Vision Namespace

The Vision namespace provides high-level image-recognition and registration facilities.

Classes

VNBarcodeObservation

A VNRectangleObservation in which a barcode was recognized.

VNBarcodeSymbologyExtensions

Extension methods relating to VNBarcodeSymbology objects.

VNClassificationObservation

Subclass of VNObservation for CoreML models that predict a single feature.

VNCoreMLFeatureValueObservation

A VNObservation that contains a feature detected by a Core ML model.

VNCoreMLModel

Wraps a CoreML model for use within the Vision namespace.

VNCoreMLRequest

A subclass of VNImageBasedRequest that uses a Core ML model for processing.

VNDetectBarcodesRequest

A subclass of VNImageBasedRequest that detects barcodes.

VNDetectedObjectObservation

A VNObservation that includes the detection of a rectangular object.

VNDetectFaceLandmarksRequest

A VNImageBasedRequest for recognizing face components.

VNDetectFaceRectanglesRequest

A VNImageBasedRequest to retrieve the bounding boxes of any faces detected in the source.

VNDetectHorizonRequest

A VNImageBasedRequest to retrieve the horizon in the source.

VNDetectRectanglesRequest

A VNImageBasedRequest to retrieve rectangular regions detected in the source.

VNDetectTextRectanglesRequest

A VNImageBasedRequest to retrieve the bounding boxes of any text blocks detected in the source.

VNErrorCodeExtensions

Extension methods for the Vision.VNErrorCode enumeration.

VNFaceLandmarkRegion

Abstract base class for classes holding information about facial landmarks.

VNFaceLandmarkRegion2D

A VNFaceLandmarkRegion that holds two-dimensional information about a recognized facial landmark.

VNFaceLandmarks

Abstract base class for classes that hold information about recognized facial landmarks.

VNFaceLandmarks2D

A VNFaceLandmarks object that hold information about landmarks in 2D space.

VNFaceObservation

A VNDetectedObjectObservation that includes a face.

VNHomographicImageRegistrationRequest

T:VNImageRegistrationRequest

VNHorizonObservation

A VNObservation that includes detection of a horizon line.

VNImageAlignmentObservation

Abstract base class of VNObservation that hold information about aligning images.

VNImageBasedRequest

Abstract base-class for VNRequests that operate on images.

VNImageHomographicAlignmentObservation

An observation whose results are a 3D warp transform for aligning two images .

VNImageOptions

A DictionaryContainer holding options to be used in Vision queries.

VNImageRegistrationRequest

Abstract sub-class of VNTargetedImageRequest that attemps to align an image with a reference image.

VNImageRequestHandler

Coordinates the vision requests of a single image.

VNImageTranslationAlignmentObservation

A VNImageAlignmentObservation in which the alignment is restricted to an affine transform.

VNObservation

Abstract base class for vision processing results.

VNPixelBufferObservation

A VNObservation that works on CVPixelBuffer object(s).

VNRecognizedObjectObservation
VNRectangleObservation

A VNDetectedObjectObservation of a rectangular region.

VNRequest

Abstract base-class for vision-processing request.

VNSequenceRequestHandler

Coordinates the vision requests of a sequence of images (such as a video stream).

VNTargetedImageRequest

Abstract VNImageBasedRequest for requests that operate on two images.

VNTextObservation

A VNDetectedObjectObservation of a text block.

VNTrackingRequest

Abstract VNImageBasedRequest that is the base class for requests that track an object between multiple images.

VNTrackObjectRequest

A VNTrackingRequest that tracks a previously recognized object across multiple frames.

VNTrackRectangleRequest

A VNTrackingRequest that tracks a rectangular region over multiple frames.

VNTranslationalImageRegistrationRequest

A VNImageRegistrationRequest that restricts the registration result to an affine transform.

VNUtils

A set of utility functions for working with images.

Interfaces

IVNFaceObservationAccepting

Interface for VNImageBasedRequest objects that may detect faces (e.g., VNDetectFaceLandmarksRequest).

IVNRequestRevisionProviding

Enums

VNBarcodeObservationRequestRevision
VNBarcodeSymbology

Enumerated supported barcode standards.

VNCoreMLRequestRevision
VNDetectBarcodesRequestRevision
VNDetectedObjectObservationRequestRevision
VNDetectFaceLandmarksRequestRevision
VNDetectFaceRectanglesRequestRevision
VNDetectHorizonRequestRevision
VNDetectRectanglesRequestRevision
VNDetectTextRectanglesRequestRevision
VNErrorCode

Enumerates errors associated with Vision requests.

VNFaceObservationRequestRevision
VNHomographicImageRegistrationRequestRevision
VNImageCropAndScaleOption

Enumerates how an image should be cropped and scaled during processing.

VNRecognizedObjectObservationRequestRevision
VNRectangleObservationRequestRevision
VNRequestRevision
VNRequestTrackingLevel

Enumerates the emphasis of the tracking algorithm.

VNTextObservationRequestRevision
VNTrackObjectRequestRevision
VNTrackRectangleRequestRevision
VNTranslationalImageRegistrationRequestRevision

Delegates

VNRequestCompletionHandler

A delegate that is called once for each feature detected in a VNRequest.

Remarks

The Vision namespace, introduced in iOS 11, provides a common interface for high-level image recognition, segmentation, and machine-learned tasks.

The Vision namespace provides a number of built-in image processing functions:

TaskRequest types
Face detection and geometry VNDetectFaceRectanglesRequest, VNDetectFaceLandmarksRequest
Barcode recognition VNDetectBarcodesRequest
Image registration VNTranslationalImageRegistrationRequest, VNHomographicImageRegistrationRequest
Text detection VNDetectTextRectanglesRequest
Horizon detection and straightening VNDetectHorizonRequest
Object detection and tracking VNDetectRectanglesRequest, T:Vision.VNTrackRectanglesRequest, VNTrackObjectRequest

In addition to the built-in functions, Vision supports flexible image-based queries to CoreMLMLModel objects. In contrast with the precise input requirements of CoreML, implementers of T:Vision.IVNTargetedImageRequest can accept a variety of image formats: CIImage, CGImage, and CVPixelBuffer objects. The system scaled and converts the image to the input format required by the MLModel.

In all cases, Vision requests and an image are passed to a VNImageRequestHandler, whose M:VNImageRequestHandler.Perform* method executes a callback, passing one or more VNObservation objects of a request-appropriate type. For example:


var rectangleRequest = new VNDetectRectanglesRequest(HandleRectangles);
var handler = new VNImageRequestHandler(img, orientation, new VNImageOptions());
DispatchQueue.DefaultGlobalQueue.DispatchAsync(()=>{
    NSError error;
    handler.Perform(new VNRequest[] {rectangleRequest}, out error);
    if (error != null)
    {
       ErrorOccurred(error); 
    }
});

void HandleRectangles(VNRequest request, NSError error){ VNRectangleObservation[] observations = request.GetResults<VNRectangleObservation>(); // ... etc ...