Face - Detect With Stream

Detect human faces in an image, return face rectangles, and optionally with faceIds, landmarks, and attributes.

  • No image will be stored. Only the extracted face feature will be stored on server. The faceId is an identifier of the face feature and will be used in Face - Identify, Face - Verify, and Face - Find Similar. The stored face feature(s) will expire and be deleted 24 hours after the original detection call.

  • Optional parameters include faceId, landmarks, and attributes. Attributes include age, gender, headPose, smile, facialHair, glasses, emotion, hair, makeup, occlusion, accessories, blur, exposure and noise. Some of the results returned for specific attributes may not be highly accurate.

  • JPEG, PNG, GIF (the first frame), and BMP format are supported. The allowed image file size is from 1KB to 6MB.

  • Up to 100 faces can be returned for an image. Faces are ranked by face rectangle size from large to small.

  • For optimal results when querying Face - Identify, Face - Verify, and Face - Find Similar ('returnFaceId' is true), please use faces that are: frontal, clear, and with a minimum size of 200x200 pixels (100 pixels between eyes).

  • The minimum detectable face size is 36x36 pixels in an image no larger than 1920x1080 pixels. Images with dimensions higher than 1920x1080 pixels will need a proportionally larger minimum face size.

  • Different 'detectionModel' values can be provided. To use and compare different detection models, please refer to How to specify a detection model | Model | Recommended use-case(s) | | ---------- | -------- | | 'detection_01': | The default detection model for Face - Detect. Recommend for near frontal face detection. For scenarios with exceptionally large angle (head-pose) faces, occluded faces or wrong image orientation, the faces in such cases may not be detected. | | 'detection_02': | Detection model released in 2019 May with improved accuracy especially on small, side and blurry faces. |

  • Different 'recognitionModel' values are provided. If follow-up operations like Verify, Identify, Find Similar are needed, please specify the recognition model with 'recognitionModel' parameter. The default value for 'recognitionModel' is 'recognition_01', if latest model needed, please explicitly specify the model you need in this parameter. Once specified, the detected faceIds will be associated with the specified recognition model. More details, please refer to How to specify a recognition model | Model | Recommended use-case(s) | | ---------- | -------- | | 'recognition_01': | The default recognition model for Face - Detect. All those faceIds created before 2019 March are bonded with this recognition model. | | 'recognition_02': | Recognition model released in 2019 March. 'recognition_02' is recommended since its overall accuracy is improved compared with 'recognition_01'. |

POST {Endpoint}/face/v1.0/detect
POST {Endpoint}/face/v1.0/detect?overload=stream&returnFaceId={returnFaceId}&returnFaceLandmarks={returnFaceLandmarks}&returnFaceAttributes={returnFaceAttributes}&recognitionModel={recognitionModel}&returnRecognitionModel={returnRecognitionModel}&detectionModel={detectionModel}

URI Parameters

Name In Required Type Description
Endpoint
path True
  • string

Supported Cognitive Services endpoints (protocol and hostname, for example: https://westus.api.cognitive.microsoft.com).

returnFaceId
query
  • boolean

A value indicating whether the operation should return faceIds of detected faces.

returnFaceLandmarks
query
  • boolean

A value indicating whether the operation should return landmarks of the detected faces.

returnFaceAttributes
query
  • array

Analyze and return the one or more specified face attributes in the comma-separated string like "returnFaceAttributes=age,gender". Supported face attributes include age, gender, headPose, smile, facialHair, glasses and emotion. Note that each face attribute analysis has additional computational and time cost.

recognitionModel
query

Name of recognition model. Recognition model is used when the face features are extracted and associated with detected faceIds, (Large)FaceList or (Large)PersonGroup. A recognition model name can be provided when performing Face - Detect or (Large)FaceList - Create or (Large)PersonGroup - Create. The default value is 'recognition_01', if latest model needed, please explicitly specify the model you need.

returnRecognitionModel
query
  • boolean

A value indicating whether the operation should return 'recognitionModel' in response.

detectionModel
query

Name of detection model. Detection model is used to detect faces in the submitted image. A detection model name can be provided when performing Face - Detect or (Large)FaceList - Add Face or (Large)PersonGroup - Add Face. The default value is 'detection_01', if another model is needed, please explicitly specify it.

Request Body

Media Types: "application/octet-stream"

Name Type Description
Image
  • object

An image stream.

Responses

Name Type Description
200 OK

A successful call returns an array of face entries ranked by face rectangle size in descending order. An empty response indicates no faces detected.

Other Status Codes

Error response.

Security

Ocp-Apim-Subscription-Key

Type: apiKey
In: header

Examples

Detect with stream example

Sample Request

POST {Endpoint}/face/v1.0/detect?overload=stream&returnFaceAttributes=[

  "age",

  "gender",

  "headPose",

  "smile",

  "facialHair",

  "glasses",

  "emotion",

  "hair",

  "makeup",

  "occlusion",

  "accessories",

  "blur",

  "exposure",

  "noise"

]&recognitionModel=recognition_01&returnRecognitionModel=True&detectionModel=detection_01
"{Image binary in base 64 format}"

Sample Response

[
  {
    "faceId": "c5c24a82-6845-4031-9d5d-978df9175426",
    "recognitionModel": "recognition_01",
    "faceRectangle": {
      "width": 78,
      "height": 78,
      "left": 394,
      "top": 54
    },
    "faceLandmarks": {
      "pupilLeft": {
        "x": 412.7,
        "y": 78.4
      },
      "pupilRight": {
        "x": 446.8,
        "y": 74.2
      },
      "noseTip": {
        "x": 437.7,
        "y": 92.4
      },
      "mouthLeft": {
        "x": 417.8,
        "y": 114.4
      },
      "mouthRight": {
        "x": 451.3,
        "y": 109.3
      },
      "eyebrowLeftOuter": {
        "x": 397.9,
        "y": 78.5
      },
      "eyebrowLeftInner": {
        "x": 425.4,
        "y": 70.5
      },
      "eyeLeftOuter": {
        "x": 406.7,
        "y": 80.6
      },
      "eyeLeftTop": {
        "x": 412.2,
        "y": 76.2
      },
      "eyeLeftBottom": {
        "x": 413,
        "y": 80.1
      },
      "eyeLeftInner": {
        "x": 418.9,
        "y": 78
      },
      "eyebrowRightInner": {
        "x": 4.8,
        "y": 69.7
      },
      "eyebrowRightOuter": {
        "x": 5.5,
        "y": 68.5
      },
      "eyeRightInner": {
        "x": 441.5,
        "y": 75
      },
      "eyeRightTop": {
        "x": 446.4,
        "y": 71.7
      },
      "eyeRightBottom": {
        "x": 447,
        "y": 75.3
      },
      "eyeRightOuter": {
        "x": 451.7,
        "y": 73.4
      },
      "noseRootLeft": {
        "x": 428,
        "y": 77.1
      },
      "noseRootRight": {
        "x": 435.8,
        "y": 75.6
      },
      "noseLeftAlarTop": {
        "x": 428.3,
        "y": 89.7
      },
      "noseRightAlarTop": {
        "x": 442.2,
        "y": 87
      },
      "noseLeftAlarOutTip": {
        "x": 424.3,
        "y": 96.4
      },
      "noseRightAlarOutTip": {
        "x": 446.6,
        "y": 92.5
      },
      "upperLipTop": {
        "x": 437.6,
        "y": 105.9
      },
      "upperLipBottom": {
        "x": 437.6,
        "y": 108.2
      },
      "underLipTop": {
        "x": 436.8,
        "y": 111.4
      },
      "underLipBottom": {
        "x": 437.3,
        "y": 114.5
      }
    },
    "faceAttributes": {
      "age": 71,
      "gender": "male",
      "smile": 0.88,
      "facialHair": {
        "moustache": 0.8,
        "beard": 0.1,
        "sideburns": 0.02
      },
      "glasses": "sunglasses",
      "headPose": {
        "roll": 2.1,
        "yaw": 3,
        "pitch": 1.6
      },
      "emotion": {
        "anger": 0.575,
        "contempt": 0,
        "disgust": 0.006,
        "fear": 0.008,
        "happiness": 0.394,
        "neutral": 0.013,
        "sadness": 0,
        "surprise": 0.004
      },
      "hair": {
        "bald": 0,
        "invisible": false,
        "hairColor": [
          {
            "color": "brown",
            "confidence": 1
          },
          {
            "color": "blond",
            "confidence": 0.88
          },
          {
            "color": "black",
            "confidence": 0.48
          },
          {
            "color": "other",
            "confidence": 0.11
          },
          {
            "color": "gray",
            "confidence": 0.07
          },
          {
            "color": "red",
            "confidence": 0.03
          }
        ]
      },
      "makeup": {
        "eyeMakeup": true,
        "lipMakeup": false
      },
      "occlusion": {
        "foreheadOccluded": false,
        "eyeOccluded": false,
        "mouthOccluded": false
      },
      "accessories": [
        {
          "type": "headWear",
          "confidence": 0.99
        },
        {
          "type": "glasses",
          "confidence": 1
        },
        {
          "type": "mask",
          "confidence": 0.87
        }
      ],
      "blur": {
        "blurLevel": "Medium",
        "value": 0.51
      },
      "exposure": {
        "exposureLevel": "GoodExposure",
        "value": 0.55
      },
      "noise": {
        "noiseLevel": "Low",
        "value": 0.12
      }
    }
  }
]

Definitions

Accessory

Accessory item and corresponding confidence level.

AccessoryType

Type of an accessory

APIError

Error information returned by the API

Blur

Properties describing any presence of blur within the image.

BlurLevel

An enum value indicating level of blurriness.

Coordinate

Coordinates within an image

DetectedFace

Detected Face object.

DetectionModel

Name of detection model. Detection model is used to detect faces in the submitted image. A detection model name can be provided when performing Face - Detect or (Large)FaceList - Add Face or (Large)PersonGroup - Add Face. The default value is 'detection_01', if another model is needed, please explicitly specify it.

Emotion

Properties describing facial emotion in form of confidence ranging from 0 to 1.

Error

Error body.

Exposure

Properties describing exposure level of the image.

ExposureLevel

An enum value indicating level of exposure.

FaceAttributes

Face Attributes

FaceLandmarks

A collection of 27-point face landmarks pointing to the important positions of face components.

FaceRectangle

A rectangle within which a face can be found

FacialHair

Properties describing facial hair attributes.

Gender

Possible gender of the face.

GlassesType

Glasses type if any of the face.

Hair

Properties describing hair attributes.

HairColor

Hair color and associated confidence

HairColorType

Name of the hair color.

HeadPose

Properties indicating head pose of the face.

Makeup

Properties describing present makeups on a given face.

Noise

Properties describing noise level of the image.

NoiseLevel

An enum value indicating level of noise.

Occlusion

Properties describing occlusions on a given face.

RecognitionModel

Name of recognition model. Recognition model is used when the face features are extracted and associated with detected faceIds, (Large)FaceList or (Large)PersonGroup. A recognition model name can be provided when performing Face - Detect or (Large)FaceList - Create or (Large)PersonGroup - Create. The default value is 'recognition_01', if latest model needed, please explicitly specify the model you need.

Accessory

Accessory item and corresponding confidence level.

Name Type Description
confidence
  • number

Confidence level of an accessory

type

Type of an accessory

AccessoryType

Type of an accessory

Name Type Description
glasses
  • string
headWear
  • string
mask
  • string

APIError

Error information returned by the API

Name Type Description
error

Error body.

Blur

Properties describing any presence of blur within the image.

Name Type Description
blurLevel

An enum value indicating level of blurriness.

value
  • number

A number indicating level of blurriness ranging from 0 to 1.

BlurLevel

An enum value indicating level of blurriness.

Name Type Description
High
  • string
Low
  • string
Medium
  • string

Coordinate

Coordinates within an image

Name Type Description
x
  • number

The horizontal component, in pixels.

y
  • number

The vertical component, in pixels.

DetectedFace

Detected Face object.

Name Type Description
faceAttributes

Face Attributes

faceId
  • string
faceLandmarks

A collection of 27-point face landmarks pointing to the important positions of face components.

faceRectangle

A rectangle within which a face can be found

recognitionModel

Name of recognition model. Recognition model is used when the face features are extracted and associated with detected faceIds, (Large)FaceList or (Large)PersonGroup. A recognition model name can be provided when performing Face - Detect or (Large)FaceList - Create or (Large)PersonGroup - Create. The default value is 'recognition_01', if latest model needed, please explicitly specify the model you need.

DetectionModel

Name of detection model. Detection model is used to detect faces in the submitted image. A detection model name can be provided when performing Face - Detect or (Large)FaceList - Add Face or (Large)PersonGroup - Add Face. The default value is 'detection_01', if another model is needed, please explicitly specify it.

Name Type Description
detection_01
  • string
detection_02
  • string

Emotion

Properties describing facial emotion in form of confidence ranging from 0 to 1.

Name Type Description
anger
  • number

A number ranging from 0 to 1 indicating a level of confidence associated with a property.

contempt
  • number

A number ranging from 0 to 1 indicating a level of confidence associated with a property.

disgust
  • number

A number ranging from 0 to 1 indicating a level of confidence associated with a property.

fear
  • number

A number ranging from 0 to 1 indicating a level of confidence associated with a property.

happiness
  • number

A number ranging from 0 to 1 indicating a level of confidence associated with a property.

neutral
  • number

A number ranging from 0 to 1 indicating a level of confidence associated with a property.

sadness
  • number

A number ranging from 0 to 1 indicating a level of confidence associated with a property.

surprise
  • number

A number ranging from 0 to 1 indicating a level of confidence associated with a property.

Error

Error body.

Name Type Description
code
  • string
message
  • string

Exposure

Properties describing exposure level of the image.

Name Type Description
exposureLevel

An enum value indicating level of exposure.

value
  • number

A number indicating level of exposure level ranging from 0 to 1. [0, 0.25) is under exposure. [0.25, 0.75) is good exposure. [0.75, 1] is over exposure.

ExposureLevel

An enum value indicating level of exposure.

Name Type Description
GoodExposure
  • string
OverExposure
  • string
UnderExposure
  • string

FaceAttributes

Face Attributes

Name Type Description
accessories

Properties describing any accessories on a given face.

age
  • number

Age in years

blur

Properties describing any presence of blur within the image.

emotion

Properties describing facial emotion in form of confidence ranging from 0 to 1.

exposure

Properties describing exposure level of the image.

facialHair

Properties describing facial hair attributes.

gender

Possible gender of the face.

glasses

Glasses type if any of the face.

hair

Properties describing hair attributes.

headPose

Properties indicating head pose of the face.

makeup

Properties describing present makeups on a given face.

noise

Properties describing noise level of the image.

occlusion

Properties describing occlusions on a given face.

smile
  • number

Smile intensity, a number between [0,1]

FaceLandmarks

A collection of 27-point face landmarks pointing to the important positions of face components.

Name Type Description
eyeLeftBottom

Coordinates within an image

eyeLeftInner

Coordinates within an image

eyeLeftOuter

Coordinates within an image

eyeLeftTop

Coordinates within an image

eyeRightBottom

Coordinates within an image

eyeRightInner

Coordinates within an image

eyeRightOuter

Coordinates within an image

eyeRightTop

Coordinates within an image

eyebrowLeftInner

Coordinates within an image

eyebrowLeftOuter

Coordinates within an image

eyebrowRightInner

Coordinates within an image

eyebrowRightOuter

Coordinates within an image

mouthLeft

Coordinates within an image

mouthRight

Coordinates within an image

noseLeftAlarOutTip

Coordinates within an image

noseLeftAlarTop

Coordinates within an image

noseRightAlarOutTip

Coordinates within an image

noseRightAlarTop

Coordinates within an image

noseRootLeft

Coordinates within an image

noseRootRight

Coordinates within an image

noseTip

Coordinates within an image

pupilLeft

Coordinates within an image

pupilRight

Coordinates within an image

underLipBottom

Coordinates within an image

underLipTop

Coordinates within an image

upperLipBottom

Coordinates within an image

upperLipTop

Coordinates within an image

FaceRectangle

A rectangle within which a face can be found

Name Type Description
height
  • integer

The height of the rectangle, in pixels.

left
  • integer

The distance from the left edge if the image to the left edge of the rectangle, in pixels.

top
  • integer

The distance from the top edge if the image to the top edge of the rectangle, in pixels.

width
  • integer

The width of the rectangle, in pixels.

FacialHair

Properties describing facial hair attributes.

Name Type Description
beard
  • number

A number ranging from 0 to 1 indicating a level of confidence associated with a property.

moustache
  • number

A number ranging from 0 to 1 indicating a level of confidence associated with a property.

sideburns
  • number

A number ranging from 0 to 1 indicating a level of confidence associated with a property.

Gender

Possible gender of the face.

Name Type Description
female
  • string
male
  • string

GlassesType

Glasses type if any of the face.

Name Type Description
noGlasses
  • string
readingGlasses
  • string
sunglasses
  • string
swimmingGoggles
  • string

Hair

Properties describing hair attributes.

Name Type Description
bald
  • number

A number describing confidence level of whether the person is bald.

hairColor

An array of candidate colors and confidence level in the presence of each.

invisible
  • boolean

A boolean value describing whether the hair is visible in the image.

HairColor

Hair color and associated confidence

Name Type Description
color

Name of the hair color.

confidence
  • number

Confidence level of the color

HairColorType

Name of the hair color.

Name Type Description
black
  • string
blond
  • string
brown
  • string
gray
  • string
other
  • string
red
  • string
unknown
  • string
white
  • string

HeadPose

Properties indicating head pose of the face.

Name Type Description
pitch
  • number
roll
  • number
yaw
  • number

Makeup

Properties describing present makeups on a given face.

Name Type Description
eyeMakeup
  • boolean

A boolean value describing whether eye makeup is present on a face.

lipMakeup
  • boolean

A boolean value describing whether lip makeup is present on a face.

Noise

Properties describing noise level of the image.

Name Type Description
noiseLevel

An enum value indicating level of noise.

value
  • number

A number indicating level of noise level ranging from 0 to 1. [0, 0.25) is under exposure. [0.25, 0.75) is good exposure. [0.75, 1] is over exposure. [0, 0.3) is low noise level. [0.3, 0.7) is medium noise level. [0.7, 1] is high noise level.

NoiseLevel

An enum value indicating level of noise.

Name Type Description
High
  • string
Low
  • string
Medium
  • string

Occlusion

Properties describing occlusions on a given face.

Name Type Description
eyeOccluded
  • boolean

A boolean value indicating whether eyes are occluded.

foreheadOccluded
  • boolean

A boolean value indicating whether forehead is occluded.

mouthOccluded
  • boolean

A boolean value indicating whether the mouth is occluded.

RecognitionModel

Name of recognition model. Recognition model is used when the face features are extracted and associated with detected faceIds, (Large)FaceList or (Large)PersonGroup. A recognition model name can be provided when performing Face - Detect or (Large)FaceList - Create or (Large)PersonGroup - Create. The default value is 'recognition_01', if latest model needed, please explicitly specify the model you need.

Name Type Description
recognition_01
  • string
recognition_02
  • string