Face Detection

Detects and recognizes faces within target media.

The Face Detection API analyzes a media file to find faces. The API returns a list of items, each of which contains the details of a face that was detected in the file.

For each face, the response includes the position of the face, as well as additional information, such as the estimated age range of the face. If the input file is a video, the response also includes the time when the face is most visible, the time at which the face first appears in the video, and the time at which it is no longer visible.

You can use this information to highlight a face, or create a timeline of where faces appear and disappear if the input file is a video.

Quick Start

You can upload a media file to the API as a file, in which case you must use a POST method.

curl -X POST http://api.havenondemand.com/1/api/async/detectfaces/v2 --form "file=@funfair.mp4"

You can also input a URL or a Haven OnDemand object store reference.

If you provide a URL, it must link directly to a media file. You cannot link to a page with an embedded media (such as a news page or YouTube link).

http://api.havenondemand.com/1/api/async/detectfaces/v2?url=https://www.havenondemand.com/sample-content/images/family.jpg

You can provide an image or video in many common formats. For a full list of supported media formats, see Supported Media Formats.

Note: Because input files for this API can be large and take a long time to process, the API runs only in asynchronous mode. See Get the Results.

Note: This API has rate and duration limits:

  • Input files are truncated after 10 minutes.
  • The processing terminates after two hours and returns only what it has completed within that time.

For more information, see Rate Limiting, Quotas, Data Expiry, and Maximums.

Filter Out Smaller Faces

You can specify a minimum face size, to exclude smaller faces from the results. For example:

http://api.havenondemand.com/1/api/async/detectfaces/v2?url=https://www.havenondemand.com/sample-content/images/family.jpg&minimum_face_diameter=100

Specify the diameter in pixels. The default value is 70 pixels.

Get the Results

The asynchronous mode returns a job-id, which you can then use to extract your results. There are two methods for this:

  • Use /1/job/status/ to get the status of the job, including results if the job is finished.
  • Use /1/job/result/, which waits until the job has finished and then returns the result.

    Note: Because /result has to wait for the job to finish before it can return a response, using it for longer operations such as processing a large video file can result in an HTTP request timeout response. The /result method returns a response either when the result is available, or after 120 seconds, whichever is sooner. If the job is not complete after 120 seconds, the /result method returns a code 7010 (job result request timeout) response. This means that your asynchronous job is still in progress. To avoid the timeout, use /status instead.

Results

The results contain demographic information about the age of the detected face as well as the position coordinates for the face. The position coordinates for faces are in pixels (unless the source is a PDF where the units are expressed as points) from the upper left corner of the frame.

When the input is a video, the results contain time information. The time information is in seconds (milliseconds appear as a decimal, for example 3.5).


{
	"age": "Adult",
	"source_region_coordinates": [
		{
			"x": 238,
			"y": 181,
			"unit": "pixels"
		},
		{
			"x": 310,
			"y": 181,
			"unit": "pixels"
		},
		{
			"x": 310,
			"y": 253,
			"unit": "pixels"
		},
		{
			"x": 238,
			"y": 253,
			"unit": "pixels"
		}
	],
	"start_time_offset": 3.069,
	"end_time_offset": 9.108,
	"time_offset": 4.07
}

The age response returns one of the following values:

  • baby
  • child
  • young_adult
  • adult
  • senior

Face Recognition

The API can optionally attempt to match detected faces against a library of famous faces. When you submit an image or video to the API, Haven OnDemand searches it for sections that match the faces in the database. You can enable this using the recognize_faces parameter, for example:

http://api.havenondemand.com/1/api/async/detectfaces/v2?url=https://www.havenondemand.com/sample-content/images/family.jpg&recognize_faces=true

The API returns the name of the person in the database that was detected, and the location of the face in your video at the time the face was clearest. If the API finds more than one possible match, it returns all possibilities. The API returns the location as the coordinates of the corners of a box that surrounds the matched face.

{
    "items": [
        {
            "start_time_offset": 0.041,
            "end_time_offset": 9.091,
            "time_offset": 8.55,
            "source_region_coordinates": [
                { "x": 228, "y": 71, "unit": "pixels" },
                { "x": 404, "y": 71, "unit": "pixels" },
                { "x": 404, "y": 247, "unit": "pixels" },
                { "x": 228, "y": 247, "unit": "pixels" }
            ],
            "age": "senior",
            "identities": [
                {
                    "identity_id": "Margaret_Thatcher",
                    "identity_name": "Margaret Thatcher",
                    "confidence": 84.8
                },
                {
                    "identity_id": "Madonna",
                    "identity_name": "Madonna",
                    "confidence": 43.49
                }
            ]
        }
    ]
}

For a full list of supported image formats, video container formats, and codecs, see Supported Media Formats

For a list of famous faces used in the training set for the API, see Famous Faces Training Set.

Filter Out Unwanted Responses

You can specify a minimum confidence score, to exclude faces with lower confidence values from the results. For example:

http://api.havenondemand.com/1/api/async/detectfaces/v2?url=https://www.havenondemand.com/sample-content/images/family.jpg&recognize_faces=true&minimum_confidence=80

{
    "items": [
        {
            "start_time_offset": 0.041,
            "end_time_offset": 9.091,
            "time_offset": 8.55,
            "source_region_coordinates": [
                { "x": 228, "y": 71, "unit": "pixels" },
                { "x": 404, "y": 71, "unit": "pixels" },
                { "x": 404, "y": 247, "unit": "pixels" },
                { "x": 228, "y": 247, "unit": "pixels" }
            ],
            "age": "senior",
            "identities": [
                {
                    "identity_id": "Margaret_Thatcher",
                    "identity_name": "Margaret Thatcher",
                    "confidence": 84.8
                }
            ]
        }
    ]
}

Optimize Results

The quality of the media file that you send, and the location and type of faces that appear, can have a large effect on the quality of the API output. For example, you might need to check that:

  • the faces are in focus.
  • the faces are natural colors (for example, faces in colored lighting or sepia-tinted photos might not be detected).
  • the faces are upright (not upside down) and not tilted too much (the nose is approximately vertical).
  • both eyes of the face are visible. Face detection also works best for faces that are looking directly at the camera.
  • the face is prominent in the image, and larger than 70 pixels across.
  • there is even illumination in the images (no shadows, no highlights, not under or over exposed).
  • there are no compression artifacts in the video. Use best quality reasonable resolution and reasonable quality video. Do not transcode videos, because this can introduce blocking or aliasing artifacts.
Asynchronous
https://api.havenondemand.com/1/api/async/detectfaces/v2

This API supports only asynchronous invocation.

Authentication

This API requires an authentication token to be supplied in the following parameter:

Parameter Description
apikey The API key to use to authenticate the API request.
Parameters

This API accepts the following parameters:

Required
Name Type Description
file
binary The media file that contains the faces to process. Multipart POST only.
reference
string A Haven OnDemand reference obtained from either the Expand Container or Store Object API. The corresponding video is passed to the API.
url
string A publicly accessible HTTP URL from which the video can be retrieved.
Optional
Name Type Description
minimum_face_diameter
integer The minimum diameter (in pixels) of the face that each result returned must have. The minimum value of this parameter is 24 pixels. Default value: 70.
minimum_confidence
number The minimum confidence value (0-100) that each result returned must have. Default value: 50.
recognize_faces
boolean Whether to match detected faces against a set of known faces. Default value: false.

This API returns a JSON response that is described by the model below. This single model is presented both as an easy to read abstract definition and as the formal JSON schema.

Asynchronous Use

Additional requests are required to get the result if this API is invoked asynchronously.

You can use /1/job/status/<job-id> to get the status of the job, including results if the job is finished.

You can also use /1/job/result/<job-id>, which waits until the job has finished and then returns the result.

Model
This is an abstract definition of the response that describes each of the properties that might be returned.
Face Detection Response {
source_information ( Source_information , optional) Metadata information about a media file
items ( array[Items] ) An array of Face objects.
}
Face Detection Response:Source_information {
mime_type ( string ) MIME type of the document.
video_information ( Video_information , optional) Information about the video track if one is present.
audio_information ( Audio_information , optional) Information about the audio track if one is present.
image_information ( Image_information , optional) Information about the image track if one is present.
document_information ( Document_information , optional) Information about the document track. This is only available if the source media is a multi-page image, presentation file, or PDF.
}
Face Detection Response:Source_information:Video_information {
width ( integer ) The width of the video in pixels.
height ( integer ) The height of the video in pixels.
codec ( string ) The algorithm used to encode the video.
pixel_aspect_ratio ( string ) The aspect ratio of pixels in the video. For example, if the video is made up of square pixels this value is 1:1.
}
Face Detection Response:Source_information:Audio_information {
codec ( string ) The algorithm used to encode the audio.
sample_rate ( integer ) The frequency at which the audio was sampled.
channels ( integer ) The number of channels present in the audio. For example, for stereo this value is 2.
}
Face Detection Response:Source_information:Image_information {
width ( integer ) The width of the image in pixels.
height ( integer ) The height of the image in pixels.
}
Face Detection Response:Source_information:Document_information {
page_count ( integer ) The estimated number of pages in the document.
}
Face Detection Response:Items {
start_time_offset ( number , optional) Time from the start of the video to where the face first appears. This value is expressed as a non integer number. This is only available if the source media is a video.
end_time_offset ( number , optional) Time from the start of the video to where the face is no longer visible. This value is expressed as a non integer number. This is only available if the source media is a video.
time_offset ( number , optional) Time offset from the start of the video to where the face was detected. This value is expressed as a non integer number. This is only available if the source media is a video.
page ( integer , optional) The page (1-based index) in the document where the face is located. This is only available if the source media is a multi-page image, presentation file, or PDF.
age ( enum<Age> , optional) The age range of the face.
source_region_coordinates ( array[Source_region_coordinates] ) An array of coordinates (relative to the top left of the source) that represents the bounding region where the face can be found. If the source media is a video then this region is where the face was found at time_offset in the input source.
identities ( array[Identities] , optional) One or more identity matches for the detected face.
}
enum<Face Detection Response:Items:Age> {
'baby' Below approximately 2 years.
'child' Approximately 2–15 years.
'young_adult' Approximately 15–35 years.
'adult' Approximately 35–55 years.
'senior' Above approximately 55 years.
}
Face Detection Response:Items:Source_region_coordinates {
x ( integer ) The horizontal co-ordinate that makes up part of a region.
y ( integer ) The vertical co-ordinate that makes up part of a region.
unit ( enum<Unit> ) The unit in which x and y are expressed.
}
enum<Face Detection Response:Items:Source_region_coordinates:Unit> {
'pixels' Pixels.
'points' Points in the PDF co-ordinate system.
}
Face Detection Response:Items:Identities {
name ( string ) The name of the person identified.
confidence ( number ) A value (0-100) of confidence in the identification.
}
Model Schema
This is a JSON schema that describes the syntax of the response. See json-schema.org for a complete reference.
{
    "properties": {
        "source_information": {
            "properties": {
                "mime_type": {
                    "type": "string"
                },
                "video_information": {
                    "properties": {
                        "width": {
                            "type": "integer",
                            "minimum": 1
                        },
                        "height": {
                            "type": "integer",
                            "minimum": 1
                        },
                        "codec": {
                            "type": "string"
                        },
                        "pixel_aspect_ratio": {
                            "type": "string"
                        }
                    },
                    "type": "object",
                    "required": [
                        "width",
                        "height",
                        "codec",
                        "pixel_aspect_ratio"
                    ]
                },
                "audio_information": {
                    "properties": {
                        "codec": {
                            "type": "string"
                        },
                        "sample_rate": {
                            "type": "integer"
                        },
                        "channels": {
                            "type": "integer"
                        }
                    },
                    "type": "object",
                    "required": [
                        "codec",
                        "sample_rate",
                        "channels"
                    ]
                },
                "image_information": {
                    "properties": {
                        "width": {
                            "type": "integer",
                            "minimum": 1
                        },
                        "height": {
                            "type": "integer",
                            "minimum": 1
                        }
                    },
                    "type": "object",
                    "required": [
                        "width",
                        "height"
                    ]
                },
                "document_information": {
                    "properties": {
                        "page_count": {
                            "type": "integer"
                        }
                    },
                    "type": "object",
                    "required": [
                        "page_count"
                    ]
                }
            },
            "required": [
                "mime_type"
            ],
            "type": "object"
        },
        "items": {
            "items": {
                "properties": {
                    "start_time_offset": {
                        "type": "number",
                        "minimum": 0
                    },
                    "end_time_offset": {
                        "type": "number",
                        "minimum": 0
                    },
                    "time_offset": {
                        "type": "number",
                        "minimum": 0
                    },
                    "page": {
                        "type": "integer",
                        "minimum": 1
                    },
                    "age": {
                        "enum": [
                            "baby",
                            "child",
                            "young_adult",
                            "adult",
                            "senior"
                        ]
                    },
                    "source_region_coordinates": {
                        "items": {
                            "properties": {
                                "x": {
                                    "type": "integer",
                                    "minimum": 0
                                },
                                "y": {
                                    "type": "integer",
                                    "minimum": 0
                                },
                                "unit": {
                                    "enum": [
                                        "pixels",
                                        "points"
                                    ]
                                }
                            },
                            "required": [
                                "x",
                                "y",
                                "unit"
                            ],
                            "type": "object"
                        },
                        "minItems": 3,
                        "type": "array"
                    },
                    "identities": {
                        "items": {
                            "properties": {
                                "name": {
                                    "type": "string"
                                },
                                "confidence": {
                                    "type": "number",
                                    "schema": {
                                        "minimum": 0,
                                        "maximum": 100
                                    }
                                }
                            },
                            "required": [
                                "name",
                                "confidence"
                            ],
                            "type": "object"
                        },
                        "type": "array"
                    }
                },
                "required": [
                    "source_region_coordinates"
                ],
                "type": "object"
            },
            "type": "array"
        }
    },
    "required": [
        "items"
    ],
    "type": "object"
}
https://api.havenondemand.com/1/api/async/detectfaces/v2
/api/api-example/1/api/async/detectfaces/v2
Examples
See this API for yourself - select one of our examples below.
Funfair Video
Motorbike Video
Recognize Video
Greyscale Image
Face Image
Faces Image
Recognize Face
Parameters
Required
Select file Change Remove
Optional
Name Type Value
minimum_face_diameter
integer
minimum_confidence
number
recognize_faces
boolean
(Default: False)

Note: This API will be invoked asynchronously.



Async – Response An error occurred making the API request
Response Code:
Response Body

	
Making API Request…
Checking result of job

To try this API with your own data and use it in your own applications, you need an API Key. You can create an API Key from your account page - API Keys.

Output Refresh An error occurred making the API request View Input
Rendered RawHtml Response
Result Display
Response Code:
Response Body:

			
Make this call with curl


If you would like to provide us with more information then please use the box below:

We will use your submission to help improve our product.