Image Classification

Classifies images within target media.

The Image Classification API classifies images within target media, against a pre-trained classifier. For example, media that contains vehicles might return classes such as Pickup Truck or Motor Scooter.

The classes on which Haven OnDemand was trained are listed in Supported Image Classes.

Note: Image Classification classifies the contents of an entire image into a broad set of conceptual classes. For example, an image of a parking lot, containing mainly cars, but also a few trucks and scooters, might be globally identified as belonging to a Car class with the highest confidence level, followed by a Truck class and a Scooter class with successively lower confidence levels. This differs from Object recognition Object Recognition, which identifies and locates specific objects, such as company logos, within an image.

Quick Start

You can upload a media file to the API as a file, in which case you must use a POST method.

curl -X POST http://api.havenondemand.com/1/api/async/classifyimages/v1 --form "file=@snail.mp4"

You can also input a URL or a Haven OnDemand object store reference.

If you provide a URL, it must link directly to a media file. You cannot link to a page with an embedded media (such as a news page or YouTube link).

http://api.havenondemand.com/1/api/async/classifyimages/v1?url=https://www.havenondemand.com/sample-content/images/snail.jpg

You can provide an image or video in many common formats. For a full list of supported media formats, see Supported Media Formats.

Note: Because input files for this API can be large and take a long time to process, the API runs only in asynchronous mode. See Get the Results.

Note: This API has rate and duration limits:

  • Input files are truncated after 10 minutes.
  • The processing terminates after two hours and returns only what it has completed within that time.

For more information, see Rate Limiting, Quotas, Data Expiry, and Maximums.

Get the Results

The asynchronous mode returns a job-id, which you can then use to extract your results. There are two methods for this:

  • Use /1/job/status/ to get the status of the job, including results if the job is finished.
  • Use /1/job/result/, which waits until the job has finished and then returns the result.

    Note: Because /result has to wait for the job to finish before it can return a response, using it for longer operations such as processing a large video file can result in an HTTP request timeout response. The /result method returns a response either when the result is available, or after 120 seconds, whichever is sooner. If the job is not complete after 120 seconds, the /result method returns a code 7010 (job result request timeout) response. This means that your asynchronous job is still in progress. To avoid the timeout, use /status instead.

Results

When the input is a video, the results contain time information. The time information is in seconds (milliseconds appear as a decimal, for example 3.5).


  {
    "classification": {
      "confidence": 27.1,
      "name": "slug"
    },
    "time_offset": 0.08
  },
  {
    "classification": {
      "confidence": 99.21,
      "name": "snail"
    },
    "time_offset": 0.09
  }

When the input is a multi-page image, presentation file, or PDF, the results contain page information.


  {
    "classification": {
      "confidence": 27.1,
      "name": "slug"
    },
    "page": 1
  },
  {
    "classification": {
      "confidence": 99.21,
      "name": "snail"
    },
    "page": 2
  }

Filter Out Unwanted Responses

You can specify a minimum confidence score, to exclude classes with lower scores from the results. For example:

http://api.havenondemand.com/1/api/async/classifyimages/v1?url=https://www.havenondemand.com/sample-content/videos/snail.mp4&minium_confidence=50


  {
    "classification": {
      "confidence": 99.21,
      "name": "snail"
    },
    "time_offset": 0.09
  }
Asynchronous
https://api.havenondemand.com/1/api/async/classifyimages/v1

This API supports only asynchronous invocation.

Authentication

This API requires an authentication token to be supplied in the following parameter:

Parameter Description
apikey The API key to use to authenticate the API request.
Parameters

This API accepts the following parameters:

Required
Name Type Description
file
binary A media file containing the images to classify. Multipart POST only.
reference
string A Haven OnDemand reference obtained from either the Expand Container or Store Object API. The corresponding video is passed to the API.
url
string A publicly accessible HTTP URL from which a video can be retrieved.
Optional
Name Type Description
minimum_confidence
number The minimum confidence value (0-100) that each result returned must have. Default value: 10.

This API returns a JSON response that is described by the model below. This single model is presented both as an easy to read abstract definition and as the formal JSON schema.

Asynchronous Use

Additional requests are required to get the result if this API is invoked asynchronously.

You can use /1/job/status/<job-id> to get the status of the job, including results if the job is finished.

You can also use /1/job/result/<job-id>, which waits until the job has finished and then returns the result.

Model
This is an abstract definition of the response that describes each of the properties that might be returned.
Image Classification Response {
source_information ( Source_information , optional) Metadata information about a media file
items ( array[Items] ) An array of image classes
}
Image Classification Response:Source_information {
mime_type ( string ) MIME type of the document.
video_information ( Video_information , optional) Information about the video track if one is present.
audio_information ( Audio_information , optional) Information about the audio track if one is present.
image_information ( Image_information , optional) Information about the image track if one is present.
document_information ( Document_information , optional) Information about the document track. This is only available if the source media is a multi-page image, presentation file, or PDF.
}
Image Classification Response:Source_information:Video_information {
width ( integer ) The width of the video in pixels.
height ( integer ) The height of the video in pixels.
codec ( string ) The algorithm used to encode the video.
pixel_aspect_ratio ( string ) The aspect ratio of pixels in the video. For example, if the video is made up of square pixels this value is 1:1.
}
Image Classification Response:Source_information:Audio_information {
codec ( string ) The algorithm used to encode the audio.
sample_rate ( integer ) The frequency at which the audio was sampled.
channels ( integer ) The number of channels present in the audio. For example, for stereo this value is 2.
}
Image Classification Response:Source_information:Image_information {
width ( integer ) The width of the image in pixels.
height ( integer ) The height of the image in pixels.
}
Image Classification Response:Source_information:Document_information {
page_count ( integer ) The estimated number of pages in the document.
}
Image Classification Response:Items {
time_offset ( number , optional) Time offset from the start of the video to where the image was classified. This value is expressed as a non integer number. This is only available if the source media is a video.
page ( integer , optional) The page (1-based index) in the document where the image is located. This is only available if the source media is a multi-page image, presentation file, or PDF.
classification ( Classification ) The classification of the image.
}
Image Classification Response:Items:Classification {
name ( string ) The name of the class to which the entire image belongs.
confidence ( number ) A value (0-100) of confidence in the classification.
}
Model Schema
This is a JSON schema that describes the syntax of the response. See json-schema.org for a complete reference.
{
    "properties": {
        "source_information": {
            "properties": {
                "mime_type": {
                    "type": "string"
                },
                "video_information": {
                    "properties": {
                        "width": {
                            "type": "integer",
                            "minimum": 1
                        },
                        "height": {
                            "type": "integer",
                            "minimum": 1
                        },
                        "codec": {
                            "type": "string"
                        },
                        "pixel_aspect_ratio": {
                            "type": "string"
                        }
                    },
                    "type": "object",
                    "required": [
                        "width",
                        "height",
                        "codec",
                        "pixel_aspect_ratio"
                    ]
                },
                "audio_information": {
                    "properties": {
                        "codec": {
                            "type": "string"
                        },
                        "sample_rate": {
                            "type": "integer"
                        },
                        "channels": {
                            "type": "integer"
                        }
                    },
                    "type": "object",
                    "required": [
                        "codec",
                        "sample_rate",
                        "channels"
                    ]
                },
                "image_information": {
                    "properties": {
                        "width": {
                            "type": "integer",
                            "minimum": 1
                        },
                        "height": {
                            "type": "integer",
                            "minimum": 1
                        }
                    },
                    "type": "object",
                    "required": [
                        "width",
                        "height"
                    ]
                },
                "document_information": {
                    "properties": {
                        "page_count": {
                            "type": "integer"
                        }
                    },
                    "type": "object",
                    "required": [
                        "page_count"
                    ]
                }
            },
            "required": [
                "mime_type"
            ],
            "type": "object"
        },
        "items": {
            "items": {
                "properties": {
                    "time_offset": {
                        "type": "number",
                        "minimum": 0
                    },
                    "page": {
                        "type": "integer",
                        "minimum": 1
                    },
                    "classification": {
                        "properties": {
                            "name": {
                                "type": "string"
                            },
                            "confidence": {
                                "type": "number",
                                "minimum": 0,
                                "maximum": 100
                            }
                        },
                        "required": [
                            "name",
                            "confidence"
                        ],
                        "type": "object"
                    }
                },
                "required": [
                    "classification"
                ],
                "type": "object"
            },
            "type": "array"
        }
    },
    "required": [
        "items"
    ],
    "type": "object"
}
https://api.havenondemand.com/1/api/async/classifyimages/v1
/api/api-example/1/api/async/classifyimages/v1
Examples
See this API for yourself - select one of our examples below.
Snail video
Snake video
Frog image
Snail image
Snake in image
Parameters
Required
Select file Change Remove
Optional
Name Type Value
minimum_confidence
number

Note: This API will be invoked asynchronously.



Async – Response An error occurred making the API request
Response Code:
Response Body

	
Making API Request…
Checking result of job

To try this API with your own data and use it in your own applications, you need an API Key. You can create an API Key from your account page - API Keys.

Output Refresh An error occurred making the API request View Input
Rendered RawHtml Response
Result Display
Response Code:
Response Body:

			
Make this call with curl


If you would like to provide us with more information then please use the box below:

We will use your submission to help improve our product.