Object Detection

Object Detection In Action

With the Object Detection feature, you can identify objects of interest in an image or each frame of live video. Each prediction returns a set of objects, each with a label, bounding box, and confidence score.

If you just need to know the contents of an image – not the location of the objects – consider using Image Labeling instead.

Models Compatible with the API

Compatibility Checklist

  1. Your model must be a single-shot multibox detector with boxes matching the default configuration found here.
  2. Your model must be in the TensorFlow Lite (.tflite) or Core ML (.mlmodel) formats.
  3. iOS Only The name of the input layer must be named Preprocessor/sub:0 and the 2 outputs concat:0 (boxPredictions) and concat_1:0 (classPredictions).
  4. Android Only The 1 input layer (Preprocessor/sub) and 4 output layers (‘outputLocations’, ‘outputClasses’, ‘outputScores’, ‘numDetections’) should be defined in the TensorFlow Lite conversion tool.
  5. The input should have the following dimensions: 1x300x300x3 (batch_size x height x width * num_channels). Height and width are configurable.
  6. iOS Only The output should have the following dimensions: 4 (box points) x num_anchor_boxes x 1 for boxPredictions and num_classes x 1 for classPredictions.
  7. Android Only The output should have the following dimensions: 1 x num_anchor_boxes x 4 (box points) for outputLocations, num_classes x 1 for outputClasses & outputScores, and 1 for numDetections.

Technical Specifications

Architecture Format(s) Model Size Input Output Benchmarks
SSDLite + MobileNet V2 variant Core ML (iOS), TensorFlow Lite (Android) ~17 MB 300x300-pixel image Offsets for >2,000 candidate bounding boxes, Class labels for each box, Confidence scores for each box 18 FPS on iPhone X, 8 FPS on Pixel 2

Prebuilt Model

  • The object detection model supports 90 labels from the COCO dataset.

Customizing Models for Object Detection

If you have your own dataset and would like to train a custom model that is compatible with the Object Detection API, sign up for the Standard Plan on Fritz to access training notebooks.

If you’ve already created a model with tools like TuriCreate, Create ML, or TensorFlow, contact us to see if the model is compatible with the API.