Android

Note

If you haven’t set up the SDK yet, make sure to go through those directions first. You’ll need to add the Core library to the app before using the specific feature or custom model libraries.

1. Add FritzVisionLabelModel via Gradle

Add our repository in order to download Fritz libraries:

repositories {
    maven { url "https://raw.github.com/fritzlabs/fritz-repository/master" }
}

Add the dependencies in app/build.gradle:

dependencies {
    implementation 'ai.fritz:core:1.3.2'
    implementation 'ai.fritz:vision-label-model:1.3.2'
}

2. Define FritzVisionLabelPredictor

First, create a FritzVisionLabelPredictor instance:

FritzVisionLabelPredictor visionPredictor = FritzVisionLabelPredictor.getInstance(this);

Note

FritzVisionLabelPredictor.getInstance requires the calling Context as an argument (e.g Application, Activity, etc). For example, if you’re getting the predictor instance from a Fragment, you’ll want to pass in the context:

FritzVisionLabelPredictor visionPredictor = FritzVisionLabelPredictor.getInstance(this.getContext());

3. Create a FritzVisionImage from an image or a video stream

  • To create a FritzVisionImage from a Bitmap:

    FritzVisionImage visionImage = FritzVisionImage.fromBitmap(bitmap);
    
  • To create a FritzVisionImage from a media.Image object when capturing the result from a camera, first determine the orientation of the image. This will rotate the image to account for device rotation and the orientation of the camera sensor.

    // Get the system service for the camera manager
    final CameraManager manager = (CameraManager) getSystemService(Context.CAMERA_SERVICE);
    
    // Gets the first camera id
    String cameraId = manager.getCameraIdList().get(0);
    
    // Determine the rotation on the FritzVisionImage from the camera orientaion and the device rotation.
    // "this" refers to the calling Context (Application, Activity, etc)
    int imageRotationFromCamera = FritzVisionOrientation.getImageRotationFromCamera(this, cameraId);
    

    Finally, create the FritzVisionImage object with the rotation

    FritzVisionImage visionImage = FritzVisionImage.fromMediaImage(image, imageRotationFromCamera);
    

4. Run prediction - Label your image

Next, convert a bitmap into a FritzVisionImage and pass the image into the predictor in order to evaluate the image labels:

List<FritzVisionLabel> labels = visionPredictor.predict(visionImage);

visionPredictor.predict() returns a list of FritzVisionLabels sorted by the confidence score. You can access the label text and the score through visionLabel.getText() and visionLabel.getConfidence() respectively.

Note

You may also pass options into FritzVisionLabelPredictor in order to set a confidence threshold for the results. For example, if you only want to return labels that have a confidence score of 0.7 or higher, you may specify it with the following:

// Create predictor options
FritzVisionLabelPredictorOptions options = new FritzVisionLabelPredictorOptions.Builder()
    .confidenceThreshold(0.7)
    .build();

// Pass it into the getInstance method
FritzVisionLabelPredictor visionPredictor = FritzVisionLabelPredictor.getInstance(this, options);