Custom SnapML Models

Creators can use Fritz AI Studio to train custom AI models without writing code.

1. Create a new Custom Trained project

To get started, create a new project in your Fritz AI Studio account. When selecting a Project Type, choose Custom Trained. Then, select the model type for your project (e.g. Image Labeling, Object Detection, etc.).

2. Collect data and train a model

Next, you'll need to collect and label data before training a model. You can read all about these steps in this Studio Quickstart Guide.

3. Export your Lens Studio project

Once your model is trained, you can export it as part of a pre-built Lens Studio project directly from the Models table in Fritz AI Studio.

4. Configure your model in Lens Studio

Once downloaded, unzip the project archive and open the .lsproj file in Snap Lens Studio. To test your model, complete the following steps:

note

If you already have a SnapML template and have only downloaded a model file, drag and drop your model file into the Resources pane of Lens Studio before proceeding to Step 1 below.

warning

The following instructions are for Image Segmentation projects. Instructions for other feature types may be slightly different. Specific instructions for your SnapML template can be found in the Public/README.js file included with each project archive.

Import your model

Upon opening the Lens Studio project file, you will be shown a "Model Import" dialog box. You do not need to change any of these settings. Just click "Import".

Add your model to the ML Component

Select the "ML Component" object from the Objects list in the top left corner of Lens Studio. Then, in the Inspector on the right, change the value of Model from "None" to the model included in your project.

Set the model's input texture

Again with the "ML Component" object selected, in the Inspector under the model's Inputs change the value of "Texture" from None to "Device Camera Texture". This will feed what the camera sees to your model.

Create an output texture for your model

Below the input Texture you configured in Step 3, click the "Create Output Texture" button in the model output options. This texture will be used by other Lens Studio elements to access the output of your model.

Connect the model output texture to the Fritz Segmentation Texture material

From the Resources list in the bottom left corner of Lens Studio, select the "Fritz Segmentation Texture" material. In the Inspector on the right, click the Model Output option under Graph Parameters and set the value to the output texture you created in Step 4.

Add a Photo or Video to test your model

In the Preview tool on the far right side of Lens Studio, open the Source drop-down menu by selecting the name of the default source (Person 2 in the image above). By selecting either the Photo or Video icons to the left of the Search bar, you can upload images or videos of objects your model is trained to segment. This will allow you to quickly and easily test your model.