Manage Model VersionsΒΆ

Listing all models in your project

To list all models in your current project run:

$ fritz models

    Model Name            | Framework | Active Version | Created          | Updated          | Model ID
    ---------------------------------------------------------------------------------------------------
    model_1               | keras     |              2 | 2019-06-27 15:57 | 2019-07-02 18:25 | abc123
    model_1_tflite        | tflite    |              2 | 2019-06-27 17:47 | 2019-06-28 14:24 | def456
    model_1_coreml        | coreml    |              2 | 2019-06-19 18:05 | 2019-06-27 21:34 | ghi789

The resulting list contains the name, format, active version, created and updated dates, and model IDs for all of the models available in your project. In this context, a model and corresponding unique Model ID represents a group of model versions that all perform the same task and require the same inputs, outputs, and format. Typically, only one of these versions (the Active Version) is in use in your application at any given time. The Model ID in the result above ties all of these versions together and should be placed in your application code.

For example, imagine you are building an app to detect the presence of pizza in an image. The app needs to run on iOS (Core ML) and Android (TensorFlow Lite), and you decide to use Keras for training. While experimenting, you train two different versions of the same MobileNet model with different hyperparameters and find that the last one performs the best. In the result above, you see three lines related to your pizza models, one for each format (keras, tflite, and coreml) and note that the active version was set to 2. .. note:

As far as your mobile app is concerned, different versions of a model are interchangable. As long as the inputs, outputs, and format remain the same, you can swap them as desired.

Getting more details on your model

You can see a more detailed view of a model by running

$ fritz model <model id>

    Model: model_1_coreml
    ID: ghi789
    ---------------------------------------
    Created At      : 2019-06-19 18:05
    Description     : None
    Model Format    : coreml
    Project ID      : project-abc-123
    Active Version  : 2
    Is Global       : False


    2 model versions in model_1_coreml

    Provided Filename         | Version | Created          | Model Size | Version ID | Loss
    -------------------------------------------------------------------------------------------
    model_1_coreml_v2.mlmodel |      2 | 2019-06-27 22:34 |    14.8 MB | xyz098      |   0.1413
    model_1_coreml_v1.mlmodel |      1 | 2019-06-27 21:34 |    14.8 MB | wvu765      |   0.1086


    Usage: fritz model [OPTIONS] [MODEL_ID] COMMAND [ARGS]...

    Commands for working with Fritz Models.

    Options:
    --help  Show this message and exit.

    Commands:
    benchmark  Get model grader report for a model version or Keras file.
    details    Get details about specific model.
    download   Download model version.
    update     Update model.
    upload     Upload model version to Fritz API.
    version    Commands related to specific versions of a model.
    versions   List Model Versions for a specific model_id.

First, you get a detailed print out of the this model, followed by a table showing all of the versions. Finally, you are presented with a list of commands you can run to interact with your models or get even more information.

Seeing metadata associated with a specific model version

You can query for the details of specific model versions by running

$ fritz model <model id> details <version id>

    ModelVersion: model_1_coreml_v2.mlmodel
            ID: xyz098
    --------------------------------------------------------------
    Model Size  : 14.8 MB
    Filename    : model_1_coreml_v2.mlmodel
    Version     : 2
    Metadata    : {"learning_rate": 0.001}

The result shows the specific name of the model file, the version id, the model size, and any associated metadata.

Every model version in Fritz can be stored along with key : value metadata pairs containing anything you find useful for organization. For example, you may store the training configuration of the model with hyperparameters like learning rate or batch size or the final loss and accuracy value. You can read more about adding metadata to your models here.

By inspecting model version details, you can sift through and identify the exact model you want, right from the command line.

Downloading models from the command line

You can download models and model versions right from the command line. To download the current active version of a model, run:

$ fritz model <model id> download

By adding the --version option, you can download a specific version of a model:

$ fritz model <model id> download --version <integer model version>

Note

The version is an integer and not the Version ID which is a unique hash.

Uploading models from the command line

You can also upload models with the Fritz CLI. To upload a completely new model to Fritz, run:

$ fritz model upload <path/to/checkpoint>

    Model: new_model
    ID: asd123
    -----------------------------------------------------
    Created At      : 2019-06-19 20:42
    Description     : None
    Model Format    : keras
    Project ID      : project-abc-123
    Active Version  : 1
    Is Global       : False
    ModelVersion: new_model_v1.h5
            ID: poi092
    ---------------------------------------------------------------
    Model Size  : 5.55 MB
    Filename    : new_model_v1.h5
    Version     : 1
    Metadata    : {}
    Loss        : None

Upon success, the output provides you with the newly created model ID you can add to your app.

To upload a new version of an existing model, run:

$ fritz model <model ID> upload <path/to/checkpoint>

By default, a new version is created, but not set to active. Setting a model to active would ship the new version to all app users. You can do this explicitly by adding the --deploy option to the command above.

You can verify that our model has been uploaded properly by running the fritz models or fritz model <model id> versions command.