Skip to content

Adding Model Version Metadata to MLflow

Darby Frey requested to merge model-versions-metadata into master

What does this MR do and why?

Related to [MLflow compatibility] Store ModelVersion tags ... (#434409 - closed) this MR adds support for model version metadata in the DB and exposes that data in the MLflow API via the tags attribute on the model_version resource.

How to set up and validate locally

  1. In the Rails console, ensure the feature flag is enabled

    Feature.enable(:ml_experiment_tracking)
Feature.enable(:model_registry)

  2. In the Rails console, create an ML model in the project

    Ml::CreateModelService.new(Project.find(PROJECT_ID), 'my-model-name').execute

  3. Enable the Model Registry feature toggle in the project by going to Settings > General > Visibility, project features, permissions and flipping the toggle for Model Registry

  4. Use the following cURL command to create a model in a local GDK project

    curl -X "POST" "http://GDKHOST/api/v4/projects/PROJECT_ID/ml/mlflow/api/2.0/mlflow/model-versions/create" \
    -H 'Authorization: Bearer PERSONAL_ACCESS_TOKEN' \
    -H 'Content-Type: application/json; charset=utf-8' \
    -d $'{
  "name": "my-model-name",
  "tags": [
    {
      "key": "key1",
      "value": "value1"
    },
    {
      "key": "key2",
      "value": "value2"
    }
  ],
  "description": "My Model Description"
}'

  5. Use the get API to look up the newly created entry.

    curl -X "GET" "http://GDKHOST/api/v4/projects/PROJECT_ID/ml/mlflow/api/2.0/mlflow/model-versions/get?name=my-model-name&version=1.0.0" \
    -H 'Authorization: Bearer PERSONAL_ACCESS_TOKEN'

Database Review

Up 
main: == [advisory_lock_connection] object_id: 181940, pg_backend_pid: 33159
main: == 20231215192706 CreateMlModelVersionMetadata: migrating =====================
main: -- create_table(:ml_model_version_metadata)
main: -- quote_column_name(:name)
main:    -> 0.0000s
main: -- quote_column_name(:value)
main:    -> 0.0000s
main:    -> 0.0094s
main: == 20231215192706 CreateMlModelVersionMetadata: migrated (0.1351s) ============

main: == [advisory_lock_connection] object_id: 181940, pg_backend_pid: 33159
Down 
main: == [advisory_lock_connection] object_id: 182640, pg_backend_pid: 32047
main: == 20231215192706 CreateMlModelVersionMetadata: reverting =====================
main: -- drop_table(:ml_model_version_metadata)
main:    -> 0.0029s
main: == 20231215192706 CreateMlModelVersionMetadata: reverted (0.0078s) ============

main: == [advisory_lock_connection] object_id: 182640, pg_backend_pid: 32047

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Darby Frey

Merge request reports