Skip to content

Resolve "Create UI to compare different Candidates"

What does this MR do and why?

Adds list of experiments and details on the candidates associated to an experiment.

Database

Scope :latest

SELECT DISTINCT ON (candidate_id, name) *
FROM "ml_candidate_metrics"
WHERE "ml_candidate_metrics"."candidate_id" IN (68, 69, 70, 71, 72)
ORDER BY candidate_id, name, id DESC;
Unique  (cost=1.01..1.02 rows=1 width=88) (actual time=0.021..0.026 rows=10 loops=1)
  ->  Sort  (cost=1.01..1.01 rows=1 width=88) (actual time=0.021..0.022 rows=10 loops=1)
"        Sort Key: candidate_id, name, id DESC"
        Sort Method: quicksort  Memory: 26kB
        ->  Seq Scan on ml_candidate_metrics  (cost=0.00..1.00 rows=1 width=88) (actual time=0.011..0.013 rows=10 loops=1)
"              Filter: (candidate_id = ANY ('{68,69,70,71,72}'::bigint[]))"
              Rows Removed by Filter: 27
Planning Time: 0.084 ms
Execution Time: 0.041 ms

Ml::Experiment.py_project_id

SELECT "ml_experiments".* 
FROM "ml_experiments" 
WHERE "ml_experiments"."project_id" = 32 
ORDER BY "ml_experiments"."id" DESC
Sort  (cost=1.01..1.01 rows=1 width=88) (actual time=0.040..0.041 rows=3 loops=1)
  Sort Key: id DESC
  Sort Method: quicksort  Memory: 25kB
  ->  Seq Scan on ml_experiments  (cost=0.00..1.00 rows=1 width=88) (actual time=0.010..0.010 rows=3 loops=1)
        Filter: (project_id = 32)
        Rows Removed by Filter: 4
Planning Time: 0.113 ms
Execution Time: 0.053 ms

Ml::Experiment.py_project_id_and_iid

SELECT "ml_experiments".* FROM "ml_experiments" WHERE "ml_experiments"."project_id" = 32 AND "ml_experiments"."iid" = 1 LIMIT 1 /*application:web,correlation_id:01GFNM5CE3CRA0P52PEDGR5CPF,endpoint_id:Projects::Ml::ExperimentsController#show,db_config_name:main,line:/app/models/ml/experiment.rb:18:in `by_project_id_and_iid'*
Limit  (cost=0.00..1.00 rows=1 width=88) (actual time=0.028..0.028 rows=1 loops=1)
  ->  Seq Scan on ml_experiments  (cost=0.00..1.00 rows=1 width=88) (actual time=0.025..0.025 rows=1 loops=1)
        Filter: ((project_id = 32) AND (iid = 1))
        Rows Removed by Filter: 4
Planning Time: 1.003 ms
Execution Time: 0.052 ms
  • if candidate.params is used:
SELECT "ml_candidate_params".* 
FROM "ml_candidate_params" 
WHERE "ml_candidate_params"."candidate_id" 
IN (68, 69, 70, 71, 72) 
Seq Scan on ml_candidate_params  (cost=0.00..1.29 rows=1 width=43) (actual time=0.020..0.022 rows=5 loops=1)
"  Filter: (candidate_id = ANY ('{68,69,70,71,72}'::bigint[]))"
  Rows Removed by Filter: 30
Planning Time: 0.534 ms
Execution Time: 0.030 ms
  • if candidate.latest_metrics is accessed:
SELECT DISTINCT ON (candidate_id, name) * 
FROM "ml_candidate_metrics" 
WHERE "ml_candidate_metrics"."candidate_id" IN (68, 69, 70, 71, 72) 
ORDER BY candidate_id, name, id DESC /*application:web,correlation_id:01GFNMRCB6TXBFJSS9C73TQ02W,endpoint_id:Projects::Ml::ExperimentsController#show,db_config_name:main,line:/app/models/ml/experiment.rb:18:in `by_project_id_and_iid'*/
Unique  (cost=2.01..2.09 rows=8 width=88)
  ->  Sort  (cost=2.01..2.04 rows=10 width=88)
"        Sort Key: candidate_id, name, id DESC"
        ->  Seq Scan on ml_candidate_metrics  (cost=0.00..1.84 rows=10 width=88)
"              Filter: (candidate_id = ANY ('{68,69,70,71,72}'::bigint[]))"

How to set up and validate locally

Numbered steps to set up and validate the change are strongly suggested.

  1. Enable the invite modal

    echo "Feature.enable(:ml_experiment_tracking)" | bundle exec rails c
  2. Populate the database. In rails console:

    experiments = 3.times.map { |v| Ml::Experiment.create!(name: "Gitlab Experiment #{v}", user_id: user_id, project_id: project_id) }
    exp = experiments[0]
    5.times.each { |i| exp.candidates.create!(user_id: user_id, start_time: 0) }
    exp.candidates.each_with_index { |c, i| c.metrics.create!(name: "auc", value: i*0.1 , tracked_at: Time.zone.now, step: 1)} 
    exp.candidates.each_with_index { |c, i| c.metrics.create!(name: "accuracy", value: i* 0.1+0.1 , tracked_at: Time.zone.now, step: 1)} 
    exp.candidates.each { |c| c.params.create!(name: "algorithm", value: ["LogisticRegression", "DecisionTree"].sample )} 
  3. Visit http://gdk.test:3000/root/ml-test/-/ml/experiments/, it should show the follow: image

  4. Visit http://gdk.test:3000/root/ml-test/-/ml/experiments/2, or click Gitlab Experiment 1, it should show the follow: image

  5. Visit http://gdk.test:3000/root/ml-test/-/ml/experiments/1, or click Gitlab Experiment 0, it should show the follow: image

  6. Without feature flag enabled, http://gdk.test:3000/root/ml-test/-/ml/experiments/ should give 404

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Related to #370480 (closed)

Edited by Eduardo Bonet

Merge request reports

Loading