Migrate old traces to artifacts
requested to merge feature/sm/4180-trace-as-artifacts-migrate into feature/sm/4180-trace-as-artifacts-os-support
What does this MR do?
Concept
Today, we have four traces.
Old to New
- ci_builds.trace (In Database)
- Gitlab::Ci::Trace#deprecated_path (with project.ci_id) e.g. builds/2018_01/10/12/1021.log
- Gitlab::Ci::Trace#default_path (without project.ci_id) e.g. builds/2018_01/12/1021.log
- JobArtifactUploader#default_path (Same with artifacts) e.g. shared/artifacts/4a/44/4a44dc15364204a80fe80e9039455cc1608281820fe2b24f1e5233ade6af1dd5/2017_12_14/938/1/trace.log
We move 1, 2 and 3 to 4.
-
Calcurate estimated processing time https://gitlab.com/gitlab-org/gitlab-ee/issues/4639#note_55581146
(Job_count/BATCH_SIZE) * delay = Estimated timing (i.e. The completion of the last queue)
e.g. (20000000 / 10000) * 2 min = 4000 min ≒ 2.7 days
-
Note: When adding background migrations you must make sure they are announced in the monthly release post along with an estimate of how long it will take to complete the migrations. https://docs.gitlab.com/ce/development/background_migrations.html#cleaning-up -
Target jobs should be completed, otherwise live-trace will be considered as un-migrated data (This will be artifacts when it's done)
Are there points in the code the reviewer needs to double check?
Why was this MR needed?
Screenshots (if relevant)
Does this MR meet the acceptance criteria?
-
Changelog entry added, if necessary -
Documentation created/updated -
API support added -
Tests added for this feature/bug - Review
-
Has been reviewed by UX -
Has been reviewed by Frontend -
Has been reviewed by Backend -
Has been reviewed by Database
-
-
Conform by the merge request performance guides -
Conform by the style guides -
Squashed related commits together -
Internationalization required/considered -
If paid feature, have we considered GitLab.com plan and how it works for groups and is there a design for promoting it to users who aren't on the correct plan
What are the relevant issue numbers?
close https://gitlab.com/gitlab-org/gitlab-ee/issues/4639 close https://gitlab.com/gitlab-org/gitlab-ce/issues/34317
Edited by Shinya Maeda