Add locked column to Ci::JobArtifact
What does this MR do?
- Adds
locked
column toCi::JobArtifact
. - Schedules a partial index on
expire_at
wherelocked = 0
to be added asynchronously during a weekend toci_job_artifacts
.
This will allow us to query ci_job_artifacts
for unlocked and expired artifacts without needing to join ci_builds
and ci_pipelines
in order to get this information.
This is part of the work in #327281 (closed) and #322817 (closed) to resolve the issue we have where expired and unlocked job artifacts are not being cleaned up.
Remaining steps after this MR
- Verify scheduled index has been created on gitlab.com.
- Add the index for self-managed using
add_concurrent_index
. - Open an MR to propagate the value from
Ci::Pipeline
toCi::JobArtifact
for new pipelines and artifacts. - Update
DestroyAllExpiredService
to use the new column and index. - Backfill the column.
Database
Migrate up
== 20210826110839 PrepareIndexesForCiJobArtifactsExpireAtUnlocked: migrating ==
== 20210826110839 PrepareIndexesForCiJobArtifactsExpireAtUnlocked: migrated (0.0035s)
== 20210826120834 AddLockedToCiJobArtifacts: migrating ========================
-- add_column(:ci_job_artifacts, :locked, :smallint, {:default=>2})
-> 0.0022s
== 20210826120834 AddLockedToCiJobArtifacts: migrated (0.0239s) ===============
Migrate down
== 20210826120834 AddLockedToCiJobArtifacts: reverting ========================
-- remove_column(:ci_job_artifacts, :locked, :smallint, {:default=>2})
-> 0.0015s
== 20210826120834 AddLockedToCiJobArtifacts: reverted (0.0243s) ===============
== 20210826110839 PrepareIndexesForCiJobArtifactsExpireAtUnlocked: reverting ==
== 20210826110839 PrepareIndexesForCiJobArtifactsExpireAtUnlocked: reverted (0.0018s)
Index to be added asynchronously over a weekend
CREATE INDEX CONCURRENTLY ci_job_artifacts_expire_at_unlocked_idx ON ci_job_artifacts (expire_at) WHERE locked = 0;
Does this MR meet the acceptance criteria?
Conformity
-
I have included changelog trailers, or none are needed. (Does this MR need a changelog?) -
I have added/updated documentation, or it's not needed. (Is documentation required?) -
I have properly separated EE content from FOSS, or this MR is FOSS only. (Where should EE code go?) -
I have added information for database reviewers in the MR description, or it's not needed. (Does this MR have database related changes?) -
I have self-reviewed this MR per code review guidelines. -
This MR does not harm performance, or I have asked a reviewer to help assess the performance impact. (Merge request performance guidelines) -
I have followed the style guides. -
This change is backwards compatible across updates, or this does not apply.
Availability and Testing
-
I have added/updated tests following the Testing Guide, or it's not needed. (Consider all test levels. See the Test Planning Process.) -
I have tested this MR in all supported browsers, or it's not needed. -
I have informed the Infrastructure department of a default or new setting change per definition of done, or it's not needed.
Related to #327281 (closed)
Edited by Matt Kasa