Add helper for batched background migrations
What does this MR do?
In the previous MR !54628 (merged), a new dynamic method of scheduling background migrations that batch over a table was introduced. This MR adds a new migration helper that can be used to kickoff a batched migration, by creating the tracking record in the database.
A previous version of this MR also added a corresponding method that could be used in the down
operation to abort a batched migration, but I have removed it for now. I think it's too early to say how we will want to handle this to introduce a method that will most likely change in a future iteration.
For now, we would have to handle the migration rollback manually, which should be fine as this new approach will have a limited rollout in one future migration: !51332 (merged)
The main reason for introducing this MR's queue_batched_background_migration
is that it will be used by the backfill_conversion_of_integer_to_bigint
method used in the events
conversion.
Example migration using them:
class ConvertEventsIdColumnToBigint < ActiveRecord::Migration[6.0]
include Gitlab::Database::MigrationHelpers
DOWNTIME = false
JOB_CLASS_NAME = 'CopyColumnUsingBackgroundMigrationJob'
def up
queue_batched_background_migration JOB_CLASS_NAME, :events, :id, 'id', 'id_convert_to_bigint',
job_interval: 2.minutes, batch_size: 100, sub_batch_size: 25
end
def down
# For now, we would manually abort the appropriate migration records
end
end
For details about the motivation/plan behind the overarching change, see: #300915 (closed)
Screenshots (strongly suggested)
Does this MR meet the acceptance criteria?
Conformity
-
📋 Does this MR need a changelog?- [-] I have included a changelog entry.
- [-] I have not included a changelog entry because _____.
- [-] Documentation (if required)
-
Code review guidelines -
Merge request performance guidelines -
Style guides -
Database guides - [-] Separation of EE specific content
Availability and Testing
-
Review and add/update tests for this feature/bug. Consider all test levels. See the Test Planning Process. - [-] Tested in all supported browsers
- [-] Informed Infrastructure department of a default or new setting change, if applicable per definition of done
Security
If this MR contains changes to processing or storing of credentials or tokens, authorization and authentication methods and other items described in the security review guidelines:
- [-] Label as security and @ mention
@gitlab-com/gl-security/appsec
- [-] The MR includes necessary changes to maintain consistency between UI, API, email, or other methods
- [-] Security reports checked/validated by a reviewer from the AppSec team