Skip to content

Implement DSL to define additional scope for batched migration

Krasimir Angelov requested to merge 357900-batched-job-dsl-define-batch-scope into master

What does this MR do and why?

Introduce new scope_to helper methods so that batched migration jobs can define additional filter to be used by PrimaryKeyBatchingStrategy and the job itself.

Example

In case we need to iterate only over subset of table, we can have batched migration job class like

class BackfillNamespaceType < BatchedMigrationJob
  scope_to ->(relation) { relation.where(type: nil) }

  def perform
    each_sub_batch(operation_name: :update_all) do |sub_batch|
      sub_batch.update_all(type: 'User')
    end
  end
end

This job will iterate only records that do not have type assigned and update the value to User.

Later MR will update the few existing jobs that subclass PrimaryKeyBatchingStrategy ans use apply_additional_filters to use this new approach, so that we can eventually remove apply_additional_filters.

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Related to #357900 (closed)

Edited by Krasimir Angelov

Merge request reports