Save background migration exceptions into sampling artifacts
requested to merge 65-display-that-a-background-migration-has-failed-testing-if-any-batch-has-failed into master
What does this MR do and why?
Currently we don't display the success / failure of background migrations (in the comment generated by db:gitlabcom-database-testing
job).
These changes will ensure we generate sampling files with correct data for later use in https://gitlab.com/gitlab-org/database-team/gitlab-com-database-testing project.
How to set up and validate locally
- Create a new Batched Background Migration
bundle exec rails g batched_background_migration failure_migration --table_name=users --column_name=id --feature_category=database
Use the following classes:
# db/post_migrate/xxx_queue_failure_migration.rb
class QueueFailureMigration < Gitlab::Database::Migration[2.1]
MIGRATION = "FailureMigration"
DELAY_INTERVAL = 2.minutes
BATCH_SIZE = 300
SUB_BATCH_SIZE = 100
disable_ddl_transaction!
restrict_gitlab_migration gitlab_schema: :gitlab_main
def up
queue_batched_background_migration(
MIGRATION,
:users,
:id,
job_interval: DELAY_INTERVAL,
batch_size: BATCH_SIZE,
sub_batch_size: SUB_BATCH_SIZE
)
end
def down
delete_batched_background_migration(MIGRATION, :users, :id, [])
end
end
# lib/gitlab/background_migration/failure_migration.rb
module Gitlab
module BackgroundMigration
class FailureMigration < BatchedMigrationJob
operation_name :update_all
feature_category :database
def perform
each_sub_batch do |_sub_batch|
random_action.sample.call
end
end
private
def random_action
[
-> () { raise 'Error!' },
-> () { Logger.new($stdout).info('== MIGRATED ==') }
]
end
end
end
end
- Add random some users
2500.times do |i|
begin
User.create!(
username: FFaker::Internet.user_name,
name: FFaker::Name.name,
email: FFaker::Internet.email,
confirmed_at: DateTime.now,
password: SecureRandom.hex.slice(0,16)
)
rescue ActiveRecord::RecordInvalid
print 'Fail'
end
end
- Run migrations. Migration will fail, because we're raising an error in the
FailureMigration
class
bundle exec rails db:migrate
- Run BBM sampling tests
mkdir -p tmp/migration-testing/main/background_migrations
echo "0" > tmp/migration-testing/main/background_migrations/last-batched-background-migration-id.txt
bundle exec rake gitlab:db:migration_testing:sample_batched_background_migrations:main
- Check your sampling files
- Check
tmp/migration-testing/main/background_migrations/FailureMigration/batch_<number>/migration-stats.json
- If the batch failed, it will shown the
success: false
and theerror_message: "Error!"
. Otherwise,success
should betrue
anderror_message: null
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.
Related to gitlab-org/database-team/gitlab-com-database-testing#65 (closed)
Edited by Leonardo da Rosa