Cleanup legacy background migration module usages from Partition helpers
What does this MR do and why?
This is the next step in closing #365354 (closed), with this all legacy background migration helper usages are removed from Table Partition helpers.
How to set up and validate locally
- Create a dummy table (to partition) with created_at and add some entries.
CREATE TABLE _test_dummy_source_table ( id SERIAL NOT NULL PRIMARY KEY, content VARCHAR(100), created_at timestamptz NOT NULL ); INSERT INTO _test_dummy_source_table (content, created_at) VALUES ('apr', '2023/04/01'); INSERT INTO _test_dummy_source_table (content, created_at) VALUES ('may', '2023/05/01');
- Add '_test_dummy_source_table' to ALLOWED_TABLES.
ALLOWED_TABLES = %w[audit_events web_hook_logs _test_dummy_source_table].freeze
- Run the below migration to the partition source table. This will create necessary partitioned_table (_test_dummy_source_table_part_a24b0274e4).
# frozen_string_literal: true class PartitionTestDummySourceTable < Gitlab::Database::Migration[2.1] include Gitlab::Database::PartitioningMigrationHelpers disable_ddl_transaction! def up partition_table_by_date :_test_dummy_source_table, :created_at end def down # Just testing end end
-
Feature.disable(:execute_batched_migrations_on_schedule)
to avoid executing batched background migrations in local. - Run a migration to enqueue backfilling batched background migration,
# frozen_string_literal: true class BackfillPartitionedTestDummySourceTable < Gitlab::Database::Migration[2.1] include Gitlab::Database::PartitioningMigrationHelpers disable_ddl_transaction! restrict_gitlab_migration gitlab_schema: :gitlab_shared def up enqueue_partitioning_data_migration :_test_dummy_source_table end def down # Just testing end end
- Now we should be able to see the BBM record created with status: 1 (active).
bm = Gitlab::Database::BackgroundMigration::BatchedMigration.find_by(table_name: '_test_dummy_source_table') bm.status
- Also check the presence of partitioned table (test_dummy_source_table_part_a24b0274e4), without any records.
gitlabhq_development=# SELECT * from _test_dummy_source_table_part_a24b0274e4; id | content | created_at ----+---------+------------ (0 rows)
- Run a migration to finalize the enqueued batched background migration,
# frozen_string_literal: true class FinalizeTestDummySourceTableBackill < Gitlab::Database::Migration[2.1] include Gitlab::Database::PartitioningMigrationHelpers disable_ddl_transaction! restrict_gitlab_migration gitlab_schema: :gitlab_shared def up finalize_backfilling_partitioned_table :_test_dummy_source_table end def down # no op end end
-
bm.reload
=> The migration should be finished now (with status 3). - And test_dummy_source_table_part_a24b0274e4 will be having all data copied.
gitlabhq_development=# SELECT * from _test_dummy_source_table_part_a24b0274e4; id | content | created_at ----+---------+------------------------ 1 | apr | 2023-04-01 00:00:00+00 2 | may | 2023-05-01 00:00:00+00 (2 rows)
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.
Related to #365354 (closed)
Edited by Prabakaran Murugesan