Finish migration to the new events setup
This MR cleans up / finishes the process of migrating events to the new database setup, including the removal of the old migrations table. For GitLab.com we expect to save at least 165 GB of storage with these changes.
Related issues/MRs:
- https://gitlab.com/gitlab-org/gitlab-ce/issues/31806
- https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/12463
- https://gitlab.com/gitlab-org/gitlab-ce/issues/37241
Migration Output
This excludes the steal migration since it doesn't produce any meaningful output.
== SwapEventMigrationTables: migrating =======================================
-- rename_table(:events, :events_old)
-> 0.1747s
-- rename_table(:events_for_migration, :events)
-> 0.0257s
-- rename_table(:events_old, :events_for_migration)
-> 0.0345s
== SwapEventMigrationTables: migrated (0.2358s) ==============================
== DropEventsForMigrationTable: migrating ====================================
-- transaction()
-- drop_table(:events_for_migration)
-> 0.0812s
-> 3.2884s
== DropEventsForMigrationTable: migrated (3.3997s) ===========================
Migration Timings
Migration | Time |
---|---|
SwapEventMigrationTables | 0.2 seconds |
DropEventsForMigrationTable | 3.4 seconds |
Production Impact
The process should have no impact visible to users. The migrations can execute in parallel to the running application. This was verified by running a simple loop that queries events
every 1 second, which produced no errors.
Size Impact
This migration will save around 165 GB based on recent estimates. I re-verified this today to be certain using a replica and a separate database restored from production. The replica had 941 GB of data, while the restored host (after the migrations) has 759 GB of data; 182 GB less.
Database Checklist
When adding migrations:
-
Updated db/schema.rb
-
Added a down
method so the migration can be reverted -
Added the output of the migration(s) to the MR body -
Added the execution time of the migration(s) to the MR body -
Made sure the migration won't interfere with a running GitLab cluster, for example by disabling transactions for long running migrations
When removing columns, tables, indexes or other structures:
-
Removed these in a post-deployment migration -
Made sure the application no longer uses (or ignores) these structures
General Checklist
-
Changelog entry added, if necessary -
Tests added for this feature/bug - Review
-
Has been reviewed by Backend -
Has been reviewed by Database
-
-
Conform by the merge request performance guides -
Conform by the style guides -
Squashed related commits together