Add single db variant of partitioning rake tasks
What does this MR do and why?
Follow up from !82213 (merged) to update partitioning rake tasks with single-database variants when multiple databases are configured.
With multiple databases configured, partition management can now be run against a single database, like:
$ rails gitlab:db:create_dynamic_partitions:main
Also, for other rake tasks that manage partitions (db:migrate
, db:schema:load
), this change will only run partition management for the matching database. For example, when running
$ rails db:migrate:ci
it will call rails gitlab:db:create_dynamic_partitions:ci
instead of rails gitlab:db:create_dynamic_partitions
How to set up and validate locally
- Setup your development environment with multiple databases as described here: https://docs.gitlab.com/ee/development/database/multiple_databases.html#development-setup
- Verify that the different rake tasks are present:
$ rails -T gitlab:db:create_dynamic_partitions rails gitlab:db:create_dynamic_partitions # Create missing dynamic database partitions rails gitlab:db:create_dynamic_partitions:ci # Create missing dynamic database partitions on the ci database rails gitlab:db:create_dynamic_partitions:main # Create missing dynamic database partitions on the main database
- Tail the development log file
$ tail -f log/development.log
- Run the ci rake task with:
$ DISABLE_POSTGRES_PARTITION_CREATION_ON_STARTUP=true GITLAB_USE_MODEL_LOAD_BALANCING=true rails gitlab:db:create_dynamic_partitions:ci
- Check the log output, to see that partition management only ran for
ci
. It should be at the end, after the app is loaded, similar to the following. At this time it will only run forloose_foreign_keys
and only fordb_config_name:ci
.Creating scope :within_timeframe. Overwriting existing method Iteration.within_timeframe. Creating scope :opened. Overwriting existing method Epic.opened. Creating scope :closed. Overwriting existing method Epic.closed. Creating scope :order_created_desc. Overwriting existing method Packages::Package.order_created_desc. Creating scope :order_name_desc. Overwriting existing method Packages::Package.order_name_desc. Gitlab::Database::PostgresPartition Load (5.1ms) SELECT "postgres_partitions".* FROM "postgres_partitions" WHERE (parent_identifier = concat(current_schema(), '.', 'loose_foreign_keys_deleted_records')) ORDER BY "postgres_partitions"."name" ASC /*application:web,db_config_name:ci,line:/lib/gitlab/database/partitioning/sliding_list_strategy.rb:21:in `map'*/ ↳ lib/gitlab/database/partitioning/sliding_list_strategy.rb:21:in `map' Gitlab::Database::PostgresPartition Load (4.1ms) SELECT "postgres_partitions".* FROM "postgres_partitions" WHERE (parent_identifier = concat(current_schema(), '.', 'loose_foreign_keys_deleted_records')) ORDER BY "postgres_partitions"."name" ASC /*application:web,db_config_name:ci,line:/lib/gitlab/database/partitioning/sliding_list_strategy.rb:21:in `map'*/ ↳ lib/gitlab/database/partitioning/sliding_list_strategy.rb:21:in `map' LooseForeignKeys::DeletedRecord Load (0.8ms) SELECT "loose_foreign_keys_deleted_records"."id", "loose_foreign_keys_deleted_records"."created_at" FROM "loose_foreign_keys_deleted_records" WHERE "loose_foreign_keys_deleted_records"."partition" = 1 ORDER BY "loose_foreign_keys_deleted_records"."id" ASC LIMIT 1 /*application:web,db_config_name:ci,line:/app/models/loose_foreign_keys/deleted_record.rb:20:in `block in <class:DeletedRecord>'*/ ↳ app/models/loose_foreign_keys/deleted_record.rb:20:in `block in <class:DeletedRecord>' Gitlab::Database::PostgresPartition Load (4.1ms) SELECT "postgres_partitions".* FROM "postgres_partitions" WHERE (parent_identifier = concat(current_schema(), '.', 'loose_foreign_keys_deleted_records')) ORDER BY "postgres_partitions"."name" ASC /*application:web,db_config_name:ci,line:/lib/gitlab/database/partitioning/sliding_list_strategy.rb:21:in `map'*/ ↳ lib/gitlab/database/partitioning/sliding_list_strategy.rb:21:in `map' (5.0ms) SELECT columns.column_default AS default_value FROM information_schema.columns columns WHERE columns.column_name = 'partition' AND columns.table_name = 'loose_foreign_keys_deleted_records' /*application:web,db_config_name:ci,line:/lib/gitlab/database/partitioning/sliding_list_strategy.rb:82:in `current_default_value'*/ ↳ lib/gitlab/database/partitioning/sliding_list_strategy.rb:82:in `current_default_value'
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.
Edited by Patrick Bair