Run ClickHouse migrations during gitlab:db:configure
What does this MR do and why?
This MR addresses:
- Clickhouse - automatically run migrations durin... (omnibus-gitlab#8266 - closed)
- Clickhouse - automatically run migrations durin... (gitlab-org/charts/gitlab#5043 - closed)
I initially tried to implement it directly on omnibus and charts: Draft: Automatically migrate ClickHouse databases (omnibus-gitlab!7272 - closed)
But then realized the both omnibus and charts right now just call the gitlab:db:configure
rake task and expect it to do all the work.
So this MR just hooks into that task.
Key things to look at:
- migrations moved to the
db/clickhouse/migrate/main
directory - we now have
gitlab:clickhouse:migrate:main
andgitlab:clickhouse:rollback:main
rake tasks - The
gitlab:clickhouse:rollback
rollback task is gone, because it no longer makes sense - But there's
gitlab:clickhouse:migrate
which just runs migrations for all databases (onlymain
for now) -
gitlab:db:configure
now also runsgitlab:clickhouse:migrate
, but it skips the clickhouse migrations if the connection is not configured. - running migrations automatically is also hidden behind the feature flag, just in case.
Screenshots or screen recordings
Tests
./bin/rails gitlab:clickhouse:rollback:main VERSION=0
== 20231114142100 CreateAuditEvents: reverting ================================
== 20231114142100 CreateAuditEvents: reverted (0.0976s) =======================
== 20231106202300 ModifyCiFinishedBuildsSettings: reverting ===================
== 20231106202300 ModifyCiFinishedBuildsSettings: reverted (0.0098s) ==========
== 20230808140217 CreateCiFinishedBuildsAggregatedQueueingDelayPercentilesMv: reverting
== 20230808140217 CreateCiFinishedBuildsAggregatedQueueingDelayPercentilesMv: reverted (0.0087s)
== 20230808070520 CreateSyncCursors: reverting ================================
== 20230808070520 CreateSyncCursors: reverted (0.0949s) =======================
== 20230724064918 CreateContributionAnalyticsEventsMaterializedView: reverting
== 20230724064918 CreateContributionAnalyticsEventsMaterializedView: reverted (0.0084s)
== 20230724064832 CreateContributionAnalyticsEvents: reverting ================
== 20230724064832 CreateContributionAnalyticsEvents: reverted (0.0958s) =======
== 20230719101806 CreateCiFinishedBuildsAggregatedQueueingDelayPercentiles: reverting
== 20230719101806 CreateCiFinishedBuildsAggregatedQueueingDelayPercentiles: reverted (0.0988s)
== 20230707151359 CreateCiFinishedBuilds: reverting ===========================
== 20230707151359 CreateCiFinishedBuilds: reverted (0.1162s) ==================
== 20230705124511 CreateEvents: reverting =====================================
== 20230705124511 CreateEvents: reverted (0.1364s) ============================
mv config/click_house.yml config/click_house.yml2
./bin/rails gitlab:db:configure
Running db:migrate:main rake task
main: == [advisory_lock_connection] object_id: 182620, pg_backend_pid: 34210
main: == [advisory_lock_connection] object_id: 182620, pg_backend_pid: 34210
Running db:migrate:ci rake task
ci: == [advisory_lock_connection] object_id: 185040, pg_backend_pid: 34927
ci: == [advisory_lock_connection] object_id: 185040, pg_backend_pid: 34927
Running gitlab:clickhouse:migrate:main rake task
The 'main' ClickHouse database is not configured, skipping migrations
./bin/rails gitlab:clickhouse:migrate
Running gitlab:clickhouse:migrate:main rake task
rails aborted!
ClickHouse::Client::ConfigurationError: The database 'main' is not configured
/Users/vlad/gdk/gitlab/gems/click_house-client/lib/click_house/client.rb:95:in `block in lookup_database'
/Users/vlad/gdk/gitlab/gems/click_house-client/lib/click_house/client.rb:94:in `lookup_database'
/Users/vlad/gdk/gitlab/gems/click_house-client/lib/click_house/client.rb:100:in `instrumented_execute'
/Users/vlad/gdk/gitlab/gems/click_house-client/lib/click_house/client.rb:35:in `select'
/Users/vlad/gdk/gitlab/lib/click_house/connection.rb:11:in `select'
/Users/vlad/gdk/gitlab/lib/click_house/connection.rb:29:in `table_exists?'
/Users/vlad/gdk/gitlab/lib/click_house/migration_support/schema_migration.rb:12:in `ensure_table'
/Users/vlad/gdk/gitlab/lib/tasks/gitlab/click_house/migration.rake:73:in `migrate'
/Users/vlad/gdk/gitlab/lib/tasks/gitlab/click_house/migration.rake:16:in `block (5 levels) in <main>'
/Users/vlad/gdk/gitlab/lib/tasks/gitlab/click_house/migration.rake:35:in `block (4 levels) in <main>'
/Users/vlad/gdk/gitlab/lib/tasks/gitlab/click_house/migration.rake:33:in `each'
/Users/vlad/gdk/gitlab/lib/tasks/gitlab/click_house/migration.rake:33:in `block (3 levels) in <main>'
Tasks: TOP => gitlab:clickhouse:migrate:main
(See full trace by running task with --trace)
mv config/click_house.yml2 config/click_house.yml
./bin/rails gitlab:db:configure
Running db:migrate:main rake task
main: == [advisory_lock_connection] object_id: 182660, pg_backend_pid: 47120
main: == [advisory_lock_connection] object_id: 182660, pg_backend_pid: 47120
Running db:migrate:ci rake task
ci: == [advisory_lock_connection] object_id: 185080, pg_backend_pid: 47831
ci: == [advisory_lock_connection] object_id: 185080, pg_backend_pid: 47831
Running gitlab:clickhouse:migrate:main rake task
== 20230705124511 CreateEvents: migrating =====================================
== 20230705124511 CreateEvents: migrated (0.0203s) ============================
== 20230707151359 CreateCiFinishedBuilds: migrating ===========================
== 20230707151359 CreateCiFinishedBuilds: migrated (0.0148s) ==================
== 20230719101806 CreateCiFinishedBuildsAggregatedQueueingDelayPercentiles: migrating
== 20230719101806 CreateCiFinishedBuildsAggregatedQueueingDelayPercentiles: migrated (0.0114s)
== 20230724064832 CreateContributionAnalyticsEvents: migrating ================
== 20230724064832 CreateContributionAnalyticsEvents: migrated (0.0150s) =======
== 20230724064918 CreateContributionAnalyticsEventsMaterializedView: migrating
== 20230724064918 CreateContributionAnalyticsEventsMaterializedView: migrated (0.0172s)
== 20230808070520 CreateSyncCursors: migrating ================================
== 20230808070520 CreateSyncCursors: migrated (0.0127s) =======================
== 20230808140217 CreateCiFinishedBuildsAggregatedQueueingDelayPercentilesMv: migrating
== 20230808140217 CreateCiFinishedBuildsAggregatedQueueingDelayPercentilesMv: migrated (0.0108s)
== 20231106202300 ModifyCiFinishedBuildsSettings: migrating ===================
== 20231106202300 ModifyCiFinishedBuildsSettings: migrated (0.0342s) ==========
== 20231114142100 CreateAuditEvents: migrating ================================
== 20231114142100 CreateAuditEvents: migrated (0.0307s) =======================
== 20231129062064 CreateContributionsTable: migrating =========================
== 20231129062064 CreateContributionsTable: migrated (0.0120s) ================
== 20231129062151 CreateContributionsMv: migrating ============================
== 20231129062151 CreateContributionsMv: migrated (0.0114s) ===================
== 20231205104100 ModifyCiFinishedBuildsStartedAtDefault: migrating ===========
== 20231205104100 ModifyCiFinishedBuildsStartedAtDefault: migrated (0.0125s) ==
== 20231205104101 ModifyCiFinishedBuildsFinishedAtDefault: migrating ==========
== 20231205104101 ModifyCiFinishedBuildsFinishedAtDefault: migrated (0.0121s) =
== 20231205112200 FixInvalidCiFinishedBuildsStartedAtValues: migrating ========
== 20231205112200 FixInvalidCiFinishedBuildsStartedAtValues: migrated (0.0110s)
./bin/rails gitlab:clickhouse:migrate
Running gitlab:clickhouse:migrate:main rake task
No migrations ran.
./bin/rails gitlab:clickhouse:rollback:main STEP=2
== 20231205112200 FixInvalidCiFinishedBuildsStartedAtValues: reverting ========
== 20231205112200 FixInvalidCiFinishedBuildsStartedAtValues: reverted (0.0000s)
== 20231205104101 ModifyCiFinishedBuildsFinishedAtDefault: reverting ==========
== 20231205104101 ModifyCiFinishedBuildsFinishedAtDefault: reverted (0.0141s) =
How to set up and validate locally
- In rails console enable the experiment fully
Feature.enable(:run_clickhouse_migrations_automatically)
- repeat commands from the section above
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.
Edited by Vladimir Shushlin