Add basic migration support for ClickHouse
What does this MR do and why?
NOTE: This MR looks very large, but more than half of the changes are renames or new fixtures.
Currently, the ClickHouse tables are setup by ClickHouseTestRunner#ensure_schema
, by running db/click_house/main/*.sql
files against the database.
This MR adds basic support for ClickHouse migrations:
-
first commit: support classes inspired by the Active Record equivalent (https://github.com/rails/rails/blob/main/activerecord/lib/active_record/migration.rb and the clickhouse-activerecord
gem). I've chosen to not rely on the equivalent ActiveRecord classes since:
- the amount of code saved is not significant.
- we'd need to catch any ActiveRecord exceptions and raise equivalent
ClickHouse
exceptions, otherwise the exception class might mislead users. - we would be exposed to potential bugs due to subtle changes in the base ActiveRecord classes.
-
second commit: Rake targets (
rake gitlab:clickhouse:migrate
andrake gitlab:clickhouse:rollback
, including support forVERSION
,VERBOSE
, andSCOPE
environment variables). -
third commit: moves the existing
db/click_house/main/*.sql
files todb/click_house/migrate/*.rb
files.
Closes #428124 (closed)
Screenshots or screen recordings
Screenshots are required for UI changes, and strongly recommended for all other merge requests.
Command | Before | After |
---|---|---|
rake gitlab:clickhouse:migrate |
||
rake gitlab:clickhouse:rollback VERSION=20230808140217 |
How to set up and validate locally
Numbered steps to set up and validate the change are strongly suggested.
-
Setup a development ClickHouse server, see https://docs.gitlab.com/ee/development/database/clickhouse/clickhouse_within_gitlab.html
-
Test several migration commands, such as:
rake gitlab:clickhouse:migrate
rake gitlab:clickhouse:rollback
You can also pass specific
VERSION
values.At any point, you can run
SELECT * FROM schema_migrations FINAL
on the ClickHouse client (clickhouse client -d gitlab_clickhouse_development
) to see the internal state of the migrations.
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.