Allow setting the shard/replica separately for ES indexes (step 1)
What does this MR do?
This MR introduces a new table for storing shards and replicas count for each index and migrates existing shard/replica settings from application_settings
to this table. We also adapt current advanced settings page to using this new table.
table schema
gitlabhq_development=# \d elastic_index_settings
Table "public.elastic_index_settings"
Column | Type | Collation | Nullable | Default
--------------------+--------------------------+-----------+----------+----------------------------------------------------
id | bigint | | not null | nextval('elastic_index_settings_id_seq'::regclass)
alias_name | text | | not null |
number_of_replicas | smallint | | not null | 1
number_of_shards | smallint | | not null | 5
created_at | timestamp with time zone | | not null |
updated_at | timestamp with time zone | | not null |
Indexes:
"elastic_index_settings_pkey" PRIMARY KEY, btree (id)
"index_elastic_index_settings_on_alias_name" UNIQUE, btree (alias_name)
Check constraints:
"check_c30005c325" CHECK (char_length(alias_name) <= 255)
Rollout plan
- Step 1: (we are here) Create a new table and migrate existing index settings and change the backend to allow using this new table instead of application_settings
- Step 2: Change the backend and the UI to allow changing settings for all existing indices. Ignoring no longer used fields (elasticsearch_replicas, elasticsearch_shards), in the
application_settings
table.
Migrations
up
$ VERSION=20210317100520 rake db:migrate:up
== 20210317100520 CreateElasticIndexSettings: migrating =======================
-- create_table(:elastic_index_settings, {})
-- quote_column_name(:alias_name)
-> 0.0000s
-> 0.0168s
-- quote_table_name("check_c30005c325")
-> 0.0000s
-- quote_table_name(:elastic_index_settings)
-> 0.0000s
-- execute("ALTER TABLE \"elastic_index_settings\"\nADD CONSTRAINT \"check_c30005c325\" CHECK (char_length(\"alias_name\") <= 255)\n")
-> 0.0013s
== 20210317100520 CreateElasticIndexSettings: migrated (0.0238s) ==============
$ VERSION=20210324131727 rake db:migrate:up
== 20210324131727 MigrateElasticIndexSettings: migrating ======================
== 20210324131727 MigrateElasticIndexSettings: migrated (0.1913s) =============
down
$ VERSION=20210324131727 rake db:migrate:down
== 20210324131727 MigrateElasticIndexSettings: reverting ======================
== 20210324131727 MigrateElasticIndexSettings: reverted (0.0036s) =============
$ VERSION=20210317100520 rake db:migrate:down
== 20210317100520 CreateElasticIndexSettings: reverting =======================
-- drop_table(:elastic_index_settings)
-> 0.0075s
== 20210317100520 CreateElasticIndexSettings: reverted (0.0076s) ==============
Screenshots (strongly suggested)
Does this MR meet the acceptance criteria?
Conformity
-
📋 Does this MR need a changelog?-
I have included a changelog entry. - [-] I have not included a changelog entry because _____.
-
- [-] Documentation (if required)
-
Code review guidelines -
Merge request performance guidelines -
Style guides -
Database guides -
Separation of EE specific content
Availability and Testing
-
Review and add/update tests for this feature/bug. Consider all test levels. See the Test Planning Process. - [-] Tested in all supported browsers
- [-] Informed Infrastructure department of a default or new setting change, if applicable per definition of done
Security
If this MR contains changes to processing or storing of credentials or tokens, authorization and authentication methods and other items described in the security review guidelines:
- [-] Label as security and @ mention
@gitlab-com/gl-security/appsec
- [-] The MR includes necessary changes to maintain consistency between UI, API, email, or other methods
- [-] Security reports checked/validated by a reviewer from the AppSec team
Related to #296882 (closed)
Edited by Dmitry Gruzd