Adapt zero-time reindexing to ILM
What does this MR do and why?
This MR is part of Advanced Search: Index lifecycle management [Phase 1]
This MR adapts Elastic::ClusterReindexingService
to using more than one index per alias (ILM). The new approach is to have 1-1 reindexing. If you have 2 indices, zero-downtime reindexing will also create 2 indices. This is done to avoid situations when we have 1 huge index after triggering reindexing.
Screenshots or screen recordings
Screenshots are required for UI changes, and strongly recommended for all other merge requests.
How to set up and validate locally
Numbered steps to set up and validate the change are strongly suggested.
Example below:
- Enable Advanced Search
- Split your index into 2 or more. You can use this script to split issues index https://gitlab.com/-/snippets/2428478
- Create a new reindexing task with only one class. For example
Elastic::ReindexingTask.create!(targets: %w[Issue])
- Trigger
ElasticClusterReindexingCronWorker.new.perform
repeatedly - Check out the reindexing progress via
Elastic::ReindexingTask.current
or/admin/application_settings/advanced_search
- Ensure that you have a new
gitlab-development-issues
indices viacurl http://localhost:9200/_cat/indices
- After reindexing is completed you should notice that
Elastic::ReindexingTask.last.state
has changed tosuccess
- Ensure that
curl http://localhost:9200/_cat/aliases/gitlab-development-issues
points to newly created indices and search still works
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.
Related to #375752 (closed)
Edited by Dmitry Gruzd