Skip to content

Adapt zero-time reindexing to ILM

Dmitry Gruzd requested to merge 375752-adapt-zero-downtime-feature-to-ilm into master

What does this MR do and why?

This MR is part of Advanced Search: Index lifecycle management [Phase 1]

This MR adapts Elastic::ClusterReindexingService to using more than one index per alias (ILM). The new approach is to have 1-1 reindexing. If you have 2 indices, zero-downtime reindexing will also create 2 indices. This is done to avoid situations when we have 1 huge index after triggering reindexing.

Screenshots or screen recordings

Screenshots are required for UI changes, and strongly recommended for all other merge requests.

How to set up and validate locally

Numbered steps to set up and validate the change are strongly suggested.

Example below:

  1. Enable Advanced Search
  2. Split your index into 2 or more. You can use this script to split issues index https://gitlab.com/-/snippets/2428478
  3. Create a new reindexing task with only one class. For example
    Elastic::ReindexingTask.create!(targets: %w[Issue])
  4. Trigger ElasticClusterReindexingCronWorker.new.perform repeatedly
  5. Check out the reindexing progress via Elastic::ReindexingTask.current or /admin/application_settings/advanced_search
  6. Ensure that you have a new gitlab-development-issues indices via curl http://localhost:9200/_cat/indices
  7. After reindexing is completed you should notice that Elastic::ReindexingTask.last.state has changed to success
  8. Ensure that curl http://localhost:9200/_cat/aliases/gitlab-development-issues points to newly created indices and search still works

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Related to #375752 (closed)

Edited by Dmitry Gruzd

Merge request reports

Loading