Skip to content

Migration to fix wrong values of label_ids in issue index

Ravi Kumar requested to merge 416091-migration-fix-label_ids-in-issues into master

What does this MR do and why?

When we removed the labels from issues, there was a bug not updating the label_ids in the issues index. This MR is to reindex all the issues which don't have new SCHEMA_VERSION = 23_09. The schema_version with 2309 is set after the underlying bug gets fixed

Screenshots or screen recordings

Screenshots are required for UI changes, and strongly recommended for all other merge requests.

Before After

How to set up and validate locally

  • Ensure elasticsearch is enabled
  • For a fresh test, create a new project. Create an issue in this project.
  • Update the schema_version and label_ids for this issue in the elastic by running the following command. Replace the project_id with the one you have just created.
curl -XPOST "http://localhost:9200/gitlab-development-issues/_update_by_query" -H "kbn-xsrf: reporting" -H "Content-Type: application/json" -d'
{
  "query": {
    "bool": {
      "filter": [
        {
          "term": {
            "project_id": project_id
          }
        },
        {
          "term": {
            "type": "issue"
          }
        }
      ]
    }
  },
  "script": {
    "source": "ctx._source.schema_version = 2308;ctx._source.label_ids=[0]"
  }
}' | json_pp
  • Ensure on Elastic that the issue has been updated successfully by running the following command. Verify that the schema_version is 2308 and label_ids is [0]. Replace the project_id with the one you just created.
curl -XGET "http://localhost:9200/gitlab-development-issues/_search" -H "kbn-xsrf: reporting" -H "Content-Type: application/json" -d'
{
  "query": {
    "bool": {
      "filter": [
        {
          "term": {
            "project_id": project_id
          }
        },
        {
          "term": {
            "type": "issue"
          }
        }
      ]
    }
  }
}' | json_pp
  • Open the rails console
bundle exec rails c
  • Run the following command
Elastic::DataMigrationService[20230908120634].send(:migration).migrate
  • And then again run the above XGET curl command and ensure the schema_version is 2309 and the label_ids is an empty array. It might take a few seconds to reflect the changes. If it takes longer than 1 minute then restart the rails-background-jobs and check again.
gdk restart rails-background-jobs

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Related to #416091 (closed)

#424128 (closed)

Runtime

~35 hours

[1] pry(main)> documents_count = 18769666
=> 18769666
[2] pry(main)> throttle_delay = 1.minute
=> 1 minute
[3] pry(main)> batch_size = 9_000
=> 9000
[4] pry(main)> ((documents_count / batch_size) * throttle_delay) / 1.hour
=> 34
Edited by Ravi Kumar

Merge request reports

Loading