Migration to fix wrong values of label_ids in issue index
What does this MR do and why?
When we removed the labels from issues, there was a bug not updating the label_ids
in the issues
index. This MR is to reindex all the issues which don't have new SCHEMA_VERSION = 23_09
. The schema_version
with 2309
is set after the underlying bug gets fixed
Screenshots or screen recordings
Screenshots are required for UI changes, and strongly recommended for all other merge requests.
Before | After |
---|---|
How to set up and validate locally
- Ensure elasticsearch is enabled
- For a fresh test, create a new project. Create an issue in this project.
- Update the
schema_version
andlabel_ids
for this issue in the elastic by running the following command. Replace theproject_id
with the one you have just created.
curl -XPOST "http://localhost:9200/gitlab-development-issues/_update_by_query" -H "kbn-xsrf: reporting" -H "Content-Type: application/json" -d'
{
"query": {
"bool": {
"filter": [
{
"term": {
"project_id": project_id
}
},
{
"term": {
"type": "issue"
}
}
]
}
},
"script": {
"source": "ctx._source.schema_version = 2308;ctx._source.label_ids=[0]"
}
}' | json_pp
- Ensure on Elastic that the issue has been updated successfully by running the following command. Verify that the
schema_version
is2308
andlabel_ids
is[0]
. Replace theproject_id
with the one you just created.
curl -XGET "http://localhost:9200/gitlab-development-issues/_search" -H "kbn-xsrf: reporting" -H "Content-Type: application/json" -d'
{
"query": {
"bool": {
"filter": [
{
"term": {
"project_id": project_id
}
},
{
"term": {
"type": "issue"
}
}
]
}
}
}' | json_pp
- Open the rails console
bundle exec rails c
- Run the following command
Elastic::DataMigrationService[20230908120634].send(:migration).migrate
- And then again run the above XGET curl command and ensure the
schema_version
is2309
and thelabel_ids
is an empty array. It might take a few seconds to reflect the changes. If it takes longer than 1 minute then restart therails-background-jobs
and check again.
gdk restart rails-background-jobs
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.
Related to #416091 (closed)
Runtime
~35 hours
[1] pry(main)> documents_count = 18769666
=> 18769666
[2] pry(main)> throttle_delay = 1.minute
=> 1 minute
[3] pry(main)> batch_size = 9_000
=> 9000
[4] pry(main)> ((documents_count / batch_size) * throttle_delay) / 1.hour
=> 34
Edited by Ravi Kumar