Skip to content

Migration to backfill archived on notes

Ravi Kumar requested to merge 414905-create-migration-backfill-archived-on-notes into master

What does this MR do and why?

This migration will backfill the archived value in the notes index.

Screenshots or screen recordings

Screenshots are required for UI changes, and strongly recommended for all other merge requests.

Before After

How to set up and validate locally

  1. Make sure the Elasticsearch is enabled.
  2. Open the rails console
bundle exec rails c
  1. Run the following commands in the console.
require_relative 'ee/elastic/migrate/20230722212041_backfill_archived_on_notes.rb' 
BackfillArchivedOnNotes.new(20230722212041).migrate
  1. Check on the elastic that all the notes must have the archived. To check that run the following curl command on the terminal and make sure the count is 0 
curl -XGET "http://localhost:9200/gitlab-development-notes/_count" -H "kbn-xsrf: reporting" -H "Content-Type: application/json" -d'
{
  "query": {
    "bool": {
      "must_not": {
        "exists": {
          "field": "archived"
        }
      }
    }
  }
}' | json_pp

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Run time

> batch_size = 9_000
=> 9000
> throttle_delay = 10.seconds
=> 1 minute
> number_of_documents = 135_280_365
=> 167616
> (number_of_documents / batch_size) * throttle_delay
=> 150310 seconds
=> ~ 42 hours

Related to #414905 (closed)

Edited by Ravi Kumar

Merge request reports