Skip to content

Add migration to reindex wiki documents

Ravi Kumar requested to merge 410777-migration-reindex-project-wiki-to-fix-permissions into master

What does this MR do and why?

Due to a bug, we have the wrong values of visibility_level, wiki_access_level, and traversal_ids for the wiki blobs in Elastic. The bug fix has already been deployed to production. Since there is no way to determine which document has the wrong values, this MR will reindex all the documents to fix the wrong values. We have already bumped the schema_version with the previous fix. This means all the documents which have older schema_version need to be reindexed. With each reindex, the schema_version will get updated. So, we will reindex until all the documents have new schema_version

How to set up and validate locally

Make sure Elasticsearch is enabled

  1. Open the Rails console
bundle exec rails c
  1. Run the following command
require_relative 'ee/elastic/migrate/202305195000123_reindex_wikis_to_fix_permissions_and_traversal_ids.rb' 
ReindexWikisToFixPermissionsAndTraversalIds.new(202305195000123).migrate
  1. Check project wikis on ES
curl -H 'Content-type: application/json' -XGET 'http://localhost:9200/gitlab-development-wikis/_search'

The schema_version must be 2305

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Approximate time to completetion

~ 19 hours

[1] pry(main)> number_of_projects = 46129
=> 46129
[2] pry(main)> throttle_delay = 5.minute
=> 5 minutes
[3] pry(main)> batch_size = 500
=> 200
[4] pry(main)> ((number_of_projects / batch_size) * throttle_delay) / 1.hour
=> 19

Related to #410777

Edited by Ravi Kumar

Merge request reports