Move projects to separate index
What does this MR do and why?
In this MR we move the projects to a separate elasticsearch index, to improve our performance.
Expected Run Time of migration: ~6 hours
Screenshots or screen recordings
Project Document Snippet on New Index
Screenshots are required for UI changes, and strongly recommended for all other merge requests.
Before | After |
---|---|
How to set up and validate locally
Numbered steps to set up and validate the change are strongly suggested.
TEST INDEXING
- Make sure the elasticsearch is enabled. Navigate to the elasticsearch settings and make sure the two checkboxes
Elasticsearch indexing
andSearch with Elasticsearch enabled
under theAdvanced Search
are enabled. - Complete the migration
migrate_projects_to_separate_index
bundle exec rake gitlab:elastic:index
- Create a new Project, and verify that it gets indexed properly
curl -H 'Content-type: application/json' -XGET 'http://localhost:9200/gitlab-development-projects/_search' | json_pp
- Update the project description from the UI and verify again that
description
gets updated in the elastic by running the above command. - Delete the project from the UI and verify that the documents get removed from the elastic by running the above command.
TEST SEARCHING
- Create a project with
name
new_project_index
anddescription
testing123
- Just for testing add the following line at the top in the method
def migration_has_finished?(name)
. It will simulate as the migrationmigrate_projects_to_separate_index
is not finished.
return false if name.eql?(:migrate_projects_to_separate_index)
- Now, create a project with a title
old_project_index
. Add thedescription
same as you have given
testing123` - Now, search
testing123
. And notice that in theProject
section you don't see the resultnew_project_index
- Now remove the line which you have added in the method
def migration_has_finished?(name)
- Again, search
testing123
. And notice that in theproject
section you don't see the resultold_project_index
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.
Related to #344644 (closed)
Edited by Siddharth Dungarwal