The source project of this merge request has been removed.
Implement progressive elasticsearch indexing for project mirrors
What does this MR do?
Currently, every time a project imports commits from another repo via the pull mirror
functionality, the elasticsearch repository index is completely regenerated. This is quite expensive - a complete reindex of the v4.4 Linux kernel, for instance, takes 40 (ruby) or 10 (experimental) minutes, depending on which indexer you use.
We store the last indexed commit in the database so it's simple to use this to make the indexing progressive, as it is for git push
operations.
Are there points in the code the reviewer needs to double check?
Why was this MR needed?
Screenshots (if relevant)
Does this MR meet the acceptance criteria?
-
Changelog entry added, if necessary -
Documentation created/updated -
API support added - Tests
-
Added for this feature/bug -
All builds are passing
-
-
Conform by the merge request performance guides -
Conform by the style guides -
Branch has no merge conflicts with master
(if it does - rebase it please) -
Squashed related commits together
What are the relevant issue numbers?
Closes #2881 (closed)
Edited by Nick Thomas