Add fix for missing file to index integrity
Problem to solve
A bug was found that means some projects will have documents in the index which do not exist in the repository anymore.
We need a way to fix that when it's detected without doing a complicated migration. The migration would be too difficult to write to support self-managed customers.
Proposal
Index integrity could handle it, we discussed this in the team meeting and here were the ideas discussed:
- Create a new worker to reindex a project's code related data, use the script from gitlab-com/gl-infra/production#17060 (comment 1628964730) as a starting point
- kick off this new worker when a blob (code) search result is clicked and returns a 404 (file is not found in repo), this should only be done if the project is indexed (and maintaining associations, we are working on indexing all projects so keep an eye on that work)
- do not want to run this every time the index integrity worker runs
We might consider opening a follow up issue to change the index integrity worker to use this new worker
Edited by Terri Chu