"Deleted projects" does not get deleted
Summary
I have deleted several projects and they have all been tagged with pending removal. After waiting for a week the projects still has not been deleted.
Steps to reproduce
- Create a project on gitlab.com
- Add something to it (or not)
- Delete project
- Wait for project to be deleted
Example Project
- https://gitlab.com/jimmy.ljungberg/nexus - Supposed to be deleted on 2020-03-27
- https://gitlab.com/jimmy.ljungberg/yatsy-server-old - Empty project, lacking warning message when it will be deleted
- https://gitlab.com/jimmy.ljungberg/sysdev - Supposed to be deleted on 2020-03-27
- https://gitlab.com/jimmy.ljungberg/yatsy-klient - Supposed to be deleted on 2020-03-23
- https://gitlab.com/jimmy.ljungberg/yatsy - Empty project, lacking warning message when it will be deleted
What is the current bug behavior?
Projects don't get deleted
What is the expected correct behavior?
Project should no longer exist
Output of checks
This bug happens on GitLab.com
Theoretical Issue
The cron worker that finds projects scheduled for delection adds an artifical spacing between jobs. This increases exponentially with each project scheduled for deletion.
The table below shows how long it would take the last project picked up by the cron job to execute.
| Last deletion starts after | |
|---|---|
| 5 projects | 25 minutes |
| 100 projects | 8 hours |
| 1000 projects | 3 days |
Proposal
Remove this artifical spacing between jobs and allow sidekiq to schedule the jobs instead.
Edited by Dan Jensen