Create feature flag for incremental repository backups
What does this MR do and why?
- Adds a feature flag named
incremental_repository_backup
. - When enabled this will provide the minimum required arguments to
gitaly-backup
to enable incremental repository backups. - The approach may change as we learn how exactly the incremental backup workflow will work.
How to set up and validate locally
- Enable the feature
Feature.enable(:incremental_repository_backup)
- Take a normal backup. Note that we cannot pack into a tar file since incremental backups need access to the previous increment and it doesn't know about tar files.
$ bundle exec rake gitlab:backup:create SKIP=tar
- Find a target project to inspect.
[1] pry(main)> Project.find(20).repository.relative_path => "@hashed/f5/ca/f5ca38f748a1d6eaf726b8a42fb575c3c71f1864a8143301782de13da2d9202b.git"
- Check that the pointer layout was indeed used. Strip off the
.git
from above and add it totmp/backups/repositories/
We expect to a file namedLATEST
. These are the "pointers".$ tree tmp/backups/repositories/@hashed/f5/ca/f5ca38f748a1d6eaf726b8a42fb575c3c71f1864a8143301782de13da2d9202b/ tmp/backups/repositories/@hashed/f5/ca/f5ca38f748a1d6eaf726b8a42fb575c3c71f1864a8143301782de13da2d9202b/ ├── 20220316012048 │ ├── 001.bundle │ ├── 001.refs │ └── LATEST └── LATEST 1 directory, 4 files
- Push changes to the target repo (anything except a branch delete).
- Take an incremental backup.
$ bundle exec rake gitlab:backup:create SKIP=tar INCREMENTAL=yes
- Check that a new increment was created. We expect to see files named
002
.$ tree tmp/backups/repositories/@hashed/f5/ca/f5ca38f748a1d6eaf726b8a42fb575c3c71f1864a8143301782de13da2d9202b/ tmp/backups/repositories/@hashed/f5/ca/f5ca38f748a1d6eaf726b8a42fb575c3c71f1864a8143301782de13da2d9202b/ ├── 20220316012048 │ ├── 001.bundle │ ├── 001.refs │ ├── 002.bundle │ ├── 002.refs │ └── LATEST └── LATEST 1 directory, 6 files
🎉 - Restore from said backup. Hopefully this wont destroy GDK
😄 $ bundle exec rake gitlab:backup:restore ... Restore task is done.
- Check that the target repository looks okay.
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.
Edited by James Fargher