Atomic repository creation
While our RPCs which create repositories all do mostly the same thing
except for how the repository is seeded, their behaviour is wildly
different when it comes to how preexisting repositories are handled.
E.g. CreateRepository()
is happy with the repository already existing,
CreateRepositoryFromBundle()
is happy with the target path to exist as
long as it is an empty directory, and the others disallow the target
path existing altogether. This behaviour is confusing, inconsistent and
has likely grown organically over time.
The issue with this divergent behaviour is that we cannot really
guarantee atomic transactional semantics for the former two RPC calls:
especially in CreateRepository()
, it is impossible to guarantee that
we either do no changes at all if the call will fail, or to do only
specific changes. This keeps us from implementing proper two-phase
voting in Gitaly Cluster given that we cannot roll back changes, and
neither can we meaningfully lock the repository for any additional
changes.
To fix this, we must assert that the target repository path does not exist at the time of creation and use proper locking at repository creation time. This ensures that we can assert an exact state of each repository on which we want to vote, it ensures that no other RPC calls modify the repository (it does not exist and cannot be created concurrently given it is locked), and it ensures that we can roll back changes in case an error happens by using a temporary repo into which all changes will first be written.
Implement a new helper function createRepository()
which handles all
this logic for us:
1. It asserts that the target path does not exist.
2. It creates a temporary repository.
3. This temporary repository is getting seeded by the caller via a
provided callback function.
4. We compute the vote, which is the complete contents of the
repository. This will guarantee that all nodes are about to
perform the same change.
5. The target repository path is locked now, where we assert after
the lock has been taken that no other concurrent RPC call has
created it meanwhile. This ensures that no two concurrent calls
can ever touch this repository and thus it cannot be modified by
anything else.
6. We vote on the state computed in step 4.
7. If the vote was successful, we know that other nodes did end up
with the same result. We thus rename the temporary directory into
place.
8. We do a finalizing vote to assert that we have performed the
change.
This mechanism is thus race-free and can be reused across the different RPCs. While it is a breaking change that will first require changes in Ruby, there really is no other way to provide atomicity in the context of Gitaly Cluster.
This MR fixes #3779 (closed), but is blocked by required upstream changes in Rails (gitlab#341009 (closed)). This MR is thus marked as draft.