Implement three-phase commit for WriteRef RPC (proof of concept)
Problem to solve
The WriteRef RPC is used by merge-refs refs/merge-requests/:iid/merge, for GitLab CI, and we are in the process of making this the default for merge request diffs. This means tests are run against the merge result, and the diff is what the resultant merge will be, but this results in lots of calls to the WriteRef RPC which triggers a mirror from the primary Gitaly node to the replicas. Mirroring is expensive relative to the small operation WriteRef RPC. It is also a relatively simple RPC to experiment with transactions.
Further details
Consistency is an objective of Gitaly HA, but something we avoided in the first iteration because it seems hard to implement comprehensively. But, we need to start developing a strategy and better understand the complexity of implementing strong consistency.
In theory, if we have okay eventual consistency, we can start optimizing the most common mutators to use transactions. For operations that have been implemented with transaction support, we don't need to trigger replication for those RPCs. This might provide a nice pathway to move from eventual consistency, to strong consistency.
Proposal
Implement a proof of concept for three-phase commit for WriteRef RPC.