Skip to content

Makefile: Apply Git patches to speed up fetches

Patrick Steinhardt requested to merge pks-git-fetch-speedups into master

For quite some time we're aware of the fact that mirror-fetches into repositories with many refs are exceedingly slow. Most importantly, this issue poses problems for our replication strategy where replication jobs take so much time that replication targets are likely to be out of date immediately after they have received a replication jobs because the primary node has received additional mutators while the replication target was fetching changes.

To address this problem, we have upstreamed a patch series into git.git which speeds fetches up somewhat. Most importantly, this patch series optimizes the way git-fetch(1) enumerates refs by making better use of the commit-graph. The result is that mirror-fetches in the benchmarking repository gitlab-org/gitlab have been sped up from originally 56s to 25s.

While it is unlikely that this speedup alone will fix our replication issue, it is definitely an important step towards improving the situation.

Closes git#95 (closed)

Merge request reports

Loading