Skip to content

Serialize delete refs for Merge Requests to avoid Gitaly deadlock

What does this MR do and why?

Serialize delete refs for Merge Requests to avoid Gitaly deadlock.

This is a fix similar to !123976 (merged). It changes both to use the same serialization mechanism to reduce the likelity of Gitaly deadlock as described in gitaly#5368 (closed).

This MR does:

  • Build on top of work in !123976 (merged).
  • Serializes all references managed by merge-requests: refs/merge-requests/IID/head, refs/merge-requests/IID/merge, refs/merge-requests/IID/train.
  • They are scheduled and run async, synchronized with deleted being done by CI Pipelines.
  • This should cover 99.99% of cases when deletes does happen in code.

Remarks:

  • Those deletes still happen one-by-one, so they do generate extra pressure on Gitaly.
  • This does show case all places where deletes do happen and generalises their handling making it easier to implement: gitaly#5369 (closed).
  • This has its own feature flag: merge_request_cleanup_ref_worker_async. To avoid deadlock the merge_request_cleanup_ref_worker_async and pipeline_cleanup_ref_worker_async should be enabled.

How to set up and validate locally

  1. In rails console enable the experiment fully
    Feature.enable(: merge_request_cleanup_ref_worker_async)

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Grzegorz Bizon

Merge request reports

Loading