Skip to content

Filter for oversized blobs using blobs sitcher

Gavin Hinfey requested to merge 438245-refactor-find-oversized-blobs into master

What does this MR do and why?

Previously we used GitalyClient::BlobsStitcher to return all blobs. We then checked if these blobs were over the filesize limit. Those operations were expensive.
In this MR we pass a filter to BlobStitcher and only return blobs over the specified size.

See #482936 (comment 2158778314) for performance comparison when pushing the Linux repo.

MR acceptance checklist

Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

How to set up and validate locally

  1. Enable the Feature Flag check_oversized_blobs_without_blob_stitcher.
  2. Clone the Linux kernel: https://github.com/torvalds/linux
  3. Create a new project in your GDK and push the Linux kernel master branch to it.
  4. Notice that the push is successful.
  5. Check the filesize check is still working by commiting a file over 100mb to the linux kernel.
  6. Create a new project in the GDK and push the Linux kernel master branch to it.
  7. Notice that the push is rejected because a file is over the size limit.

Related to #438245

Edited by Gavin Hinfey

Merge request reports