Skip to content

Increase VSA upsert count

Adam Hegyi requested to merge 451275-increase-batch-sizes-for-vsa-aggregation into master

What does this MR do and why?

This MR increases the total number of records we upsert (during the worker run, not in one batch) for the VSA workers. The goal is to speed up the full re-aggregation process. Currently, it seems like we don't utilize the full runtime (250s), we reach the max upsert limit earlier than the allowed runtime:

gitlab-org:
full_runtimes_in_seconds | {17,71,60,65,76,49,69,77,90,71}
gitlab-com:
full_runtimes_in_seconds | {56,54,63,59,29,42,32,83,79,51}

By increasing the maximum number of records to process per worker invocation should speed things up considerably.

MR acceptance checklist

Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Related to #451275 (closed)

Merge request reports