Skip to content

Change batch and cadence for dot_com_rollout task

Terri Chu requested to merge tchu-adjust-zoekt-dot_com_rollout-task-cadence into master

What does this MR do and why?

slack thread (internal)

This MR makes two changes for the dot_com_rollout task in the Search::Zoekt::SchedulingService :

  • increase batch size for namespaces processed
  • remove the 2 hour limit, the task will run every minute (when the worker runs)

The why

The limitations were added when Zoekt was initially being rolled out to prevent too many namespaces from having search enabled at once. We are past the initial rollout point and working on 100% rollout and can safely roll out more namespaces.

Additionally, the Zoekt architecture includes namespace reallocation when a node runs out of PVC space: namespaces are moved from one node to another. The namespaces need to be re-assigned to other nodes (node_assignemt task does this) and have search re-enabled as quickly as possible.

We currently have a backlog of reallocated namespaces that are waiting to have search set to true by this task. This should help clear those out and keep them from building up when reallocation happens in the future.

MR acceptance checklist

Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Screenshots or screen recordings

N/A

How to set up and validate locally

N/A

Edited by Terri Chu

Merge request reports