Add Sidekiq RoutingRules configuration
What does this MR do?
In gitlab-org/gitlab!59604 (merged) is merged, we introduced Sidekiq's routing_queues
configuration that allow use re-route a sidekiq job into a particular queue based on a set of routing rules tuples. This is an example for routing rules configuration:
sidekiq:
routingRules:
- ["resource_boundary=cpu", "cpu_boundary"]
- ["feature_category=pages", null]
- ["*", "default"]
The new configuration is added into Gitlab global configuration, as it affect the Sidekiq clients at the schedule time.
Link to equivalent omnibus-gitlab change: gitlab-org/omnibus-gitlab!5202 (merged)
Testing
This MR affects the gitlab.yml
configuration, which then leads to the restructuring of job queues. I design a small test in my local environment with minikube.
Scenario 1
After applying this chart using helm
with default values, the routing rules are not configured. As a result, all the jobs are sent to the queues generated from the worker names. For examples, in this case, PagesWorker
is sent to pages
queue.
Scenario 2
In gitlab-com/gl-infra/scalability#1018 (closed), we introduced worker_name
predicate allowing us to re-route a particular worker.
- I updated the value file as following and upgraded with
helm upgrade
:
- Any new jobs related to PagesWorker are not routed to
hello
queue, as expected:
Scenario 3
Let's try a more complicated configuration:
I tested some workers representative for each shard. THe result looks like this:
- PagesWorker jobs are still routed to
hello
- ProjectExportWorker (worker_boundary=memory) is routed to
memory-bound
- ExpirePipelineCacheWorker (worker_boundary=cpu) matches the
nil
queue, equivalent to queue generated from worker name.
- Other jobs from random workers are routed to
default
- When looking at
/srv/gitlab/config/gitlab.yml
, therouting_rules
configuration is set in the Sidekiq section:
Related issues
Checklist
See Definition of done.
For anything in this list which will not be completed, please provide a reason in the MR discussion.
Required
-
Merge Request Title and Description are up to date, accurate, and descriptive -
MR targeting the appropriate branch -
MR has a green pipeline on GitLab.com
Expected (please provide an explanation if not completing)
-
Test plan indicating conditions for success has been posted and passes -
Documentation created/updated -
Tests added -
Integration tests added to GitLab QA -
Equivalent MR/issue for omnibus-gitlab opened