Refactor Sidekiq .perform_bulk usage to use .bulk_perform_async
What does this MR do and why?
This MR replaces perform_bulk
with bulk_perform_async
which is our wrapper over the push_bulk
method. bulk_perform_async
is shard-aware compared to perform_bulk
.
Both bulk_perform_async
and perform_bulk
calls push_bulk
This is a follow-up to !145495 (comment 1804849962)
MR acceptance checklist
Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.
Screenshots or screen recordings
Screenshots are required for UI changes, and strongly recommended for all other merge requests.
Before | After |
---|---|
How to set up and validate locally
The 2 methods are interchangeable
-
Stop sidekiq using
gdk stop rails-background-jobs
. -
On a gdk rails console, run both commands
Chaos::SleepWorker.bulk_perform_async([[1], [2]])
Chaos::SleepWorker.perform_bulk([[1], [2]])
- Verify in the redis that 4 jobs are in the queue.
redis /Users/sylvesterchin/work/gitlab-development-kit/redis/redis.socket[1]> lrange queue:default 0 -1
1) "{\"retry\":3,\"queue\":\"default\",\"backtrace\":true,\"version\":0,\"store\":null,\"queue_namespace\":\"chaos\",\"class\":\"Chaos::SleepWorker\",\"args\":[2],\"jid\":\"962b7e9a593d4ed0944f25f5\",\"created_at\":1709862000.920937,\"meta.sidekiq_destination_shard_redis\":\"main\",\"correlation_id\":\"60625f9d925f673bc42df52cd0c666ed\",\"worker_data_consistency\":\"always\",\"idempotency_key\":\"resque:gitlab:duplicate:default:24808edb6838c48efcad29c8e4b7b5b1c8243aa7a545bb43a333e8284b18e49e\",\"duplicate-of\":\"ee33762f33033a2e8d690926\",\"size_limiter\":\"validated\",\"enqueued_at\":1709862000.933753}"
2) "{\"retry\":3,\"queue\":\"default\",\"backtrace\":true,\"version\":0,\"store\":null,\"queue_namespace\":\"chaos\",\"class\":\"Chaos::SleepWorker\",\"args\":[1],\"jid\":\"733ca1ec589ebf94d459b233\",\"created_at\":1709862000.920937,\"meta.sidekiq_destination_shard_redis\":\"main\",\"correlation_id\":\"60625f9d925f673bc42df52cd0c666ed\",\"worker_data_consistency\":\"always\",\"idempotency_key\":\"resque:gitlab:duplicate:default:2f0bc834677c58fe9b274ea1acbbcd384d61ad88243f7ae8b6c38429900bba4f\",\"duplicate-of\":\"b3e81a65e845c586de6de6d4\",\"size_limiter\":\"validated\",\"enqueued_at\":1709862000.933753}"
3) "{\"retry\":3,\"queue\":\"default\",\"backtrace\":true,\"version\":0,\"store\":null,\"queue_namespace\":\"chaos\",\"class\":\"Chaos::SleepWorker\",\"args\":[2],\"jid\":\"ee33762f33033a2e8d690926\",\"created_at\":1709861987.61459,\"correlation_id\":\"144544240ef63d68434cfedd0bb72fb3\",\"worker_data_consistency\":\"always\",\"idempotency_key\":\"resque:gitlab:duplicate:default:24808edb6838c48efcad29c8e4b7b5b1c8243aa7a545bb43a333e8284b18e49e\",\"size_limiter\":\"validated\",\"enqueued_at\":1709861987.6437812}"
4) "{\"retry\":3,\"queue\":\"default\",\"backtrace\":true,\"version\":0,\"store\":null,\"queue_namespace\":\"chaos\",\"class\":\"Chaos::SleepWorker\",\"args\":[1],\"jid\":\"b3e81a65e845c586de6de6d4\",\"created_at\":1709861987.61459,\"correlation_id\":\"c734976ec823003a8b36f90bd618592c\",\"worker_data_consistency\":\"always\",\"idempotency_key\":\"resque:gitlab:duplicate:default:2f0bc834677c58fe9b274ea1acbbcd384d61ad88243f7ae8b6c38429900bba4f\",\"size_limiter\":\"validated\",\"enqueued_at\":1709861987.6437812}"
Edited by Sylvester Chin