Skip to content

Change AdjustIndicesReservedStorageBytesEventWorker to process 1 batch

Terri Chu requested to merge 511071-move-event-worker-to-process-one-batch into master

What does this MR do and why?

This MR moves adjust_indices_reserved_storage_bytes task in the Search::Zoekt::SchedulingService to process one batch at a time:

  1. emit an event with no index ids
  2. event worker reads in 1 batch of 1000 indexes
  3. call find_each instead of each in worker

References

Please include cross links to any resources that are relevant to this MR. This will give reviewers and future readers helpful context to give an efficient review of the changes introduced.

MR acceptance checklist

Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Screenshots or screen recordings

N/A

How to set up and validate locally

This is covered pretty well by testing. If you wanted to validate it manually the steps are:

  1. enable zoekt for gdk
  2. setup some indices manually with either: watermark_levels = overprovisioned and state = ready or watermark_levels = high_watermark_exceeded
  3. tail the background jobs log: gdk tail rails-background-jobs
  4. run the scheduling service task: Search::Zoekt::SchedulingService.execute(:adjust_indices_reserved_storage_bytes)
  5. verify that the indices are updated
  6. verify you see the sidekiq log with the extra metadata
sidekiq done log
{"severity":"INFO","time":"2025-01-07T18:28:44.556Z","retry":1,"queue":"default","backtrace":true,"version":0,"store":null,"args":["Search::Zoekt::AdjustIndicesReservedStorageBytesEvent","{}"],"class":"Search::Zoekt::AdjustIndicesReservedStorageBytesEventWorker","jid":"a08d83a1d6a660c27bf95857","created_at":"2025-01-07T18:28:44.454Z","trace_propagation_headers":{"sentry-trace":"7c59c5a4effd420e91c4724633e01fd4-feace167fa134147","baggage":"sentry-trace_id=7c59c5a4effd420e91c4724633e01fd4,sentry-environment=,sentry-release=dba4265cc85"},"meta.feature_category":"global_search","correlation_id":"f9626663dc988ad3f572caa0fd635d50","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:65301fa02f6c642f99b130161b7958b797581b6c3f274c0fb4d6f4e80ef4c429","size_limiter":"validated","enqueued_at":"2025-01-07T18:28:44.472Z","job_size_bytes":60,"pid":73925,"sidekiq_tid":"15dt","sidekiq_thread_name":"sidekiq.default/processor","message":"Search::Zoekt::AdjustIndicesReservedStorageBytesEventWorker JID-a08d83a1d6a660c27bf95857: done: 0.083529 sec","job_status":"done","queue_duration_s":0.000633,"scheduling_latency_s":0.000633,"db_count":55,"db_write_count":10,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":55,"db_replica_write_count":0,"db_primary_write_count":10,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.051,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":55,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":10,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.051,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.031019,"worker_id":"sidekiq_0","rate_limiting_gates":[],"extra.search_zoekt_adjust_indices_reserved_storage_bytes_event_worker.indices_updated_count":2,"duration_s":0.083529,"completed_at":"2025-01-07T18:28:44.556Z","load_balancing_strategy":"primary","db_duration_s":0.004321,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}
Edited by Terri Chu

Merge request reports

Loading