Skip to content

Add urgency and duration fields to Sidekiq logs

Gregorius Marco requested to merge mg-add-urgency-duration-to-sidekiq-logs into master

What does this MR do and why?

Part of gitlab-com/gl-infra/scalability#1646 (closed), this MR adds urgency and target_duration_s to Sidekiq logs.

How to set up and validate locally

  1. Run gdk
  2. Enter EmailReceiverWorker.perform_async "foo" in rails console
  3. Observe log/sidekiq.log, it should contain the new fields like: (scroll all the way to the right)
    {"severity":"INFO","time":"2023-04-12T10:44:38.328Z","retry":3,"queue":"default","backtrace":true,"version":0,"args":["[FILTERED]"],"class":"EmailReceiverWorker","jid":"a2f9adfcd7bf9c80c29bdd30","created_at":"2023-04-12T10:44:38.325Z","meta.feature_category":"team_planning","correlation_id":"9346ec6de13f7b8a521ab8e2ed582f11","worker_data_consistency":"always","idempotency_key":"resque:gitlab:duplicate:default:78ed7017e7e8f88d00049e2fb939deb575bf051ae396cf6460c1e25a32166c5c","size_limiter":"validated","enqueued_at":"2023-04-12T10:44:38.327Z","job_size_bytes":7,"pid":37082,"message":"EmailReceiverWorker JID-a2f9adfcd7bf9c80c29bdd30: start","job_status":"start","scheduling_latency_s":0.000772}
    {"severity":"INFO","time":"2023-04-12T10:44:38.400Z","retry":3,"queue":"default","backtrace":true,"version":0,"args":["[FILTERED]"],"class":"EmailReceiverWorker","jid":"a2f9adfcd7bf9c80c29bdd30","created_at":"2023-04-12T10:44:38.325Z","meta.feature_category":"team_planning","correlation_id":"9346ec6de13f7b8a521ab8e2ed582f11","worker_data_consistency":"always","idempotency_key":"resque:gitlab:duplicate:default:78ed7017e7e8f88d00049e2fb939deb575bf051ae396cf6460c1e25a32166c5c","size_limiter":"validated","enqueued_at":"2023-04-12T10:44:38.327Z","job_size_bytes":7,"pid":37082,"message":"EmailReceiverWorker JID-a2f9adfcd7bf9c80c29bdd30: done: 0.072558 sec","job_status":"done","scheduling_latency_s":0.000772,"redis_calls":2,"redis_duration_s":0.000305,"redis_read_bytes":2,"redis_write_bytes":186,"redis_queues_calls":2,"redis_queues_duration_s":0.000305,"redis_queues_read_bytes":2,"redis_queues_write_bytes":186,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_replica_count":0,"db_primary_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.065316,"worker_id":"sidekiq_0","rate_limiting_gates":[],"duration_s":0.072558,"completed_at":"2023-04-12T10:44:38.400Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"high","target_duration_s":10}

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Gregorius Marco

Merge request reports

Loading