Skip to content

Make the AuditEventStreamingWorker backward compatible temporarily

Huzaifa Iftikhar requested to merge 361931_tmp_make_streaming_worker_compatible_for_old_jobs into master

What does this MR do and why?

  1. We created the MR !86881 (merged) but missed following the correct process to add a new argument to the sidekiq worker.
  2. This caused inconsistencies such as:
    1. A job scheduled with new arguments but was picked by the old worker code. This happened during the canary deployment, however, these jobs are eventually successful once a worker with the new code handles the job.
    2. A job is scheduled with old arguments but is picked by the new worker code. This causes the job to fail with ActiveRecord::RecordNotFound exceptions because of inconsistencies in the actual argument passed when the job was created and the arguments that the new worker code expects.
    3. More details can be found in the issue description.
  3. Changelog is not added because "A fix for a regression introduced and then fixed in the same release (such as fixing a bug introduced during a monthly release candidate) should not have a changelog entry."

Screenshots or screen recordings

These are strongly recommended to assist reviewers and reduce the time to merge your change.

How to set up and validate locally

  1. You should have an ultimate license.
  2. We need to use rails console to manually schedule jobs that are identical to the ones that were created before we merged !86881 (merged).
  3. Create a root group or use an existing one and add an audit event streaming destination for that root group.
  4. Manually create an audit event json. For example audit_event_json = AuditEvent.where(entity_id: <root_group_id>).first.to_json
  5. Use the above json to stream an event, AuditEvents::AuditEventStreamingWorker.perform_async(nil, audit_event_json). We can also use AuditEvents::AuditEventStreamingWorker.new.perform(nil, audit_event_json).
  6. Observe in logs that ActiveRecord::RecordNotFound is raised once and then the job is re-enqueued with correct arguments and finally gets streamed successfully.

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Relates to https://gitlab.com/gitlab-org/gitlab/-/issues/361931

Edited by Huzaifa Iftikhar

Merge request reports