Optimize index_should_be_marked_as_orphaned_check with a single batch
Background
index_should_be_marked_as_orphaned_check
runs on full records of zoekt_indices
. This can put too much stress on the database. The second issue could be that in a single run, all zoekt_indices might not get processed and they will get re-enqueued after 10 minutes which will increase redundancy.
Proposal
Publish the event in the scheduling_service
without any zoekt_indices_id. In the worker OrphanedIndexEventWorker
add order(:id)
and limit(1000)
on the scope should_be_marked_as_orphaned
to process just one batch.