Move indices_to_evict_check to process one batch

What does this MR do and why?

This MR moves indices_to_evict_check task in the Search::Zoekt::SchedulingService to process one batch at a time:

1. emit an event with no index ids
2. Search::Zoekt::IndexToEvictEventWorker worker reads in 1 batch of 1000 indexes
3. Search::Zoekt::NodeWithNegativeUnclaimedStorageEventWorker now updates indices watermark_level to critical_watermark_exceeded and emits an event with no index ids

References

Please include cross links to any resources that are relevant to this MR. This will give reviewers and future readers helpful context to give an efficient review of the changes introduced.

• Related to Optimize indices_to_evict_check with a single b... (#511074)

MR acceptance checklist

Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Screenshots or screen recordings

N/A

How to set up and validate locally

1. enable zoekt for gdk
2. setup some indices manually with: watermark_levels = critical_watermark_exceeded
3. tail the background jobs log: gdk tail rails-background-jobs
4. run the scheduling service task: Search::Zoekt::SchedulingService.execute(:indices_to_evict_check)
5. verify that the replicas are deleted for the indices
6. manually set nodes with negative unclaimed storage bytes
7. run the scheduling service task: Search::Zoekt::SchedulingService.execute(:node_with_negative_unclaimed_storage_bytes_check)
8. verify that the indices records are updated to watermark_levels = critical_watermark_exceeded
9. verify that the replicas are deleted when the emitted event is processed