Delete stale npm metadata caches with background migration

Context

In Use LFK to mark npm metadata caches for destruc... (!165804 - merged) we've introduced the new status pending_destruction for Packages::Npm::MetadataCache. It's affected the Packages::Npm::CleanupStaleMetadataCacheWorker that's working with .pending_destruction scope. Before the changes the worker was searching and deleting the entries with project_id = NULL, but now the condition has changed to status = 2 (pending_destruction).

Given above, we got a bunch of entries in the database with project_id = NULL that won't be cleaned with the background worker anymore.

What does this MR do and why?

This MR adds the batched background migration to delete the entries with project_id = NULL from the packages_npm_metadata_caches table.

MR acceptance checklist

Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Screenshots or screen recordings

No.

How to set up and validate locally

Let's prepare a new orphaned metadata cache entry and trigger the background migration to delete it.

1. Create npm metadata cache entry

   # stub file upload
   def fixture_file_upload(*args, **kwargs)
     Rack::Test::UploadedFile.new(*args, **kwargs)
   end
   
   c = FactoryBot.create(:npm_metadata_cache) # note down the `id`
   c.update_attribute(:project_id, nil)

2. Run the migration to add the cleanup job to the queue

   $ rails db:migrate:up:main VERSION=20240911115256

3. Give sidekiq a bit time to process the job and verify that the npm metadata cache was deleted

   Packages::Npm::MetadataCache.find(c.id) # => ActiveRecord::RecordNotFound

   To verify the background migration status visit http://gdk.test:3000/admin/background_migrations

Related to #461787 (closed)