Delete stale npm metadata caches with background migration
Context
In Use LFK to mark npm metadata caches for destruc... (!165804 - merged) we've introduced the new status pending_destruction
for Packages::Npm::MetadataCache
. It's affected the Packages::Npm::CleanupStaleMetadataCacheWorker
that's working with .pending_destruction
scope. Before the changes the worker was searching and deleting the entries with project_id = NULL
, but now the condition has changed to status = 2
(pending_destruction).
Given above, we got a bunch of entries in the database with project_id = NULL
that won't be cleaned with the background worker anymore.
What does this MR do and why?
This MR adds the batched background migration to delete the entries with project_id = NULL
from the packages_npm_metadata_caches
table.
MR acceptance checklist
Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.
Screenshots or screen recordings
No.
How to set up and validate locally
Let's prepare a new orphaned metadata cache entry and trigger the background migration to delete it.
-
Create npm metadata cache entry
# stub file upload def fixture_file_upload(*args, **kwargs) Rack::Test::UploadedFile.new(*args, **kwargs) end c = FactoryBot.create(:npm_metadata_cache) # note down the `id` c.update_attribute(:project_id, nil)
-
Run the migration to add the cleanup job to the queue
$ rails db:migrate:up:main VERSION=20240911115256
-
Give sidekiq a bit time to process the job and verify that the npm metadata cache was deleted
Packages::Npm::MetadataCache.find(c.id) # => ActiveRecord::RecordNotFound
To verify the background migration status visit http://gdk.test:3000/admin/background_migrations
Related to #461787 (closed)