Create a service to build npm metadata cache record
🌱 Context
In !114312 (merged) we introduced a new model to manage npm metadata caches,
that suppose to operate like a cache and to reduce the load and speed up the metadata endpoints.
Now it's time to provide a convenient way of creating/updating npm metadata cache entry, therefore this MR introduces a new service.
🚢 What does this MR do and why?
-
Creates the new service
Packages::Npm::CreateMetadataCacheService
to create a new entity of npm metadata cache or update the metadata of existing one. -
Additionally it introduces the new uploader
Packages::Npm::MetadataCacheUploader
to manage the metadata objects in the storage.
📷 Screenshots or screen recordings
No.
💾 Database migrations
⬆ Migration up
$ rails db:migrate:main
main: == [advisory_lock_connection] object_id: 274380, pg_backend_pid: 4527
main: == 20230403093349 EnsurePackagesNpmMetadataCachesIsEmpty: migrating ===========
main: -- quote_table_name("packages_npm_metadata_caches")
main: -> 0.0000s
main: -- execute("TRUNCATE TABLE \"packages_npm_metadata_caches\"")
main: -> 0.0067s
main: == 20230403093349 EnsurePackagesNpmMetadataCachesIsEmpty: migrated (0.0775s) ==
main: == 20230405071033 AddObjectStorageKeyToPackagesNpmMetadataCaches: migrating ===
main: -- column_exists?(:packages_npm_metadata_caches, :object_storage_key)
main: -> 0.0053s
main: -- add_column(:packages_npm_metadata_caches, :object_storage_key, :text, {:null=>false})
main: -> 0.0007s
main: -- transaction_open?()
main: -> 0.0000s
main: -- view_exists?(:postgres_partitions)
main: -> 0.0005s
main: -- index_exists?(:packages_npm_metadata_caches, :object_storage_key, {:unique=>true, :name=>"index_packages_npm_metadata_caches_on_object_storage_key", :algorithm=>:concurrently})
main: -> 0.0021s
main: -- execute("SET statement_timeout TO 0")
main: -> 0.0003s
main: -- add_index(:packages_npm_metadata_caches, :object_storage_key, {:unique=>true, :name=>"index_packages_npm_metadata_caches_on_object_storage_key", :algorithm=>:concurrently})
main: -> 0.0014s
main: -- execute("RESET statement_timeout")
main: -> 0.0002s
main: == 20230405071033 AddObjectStorageKeyToPackagesNpmMetadataCaches: migrated (0.0204s)
main: == 20230503191056 AddTextLimitToPackagesNpmMetadataCachesObjectStorageKey: migrating
main: -- transaction_open?()
main: -> 0.0000s
main: -- transaction_open?()
main: -> 0.0000s
main: -- execute("ALTER TABLE packages_npm_metadata_caches\nADD CONSTRAINT check_f97c15aa60\nCHECK ( char_length(object_storage_key) <= 255 )\nNOT VALID;\n")
main: -> 0.0006s
main: -- execute("ALTER TABLE packages_npm_metadata_caches VALIDATE CONSTRAINT check_f97c15aa60;")
main: -> 0.0004s
main: == 20230503191056 AddTextLimitToPackagesNpmMetadataCachesObjectStorageKey: migrated (0.0187s)
main: == [advisory_lock_connection] object_id: 274380, pg_backend_pid: 4527
⬇ Migration down
$ rails db:migrate:down:main VERSION=20230405071033
main: == [advisory_lock_connection] object_id: 274160, pg_backend_pid: 3290
main: == 20230405071033 AddObjectStorageKeyToPackagesNpmMetadataCaches: reverting ===
main: -- remove_column(:packages_npm_metadata_caches, :object_storage_key)
main: -> 0.0050s
main: == 20230405071033 AddObjectStorageKeyToPackagesNpmMetadataCaches: reverted (0.0158s)
$ rails db:migrate:down:main VERSION=20230403093349
main: == [advisory_lock_connection] object_id: 274120, pg_backend_pid: 3701
main: == 20230403093349 EnsurePackagesNpmMetadataCachesIsEmpty: reverting ===========
main: == 20230403093349 EnsurePackagesNpmMetadataCachesIsEmpty: reverted (0.0039s) ==
main: == [advisory_lock_connection] object_id: 274120, pg_backend_pid: 3701
$ rails db:migrate:down:main VERSION=20230503191056
main: == [advisory_lock_connection] object_id: 274120, pg_backend_pid: 4115
main: == 20230503191056 AddTextLimitToPackagesNpmMetadataCachesObjectStorageKey: reverting
main: -- transaction_open?()
main: -> 0.0000s
main: -- transaction_open?()
main: -> 0.0000s
main: -- execute(" ALTER TABLE packages_npm_metadata_caches\n DROP CONSTRAINT IF EXISTS check_f97c15aa60\n")
main: -> 0.0005s
main: == 20230503191056 AddTextLimitToPackagesNpmMetadataCachesObjectStorageKey: reverted (0.0200s)
main: == [advisory_lock_connection] object_id: 274120, pg_backend_pid: 4115
🔬 How to set up and validate locally
All steps need to be executed in rails console
-
Create a new npm package:
def fixture_file_upload(*args, **kwargs) Rack::Test::UploadedFile.new(*args, **kwargs) end package = FactoryBot.create(:npm_package, project: Project.first)
-
Fetch packages as
ActiveRecord::Relation
:packages = Packages::Package.with_name(package.name)
-
Execute the service to create a new entity of
Packages::Npm::MetadataCache
:Packages::Npm::CreateMetadataCacheService.new(Project.first, package.name, packages).execute
-
Verify that a new entity of
Packages::Npm::MetadataCache
is created with the metadata:metadata_cache = Packages::Npm::MetadataCache.last Gitlab::Json.parse(metadata_cache.file.read)
-
Create a new tag:
Packages::Npm::CreateTagService.new(package, "my-new-tag").execute
-
Trigger the
Packages::Npm::CreateMetadataCacheService
to update the metadata of existingPackages::Npm::MetadataCache
entry:Packages::Npm::CreateMetadataCacheService.new(Project.first, package.name, packages).execute
-
Verify if the metadata gets updated and now contain a new
my-new-tag
tag:metadata_cache = Packages::Npm::MetadataCache.last Gitlab::Json.parse(metadata_cache.file.read)
✅ MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.
Related to #393635 (closed)