Support publishing Composer package to one central project
What does this MR do and why?
Implements #250633.
This MR adds source_project
parameter to Composer package creation endpoint.
It achieves that by:
- Adding
source_project_id
column topackages_composer_metadata
table. - Migrating all existing composer packages to fill in
source_project_id
frompackages_packages.project_id
- before this change the source was always in the same project as composer repository. Note that since this migration runs in post_migrate, I had to allow null value insource_project_id
for now. In the future version we could rollback commitAllow for composer source_project_id column to be null
. - Using the new column
source_project_id
when generating links in composer responses (both for source and distribution) to make source composer uses the source project when fetching data.
Screenshots or screen recordings
This is an API only change. Here's example of composer repository with packages from multiple projects added:
Database migration log
Migration log
$ rake db:migrate
== 20211212174031 AddComposerPackageSourceProjectId: migrating ================
-- add_column(:packages_composer_metadata, :source_project_id, :integer)
-> 0.0006s
== 20211212174031 AddComposerPackageSourceProjectId: migrated (0.0006s) =======
== 20211212174638 AddIndexForComposerPackageSourceProjectId: migrating ========
-- transaction_open?()
-> 0.0000s
-- index_exists?(:packages_composer_metadata, :source_project_id, {:name=>"index_packages_composer_metadata_on_source_project_id", :algorithm=>:concurrently})
-> 0.0015s
-- execute("SET statement_timeout TO 0")
-> 0.0003s
-- add_index(:packages_composer_metadata, :source_project_id, {:name=>"index_packages_composer_metadata_on_source_project_id", :algorithm=>:concurrently})
-> 0.0074s
-- execute("RESET statement_timeout")
-> 0.0004s
== 20211212174638 AddIndexForComposerPackageSourceProjectId: migrated (0.0107s)
== 20211213062010 AddFkForComposerPackageSourceProjectId: migrating ===========
-- transaction_open?()
-> 0.0000s
-- foreign_keys(:packages_composer_metadata)
-> 0.0016s
-- transaction_open?()
-> 0.0000s
-- execute("ALTER TABLE packages_composer_metadata\nADD CONSTRAINT fk_b959c6a2d5\nFOREIGN KEY (source_project_id)\nREFERENCES projects (id)\nON DELETE CASCADE\nNOT VALID;\n")
-> 0.0011s
-- execute("ALTER TABLE packages_composer_metadata VALIDATE CONSTRAINT fk_b959c6a2d5;")
-> 0.0015s
== 20211213062010 AddFkForComposerPackageSourceProjectId: migrated (0.0063s) ==
== 20211213062612 BackfillComposerPackageSourceProjectId: migrating ===========
-- transaction_open?()
-> 0.0000s
-- execute("UPDATE packages_composer_metadata\nSET source_project_id = project_id\nFROM packages_packages\nWHERE source_project_id IS NULL AND package_id = packages_packages.id AND package_id BETWEEN 1 AND 37\n")
-> 0.0019s
== 20211213062612 BackfillComposerPackageSourceProjectId: migrated (0.0191s) ==
$ rake db:migrate:down VERSION=20211213062612
== 20211213062612 BackfillComposerPackageSourceProjectId: reverting ===========
== 20211213062612 BackfillComposerPackageSourceProjectId: reverted (0.0000s) ==
$ rake db:migrate:down VERSION=20211213062010
== 20211213062010 AddFkForComposerPackageSourceProjectId: reverting ===========
-- transaction_open?()
-> 0.0000s
-- remove_foreign_key(:packages_composer_metadata, {:column=>:source_project_id})
-> 0.0028s
== 20211213062010 AddFkForComposerPackageSourceProjectId: reverted (0.0060s) ==
$ rake db:migrate:down VERSION=20211212174638
== 20211212174638 AddIndexForComposerPackageSourceProjectId: reverting ========
-- transaction_open?()
-> 0.0000s
-- indexes(:packages_composer_metadata)
-> 0.0015s
-- execute("SET statement_timeout TO 0")
-> 0.0003s
-- remove_index(:packages_composer_metadata, {:algorithm=>:concurrently, :name=>"index_packages_composer_metadata_on_source_project_id"})
-> 0.0036s
-- execute("RESET statement_timeout")
-> 0.0003s
== 20211212174638 AddIndexForComposerPackageSourceProjectId: reverted (0.0068s)
$ rake db:migrate:down VERSION=20211212174031
== 20211212174031 AddComposerPackageSourceProjectId: reverting ================
-- remove_column(:packages_composer_metadata, :source_project_id, :integer)
-> 0.0006s
== 20211212174031 AddComposerPackageSourceProjectId: reverted (0.0013s) =======
How to set up and validate locally
The easiest option is to run bundle exec rake db:seed_fu FILTER=composer SEED_COMPOSER=1
.
This will create project composer/composer-repository that will have composer packages with sources from another repo.
After running this seed you can go to <gdk-host>/composer/composer-repository/-/packages
.
To test it manually and check the API endpoint you can:
- Create a project A with a valid composer.json.
- Create a separate project B (in a group) that has package registry enabled.
- Create a composer package in a registry in project B, but with source from project A by using a new parameter
source_project
in Composer package creation endpoint. - Verify the dist and source links in composer package metadata endpoint.
Next, follow https://docs.gitlab.com/ee/user/packages/composer_repository/#install-a-composer-package to test with composer that package is installable.
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.