Multiple package pipelines for package history
🌲 Context
The GitLab package registry allows users to publish their own packages to GitLab projects. Currently, if a package is published for the very first time using GitLab CI, a build_infos
record is created that associating the package record with the pipeline that created it.
Users would like to be able to not only see the first pipeline, but anytime a pipeline may have republished a version. This is a common pattern with Maven SNAPSHOT packages. The same version may be published many times, but due to the current limitations, we only see the pipeline from the very first one that was published which may no longer be relevant.
🔍 What does this MR do?
This MR updates the relationships between packages and pipelines to change from a one-to-one to a one-to-many relationship. We do this by:
- Updating the relationships on
Packages::Package
andPackages::BuildInfo
- Removing the
UNIQUE
index onpackages_build_infos
that prevented us from having more than one record per package. - Update the logic of when a
build_info
is created to not only occur when a package is first created, but also when it is updated.
The MR creates a new relationship between package files and pipelines so in addition to seeing when a package record was created/updated, we can see the pipelines that created the individual files belonging to that package. There are cases where the package files may be created in a different pipeline from when the package was created. We do this by:
- Create a new model:
Packages::PackageFileBuildInfo
that relates package files to pipelines. - Add logic to create these records anytime a new
package_file
record is created.
A few notes:
- This MR only updates the backend and the API payload. We add a method to
Packages::Package#build_info
to mimic the previously existing association and allow the front end to continue to function, displaying only the firstbuild_info
- We have
pipeline
andpipelines
now returned in the API entity. Once the frontend is updated to usepipelines
, we will removepipeline
and the#build_info
method. - This only updates Maven, NPM, and generic packages. The other package types do not yet track pipeline data, which will be added in a separate MR
- The frontend updates to display the new pipeline information will be implemented in a separate MR.
🐘 Database
UpdatePackagesBuildInfosIndex
Up:
== 20201103202213 UpdatePackagesBuildInfosIndex: migrating ====================
-- transaction_open?()
-> 0.0000s
-- index_exists?(:packages_build_infos, :package_id, {:name=>"idx_packages_build_infos_on_package_id", :algorithm=>:concurrently})
-> 0.0029s
-- execute("SET statement_timeout TO 0")
-> 0.0003s
-- add_index(:packages_build_infos, :package_id, {:name=>"idx_packages_build_infos_on_package_id", :algorithm=>:concurrently})
-> 0.0030s
-- execute("RESET ALL")
-> 0.0003s
-- transaction_open?()
-> 0.0000s
-- indexes(:packages_build_infos)
-> 0.0028s
-- remove_index(:packages_build_infos, {:algorithm=>:concurrently, :name=>"index_packages_build_infos_on_package_id"})
-> 0.0032s
== 20201103202213 UpdatePackagesBuildInfosIndex: migrated (0.0140s) ===========
Down:
== 20201103202213 UpdatePackagesBuildInfosIndex: reverting ====================
== 20201103202213 UpdatePackagesBuildInfosIndex: reverted (0.0000s) ===========
CreatePackagesPackageFileBuildInfos
Up:
== 20201103184333 CreatePackagesPackageFileBuildInfos: migrating ==============
-- table_exists?(:packages_package_file_build_infos)
-> 0.0009s
-- create_table(:packages_package_file_build_infos)
-> 0.0112s
== 20201103184333 CreatePackagesPackageFileBuildInfos: migrated (0.0189s) =====
Down:
== 20201103184333 CreatePackagesPackageFileBuildInfos: reverting ==============
-- drop_table(:packages_package_file_build_infos)
-> 0.0032s
== 20201103184333 CreatePackagesPackageFileBuildInfos: reverted (0.0092s) =====
Screenshots
Does this MR meet the acceptance criteria?
Conformity
-
Changelog entry -
Documentation (if required) -
Code review guidelines -
Merge request performance guidelines -
Style guides -
Database guides - [-] Separation of EE specific content
Availability and Testing
-
Review and add/update tests for this feature/bug. Consider all test levels. See the Test Planning Process. - [-] Tested in all supported browsers
- [-] Informed Infrastructure department of a default or new setting change, if applicable per definition of done
Security
If this MR contains changes to processing or storing of credentials or tokens, authorization and authentication methods and other items described in the security review guidelines:
- [-] Label as security and @ mention
@gitlab-com/gl-security/appsec
- [-] The MR includes necessary changes to maintain consistency between UI, API, email, or other methods
- [-] Security reports checked/validated by a reviewer from the AppSec team
Related to #239518 (closed)