Incorporate Go modules into the Packages UI and API

Background

Go uses a source-based dependency management system, whereas most other dependency management systems are artifact-based. This is to say, Go dependencies are ultimately fetched directly from their source VCS repository, but dependencies in other systems are artifacts that have been uploaded to a package repository. Another unique feature of the Go ecosystem is the name of a package (excluding stdlib) must be a valid URL, sans the scheme (e.g. golang.org/x/text). Thus, Go modules are defined by the source repository and have unique names.

Problem to solve

GitLab should incorporate Go modules into the Packages UI and API.

For the artifact-based dependency management systems that GitLab supports (i.e. all of them except Go), database entries are created when the user pushes a package. However, Go dependency management is VCS-based, which in the context of GitLab means Git-based, thus a Git tag is a module release and there is no other way to release modules (technically anything is possible, but that would be a Bad Idea).

In principal, Go modules are not special and can be easily incorporated into the UI and API. However, since releasing a module (version) is a Git action and not an API action, populating the database is potentially problematic.

Intended users

• Rachel (Release Manager)
• Delaney (Development Team Lead)
• Sasha (Software Developer)
• Devon (DevOps Engineer)
• Sidney (Systems Administrator)

Further details

TODO

Proposal

The key issue is either populating the database or working around the database. The latter is possible, but not a real option, because it requires some nasty hacks. Potential strategies:

• Triggered by an action: Database entries are created synchronously when the user executes an action.
• Scheduled periodically: Database entries are created asynchronously by a cron job.
• Scheduled by an action: Database entries are created asynchronously by a scheduled job that is created when a user executes an action.
• Some combination of these options.

Potential actions:

• Pushing a tag, if this can be hooked reliably
• Querying the packages API (or equivalent UI operation)
• Querying the Go modules proxy
• A specific API endpoint that updates the database

I made a quick proof of concept which utilizes the aforementioned nasty hacks to work around the database issue.

Permissions and Security

Calling the appropriate helpers in the API should be sufficient:

before do
  require_packages_enabled!
  authorize_read_package!
  authorize_packages_feature!

  # for creating DB entries
  authorize_create_package!

  # for deleting DB entries
  authorize_destroy_package!
end

Documentation

• administration/packages/index.md
• user/packages/index.md
• user/packages/go_proxy/index.md
• development/packages.md
• api/packages.md

Availability & Testing

TODO

What does success look like, and how can we measure that?

Go modules are included in responses from the Packages API, listed in the Packages UI, and there is a tab for Go modules in the Packages UI.

What is the type of buyer?

• Individual Contributor
• Manager
• Director

Is this a cross-stage feature?

Not sure.

Links / references

!27746 (merged), #27376 (closed)