Go repository MVC followup
I am creating this as a place to discuss possible followup to #27376 (closed) and !27746 (merged). I am not requesting these features, but asking, "Should GitLab implement these features?" I have created separate feature request issues for features I think definitely should be added.
Background
Go uses a source-based dependency management system, whereas most other dependency management systems are artifact-based. This is to say, Go dependencies are ultimately fetched directly from their source VCS repository, but dependencies in other systems are artifacts that have been uploaded to a package repository. Another unique feature of the Go ecosystem is the name of a package (excluding stdlib) must be a valid URL, sans the scheme (e.g. golang.org/x/text
). Thus, Go modules are defined by the source repository and have unique names.
Dependency proxy?
I take it as given that a Go dependency proxy is included in GitLab's long term goals. I have not created a separate issue for it because I don't know how it should be done and it is not something I have a need for.
Buyer: Director/Executive (complete control over in-house Go dev ecosystem).
Support for legacy packages?
Go 1.10 and earlier did not have any notion of versioning or dependency management, beyond "You imported 'golang.org/x/text' so I'm going to find the repository associated with that and clone the latest commit of the default branch." Go 1.11 added beta support for modules and proper versioning. Go 1.13 released this as a mature strategy. !27746 (merged) implements MVC support for projects that have opted-in to the new strategy, and &3043 expands on this.
Should GitLab support pre-module-style Go packages? Many of these packages do not have any form of versioning (beyond Git history) and those that are well versioned do not conform to any particular standard. There is no indicator that "This project is a Go package" beyond "It contains *.go files".
Buyers are:
- Users of unmaintained packages that have not updated to the new system.
- Organizations that provide packages and have decided not to use the new system, or to delay adoption.
For large Go-based systems (kubernetes, AWS Go API, etc), the cost of adoption may be high, especially if they have already invested in one of the other package management tools (most commonly, vendoring managed by dep, Godep, Govendor, trash, etc).
Checksum database?
This ties in to a Go dependency proxy and is most relevant in the context of managing in-house use of Go packages. The public checksum database should cover most use cases for public projects, and private projects do not have the same concerns.
sum.golang.org
is an "auditable checksum database [that is] used by the go command to authenticate modules. Check out the Secure the Public Go Module Ecosystem Proposal for more details."
Essentially, the source of a given version of a package/module/dependency should never change, and if it does change that is a potential security issue. Go (as of 1.13) automatically hashes dependencies and saves the hash in go.sum
(next to go.mod
). This file should be committed. When the module is (fetched and) built, if the fetched dependencies do not match the hash in go.sum
, Go issues a build failure. When Go fetches a dependency, it queries sum.golang.org
for the checksum (the user can configure it to skip this, use a different server, or use multiple servers). Thus the authenticity of fetched dependencies can be verified, if they are in the public database, even if they are not present in go.sum
.
Should GitLab implement a Go checksum database? The buyer is a director or executive looking to establish control/management over in-house use of the Go ecosystem.
Package index?
This ties in to a Go dependency proxy and is most relevant in the context of managing in-house use of Go packages. The public index should cover most use cases for public projects. A package index may have relevance to private packages.
index.golang.org
is an "index which serves a feed of new module versions that become available by proxy.golang.org. The feed can be viewed at https://index.golang.org/index."
The use-case I am familiar with is systems that want to know what packages can be found. Previously, systems such as godoc.org
dynamically generated content in response to a user query, if the request points to an accessible Go package. The new documentation system, pkg.go.dev
, is closed source, but based on the proposal that introduced the index, pkg.go.dev
may be using the index to discover new modules (and new versions) and using that to schedule content generation. The index could be useful to any system that generates content or otherwise acts on arbitrary Go modules and thus would benefit from a feed of known modules and their versions.
Should GitLab implement a Go package index? The main buyer is a director or executive looking to establish control/management over in-house use of the Go ecosystem. Another buyer may be a manager or sysadmin looking to set up something like godoc.org
or pkg.go.dev
for private projects.