Skip to content

Use npmjs.org as a default remote repository when the package is not found in the GitLab NPM Registry

frontend weight: 1
backend weight: 2

Problem to solve

The GitLab NPM Registry allows users to build, publish and share npm packages right alongside their source code and pipelines.

The NPM Registry documentation indicates that:

  • Scoped packages must be used
  • The scope must be the same as the root group name in gitlab
  • The project name is preferred to be the same name as the project.

This works fine if the GitLab NPM Registry is the only source of packages with the GitLab <group_name> scope. However, it's common that users also publish open source packages to the global npm registry using the same scope. In addition, many private packages may depend on those public packages.

Intended users

Further details

Use cases

The primary use case is for organizations that publish both public and private scoped packages, where their private npm packages depend on their public npm packages, that utilize the GitLab group and project as their scope.

Implementation

  • This feature is in-line with our goals for the Dependency Proxy, which proxies and caches Docker images for use in CI/CD and helps with faster, more reliable builds. We will evaluate if this can be done via the Dependency Proxy or if there is a faster path to value by just creating multiple remotes.

Examples

For example, let's assume that the gitlab root namespace is acme, found at https://gitlab.com/acme, that there's a private project at https://gitlab.com/acme/child, and that that project depends on a public npm package @acme/parent. The name of the child project, to be compliant with gitlab.com's requirements, must use scope @acme, and is recommended to be @acme/child.

Assume the .gitlab-ci.yml looks like the following:

.authenticate-to-npmjs:
  - &authenticate-to-npmjs |
    echo "$NPM_SCOPE:registry=https://$NPM_REGISTRY_HOST_PATH" >> ~/.npmrc
    echo "//$NPM_REGISTRY_HOST_PATH:_authToken=$NPM_WRITE_TOKEN" >> ~/.npmrc
    echo "//$NPM_PROJECT_HOST_PATH_PREFIX$CI_PROJECT_ID$NPM_PROJECT_HOST_PATH_SUFFIX:_authToken=$NPM_WRITE_TOKEN" >> ~/.npmrc

stages:
  - build
  - publish

build:
  stage: build
  image: node:8-alpine
  script:
    - npm run build

publish:
  stage: publish
  image: node:8-alpine
  script:
    - *authenticate-to-npmjs
    - npm install
    - npm run make-distribution # invokes babel transpilation to lib/main
    - cd lib/main
    - npm publish --access restricted
  only:
    - /^([0-9]{1,}\.[0-9]{1,}\.[0-9]{1,}(-(pre|rc)\.[0-9]{1,})?)$/

The publish job fails at npm install because gitlab's npm can't find package @acme/parent; it's in the global npm registry, not the gitlab private npm registry. The gitlab.com npm registry should forward requests for missing scoped packages to the global npm registry.

Proposal

Allow users of the NPM Registry to forward scoped package requests to the global npm registry when the package is not found in the gitlab private registry.

  • By default this feature should be enabled
  • Give users the ability to disable this feature from within the user interface
  • Count number of packages pulled/pushed and distinguish between GitLab NPM Registry and npmjs.org

User interface

Screenshot_2020-02-13_at_10.19.25

What we are not doing

  • As a first iteration, we will limit the scope to one additional remote. However, in the future we may allow users to add multiple remotes.

Permissions and Security

  • Developers and above can enable and disable this feature

Documentation

Testing

Automated testing of this feature is possible.

What does success look like, and how can we measure that?

In the gitlab.com CI/CD pipeline, npm & yarn, when installing dependencies for a private, scoped package, should be able to resolve a public package with the same scope in the gitlab.com CI/CD pipeline.

Metrics

  • Count number of requests that are sent to GitLab NPM Registry vs. npmjs.org
  • Measure ratio of the above
  • We can utilize this data to better understand the priority and use cases for the Dependency Proxy.

Links / references

Edited by Tim Rizzi