Skip to content

Add authentication for GitLab -> Zoekt HTTP calls

There is a lot more detail at !107891 (comment 1248586481) and gitlab-org/cloud-native/charts/gitlab-zoekt!7 (comment 1323530208) but we need to work with appsec and infra to figure out a suitable authentication scheme between GitLab and Zoekt.

It seems most (or all) internal service calls still have some form of authentication (Redis, Registry, Postgres, Elasticsearch, ...) so we should implement an authentication layer for Zoekt as well.

Proposal

It seems Zoekt does support terminating SSL but it does not have any authentication mechanism. So we either have the option of building some authentication mechanism into Zoekt or instead adding an NGinx proxy in front of Zoekt. I propose an NGinx proxy because it will be simpler and doesn't require us working with Zoekt maintainers to find a solution that they are happy with.

In our K8s infrastructure we can add another Nginx container to a pod and then use either mututal SSL or Basic Auth with SSL for authenticating requests. Mutual SSL will be preferred but this requires deploying client certificates with our rails deployment. If we don't already have a straightforward mechanism for this then we can at least use Basic Auth (which can already be encoded in our Zoekt configuration in Rails) but we would also want SSL on the Nginx server so that the password is not being sent in plain text.

Additionally once we have this we should update our NetworkPolicy rules (see gitlab-org/cloud-native/charts/gitlab-zoekt!7 (comment 1321679998) ) such that only the NGinx container port is accessible on each pod and it forwards requests to the Zoekt container ports. The Zoekt container ports should block all ingress traffic from the cluster (ie. only accepting local traffic from the Nginx server).

TODO

  1. gitlab-zoekt chart: Add an nginx container that proxies the webserver gitlab-org/cloud-native/charts/gitlab-zoekt!14 (merged)
  2. gitlab-zoekt chart: Nginx container can mount and use a named Certificate gitlab-org/cloud-native/charts/gitlab-zoekt!14 (merged)
  3. gitlab-zoekt chart: Nginx container can mount and use a named Secret that will be used for basic auth gitlab-org/cloud-native/charts/gitlab-zoekt!14 (merged)
  4. gitlab chart: Rails container can mount a named Secret as .gitlab_zoekt_secret gitlab-org/charts/gitlab!3184 (merged)
  5. gitlab rails: Rails will use (if present) a .gitlab_zoekt_secret when making requests to Zoekt !127203 (merged)
  6. Update gitlab-com to configure TLS and basic auth for Zoekt
  7. Consider if we should make this .gitlab_zoekt_secret mandatory in Rails, gitlab-zoekt chart or both
  8. Remove indexserver and webserver from load balancer and service gitlab-org/cloud-native/charts/gitlab-zoekt!28 (merged)
  9. Remove open ports for indexserver and webserver gitlab-com/gl-infra/k8s-workloads/gitlab-com!2923 (merged)
Old plan with mTLS
  1. Allow configuring mutual TLS client certificate in GitLab Rails: !120406 (closed)
  2. Figure out if will be possible to scrape prometheus metrics and do K8s health checks with mutual TLS
  3. Allow configuring mutual TLS verification in Zoekt: https://github.com/DylanGriffith/zoekt/tree/allow-mutual-tls
  4. Update the main CNG helm chart to allow configuring secrets and config/gitlab.yml for the client PEM and CA Cert
  5. Update the zoekt-webserver image to allow configure the SSL options
  6. Update the gitlab-zoekt helm chart to allow configuring SSL options as secrets
  7. Update our production config to configure these options in the Helm charts
Edited by Dylan Griffith