Webservice: Implement generator pattern to allow traffic splitting across scalable Deployments via Ingress

Summary

Convert Webservice to a "generator" pattern per precedent set with Sidekiq, allowing 1 or more Deployment/Service/PodDisruption/HorizontalPodAutoscaler to be generated, but sharing Ingress/NetworkPolicy. We should only require simple option differences to start: min / max Pods, Ingress path (/api), annotation/label, resources (maybe).

Spawned from this comment in #2212 (closed)

Conceptual map, based on manual testing implementation:

Webservice becomes a "generator" pattern, using label/annotations for delineation of work.

• Structure:
  • A single Ingress is made, which sends to the destination Service(s) based on /path
  • Service/Deployment/HPA/PDB are generated with the according label / annotations accounted for
• Configuration:
  • Allows "splitting" similar to how we've implemented Sidekiq in this pattern (including min/max pods)

This could be implemented via alias-ing gitlab-webservice several times and detecting the difference in configuration. We can look into this, but due to the inheritance patterns, this could lead to more complex and possibly highly repetitive configuration.

Details

This can be manually performed and tested, as referenced in the comment above. This was demo'd on 2020-10-01

• Deploy the existing chart
• Use helm get manifest to fetch all objects
• Remove objects other than those for Webservice
• Copy necessary items
  • Add a suffix to metadata.name per object (web/api/git-https)
  • Add entry to metadata.labels per object, gitlab.com/webservice: X
  • Use the above label within the Selectors for each managing object.
• Edit the Ingress for gitlab.example.com, adding to paths array

        paths:
          - path: /
            backend:
              serviceName: {{ template "fullname" . }}-web
              servicePort: {{ .Values.service.workhorseExternalPort }}
          - path: /admin/sidekiq
            backend:
              serviceName: {{ template "fullname" . }}-web
              servicePort: {{ .Values.service.externalPort }}
          - path: /api
            backend:
              serviceName: {{ template "fullname" . }}-api
              servicePort: {{ .Values.service.workhorseExternalPort }}
          - path: /.*\.git
            backend:
              serviceName: {{ template "fullname" . }}-git-https
              servicePort: {{ .Values.service.workhorseExternalPort }}

Conceptual design

Configuration of gitlab.webservice

gitlab:
  webservice:
    deployments:
      web:
        minReplicas: 2
        maxReplicas: 10
        # docs: https://kubernetes.io/docs/concepts/services-networking/ingress/#ingress-rules
        ingress:
           path: /
      api:
        minReplicas: 4
        maxReplicas: 10
        ingress:
           path: /api
      git-https:
        minReplicas: 1
        maxReplicas: 4
        ingress:
           # doc: https://kubernetes.github.io/ingress-nginx/user-guide/ingress-path-matching/
           path: /.*\.git

Current behavior

All traffic goes to all Webservice pods.

Expected behavior

Traffic can be divided by path across different Deployments of GitLab Webservice chart, allowing division of workloads.