Support NGINX Ingress for Geo traffic

What does this MR do?

Support NGINX Ingress for Geo traffic

Support reconciling the bundled NGINX chart for Geo workloads. This is a dependency to support unified URLs.

Chart documentation: https://docs.gitlab.com/charts/charts/nginx/#gitlab-geo

Changelog: added

Checklist

See Definition of done.

For anything in this list which will not be completed, please provide a reason in the MR discussion.

Required

• Merge Request Title and Description are up to date, accurate, and descriptive
• MR targeting the appropriate branch
• MR has a green pipeline on GitLab.com
• When ready for review, MR is labeled "~workflow::ready for review" per the Distribution MR workflow

Expected (please provide an explanation if not completing)

• Test plan indicating conditions for success has been posted and passes
• Documentation created/updated
  • Will be added in !960 (merged)
• Tests added
• Integration tests added to GitLab QA
• Equivalent MR/issue for omnibus-gitlab opened
• Validate potential values for new configuration settings. Formats such as integer 10, duration 10s, URI scheme://user:passwd@host:port may require quotation or other special handling when rendered in a template and written to a configuration file.

Test plan

1. Create the Geo NGINX IngressClass

   $ kubectl apply -f - <<EOF
   apiVersion: networking.k8s.io/v1
   kind: IngressClass
   metadata:
     name: gitlab-nginx-geo
   spec:
     controller: k8s.io/ingress-nginx-geo
   EOF

2. Create GitLab and enable the controller

   $ kubectl apply -f - <<EOF
   apiVersion: apps.gitlab.com/v1beta1
   kind: GitLab
   metadata:
     name: gitlab
     namespace: gitlab-system
   spec:
     chart:
       version: 8.3.1
       values:
         nginx-ingress-geo:
           enabled: true
   EOF

3. Confirm both the default NGINX and the Geo NGIX controller are healthy

Related issues

Relates #1568 (closed)