WIP: POC/Discovery for group level clusters
What does this MR do?
Validate and note what assumptions we violate. See what is the list of mergeable MRs
Possible MRs
-
backend Allow
project#deployment_platform
to choose group level cluster if match environment scope -
backend Create cluster on group level
- Only install helm and nginx-ingress
- Refactor
Clusters::CreateService
to take in either group or project as a param. - Refactor
Clusters::Applications::ScheduleInstallationService
to take in either group or project as a param. - Significant code re-use for
Projects::ClustersController
andProjects::Clusters::ApplicationController
- frontend Cluster application install vue component with only helm and ingress
- frontend backend Show group level clusters on project cluster page
-
backend Allow
Clusters::Platform::Kubernetes
to function withgroup
, not justproject
- Auto Devops domain - Document that users cannot use the same base domain name for 2 different projects (or prefix with project name)
Discard pile
Refactor to have multiple project-namespace pairs-
Possibly re-useClusters::Platforms::Kubernetes
and refactor to handleprojects
not `project.Work out how to break compat withKubernetesService
cleanlyCreate the project namespace, persist with a database table (cluster_projects)
Persistgitlab-managed-apps
namespace into database as wellInstall cluster applications on project level, into project namespace
POC Tasks
-
Create a group-level cluster -
install helm -
install ingress -
Find appropriate group or project level cluster - Environment scope ? -
destroy group-level cluster -
Only allow installing of helm and nginx-ingress for a group-level cluster -
Project, use group-level cluster for this project - [-]
Install runner into project level namespace for the cluster
Discovery tasks
-
Precedence of Environment Scope
for picking which cluster to deploy to -
database models we'll use (do we use the same tables or new tables?) - Can mostly re-use the same tables
-
clusters
can be linked to multiple projects viacluster_projects
-
inheritance of env variables down to project levels - The variables involved here looks like it will be :
- KUBE_URL
- KUBE_TOKEN
- KUBE_NAMESPACE
- KUBECONFIG
- KUBE_CA_PEM
- KUBE_CA_PEM_FILE
- The variables involved here looks like it will be :
-
Groups, subgroups edge cases -
k8s namespaces we'll use (different for each project?) - Decision Start with using the generated namespace each cluster will generate.
- Unsure what will happen if the user enters one - disallow that for group-level clusters ?
-
Installing helm applications for groups or keep doing this per project, what are the trade-offs? - Decision Helm applications should be installed only for the cluster.
- Installing helm applications for a group-level cluster as per current should be possible with no change.
- If per project, will have to install into project unique namespaces instead of fixed namespace
gitlab-managed-apps
. Cannot "share" applications
-
Will RBAC require more consideration to make this work (more service accounts now?) - Does not look like it if we are not changing how we map project environments to namespaces
- Each project namespace which has Helm Tiller will need service account isolated to that namespace
-
how does our solution extend to instance wide as this is coming up soon (ignoring instance wide on gitlab.com as this brings it's own completely different security challenges) -
Ingress integration : cluster-wide or per project/namespace ? - Decision Cluster-wide but should not use the same base domain with 2 different apps!
- Consider hiding the help text under the domain name input as it encourages using the same nip.io domain for both. Or we prefix this domain name with the project name or something else to ensure they do not conflict.
-
Prometheus integration : How will prometheus work at the group level? Currently, prometheus is a service-level integration, so there's nothing at the group level. How will we have service integration for prometheus? This leads to other questions such as how to manage it or how to configure custom metrics. - Decision 2nd iteration
- (Dylan) Prometheus also makes most sense at group level but we just need the monitoring team to take a look at what it will take to support monitoring multiple apps (ie. how much code has assumed there is only a single app running?)
-
Runner integration : How will runner work at the group level ? - Decision 2nd iteration
- (Dylan) Runner can be installed at group level, but it just requires a little work. I think group level is a more useful solution here anyway and the need for project level may not be that great
-
Jupyterhub integration : Integration with just one project ? - Decision 2nd iteration
- (Dylan) JupyterHub may also end up making sense to run at the group level too since it should handle multiple users etc quite well. Since it's heavy-weight software it will probably end up being useful multi-tenanted too
-
Auto DevOps domain: currently, every project has a field for "auto devops domain", that doesn't work well when the cluster is at the group level (or multiple clusters at the group level). - (Daniel) One possible solution would be to NOT allow project-level domain configuration when there's group level clusters (thought it feels heavy-handed).
-
Who can access group-level clusters? I'd imagine (just as in project-level) that group admin & maintainer are able to manage and developer can view. - This should be the same as project-level cluster
-
Auto DevOps - See Environment scope
-
$CI_ENVIRONMENT_SLUG
which expands to e.g.production
. What does it mean to deploy project X intoproduction
environment ? How will that clash with deploying projectY
intoproduction
environment too ?
-
Compatibility with KubernetesService
- instance level cluster required replacement
What are the relevant issue numbers?
POC for https://gitlab.com/gitlab-org/gitlab-ee/issues/7412
Does this MR meet the acceptance criteria?
- [-] Changelog entry added, if necessary
- [-] Documentation created/updated
- [-] Tests added for this feature/bug
- [-] Conforms to the code review guidelines
- [-] Conforms to the merge request performance guidelines
- [-] Conforms to the style guides
- [-] Conforms to the database guides
Edited by 🤖 GitLab Bot 🤖