Skip to content

Set and use fields for workspace sudo access

Vishal Tak requested to merge vtak/workspace_sudo_access_backend into master

Issue: Backend: Set and use fields for sudo access (#474970 - closed)

What does this MR do and why?

In order to enable users to use sudo securely within a workspace, we have identified 3 ways - configure Sysbox, configure Kata containers, and configure Kubernetes cluster to use user namespaces feature.

To achieve this, users needs to provide certain values - labels, annotations, default_runtime_class, allow_privilege_escalation, use_kubernetes_user_namespaces. Details about what combination of fields need to be configured for each case(Sysbox, Kata Containers, User Namespaces) can be found in the comment.

This MR adds these fields for workspace sudo access in the agent config update. These values are then used to drive the behaviour of the Workspace Kubernetes resources generated during reconciliation.

The values are set in such a way that they keep the existing behaviour as it is. Existing workspaces will continue to have the same behaviour and will not be affected by this change.

What has changed in the code?

  • Commit 1 - Added the new settings in the Remote Development Settings module in ee/lib/remote_development/settings/default_settings.rb.
  • Commit 1 - Update the agent config updater to set these new values in the database in ee/lib/remote_development/agent_config_operations/updater.rb.
  • Commit 2 - Abstract Kubernetes resources parameters required for generating desired config in ee/lib/remote_development/workspace_operations/reconcile/output/kubernetes_resources_params.rb.
  • Commit 3 - When converting the devfile to Kubernetes resources in ee/lib/remote_development/workspace_operations/reconcile/output/devfile_parser.rb, set the required values.
    • Deployment's .spec.template.spec.hostUsers to the associated agent's use_kubernetes_user_namespaces if true.
    • Deployment's .spec.template.spec.runtimeClassName to the associated agent's default_runtime_class_name if present.
    • Deployment's .spec.template.spec.containers[*].securityContext.allowPrivilegeEscalation to the associated agent's allow_privilege_escalation.
  • Following commits are based on review feedback.

MR acceptance checklist

Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Screenshots or screen recordings

Screenshots are required for UI changes, and strongly recommended for all other merge requests.

Before After
image image

How to set up and validate locally

Numbered steps to set up and validate the change are strongly suggested.

A fully configured test environment has been setup to test this MR. The details of how to access it will be shared with you on Slack when you are assigned as a reviewer/maintainer.

For posterity, following are the steps to setup the said test environment.

  1. Setup 3 Kubernetes clusters and configure Sysbox, Kata Containers and User Namespaces respectively. Configure GitLab Workspaces Proxy in all of them.

  2. Create a new GitLab instance on AWS EC2 by following the below steps. EC2 instance type c5.4xlarge, disk size 100 gb, Ubuntu 24

  3. Login to instance

    ssh -i ~/.ssh/key-pair.pem ubuntu@PUBLIC_IP
mkdir -p gdk
cd gdk

  4. Install GDK - https://gitlab.com/gitlab-org/gitlab-development-kit/-/blob/main/doc/index.md#one-line-installation

  5. Follow local network binding guide for 172.16.123.1 - https://gitlab.com/gitlab-org/gitlab-development-kit/-/blob/main/doc/index.md#one-line-installation

  6. Setup the gdk.yml

    ---
gitlab_k8s_agent:
  enabled: true
listen_address: 172.16.123.1
gitlab:
  rails:
    hostname: 'gitlab.remote-development-test.com'
    allowed_hosts:
    - 'gitlab.remote-development-test.com'
vite:
  enabled: true
webpack:
  enabled: false

  7. Checkout relevant branch in GitLab repository and run db migrations

    cd ~/gdk/gitlab-development-kit/gitlab
git checkout vtak/workspace_sudo_access_backend
bin/rails db:migrate RAILS_ENV=development

  8. Because of the way the GDK will be exposed publically, we need to make a small patch to the project URL to make it accessible. Save the following patch inside ~/gdk/gitlab-development-kit/gitlab folder as project_url.patch and apply it using git apply project_url.patch. Workspaces not working for public GDK (#486020) is created to look into this further.

    diff --git a/ee/lib/remote_development/workspace_operations/create/project_cloner_component_injector.rb b/ee/lib/remote_development/workspace_operations/create/project_cloner_component_injector.rb
index 1813da2d51d4..c8001b62f8a7 100644
--- a/ee/lib/remote_development/workspace_operations/create/project_cloner_component_injector.rb
+++ b/ee/lib/remote_development/workspace_operations/create/project_cloner_component_injector.rb
@@ -30,7 +30,7 @@ def self.inject(context)
          #       replace the alpine/git docker image with one that is published by gitlab for security / reliability
          #       reasons
          clone_dir = "#{volume_path}/#{project.path}"
-          project_url = project.http_url_to_repo
+          project_url = project.http_url_to_repo.sub('http://', 'https://').sub(':3000', '')

          # The project should be cloned only if one is not cloned successfully already.
          # This is required to avoid resetting user's modifications to the files.
diff --git a/ee/lib/remote_development/workspace_operations/create/workspace_variables.rb b/ee/lib/remote_development/workspace_operations/create/workspace_variables.rb
index 3a4ade8e8498..d6ddd91b054e 100644
--- a/ee/lib/remote_development/workspace_operations/create/workspace_variables.rb
+++ b/ee/lib/remote_development/workspace_operations/create/workspace_variables.rb
@@ -138,7 +138,7 @@ def self.variables(
            # variables with prefix `GITLAB_WORKFLOW_` are used for configured GitLab Workflow extension for VS Code
            {
              key: 'GITLAB_WORKFLOW_INSTANCE_URL',
-              value: Gitlab::Routing.url_helpers.root_url,
+              value: Gitlab::Routing.url_helpers.root_url.sub('http://', 'https://').sub(':3000', ''),
              variable_type: RemoteDevelopment::Enums::Workspace::WORKSPACE_VARIABLE_TYPES[:environment],
              workspace_id: workspace_id
            },

  9. Enable workspaces extensions marketpalces feature flag

    cd ~/gdk/gitlab-development-kits/gitlab
bin/rails c
Feature.enable(:allow_extensions_marketplace_in_workspace, Group.find_by_path('gitlab-org'))

  10. Install Bazel - Follow step 1 of https://gitlab.com/gitlab-org/gitlab-development-kit/-/blob/main/doc/howto/kubernetes_agent.md

  11. Expose gdk publically - https://gitlab.com/gitlab-org/gitlab-development-kit/-/blob/main/doc/howto/public_gdk.md

    cd ~/gdk/gitlab-development-kit
cat >Caddyfile <<EOL
gitlab.remote-development-test.com {
  reverse_proxy 172.16.123.1:3000
}
EOL
    sudo caddy start

    If nginx service is running on the instance, disbale it sudo systemctl disable nginx && sudo systemctl stop nginx.

  12. Reconfigure and restart GDK

    cd ~/gdk/gitlab-development-kit
gdk reconfigure
gdk restart

  13. Login to GitLab instance as root user, disable sign ups, add license

  14. Setup AWS CLI by following https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html

  15. Create new access keys and set them up.

    aws configure --profile gl-sandbox-group-remote-vtak-root

  16. Fetch the Kube Config for the 2 Kubernetes clusters

    aws eks update-kubeconfig --name "eks-sysbox" --region "ap-south-1" --profile gl-sandbox-group-remote-vtak-root
aws eks update-kubeconfig --name "eks-kata" --region "ap-south-1" --profile gl-sandbox-group-remote-vtak-root

  17. Copy the kubeconfig file from the User Namespace Kubeadm Kubernetes cluster from location /etc/kubernetes/admin.conf(root user).

    • On kubeadm control plane machine - cat /etc/kubernetes/admin.conf
    • On GitLab instance machine, create the file at ~/.kube/userns-cluster.config with the above content

  18. Setup kubectl by following https://kubernetes.io/docs/tasks/tools/install-kubectl-linux/

    curl -LO https://dl.k8s.io/release/v1.30.0/bin/linux/amd64/kubectl
sudo install -o root -g root -m 0755 kubectl /usr/local/bin/kubectl
verify - kubectl get po -A
# setup krew for kubectl - https://krew.sigs.k8s.io/docs/user-guide/setup/install/
# install ctx and ns plugins for kubectll - https://github.com/ahmetb/kubectx
kubectl krew install ctx
kubectl krew install ns

  19. Create agent project at gitlab-org/gitlab-agent-configurations

    • Add file for sysbox - .gitlab/agents/sysbox/config.yaml 
      remote_development:
  enabled: true
  dns_zone: sysbox-testing.remote-development-test.com
  allow_privilege_escalation: true
  default_runtime_class: sysbox-runc
  annotations:
    "io.kubernetes.cri-o.userns-mode": "auto:size=65536"

# for debugging locally to increase log verbosity
observability:
  logging:
    level: debug
    grpc_level: warn
    • Add file for kata - .gitlab/agents/kata/config.yaml 
      remote_development:
  enabled: true
  dns_zone: kata-testing.remote-development-test.com
  allow_privilege_escalation: true
  default_runtime_class: kata-qemu

# for debugging locally to increase log verbosity
observability:
  logging:
    level: debug
    grpc_level: warn
    • Add file for userns - .gitlab/agents/userns/config.yaml 
      remote_development:
  enabled: true
  dns_zone: userns-testing.remote-development-test.com
  allow_privilege_escalation: true
  use_kubernetes_user_namespaces: true

# for debugging locally to increase log verbosity
observability:
  logging:
    level: debug
    grpc_level: warn

  20. Run agentk' for the above configured config files

    cd ~/gdk/gitlab-development-kit/gitlab-k8s-agent

POD_NAMESPACE=default POD_NAME=remotedev AGENTK_TOKEN="" bazel run //cmd/agentk -- --kas-address=grpc://172.16.123.1:8150 --context="arn:aws:eks:ap-south-1:014073590673:cluster/eks-sysbox" --observability-listen-address=:8980 >sysbox-agent.log 2>&1 &

POD_NAMESPACE=default POD_NAME=remotedev AGENTK_TOKEN="" bazel run //cmd/agentk -- --kas-address=grpc://172.16.123.1:8150 --context="arn:aws:eks:ap-south-1:014073590673:cluster/eks-kata" --observability-listen-address=:8981 >kata-agent.log 2>&1 &

POD_NAMESPACE=default POD_NAME=remotedev AGENTK_TOKEN="" bazel run //cmd/agentk -- --kas-address=grpc://172.16.123.1:8150 --context="kubernetes-admin@test-user-ns-kubeadm" --kubeconfig="/home/ubuntu/.kube/userns-cluster.config" --observability-listen-address=:8982 >userns-agent.log 2>&1 &

  21. Authorize the 3 agents at gitlab-org group by following https://docs.gitlab.com/ee/user/workspace/gitlab_agent_configuration.html#allow-a-cluster-agent-for-workspaces-in-a-group create .devfile.yaml in project gitlab-org/gitlab-shell with the following content

  22. Create a devfile for gitlab-org/gitlab-shell project

    schemaVersion: 2.2.0
components:
  - name: tooling-container
    attributes:
      gl/inject-editor: true
    container:
      image: "registry.gitlab.com/gitlab-org/remote-development/gitlab-remote-development-docs/ubuntu:22.04"

  23. Create a workspace from the project and select the respective agent.

  24. Run the following commands in terminal of each workspace.

    sudo apt update
sudo apt install -y vim

vim

  25. For each workspace, verify all the pre-installed extensions are working as expected. Install/uninstall a new extension from the marketplace as well.

Edited by Vishal Tak

Merge request reports