Receptive agent support Part 1 - using JWT & no TLS

Part of Closes #60 (closed).

Support receptive agents with JWT authentication and no TLS. TLS support will be added in the different MR.

This includes:

• Support new command line flags. Add new configurations under values.config for each of:
  • api-listen-network
  • api-listen-address
  • api-jwt-file
  • private-api-listen-network
  • private-api-listen-address
  • private-api-jwt-file
• Support new env vars.
  • POD_SELECTOR_LABELS: Construct it from agent pod labels.
  • OWN_PRIVATE_API_URL: Construct it from configurations.
• Add Service resource.
• Modify agent pod's ports.

Not included: TLS support for API & Private API, Ingress, cert manager, cert manager issuer.

How to verify

• Start GDK with the latest commit.
• Check Enable receptive mode and Save changes in Admin => Setting => General => open GitLab Agent for Kubernetes.
• Get your project id and agent id.
• Create URL configuration:

curl --header "Private-Token: <your_access_token>" \
"https://gdk.test:3443/api/v4/projects/<project id>/cluster_agents/<agent id>/url_configurations" \
-H "Content-Type:application/json" \
-X POST \
--data '{"url":"grpc://127.0.0.1:8182"}'

• Note your public_key from the response above.

$ helm install my-release . --debug \
--set config.token=<YOUR_TOKEN> \
--set image.tag=v17.4.0-rc1-debug \
--set config.receptive.enabled=true \
--set config.api.jwtPublicKey=<PUBLIC_KEY_RECEIVED_IN_THE_PREVIOUS_STEP>

• (If you test locally): Port forward 8182 (= API external port) so that you can test it locally.

$ kubectl port-forward service/<YOUR_RELEASE_NAME>-gitlab-agent-service 8182:8182

• Confirm the agents are connected to KAS:

Logs from KAS

{"time":"2024-08-30T16:43:28.306954+09:00","level":"INFO","msg":"Starting worker","mod_name":"kas2agentk_tunnel","agent_id":5}
{"time":"2024-08-30T16:43:31.184613+09:00","level":"INFO","msg":"Registering agent","agent_id":5,"agent_version":"v17.4.0-rc1","expires":"2024-08-30T07:58:31.18461Z","pod_name":"my-release-gitlab-agent-v2-5b8648f9d6-5tmdn","pod_namespace":"default"}
{"time":"2024-08-30T16:43:31.187361+09:00","level":"DEBUG","msg":"Handled a connection successfully","mod_name":"kas2agentk_tunnel","agent_id":5}
{"time":"2024-08-30T16:43:31.30875+09:00","level":"INFO","msg":"Config: new commit","grpc_service":"gitlab.agent.agent_configuration.rpc.AgentConfiguration","grpc_method":"GetConfiguration","agent_id":5,"project_id":"sandbox/simple-rails","commit_id":"9b0ce59973ab7382e5fa3c519238bee4b3419624"}

Logs from Agents

$ k logs pod/my-release-gitlab-agent-v2-5b8648f9d6-5tmdn
{"time":"2024-08-30T07:43:18.963690082Z","level":"INFO","msg":"Using own private API URL","url":"grpc://10.42.0.138:8081"}
{"time":"2024-08-30T07:43:18.963827082Z","level":"INFO","msg":"Using Pod label selector to find agentk Pods within the namespace","pod_namespace":"default","label_selector":"app.kubernetes.io/instance=my-release,app.kubernetes.io/managed-by=Helm,app.kubernetes.io/name=gitlab-agent,app.kubernetes.io/version=v17.3.1,helm.sh/chart=gitlab-agent-2.6.2"}
{"time":"2024-08-30T07:43:18.965785124Z","level":"INFO","msg":"Flux is not installed, skipping module. A restart is required for this to be checked again","mod_name":"flux"}
{"time":"2024-08-30T07:43:18.970748374Z","level":"INFO","msg":"Private API endpoint is up","net_network":"tcp","net_address":"[::]:8081"}
{"time":"2024-08-30T07:43:18.970771666Z","level":"INFO","msg":"API endpoint is up","net_network":"tcp","net_address":"[::]:8082"}
{"time":"2024-08-30T07:43:18.970823082Z","level":"INFO","msg":"Observability endpoint is up","mod_name":"observability","net_network":"tcp","net_address":"[::]:8080"}
{"time":"2024-08-30T07:43:30.687659463Z","level":"INFO","msg":"attempting to acquire leader lease default/agent-5-lock...","agent_id":5}
{"time":"2024-08-30T07:43:47.916606846Z","level":"INFO","msg":"successfully acquired lease default/agent-5-lock","agent_id":5}
{"time":"2024-08-30T07:43:47.918248471Z","level":"INFO","msg":"Event occurred","agent_id":5,"object":{"name":"agent-5-lock","namespace":"default"},"fieldPath":"","kind":"Lease","apiVersion":"coordination.k8s.io/v1","type":"Normal","reason":"LeaderElection","message":"my-release-gitlab-agent-v2-5b8648f9d6-5tmdn became leader"}
{"time":"2024-08-30T07:43:47.921152263Z","level":"DEBUG","msg":"Trying tunnel","agent_id":5,"gateway_url":"grpc://10.42.0.138:8081"}
{"time":"2024-08-30T07:43:48.000682596Z","level":"DEBUG","msg":"ContainerScanning config is empty, security policies are disabled","mod_name":"starboard_vulnerability","agent_id":5}