report container configuration errors in the kubernetes executor
Description
When KUBERNETES_POD_SECURITY_CONTEXT_RUN_AS_NON_ROOT=true
is set for the kubernetes executor, jobs that attempt to run pods as root
will timeout and fail.
Job log:
Running with gitlab-runner 12.1.0 (de7731dd)
on gitlab-runner-gitlab-runner-58c76458c4-5hdjf 8NZcgmbi
Using Kubernetes namespace: gitlab-runner
Using Kubernetes executor with image ${RELEASE_IMAGE}:${RELEASE_IMAGE_TAG} ...
Waiting for pod gitlab-runner/runner-8nzcgmbi-project-2-concurrent-0xtnld to be running, status is Pending
Waiting for pod gitlab-runner/runner-8nzcgmbi-project-2-concurrent-0xtnld to be running, status is Pending
...
ERROR: Job failed (system failure): timed out waiting for pod to start
Pod description:
...
State: Waiting
Reason: CreateContainerConfigError
Ready: False
...
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 3m58s default-scheduler Successfully assigned gitlab-runner/runner-8nzcgmbi-project-2-concurrent-0xtnld to k8s04b.test.jt.dev.example.com
Normal Pulled 3m27s (x4 over 3m55s) kubelet, k8s04b.test.jt.dev.example.com Successfully pulled image "docker.example.com/example/gitlab-ci-helm-release:2019.0.0"
Warning Failed 3m27s (x4 over 3m55s) kubelet, k8s04b.test.jt.dev.example.com Error: container has runAsNonRoot and image will run as root
Normal Pulling 3m27s (x4 over 3m55s) kubelet, k8s04b.test.jt.dev.example.com Pulling image "docker.example.com/gitlab/gitlab-runner-helper:x86_64-de7731dd"
Normal Pulled 3m27s (x4 over 3m55s) kubelet, k8s04b.test.jt.dev.example.com Successfully pulled image "docker.example.com/gitlab/gitlab-runner-helper:x86_64-de7731dd"
Warning Failed 3m27s (x4 over 3m55s) kubelet, k8s04b.test.jt.dev.example.com Error: container has runAsNonRoot and image will run as root
Normal Pulling 3m15s (x5 over 3m57s) kubelet, k8s04b.test.jt.dev.example.com Pulling image "docker.example.com/example/gitlab-ci-helm-release:2019.0.0"
Proposal
It would be a UX improvement to show the pod error message in the job log.
There is minor risk for information leakage about the k8s cluster from the error messages, but I think if the messages are filtered to only include "Error:" strings, that risk is minimal (that statement made without fully knowing what k8s can include in those logs).
Edited by Aron Parsons