Skip to content

Allow user to specify multiple pull policies for Docker executor

Pedro Pombeiro requested to merge 26558-multiple-docker-pull-policies into master

What does this MR do?

Tries to solve the problem specified in #26558 (closed) in a different way. Instead of introducing a new pull policy, updates the pull_policy config to support multiple values, for example pull_policy = ["always", "if-not-present"]. It makes parsing of pull_policy accept an array but also keep backward compatibility.

Benefits

  • Implement it for Kubernetes executor: Previously this pull policy was only for Docker/Docker+machine executor, with this we can also implement it for the Kubernetes executor. We can see if the error on the pod is about pulling the image. If it is, we just recreate the pod but this time with the if-not-present policy.
  • Simpler to implement: We can just check the error of the pull, if there is an error we log it and move onto the next implementation.
  • Not introducing a specific policy for the runner: Not custom/new policy for the runner that other tools and the industry doesn't use.
  • Not introducing a new policy with the same security flaws: This is the biggest concern we had when discussing this, there is still the security flaw because we have if-not-present as fallback but at least that is already well documented, and most users are educated around it.
  • Stackable: We can stack multiple policies on top of each other, and this might give even more control to what the user might want.
  • Less risky to change: We don't have to touch or change any of the existing logic, we just have to wrap it and check if there is an error or not, if there is move onto the next one in line.

Drawbacks

  • We can gradually change this as we did for docker services where we support both syntax for a while and then deprecate it (if we want to remove it). This was done in 52f009a9.

Why was this MR needed?

Allow users to keep using CI even when the registry is down. It keeps the images up to date if the registry is available with always and if pulling fails, it uses the if-not-present policy.

What's the best way to test this MR?

  1. docker pull alpine:3.12

  2. docker tag alpine:3.12 localonly/alpine:3.12

  3. Have the following config.toml

    config.toml
    concurrent = 1
    check_interval = 0
    
    [session_server]
      session_timeout = 1800
    
    [[runners]]
      name = "steve-mbp-gitlab.local"
      url = "https://gitlab.com/"
      token = "5RYH_2sGcGgbmxpq799-"
      executor = "docker"
      [runners.docker]
        tls_verify = false
        image = "localonly/alpine:3.12"
        privileged = false
        disable_entrypoint_overwrite = false
        oom_kill_disable = false
        disable_cache = false
        volumes = ["/cache"]
        pull_policy = ["always", "if-not-present"]
        shm_size = 0
  4. Run job, for example https://gitlab.com/steveazz/playground/-/jobs/872091301

What are the relevant issue numbers?

Closes #26558 (closed)

Edited by Steve Xuereb

Merge request reports

Loading