Skip to content

Downstream pipeline UI is caching and not updating correctly

Summary

When running multiple downstream pipelines, GitLab will return fresh JSON when expanding downstream pipelines for the first time. Subsequent refreshes to the same pipeline JSON endpoint will return 304 instead of 200, resulting in the UI showing incorrect information.

Steps to reproduce

  1. Create a new project, using the following CI config. Start a new pipeline.

    ## .gitlab-ci.yml

alpha:
    stage: deploy
    trigger:
        include: downstream.yml

beta:
    stage: deploy
    trigger:
        include: downstream.yml

gamma:
    stage: deploy
    trigger:
        include: downstream.yml

delta:
    stage: deploy
    trigger:
        include: downstream.yml

epsilon:
    stage: deploy
    trigger:
        include: downstream.yml

zeta:
    stage: deploy
    trigger:
        include: downstream.yml


## downstream.yml
tahi:
    stage: build
    script:
        - sleep 10

rua:
    stage: build
    script:
        - sleep 10
    
toru:
    stage: test
    script:
        - sleep 10

wha:
    stage: test
    script:
        - sleep 10
    
rima:
    stage: deploy
    script:
        - sleep 10

onu:
    stage: deploy
    script:
        - sleep 10

  2. Open the Chrome Developer Tools and goto the Network tab. Ensure that the Disable cache checkbox is not selected.

  3. Expand the first downstream pipeline and observe that the XHR request is a 200. As jobs complete, observe that:

    • Subsequent XHR requests to refresh the expanded downstream pipeline are 304
    • As a result, as jobs change statuses, the expanded downstream pipeline UI never updates

  4. Enable the Disable cache checkbox in the Chrome Developer Tools

  5. Observe that any refreshes done now are returning 200.

See the video attached demonstrating this bug: downstream-pipeline-ui-cache-bug

Example Project

Example pipeline https://gitlab.com/anton/downstream-ui-cache-bug/pipelines/149184413

Although, this bug is best observed while a pipeline is running.

What is the current bug behavior?

Subsequent XHR requests made to refresh the pipeline status are being cached, as a result the UI never updates.

What is the expected correct behavior?

Subsequent XHR requests made to refresh the pipeline status should never be cached, so the UI has the correct and up to date information on the pipeline.

Relevant logs and/or screenshots

(Paste any relevant logs - please use code blocks (```) to format console output, logs, and code as it's tough to read otherwise.)

Output of checks

The customer had this problem on GitLab 12.9.3-ee

I was able to replicate this on GitLab 13.0.0-ee

The bug also occurs on GitLab.com

Results of GitLab environment info

Expand for output related to GitLab environment info 

(For installations with omnibus-gitlab package run and paste the output of:
`sudo gitlab-rake gitlab:env:info`)

(For installations from source run and paste the output of:
`sudo -u git -H bundle exec rake gitlab:env:info RAILS_ENV=production`)

Results of GitLab application Check

Expand for output related to the GitLab application check 
(For installations with omnibus-gitlab package run and paste the output of:
sudo gitlab-rake gitlab:check SANITIZE=true)


(For installations from source run and paste the output of:
sudo -u git -H bundle exec rake gitlab:check RAILS_ENV=production SANITIZE=true)


(we will only investigate if the tests are passing)

Possible fixes

(If you can, link to the line of code that might be responsible for the problem)

I think there is a problem with ETag cache workers/services:

Workarounds

Disable the Chrome cache https://stackoverflow.com/questions/5690269/disabling-chrome-cache-for-website-development#answer-7000899 while viewing the pipeline status page. On subsequent XHR requests, this will force Chrome to fetch the latest JSON.

Edited by Marius Bobin