`queued_duration` for canceled jobs keeps increasing
Summary
Canceled jobs have their queued_duration
being displayed as a non-null value and increasing on each request.
Steps to reproduce
- create a MR with a pipeline config triggering pipelines for every commit in the MR
- pipeline was created
- the first job was not picked up immediately but became queued (in UI it's "Pending" state with yellow pause symbol) -- this is mandatory for the bug to be reproduced. The job has to be in pending state long enough for me to open it in the UI
- manually cancel the job, it moves to Cancelled status without being run
- Take the job ID and query it via API - it should return like this:
...
"started_at": null,
"duration": null,
"queued_duration": 2430.001029147,
...
Example job that was canceled:
API output:
faleksic@faleksic-m1 (10:58:37)
~/Desktop $ curl -XGET https://gitlab.com/api/v4/projects/$PROJECT_ID/jobs/$JOB_ID -H "PRIVATE-TOKEN: $PRIVATE_TOKEN" -sS | jq .queued_duration,.status
2260.146803837
"canceled"
faleksic@faleksic-m1 (10:58:39)
~/Desktop $ curl -XGET https://gitlab.com/api/v4/projects/$PROJECT_ID/jobs/$JOB_ID -H "PRIVATE-TOKEN: $PRIVATE_TOKEN" -sS | jq .queued_duration,.status
2262.60396861
"canceled"
Example Project
What is the current bug behavior?
The queued_duration
is not null
, since canceled jobs cannot be restarted this value should be null.
What is the expected correct behavior?
The queued_duration
being null
. Since the job cannot be restarted this field should be set to 0 instead of the value increasing.
Relevant logs and/or screenshots
Output of checks
This bug happens on GitLab.com
Results of GitLab environment info
Expand for output related to GitLab environment info
(For installations with omnibus-gitlab package run and paste the output of: `sudo gitlab-rake gitlab:env:info`) (For installations from source run and paste the output of: `sudo -u git -H bundle exec rake gitlab:env:info RAILS_ENV=production`)
Results of GitLab application Check
Expand for output related to the GitLab application check
(For installations with omnibus-gitlab package run and paste the output of:
sudo gitlab-rake gitlab:check SANITIZE=true
)(For installations from source run and paste the output of:
sudo -u git -H bundle exec rake gitlab:check RAILS_ENV=production SANITIZE=true
)(we will only investigate if the tests are passing)
Possible fixes
Fallback on using finished_at
when started_at
is missing:
Implementation Plan
# app/models/commit_status.rb
def queued_duration
calculate_duration(queued_at, started_at || finished_at)
end
# app/models/ci/pipeline.rb
def queued_duration
queueing_finished_time = started_at || finished_at
return unless queueing_finished_time
return unless created_at
seconds = (queueing_finished_time - created_at).to_i
seconds unless seconds == 0
end