Investigate SQL statement timeouts in Ci::UnlockPipelinesInQueueWorker
We've recently been seeing failed executions of Ci::UnlockPipelinesInQueueWorker
.
Looking at https://log.gprd.gitlab.net/app/r/s/JJhCU, it seems to have been caused by an SQL query timing out.
The problematic query seems to be:
SELECT "p_ci_builds"."id" FROM "p_ci_builds" WHERE "p_ci_builds"."type" = $1 AND "p_ci_builds"."commit_id" = $2 AND "p_ci_builds"."partition_id" = $3 ORDER BY "p_ci_builds"."id" ASC LIMIT $4
Looking at https://log.gprd.gitlab.net/app/r/s/mc1Qk, this seems to have started on Dec 14, and the amount of statement timeouts for that query have increased since then.
But I looked at the pipeline IDs of those that timed out and eventually, they were unlocked after re-enqueueing them.