Link fast-forward MRs to deployment
What does this MR do and why?
Related to #384104 (closed)
Problem
GitLab tracks newly included merge requests in a deployment. Merge requests are linked to a deployment through the Deployments::LinkMergeRequestsService
. These Deployment-linked MRs can then be fetched through the Deployments API -> List of merge requests associated with a deployment.
If a Project's merge method is Fast-forward merge, merge requests are not linked to a deployment. This discrepancy is due to how Deployments::LinkMergeRequestsService
gathers Merge Requests for linking to a Deployment:
- The
commits
between the "current deployment" and "previous deployment" are fetched (see code) - There is a query on the merge requests with "
merge_commit
INcommits
" from step 1 (see code) - Link all merge requests from step 2 to the current deployment (see code)
The problem is that fast-forward merge requests do not have a merge_commit
.
Resolution introduced in this MR
This MR introduces a change in step 2 above so that the query is for merge requests with "merge_commit
IN commits
" OR "merge_request -> diff -> HEAD
IN commits
".
Possible performance concerns and de-risking
This makes the query a little more complex, from a simple query on the merge_requests
record to a UNION query on the merge_requests
and merge_request_diffs
record. This is not a major concern because this is a service that is purposely run in its own asynchronous worker.
However, this change is introduced behind a Feature Flag (link_fast_forward_merge_requests_to_deployment
) for further de-risking.
MR acceptance checklist
Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.
Screenshots or screen recordings
N/A - this is purely backend change tested through a GET API. See test steps below.
How to set up and validate locally
Setup
-
Create a project
-
Set the project's merge method to Fast-forward merge
- Navigate to Settings -> Merge Requests
- Select Fast-forward merge, and save
-
Set up a build pipeline for the project with a deploy job. The deploy job must target
production
orstaging
and only run in one branch (e.g.:main
). See example.gitlab-ci.yml
configuration below:deploy: stage: deploy script: echo "deploying..." environment: name: staging action: start only: refs: - main
Test with Feature Flag disabled
-
Make sure the
link_fast_forward_merge_requests_to_deployment
FF is disabledFeature.disable(:link_fast_forward_merge_requests_to_deployment)
-
Create a merge request with commits and merge it
-
Wait for the deploy job to finish
-
Fetch the associated merge requests of the current deployment through the API (see "Fetching deployment merge requests" section below)
-
Verify that the result is empty
Test with Feature Flag enabled
-
Enable the
link_fast_forward_merge_requests_to_deployment
FFFeature.enable(:link_fast_forward_merge_requests_to_deployment)
-
Create a merge request with commits and merge it
-
Wait for the deploy job to finish
-
Fetch the associated merge requests of the current deployment through the API (see "Fetching deployment merge requests" section below)
-
Verify that the merge request your created is in the result
Fetching deployment merge requests
-
Navigate to your project's Environment's page (Operate -> Environments)
-
Verify that there is an active Environment in the list. In this example, it should be
staging
-
Expand the
staging
environment accordion -
Check the
iid
of the latest deployment -
Get the
deployment.id
from thedeployment.iid
and theproject.id
. In the rails console:deployment = Deployment.where(iid: <iid-from-step-4>, project_id: <id-of-your-project>)
-
Fetch the Deployment Merge Requests through the API
curl -k -X GET \ --header "Authorization: Bearer $PERSONAL_ACCESS_TOKEN" \ "https://gdk.test:3443/api/v4/projects/<id-of-your-project-id>/deployments/<deployment-id>/merge_requests" \ | json_pp -json_opt pretty,canonical
Note: you can generate your
PERSONAL_ACCESS_TOKEN
in your profile settings
Query Plans
There is only one statement to query merge_requests
and insert them into the deployment_merge_requests
records. The structure of the statement goes:
INSERT INTO deployment_merge_requests (merge_request_id, deployment_id, environment_id)
SELECT "merge_requests"."id", 2 as deployment_id, 2 as environment_id
FROM (<query here>) /* this is the part of the statement affected by the change */
ON CONFLICT DO NOTHING
Query before the change
SELECT "merge_requests"."id", 2 as deployment_id, 2 as environment_id
FROM "merge_requests"
WHERE "merge_requests"."target_project_id" = 2 AND "merge_requests"."state_id" = 3
AND "merge_requests"."merge_commit_sha" IN ('c1c67abbaf91f624347bb3ae96eabe3a1b742478', '1e292f8fedd741b75372e19097c76d327140c312', '2d1db523e11e777e49377cfb22d368deec3f0793', 'ddd0f15ae83993f5cb66a927a28673882e99100b')
Query after the change
With INNER JOIN
INSERT INTO deployment_merge_requests (merge_request_id, deployment_id, environment_id)
SELECT "merge_requests"."id", 39 as deployment_id, 37 as environment_id
FROM (
(
SELECT "merge_requests".* FROM "merge_requests"
WHERE "merge_requests"."target_project_id" = 278 AND "merge_requests"."state_id" = 3
AND "merge_requests"."merge_commit_sha" IN ('c1c67abbaf91f624347bb3ae96eabe3a1b742478', '1e292f8fedd741b75372e19097c76d327140c312', '2d1db523e11e777e49377cfb22d368deec3f0793', 'ddd0f15ae83993f5cb66a927a28673882e99100b')
)
UNION
(
SELECT "merge_requests".* FROM "merge_requests"
INNER JOIN "merge_request_diffs" ON "merge_request_diffs"."id" = "merge_requests"."latest_merge_request_diff_id"
WHERE "merge_requests"."target_project_id" = 278 AND "merge_requests"."state_id" = 3 AND
"merge_request_diffs"."head_commit_sha" IN ('c1c67abbaf91f624347bb3ae96eabe3a1b742478', '1e292f8fedd741b75372e19097c76d327140c312', '2d1db523e11e777e49377cfb22d368deec3f0793', 'ddd0f15ae83993f5cb66a927a28673882e99100b')
)
) merge_requests WHERE "merge_requests"."target_project_id" = 278 AND "merge_requests"."state_id" = 3
ON CONFLICT DO NOTHING
With EXISTS
/WHERE
- this is no longer used (see !145211 (comment 1784886442))
SELECT "merge_requests"."id", 1 as deployment_id, 1 as environment_id
FROM (
(
SELECT "merge_requests".* FROM "merge_requests" WHERE "merge_requests"."target_project_id" = 1 AND "merge_requests"."state_id" = 3
AND "merge_requests"."merge_commit_sha" IN ('c1c67abbaf91f624347bb3ae96eabe3a1b742478', '1e292f8fedd741b75372e19097c76d327140c312', '2d1db523e11e777e49377cfb22d368deec3f0793', 'ddd0f15ae83993f5cb66a927a28673882e99100b')
)
UNION
(
SELECT "merge_requests".* FROM "merge_requests" WHERE "merge_requests"."target_project_id" = 1 AND "merge_requests"."state_id" = 3
AND (
EXISTS (
SELECT 1 FROM "merge_request_diffs"
WHERE (merge_requests.latest_merge_request_diff_id = merge_request_diffs.id)
AND "merge_request_diffs"."head_commit_sha" IN ('c1c67abbaf91f624347bb3ae96eabe3a1b742478', '1e292f8fedd741b75372e19097c76d327140c312', '2d1db523e11e777e49377cfb22d368deec3f0793', 'ddd0f15ae83993f5cb66a927a28673882e99100b')
)
)
)
) merge_requests
WHERE "merge_requests"."target_project_id" = 1 AND "merge_requests"."state_id" = 3
Database Labs EXPLAIN links
- Query plan for before the change / FF disabled
- Query plan for after the change / FF enabled
- with INNER JOIN
- with EXISTS/WHERE - this is no longer used (see !145211 (comment 1784886442))