Link fast-forward MRs to deployment (!145211) · Merge requests · GitLab.org / GitLab

Pam Artiaga requested to merge 384104-ff-merge-in-api into master Feb 20, 2024

What does this MR do and why?

Problem

GitLab tracks newly included merge requests in a deployment. Merge requests are linked to a deployment through the Deployments::LinkMergeRequestsService. These Deployment-linked MRs can then be fetched through the Deployments API -> List of merge requests associated with a deployment.

If a Project's merge method is Fast-forward merge, merge requests are not linked to a deployment. This discrepancy is due to how Deployments::LinkMergeRequestsService gathers Merge Requests for linking to a Deployment:

The commits between the "current deployment" and "previous deployment" are fetched (see code)
There is a query on the merge requests with "merge_commit IN commits" from step 1 (see code)
Link all merge requests from step 2 to the current deployment (see code)

The problem is that fast-forward merge requests do not have a merge_commit.

Resolution introduced in this MR

This MR introduces a change in step 2 above so that the query is for merge requests with "merge_commit IN commits" OR "merge_request -> diff -> HEAD IN commits".

Possible performance concerns and de-risking

This makes the query a little more complex, from a simple query on the merge_requests record to a UNION query on the merge_requests and merge_request_diffs record. This is not a major concern because this is a service that is purposely run in its own asynchronous worker.

However, this change is introduced behind a Feature Flag (link_fast_forward_merge_requests_to_deployment) for further de-risking.

MR acceptance checklist

Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Screenshots or screen recordings

N/A - this is purely backend change tested through a GET API. See test steps below.

How to set up and validate locally

Setup

Create a project
Set the project's merge method to Fast-forward merge
1. Navigate to Settings -> Merge Requests
2. Select Fast-forward merge, and save
Set up a build pipeline for the project with a deploy job. The deploy job must target production or staging and only run in one branch (e.g.: main). See example .gitlab-ci.yml configuration below:
```
deploy:
  stage: deploy
  script: echo "deploying..."
  environment:
    name: staging
    action: start
  only:
    refs:
      - main
```

Test with Feature Flag disabled

Make sure the link_fast_forward_merge_requests_to_deployment FF is disabled
```
Feature.disable(:link_fast_forward_merge_requests_to_deployment)
```
Create a merge request with commits and merge it
Wait for the deploy job to finish
Fetch the associated merge requests of the current deployment through the API (see "Fetching deployment merge requests" section below)
Verify that the result is empty

Test with Feature Flag enabled

Enable the link_fast_forward_merge_requests_to_deployment FF

Feature.enable(:link_fast_forward_merge_requests_to_deployment)

Create a merge request with commits and merge it
Wait for the deploy job to finish
Fetch the associated merge requests of the current deployment through the API (see "Fetching deployment merge requests" section below)
Verify that the merge request your created is in the result

Fetching deployment merge requests

Navigate to your project's Environment's page (Operate -> Environments)
Verify that there is an active Environment in the list. In this example, it should be staging
Expand the staging environment accordion
Check the iid of the latest deployment

Get the deployment.id from the deployment.iid and the project.id. In the rails console:

deployment = Deployment.where(iid: <iid-from-step-4>, project_id: <id-of-your-project>)

Fetch the Deployment Merge Requests through the API

curl -k -X GET \
--header "Authorization: Bearer $PERSONAL_ACCESS_TOKEN" \
"https://gdk.test:3443/api/v4/projects/<id-of-your-project-id>/deployments/<deployment-id>/merge_requests" \
| json_pp -json_opt pretty,canonical

Note: you can generate your PERSONAL_ACCESS_TOKEN in your profile settings

Query Plans

There is only one statement to query merge_requests and insert them into the deployment_merge_requests records. The structure of the statement goes:

INSERT INTO deployment_merge_requests (merge_request_id, deployment_id, environment_id)
SELECT "merge_requests"."id", 2 as deployment_id, 2 as environment_id 
FROM (<query here>) /* this is the part of the statement affected by the change */
ON CONFLICT DO NOTHING

Query before the change

SELECT "merge_requests"."id", 2 as deployment_id, 2 as environment_id 
FROM "merge_requests" 
WHERE "merge_requests"."target_project_id" = 2 AND "merge_requests"."state_id" = 3 
AND "merge_requests"."merge_commit_sha" IN ('c1c67abbaf91f624347bb3ae96eabe3a1b742478', '1e292f8fedd741b75372e19097c76d327140c312', '2d1db523e11e777e49377cfb22d368deec3f0793', 'ddd0f15ae83993f5cb66a927a28673882e99100b')

Query after the change

With INNER JOIN

INSERT INTO deployment_merge_requests (merge_request_id, deployment_id, environment_id)
SELECT "merge_requests"."id", 39 as deployment_id, 37 as environment_id 
FROM (
  (
    SELECT "merge_requests".* FROM "merge_requests" 
    WHERE "merge_requests"."target_project_id" = 278 AND "merge_requests"."state_id" = 3 
    AND "merge_requests"."merge_commit_sha" IN ('c1c67abbaf91f624347bb3ae96eabe3a1b742478', '1e292f8fedd741b75372e19097c76d327140c312', '2d1db523e11e777e49377cfb22d368deec3f0793', 'ddd0f15ae83993f5cb66a927a28673882e99100b')
  )
  UNION
  (
    SELECT "merge_requests".* FROM "merge_requests" 
    INNER JOIN "merge_request_diffs" ON "merge_request_diffs"."id" = "merge_requests"."latest_merge_request_diff_id" 
    WHERE "merge_requests"."target_project_id" = 278 AND "merge_requests"."state_id" = 3 AND 
    "merge_request_diffs"."head_commit_sha" IN ('c1c67abbaf91f624347bb3ae96eabe3a1b742478', '1e292f8fedd741b75372e19097c76d327140c312', '2d1db523e11e777e49377cfb22d368deec3f0793', 'ddd0f15ae83993f5cb66a927a28673882e99100b')
  )
) merge_requests WHERE "merge_requests"."target_project_id" = 278 AND "merge_requests"."state_id" = 3
ON CONFLICT DO NOTHING

With EXISTS/WHERE - this is no longer used (see !145211 (comment 1784886442))

SELECT "merge_requests"."id", 1 as deployment_id, 1 as environment_id 
FROM (
  (
    SELECT "merge_requests".* FROM "merge_requests" WHERE "merge_requests"."target_project_id" = 1 AND "merge_requests"."state_id" = 3 
    AND "merge_requests"."merge_commit_sha" IN ('c1c67abbaf91f624347bb3ae96eabe3a1b742478', '1e292f8fedd741b75372e19097c76d327140c312', '2d1db523e11e777e49377cfb22d368deec3f0793', 'ddd0f15ae83993f5cb66a927a28673882e99100b')
  )
  UNION
  (
    SELECT "merge_requests".* FROM "merge_requests" WHERE "merge_requests"."target_project_id" = 1 AND "merge_requests"."state_id" = 3 
    AND (
      EXISTS (
        SELECT 1 FROM "merge_request_diffs" 
        WHERE (merge_requests.latest_merge_request_diff_id = merge_request_diffs.id) 
        AND "merge_request_diffs"."head_commit_sha" IN ('c1c67abbaf91f624347bb3ae96eabe3a1b742478', '1e292f8fedd741b75372e19097c76d327140c312', '2d1db523e11e777e49377cfb22d368deec3f0793', 'ddd0f15ae83993f5cb66a927a28673882e99100b')
      )
    )
  )
) merge_requests 
WHERE "merge_requests"."target_project_id" = 1 AND "merge_requests"."state_id" = 3

Database Labs EXPLAIN links

Query plan for before the change / FF disabled
Query plan for after the change / FF enabled
- with INNER JOIN
- with EXISTS/WHERE - this is no longer used (see !145211 (comment 1784886442))

Edited Feb 23, 2024 by Pam Artiaga

Link fast-forward MRs to deployment