Display blob metadata for found secrets in secrets push check
What does this MR do and why?
This merge request updates the secrets push check introduced over a number of earlier merge requests:
- !135032 (merged)
- !135036 (merged)
- !135164 (merged)
- !135273 (merged)
- !136896 (merged)
- !137812 (merged)
To perform secret detection scans on git
blobs using the gitlab-secret_detection
gem introduced in !136381 (merged).
In this merge request, we update the check to load up a blob metadata using GetTreeEntries()
gitaly RPC by listing tree entries for the new revisions (from newrev
in changes_access.changes
) pushed in a specific push, then cross referencing those entries with the blobs in which a secret is found, allowing us to display the file path and the commit sha for a certain secret to the user.
Note: the merge request is part of a list of related merge requests, which were created iteratively to ensure ease of review and focused scope. Therefore, it shouldn't be reviewed in isolation from the rest of those merge requests.
Resolves #427047 (closed), and built on top of !136896 (merged), !136381 (merged), and !137812 (merged).
Related Merge Requests
Step | Merge Request | Description |
---|---|---|
5 | !136896 (merged) | Updates the check to list and filter blobs of new git-push operations. |
6 | !137812 (merged) | Invokes gitlab-secret_detection gem to scan blobs filtered. |
7 | This one. | Updates the check to add details of secrets detected (e.g. file path/commit sha). |
8 | !138831 (merged) | Updates the check to introduce a bypass mechanism via commit special flag. |
Screenshots or screen recordings
Before | After |
---|---|
How to set up and validate locally
- Create a new project or use an existing one.
- In your
rails
console, execute the following commands:
pry(main)> Gitlab::CurrentSettings.update!(pre_receive_secret_detection_enabled: true) # to ensure feature is enabled instance-wide
pry(main)> project = Project.find(PROJECT_ID)
pry(main)> Feature.enable(:pre_receive_secret_detection_push_check, project)
- Ensure your GDK is licensed as
ultimate
. - In your terminal, navigate to the project folder.
- Create a new file, e.g.
.env
, and add a gitlab personal access token:
TOKEN=glpat-JUST20LETTERSANDNUMB
- Run
git add .
andgit commit -m 'test'
to commit the file. - Run the command to push the commited file
git push
. - Verify the push fails due to the secret detected and it shows the correct file path and commit sha.
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.