Security target issue processor
What does this MR do and why?
This MR adds a service ReleaseTools::Security::TargetIssuesProcessor that gets all security implementation issues labeled with security-target and checks if they are ready for processing or not. It takes one of the four following actions:
- If the issue is linked and ready for processing - the service does nothing
- If the issue is unlinked and not ready for processing - the service does nothing
- If the issue is unlinked and ready for processing - the issue is linked.
- If the issue is linked and not ready for processing - the issue is unlinked.
This issue only links and unlinks the issues, logging which specific action is taken. Future issues and MRs will add additional logic to communicate the changes to issue assignees and also communicate reasons why issues are not ready for processing to issue assignees.
Related to gitlab-com/gl-infra/delivery#19568 (closed)
Testing
To test this I created 4 mock security implementation issues:
- https://gitlab.com/gitlab-org/security/gitlab/-/issues/967
- https://gitlab.com/gitlab-org/security/gitlab/-/issues/969
- https://gitlab.com/gitlab-org/security/gitlab/-/issues/970
- https://gitlab.com/gitlab-org/security/gitlab/-/issues/971
And one mock security release tracking issue:
I linked issues 967 and 969 to the tracking issue:
Next I modified my local code so that the processor would use the fake tracking issue and also consider issues 967 and 970 as being ready to be processed:
Click to expand Git Diff
diff --git a/lib/release_tools/security/implementation_issue.rb b/lib/release_tools/security/implementation_issue.rb
index 6a73ab47..840d64c7 100644
--- a/lib/release_tools/security/implementation_issue.rb
+++ b/lib/release_tools/security/implementation_issue.rb
@@ -51,17 +51,22 @@ module ReleaseTools
end
def ready_to_be_processed?
- if missing_backports?
- reject('missing merge requests')
- elsif !merge_requests_with_allowed_status?
- reject('invalid merge requests status')
- elsif !merge_requests_assigned_to_the_bot?
- reject('unassigned merge requests')
- elsif !valid_merge_requests?
- reject('invalid merge requests')
- else
+ if iid == 967 || iid == 970
true
+ else
+ false
end
+ # if missing_backports?
+ # reject('missing merge requests')
+ # elsif !merge_requests_with_allowed_status?
+ # reject('invalid merge requests status')
+ # elsif !merge_requests_assigned_to_the_bot?
+ # reject('unassigned merge requests')
+ # elsif !valid_merge_requests?
+ # reject('invalid merge requests')
+ # else
+ # true
+ # end
end
def reject(reason)
diff --git a/lib/release_tools/security/issue_crawler.rb b/lib/release_tools/security/issue_crawler.rb
index e96e7369..a528a2d6 100644
--- a/lib/release_tools/security/issue_crawler.rb
+++ b/lib/release_tools/security/issue_crawler.rb
@@ -12,7 +12,7 @@ module ReleaseTools
SECURITY_NAMESPACE = 'gitlab-org/security'
# The label required by security root/meta issues.
- ROOT_ISSUE_LABEL = 'upcoming security release'
+ ROOT_ISSUE_LABEL = 'security-target'
# The label required for a related issue or merge request to be considered
# by the crawler.
@@ -95,7 +95,7 @@ module ReleaseTools
# Returns issues that are related to the security release tracking issue.
def related_security_issues
- security_issues_for(release_issue.iid)
+ security_issues_for(424529)
end
# Returns issues on the security mirrors for projects under managed versioning thatSo this shows the state before I ran the processor and the expected state for after the processor has run:
| Issue | Ready for processing? | Linked before? | Linked after? |
|---|---|---|---|
| https://gitlab.com/gitlab-org/security/gitlab/-/issues/967 | True | Yes | Yes |
| https://gitlab.com/gitlab-org/security/gitlab/-/issues/969 | False | Yes | No |
| https://gitlab.com/gitlab-org/security/gitlab/-/issues/970 | True | No | Yes |
| https://gitlab.com/gitlab-org/security/gitlab/-/issues/971 | False | No | No |
I ran the processor in my local console, note the logger output shows that each issue was processed as expected:
[1] pry(main)> ReleaseTools::Security::TargetIssuesProcessor.new.execute
2023-09-07 14:46:16.591810 D [dry-run] ReleaseTools::GitlabClient -- [HTTParty] [2023-09-07 14:46:16 -0600] 200 "GET https://gitlab.com/api/v4/projects/gitlab-org%2Fsecurity%2Fcharts%2Fcomponents%2Fimages/issues" 2
2023-09-07 14:46:16.851985 D [dry-run] ReleaseTools::GitlabClient -- [HTTParty] [2023-09-07 14:46:16 -0600] 200 "GET https://gitlab.com/api/v4/projects/gitlab-org%2Fsecurity%2Fgitaly/issues" 2
2023-09-07 14:46:17.355752 D [dry-run] ReleaseTools::GitlabClient -- [HTTParty] [2023-09-07 14:46:17 -0600] 200 "GET https://gitlab.com/api/v4/projects/gitlab-org%2Fsecurity%2Fgitlab/issues" -
2023-09-07 14:46:17.652345 D [dry-run] ReleaseTools::GitlabClient -- [HTTParty] [2023-09-07 14:46:17 -0600] 200 "GET https://gitlab.com/api/v4/projects/gitlab-org%2Fsecurity%2Fgitlab-pages/issues" 2
2023-09-07 14:46:17.926364 D [dry-run] ReleaseTools::GitlabClient -- [HTTParty] [2023-09-07 14:46:17 -0600] 200 "GET https://gitlab.com/api/v4/projects/gitlab-org%2Fsecurity%2Fomnibus-gitlab/issues" 2
2023-09-07 14:46:18.208054 D ReleaseTools::GitlabClient -- [HTTParty] [2023-09-07 14:46:18 -0600] 200 "GET https://gitlab.com/api/v4/projects/15642544/issues/971/related_merge_requests" 2
2023-09-07 14:46:18.222682 D ReleaseTools::GitlabClient -- [HTTParty] [2023-09-07 14:46:18 -0600] 200 "GET https://gitlab.com/api/v4/projects/15642544/issues/969/related_merge_requests" 2
2023-09-07 14:46:18.233196 D ReleaseTools::GitlabClient -- [HTTParty] [2023-09-07 14:46:18 -0600] 200 "GET https://gitlab.com/api/v4/projects/15642544/issues/970/related_merge_requests" 2
2023-09-07 14:46:18.451376 D ReleaseTools::GitlabClient -- [HTTParty] [2023-09-07 14:46:18 -0600] 200 "GET https://gitlab.com/api/v4/projects/15642544/issues/967/related_merge_requests" -
2023-09-07 14:46:18.817451 D [dry-run] ReleaseTools::GitlabClient -- [HTTParty] [2023-09-07 14:46:18 -0600] 200 "GET https://gitlab.com/api/v4/projects/gitlab-org%2Fgitlab/issues" -
2023-09-07 14:46:18.821968 W [dry-run] Ruby -- WARNING: Please convert ObjectifiedHash object to hash before calling Hash methods on it.
-- {:source=>"ruby_warnings", :stacktrace=>"/Users/steveabrams/workspace/gitlab-org/release-tools/lib/release_tools/issuable.rb:6:in `initialize'"}
2023-09-07 14:46:18.823106 I [dry-run] ReleaseTools::Security::TargetIssuesProcessor -- 4 target issues found. They will be evaluated and considered for linking to the security release tracking issue: https://gitlab.com/gitlab-org/gitlab/-/issues/424529.
2023-09-07 14:46:19.157518 D [dry-run] ReleaseTools::GitlabClient -- [HTTParty] [2023-09-07 14:46:19 -0600] 200 "GET https://gitlab.com/api/v4/projects/gitlab-org%2Fgitlab/issues/424529/links" -
2023-09-07 14:46:19.412757 D ReleaseTools::GitlabClient -- [HTTParty] [2023-09-07 14:46:19 -0600] 200 "GET https://gitlab.com/api/v4/projects/15642544/issues/969/related_merge_requests" 2
2023-09-07 14:46:19.992219 D ReleaseTools::GitlabClient -- [HTTParty] [2023-09-07 14:46:19 -0600] 200 "GET https://gitlab.com/api/v4/projects/15642544/issues/967/related_merge_requests" -
2023-09-07 14:46:19.993632 I [dry-run] ReleaseTools::Security::TargetIssuesProcessor -- https://gitlab.com/gitlab-org/security/gitlab/-/issues/971 is not ready to be processed or linked to the security release tracking issue.
2023-09-07 14:46:20.361373 D [dry-run] ReleaseTools::GitlabClient -- [HTTParty] [2023-09-07 14:46:20 -0600] 200 "GET https://gitlab.com/api/v4/projects/gitlab-org%2Fgitlab/issues/424529/links" -
2023-09-07 14:46:20.748204 D ReleaseTools::GitlabClient -- [HTTParty] [2023-09-07 14:46:20 -0600] 200 "GET https://gitlab.com/api/v4/projects/15642544/issues/969/related_merge_requests" 2
2023-09-07 14:46:21.052021 D ReleaseTools::GitlabClient -- [HTTParty] [2023-09-07 14:46:21 -0600] 200 "GET https://gitlab.com/api/v4/projects/15642544/issues/967/related_merge_requests" -
2023-09-07 14:46:21.053266 I [dry-run] ReleaseTools::Security::TargetIssuesProcessor -- https://gitlab.com/gitlab-org/security/gitlab/-/issues/970 is ready to be processed and will be linked to the security release tracking issue.
2023-09-07 14:46:21.547938 D [dry-run] ReleaseTools::GitlabClient -- [HTTParty] [2023-09-07 14:46:21 -0600] 200 "GET https://gitlab.com/api/v4/projects/gitlab-org%2Fgitlab/issues" -
2023-09-07 14:46:22.233667 D [dry-run] ReleaseTools::GitlabClient -- [HTTParty] [2023-09-07 14:46:22 -0600] 201 "POST https://gitlab.com/api/v4/projects/gitlab-org%2Fgitlab/issues/424529/links" 3063
2023-09-07 14:46:22.897274 D [dry-run] ReleaseTools::GitlabClient -- [HTTParty] [2023-09-07 14:46:22 -0600] 200 "GET https://gitlab.com/api/v4/projects/gitlab-org%2Fgitlab/issues/424529/links" -
2023-09-07 14:46:23.204996 D ReleaseTools::GitlabClient -- [HTTParty] [2023-09-07 14:46:23 -0600] 200 "GET https://gitlab.com/api/v4/projects/15642544/issues/969/related_merge_requests" 2
2023-09-07 14:46:23.205115 D ReleaseTools::GitlabClient -- [HTTParty] [2023-09-07 14:46:23 -0600] 200 "GET https://gitlab.com/api/v4/projects/15642544/issues/970/related_merge_requests" 2
2023-09-07 14:46:23.647255 D ReleaseTools::GitlabClient -- [HTTParty] [2023-09-07 14:46:23 -0600] 200 "GET https://gitlab.com/api/v4/projects/15642544/issues/967/related_merge_requests" -
2023-09-07 14:46:23.647810 I [dry-run] ReleaseTools::Security::TargetIssuesProcessor -- https://gitlab.com/gitlab-org/security/gitlab/-/issues/969 will be unlinked from the security release tracking issue as it is no longer ready to be processed.
2023-09-07 14:46:23.979452 D [dry-run] ReleaseTools::GitlabClient -- [HTTParty] [2023-09-07 14:46:23 -0600] 200 "GET https://gitlab.com/api/v4/projects/gitlab-org%2Fgitlab/issues/424529/links" -
2023-09-07 14:46:24.614456 D [dry-run] ReleaseTools::GitlabClient -- [HTTParty] [2023-09-07 14:46:24 -0600] 200 "DELETE https://gitlab.com/api/v4/projects/gitlab-org%2Fgitlab/issues/424529/links/3275614" -
2023-09-07 14:46:24.946716 D [dry-run] ReleaseTools::GitlabClient -- [HTTParty] [2023-09-07 14:46:24 -0600] 200 "GET https://gitlab.com/api/v4/projects/gitlab-org%2Fgitlab/issues/424529/links" -
2023-09-07 14:46:25.212280 D ReleaseTools::GitlabClient -- [HTTParty] [2023-09-07 14:46:25 -0600] 200 "GET https://gitlab.com/api/v4/projects/15642544/issues/970/related_merge_requests" 2
2023-09-07 14:46:25.565647 D ReleaseTools::GitlabClient -- [HTTParty] [2023-09-07 14:46:25 -0600] 200 "GET https://gitlab.com/api/v4/projects/15642544/issues/967/related_merge_requests" -
2023-09-07 14:46:25.566736 I [dry-run] ReleaseTools::Security::TargetIssuesProcessor -- https://gitlab.com/gitlab-org/security/gitlab/-/issues/967 is already linked to the security release tracking issue and still ready to be processed.Here is a screenshot of the fake tracking issue after the processor was run showing that 970 was linked and 969 was unlinked:
Author Check-list
- [-] Has documentation been updated?
Edited by Steve Abrams