Add RelatedPipelinesFinder for security policies
What does this MR do and why?
This MR adds a new finder for querying latest pipelines with matching sha of a given pipeline grouped by source. This is needed for scan result policy to compare the security findings from multiple pipelines for a given sha (Compare results from all pipelines related to c... (#379108 - closed))
Consider this case for main
default branch and feature-branch
id | source | branch | security scans |
---|---|---|---|
1 | push | main | [sast, dependency_scanning] |
2 | push | feature-branch | [sast, dependency_scanning] |
3 | schedule | main | [dast] |
4 | security_orchestration_policy | main | [container_scanning] |
5 | schedule | feature-branch | [dast] |
6 | security_orchestration_policy | main | [container_scanning] |
8 | merge_request_event | feature-branch | [container_scanning] |
9 | security_orchestration_policy | feature-branch | [container_scanning] |
10 | schedule | feature-branch | [dast] |
11 | security_orchestration_policy | feature-branch | [container_scanning] |
12 | schedule | main | [dast] |
graph TD
subgraph source
direction LR
B[Source Branch Latest SHA] -->|Push| D[Pipeline 2]
B -->|merge_request_event| E[Pipeline 8]
B -->|schedule| F[Pipeline 10]
F --> H[Pipeline 5]
B -->|security_orchestration_policy| G[Pipeline 11]
G --> J[Pipeline 9]
end
subgraph target
direction RL
K[Target Branch Latest SHA] -->|Push| L[Pipeline 1]
K -->|schedule| Q[Pipeline 12]
Q --> M[Pipeline 3]
K -->|security_orchestration_policy| R[Pipeline 6]
R --> S[Pipeline 4]
end
The finder returns pipeline 2, 8, 10, 11 for the source branch and pipeline 1, 12, 6 for the target branch:
Database query plan
SELECT
max(id) as id
FROM
"ci_pipelines"
WHERE
"ci_pipelines"."project_id" = 278964
AND (
"ci_pipelines"."status" IN (
'success','failed','canceled','skipped'
)
)
AND "ci_pipelines"."sha" = '6418b07f2e90539ce87dbe51b8624d2e486b6ee0'
AND (
"ci_pipelines"."source" IN (
1, 2, 3, 4, 5, 6, 7, 8, 10, 11, 15
)
OR "ci_pipelines"."source" IS NULL
)
GROUP BY
"ci_pipelines"."source"
Aggregate (cost=3.75..3.77 rows=1 width=8) (actual time=78.052..78.057 rows=2 loops=1)
Group Key: ci_pipelines.source
Buffers: shared hit=3 read=21 dirtied=2
I/O Timings: read=77.587 write=0.000
-> Sort (cost=3.75..3.75 rows=1 width=8) (actual time=78.039..78.041 rows=5 loops=1)
Sort Key: ci_pipelines.source
Sort Method: quicksort Memory: 25kB
Buffers: shared hit=3 read=21 dirtied=2
I/O Timings: read=77.587 write=0.000
-> Index Scan using index_ci_pipelines_on_project_id_and_sha on public.ci_pipelines (cost=0.70..3.74 rows=1 width=8) (actual time=27.555..77.993 rows=5 loops=1)
Index Cond: ((ci_pipelines.project_id = 278964) AND ((ci_pipelines.sha)::text = '6418b07f2e90539ce87dbe51b8624d2e486b6ee0'::text))
Filter: (((ci_pipelines.status)::text = ANY ('{success,failed,canceled,skipped}'::text[])) AND ((ci_pipelines.source = ANY ('{1,2,3,4,5,6,7,8,10,11,15}'::integer[])) OR (ci_pipelines.source IS NULL)))
Rows Removed by Filter: 12
Buffers: shared read=21 dirtied=2
I/O Timings: read=77.587 write=0.000
Time: 85.323 ms
- planning: 7.194 ms
- execution: 78.129 ms
- I/O read: 77.587 ms
- I/O write: 0.000 ms
Shared buffers:
- hits: 3 (~24.00 KiB) from the buffer pool
- reads: 21 (~168.00 KiB) from the OS file cache, including disk I/O
- dirtied: 2 (~16.00 KiB)
- writes: 0
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.
Edited by Sashi Kumar Kumaresan