Improve vulnerabilities scan_finding count by uuid and state operation
What does this MR do and why?
Runs the count operation query in batches as it can increase query parsing size for PostreSQL and also other places when logged as well. See related infra issue
Related to #385634 (closed)
Database
For the issue observed project 20057221
, the pipeline_id with max findings is 467742628
Vulnerabilities::FindingPipeline.joins(:finding).where('vulnerability_occurrences.project_id' => 20057221).select('vulnerability_occurrence_pipelines.pipeline_id, count(vulnerability_occurrence_pipelines.pipeline_id)').group('vulnerability_occurrence_pipelines.pipeline_id').limit(100000).order('count(vulnerability_occurrence_pipelines.pipeline_id) desc').first
=> 467742628
Vulnerabilities::FindingPipeline.where(:pipeline_id => 467742628).size
=> 63104
Number of batches: 63104/50 = 1262
Before batching(limiting to first 100 uuids, not all 63104)
After batching
How to set up and validate locally
Feature functionality can be tested by following same steps from !105248 (merged)
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.
Related to #385634 (closed)
Edited by Bala Kumar