Track security scan report size in snowplow

Why are we doing this work

Security scan jobs that produce large reports can cause performance issues and timeouts (e.g. #331679 (closed)).

We'd like to collect size information for scan reports so that we can introduce a size limit that will minimise impact to users.

Relevant links

Non-functional requirements

• Documentation:
• Feature flag:
• Performance:
• Testing:

Implementation plan

1. Add a new version of the IGLU Secure metric schema to include new fields:
   • report_size
   • findings_count
2. Update the schema version in the TrackScanService.
3. Add report size to Security::Report, this should come from ci_job_artifacts.size.
4. Add new fields to data_to_track:
   1. report_size
   2. findings_count (this should correspond to report.findings.count)
5. Add a histogram of report size to the periscope Secure Scan Metrics dashboard. The chart should use the secure_scans view and be filtered by the filters on the dashboard, i.e.

   FROM   [secure_scans]
   SELECT ...
   WHERE  [analyzer=Analyzer]
   AND [scanner=Scanner]
   AND [scan_type=ScanType]
   AND [start_time=daterange]

   See other charts on the same dashboard for other examples.
6. As above for findings_count.
7. Using Snowplow Micro, verify that the metrics can be tracked locally. Note that the metrics will not be tracked if it doesn't conform to the schema or the schema has not been released.