Save `finding_data` for security findings
What does this MR do and why?
With this MR, we will start ingesting all the data(except the remediations) for report findings into the database. This way, we will no longer need to parse all the JSON report artifacts for each HTTP request to generate the response of a collection of vulnerability findings.
Remediations are not ingested because they are base64 encoded git diffs and we don't know how big a remediation can be, nor we can truncate it as truncation breaks the patch, therefore, we just store their byte offsets for each vulnerability they are associated with and later read the JSON report artifact partially to get the data.
If this byte offset approach succeeds, we can later store only the byte offsets of vulnerability objects and follow the same approach as remediations to reduce the amount of data we store in the database(basically a platform-agnostic object file storage which stores the indices on PG).
The oj-introspect
gem repository can be found here: https://github.com/meinac/oj-introspect
Related to Save the data of report finding in Security::St... (#376213 - closed).
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.