Create X-Ray scanner uploader job
Description:
The metadata JSON produced by the X-Ray scanner job needs to be imported into the GitLab Rails PostgreSQL database for persistence and further usage.
This should happen asynchronously via a background job triggered upon a successful X-Ray scanner CI job completion.
Requirements:
- Add a background job to handle upload
- Job should be idempotent to allow retry logic
- Create a new database table to store the scan data
- Could use a jsonb column to store raw JSON data at first
- Apply unique constraint on project/repo ID
- Use UPSERT to only persist latest scan data per project
The existing pipeline security scan data integration can be used as a reference:
- Pipeline completion schedules
StoreScansWorker
https://gitlab.com/gitlab-org/gitlab/blob/a065bee3ff5e2fa7c03276dca5e59cd0b6cdab0b/ee/app/models/ee/ci/pipeline.rb#L67 - Calls
StoreScansService
to save artifacts in batches https://gitlab.com/gitlab-org/gitlab/blob/41f2a30c9881ad8aaf2352e5ad9792263fdb3d36/ee/app/services/security/store\_scans\_service.rb#L16 -
StoreGroupedScansService
parses and saves each one
Testing and error handling should follow established patterns.
Please advise if any part of the database integration needs clarification or adjustment.
Edited by Mikołaj Wawrzyniak