Validate security report artifacts
What does this MR do?
With this change, we will start validating the security report artifacts based on their corresponding report schemas if the VALIDATE_SCHEMA
environment variable is set as "true"
for the related build. This change will enable us to test the logic around the validation process and at some point, we will run the validation by default for all security report artifacts.
Related to #321918 (closed).
Scenarios
Report contains malformed JSON
In this case, the report must be marked as errored as the parsing fails.
Test file: scenario_1.json
Report contains a wellformed JSON
In this case, the validation logic runs and validates the report artifact.
Test Files
- JSON primitive: scenario_2.json
- Invalid JSON object: scenario_3.json
- Valid JSON object: scenario_4.json
How to test this locally?
The following YAML configuration adds 4 `sast` jobs into the `gitlab-ci.yml` to make it easier to test the setup on local environment.
.test-sast:
variables:
VALIDATE_SCHEMA: "true"
artifacts:
reports:
sast: "${CI_JOB_NAME}.json"
script:
- echo "$REPORT_CONTENT" > "${CI_JOB_NAME}.json"
test-sast-1:
extends: .test-sast
variables:
REPORT_CONTENT: 'This is an invalid JSON!'
test-sast-2:
extends: .test-sast
variables:
REPORT_CONTENT: '"This is a valid JSON!"'
test-sast-3:
extends: .test-sast
variables:
REPORT_CONTENT: '{"foo": "BAR"}'
test-sast-4:
extends: .test-sast
variables:
REPORT_CONTENT: '{"version": "14.0.0", "vulnerabilities": []}'
After running a pipeline with the above configuration, we can check the validation logic on the console with the following script;
build_id = # The ID of the build
build = Ci::Build.find(build_id)
artifact = build.job_artifacts.find { |artifact| artifact.file_type == 'sast' }
report = artifact.security_report(validate: true)
report.errors # The list of the errors
Does this MR meet the acceptance criteria?
Conformity
-
📋 Does this MR need a changelog?-
I have included a changelog entry.
-
-
Documentation (if required) -
Code review guidelines -
Merge request performance guidelines -
Style guides - [-] Database guides
-
Separation of EE specific content
Availability and Testing
- [-] Review and add/update tests for this feature/bug. Consider all test levels. See the Test Planning Process.
- [-] Tested in all supported browsers
- [-] Informed Infrastructure department of a default or new setting change, if applicable per definition of done
Security
If this MR contains changes to processing or storing of credentials or tokens, authorization and authentication methods and other items described in the security review guidelines:
- [-] Label as security and @ mention
@gitlab-com/gl-security/appsec
- [-] The MR includes necessary changes to maintain consistency between UI, API, email, or other methods
- [-] Security reports checked/validated by a reviewer from the AppSec team