Associate Build with DAST Site and Scanner profile [RUN ALL RSPEC] [RUN AS-IF-FOSS]
What does this MR do?
adds an association between ci_builds
and dast_site_profiles + dast_scanner_profiles
in order to deliver calculated variables to ci_builds
.
Example
when you specify a site
or scanner
profile in the yaml
config, we will look up those records in the database by name and add the calculated variables to the build
using DastSiteProfile#ci_variables
and DastScannerProfile#ci_variables
.
dast:
stage: dast
dast_configuration:
site_profile: "site-profile-name"
scanner_profile: "scanner-profile-name"
script:
- echo
the merge request that adds that adds the job-level keyword is here and the merge request that seeds the build is here.
Context
with dast
we have the concept of on-demand scans, which essentially entails storing dast
config in the database (dast_scanner_profiles
and dast_site_profiles
) and triggering scans via GraphQL on a manual basis. the proposal is to reference those database entities by name in the .gitlab-ci.yml
and seed their configuration into pipelines triggered via the regular ci/cd process.
Why?
whilst refining this feature we got feedback that indicate the modelling we already had in place (between dast_profiles
and ci_pipelines
) would fit with what we were trying to achieve. it was therefore necessary to introduce a different relation to better capture that there might be many dast
builds for a given pipeline.
Related Issue(s)
Related Merge Requests
- Add job-level keyword for DAST configuration
- Associate Build with DAST Site and Scanner profile
- Deliver DAST on-demand CI variables to builds
Database
Migration
rails db:migrate:up VERSION=20210604032738 && \
rails db:migrate:up VERSION=20210604034158 && \
rails db:migrate:up VERSION=20210604034354 && \
rails db:migrate:up VERSION=20210604051330 && \
rails db:migrate:up VERSION=20210604051742 && \
rails db:migrate:up VERSION=20210604051917
== 20210604032738 CreateDastSiteProfilesBuilds: migrating =====================
-- create_table(:dast_site_profiles_builds, {:primary_key=>[:dast_site_profile_id, :ci_build_id], :comment=>"{\"owner\":\"group::dynamic analysis\",\"description\":\"Join table between DAST Site Profiles and CI Builds\"}"})
-> 0.0135s
== 20210604032738 CreateDastSiteProfilesBuilds: migrated (0.0137s) ============
== 20210604034158 AddCiBuildIdFkToDastSiteProfilesBuilds: migrating ===========
-- transaction_open?()
-> 0.0000s
-- foreign_keys(:dast_site_profiles_builds)
-> 0.0048s
-- execute("ALTER TABLE dast_site_profiles_builds\nADD CONSTRAINT fk_a325505e99\nFOREIGN KEY (ci_build_id)\nREFERENCES ci_builds (id)\nON DELETE CASCADE\nNOT VALID;\n")
-> 0.0041s
-- execute("SET statement_timeout TO 0")
-> 0.0014s
-- execute("ALTER TABLE dast_site_profiles_builds VALIDATE CONSTRAINT fk_a325505e99;")
-> 0.0067s
-- execute("RESET ALL")
-> 0.0012s
== 20210604034158 AddCiBuildIdFkToDastSiteProfilesBuilds: migrated (0.0358s) ==
== 20210604034354 AddDastSiteProfileIdFkToDastSiteProfilesBuilds: migrating ===
-- transaction_open?()
-> 0.0000s
-- foreign_keys(:dast_site_profiles_builds)
-> 0.0039s
-- execute("ALTER TABLE dast_site_profiles_builds\nADD CONSTRAINT fk_94e80df60e\nFOREIGN KEY (dast_site_profile_id)\nREFERENCES dast_site_profiles (id)\nON DELETE CASCADE\nNOT VALID;\n")
-> 0.0021s
-- execute("SET statement_timeout TO 0")
-> 0.0005s
-- execute("ALTER TABLE dast_site_profiles_builds VALIDATE CONSTRAINT fk_94e80df60e;")
-> 0.0047s
-- execute("RESET ALL")
-> 0.0006s
== 20210604034354 AddDastSiteProfileIdFkToDastSiteProfilesBuilds: migrated (0.0193s)
== 20210604051330 CreateDastScannerProfilesBuilds: migrating ==================
-- create_table(:dast_scanner_profiles_builds, {:primary_key=>[:dast_scanner_profile_id, :ci_build_id], :comment=>"{\"owner\":\"group::dynamic analysis\",\"description\":\"Join table between DAST Scanner Profiles and CI Builds\"}"})
-> 0.0154s
== 20210604051330 CreateDastScannerProfilesBuilds: migrated (0.0156s) =========
== 20210604051742 AddCiBuildIdFkToDastScannerProfilesBuilds: migrating ========
-- transaction_open?()
-> 0.0000s
-- foreign_keys(:dast_scanner_profiles_builds)
-> 0.0047s
-- execute("ALTER TABLE dast_scanner_profiles_builds\nADD CONSTRAINT fk_e4c49200f8\nFOREIGN KEY (ci_build_id)\nREFERENCES ci_builds (id)\nON DELETE CASCADE\nNOT VALID;\n")
-> 0.0022s
-- execute("SET statement_timeout TO 0")
-> 0.0009s
-- execute("ALTER TABLE dast_scanner_profiles_builds VALIDATE CONSTRAINT fk_e4c49200f8;")
-> 0.0061s
-- execute("RESET ALL")
-> 0.0010s
== 20210604051742 AddCiBuildIdFkToDastScannerProfilesBuilds: migrated (0.0253s)
== 20210604051917 AddDastScannerProfileIdFkToDastScannerProfilesBuilds: migrating
-- transaction_open?()
-> 0.0000s
-- foreign_keys(:dast_scanner_profiles_builds)
-> 0.0047s
-- execute("ALTER TABLE dast_scanner_profiles_builds\nADD CONSTRAINT fk_5d46286ad3\nFOREIGN KEY (dast_scanner_profile_id)\nREFERENCES dast_scanner_profiles (id)\nON DELETE CASCADE\nNOT VALID;\n")
-> 0.0023s
-- execute("SET statement_timeout TO 0")
-> 0.0009s
-- execute("ALTER TABLE dast_scanner_profiles_builds VALIDATE CONSTRAINT fk_5d46286ad3;")
-> 0.0029s
-- execute("RESET ALL")
-> 0.0006s
== 20210604051917 AddDastScannerProfileIdFkToDastScannerProfilesBuilds: migrated (0.0226s)
rails db:migrate:down VERSION=20210604051917 && \
rails db:migrate:down VERSION=20210604051742 && \
rails db:migrate:down VERSION=20210604051330 && \
rails db:migrate:down VERSION=20210604034354 && \
rails db:migrate:down VERSION=20210604034158 && \
rails db:migrate:down VERSION=20210604032738
== 20210604051917 AddDastScannerProfileIdFkToDastScannerProfilesBuilds: reverting
-- remove_foreign_key(:dast_scanner_profiles_builds, {:column=>:dast_scanner_profile_id})
-> 0.0066s
== 20210604051917 AddDastScannerProfileIdFkToDastScannerProfilesBuilds: reverted (0.0142s)
== 20210604051742 AddCiBuildIdFkToDastScannerProfilesBuilds: reverting ========
-- remove_foreign_key(:dast_scanner_profiles_builds, {:column=>:ci_build_id})
-> 0.0058s
== 20210604051742 AddCiBuildIdFkToDastScannerProfilesBuilds: reverted (0.0149s)
== 20210604051330 CreateDastScannerProfilesBuilds: reverting ==================
-- drop_table(:dast_scanner_profiles_builds)
-> 0.0035s
== 20210604051330 CreateDastScannerProfilesBuilds: reverted (0.0036s) =========
== 20210604034354 AddDastSiteProfileIdFkToDastSiteProfilesBuilds: reverting ===
-- remove_foreign_key(:dast_site_profiles_builds, {:column=>:dast_site_profile_id})
-> 0.0061s
== 20210604034354 AddDastSiteProfileIdFkToDastSiteProfilesBuilds: reverted (0.0146s)
== 20210604034158 AddCiBuildIdFkToDastSiteProfilesBuilds: reverting ===========
-- remove_foreign_key(:dast_site_profiles_builds, {:column=>:ci_build_id})
-> 0.0061s
== 20210604034158 AddCiBuildIdFkToDastSiteProfilesBuilds: reverted (0.0149s) ==
== 20210604032738 CreateDastSiteProfilesBuilds: reverting =====================
-- drop_table(:dast_site_profiles_builds)
-> 0.0054s
== 20210604032738 CreateDastSiteProfilesBuilds: reverted (0.0054s) ============
Does this MR meet the acceptance criteria?
Conformity
-
I have included a changelog entry, or it's not needed. (Does this MR need a changelog?) -
I have added/updated documentation, or it's not needed. (Is documentation required?) -
I have properly separated EE content from FOSS, or this MR is FOSS only. (Where should EE code go?) -
I have added information for database reviewers in the MR description, or it's not needed. (Does this MR have database related changes?) -
I have self-reviewed this MR per code review guidelines. -
This MR does not harm performance, or I have asked a reviewer to help assess the performance impact. (Merge request performance guidelines) -
I have followed the style guides. -
This change is backwards compatible across updates, or this does not apply.
Availability and Testing
-
I have added/updated tests following the Testing Guide, or it's not needed. (Consider all test levels. See the Test Planning Process.) -
I have tested this MR in all supported browsers, or it's not needed. -
I have informed the Infrastructure department of a default or new setting change per definition of done, or it's not needed.
Security
Does this MR contain changes to processing or storing of credentials or tokens, authorization and authentication methods or other items described in the security review guidelines? If not, then delete this Security section.
-
Label as security and @ mention @gitlab-com/gl-security/appsec
-
The MR includes necessary changes to maintain consistency between UI, API, email, or other methods -
Security reports checked/validated by a reviewer from the AppSec team