Scan execution policy CI variables not being applied in scan

Summary

There are two issues that seem similar, but I am not sure they are the same When creating a project-level scan execution policy with a secret detection action and CI variable SECRET_DETECTION_HISTORIC_SCAN: 'true'

Steps to reproduce

1. Upload a GitLab Ultimate license
2. Navigate to a project/group => Secure => Policies => New policy => Scan execution
3. Create the following policies (or something similar)

--
scan_execution_policy:
- name: Secret Detection Execution
  description: ''
  enabled: true
  rules:
  - type: pipeline
    branch_type: all
  actions:
  - scan: secret_detection
    tags:
    - saas-linux-small-amd64
    variables:
      SECRET_DETECTION_HISTORIC_SCAN: 'true'
      SECURE_LOG_LEVEL: debug
      SECRET_DETECTION_RULESET_GIT_REFERENCE: "$USER_EXT_RULESET_SEC_POL:$EXT_RULESET_SEC_POL@gitlab.com/roche/playground/gitlab-duo/group-security-policies"
scan_result_policy:
- name: Found Secret Block
  description: ''
  enabled: true
  rules:
  - type: scan_finding
    scanners:
    - secret_detection
    vulnerabilities_allowed: 0
    severity_levels:
    - critical
    - high
    vulnerability_states:
    - new_needs_triage
    - new_dismissed
    branch_type: protected
  actions:
  - type: require_approval
    approvals_required: 1
    group_approvers_ids:
    - 70377947

4. Save the policy, merge it, run a pipeline with it
5. Inspect the secret detection job logs

Example Project

https://staging.gitlab.com/govern-team-test/cnp-alert-demo/-/security/policies/Verify%20CI%20Variables/edit?type=scan_execution_policy

What is the current bug behavior?

When a user creates the CI variable SECRET_DETECTION_HISTORIC_SCAN: 'true', the secret detection job in the pipeline does not use true

What is the expected correct behavior?

When a user creates the CI variable SECRET_DETECTION_HISTORIC_SCAN: 'true', the secret detection job in the pipeline does use true

Relevant logs and/or screenshots

1. walkthrough video of group-level policy not using historic scan

2. walkthrough video of project-level policy not using group path

Output of checks

Results of GitLab environment info

Expand for output related to GitLab environment info

(For installations with omnibus-gitlab package run and paste the output of:
`sudo gitlab-rake gitlab:env:info`)

(For installations from source run and paste the output of:
`sudo -u git -H bundle exec rake gitlab:env:info RAILS_ENV=production`)

Results of GitLab application Check

Expand for output related to the GitLab application check

(For installations with omnibus-gitlab package run and paste the output of:
sudo gitlab-rake gitlab:check SANITIZE=true)
(For installations from source run and paste the output of:
sudo -u git -H bundle exec rake gitlab:check RAILS_ENV=production SANITIZE=true)
(we will only investigate if the tests are passing)

Implementation

• backend update security_orchestration_policies/scan_pipeline_service.rb to allow for customization of the SECRET_DETECTION_HISTORIC_SCAN property. We might want to add this behind a feature flag because the reasoning of the default value isn't clear and we need to coordinate with the frontend changes.

--- a/ee/app/services/security/security_orchestration_policies/ci_configuration_service.rb
+++ b/ee/app/services/security/security_orchestration_policies/ci_configuration_service.rb
@@ -40,6 +40,7 @@ def pipeline_configuration(action, ci_variables, index)
           apply_tags!(job_configuration, action[:tags])
           remove_extends!(job_configuration)
           remove_rule_to_disable_job!(job_configuration)
+          job_configuration[:script] = 'env' # TODO remove this change. It can be used to see the output of all variables in the job
         end
 
         ci_configuration
diff --git a/ee/app/services/security/security_orchestration_policies/scan_pipeline_service.rb b/ee/app/services/security/security_orchestration_policies/scan_pipeline_service.rb
index fff3a573f548..d63dd5c98ad4 100644
--- a/ee/app/services/security/security_orchestration_policies/scan_pipeline_service.rb
+++ b/ee/app/services/security/security_orchestration_policies/scan_pipeline_service.rb
@@ -62,7 +62,7 @@ def prepare_policy_configuration(action, index)
 
         ::Security::SecurityOrchestrationPolicies::CiConfigurationService
           .new
-          .execute(action, action_variables.merge(scan_variables(action)), index)
+          .execute(action, scan_variables(action).merge(action_variables), index)
           .deep_symbolize_keys
       end

• frontend update scan result policy editor page to show the ci variable by default as it has a default value

design