Extend DAST policy to support scheduled execution
Why are we doing this work
To support scheduled scans configured in Security Orchestration Policies we need to add workers and services to configure and execute scheduled jobs with selected scans.
Relevant links
Non-functional requirements
- [-] Documentation: added in separate issue,
-
Feature flag: all work will be hidden behind already created feature flag: security_orchestration_policies_configuration
- [-] Performance: verify the impact of additional cron jobs on the system, coordinate with
CI
team, -
Testing: -
verify if pipeline
scan policies are working properly, -
verify if schedule
scan policies are working properly:-
check multiple cron expressions (with *
,SUN,MON
, numbers, etc.) -
check if jobs are properly scheduled when the repository with policies is updated,
-
-
Implementation plan
-
backend extend config/initializers/1_settings.rb
withSecurity::CreateOrchestrationPolicyWorker
andSecurity::OrchestrationPolicyRuleScheduleWorker
, properly setcron
value for these jobs,CreateOrchestrationPolicyWorker
can work every hour, butOrchestrationPolicyRuleScheduleWorker
must work quite often (ie. every 15 minutes) to make sure we can run scheduled scans on provided by users schedule (mention that in the documentation), -
backend extend ee/app/models/security/orchestration_policy_configuration.rb
with scope:with_outdated_configuration
that will get all policy configurations that does not have value forconfigured_at
field or for which value ofconfigured_at
is earlier thansecurity_policy_management_project.last_activity_at
-> this is not a perfect way to get information when was the last push tosecurity_policy_management_project
(and potential policy update), but it will significantly reduce calls to database to createOrchestrationPolicyRuleSchedule
inee/app/workers/security/create_orchestration_policy_worker.rb
, -
backend extend ee/app/models/security/orchestration_policy_rule_schedule.rb
with scoperunnable_schedules
and methods needed bySchedulable
concern:ideal_next_run_from
,cron_worker_next_run_from
andset_next_run_at
;policy
method (to load policy configuration), -
backend add ee/app/workers/security/create_orchestration_policy_worker.rb
that will go throughSecurity::OrchestrationPolicyConfiguration.with_outdated_configuration
in batches, and for each outdated configuration it will callSecurity::OrchestrationPolicies::ProcessRuleService
service, -
backend add ee/app/services/security/orchestration_policies/process_rule_service.rb
that will recreateSecurity::OrchestrationPolicyRuleSchedule
in database for eachschedule
policy and setconfigured_at
forOrchestrationPolicyConfiguration
, -
backend add ee/app/workers/security/orchestration_policy_rule_schedule_worker.rb
that will go throughSecurity::OrchestrationPolicyRuleSchedule.runnable_schedules
(all schedules withnext_run_at < Time.now
) in batches, and for each runnable schedule it will callSecurity::OrchestrationPolicies::RuleScheduleService
service, -
backend add ee/app/services/security/orchestration_policies/rule_schedule_service.rb
that will set new value fornext_run_at
field and go through actions in provided policy and start new pipeline with configured DAST On Demand Scan using::DastOnDemandScans::CreateService
, when triggering a pipeline take last user that edited the policy file,
Edited by Sashi Kumar Kumaresan