Skip to content

Bulk create license scan result policies

What does this MR do and why?

This replaces the use of SoftwareLicensePolicies::CreateService with a new bulk create service to create multiple licenses and license policies at once.

There are two bulk operations:

  1. Insert unknown licenses into software_licenses
  2. Insert new license policies into software_license_policies

This will probably improve worker performance, especially when updating license policies with multiple licenses for a large namespace. The downside is that bulk operations ignore validations and callbacks. Adding a new validation or callback to SoftwareLicense or SoftwareLicensePolicy might cause problems.

Database notes

The bulk insert service executes 3 queries:

I experimented with different batch sizes. 500 was quite slow and 250 surprisingly fast. The second query probably benefited from a warm cache, but I think 250 is the safer option anyway.

Screenshots or screen recordings

There should be no user visible change

How to set up and validate locally

  1. Enable the feature flag echo "Feature.enable(:feature_flag_name)" | rails c
  2. Create a new project
  3. Go to Secure -> Policies
  4. Select New policy
  5. Select Scan result policy
  6. Switch to .yaml mode and paste the policy with a custom license:
    type: scan_result_policy
    name: license
    description: ''
    enabled: true
    policy_scope:
      compliance_frameworks: []
    rules:
      - type: license_finding
        match_on_inclusion: true
        license_types:
          - test MR 139163 license
        license_states:
          - newly_detected
          - detected
        branch_type: protected
    actions:
      - type: require_approval
        approvals_required: 1
        role_approvers:
          - maintainer
    approval_settings:
      block_branch_modification: true
      prevent_pushing_and_force_pushing: true
  7. Select Configure with a merge request and merge the MR
  8. Open the rails console and see if a new license_policy was created:
    SoftwareLicensePolicy.joins(:software_license).where(software_licenses: {name: "test MR 139163 license"})
  9. This should return at least one result. (You might have to restart GDK to update the background workers)

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Andy Schoenen

Merge request reports

Loading