Skip to content

Backfill owasp_top_10 null with -1 on vulnerability_reads

Bala Kumar requested to merge bk/473748-backfill-owasp-top-10-null into master

What does this MR do and why?

Change owasp_top_10 column default to int on vulnerability_reads

3rd MR of #473748 (closed)

Changelog: other

Database

1. Migration up > bundle exec rake db:migrate:up:main VERSION=20240805205657

main: == [advisory_lock_connection] object_id: 127260, pg_backend_pid: 57166
main: == 20240805205657 TmpIndexForOwaspOnVulnerabilityReads: migrating =============
main: -- transaction_open?(nil)
main:    -> 0.0000s
main: -- view_exists?(:postgres_partitions)
main:    -> 0.0260s
main: -- index_exists?(:vulnerability_reads, [:vulnerability_id], {:name=>"tmp_index_for_owasp_null_on_vulnerability_reads", :where=>"owasp_top_10 IS NULL", :algorithm=>:concurrently})
main:    -> 0.0267s
main: -- execute("SET statement_timeout TO 0")
main:    -> 0.0002s
main: -- add_index(:vulnerability_reads, [:vulnerability_id], {:name=>"tmp_index_for_owasp_null_on_vulnerability_reads", :where=>"owasp_top_10 IS NULL", :algorithm=>:concurrently})
main:    -> 0.0175s
main: -- execute("RESET statement_timeout")
main:    -> 0.0003s
main: == 20240805205657 TmpIndexForOwaspOnVulnerabilityReads: migrated (0.1198s) ====

main: == [advisory_lock_connection] object_id: 127260, pg_backend_pid: 57166

2. Migration down > bundle exec rake db:migrate:down:main VERSION=20240805205657

main: == [advisory_lock_connection] object_id: 127260, pg_backend_pid: 57602
main: == 20240805205657 TmpIndexForOwaspOnVulnerabilityReads: reverting =============
main: -- transaction_open?(nil)
main:    -> 0.0000s
main: -- view_exists?(:postgres_partitions)
main:    -> 0.0139s
main: -- indexes(:vulnerability_reads)
main:    -> 0.0117s
main: -- execute("SET statement_timeout TO 0")
main:    -> 0.0002s
main: -- remove_index(:vulnerability_reads, {:algorithm=>:concurrently, :name=>"tmp_index_for_owasp_null_on_vulnerability_reads"})
main:    -> 0.0054s
main: -- execute("RESET statement_timeout")
main:    -> 0.0002s
main: == 20240805205657 TmpIndexForOwaspOnVulnerabilityReads: reverted (0.0467s) ====

main: == [advisory_lock_connection] object_id: 127260, pg_backend_pid: 57602
  • Explain plan for the final update query:
UPDATE
  vulnerability_reads vr
SET
  owasp_top_10 = $1
FROM
  (
    WITH
      batched_relation AS MATERIALIZED (
        SELECT
          vulnerability_reads.*
        FROM
          vulnerability_reads
        WHERE
          vulnerability_reads.vulnerability_id BETWEEN $2 AND $3
          AND vulnerability_reads.owasp_top_10 IS NULL
          AND vulnerability_reads.vulnerability_id >= $4
          AND vulnerability_reads.vulnerability_id < $5
        LIMIT
          $6
      )
    SELECT
      vulnerability_reads.id AS id
    FROM
      batched_relation vulnerability_reads
  ) selected_ids
WHERE
  vr.id = selected_ids.id

https://console.postgres.ai/gitlab/gitlab-production-main/sessions/31012/commands/96323

  • Queries histogram analysis during the DB migration testing pipeline:

Screenshot_2024-08-24_at_11.57.24_PM

MR acceptance checklist

Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Bala Kumar

Merge request reports

Loading