Skip to content

Add present on default branch to common_finder_query index

What does this MR do and why?

Describe in detail what your merge request does and why.

This Merge Request adds the present_on_default_branch attribute to the common_finder_query index on the vulnerabilities table.

The present_on_default_branch field will be used on most queries, and the vulnerabilities indices should be rearranged.

More context can be found on this comment and this issue

This Merge Request is related to the issue #368323 (closed)

Migration Up

main: == 20220728134255 UpdateIndexVulnerabilitiesCommonFinder: migrating ===========
main: -- transaction_open?()
main:    -> 0.0000s
main: -- index_exists?(:vulnerabilities, [:project_id, :state, :report_type, :present_on_default_branch, :severity, :id], {:name=>"index_vulnerabilities_common_finder_query_on_default_branch", :algorithm=>:concurrently})
main:    -> 0.0144s
main: -- execute("SET statement_timeout TO 0")
main:    -> 0.0003s
main: -- add_index(:vulnerabilities, [:project_id, :state, :report_type, :present_on_default_branch, :severity, :id], {:name=>"index_vulnerabilities_common_finder_query_on_default_branch", :algorithm=>:concurrently})
main:    -> 0.0028s
main: -- execute("RESET statement_timeout")
main:    -> 0.0004s
main: -- transaction_open?()
main:    -> 0.0000s
main: -- indexes(:vulnerabilities)
main:    -> 0.0108s
main: -- remove_index(:vulnerabilities, {:algorithm=>:concurrently, :name=>"index_vulnerabilites_common_finder_query"})
main:    -> 0.0021s
main: == 20220728134255 UpdateIndexVulnerabilitiesCommonFinder: migrated (0.0407s) ==

Migration Down

main: == 20220728134255 UpdateIndexVulnerabilitiesCommonFinder: reverting ===========
main: -- transaction_open?()
main:    -> 0.0000s
main: -- index_exists?(:vulnerabilities, [:project_id, :state, :report_type, :severity, :id], {:name=>"index_vulnerabilites_common_finder_query", :algorithm=>:concurrently})
main:    -> 0.0146s
main: -- Index not created because it already exists (this may be due to an aborted migration or similar): table_name: vulnerabilities, column_name: [:project_id, :state, :report_type, :severity, :id]
main: -- transaction_open?()
main:    -> 0.0000s
main: -- indexes(:vulnerabilities)
main:    -> 0.0099s
main: -- execute("SET statement_timeout TO 0")
main:    -> 0.0003s
main: -- remove_index(:vulnerabilities, {:algorithm=>:concurrently, :name=>"index_vulnerabilities_common_finder_query_on_default_branch"})
main:    -> 0.0022s
main: -- execute("RESET statement_timeout")
main:    -> 0.0003s
main: == 20220728134255 UpdateIndexVulnerabilitiesCommonFinder: reverted (0.0356s) ==

Index creation time

exec CREATE INDEX index_vulnerabilities_common_finder_query_on_default_branch ON vulnerabilities USING btree (project_id, state, report_type, present_on_default_branch, severity, id);

The query has been executed. Duration: 3.896 min

Query explanation

Before index
SELECT
    *
FROM
    "vulnerabilities"
WHERE
    "project_id" = 278964
    AND "report_type" IN (2, 3, 1, 0, 4, 5, 6)
    AND "state" IN (1, 4)
    AND "present_on_default_branch" = TRUE 
ORDER BY
    "severity" DESC,
    "id" DESC
LIMIT 20;
Time: 5.357 s  
  - planning: 3.190 ms  
  - execution: 5.354 s  
    - I/O read: 5.213 s  
    - I/O write: 0.000 ms  
  
Shared buffers:  
  - hits: 4865 (~38.00 MiB) from the buffer pool  
  - reads: 5388 (~42.10 MiB) from the OS file cache, including disk I/O  
  - dirtied: 471 (~3.70 MiB)  
  - writes: 0  

https://explain.depesz.com/s/JA3Bm

After index
SELECT
    *
FROM
    "vulnerabilities"
WHERE
    "project_id" = 278964
    AND "report_type" IN (2, 3, 1, 0, 4, 5, 6)
    AND "state" IN (1, 4)
    AND "present_on_default_branch" = TRUE 
ORDER BY
    "severity" DESC,
    "id" DESC
LIMIT 20;

Cold cache

Time: 439.333 ms  
  - planning: 3.049 ms  
  - execution: 436.284 ms  
    - I/O read: 396.615 ms  
    - I/O write: 0.000 ms  
  
Shared buffers:  
  - hits: 4804 (~37.50 MiB) from the buffer pool  
  - reads: 7241 (~56.60 MiB) from the OS file cache, including disk I/O  
  - dirtied: 189 (~1.50 MiB)  
  - writes: 0  

https://explain.depesz.com/s/6ZLP

Warm cache

Time: 34.882 ms  
  - planning: 0.531 ms  
  - execution: 34.351 ms  
    - I/O read: 0.000 ms  
    - I/O write: 0.000 ms  
  
Shared buffers:  
  - hits: 12045 (~94.10 MiB) from the buffer pool  
  - reads: 0 from the OS file cache, including disk I/O  
  - dirtied: 0  
  - writes: 0  

https://explain.depesz.com/s/NilX

Screenshots or screen recordings

These are strongly recommended to assist reviewers and reduce the time to merge your change.

How to set up and validate locally

Numbered steps to set up and validate the change are strongly suggested.

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Marcos Rocha

Merge request reports

Loading