Add partial index to improve mirrors update
What does this MR do?
Improves slow (16-18s) query produced by UpdateAllMirrorsWorker
.
More at #325503 (closed)
The main offenders were:
- Filtering on top of the index by the status in
project_mirror_data
table
Postgres.ai reports
After reset: Without any changes
https://console.postgres.ai/gitlab/gitlab-production-tunnel/sessions/2972/commands/9651
depesz + postgres.ai doesn't work me, so here's the manual paste: https://explain.depesz.com/s/kPSG
project_mirror_data
After reset: With the addition the index on https://console.postgres.ai/gitlab/gitlab-production-tunnel/sessions/2972/commands/9665
After warm-up*: Without any changes
https://console.postgres.ai/gitlab/gitlab-production-tunnel/sessions/2972/commands/9661
project_mirror_data
After warm-up*: With the addition the index on https://console.postgres.ai/gitlab/gitlab-production-tunnel/sessions/2972/commands/9667
*
warm-up = reset + run the target query multiple times in a row
The output of both migrating and rolling back for all migrations
be rake db:migrate
== 20210324112439 AddIndexMirrorDataOnRetryNextExecutionWhereStatus: migrating
-- transaction_open?()
-> 0.0000s
-- index_exists?(:project_mirror_data, [:next_execution_timestamp, :retry_count], {:where=>"(status)::text <> ALL ('{scheduled,started}'::text[])", :name=>"index_mirror_data_non_scheduled_or_started", :algorithm=>:concurrently})
-> 0.0032s
-- execute("SET statement_timeout TO 0")
-> 0.0005s
-- add_index(:project_mirror_data, [:next_execution_timestamp, :retry_count], {:where=>"(status)::text <> ALL ('{scheduled,started}'::text[])", :name=>"index_mirror_data_non_scheduled_or_started", :algorithm=>:concurrently})
-> 0.0107s
-- execute("RESET ALL")
-> 0.0004s
== 20210324112439 AddIndexMirrorDataOnRetryNextExecutionWhereStatus: migrated (0.0157s)
be rake db:rollback
== 20210324112439 AddIndexMirrorDataOnRetryNextExecutionWhereStatus: reverting
-- transaction_open?()
-> 0.0000s
-- index_exists?(:project_mirror_data, [:next_execution_timestamp, :retry_count], {:where=>"(status)::text <> ALL ('{scheduled,started}'::text[])", :name=>"index_mirror_data_non_scheduled_or_started", :algorithm=>:concurrently})
-> 0.0035s
-- execute("SET statement_timeout TO 0")
-> 0.0005s
-- remove_index(:project_mirror_data, {:where=>"(status)::text <> ALL ('{scheduled,started}'::text[])", :name=>"index_mirror_data_non_scheduled_or_started", :algorithm=>:concurrently, :column=>[:next_execution_timestamp, :retry_count]})
-> 0.0068s
-- execute("RESET ALL")
-> 0.0005s
== 20210324112439 AddIndexMirrorDataOnRetryNextExecutionWhereStatus: reverted (0.0121s)
Does this MR meet the acceptance criteria?
Conformity
-
📋 Does this MR need a changelog?-
I have included a changelog entry. - [-] I have not included a changelog entry because _____.
-
- [-] Documentation (if required)
-
Code review guidelines -
Merge request performance guidelines -
Style guides -
Database guides - [-] Separation of EE specific content
Availability and Testing
Run the migration locally & rollbacked.
Run the index addition on postgres.ai (database-lab channel) and verified the query plans.
Security
N/A
Related to #325503 (closed)