Add NOT NULL constraint to `uuid` column in `security_findings` table
What does this MR do and why?
According to our guidelines for adding NOT NULL constraints on big tables this MR:
- Adds a not-valid
NOT NULL
constraint - Schedules
DropInvalidSecurityFindings
background migration for the entiresecurity_findings
table in batches of 10 000
Related to #284996 (closed)
Database review
db:migrate and db:rollback
➜ bundle exec rails db:migrate
== 20211110143306 AddNotNullConstraintToSecurityFindingsUuid: migrating =======
-- current_schema()
-> 0.0002s
-- transaction_open?()
-> 0.0000s
-- current_schema()
-> 0.0003s
-- transaction_open?()
-> 0.0000s
-- execute("ALTER TABLE security_findings\nADD CONSTRAINT check_6c2851a8c9\nCHECK ( uuid IS NOT NULL )\nNOT VALID;\n")
-> 0.0026s
== 20211110143306 AddNotNullConstraintToSecurityFindingsUuid: migrated (0.0265s)
➜ bundle exec rails db:rollback
== 20211110143306 AddNotNullConstraintToSecurityFindingsUuid: reverting =======
-- transaction_open?()
-> 0.0000s
-- transaction_open?()
-> 0.0000s
-- execute("ALTER TABLE security_findings\nDROP CONSTRAINT IF EXISTS check_6c2851a8c9\n")
-> 0.0018s
== 20211110143306 AddNotNullConstraintToSecurityFindingsUuid: reverted (0.0080s)
Background migration
Number of rows affected
Unknown, according to !63937 (comment 612763457) we already ran a migration on GitLab.com but on-premises customers can have some of these rows so I opted for a batch size of 10 000 and measured the time required to drop a WHOLE batch.
db:migrate and db:rollback
➜ bundle exec rails db:migrate
== 20211110151350 ScheduleDropInvalidSecurityFindings: migrating ==============
-- Scheduled 0 DropInvalidSecurityFindings jobs with a maximum of 10000 records per batch and an interval of 120 seconds.
The migration is expected to take at least 0 seconds. Expect all jobs to have completed after 2021-11-10 16:12:07 UTC."
== 20211110151350 ScheduleDropInvalidSecurityFindings: migrated (0.0039s) =====
➜ bundle exec rails db:rollback
== 20211110151350 ScheduleDropInvalidSecurityFindings: reverting ==============
== 20211110151350 ScheduleDropInvalidSecurityFindings: reverted (0.0000s) =====
Dropping 10_000 security_findings
Cold cache: https://console.postgres.ai/gitlab/gitlab-production-tunnel-pg12/sessions/7295/commands/25953
Warm cache: https://console.postgres.ai/gitlab/gitlab-production-tunnel-pg12/sessions/7295/commands/25954
Total migration duration
Table size: 405 557 791
Batch size: 10 000
Batches needed: 40 556
Time per batch: ~2 minutes
Total running time: 2*40556 ~= 56 days
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.
Edited by Michał Zając