Remove records without group from webhooks
What does this MR do?
After adding a NOT VALID
foreign key from web_hooks to groups to prevent invalid records with the foreign key !57735 (merged).
This is the data migration to fix the existing records before we can validate the foreign key.
Count number of invalid records
It looks like we don't have any invalid records in the database (this is a query to check the number of invalid records, we are not executing this query in the migration).
https://postgres.ai/console/gitlab/gitlab-production-tunnel/sessions/3052/commands/10061
SELECT "web_hooks"."id"
FROM "web_hooks"
WHERE "web_hooks"."type" = 'GroupHook'
AND "web_hooks"."group_id" IS NOT NULL
AND "web_hooks"."group_id" NOT IN
(SELECT "namespaces"."id"
FROM "namespaces");
Queries
each_batch
generates two queries to get the start
id and end
id for the relation (the second query is the same with an OFFSET
of the batch size (1_000
in this case).
SELECT "web_hooks"."id"
FROM "web_hooks"
WHERE "web_hooks"."type" = 'GroupHook'
ORDER BY "web_hooks"."id" ASC
LIMIT 1
Time: 669.741 ms
- planning: 0.175 ms
- execution: 669.566 ms
- I/O read: 639.779 ms
- I/O write: N/A
https://postgres.ai/console/gitlab/gitlab-production-tunnel/sessions/3099/commands/10245
and then we have the DELETE
statement, I guess this will use a BETWEEN
in production:
DELETE
FROM "web_hooks"
WHERE "web_hooks"."type" = 'GroupHook'
AND "web_hooks"."id" BETWEEN 548001 AND 549000
AND "web_hooks"."group_id" NOT IN
(SELECT "namespaces"."id"
FROM "namespaces")
Time: 15.457 ms
- planning: 3.207 ms
- execution: 12.250 ms
- I/O read: 6.112 ms
- I/O write: N/A
https://postgres.ai/console/gitlab/gitlab-production-tunnel/sessions/3099/commands/10249
Migration
$ rails db:migrate
== 20210330091751 RemoveRecordsWithoutGroupFromWebhooksTable: migrating =======
== 20210330091751 RemoveRecordsWithoutGroupFromWebhooksTable: migrated (0.0230s)
$ rails db:migrate:down VERSION=20210330091751
== 20210330091751 RemoveRecordsWithoutGroupFromWebhooksTable: reverting =======
== 20210330091751 RemoveRecordsWithoutGroupFromWebhooksTable: reverted (0.0000s)
Related to #202423 (closed)