Recreate async index `index_vuln_findings_on_uuid_including_vuln_id`
requested to merge 413010-recreate-asychronous-index-index_vuln_findings_on_uuid_including_vuln_id into master
What does this MR do and why?
This migration was originally added for issue 390198. The migration ended up failing due to white-space being included in the SQL definition.
To prevent this going forward:
- a
before_validation
callback was added to the async model class to strip whitespace -
definition.to_s.strip
was added to the migration helper as suggested by this comment
to correct the existing records in production
- a migration was added that updates the existing records by calling
strip
on thedefinition
column
Changelog: fixed
How to set up and validate locally
- before running the migration, you can create an invalid record in the database with the following lambda:
lambda { index_name = 'index_vuln_findings_on_uuid_including_vuln_id' index_sql = <<-SQL CREATE UNIQUE INDEX CONCURRENTLY #{index_name} ON vulnerability_occurrences (uuid) include (vulnerability_id); SQL async_index = Gitlab::Database::AsyncIndexes::PostgresAsyncIndex.find_or_create_by!(name: index_name) do |rec| rec.table_name = :vulnerability_occurrences rec.definition = index_sql end async_index.update_column(:definition, index_sql) }.call
- check that an async index was created and that the
definition
field starts with whitespace[3] pry(main)> Gitlab::Database::AsyncIndexes::PostgresAsyncIndex.all => [#<Gitlab::Database::AsyncIndexes::PostgresAsyncIndex:0x00007fdf6021b2d0 id: 10, created_at: Mon, 10 Jul 2023 23:38:47.973580000 UTC +00:00, updated_at: Mon, 10 Jul 2023 23:38:47.973580000 UTC +00:00, name: "index_vuln_findings_on_uuid_including_vuln_id", definition: " CREATE UNIQUE INDEX CONCURRENTLY index_vuln_findings_on_uuid_including_vuln_id ON vulnerability_occurrences (uuid) include (vulnerability_id);\n", table_name: "vulnerability_occurrences", attempts: 0, last_error: nil>]
- run the migration:
bundle exec rails db:migrate
- check the db again, the whitespace should have been removed:
[1] pry(main)> Gitlab::Database::AsyncIndexes::PostgresAsyncIndex.all => [#<Gitlab::Database::AsyncIndexes::PostgresAsyncIndex:0x00007ff58c287718 id: 10, created_at: Mon, 10 Jul 2023 23:38:47.973580000 UTC +00:00, updated_at: Mon, 10 Jul 2023 23:53:44.272120000 UTC +00:00, name: "index_vuln_findings_on_uuid_including_vuln_id", definition: "CREATE UNIQUE INDEX CONCURRENTLY index_vuln_findings_on_uuid_including_vuln_id ON vulnerability_occurrences (uuid) include (vulnerability_id);", table_name: "vulnerability_occurrences", attempts: 0, last_error: nil>] [2] pry(main)>
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.
Related to #413010 (closed)
Edited by Michael Becker