Skip to content

Ci Resouce Group models and parser (with migration fix)

Shinya Maeda requested to merge ci-resource-group-model-v2 into master

What does this MR do?

This MR cherry picks the reverted commit due to a migration failure. The problem is explained well in https://gitlab.com/gitlab-com/gl-infra/infrastructure/issues/8718#note_261176518 and !20950 (comment 261142220). But simply put,

  • We should not create a FK to a big table in transaction because it locks the table and make some queries (e.g. UPDATE) wait.
  • We should not execute FK creation and column addition to the same table in the same transaction because it causes deadlock.

Previously we had these migrations:

  1. db/migrate/20191128145231_add_ci_resource_groups.rb ... It creates ci_resource_groups and ci_resources. It adds FKs to the build_id and project_id columns. It also adds resource_group_id and waiting_for_resource_at columns to ci_builds. All happens in a single transaction thus failed.
  2. db/migrate/20191129144631_add_index_to_resource_group_id.rb ... It adds FK and index to ci_builds.resource_group_id to ci_resource_groups table.

And this MR introduces the following migrations, instead. Basically, splitting the 20191128145231_add_ci_resource_groups into four migrations. See

  1. db/migrate/20191128145231_add_ci_resource_groups.rb ... It creates ci_resource_groups and ci_resources tables in transaction. It doesn't add FK at this moment.
  2. db/migrate/20191128145232_add_fk_to_ci_resources_build_id.rb ... It adds FK to ci_resources.build_id to ci_builds table with add_concurrent_foreign_key.
  3. db/migrate/20191128145233_add_fk_to_ci_resource_groups_project_id.rb ... It adds FK to ci_resource_groups.project_id to projects table with add_concurrent_foreign_key.
  4. db/migrate/20191129144630_add_resource_group_id_to_ci_builds.rb ... It adds resource_group_id and waiting_for_resource_at columns to ci_builds in transaction.
  5. db/migrate/20191129144631_add_index_to_resource_group_id.rb ... It adds FK and index to ci_builds.resource_group_id to ci_resource_groups table.

Screenshots

Does this MR meet the acceptance criteria?

Conformity

Availability and Testing

Edited by Mayra Cabrera

Merge request reports