Migrate compliance frameworks to the new table
What does this MR do?
Related issue: #251113 (closed)
This MR connects the newly introduced ComplianceManagement::Framework
model with the ComplianceManagement::ComplianceFramework::ProjectSettings
model in a backward compatible manner. It also fixes a mistake with the original schema: we should associate the ComplianceManagement::Framework
model with Namespace
and not with the Group
.
Goal (not part of this MR): allow groups to customize their compliance frameworks and migrate the existing data (enum) to the newly created table.
How to test it
There are no user-facing changes.
- Log in to a local gitlab instance with ultimate license.
- Go to a project / Settings / General
- Pick a
Compliance framework
- Check the logs and verify that the
framework
andframework_id
columns are being set. - Check if an
INSERT
happens tocompliance_management_frameworks
table.
Development
The current status (no connection):
This MR:
End goal (next MR/milestone):
Database migration
- Add
namespace_id
column. - Fix the mistake where we used
group_id
column instead ofnamespace_id
.- Remove the FK, remove the NOT NULL constraint and ignore the column (
group_id
)
- Remove the FK, remove the NOT NULL constraint and ignore the column (
- Add
framework_id
column. - Migrate the data from the
framework
enum.- About 1000 records to be created.
- Add not null constraint on the
framework_id
column
Note: I tested it on postgres.ai, the migrations ran without errors.
Up:
== 201005092703 AddNamespaceColumnToFrameworks: migrating =====================
-- column_exists?(:compliance_management_frameworks, :namespace_id)
-> 0.0011s
-- add_column(:compliance_management_frameworks, :namespace_id, :integer)
-> 0.0024s
-- change_column_null(:compliance_management_frameworks, :namespace_id, false)
-> 0.0011s
-- transaction_open?()
-> 0.0000s
-- foreign_keys(:compliance_management_frameworks)
-> 0.0023s
-- execute("ALTER TABLE compliance_management_frameworks\nADD CONSTRAINT fk_b74c45b71f\nFOREIGN KEY (namespace_id)\nREFERENCES namespaces (id)\nON DELETE CASCADE\nNOT VALID;\n")
-> 0.0010s
-- execute("SET statement_timeout TO 0")
-> 0.0001s
-- execute("ALTER TABLE compliance_management_frameworks VALIDATE CONSTRAINT fk_b74c45b71f;")
-> 0.0017s
-- execute("RESET ALL")
-> 0.0002s
-- transaction_open?()
-> 0.0000s
-- index_exists?(:compliance_management_frameworks, [:namespace_id, :name], {:unique=>true, :name=>"idx_on_compliance_management_frameworks_namespace_id_name", :algorithm=>:concurrently})
-> 0.0017s
-- add_index(:compliance_management_frameworks, [:namespace_id, :name], {:unique=>true, :name=>"idx_on_compliance_management_frameworks_namespace_id_name", :algorithm=>:concurrently})
-> 0.0136s
== 201005092703 AddNamespaceColumnToFrameworks: migrated (0.0321s) ============
== 201005092709 RemoveComplianceFrameworksGroupIdFk: migrating ================
-- change_column_null(:compliance_management_frameworks, :group_id, true)
-> 0.0024s
-- foreign_keys(:compliance_management_frameworks)
-> 0.0083s
-- remove_foreign_key(:compliance_management_frameworks, :namespaces, {:column=>:group_id})
-> 0.0115s
-- transaction_open?()
-> 0.0000s
-- indexes(:compliance_management_frameworks)
-> 0.0053s
-- remove_index(:compliance_management_frameworks, {:algorithm=>:concurrently, :name=>"index_compliance_management_frameworks_on_group_id_and_name"})
-> 0.0039s
== 201005092709 RemoveComplianceFrameworksGroupIdFk: migrated (0.0332s) =======
== 20201005092753 AddFrameworkIdToProjectFrameworkSettings: migrating =========
-- column_exists?(:project_compliance_framework_settings, :framework_id)
-> 0.0031s
-- add_column(:project_compliance_framework_settings, :framework_id, :bigint)
-> 0.0012s
-- transaction_open?()
-> 0.0000s
-- index_exists?(:project_compliance_framework_settings, :framework_id, {:algorithm=>:concurrently})
-> 0.0046s
-- add_index(:project_compliance_framework_settings, :framework_id, {:algorithm=>:concurrently})
-> 0.0125s
-- transaction_open?()
-> 0.0000s
-- foreign_keys(:project_compliance_framework_settings)
-> 0.0073s
-- execute("ALTER TABLE project_compliance_framework_settings\nADD CONSTRAINT fk_be413374a9\nFOREIGN KEY (framework_id)\nREFERENCES compliance_management_frameworks (id)\nON DELETE CASCADE\nNOT VALID;\n")
-> 0.0018s
-- execute("ALTER TABLE project_compliance_framework_settings VALIDATE CONSTRAINT fk_be413374a9;")
-> 0.0033s
== 20201005092753 AddFrameworkIdToProjectFrameworkSettings: migrated (0.0479s)
== 20201005094331 MigrateComplianceFrameworkEnumToDatabaseFrameworkRecord: migrating
== 20201005094331 MigrateComplianceFrameworkEnumToDatabaseFrameworkRecord: migrated (0.0124s)
== 20201005153955 AddNotNullConstraintToComplianceProjectSettings: migrating ==
-- current_schema()
-> 0.0008s
-- transaction_open?()
-> 0.0000s
-- current_schema()
-> 0.0007s
-- execute("ALTER TABLE project_compliance_framework_settings\nADD CONSTRAINT check_d348de9e2d\nCHECK ( framework_id IS NOT NULL )\nNOT VALID;\n")
-> 0.0012s
-- current_schema()
-> 0.0007s
-- execute("ALTER TABLE project_compliance_framework_settings VALIDATE CONSTRAINT check_d348de9e2d;")
-> 0.0023s
== 20201005153955 AddNotNullConstraintToComplianceProjectSettings: migrated (0.0314s)
Down:
== 20201005153955 AddNotNullConstraintToComplianceProjectSettings: reverting ==
-- execute("ALTER TABLE project_compliance_framework_settings\nDROP CONSTRAINT IF EXISTS check_d348de9e2d\n")
-> 0.0005s
== 20201005153955 AddNotNullConstraintToComplianceProjectSettings: reverted (0.0037s)
== 20201005094331 MigrateComplianceFrameworkEnumToDatabaseFrameworkRecord: reverting
== 20201005094331 MigrateComplianceFrameworkEnumToDatabaseFrameworkRecord: reverted (0.0000s)
== 20201005092753 AddFrameworkIdToProjectFrameworkSettings: reverting =========
-- foreign_keys(:project_compliance_framework_settings)
-> 0.0025s
-- remove_foreign_key(:project_compliance_framework_settings, :compliance_management_frameworks, {:column=>:framework_id})
-> 0.0037s
-- remove_column(:project_compliance_framework_settings, :framework_id)
-> 0.0004s
== 20201005092753 AddFrameworkIdToProjectFrameworkSettings: reverted (0.0110s)
== 201005092709 RemoveComplianceFrameworksGroupIdFk: reverting ================
-- change_column_null(:compliance_management_frameworks, :group_id, false)
-> 0.0016s
-- transaction_open?()
-> 0.0000s
-- foreign_keys(:compliance_management_frameworks)
-> 0.0027s
-- execute("ALTER TABLE compliance_management_frameworks\nADD CONSTRAINT fk_d3240d6339\nFOREIGN KEY (group_id)\nREFERENCES namespaces (id)\nON DELETE CASCADE\nNOT VALID;\n")
-> 0.0011s
-- execute("SET statement_timeout TO 0")
-> 0.0002s
-- execute("ALTER TABLE compliance_management_frameworks VALIDATE CONSTRAINT fk_d3240d6339;")
-> 0.0021s
-- execute("RESET ALL")
-> 0.0002s
-- transaction_open?()
-> 0.0000s
-- index_exists?(:compliance_management_frameworks, [:group_id, :name], {:unique=>true, :name=>"index_compliance_management_frameworks_on_group_id_and_name", :algorithm=>:concurrently})
-> 0.0018s
-- add_index(:compliance_management_frameworks, [:group_id, :name], {:unique=>true, :name=>"index_compliance_management_frameworks_on_group_id_and_name", :algorithm=>:concurrently})
-> 0.0169s
== 201005092709 RemoveComplianceFrameworksGroupIdFk: reverted (0.0316s) =======
== 201005092703 AddNamespaceColumnToFrameworks: reverting =====================
-- transaction_open?()
-> 0.0000s
-- indexes(:compliance_management_frameworks)
-> 0.0021s
-- execute("SET statement_timeout TO 0")
-> 0.0003s
-- remove_index(:compliance_management_frameworks, {:algorithm=>:concurrently, :name=>"idx_on_compliance_management_frameworks_namespace_id_name"})
-> 0.0022s
-- execute("RESET ALL")
-> 0.0002s
-- foreign_keys(:compliance_management_frameworks)
-> 0.0022s
-- remove_foreign_key(:compliance_management_frameworks, :namespaces, {:column=>:namespace_id})
-> 0.0033s
-- remove_column(:compliance_management_frameworks, :namespace_id)
-> 0.0009s
== 201005092703 AddNamespaceColumnToFrameworks: reverted (0.0116s) ============
Does this MR meet the acceptance criteria?
Conformity
-
Changelog entry -
Documentation (if required) -
Code review guidelines -
Merge request performance guidelines -
Style guides -
Database guides -
Separation of EE specific content
Availability and Testing
-
Review and add/update tests for this feature/bug. Consider all test levels. See the Test Planning Process. -
Tested in all supported browsers -
Informed Infrastructure department of a default or new setting change, if applicable per definition of done
Security
If this MR contains changes to processing or storing of credentials or tokens, authorization and authentication methods and other items described in the security review guidelines:
-
Label as security and @ mention @gitlab-com/gl-security/appsec
-
The MR includes necessary changes to maintain consistency between UI, API, email, or other methods -
Security reports checked/validated by a reviewer from the AppSec team