Implement syncing ci_project_mirrors and ci_namespace_mirrors tables
What does this MR do and why?
This MR is an updated and simplified version of !74899 (closed). In this MR, we only create tables and start syncing. The next MR will be about backfilling the mirror tables.
These tables will be shadow tables from the main DB to CI DB. In this MR, we
-
create=> !75621 (merged)ci_project_mirrors
andci_namespace_mirrors
tables - create
projects_sync_events
andnamespaces_sync_events
tables - create PG triggers to populate events when changing namespace or project
- introduce a feature flag
ci_namespace_project_mirrors
#346786.
Syncing the deletion is covered by loose foreign keys: !75559 (merged) and !75621 (merged)
Triggers
namespaces
When;
-
AFTER INSERT
ofnamespaces
-
AFTER UPDATE
ofnamespaces
with changing ofparent_id
Then;
INSERT INTO namespaces_sync_events
projects
When;
-
AFTER INSERT
ofprojects
-
AFTER UPDATE
ofprojects
with changing ofnamespace_id
Then;
INSERT INTO projects_sync_events
DB
UP
== 20211011140932 CreateNamespacesSyncEvents: migrating =======================
-- create_table(:namespaces_sync_events, {})
-> 0.0090s
== 20211011140932 CreateNamespacesSyncEvents: migrated (0.0090s) ==============
== 20211011141239 CreateProjectsSyncEvents: migrating =========================
-- create_table(:projects_sync_events, {})
-> 0.0080s
== 20211011141239 CreateProjectsSyncEvents: migrated (0.0081s) ================
== 20211011141242 CreateNamespacesSyncTrigger: migrating ======================
-- execute("CREATE OR REPLACE FUNCTION insert_namespaces_sync_event()\nRETURNS TRIGGER AS\n$$\nBEGIN\nINSERT INTO namespaces_sync_events (namespace_id)\nVALUES(COALESCE(NEW.id, OLD.id));\nRETURN NULL;\n\nEND\n$$ LANGUAGE PLPGSQL\n")
-> 0.0037s
-- execute("CREATE TRIGGER trigger_namespaces_parent_id_on_insert\nAFTER INSERT ON namespaces\nFOR EACH ROW\n\nEXECUTE FUNCTION insert_namespaces_sync_event()\n")
-> 0.0015s
-- execute("CREATE TRIGGER trigger_namespaces_parent_id_on_update\nAFTER UPDATE ON namespaces\nFOR EACH ROW\nWHEN (OLD.parent_id IS DISTINCT FROM NEW.parent_id)\n\nEXECUTE FUNCTION insert_namespaces_sync_event()\n")
-> 0.0009s
== 20211011141242 CreateNamespacesSyncTrigger: migrated (0.0063s) =============
== 20211011141243 CreateProjectsSyncTrigger: migrating ========================
-- execute("CREATE OR REPLACE FUNCTION insert_projects_sync_event()\nRETURNS TRIGGER AS\n$$\nBEGIN\nINSERT INTO projects_sync_events (project_id)\nVALUES(COALESCE(NEW.id, OLD.id));\nRETURN NULL;\n\nEND\n$$ LANGUAGE PLPGSQL\n")
-> 0.0016s
-- execute("CREATE TRIGGER trigger_projects_parent_id_on_insert\nAFTER INSERT ON projects\nFOR EACH ROW\n\nEXECUTE FUNCTION insert_projects_sync_event()\n")
-> 0.0013s
-- execute("CREATE TRIGGER trigger_projects_parent_id_on_update\nAFTER UPDATE ON projects\nFOR EACH ROW\nWHEN (OLD.namespace_id IS DISTINCT FROM NEW.namespace_id)\n\nEXECUTE FUNCTION insert_projects_sync_event()\n")
-> 0.0011s
== 20211011141243 CreateProjectsSyncTrigger: migrated (0.0041s) ===============
DOWN
== 20211011141243 CreateProjectsSyncTrigger: reverting ========================
-- execute("DROP TRIGGER IF EXISTS trigger_projects_parent_id_on_insert ON projects")
-> 0.0022s
-- execute("DROP TRIGGER IF EXISTS trigger_projects_parent_id_on_update ON projects")
-> 0.0009s
-> 0.0009s
-- execute("DROP FUNCTION IF EXISTS insert_projects_sync_event()")
-> 0.0007s
== 20211011141243 CreateProjectsSyncTrigger: reverted (0.0049s) ===============
== 20211011141242 CreateNamespacesSyncTrigger: reverting ======================
-- execute("DROP TRIGGER IF EXISTS trigger_namespaces_parent_id_on_insert ON namespaces")
-> 0.0022s
-- execute("DROP TRIGGER IF EXISTS trigger_namespaces_parent_id_on_update ON namespaces")
-> 0.0008s
-> 0.0008s
-- execute("DROP FUNCTION IF EXISTS insert_namespaces_sync_event()")
-> 0.0007s
== 20211011141242 CreateNamespacesSyncTrigger: reverted (0.0047s) =============
== 20211011141239 CreateProjectsSyncEvents: reverting =========================
-- drop_table(:projects_sync_events, {})
-> 0.0038s
== 20211011141239 CreateProjectsSyncEvents: reverted (0.0058s) ================
== 20211011140932 CreateNamespacesSyncEvents: reverting =======================
-- drop_table(:namespaces_sync_events, {})
-> 0.0036s
== 20211011140932 CreateNamespacesSyncEvents: reverted (0.0055s) ==============
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.
Edited by Furkan Ayhan