XS Add sharding key YAML config to p_ci_finished_build_ch_sync_events
What we need to do
The p_ci_finished_build_ch_sync_events
has build_id
, so we can add project_id
and use ci_builds.project_id
to backfill p_ci_finished_build_ch_sync_events
How to do it
If we can get by with adding a LFK, that would be the easiest route. The lack of a partition_id
shouldn't be a blocker, since the p_ci_builds
table currently has only 3 partitions in .com.
Alternative
If we cannot use desired_sharding_key
, we can:
-
(!163926 (merged)) add a non-nullable project_id
column (default -1) and an LFK pointing to theprojects
table. The reason for a non-nullable column is to avoid extra work later on. It shouldn't be an issue since these records are getting deleted frequently, therefore in at most 30 days we'll have a table with all valid project IDs. -
(!163926 (merged)) add a validates :project_id, presence: true, on: :create
validation to ensure that theproject_id
field is present. -
(!164766 (merged)) after some days, merge an MR with a post-deployment migration truncating p_ci_finished_build_ch_sync_events
if ClickHouse unconfigured. Since .com has ClickHouse configured, this won't have an impact there. -
(!166733 (merged)) in %17.5, once all records are valid, change the validation to work at all times. -
(!166736 (merged)) remove column default. -
(!166736 (merged)) Add sharding_key
todb/docs/p_ci_finished_build_ch_sync_events.yml
Edited by Pedro Pombeiro