Convert events.id to bigint - Step 1: Add new columns and sync data
events
is the largest table in GitLab's database that is defined using an integer (int4
) Primary Key.
On November 24th, we are at 47% capacity used
# select current_date, max(id) as current_max, 2147483647 as max_integer, ((max(id) / 2147483647.0) * 100)::integer as percentage_used from events;
current_date | current_max | max_integer | percentage_used
--------------+-------------+-------------+-----------------
2020-11-24 | 1003520786 | 2147483647 | 47
The first step to address the problem is to create a new column of type bigint
, load all the data by using background jobs from the current id
column to it and keep them in sync with a trigger.
We have to also follow the same approach for all tables that reference events.id
, in this case push_event_payloads.event_id
So the overall process in %13.7 will be as follows:
-
Create new column
events.new_id
, withbigint NOT NULL DEFAULT 0
-
Install sync triggers (
install_rename_triggers
should work here) -
Start background
CopyColumn
jobs (we'll update it to use our new background migration jobs framework if we have not) -
Same for the column
push_event_payloads.event_id
that referencesevents.id
We'll follow that with a cleanup migration in %13.7 and then add the necessary indexes and swap the PK and FK.
Related Tracking Issue: #220023 (closed)