Skip to content

Set retention period of 1 month for web_hook_logs

Max Orefice requested to merge morefice/adjust-webhook-partition-size into master

Ref: https://gitlab.com/gitlab-org/gitlab/-/issues/498252

What does this MR do and why?

First step towards improving data retention for web_hook_logs.

This MR updates the retention policy to retain only 1 month of data as we are only filtering the last 7 days.

This will free up ~2TB on production.

Testing

  1. Open a psql console to check the number of partitions for web_hook_logs
  2. Sync up all partitions bin/rails runner "Gitlab::Database::Partitioning.sync_partitions_ignore_db_error"
  3. Check that 2 partitions are now gone
gitlabhq_development=# \d+ web_hook_logs
                                                                   Partitioned table "public.web_hook_logs"
Partitions:
            gitlab_partitions_dynamic.web_hook_logs_202409 FOR VALUES FROM ('2024-09-01 00:00:00') TO ('2024-10-01 00:00:00'),
            gitlab_partitions_dynamic.web_hook_logs_202410 FOR VALUES FROM ('2024-10-01 00:00:00') TO ('2024-11-01 00:00:00'),
            gitlab_partitions_dynamic.web_hook_logs_202411 FOR VALUES FROM ('2024-11-01 00:00:00') TO ('2024-12-01 00:00:00'),
            gitlab_partitions_dynamic.web_hook_logs_202412 FOR VALUES FROM ('2024-12-01 00:00:00') TO ('2025-01-01 00:00:00'),
            gitlab_partitions_dynamic.web_hook_logs_202501 FOR VALUES FROM ('2025-01-01 00:00:00') TO ('2025-02-01 00:00:00'),
            gitlab_partitions_dynamic.web_hook_logs_202502 FOR VALUES FROM ('2025-02-01 00:00:00') TO ('2025-03-01 00:00:00'),
            gitlab_partitions_dynamic.web_hook_logs_202503 FOR VALUES FROM ('2025-03-01 00:00:00') TO ('2025-04-01 00:00:00'),
            gitlab_partitions_dynamic.web_hook_logs_202504 FOR VALUES FROM ('2025-04-01 00:00:00') TO ('2025-05-01 00:00:00')
Edited by Max Orefice

Merge request reports