RemoveExpiredMembersWorker fails due to a slow query
Over the past 7 days, the worker failed twice in row. It fails with:
PG::QueryCanceled: ERROR: canceling statement due to statement timeout
In the slow query log, the following query times out:
"database_name": "gitlabhq_production",
"db_config_name": "main",
"error_severity": "ERROR",
"message": "canceling statement due to statement timeout",
"sql": "SELECT \"members\".* FROM \"members\" WHERE \"members\".\"expires_at\" <= $1 ORDER BY \"members\".\"id\" ASC LIMIT $2",
"jid": "196f9348035954550d8f8bb4",
"time": "2024-07-04T00:10:19.122Z",
"correlation_id": "086681ce54bacd01ee27b9870c8fa239",
"endpoint_id": "RemoveExpiredMembersWorker",
Query plan: https://console.postgres.ai/gitlab/gitlab-production-main/sessions/29654/commands/92041
There are 2762
expired members currently in the table:
gitlabhq_production=> select count(*) from members where expires_at <= '2024-07-04';
count
-------
2762
(1 row)