Add keyset pagination to group audit events api
What does this MR do and why?
-
Adds optional support for keyset pagination for the
[GET] /api/v4/groups/:id/audit_events
endpoint. -
This is a non-breaking change as the default behviour does not change an existing API consumers need not make any changes.
-
Offset pagination for groups with large numbers of group-level audit events often causes timeouts when querying above record 50,000 (ish) due to poor database performance. (See postgres.ai internal link https://console.postgres.ai/gitlab/gitlab-production-tunnel-pg12/sessions/10946/commands/39346)
-
A similar query using offset pagination is many orders of magnitude faster: https://console.postgres.ai/gitlab/gitlab-production-tunnel-pg12/sessions/10946/commands/39350 (22ms)
How to set up and validate locally
- Create a group (or use one that already exists)
- Ensure that it contains a lot of group-level audit events.
- Make an API query such as this (as a group owner or instance administrator)
GET /api/v4/groups/35/audit_events?per_page=2&pagination=keyset&order_by=created_at
- In the response headers, check the
Link
header which should be in this format:
<http://127.0.0.1:3000/api/v4/groups/35/audit_events?cursor=eyJpZCI6IjIwMTYzOSIsIl9rZCI6Im4ifQ%3D%3D&id=35&order_by=created_at&page=1&pagination=keyset&per_page=2>; rel="next"
- Make the same request again, adding
cursor={CURSOR_FROM_HEADER_ABOVE}
to the URL params and check that the results are the next ones in the set.
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.
Relates to #333968 (closed)