Paginated API results could return duplicates and skip other results
Zendesk: https://gitlab.zendesk.com/agent/tickets/85405
Customer reports that when they iterated through 3,000+ users via the API, they encountered some duplicates. As a result of the duplicates they also were missing some other users. The total records match what they expect for total users.
In looking at this I notice that we using Rails #page
and #per
methods. I wonder if the duplication is because we're not specifying an order on :id
. Each subsequent query is not guaranteed to return items in the same order so weird things could happen during pagination. For example, what happens if a new user gets added during the iteration through user pages - if the call returns newer users first by default then the new user is going to cause each additional page to be shifted by one result. This is my working theory and hasn't been proven yet. I'm open to other theories.