Increase the page size exponentially for the repository file tree batch requests

Problem to solve

For folders with very large amounts of files, the number of requests pile up and end up in a very long loading phase.

This can actually be applied both to the files and the last commit.

Further details

Some links where you can see the strain of performance:

• https://gitlab.com/gitlab-org/gitlab/tree/master/app/assets/images/emoji
• https://gitlab.com/ddevault/scdoc/-/tree/master/src (HN thread)

Proposal

Instead of always requesting the same number of items per request, we could increase the number of items per page.

The actual growth rate needs to be tested and tweaked to ensure a good balance between FE and BE performance.

If the page size isn't the best path, we can, instead, consider parallelising requests.

Permissions and Security

Availability & Testing

What does success look like, and how can we measure that?

• Browser doesn't freeze
• faster loading times for those critical pages.

Is this a cross-stage feature?

No, it should only impact devopscreate groupsource code.

Links / references