Improve performance of Projects List API under load into next tier

The Projects List API is showing to perform over our new performance target of 500ms.

We (Quality) have recently expanded performance test coverage to include "horizontal" test areas like Projects in a consistent way. As such we now test the API on our 10k Reference Environment against 250 groups with 10 projects each (a total of 2500 projects) at 200 RPS:

* Environment:                10k
* Environment Version:        13.2.0-pre `744484487f2`
* Option:                     60s_200rps
* Date:                       2020-06-30
* Run Time:                   1h 4m 21.39s (Start: 01:27:22 UTC, End: 02:31:44 UTC)
* GPT Version:                v2.0.3

NAME                                                     | RPS   | RPS RESULT           | TTFB AVG   | TTFB P90              | REQ STATUS      | RESULT 
---------------------------------------------------------|-------|----------------------|------------|-----------------------|-----------------|--------
api_v4_projects                                          | 200/s | 50.68/s (>32.00/s)   | 3490.03ms  | 4244.82ms (<7500ms)   | 100.00% (>95%)  | Passed 

Metrics:

For this endpoint in particular there's some more data to point out. On gitlab.com, data looks to show it performing substantially worse than we see on our smaller environments:

Worst performing Rails API routes in the applications, by p95 latency

/api/:version/projects

p95 latency for this route: 15s (find related issues)

The endpoint appears to be then exponential in how it degrades as the number of projects rise. Although .com is indeed an outlier and has a significant number of projects.

As part of the new performance targets this endpoint is currently over our TTFB target and is close enough that it falls just into the ~S2 tier. Although if you want to weigh .com more it could be considered higher. But .com is a live environment with many moving parts so it's not completely reliable so this will be raised as a S3.

Task is to reduce the performance of this endpoint consistently to the next tier of ~S3 (<2000ms).

To-do:

!66598 (closed), blocked by:

!67936 (closed), blocked by:

we need to Introduce a special version of EachBatch that can use composite row comparison.

Doing:

Done: