Cache highlighed merge request diffs file-by-file
Problem to solve
Our current merge request diff highlighting cache implementation caches all MR diff files highlighting data into a single key on Redis. That's bad when we want to read a single diff file and highlight it.
This issue goal is to refactor it in order to read/write a single or multiple MR diff files at once, similar to what we have at Gitlab::DiscussionsDiff::HighlightCache
.
Further details
Because we now have paginated APIs for loading the merge request diff #31290 (closed), the backend is doing needless heavy lifting for every page in the response. This reduces the performance of the API, this impacting how quickly the page loads, and is exerting more load on the server than before the pagination change.
Proposal
Cache highlighted merge request diffs on a file-by-file basis so that we can quickly retrieve only the data we need, rather than everything, and then discarding large amounts of data.