Improve efficiency of serialized diff caching for merge requests
In gitlab-com/gl-infra/scalability#1601 (comment 1015069256) we've seen that caches for the PaginatedDiffSerializer
account for more than 5GB of memory used. This seems to come from Projects::MergeRequests::DiffsController#diffs_batch
.
Similar to #367098 (closed), this cache is also scoped to the current user and how they prefer to view. Meaning that this cache can't be shared across users.
Furthermore, the data from the diffs is also cached in Redis another level down. I believe this cache can be reused across users, but this one duplicates the information across users.
The cache namespace cache:gitlab:PaginatedDiffSerializer:
accounted for about 13% of the memory usage of the total cache:
namespace
Proposal
Since this part of the cache is also bound to the user and their preferences. Would it make sense to use ETag caching for this as well?