Fallback to alternative estimation method when measuring namespace usage
Context
Part of Estimated registry usage for very large namespaces (&9413).
In Container Registry: Scaling limitations on top-... (#779 - closed) we have identified a performance/scaling issue that leads to failures when calculating the deduplicated registry usage with maximum precision for very large namespaces (~1%).
Among the several mitigation strategies identified in #779 (comment 1179923688), this epic focuses on delivering the registry changes for the first one, Option A
.
Option A
consists in falling back to a faster usage calculation method (less accurate) when the main one (maximum accuracy) fails due to the size of the target namespace. This alternative calculation method consists in ignoring which images are tagged or not (with the default method we only account for image layers that are tagged at least once).
Tasks
-
Add support for skipping post-deployment migrations in CNG (gitlab-org/build/CNG!1208 (merged)) -
Skip post-deployment migrations by default in Charts (gitlab-org/charts/gitlab!2867 (closed)) -
Add new indexes required to speed up the new simplified query (!1163 (merged)) -
Production change request to apply post-deployment migrations with new indexes -
Introduce the new query and fallback to it if the main one fails, flagging it on API responses