Prevent Web IDE Commits when Namespace Storage Limits are Exceeded

What does this MR do and why?

Prevent commits via the Web IDE when namespace storage limits are exceeded.

This MR also prevents commits via a number of other routes including the Repository files API and the Commits API.

Issue: https://gitlab.com/gitlab-org/gitlab/-/issues/362937

Screenshots or screen recordings

Commit is rejected when over the storage limit

RejectWebIDECommit-small

Commit is accepted when under the storage limit. A commit that would exceed the limit is rejected.

RejectWebIDECommit2MB-small

How to set up and validate locally

Basic Setup

1. Make sure the automatic_purchased_storage_allocation and enforce_namespace_storage_limit application settings are both enabled (true).

gitlabhq_development=# SELECT automatic_purchased_storage_allocation, enforce_namespace_storage_limit FROM application_settings;
 automatic_purchased_storage_allocation | enforce_namespace_storage_limit 
----------------------------------------+---------------------------------
 t                                      | t
(1 row)

gitlabhq_development=# 

2. Enable the :namespace_storage_limit, :enforce_storage_limit_for_paid, :enforce_storage_limit_for_free, and :namespace_storage_limit_bypass_date_check feature flags in a rails console.

[1] pry(main)> Feature.enable(:namespace_storage_limit)
[...]
[2] pry(main)> Feature.enable(:enforce_storage_limit_for_paid)
[...]
[3] pry(main)> Feature.enable(:enforce_storage_limit_for_free)
[...]
[4] pry(main)> Feature.enable(:namespace_storage_limit_bypass_date_check)
[...]
[5] pry(main)> 

3. Apply the following patch to your local gitlab instance. This patch disables some caching around storage size and storage limits.

diff --git a/ee/app/models/ee/namespace/root_storage_size.rb b/ee/app/models/ee/namespace/root_storage_size.rb
index fd612db00ec..b37918bdfbc 100644
--- a/ee/app/models/ee/namespace/root_storage_size.rb
+++ b/ee/app/models/ee/namespace/root_storage_size.rb
@@ -26,16 +26,16 @@ def usage_ratio
     end
 
     def current_size
-      @current_size ||= Rails.cache.fetch(['namespaces', root_namespace.id, CURRENT_SIZE_CACHE_KEY], expires_in: EXPIRATION_TIME) do
+      # @current_size ||= Rails.cache.fetch(['namespaces', root_namespace.id, CURRENT_SIZE_CACHE_KEY], expires_in: EXPIRATION_TIME) do
         root_namespace.root_storage_statistics&.storage_size
-      end
+      # end
     end
 
     def limit
-      @limit ||= Rails.cache.fetch(['namespaces', root_namespace.id, LIMIT_CACHE_KEY], expires_in: EXPIRATION_TIME) do
+      # @limit ||= Rails.cache.fetch(['namespaces', root_namespace.id, LIMIT_CACHE_KEY], expires_in: EXPIRATION_TIME) do
         root_namespace.actual_limits.storage_size_limit.megabytes +
             root_namespace.additional_purchased_storage_size.megabytes
-      end
+      # end
     end
 
     def remaining_storage_percentage

4. Restart your GDK simulating saas mode with $ GITLAB_SIMULATE_SAAS=1 gdk start.

5. Navigate to your project's group page. In the example here, my project is "Storage Test Project" under the "Storage Test Group" group so I navigate to the "Storage Test Group" page, pictured below.

Navigate to Settings > Usage Quotas. Click the Storage tab. You can see how much total namespace storage you currently have here.

This can give you a good idea of what number to use in the SQL UPDATE statements in the following steps.

6. Find the plan_limit.id of your local plan_limit for the free plan. (The project in these examples is in a group on the free plan. You may need to use a different plan if your project is on an Ultimate or Premium plan.)

gitlabhq_development=# SELECT p.id, p.name, p.title, l.id, l.storage_size_limit FROM plans AS p JOIN plan_limits AS l ON p.id = l.plan_id;
 id |      name      |           title           | id | storage_size_limit 
----+----------------+---------------------------+----+--------------------
  1 | default        | Default                   |  1 |                  0
  2 | bronze         | Bronze                    |  6 |                  0
  3 | silver         | Silver                    |  7 |                  0
  4 | gold           | Gold                      |  8 |                  0
  5 | free           |                           |  5 |                 30
  6 | premium        | Premium (Formerly Silver) | 23 |                  0
  7 | ultimate       | Ultimate (Formerly Gold)  | 26 |                  0
  8 | ultimate_trial | Ultimate Trial            |  3 |                  0
  9 | premium_trial  | Premium Trial             |  4 |                  0
 10 | opensource     | Open Source Program       | 28 |                  0
(10 rows)

gitlabhq_development=# 

Validate that Web IDE commits are rejected with an exceeded namespace storage limit

1. Update the storage limit. Pick a number below the total namespace storage.

gitlabhq_development=# UPDATE plan_limits SET storage_size_limit = 3 WHERE id = 5;
UPDATE 1
gitlabhq_development=# 

2. Navigate to the Web IDE for the project. Try to edit and commit a change to a file. The change is rejected.

Validate that Web IDE commits are accepted when under the namespace storage limit. But changes that would exceed the limit are rejected.

1. Update the storage limit. Pick a number just over the total namespace storage. In this example, we set the limit to 12, since the total namespace storage is 11.4 MB.

gitlabhq_development=# UPDATE plan_limits SET storage_size_limit = 12 WHERE id = 5;
UPDATE 1
gitlabhq_development=# 

2. Navigate to the Web IDE for the project. Try to edit and commit a change to a file. The change is accepted.

3. Try to upload and commit a file of at least 1 MB in size. The change is rejected.

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

• I have evaluated the MR acceptance checklist for this MR.