Refactor backend logic for storage limit notification
What does this MR do and why?
Relates to https://gitlab.com/gitlab-org/gitlab/-/issues/349042
This is the second MR in order to amend the storage notifications. I wanted to refactor this code so it's easier to understand as we progress with this feature.
MR | description |
---|---|
!89455 (merged) | Remove CE code |
!88912 (merged) |
|
!91878 (merged) | Refactor banner into ViewComponent and add user_callouts for each threshold |
!92642 (merged) | Add remaining details to storage notification banner |
This is not the first time I spend hours reasoning this code (had this same bad experience when working on !73410 (merged)) so now that I have more context I decided to refactor it.
Screenshots or screen recordings
Screenshots will be added in the next section, since each banner variation has a different setup I thought would be better to couple the setup and screenshots
How to set up and validate locally
First, apologies for the long description. This code was really hard for me to follow (and one of the main reasons I wanted to refactor it) and this comes from my personal notes (and previous work !73410 (merged)) when trying to reason this code.
Before we start, let's make sure we have:
::Gitlab::CurrentSettings.update(automatic_purchased_storage_allocation: true)
::Gitlab::CurrentSettings.update(check_namespace_plan: true)
- Make sure you're simulating SaaS
Repository level storage notifications
This section covers all the code/logic inside CheckExcessStorageSizeService
. I know the name of this service suggests something else, I tried to clear that up in the refactoring.
- This is the legacy lock, so we need to disable the FF:
Feature.disable(:namespace_storage_limit)
First banner - project locked due to project-level (repository) storage limit
-
To make sure we lock projects over the storage limit, let's add a 10 Bytes limit:
::Gitlab::CurrentSettings.update!(repository_size_limit: 10)
-
Now let's add some storage statistics to the project so it goes over the limit and gets locked:
p = Project.find(<your project id here>) p.statistics.update( repository_size: 3900000000, lfs_objects_size: 4800000000, build_artifacts_size: 400000000, wiki_size: 300000000, packages_size: 3800000000 )
-
Now if you navigate to the group's home page you should see the banner.
-
To toggle the message let's add some additional purchased storage
n = Group.find(<your group id here>) n.update(additional_purchased_storage_size: 15)
Second banner - namespace usage notification based on additional purchase + project-level (repository) storage limit
To see the remaining code flow in check_excess_storage_size_service.rb we need to have contains_locked_projects? return false. In order to see that we need to have a bigger additional_purchased_storage_size
. To do this, on the same group as above:
- Check what you have as
total_repository_size_excess
(on rails console:Group.find(<your group id here>).total_repository_size_excess
), - Have
X.megabites
bigger thantotal_repository_size_excess
.- For example, in my case
total_repository_size_excess
is8 699 999 990
, so I'll use 8700 because8700.megabytes
is9 122 611 200
- You can also toggle this difference so we can test different usage thresholds (50%, 75%, 95%, and 100%) but having 100% could trigger the other banner depending on the rounding.
- For example, in my case
- Then update the namespace accordingly
Group.find(232).update(additional_purchased_storage_size: 8700)
- You should see a banner with the following message:
Namespace level storage notifications
These banners are based on the plan
limit - as part of this bigger effort https://gitlab.com/groups/gitlab-org/-/epics/6587.
- First step is to enable the FF that toggles the type of enforcement from
repository
tonamespace
storage:Feature.enable(:namespace_storage_limit)
- This will make the repository-level storage banners (mentioned above) to stop showing
- Next we need to enable other feature flags that control other parts of this feature
Feature.enable(:namespace_storage_limit_bypass_date_check)
Feature.enable(:enforce_storage_limit_for_paid)
Feature.enable(:enforce_storage_limit_for_free)
- Let's add a limit to the free plan (
10
here means10 megabytes
):Plan.default.actual_limits.update!(storage_size_limit: 10)
Plan.free.actual_limits.update!(storage_size_limit: 10)
- Now let's add some namespace storage statistics. If the limit is
10.megabytes
(10485760
) here are some examples:-
info banner (
50%
):5242880
-
warning banner (
75%
):7864320
-
alert banner (
95%
):9961472
-
error banner (
100%
):10485760
-
error banner over limit (
100+%
):11000000
- You can use these numbers in one storage statistic, or distributed amongst them. Here's an example with 95% storage used:
n = Group.find(238) n.root_storage_statistics&.destroy! s = Namespace::RootStorageStatistics.new( namespace: n, build_artifacts_size: 0, wiki_size: 0, repository_size: 0, packages_size: 0, lfs_objects_size: 9961472 ) s.storage_size = s.repository_size + s.lfs_objects_size + s.wiki_size + s.build_artifacts_size + s.packages_size s.save!
-
info banner (
With that we can test these banners before and after the refactoring, and assert they're working fine.
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.