Skip to content

Draft: Add resource_usage_limits column to application_settings

Sylvester Chin requested to merge sc1-define-resource-limit into master

What does this MR do and why?

This MR adds a resource_usage_limits column to application_settings. Referencing https://docs.gitlab.com/ee/development/application_settings.html#add-a-new-application-setting,

  • Add a new JSON column to the application_settings table to store
  • Add a test for the default value, if the setting has a default value.
  • Add a validation for the new field to the ApplicationSetting model.
  • Add a model test for the validation and default value
  • Add the new setting to it to ApplicationSettingImplementation#defaults, if the setting has a default value.

This MR borrows the idea from !169103 (comment 2158582170) to store a jsonb in application_settings for SRE/admin use initially until an external ratelimiting service is in place. There are no view changes for now since it is meant for SREs to control sidekiq resource usage limits.

References

Please include cross links to any resources that are relevant to this MR This will give reviewers and future readers helpful context to give an efficient review of the changes introduced.

MR acceptance checklist

Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Screenshots or screen recordings

Screenshots are required for UI changes, and strongly recommended for all other merge requests.

Before After

How to set up and validate locally

  1. Apply this diff so that we can test it locally.
diff --git a/app/helpers/application_settings_helper.rb b/app/helpers/application_settings_helper.rb
index d123f16fcc29..922420b97df2 100644
--- a/app/helpers/application_settings_helper.rb
+++ b/app/helpers/application_settings_helper.rb
@@ -540,9 +540,10 @@ def visible_attributes
       :require_personal_access_token_expiry,
       :observability_backend_ssl_verification_enabled
     ].tap do |settings|
-      unless Gitlab.com?
         settings << :resource_usage_limits
         settings << :resource_usage_limits_raw
+      unless Gitlab.com?
+
         settings << :deactivate_dormant_users
         settings << :deactivate_dormant_users_period
         settings << :nuget_skip_metadata_url_validation

Prepare a file rule.txt

{
  "rules": [
    {
      "name": "hi",
      "rules": [
        {
          "interval": 60,
          "selector": "worker_name=PipelineProcessWorker",
          "threshold": 1000
        },
        {
          "interval": 60,
          "selector": "*",
          "threshold": 100
        }
      ],
      "scopes": [
        "worker_name"
      ],
      "metadata": {
        "db_config_name": "ci"
      },
      "resource_key": "db_ci_duration_s"
    }
  ]
}
  1. Run gdk restart rails
  2. Create a PAT using the root account in gdk.
  3. Make a PUT request
➜  gitlab git:(sc1-define-resource-limit) ✗ curl  --header "PRIVATE-TOKEN: XXXXX" 'localhost:3000/api/v4/application/settings?resource_usage_limits' | jq '.resource_usage_limits'
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 17668  100 17668    0     0  13831      0  0:00:01  0:00:01 --:--:-- 13835
{}

➜  gitlab git:(sc1-define-resource-limit) ✗ curl --request PUT --header "PRIVATE-TOKEN: XXXX" "localhost:3000/api/v4/application/settings?resource_usage_limits_raw=$(cat rule.txt| jq -c . | jq -sRr @uri)"
....

➜  gitlab git:(sc1-define-resource-limit) ✗ curl  --header "PRIVATE-TOKEN: XXXXX" 'localhost:3000/api/v4/application/settings?resource_usage_limits' | jq '.resource_usage_limits'
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 18214  100 18214    0     0   158k      0 --:--:-- --:--:-- --:--:--  158k
{
  "rules": [
    {
      "name": "hi",
      "rules": [
        {
          "interval": 60,
          "selector": "worker_name=PipelineProcessWorker",
          "threshold": 1000
        },
        {
          "interval": 60,
          "selector": "*",
          "threshold": 100
        }
      ],
      "scopes": [
        "worker_name"
      ],
      "metadata": {
        "db_config_name": "ci"
      },
      "resource_key": "db_ci_duration_s"
    }
  ]
}
Edited by Sylvester Chin

Merge request reports

Loading