Backend: Add application limits to projects and group variables
Release notes
Problem to solve
Project and group variables are scoped to environments, which means that we need to execute a database query for each environment and each type of variable. And because these variables are encrypted, we need CPU time to decrypt them.
From !78440 (comment 814018166):
An example: 160 environments * 120 CI variables means you still have over 19,000 decryptions to process. Caching by environment only works well if you have a few of them.
But we have a limit of 500 environments and no limit for the variables, currently the max number of variables for projects is 2018 and for groups is 997. If the project that has the 2000 variables would use 500 environments, we'd do 1 million decryptions.
!78440 (merged) is an optimization band-aid, but we need to treat it as ~"technical debt" and remove as soon as possible, but to do it we need to add a reasonable limit on the number of variables that will allow us to load all the variables into memory and do the environment filtering in Ruby. This should help us cache the data more efficiently and remove the RequestCache.
Proposal
Add a 200 variables limit to the project and group variables.
User experience specs
- Update the Variables section description.
Project:
Variables store information, like passwords and secret keys, that you can use in job scripts. Each project can define a maximum of 200 variables. Learn more
Group:
Variables store information, like passwords and secret keys, that you can use in job scripts. Each group can define a maximum of 200 variables. Learn more
| Project updated variables description | Group updated variables description |
|---|---|
 |
 |
-
When a project or a group has 200+ variables defined, we will show an info alert at the top of the variables section notifying the user about the reached limit. The alert will be non-dismissable.
-
When user deletes some variables and the total number of variables in a project or group is 199 or lower, we'll remove the notification and enable the "Add variable" button again.
| Project variables limit notification | Group variables limit notification |
|---|---|
 |
 |
- When a project or a group has 200+ variables defined, we will disable the "Add variable" button in the CI/CD Settings variables form, preventing the user from adding more variables. When hovering over the disabled "Add variable" button we'll show a popover explaining the limitation. The popover should also be accessible via keyboard.
Project:
Maximum number of variables reached. This project has 225 defined CI/CD variables, which is over the limit of 200 variables per project. To add new variables, you must reduce the number of defined variables.
Group:
Maximum number of variables reached. This group has 225 defined CI/CD variables, which is over the limit of 200 variables per group. To add new variables, you must reduce the number of defined variables.
| Project button popover | Group button popover |
|---|---|
 |
 |
Implementation plan
| Name | Link |
|---|---|
| Design: Add application limits to projects and group variables | https://gitlab.com/gitlab-org/gitlab/-/issues/350643 |
| Frontend: Add application limits to projects and group variables | #362227 (closed) |
| Backend: Add application limits to projects and group variables |
|