Throttle pipelines creation rate
What does this MR do and why?
Related to https://gitlab.com/gitlab-org/gitlab/-/issues/352232
Since we already have a limit for how many active pipelines a project could have, I think it would be a good idea to add a limit on how many pipelines could be created by a user for a commit in a short period of time. This should stop rogue pipelines that the other two limits fail to catch.
This limits the number of pipelines created for the same project/commit/user tuple to 20 per minute. This value is not definitive yet.
Rollout plan
- enable
ci_track_throttle_pipelines_creation
for a week and monitor the logs - if there isn't any entry in the log file, proceed to disable
ci_throttle_pipelines_creation_dry_run
and start enforcing the limit - if there are log entries, analyze them and reconsider the limit.
- as a follow-up, move the limit into an
application_settings
column
Screenshots or screen recordings
How to set up and validate locally
- Run database migrations:
bin/rails db:migrate
- Update the limit to a lower value:
ApplicationSetting.current.update!(pipelines_create_limit: 2)
- Enable the
ci_throttle_pipelines_creation
feature flag:Feature.enable(:ci_throttle_pipelines_creation)
- Disable the
ci_throttle_pipelines_creation_dry_run
feature flag to enforce the limit:Feature.disable(:ci_throttle_pipelines_creation_dry_run)
- Open 3 tabs on the new pipelines in the same project:
namespace/project/-/pipelines/new
- Press the
Run pipeline
button in each tab
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.
Edited by Marius Bobin