Skip to content

Add specific rate limit for Git LFS requests

What does this MR do?

If Git LFS is used for a repository, many Git LFS reqeusts are made during the common Git operations. In this context, the general User and IP rate limits may be reached that are defined in in Admin Area > Settings > Network > User and IP rate limits (see docs).

To address this issue, this MR allows to define a specific rate limit for Git LFS requests in Admin Area > Settings > Network > Git LFS Rate Limits:

  • Authenticated Git LFS requests

This limit is disabled by default. But if enabled, it supersede the general user and IP rate limits for Git LFS requests. This allows to keep the general user and IP rate limits, while the rate limits for Git LFS requests can be increased if necessary.

Besides this precedence, there are no differences in functionality compared to the general user and IP rate limits.

The following endpoints are detected as Git LFS requests:

  • <base_URL>/<namespace>/<project-path>.git/info/lfs/*
  • <base_URL>/<namespace>/<project-path>.git/gitlab-lfs/*

🛠 with at Siemens

/cc @bufferoverflow

Screenshots or Screencasts

image

How to setup and validate locally

  1. Create a repository using Git LFS (see docs)
  2. Visit the application settings in the admin area (http://localhost:3000/admin/application_settings/network) and enable Git LFS Rate Limits. Set requests per period per user: 1 and period in seconds: 10.
  3. Make Git LFS request curl <basic-url>/<namespace>/<project-path>.git/info/lfs/locks --header "Authorization: Basic <...>" Response OK
  4. Make same request again within 10 seconds Response Retry later (429 Too Many Requests)
  5. Wait 10 seconds
  6. Make same request again Response OK

Database Migrations

rails db:migrate:up VERSION=20210819085837
== 20210819085837 AddThrottleAuthenticatedGitLfsColumns: migrating ============
-- add_column(:application_settings, :throttle_authenticated_git_lfs_requests_per_period, :integer, {:default=>1000, :null=>false})
   -> 0.0479s
-- add_column(:application_settings, :throttle_authenticated_git_lfs_period_in_seconds, :integer, {:default=>60, :null=>false})
   -> 0.0056s
-- add_column(:application_settings, :throttle_authenticated_git_lfs_enabled, :boolean, {:default=>false, :null=>false})
   -> 0.0093s
== 20210819085837 AddThrottleAuthenticatedGitLfsColumns: migrated (0.0673s) ===
rails db:migrate:down VERSION=20210819085837
== 20210819085837 AddThrottleAuthenticatedGitLfsColumns: reverting ============
-- remove_column(:application_settings, :throttle_authenticated_git_lfs_enabled, :boolean, {:default=>false, :null=>false})
   -> 0.0254s
-- remove_column(:application_settings, :throttle_authenticated_git_lfs_period_in_seconds, :integer, {:default=>60, :null=>false})
   -> 0.0035s
-- remove_column(:application_settings, :throttle_authenticated_git_lfs_requests_per_period, :integer, {:default=>1000, :null=>false})
   -> 0.0038s
== 20210819085837 AddThrottleAuthenticatedGitLfsColumns: reverted (0.0359s) ===

Does this MR meet the acceptance criteria?

Conformity

Availability and Testing

Security

Does this MR contain changes to processing or storing of credentials or tokens, authorization and authentication methods or other items described in the security review guidelines? If not, then delete this Security section.

  • [-] Label as security and @ mention @gitlab-com/gl-security/appsec
  • [-] The MR includes necessary changes to maintain consistency between UI, API, email, or other methods
  • [-] Security reports checked/validated by a reviewer from the AppSec team
Edited by Jonas Wälter

Merge request reports

Loading