Add specific rate limit for Git LFS requests

What does this MR do?

If Git LFS is used for a repository, many Git LFS reqeusts are made during the common Git operations. In this context, the general User and IP rate limits may be reached that are defined in in Admin Area > Settings > Network > User and IP rate limits (see docs).

To address this issue, this MR allows to define a specific rate limit for Git LFS requests in Admin Area > Settings > Network > Git LFS Rate Limits:

• Authenticated Git LFS requests

This limit is disabled by default. But if enabled, it supersede the general user and IP rate limits for Git LFS requests. This allows to keep the general user and IP rate limits, while the rate limits for Git LFS requests can be increased if necessary.

Besides this precedence, there are no differences in functionality compared to the general user and IP rate limits.

The following endpoints are detected as Git LFS requests:

• <base_URL>/<namespace>/<project-path>.git/info/lfs/*
• <base_URL>/<namespace>/<project-path>.git/gitlab-lfs/*

🛠 with ❤ at Siemens

/cc @bufferoverflow

Screenshots or Screencasts

How to setup and validate locally

1. Create a repository using Git LFS (see docs)
2. Visit the application settings in the admin area (http://localhost:3000/admin/application_settings/network) and enable Git LFS Rate Limits. Set requests per period per user: 1 and period in seconds: 10.
3. Make Git LFS request curl <basic-url>/<namespace>/<project-path>.git/info/lfs/locks --header "Authorization: Basic <...>" ▶ Response OK
4. Make same request again within 10 seconds ▶ Response Retry later (429 Too Many Requests)
5. Wait 10 seconds
6. Make same request again ▶ Response OK

Database Migrations

rails db:migrate:up VERSION=20210819085837
== 20210819085837 AddThrottleAuthenticatedGitLfsColumns: migrating ============
-- add_column(:application_settings, :throttle_authenticated_git_lfs_requests_per_period, :integer, {:default=>1000, :null=>false})
   -> 0.0479s
-- add_column(:application_settings, :throttle_authenticated_git_lfs_period_in_seconds, :integer, {:default=>60, :null=>false})
   -> 0.0056s
-- add_column(:application_settings, :throttle_authenticated_git_lfs_enabled, :boolean, {:default=>false, :null=>false})
   -> 0.0093s
== 20210819085837 AddThrottleAuthenticatedGitLfsColumns: migrated (0.0673s) ===

rails db:migrate:down VERSION=20210819085837
== 20210819085837 AddThrottleAuthenticatedGitLfsColumns: reverting ============
-- remove_column(:application_settings, :throttle_authenticated_git_lfs_enabled, :boolean, {:default=>false, :null=>false})
   -> 0.0254s
-- remove_column(:application_settings, :throttle_authenticated_git_lfs_period_in_seconds, :integer, {:default=>60, :null=>false})
   -> 0.0035s
-- remove_column(:application_settings, :throttle_authenticated_git_lfs_requests_per_period, :integer, {:default=>1000, :null=>false})
   -> 0.0038s
== 20210819085837 AddThrottleAuthenticatedGitLfsColumns: reverted (0.0359s) ===

Does this MR meet the acceptance criteria?

Conformity

• I have included changelog trailers, or none are needed. (Does this MR need a changelog?)
• I have added/updated documentation, or it's not needed. (Is documentation required?)
• I have properly separated EE content from FOSS, or this MR is FOSS only. (Where should EE code go?)
• I have added information for database reviewers in the MR description, or it's not needed. (Does this MR have database related changes?)
• I have self-reviewed this MR per code review guidelines.
• This MR does not harm performance, or I have asked a reviewer to help assess the performance impact. (Merge request performance guidelines)
• I have followed the style guides.
• This change is backwards compatible across updates, or this does not apply.

Availability and Testing

• I have added/updated tests following the Testing Guide, or it's not needed. (Consider all test levels. See the Test Planning Process.)
• I have tested this MR in all supported browsers, or it's not needed.
• I have informed the Infrastructure department of a default or new setting change per definition of done, or it's not needed.

Security

Does this MR contain changes to processing or storing of credentials or tokens, authorization and authentication methods or other items described in the security review guidelines? If not, then delete this Security section.

• [-] Label as security and @ mention @gitlab-com/gl-security/appsec
• [-] The MR includes necessary changes to maintain consistency between UI, API, email, or other methods
• [-] Security reports checked/validated by a reviewer from the AppSec team