Skip to content

Add deny all requests setting to UrlBlocker: Part 1

Luke Duncalfe requested to merge 377371-deny-all-requests into master

What does this MR do and why?

This merge request adds a new ApplicationSetting to deny all requests #377371 (closed).

When enabled, this setting will:

  • Always allow internal web and shell URIs.
  • Always allow object storage URIs.
  • Allow URIs that are in the allow list (either host, or IP, local or external)
  • Deny all other URIs (both local and external).

⚠ Limitation of this iteration:

This iteration suffers from a problem outlined in #377371 (comment 1271330392) which can lead the ApplicationSetting model to be saved in an invalid state due to the new setting. A second merge request !111510 (merged) will address this. The feature flag will not be enabled until both merge requests have merged.

QA steps:

Preparation for workaround:

As a workaround for the limitation mentioned above, we need to set the public_runner_releases_url value from its default gitlab.com URI to point to our local instance instead. After !111510 (merged) this will not longer be needed.

current_settings = ApplicationSetting.find_or_create_without_cache;
old_public_runner_releases_url = current_settings.public_runner_releases_url
ApplicationSettings::UpdateService.new(current_settings, nil, public_runner_releases_url: Gitlab.config.gitlab.base_url).execute # => true

QA steps:

  1. Enable the feature flag: 
     Feature.enable(:deny_all_requests_except_allowed)
  2. Test on the console should return an Array containing an Addressable::URI, to signal the URI is valid. 
     Gitlab::UrlBlocker.validate!('http://example.com', schemes: %w(http))
  3. Enable the new setting locally on the rails console: 
     current_settings = ApplicationSetting.find_or_create_without_cache;
 ApplicationSettings::UpdateService.new(current_settings, nil, deny_all_requests_except_allowed: true).execute
  4. Test on the console should raise a Gitlab::UrlBlocker::BlockedUrlError: 
     Gitlab::UrlBlocker.validate!('http://example.com', schemes: %w(http))
  5. Add example.com to your instance's allow list:
    1. Go to /admin/application_settings/network and click Outbound requests
    2. Add example.com to the allow list (Local IP addresses and domain names that hooks and services may access), click Save changes
  6. Either wait a minute before the next step, or clear the settings cache now: 
     Gitlab::CurrentSettings.expire_current_application_settings
  7. Test on the console should return an Array containing an Addressable::URI again. 
     Gitlab::UrlBlocker.validate!('http://example.com', schemes: %w(http))

Undo workaround:

Undo the workaround we made earlier:

  1. First, ensure the new setting is disabled: 
     current_settings = ApplicationSetting.find_or_create_without_cache;
 ApplicationSettings::UpdateService.new(current_settings, nil, deny_all_requests_except_allowed: false).execute # => true
  2. Restore the old public_runner_releases_url: 
    current_settings = ApplicationSetting.find_or_create_without_cache;
ApplicationSettings::UpdateService.new(current_settings, nil, public_runner_releases_url: old_public_runner_releases_url).execute # => true

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Related to #377371 (closed)

Edited by Luke Duncalfe

Merge request reports