Skip to content

Improve sensitive URL parameter masking

Arran Walker requested to merge 4625-url-scrubbing into main

What does this MR do?

Adds URL parameters masking.

Why was this MR needed?

This is something we already have implemented with regex rules, but it doesn't work everywhere (for example, on logrus fields) and is really quite slow.

This new solution is significantly faster and works for anything written to the trace output (therefore addressing #4625 (closed)).

What's the best way to test this MR?

One way of manually testing this is to create a runner config with invalid S3 credentials:

[runners]]
  name = "nonsense"
  url = "https://gitlab.com/"
  token = "<token>"
  executor = "shell"
  [runners.custom_build_dir]
  [runners.cache]
    Type = "s3"
    Path = "path/to/prefix"
    Shared = false
    [runners.cache.s3]
      ServerAddress = "s3.amazonaws.com"
      AccessKey = "AWS_S3_ACCESS_KEY"
      SecretKey = "AWS_S3_SECRET_KEY"
      BucketName = "runners-cache"
      BucketLocation = "eu-west-1"
      Insecure = false
  [runners.feature_flags]
    FF_USE_IMPROVED_URL_MASKING = false

and run a simple cache job:

upload:
  script:
    - echo "https://example.com?X-Amz-Credential=SENSITIVE_KEY"
    - echo "hi" > file
  cache:
    paths:
      - file

With FF_USE_IMPROVED_URL_MASKING = true, the parameters will be masked. With FF_USE_IMPROVED_URL_MASKING = false the parameters will be exposed.

What are the relevant issue numbers?

Closes #4625 (closed)

Merge request reports