Show that we proxied the request via the HTTP Router for Cells

Overview

For Cells we'll be deploying the HTTP router and we'll be doing a progressive rollout in staging (&14413 (closed)) and production (&14414 (closed), &14412 (closed)).

We need an easy way to track progress of how many requests we are proxying through the router and how many we aren't, ideally something we can easy chart.

❌ Option 1: Using Cloudflare logs

Cloudflare provides Instant logs where we can see the it provides some worker information:

{
  ...
  "WorkerStatus": "unknown",
  "WorkerSubrequest": false,
  "WorkerSubrequestCount": 0,
  ...
}

I’m not 100% sure about this, but WorkerStatus should give us the information of the worker that was invoked. We don’t have permanent storage of these logs anywhere at the moment and it would be really expensive to store all of them without sampling.

❌ Option 2: Cloudflare Analytics

Cloudflare has analytics but this doesn’t provide information if a request went through a worker or not

🛠️ Option 3: Implement our own solution

By default when a worker sends a sub-request it will add the CF-worker header which specifies that the request came from a worker and what is the origin:

{
  "args": {}, 
  "data": "", 
  "files": {}, 
  "form": {}, 
  "headers": {
    "Accept-Encoding": "gzip, br", 
    "Cdn-Loop": "cloudflare; subreqs=1", 
    "Cf-Connecting-Ip": "2a06:98c0:3600::103", 
    "Cf-Ew-Via": "15", 
    "Cf-Ray": "8a845d8ad146715d-DUS", 
    "Cf-Visitor": "{\"scheme\":\"https\"}", 
    "Cf-Worker": "sxuereb.workers.dev", 
    "Host": "httpbin.org", 
    "X-Amzn-Trace-Id": "Root=1-66a10611-5ac20a354a08a06e7d935c07"
  }, 
  "json": null, 
  "method": "GET", 
  "origin": "2a06:98c0:3600::103, 198.41.242.220", 
  "url": "https://httpbin.org/anything"
}

We can then update our rails logger to print the value of Cf-Worker if it’s present, if it’s not we don’t log the field at all. With that, we’ll be able to create graphs in Kibana to show which requests have been proxied via the worker and which ones haven’t. The log field can be set in multiple ways:

1. Either a bool true/false if the request went through a worker
2. A string printing the value of the Cf-Worker header.

Action Items

• HTTP Router: Add a new header X-GitLab-HTTP-Router-Rule-Action & X-GitLab-HTTP-Router-Rule-Type with the value of the chosen rule and it's type. Type is added only if the action is classify. - Completed with gitlab-org/cells/http-router!190 (merged)
• Rails: Log the value of X-GitLab-HTTP-Router-Rule-Action & X-GitLab-HTTP-Router-Rule-Type - completed with !162543 (merged)
• Rails: Log the value of Cf-Worker when present - Completed with !162987 (merged).
• Rails: increment a counter of that metrics X-GitLab-HTTP-Router-Rule-Action & X-GitLab-HTTP-Router-Rule-Type when present - In review: !163290 (merged)