Connect to origin using proxied dns record
Why is this change being made?
Before MR
We hardcode the origin address as storage.googleapis.com
for prod + review apps.
For prod, before we connect to the origin (GCS), we prefix the path with the /<bucket name>/
followed by the path in the requested URL (eg. storage.googleapis.com/about.gitlab.com/index.html
).
For review apps, we also have to add the branch name after the bucket name, so you end up with something like storage.googleapis.com/about.gitlab-review.app/<branch name>/path
.
If we get a 404
from the origin, we go back to the origin and fetch 404.html
from the base path.
This works fine, however in production, there are a significant number of 404s and it would be good if we didn't have the extra round-trip to fetch the 404.html
. Also, statically connecting to storage.googleapis.com
in the worker code means that the target address in the CNAME record is meaningless (nit pick).
After MR
We will use the target address in the proxied DNS record to connect to the origin. Additionally, we will use c.storage.googleapis.com
for non-review (eg. production) as it means that Google will treat the bucket like a website and return 404.html
for 404 responses (saves an extra round-trip).
Issue: https://gitlab.com/gitlab-com/gl-infra/reliability/-/issues/18945
Author Checklist
-
Provided a concise title for this Merge Request (MR) -
Added a description to this MR explaining the reasons for the proposed change, per say why, not just what - Copy/paste the Slack conversation to document it for later, or upload screenshots. Verify that no confidential data is added, and the content is SAFE
-
Assign reviewers for this MR to the correct Directly Responsible Individual/s (DRI) - If the DRI for the page/s being updated isn’t immediately clear, then assign it to one of the people listed in the
Maintained by
section on the page being edited - If your manager does not have merge rights, please ask someone to merge it AFTER it has been approved by your manager in #mr-buddies
- The when to get approval handbook section explains the workflow in more detail
- If the DRI for the page/s being updated isn’t immediately clear, then assign it to one of the people listed in the
-
If the changes affect team members, or warrant an announcement in another way, please consider posting an update in #whats-happening-at-gitlab linking to this MR - If this is a change that directly impacts the majority of global team members, it should be a candidate for #company-fyi. Please work with internal communications and check the handbook for examples.