Geo container registry sync does not sync images with foreign layers
Summary
When Geo attempts to sync an image where the manifest contains a foreign layer, we attempt to sync that from the primary as well, when this does not actually exist.
From the Docker Image Manifest schema, this should not be synced:
application/vnd.docker.image.rootfs.foreign.diff.tar.gzip
- “Layer”, as a gzipped tar that should never be pushed
At the moment, it looks like the Windows images (as an example) use this extensively, here's an example:
mitmproxy -p 8081 # in one terminal
SSL_CERT_FILE=~/.mitmproxy/mitmproxy-ca-cert.pem HTTPS_PROXY=http://127.0.0.1:8081 dockerd # in another terminal
docker pull mcr.microsoft.com/windows:2004
# in mitmproxy, we can see:
16:02:31 HTTPS HEAD mcr.microsoft.com /v2/windows/manifests/2004 200 [no content] 272ms
16:02:31 HTTPS GET mcr.microsoft.com /v2/windows/manifests/sha256:012c9aed3adf51c93da6a9073bb1b6ddb796e11206b41ddaea44af3caa16732d 200 …fest.list.v2+json 479b 92ms
root@cat:~# curl https://mcr.microsoft.com/v2/windows/manifests/sha256:012c9aed3adf51c93da6a9073bb1b6ddb796e11206b41ddaea44af3caa16732d -sL | jq .manifests[].digest
"sha256:5d55ed8037743fd30751574ebfbc823caa0cae75a662da874ca6bebb7e490fd5"
root@cat:~# curl https://mcr.microsoft.com/v2/windows/manifests/sha256:5d55ed8037743fd30751574ebfbc823caa0cae75a662da874ca6bebb7e490fd5 -sL | jq '.layers | pluck(["mediaType", "urls"])'
[
[
"application/vnd.docker.image.rootfs.foreign.diff.tar.gzip",
[
"https://mcr.microsoft.com/v2/windows/blobs/sha256:830e00eda3f1e66421f779823d61a1e8f5529ddb20ac80716c817e2e9658d313"
]
],
[
"application/vnd.docker.image.rootfs.foreign.diff.tar.gzip",
[
"https://mcr.microsoft.com/v2/windows/blobs/sha256:1d43695385c669fbad5bdb8e257b5d2f910d74d20d731b0b0b4a67a254d07070"
]
]
]
Steps to reproduce
Push an image containing a foreign layer (unsure how to properly recreate this yet, probably patching a custom manifest - or maybe testing with a simple Windows image)
What is the current bug behavior?
Container repositories containing foreign blobs do not sync at all on the Geo secondary, and error out instead.
What is the expected correct behavior?
Container repositories should sync to the Geo secondary.
Relevant logs and/or screenshots
Possible fixes
Maybe we can check when syncing and skip those specific media types, somewhere near the blob exist check