Add docker media types in OCI formats #2695
Conversation
|
Thanks. At a high level, I’m really unsure. For reading images, sure, that’s ~easy enough. But are the users
8000
then going to ask to also supporting writes with non-OCI images? Does the Approximately, I generally think that any producer creating an OCI-layout directory structure is obviously aware of OCI, so I don’t know why any new user of this code should start to want to support pre-OCI images. What’s the use case? If the goal were to write some kind of general-purpose image storage mechanism, note that the OCI transport (currently) can’t do that, e.g. it doesn’t support reading/writing signatures along with the image. |
|
The main issue I'm looking to address here is cases where tooling such as the Docker Daemon pull existing images from upstreams, and those upstream images still use docker media types. With the recent switch for the Docker Daemon to save dual format archives (both oci and docker archive), we run into an issue where the archive is still an OCI layout, but it maintained its original media types, thus breaking tools like Skopeo. I do agree that when creating OCI layouts, the creator is likely aware of the OCI types and should conform to them when possible. That's why I focused primarily on only read operations so as to better handle the case where tooling changes the wrapper. That being said, if there are other areas of the code that I should fill out, I'm happy to do so. |
|
@mtrmac Sorry to ping you, but I'd like to follow up here if there is a path forward to supporting the docker media types. |
|
I’m sorry, I couldn’t return to look into this in detail — but, the short version of it is that if |
|
@mtrmac Sorry to bump this again. I updated the PR after your comments on V2S1. Let me know if there's any other changes I should add. |
I can’t reproduce this: # rpm -q moby-engine skopeo
moby-engine-27.3.1-2.fc41.x86_64
skopeo-1.17.0-1.fc41.x86_64
# skopeo inspect docker://quay.io/libpod/alpine | jq '.Digest,.LayersData[].MIMEType'
"sha256:fa93b01658e3a5a1686dc3ae55f170d8de487006fb53a28efcd12ab0710a2e5f"
"application/vnd.docker.image.rootfs.diff.tar.gzip"
# docker pull quay.io/libpod/alpine
…
# docker save -o foo.tar quay.io/libpod/alpine
# skopeo inspect --raw oci-archive:foo.tar | jq -r .mediaType
application/vnd.oci.image.manifest.v1+json(and extracting the archive, Is there some other way this can be created using Docker? |
I see different behavior on my machine. I run Windows, though we have run into this on Linux hosts as well. PS C:\Users\brbayes\Downloads> docker -v
Docker version 27.5.1, build 9f9e405
PS C:\Users\brbayes\Downloads> skopeo -v
skopeo version 1.17.0
PS C:\Users\brbayes\Downloads> skopeo inspect docker://quay.io/libpod/alpine --override-os linux | jq '.Digest,.LayersData[].MIMEType'
"sha256:fa93b01658e3a5a1686dc3ae55f170d8de487006fb53a28efcd12ab0710a2e5f"
"application/vnd.docker.image.rootfs.diff.tar.gzip"
PS C:\Users\brbayes\Downloads> docker pull quay.io/libpod/alpine
Using default tag: latest
latest: Pulling from libpod/alpine
Digest: sha256:fa93b01658e3a5a1686dc3ae55f170d8de487006fb53a28efcd12ab0710a2e5f
Status: Image is up to date for quay.io/libpod/alpine:latest
quay.io/libpod/alpine:latest
What's next:
View a summary of image vulnerabilities and recommendations → docker scout quickview quay.io/libpod/alpine
PS C:\Users\brbayes\Downloads> docker save -o libpod.tar quay.io/libpod/alpine
PS C:\Users\brbayes\Downloads> skopeo inspect --raw oci-archive:libpod.tar | jq -r .mediaType
application/vnd.docker.distribution.manifest.list.v2+json |
|
Thanks. I’m looking at https://github.com/moby/moby/blob/8ca767963101763af2d8e44ea24ae0756adcfb05/image/tarexport/save.go#L250, that’s clearly not that path. This is with the containerd backend, isn’t it? |
|
That's correct, this is using the containerd backend both on my local machine, and on the linux hosts. |
|
Thanks, reproduced. Fine, let’s do this. … and it saves the original multi-platform manifest list, but the archive only contains the one pulled architecture. I.e. the image is sparse. That’s going to be fun. |
Signed-off-by: Brandyn Bayes <brbayes@microsoft.com>
8e90432 to
5771973
Compare
|
The changes are squashed and rebased. |
This pull request adds the docker media types when reading back OCI formats, along with the corresponding tests. This is from running into cases where docker images were built with the docker media types. To avoid having to convert all of the existing manifests so that the original types can be maintained, this code adds support for reading the docker specific formats since the OCI formats are backwards compatible with the docker formats (https://github.com/opencontainers/image-spec/blob/main/media-types.md).