8000 Split OCI Image Index from Docker Manifest List by brackendawson · Pull Request #3869 · distribution/distribution · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Split OCI Image Index from Docker Manifest List #3869

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Jul 19, 2023

Conversation

brackendawson
Copy link
Contributor
@brackendawson brackendawson commented Mar 31, 2023

Move implementation of the index from the manifestlist package to the ocischema package so that other modules making empty imports support the manifest types their authors would expect. This is a breaking change to distribution as a library but not the registry.

As OCI 1.0 released the manifest and index together, that is a good package from which to initialise both manifests. The docker manifest and manifest list remain in separate packages because one was released later.

The image index and manifest list still share common code in many functions not intended for import by other modules.

This now supports (omits) empty platform objects in OCI Image Indices.

This adds support for annotations on OCI Image Indices.

fixes #3836
fixes #2652

@codecov-commenter
Copy link
codecov-commenter commented Mar 31, 2023

Codecov Report

Patch coverage: 77.27% and project coverage change: +0.53 🎉

Comparison is base (0c95801) 56.58% compared to head (9d1a8fc) 57.11%.

❗ Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3869      +/-   ##
==========================================
+ Coverage   56.58%   57.11%   +0.53%     
==========================================
  Files         105      108       +3     
  Lines       10636    10759     +123     
==========================================
+ Hits         6018     6145     +127     
+ Misses       3946     3940       -6     
- Partials      672      674       +2     
Impacted Files Coverage Δ
registry/storage/ociindexhandler.go 62.50% <62.50%> (ø)
registry/storage/manifestlisthandler.go 50.00% <66.66%> (+7.14%) ⬆️
manifest/ocischema/index.go 73.41% <73.41%> (ø)
manifest/manifestlist/manifestlist.go 78.31% <90.00%> (+6.88%) ⬆️
manifest/ocischema/manifest.go 75.00% <100.00%> (+0.80%) ⬆️
registry/storage/manifeststore.go 77.77% <100.00%> (+1.03%) ⬆️
registry/storage/registry.go 88.57% <100.00%> (+0.27%) ⬆️

... and 10 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

Move implementation of the index from the manifestlist package to the ocischema package so that other modules making empty imports support the manifest types their authors would expect. This is a breaking change to distribution as a library but not the registry.

As OCI 1.0 released the manifest and index together, that is a good package from which to initialise both manifests. The docker manifest and manifest list remain in separate packages because one was released later.

The image index and manifest list still share common code in many functions not intended for import by other modules.

Signed-off-by: Bracken Dawson <abdawson@gmail.com>
Empty platform structs were already supported after splitting OCI Image
Index out from Docker Manifest List.

Signed-off-by: Bracken Dawson <abdawson@gmail.com>
@brackendawson brackendawson marked this pull request as ready for review March 31, 2023 13:14
@milosgajdos milosgajdos requested a review from thaJeztah March 31, 2023 13:43
Copy link
Member
@milosgajdos milosgajdos left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks great. I need to give it another pass due to the volume of this change but on the first pass this LGTM. I have a few comments and suggestions but none of them is related to the core logic -- they're mostly just nits.

// OCISchemaVersion provides a pre-initialized version structure for this
// packages OCIschema version of the manifest.
var OCISchemaVersion = manifest.Versioned{
SchemaVersion: 2,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same comment as earlier. Feel free to ignore me, but I figured it might be worth changing it as we use this const value in a few places in this file.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I ended up using this definition as the source of truth for the other uses in the Index. I know it's not const because it's a struct, I can isolate it further to a read const if you like?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is fine.

@milosgajdos
Copy link
Member

@thaJeztah do you want to give this a pass, too, please? In case I missed something, which I'm pretty sure I did 😄

@thaJeztah
Copy link
Member

@thaJeztah do you want to give this a pass, too, please? In case I missed something, which I'm pretty sure I did 😄

Yes, I can give this one a pass; currently hanging out at KubeCon and was looking at it from my phone (bit hard to read the diff 😆).

I recall I looked at some of this, and seem to remember some "docker" types could potentially be just aliases for the OCI types if we get rid of this mostly useless "Describable" interface;

type Describable interface {

Which was implemented in various places "for convenience"

// to make it more convenient to pass actual descriptors to functions that

(I may have a branch that gets rid of that)

- DRY out SchemaVersion literals
- Better name the predefined Versioned struct for the Image Index
- Var names, declarations, else cases.

Co-authored-by: Milos Gajdos <milosthegajdos@gmail.com>
Signed-off-by: Bracken Dawson <abdawson@gmail.com>
It is desirable to remove Platform from distribution.Descriptor because it doesn't really belong there. However this would be a further breaking change because the References() call would no longer be returning plaform information when it reurns descriptors of manifests, which is started to for OCI Indices after c94f288 and this feature was added to Docker Manifest Lists in 1a059fe. I don't want to take away something people clearly want.

Signed-off-by: Bracken Dawson <abdawson@gmail.com>
@brackendawson
Copy link
Contributor Author

@milosgajdos @thaJeztah while working on implementing the referrers api for #3716 I found that the current shared manifestlist type can't be used for the referrers API response because the manifests[].platform keys cannot be suppressed from the JSON, even with omitempty, because non-pointer type structs are never empty. It's much easier to do after this refactor because the dedicated OCI Index can be used as is. Any chance we can get this reviewed? If we decide not to merge this PR then I'll need to take a different approach for the referrers API, probably duplicating the Index type just for that one response.

@milosgajdos
Copy link
Member

This approach is fine with me. I'm waiting for @thaJeztah and @squizzi to have a look.

@milosgajdos
Copy link
Member
milosgajdos commented Jun 27, 2023

@milosgajdos milosgajdos requested review from corhere and wy65701436 July 7, 2023 09:27
@davidspek
Copy link
Collaborator

ping @wy65701436

@davidspek
Copy link
Collaborator

Ping @deleteriousEffect

Copy link
Collaborator
@wy65701436 wy65701436 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@wy65701436 wy65701436 merged commit 46b3d62 into distribution:main Jul 19, 2023
@brackendawson brackendawson deleted the split-oci-index branch July 20, 2023 10:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

manifestlist implicitly supports OCI Indices in client code The last 1% for OCI spec support - OCI image index needs...
6 participants
0