-
Notifications
You must be signed in to change notification settings - Fork 2.6k
Implement schema2 manifest formats #1281
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
7 commits
Select commit
Hold shift + click to select a range
2ff77c0
Add schema2 manifest support
aaronlehmann befd4d6
Factor out schema-specific portions of manifestStore
aaronlehmann 3f746a8
Recognize clients that don't support schema2, and convert manifests t…
aaronlehmann 9284810
Add API unit testing for schema2 manifest
aaronlehmann 9c416f0
Add support for manifest list ("fat manifest")
aaronlehmann 697af09
Recognize clients that don't support manifest lists
aaronlehmann 6d17423
Move MediaType into manifest.Versioned
aaronlehmann File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,147 @@ | ||
package manifestlist | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Might be good to have a package doc for the concept of manifest lists with a few examples of how they might be used. Let's not hold up the PR on this, but it might be helpful. |
||
|
||
import ( | ||
"encoding/json" | ||
"errors" | ||
"fmt" | ||
|
||
"github.com/docker/distribution" | ||
"github.com/docker/distribution/digest" | ||
"github.com/docker/distribution/manifest" | ||
) | ||
|
||
// MediaTypeManifestList specifies the mediaType for manifest lists. | ||
const MediaTypeManifestList = "application/vnd.docker.distribution.manifest.list.v2+json" | ||
|
||
// SchemaVersion provides a pre-initialized version structure for this | ||
// packages version of the manifest. | ||
var SchemaVersion = manifest.Versioned{ | ||
SchemaVersion: 2, | ||
MediaType: MediaTypeManifestList, | ||
} | ||
|
||
func init() { | ||
manifestListFunc := func(b []byte) (distribution.Manifest, distribution.Descriptor, error) { | ||
m := new(DeserializedManifestList) | ||
err := m.UnmarshalJSON(b) | ||
if err != nil { | ||
return nil, distribution.Descriptor{}, err | ||
} | ||
|
||
dgst := digest.FromBytes(b) | ||
return m, distribution.Descriptor{Digest: dgst, Size: int64(len(b)), MediaType: MediaTypeManifestList}, err | ||
} | ||
err := distribution.RegisterManifestSchema(MediaTypeManifestList, manifestListFunc) | ||
if err != nil { | ||
panic(fmt.Sprintf("Unable to register manifest: %s", err)) | ||
} | ||
} | ||
|
||
// PlatformSpec specifies a platform where a particular image manifest is | ||
// applicable. | ||
type PlatformSpec struct { | ||
// Architecture field specifies the CPU architecture, for example | ||
// `amd64` or `ppc64`. | ||
Architecture string `json:"architecture"` | ||
|
||
// OS specifies the operating system, for example `linux` or `windows`. | ||
OS string `json:"os"` | ||
|
||
// Variant is an optional field specifying a variant of the CPU, for | ||
// example `ppc64le` to specify a little-endian version of a PowerPC CPU. | ||
Variant string `json:"variant,omitempty"` | ||
|
||
// Features is an optional field specifuing an array of strings, each | ||
// listing a required CPU feature (for example `sse4` or `aes`). | ||
Features []string `json:"features,omitempty"` | ||
} | ||
|
||
// A ManifestDescriptor references a platform-specific manifest. | ||
type ManifestDescriptor struct { | ||
distribution.Descriptor | ||
|
||
// Platform specifies which platform the manifest pointed to by the | ||
// descriptor runs on. | ||
Platform PlatformSpec `json:"platform"` | ||
} | ||
|
||
// ManifestList references manifests for various platforms. | ||
type ManifestList struct { | ||
manifest.Versioned | ||
|
||
// Config references the image configuration as a blob. | ||
Manifests []ManifestDescriptor `json:"manifests"` | ||
} | ||
|
||
// References returnes the distribution descriptors for the referenced image | ||
// manifests. | ||
func (m ManifestList) References() []distribution.Descriptor { | ||
dependencies := make([]distribution.Descriptor, len(m.Manifests)) | ||
for i := range m.Manifests { | ||
dependencies[i] = m.Manifests[i].Descriptor | ||
} | ||
|
||
return dependencies | ||
} | ||
|
||
// DeserializedManifestList wraps ManifestList with a copy of the original | ||
// JSON. | ||
type DeserializedManifestList struct { | ||
ManifestList | ||
|
||
// canonical is the canonical byte representation of the Manifest. | ||
canonical []byte | ||
} | ||
|
||
// FromDescriptors takes a slice of descriptors, and returns a | ||
// DeserializedManifestList which contains the resulting manifest list | ||
// and its JSON representation. | ||
func FromDescriptors(descriptors []ManifestDescriptor) (*DeserializedManifestList, error) { | ||
m := ManifestList{ | ||
Versioned: SchemaVersion, | ||
} | ||
|
||
m.Manifests = make([]ManifestDescriptor, len(descriptors), len(descriptors)) | ||
copy(m.Manifests, descriptors) | ||
|
||
deserialized := DeserializedManifestList{ | ||
ManifestList: m, | ||
} | ||
|
||
var err error | ||
deserialized.canonical, err = json.MarshalIndent(&m, "", " ") | ||
return &deserialized, err | ||
} | ||
|
||
// UnmarshalJSON populates a new ManifestList struct from JSON data. | ||
func (m *DeserializedManifestList) UnmarshalJSON(b []byte) error { | ||
m.canonical = make([]byte, len(b), len(b)) | ||
// store manifest list in canonical | ||
copy(m.canonical, b) | ||
|
||
// Unmarshal canonical JSON into ManifestList object | ||
var manifestList ManifestList | ||
if err := json.Unmarshal(m.canonical, &manifestList); err != nil { | ||
return err | ||
} | ||
|
||
m.ManifestList = manifestList | ||
|
||
return nil | ||
} | ||
|
||
// MarshalJSON returns the contents of canonical. If canonical is empty, | ||
// marshals the inner contents. | ||
func (m *DeserializedManifestList) MarshalJSON() ([]byte, error) { | ||
if len(m.canonical) > 0 { | ||
return m.canonical, nil | ||
} | ||
|
||
return nil, errors.New("JSON representation not initialized in DeserializedManifestList") | ||
} | ||
|
||
// Payload returns the raw content of the manifest list. The contents can be | ||
// used to calculate the content identifier. | ||
func (m DeserializedManifestList) Payload() (string, []byte, error) { | ||
return m.MediaType, m.canonical, nil | ||
} |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,111 @@ | ||
package manifestlist | ||
|
||
import ( | ||
"bytes" | ||
"encoding/json" | ||
"reflect" | ||
"testing" | ||
|
||
"github.com/docker/distribution" | ||
) | ||
|
||
var expectedManifestListSerialization = []byte(`{ | ||
"schemaVersion": 2, | ||
"mediaType": "application/vnd.docker.distribution.manifest.list.v2+json", | ||
"manifests": [ | ||
{ | ||
"mediaType": "application/vnd.docker.distribution.manifest.v2+json", | ||
"size": 985, | ||
"digest": "sha256:1a9ec845ee94c202b2d5da74a24f0ed2058318bfa9879fa541efaecba272e86b", | ||
"platform": { | ||
"architecture": "amd64", | ||
"os": "linux", | ||
"features": [ | ||
"sse4" | ||
] | ||
} | ||
}, | ||
{ | ||
"mediaType": "application/vnd.docker.distribution.manifest.v2+json", | ||
"size": 2392, | ||
"digest": "sha256:6346340964309634683409684360934680934608934608934608934068934608", | ||
"platform": { | ||
"architecture": "sun4m", | ||
"os": "sunos" | ||
} | ||
} | ||
] | ||
}`) | ||
|
||
func TestManifestList(t *testing.T) { | ||
manifestDescriptors := []ManifestDescriptor{ | ||
{ | ||
Descriptor: distribution.Descriptor{ | ||
Digest: "sha256:1a9ec845ee94c202b2d5da74a24f0ed2058318bfa9879fa541efaecba272e86b", | ||
Size: 985, | ||
MediaType: "application/vnd.docker.distribution.manifest.v2+json", | ||
}, | ||
Platform: PlatformSpec{ | ||
Architecture: "amd64", | ||
OS: "linux", | ||
Features: []string{"sse4"}, | ||
}, | ||
}, | ||
{ | ||
Descriptor: distribution.Descriptor{ | ||
Digest: "sha256:6346340964309634683409684360934680934608934608934608934068934608", | ||
Size: 2392, | ||
MediaType: "application/vnd.docker.distribution.manifest.v2+json", | ||
}, | ||
Platform: PlatformSpec{ | ||
Architecture: "sun4m", | ||
OS: "sunos", | ||
}, | ||
}, | ||
} | ||
|
||
deserialized, err := FromDescriptors(manifestDescriptors) | ||
if err != nil { | ||
t.Fatalf("error creating DeserializedManifestList: %v", err) | ||
} | ||
|
||
mediaType, canonical, err := deserialized.Payload() | ||
|
||
if mediaType != MediaTypeManifestList { | ||
t.Fatalf("unexpected media type: %s", mediaType) | ||
} | ||
|
||
// Check that the canonical field is the same as json.MarshalIndent | ||
// with these parameters. | ||
p, err := json.MarshalIndent(&deserialized.ManifestList, "", " ") | ||
if err != nil { | ||
t.Fatalf("error marshaling manifest list: %v", err) | ||
} | ||
if !bytes.Equal(p, canonical) { | ||
t.Fatalf("manifest bytes not equal: %q != %q", string(canonical), string(p)) | ||
} | ||
|
||
// Check that the canonical field has the expected value. | ||
if !bytes.Equal(expectedManifestListSerialization, canonical) { | ||
t.Fatalf("manifest bytes not equal: %q != %q", string(canonical), string(expectedManifestListSerialization)) | ||
} | ||
|
||
var unmarshalled DeserializedManifestList | ||
if err := json.Unmarshal(deserialized.canonical, &unmarshalled); err != nil { | ||
t.Fatalf("error unmarshaling manifest: %v", err) | ||
} | ||
|
||
if !reflect.DeepEqual(&unmarshalled, deserialized) { | ||
t.Fatalf("manifests are different after unmarshaling: %v != %v", unmarshalled, *deserialized) | ||
} | ||
|
||
references := deserialized.References() | ||
if len(references) != 2 { | ||
t.Fatalf("unexpected number of references: %d", len(references)) | ||
} | ||
for i := range references { | ||
if !reflect.DeepEqual(references[i], manifestDescriptors[i].Descriptor) { | ||
t.Fatalf("unexpected value %d returned by References: %v", i, references[i]) | ||
} | ||
} | ||
} |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This does not satisfy the
Describable
interface.Describable
must describe itself (in this case, the descriptor), not the thing it is describing.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There's a common use case where some code has a slice of descriptors and wants to add those descriptors to a manifest through a
ManifestBuilder
. SinceManifestBuilder.AppendReference
takes aDescribable
, not having this method defined would force the creation of a wrapper type just to return a descriptor.Instead of having
Descriptor
satisfyDescribable
, would you be okay with changingAppendReference
to take aDescriptor
?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I see. The idea of having an interface there is to allow implementations to type switch on elements being added to the manifest and support specialized behavior. I suppose having
Descriptor
implementDescribable
is a fine exception to make this work. Let's add a comment there, indicating this is the exception and not the rule.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Added a comment explaining this.