8000 Implement schema2 manifest formats by aaronlehmann · Pull Request #1281 · distribution/distribution · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Implement schema2 manifest formats #1281

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
Jan 8, 2016
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions blobs.go
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,15 @@ type Descriptor struct {
// depend on the simplicity of this type.
}

// Descriptor returns the descriptor, to make it satisfy the Describable
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This does not satisfy the Describable interface. Describable must describe itself (in this case, the descriptor), not the thing it is describing.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This does not satisfy the Describable interface. Describable must describe itself (in this case, the descriptor), not the thing it is describing.

There's a common use case where some code has a slice of descriptors and wants to add those descriptors to a manifest through a ManifestBuilder. Since ManifestBuilder.AppendReference takes a Describable, not having this method defined would force the creation of a wrapper type just to return a descriptor.

Instead of having Descriptor satisfy Describable, would you be okay with changing AppendReference to take a Descriptor?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see. The idea of having an interface there is to allow implementations to type switch on elements being added to the manifest and support specialized behavior. I suppose having Descriptor implement Describable is a fine exception to make this work. Let's add a comment there, indicating this is the exception and not the rule.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added a comment explaining this.

// interface. Note that implementations of Describable are generally objects
// which can be described, not simply descriptors; this exception is in place
// to make it more convenient to pass actual descriptors to functions that
// expect Describable objects.
func (d Descriptor) Descriptor() Descriptor {
return d
}

// BlobStatter makes blob descriptors available by digest. The service may
// provide a descriptor of a different digest if the provided digest is not
// canonical.
Expand Down
2 changes: 1 addition & 1 deletion docs/spec/manifest-v2-2.md
Original file line number Diff line number Diff line change
Expand Up @@ -86,7 +86,7 @@ image manifest based on the Content-Type returned in the HTTP response.

- **`os`** *string*

The architecture field specifies the operating system, for example
The os field specifies the operating system, for example
`linux` or `windows`.

- **`variant`** *string*
Expand Down
147 changes: 147 additions & 0 deletions manifest/manifestlist/manifestlist.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,147 @@
package manifestlist
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might be good to have a package doc for the concept of manifest lists with a few examples of how they might be used. Let's not hold up the PR on this, but it might be helpful.


import (
"encoding/json"
"errors"
"fmt"

"github.com/docker/distribution"
"github.com/docker/distribution/digest"
"github.com/docker/distribution/manifest"
)

// MediaTypeManifestList specifies the mediaType for manifest lists.
const MediaTypeManifestList = "application/vnd.docker.distribution.manifest.list.v2+json"

// SchemaVersion provides a pre-initialized version structure for this
// packages version of the manifest.
var SchemaVersion = manifest.Versioned{
SchemaVersion: 2,
MediaType: MediaTypeManifestList,
}

func init() {
manifestListFunc := func(b []byte) (distribution.Manifest, distribution.Descriptor, error) {
m := new(DeserializedManifestList)
err := m.UnmarshalJSON(b)
if err != nil {
return nil, distribution.Descriptor{}, err
}

dgst := digest.FromBytes(b)
return m, distribution.Descriptor{Digest: dgst, Size: int64(len(b)), MediaType: MediaTypeManifestList}, err
}
err := distribution.RegisterManifestSchema(MediaTypeManifestList, manifestListFunc)
if err != nil {
panic(fmt.Sprintf("Unable to register manifest: %s", err))
}
}

// PlatformSpec specifies a platform where a particular image manifest is
// applicable.
type PlatformSpec struct {
// Architecture field specifies the CPU architecture, for example
// `amd64` or `ppc64`.
Architecture string `json:"architecture"`

// OS specifies the operating system, for example `linux` or `windows`.
OS string `json:"os"`

// Variant is an optional field specifying a variant of the CPU, for
// example `ppc64le` to specify a little-endian version of a PowerPC CPU.
Variant string `json:"variant,omitempty"`

// Features is an optional field specifuing an array of strings, each
// listing a required CPU feature (for example `sse4` or `aes`).
Features []string `json:"features,omitempty"`
}

// A ManifestDescriptor references a platform-specific manifest.
type ManifestDescriptor struct {
distribution.Descriptor

// Platform specifies which platform the manifest pointed to by the
// descriptor runs on.
Platform PlatformSpec `json:"platform"`
}

// ManifestList references manifests for various platforms.
type ManifestList struct {
manifest.Versioned

// Config references the image configuration as a blob.
Manifests []ManifestDescriptor `json:"manifests"`
}

// References returnes the distribution descriptors for the referenced image
// manifests.
func (m ManifestList) References() []distribution.Descriptor {
dependencies := make([]distribution.Descriptor, len(m.Manifests))
for i := range m.Manifests {
dependencies[i] = m.Manifests[i].Descriptor
}

return dependencies
}

// DeserializedManifestList wraps ManifestList with a copy of the original
// JSON.
type DeserializedManifestList struct {
ManifestList

// canonical is the canonical byte representation of the Manifest.
canonical []byte
}

// FromDescriptors takes a slice of descriptors, and returns a
// DeserializedManifestList which contains the resulting manifest list
// and its JSON representation.
func FromDescriptors(descriptors []ManifestDescriptor) (*DeserializedManifestList, error) {
m := ManifestList{
Versioned: SchemaVersion,
}

m.Manifests = make([]ManifestDescriptor, len(descriptors), len(descriptors))
copy(m.Manifests, descriptors)

deserialized := DeserializedManifestList{
ManifestList: m,
}

var err error
deserialized.canonical, err = json.MarshalIndent(&m, "", " ")
return &deserialized, err
}

// UnmarshalJSON populates a new ManifestList struct from JSON data.
func (m *DeserializedManifestList) UnmarshalJSON(b []byte) error {
m.canonical = make([]byte, len(b), len(b))
// store manifest list in canonical
copy(m.canonical, b)

// Unmarshal canonical JSON into ManifestList object
var manifestList ManifestList
if err := json.Unmarshal(m.canonical, &manifestList); err != nil {
return err
}

m.ManifestList = manifestList

return nil
}

// MarshalJSON returns the contents of canonical. If canonical is empty,
// marshals the inner contents.
func (m *DeserializedManifestList) MarshalJSON() ([]byte, error) {
if len(m.canonical) > 0 {
return m.canonical, nil
}

return nil, errors.New("JSON representation not initialized in DeserializedManifestList")
}

// Payload returns the raw content of the manifest list. The contents can be
// used to calculate the content identifier.
func (m DeserializedManifestList) Payload() (string, []byte, error) {
return m.MediaType, m.canonical, nil
}
111 changes: 111 additions & 0 deletions manifest/manifestlist/manifestlist_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,111 @@
package manifestlist

import (
"bytes"
"encoding/json"
"reflect"
"testing"

"github.com/docker/distribution"
)

var expectedManifestListSerialization = []byte(`{
"schemaVersion": 2,
"mediaType": "application/vnd.docker.distribution.manifest.list.v2+json",
"manifests": [
{
"mediaType": "application/vnd.docker.distribution.manifest.v2+json",
"size": 985,
"digest": "sha256:1a9ec845ee94c202b2d5da74a24f0ed2058318bfa9879fa541efaecba272e86b",
"platform": {
"architecture": "amd64",
"os": "linux",
"features": [
"sse4"
]
}
},
{
"mediaType": "application/vnd.docker.distribution.manifest.v2+json",
"size": 2392,
"digest": "sha256:6346340964309634683409684360934680934608934608934608934068934608",
"platform": {
"architecture": "sun4m",
"os": "sunos"
}
}
]
}`)

func TestManifestList(t *testing.T) {
manifestDescriptors := []ManifestDescriptor{
{
Descriptor: distribution.Descriptor{
Digest: "sha256:1a9ec845ee94c202b2d5da74a24f0ed2058318bfa9879fa541efaecba272e86b",
Size: 985,
MediaType: "application/vnd.docker.distribution.manifest.v2+json",
},
Platform: PlatformSpec{
Architecture: "amd64",
OS: "linux",
Features: []string{"sse4"},
},
},
{
Descriptor: distribution.Descriptor{
Digest: "sha256:6346340964309634683409684360934680934608934608934608934068934608",
Size: 2392,
MediaType: "application/vnd.docker.distribution.manifest.v2+json",
},
Platform: PlatformSpec{
Architecture: "sun4m",
OS: "sunos",
},
},
}

deserialized, err := FromDescriptors(manifestDescriptors)
if err != nil {
t.Fatalf("error creating DeserializedManifestList: %v", err)
}

mediaType, canonical, err := deserialized.Payload()

if mediaType != MediaTypeManifestList {
t.Fatalf("unexpected media type: %s", mediaType)
}

// Check that the canonical field is the same as json.MarshalIndent
// with these parameters.
p, err := json.MarshalIndent(&deserialized.ManifestList, "", " ")
if err != nil {
t.Fatalf("error marshaling manifest list: %v", err)
}
if !bytes.Equal(p, canonical) {
t.Fatalf("manifest bytes not equal: %q != %q", string(canonical), string(p))
}

// Check that the canonical field has the expected value.
if !bytes.Equal(expectedManifestListSerialization, canonical) {
t.Fatalf("manifest bytes not equal: %q != %q", string(canonical), string(expectedManifestListSerialization))
}

var unmarshalled DeserializedManifestList
if err := json.Unmarshal(deserialized.canonical, &unmarshalled); err != nil {
t.Fatalf("error unmarshaling manifest: %v", err)
}

if !reflect.DeepEqual(&unmarshalled, deserialized) {
t.Fatalf("manifests are different after unmarshaling: %v != %v", unmarshalled, *deserialized)
}

references := deserialized.References()
if len(references) != 2 {
t.Fatalf("unexpected number of references: %d", len(references))
}
for i := range references {
if !reflect.DeepEqual(references[i], manifestDescriptors[i].Descriptor) {
t.Fatalf("unexpected value %d returned by References: %v", i, references[i])
}
}
}
Loading
0