Proposal: Registry v2 authorization service #9081

jlhawn · 2014-11-11T05:08:47Z

Outlines usage for endpoints which will be used by the
docker registry client for registry v2 non-standalone.

Docker-DCO-1.1-Signed-off-by: Josh Hawn josh.hawn@docker.com (github: jlhawn)

stevvooe · 2014-11-11T19:17:23Z

registry/AUTH.md

+
+    The Claim Set will also contain a private claim name unique to this authorization server specification:
+
+    <dt>"scope"</dt>


The dl needs to be reopened here to get the right formatting.

wking · 2014-11-11T21:21:27Z

“By adding your own HTTP authorization requirements in a proxy placed
between the client and the registry you can control access on an
all-or-nothing basis.”

You can make this as granular as you like in proxy-based auth, as long
as any information you need to determine auth is included in the
request header or negotiated through WWW-Authenticate headers 1.
Using long-duration tokens is fine, but I don't think “you can't do
this kind of auth in a reverse-proxy” should be the argument you use
to motivate it (“Nginx buffers upload bodies and there's currently no
way to turn that off” is a good argument though 2 ;).

wking · 2014-11-11T21:31:53Z

“{"typ":"JWT","alg":"ES256","kid":…}”

Shouldn't these whitespace-less versions for signing also have their
keys alphabetized? Or do consumers have to preserve the original key
ordering when shipping these about or validating them? Or is there
some canonical key ordering?

wking · 2014-11-11T21:41:05Z

“Check that the JWT ID (jti claim) value has not been seen before”

So we're going to need a new token for every registry request? There
were some concerns about high availability before (in the context of
registry ↔ auth service connections 1), which I expect extend to the
connection between the auth server and the client. Maybe folks
worried about network issues can just disable the JWT ID check in
their registry, and have clients optimistically try and reuse their
token?

jlhawn · 2014-11-11T22:00:25Z

@wking you're right, I realize you can configure nginx to place the authenticated user's username in a header and do namespace authorization that way, and there's definitely a variety of other creative things you could. These schemes (including the one outlined here) would all require pluggable authentication and authorization layers in the registry itself to interpret the headers correctly and recognize what level of access is required for a given request. (unless you decide to put all of this logic in the proxy itself - I don't have any experience with proxy auth outside of TLS client certs and basic authentication but it sounds like you'd have to do something custom no matter what)

For the JWT headers, yes, the original version of the key is what is signed. You can look at the JWT spec for more information: https://tools.ietf.org/html/draft-ietf-oauth-json-web-token-30#section-7

Note that white space is explicitly allowed in the representation and no canonicalization need be performed before encoding.

So whitespace is allowed, I just removed them to make the token more compact.

For your comment on using jti, yes, I think whether to require single use tokens should be configurable by the registry. Since a push or pull operation actually requires many requests (on the order of number of layers in an image - even more with chunked upload and download) I would imagine that the registry would exchange this JWT for a session identifier to handle the rest of the push/pull operation without requiring a new token for every unique request.

jlhawn · 2014-11-11T22:05:42Z

rebased with a picture describing the proposed process and how it differs from the currently existing process involving an index service.

wking · 2014-11-11T22:25:57Z

On Tue, Nov 11, 2014 at 02:00:35PM -0800, Josh Hawn wrote:

These schemes … would all require pluggable authentication and
authorization layers in the registry itself to interpret the headers
correctly and recognize what level of access is required for a given
request.

With the reverse-proxy, http_auth_request approach the
reverse-proxy/registry combo only has to know the URI of the auth
service. But I think your point is that you're going to need auth
code somewhere, and that makes sense to me ;). My point was just that
you should pick another phrase here to motivate the token-based auth,
if you think it needs motivating at all.

Note that white space is explicitly allowed in the representation
and no canonicalization need be performed before encoding.

So whitespace is allowed, I just removed them to make the token more
compact.

I guess I just wanted to clarify that the signed data needs to be
stored as an opaque byte stream until the signature is verified, since
round-tripping through something that can reorder keys is not going to
work. For example, Python stores decoded JSON in unsorted dicts:

$ python -c "import json; print(json.dumps(json.loads('{"b": 1, "a": 2}')))"
{"a": 2, "b": 1}

which would no longer validate.

For your comment on using jti, yes, I think whether to require
single use tokens should be configurable by the registry. Since a
push or pull operation actually requires many requests (on the
order of number of layers in an image - even more with chunked
upload and download) I would imagine that the registry would
exchange this JWT for a session identifier to handle the rest of the
push/pull operation without requiring a new token for every unique
request.

Hmm, so “don't use these tokens to track state, come up with your own
persistent session handling mechanism”? If you're not worried about
replay attacks against that mechanism (whatever it is) (and I think
not worring is fair over HTTPS), then why not just relax the
single-use token requirement?

jlhawn · 2014-11-11T22:44:13Z

... you should pick another phrase here to motivate the token-based auth, if you think it needs motivating at all

I'll clarify that to better express that we'd like a native authorization mechanism that's public key based with ACLs managed separately.

... then why not just relax the single-use token requirement?

Yeah, I think it should be every actor's responsibility to make sure that communication channels are secure and that the token is not leaked to any other parties, but the token is significantly larger and its verification more computationally intensive than a simple shared secret/session string. So I may just be prematurely optimizing with this detail. As was suggested above though, I think it should be configureable on the registry server.

wking · 2014-11-11T22:52:55Z

On Tue, Nov 11, 2014 at 02:44:21PM -0800, Josh Hawn wrote:

... you should pick another phrase here to motivate the
token-based auth, if you think it needs motivating at all

I'll clarify that to better express that we'd like a native
authorization mechanism that's public key based with ACLs managed
separately.

Works for me.

... then why not just relax the single-use token requirement?

… As was suggested above though, I think it should be configureable
on the registry server.

This works for me too, but I'd point out in the spec that the JWT ID
check for single use tokens is optional, as far as the API is
concerned. A way for the registry to tell the client that its ok to
reuse tokens would be nice too. Or maybe just assume they're
reusable, since registries that don't allow it will just give you a
401 telling you to get a new token?

8000

jlhawn · 2014-11-11T22:57:27Z

thanks @wking

I've just updated the document to clarify the motivation of using this auth mechanism:

... While adding your own HTTP authorization requirements in a proxy placed between the client and the registry can give you greater access control, we'd like a native authorization mechanism that's public key based with access control lists managed separately with the ability to have fine granularity in access control on a by-key, by-user, by-namespace, and by-repository basis. ...

And to clarify that single-use tokens (checking the jti claim) is optional:

If enforcing single-use tokens, check that the JWT ID (jti claim) value has not been seen before.

To enforce this, the registry may keep a record of jtis it has seen for up to the exp time of the token to prevent token replays.

ncdc · 2014-11-12T14:06:14Z

registry/AUTH.md

+3. The registry client makes a request to the authorization service for a signed JSON Web Token.
+4. The authorization service returns a token.
+5. The client retries the original request with the token embedded in the request header.
+6. The Registry authorizes the client and begins the push/pull session as ussual. 


typo: ussual

aweiteka · 2014-11-12T18:57:39Z

@jlhawn @dmcgowan can you help me understand how this relates to the libtrust server proposal? docker/libtrust#36

smarterclayton · 2014-11-12T19:20:57Z

registry/AUTH.md

+
+## How to authenticate
+
+Today, registry clients first contact the index to initiate a push or pull. For v2, clients should contact the registry first. If the registry server requires authentication it will return a `401 Unauthorized` response with a `WWW-Authenticate` header detailing how to authenticate to this registry.


This proposal seems to step away from some of the earlier discussion about authorization and authentication that occurred in the early registry meetings on IRC, specifically about being more flexible to third party auth solutions. Is this proposal saying that only one type of authentication (JWT token auth integration with a docker auth server) is supported from the client? Or is this proposal spec'd just to that type?

If the latter, it would be good to highlight in this section that this is specifically identifying the JWT token flow and describe Basic / Negotiate (or link to a separate proposal)

@smarterclayton docker-archive/docker-registry#623 - basic authentication (implemented by a proxy design in front of the registry service) is validated and will be implemented - this proposal here only covers the alternative key-based, JWT support.

So, I believe this is in line with previous discussions on that subject:

allow for third-party to plug themselves into a proxy talking basic auth with the engine

support for a key based auth mechanism to be used at least by the Hub, and that could also be used by other people granted they would implement a matching auth service

You are right though, this should be made more clear in that document here.

Thanks for clarifying - so many good proposals bubbling around it's easy to miss one.

On Nov 12, 2014, at 5:25 PM, Olivier Gambier notifications@github.com wrote:

In registry/AUTH.md:

+Instructs the server to associate the given key with the specified and authenticated account. Requires HTTP Authorization Header with basic auth (username:password) for the given account. Payload is public key with in JWK (JSON Web Key) format.
+
+Example Payload from client:
+
+```
+{

"kty": "EC",

"crv": "P-256",

"x": "yGnNRQwOFTbHElKcy-PUEbCSO5g6LEzKgmYJMLwwG8A",

"y": "u9FwaoJLfXAxj0Z3dHsR7Fm542XLenQw7DVrIJ2CNFc"
+}
+```

+## How to authenticate
+
+Today, registry clients first contact the index to initiate a push or pull. For v2, clients should contact the registry first. If the registry server requires authentication it will return a401 Unauthorizedresponse with aWWW-Authenticate header detailing how to authenticate to this registry.
@smarterclayton docker-archive/docker-registry#623 - basic authentication (implemented by a proxy design in front of the registry service) is validated and will be implemented - this proposal here only covers the alternative key-based, JWT support.

So, I believe this is in line with previous discussions on that subject:

allow for third-party to plug themselves into a proxy talking basic auth with the engine
support for a key based auth mechanism to be used at least by the Hub, and that could also be used by other people granted they would implement a matching auth service
You are right though, this should be made more clear in that document here.

—
Reply to this email directly or view it on GitHub.

jessfraz · 2014-12-02T22:02:38Z

SGTM

dmp42 · 2014-12-03T00:01:08Z

@shykes

yes
expiracy is the "revocation" mechanism for baretokens, yes (and the used life length should probably be in minutes)

Is this supposed to use the same trust infrastructure as image signature?

Certainly - and which is also the same users database / grants that the hub uses currently.
And indeed, grants, account/key associations will be common with it.
Now, this is a first step which (limited) objective is to upgrade the authentication scheme used by and for registries to make it pluggable to the trust graph down the road.

How does authorization work exactly?

Short time, it will work through the existing hub mechanisms: authorize user x to repository y, clicky click.
Later when we can get the other bits of the trust service in production (keys, and the trust proposal), endpoints to grant authorization / associate/revoke keys will be added, and the hub will move to use the trust graph as well (<- this only (eg: upgrading the hub to a new ACL database) is a huge piece of work).

Right now, this PR is about getting the engine to move away from the v1 authentication protocol which prevents us from changing the hub, and which shortcomings are no longer bearable: poor scalability (on-pull, per entity auth doesn't fly), reliability issues, granularity issues, to a JWT signed token protocol that is compatible with the "trust" vision that you devised.

Hope that clarifies.

stevvooe · 2014-12-11T19:34:18Z

registry/AUTH.md

+Once the client has a token, it will try the registry request again with the token placed in the HTTP `Authorization` header like so:
+
+```
+Authorization: Token eyJ0eXAiOiJKV1QiLCJhbGciOiJFUzI1NiIsImtpZCI6IkJWM0Q6MkFWWjpVQjVaOktJQVA6SU5QTDo1RU42Ok40SjQ6Nk1XTzpEUktFOkJWUUs6M0ZKTDpQT1RMIn0.eyJpc3MiOiJhdXRoLmRvY2tlci5jb20iLCJzdWIiOiJCQ0NZOk9VNlo6UUVKNTpXTjJDOjJBVkM6WTdZRDpBM0xZOjQ1VVc6NE9HRDpLQUxMOkNOSjU6NUlVTCIsImF1ZCI6InJlZ2lzdHJ5LmRvY2tlci5jb20iLCJleHAiOjE0MTUzODczMTUsIm5iZiI6MTQxNTM4NzAxNSwiaWF0IjoxNDE1Mzg3MDE1LCJqdGkiOiJ0WUpDTzFjNmNueXk3a0FuMGM3cktQZ2JWMUgxYkZ3cyIsInNjb3BlIjoiamxoYXduOnJlcG9zaXRvcnk6c2FtYWxiYS9teS1hcHA6cHVzaCxwdWxsIGpsaGF3bjpuYW1lc3BhY2U6c2FtYWxiYTpwdWxsIn0.Y3zZSwaZPqy4y9oRBVRImZyv3m_S9XDHF1tWwN7mL52C_IiA73SJkWVNsvNqpJIn5h7A2F8biv_S2ppQ1lgkbw


Should this be "Bearer" rather than "Token" to be compatible with RFC 6750?

Outlines usage for endpoints which will be used by the docker registry client for registry v2 non-standalone. Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)

- Updated entire document to wrap after 80 columns where appropriate. - Clarified private claim set fields - now using `access` struct instead of difficult to understand `scope` string. - linked to up-to-date and relevent RFCs where appropriate - JWA draft spec - Oauth 2.0 RFC - Bearer Token Usage - updated signature example to reflect new claim set structure. Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)

jlhawn · 2014-12-12T20:02:37Z

@stevvooe @dmcgowan @jfrazelle I've updated the draft spec, please review again when you get a chance!

stevvooe · 2014-12-13T01:11:13Z

LGTM!

Particularly, I like how we've cleared up the relationship to OAuth2.

SvenDowideit · 2014-12-19T03:30:24Z

@jlhawn can you do potential client implementors a favour and add examples using curl?

or in a more realistic place - LGTM

jlhawn · 2014-12-23T22:17:20Z

client implementation developed in #9784

SvenDowideit · 2014-12-27T01:25:38Z

y - the point is to document the server with examples in the documentation that can show how it works.

t5unamie · 2015-01-07T10:43:40Z

Do you know when this will be included into a stable release of docker?

jessfraz · 2015-01-20T19:15:35Z

@jlhawn do we need these docs now that #9784 is in?

mmdriley · 2015-01-26T18:37:06Z

Why was the implementation checked in before the spec?

dmp42 · 2015-01-26T19:19:42Z

@mmdriley it's not clear yet what the process should be to merge such proposals (eg: before or after implementation) - for example: should documentation be merged before a feature lands?
Either way, this should be merged indeed.

SvenDowideit · 2015-01-26T23:38:47Z

@dmp42 generally the documentation and the code should to be merged at the same time. In 1.6, We'd like to make this simpler, by moving the registry documentation into the registry repository, and importing it into the docs from there.

jlhawn · 2015-01-26T23:42:32Z

@jfrazelle @icecrime do you think it's best to move this to the https://github.com/docker/distribution repo? It may be more relevant here: https://github.com/docker/distribution/tree/master/auth/token

jessfraz · 2015-01-26T23:47:14Z

yep that makes sense :)

dmp42 · 2015-01-26T23:51:49Z

@jlhawn +1

jlhawn · 2015-01-27T19:32:52Z

Reopened as distribution/distribution#110

stevvooe reviewed Nov 11, 2014
View reviewed changes

dmp42 mentioned this pull request Nov 11, 2014

NG: Auth model docker-archive/docker-registry#623

Closed

jlhawn force-pushed the registry_auth_md branch from 4fffdb7 to b6d962b Compare November 11, 2014 21:16

jlhawn force-pushed the registry_auth_md branch from b6d962b to 9b12471 Compare November 11, 2014 22:04

jlhawn force-pushed the registry_auth_md branch from 9b12471 to 403d647 Compare November 11, 2014 22:47

jlhawn force-pushed the registry_auth_md branch from 403d647 to e28cb89 Compare November 11, 2014 22:54

jlhawn force-pushed the registry_auth_md branch from e28cb89 to 998b05b Compare November 11, 2014 22:58

jlhawn changed the title ~~Initial spec for registry authentication server~~ Proposal: Registry v2 authorization service Nov 11, 2014

jlhawn added Proposal labels Nov 11, 2014

ncdc reviewed Nov 12, 2014
View reviewed changes

jlhawn force-pushed the registry_auth_md branch from 998b05b to cba6c21 Compare November 12, 2014 17:52

smarterclayton reviewed Nov 12, 2014
View reviewed changes

This was referenced Nov 14, 2014

docker registry with ssl works well but has nginx error docker-archive/docker-registry#732

Open

Proposal: JSON Registry API V2.1 #9015

Closed

This was referenced Nov 20, 2014

Question: OAuth / OpenID for Docker Registry docker-archive/docker-registry#759

Closed

Update Hub and Registry Spec to Include Digital Signature Steps #9306

Closed

jlhawn mentioned this pull request Dec 7, 2014

Ownership, Governance, TM, etc rkt/rkt#139

Closed

philips mentioned this pull request Dec 9, 2014

Ownership, Governance, TM, etc appc/spec#21

Closed

stevvooe reviewed Dec 11, 2014
View reviewed changes

Josh Hawn added 2 commits December 12, 2014 10:45

Initial spec for registry authentication server

5216ebe

Outlines usage for endpoints which will be used by the docker registry client for registry v2 non-standalone. Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)

jlhawn force-pushed the registry_auth_md branch from af5054d to 51bea83 Compare December 12, 2014 19:59

dmp42 mentioned this pull request Dec 16, 2014

Authentication docker-archive/docker-registry#852

Open

jlhawn mentioned this pull request Dec 17, 2014

Adds auth package with token auth backend docker-archive/docker-registry#859

Merged

SvenDowideit added the /project/doc label Dec 19, 2014

dmcgowan mentioned this pull request Jan 25, 2015

dockercfg insecure #10318

Closed

jlhawn closed this Jan 27, 2015

jlhawn mentioned this pull request Jan 27, 2015

doc/spec: documentation for token based auth distribution/distribution#110

Merged

jlhawn deleted the registry_auth_md branch July 31, 2015 18:04


		The Claim Set will also contain a private claim name unique to this authorization server specification:

		<dt>"scope"</dt>


		## How to authenticate

		Today, registry clients first contact the index to initiate a push or pull. For v2, clients should contact the registry first. If the registry server requires authentication it will return a `401 Unauthorized` response with a `WWW-Authenticate` header detailing how to authenticate to this registry.

Proposal: Registry v2 authorization service #9081

Proposal: Registry v2 authorization service #9081

Uh oh!

Conversation

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!