Extending with custom client resolution #133

damooo · 2023-10-31T18:04:19Z

damooo
Oct 31, 2023

Solid-Oidc protocol extends oidc, and makes client registration optional, to enable decentralised ephemereal clients. It does so by allowing "bring your own id" model, where client_id can be a webid, essentially a http uri that can be supplied by a client, that derefers to a webid doc. That doc contains client details to be used.

Any quick guidance on how to extend rauthy would be highly appreciated.

rauthy can be one of the first robust solid-oidc compat open source idp for solid ecosystem with this last extension.

Thanks again.

sebadob · 2023-10-31T18:10:31Z

sebadob
Oct 31, 2023
Maintainer

You mean, the request to the /oidc/authorize endpoint would be the same, but instead of client_id=my-client-123 it would be something like webid=https://example.com ?

Do you have an example how this doc would look like?

1 reply

damooo Oct 31, 2023
Author

yes.

This section has an example doc.

damooo · 2023-10-31T18:18:09Z

damooo
Oct 31, 2023
Author

This quick primer has basic example flow enacted.

4 replies

sebadob Oct 31, 2023
Maintainer

This looks actually pretty easy.

I guess Rauthy could implement a config variable, something like ENABLE_EMPHEMERAL_CLIENTS for instance.
I that case, the only thing Rauthy needs to do is to fetch information from the url, if the client_id has the format of a URL.
Rauthy needs and uses more values than provided for its clients, but the missing ones could just be the default values. Since S256 PKCE is the default anyway when you register a new client in the UI, this would already work out.

So it would basically mean:

implement DPoP tokens
add the new config variable
have most probably some internal caching mechanism for these ephemeral client lookup URLs (I guess?)
if the client URL is not in the cache, fetch the data, build a Rauthy client from the values + defaults
Just go on like normal with the login procedure

...right?

edit:

Just another thought... how often do these ephemeral clients change? Or could it be, that it does just one request and thats it? In that case I guess I would not even bother about storing any information about that client at all.

damooo Oct 31, 2023
Author

Yes, they are supposed to be freshly resolved per token request. But cache with configurable invalidation time could be helpful.

Another required orthogonal extension is to associate a webid for each user. And provide a webid endpoint that serve webid document containing simple user profile, and info about authoritative idp, and any other extensions like pub key etc. This can be easily implemented as a layer on top of existing functionality i suppose.

sebadob Oct 31, 2023
Maintainer

Okay, that sounds like not such a big overhead and would actually be a nice feature.

I will implement DPoP first and make sure it works.

The other part would not be too complicated and could come afterwards.
I saw that solid would work with a static pre-configured client as well, if the OP does not support the ephemeral one. Which means, if I got it right, that DPoP is the most important missing piece for that.

damooo Oct 31, 2023
Author

And then if token request contain webid scope, granted id token payload must include webid claim.

webid must be included in scopes_supported in idp's discovery resource.

sebadob · 2023-11-08T15:36:06Z

sebadob
Nov 8, 2023
Maintainer

The nightly version from today should (hopefully) provide full support for Solid OIDC.

You need to do a few things to enable full compatibility, since it is opt-in:

There is a whole new section in the config file called EPHEMERAL CLIENTS.
Default values should be good, you just need to enable the following:

# Can be set to 'true' to allow the dynamic client lookup via URLs as
# 'client_id's during authorization_code flow initiation.
# default: false
ENABLE_EPHEMERAL_CLIENTS=true

# If set to 'true', a possibly existing 'webid' scope with an existing custom
# attribute for a user will be mapped into the ID token's root (instead of
# custom.webid) with the claim name 'webid'
# default: false
ENABLE_WEBID_MAPPING=true

# If set to 'true', 'solid' will be added to the 'aud' claim from the ID token
# for ephemeral clients.
# default: false
ENABLE_SOLID_AUD=true

The webid scope / claim will be handled like any other custom scope. The only difference is, that the ENABLE_WEBID_MAPPING variable will pull the webid claim out of the custom section and map it into the token payload root, to be compliant with the Solid OIDC spec.
But since it is just like any other custom scope, you need to add it to Rauthy:

Create a new custom attribute called webid
Create a new custom scope called webid
Edit the new scope and add a token mapping for the ID token
Set the correct webid value for each of your users

This should make everything work.

You can of course adjust the other new values from the EPHEMERAL CLIENTS to your liking. Rauthy will actually sanitize the requests scrops and auth flows from an ephemeral client, if the requested ones are not configured with EPHEMERAL_CLIENTS_ALLOWED_FLOWS and EPHEMERAL_CLIENTS_ALLOWED_SCOPES. Basically, the behavior is the same as for all other pre-registered clients.
A sidenote: ephemeral clients are currently forced to do S256 PKCE and this cannot be changed.

Rauthy does not host any webid document and I am not sure, if it ever will. I read into the RFC for webid's and they are basically something that a social network would / should provide. It cares about connections between users and all such stuff, which Rauthy just does not by definition. We could maybe think about putting that behind a feature flag and compile a version in the future that does support this, but the overhead is pretty big. A proper implementation of this feature would require quite a lot of work.

Please let me know, how it works out for you.

Edit:

Sorry forgot to add the nightly version:

ghcr.io/sebadob/rauthy:0.18.1-20231108

or

ghcr.io/sebadob/rauthy:0.18.1-20231108-lite

8 replies

sebadob Nov 8, 2023
Maintainer

10000

I thought about pre-defining the webid scope, but I left it fully open on purpose. Other users maybe do not even want or care about Solid OIDC and have webid in use in some other logic and have completely a different use case for this. That's why I left the webid a custom scope just like any other scope too. Additionally, if it is left fully open, users could either use the webid from Rauthy, or just refer to an already existing webid in another place. They would not be locked in to using it from Rauthy's side.

About the metadata for the web document:
In which format would the user be expected to provide this? Because I always think about non-technical persons as well, since these usually have the biggest problems with things like that.

damooo Nov 8, 2023
Author

We cannot link to a custom webid as token claim, until we can prove user indeed owns that. We are only sure that, the webid provided by Rauthy is owned by the user. Thus an identity provider provides a webid. User can always link custom webids. But Rauthy can only provide it's provisoned webid in the token claim.

Thus we can have a single config for entire webid basic support, like ENABLE_WEBID, that will:

Enable webid document endpoint,
Predefines webid scope.
Serves webid claim in id token, when webid scope is requested.

This will allow seemless, and configurable experience.

In which format would the user be expected to provide this? Because I always think about non-technical persons as well, since these usually have the biggest problems with things like that.

It would be turtle. But that can be abstracted away with custom ui for simple key-value like editing. I would address that in seperate PR.

sebadob Nov 8, 2023
Maintainer

Okay, got it.
Yes let's do turtle format first and do a custom key/value UI part later on.

I guess a new table would make sense as well where the webid information would be stored for each user, or at least any custom additions. Pre-defining the webid scope should of course only be done, if it does not exist already. If so, it should not overwrite it, since it may have some totally different meaning to the admin and possibly linked attributes and so on.

I guess I would even make the custom metadata in a separate PR and do the basic webid first. The most important thing to me is, that the user decides himself in the end from the account page, if the webid should be exposed or not. I can do some more adoptions tomorrow and prepare all of this and you could do the PR for the turtle format stuff maybe, because I have never worked with this, if you like.

damooo Nov 8, 2023
Author

that the user decides himself in the end from the account page, if the webid should be exposed or not

Can that be configurable system-wide too? I.e. admin can configure and start RAuthy in a way, where every user of his service must have webid.

I can do some more adoptions tomorrow and prepare all of this and you could do the PR for the turtle format stuff maybe, because I have never worked with this, if you like.

Indeed _/\_. I would make pr, once you give green flag.
Allowing custom metadata will be essential, as one have to provide his pub-keys for http-signatures, etc. If not a cusom-field, the a column in the new table can persist metadata.

damooo Nov 8, 2023
Author

Pre-defining the webid scope should of course only be done, if it does not exist already. If so, it should not overwrite it, since it may have some totally different meaning to the admin and possibly linked attributes and so on.

The config option ENABLE_WEBID can imply admin's choice for the webid semantics.

sebadob · 2023-11-09T17:01:40Z

sebadob
Nov 9, 2023
Maintainer

So...

with PR #158 I did almost all of it.
It was quite a bit more work in the end then expected.

The 3 config values needed are now:

# Can be set to 'true' to allow the dynamic client lookup via URLs as
# 'client_id's during authorization_code flow initiation.
# default: false
ENABLE_EPHEMERAL_CLIENTS=true

# Can be set to 'true' to enable WebID functionality like needed
# for things like Solid OIDC.
# default: false
ENABLE_WEB_ID=true

# If set to 'true', 'solid' will be added to the 'aud' claim from the ID token
# for ephemeral clients.
# default: false
ENABLE_SOLID_AUD=true

Some things left:

The webid scope still needs to be created manually once and it will stay like that. The reason is simply compatibility and not to give up any abilities to do custom scope mapping. Like it is implemented now, we basically have the best of both worlds:
Rauthy will serve the WebID document, which you can add custom key / value pairs to and you will have webid as a custom scope which you can use like shown above to do any kind of custom mapping that Rauthy can do.
For the custom data, I assumed String / String now as key / value pairs. I hope that this works out.
I saw that turtle works like graph query language, for instance like sparql. This would mean we need a predicate for the custom key / value pairs. Will this in this case always be the same, which we could hardcode, or do we need to make this dynamic and user-selectable too?
Everything should be working, except for one fn where I would like you to provide a PR:
WebIdResponse::as_turtle() in rauthy-models/src/response.rs
This is called from the get_user_webid handler in rauthy-handlers/src/users.rs
The content type header is set already to text/turle (hope this is correct)

I have provided some values for the WebIdResponse already. I hope they are good. If not, just add some more in the get_user_webid where you should have everything you would need, I guess. Currently, the as_turle() just returns some String to see if it works. I would really appreciate it if we can make this work properly without adding additional dependencies, because the compilation takes such a long time already.

I have tested the UI part and all the test cases are fine, but it could be, that there are still some bugs in the new UI parts.

I have not written any "developer introduction" yet, but to start it up locally, I would highly suggest that you have just available, which makes this really easy. Apart from that, you only need npm to do an npm install in the frontend folder. Afterwards, you should be able to:

create DB with just migrate-sqlite
start the backend with just run-sqlite
start the frontend (in another terminal) with just run-ui
you can access it on localhost:5173

There is only one thing about the UI in local dev. The way the static file adapter from svelte is set up right now, you will not be able to access localhost:5173/auth/v1, you actually would need to append /index here once. The rest works just the same and like expected: http://localhost:5173/auth/v1/index

As long as you don't use passkeys, it should work right away. If you however want to test passkeys with the local dev ui, you need to adjust the port for RP_ORIGIN

Edit:
You don't need to do all the other steps from above for the webid scope like attribute creation and all these things. You only need to create the scope itself like in step 2 without anything else, that's it.

15 replies

damooo Nov 13, 2023
Author

On the ui side, you need to use n3.js (very tiny and capable) lib for serde of graphs. The rio turtle parser is low level, but highly standards compatible otherwise, and shouldn't give any problems. I should check the new version and follow up as specified.

sebadob Nov 13, 2023
Maintainer

To quickly execute the test cases, I would suggest that you cd rauthy-models and execute cargo test --features sqlite test_web_id_response and PUB_URL="localhost:8080" cargo test --features sqlite test_web_id_response directly. This avoids the costly compilation of all the integration tests from the rauthy-main crate each time.

sebadob Nov 13, 2023
Maintainer

On the ui side, you need to use n3.js (very tiny and capable) lib for serde of graphs.

We don't need to have this additional dependency and validation on the UI, when we can just use the error from the backend, which then is the exact same code that should work later on as well I would say. This is just another possible source for errors and bugs.

damooo Nov 13, 2023
Author

The issue was, the custom data is not persisted in turtle, but n-triples, as previously documented. n-triples is a subset of turtle, that allows for zero cost serialization and deserialization. As Webid is being serialized and deserialized a lot to and from cache, n-triples was used to canonically represent graph. This allows for all graph terms and triples to be backed by just one string, instead of allocating for many many terms and triples for every ser/de.

When we take input from user in turtle instead of n-triples, we should canonicalize back to n-triples subset before persisting.

sebadob Nov 13, 2023
Maintainer

Ahh got it, yes taking it in as turtle is a way better UX I would say, because then you basically write the format that you would expect in the output.

I actually did not want to persist the data cleaned up on purpose, because when you come back to the UI later, you get your data in a totally different format.
I guess I would rather do something else like persisting an additional copy or something, so we can persist a pre-computed, cleaned up and correct form we can use for each GET request, but maybe return the original data the user has entered when coming back to the UI component. But this is optimization when everything is working fine.

Uh oh!

Extending with custom client resolution #133

Uh oh!

Uh oh!

damooo Oct 31, 2023

Replies: 4 comments · 28 replies

Uh oh!

sebadob Oct 31, 2023 Maintainer

Uh oh!

damooo Oct 31, 2023 Author

Uh oh!

damooo Oct 31, 2023 Author

Uh oh!

Uh oh!

sebadob Oct 31, 2023 Maintainer

Uh oh!

Uh oh!

damooo Oct 31, 2023 Author

Uh oh!

sebadob Oct 31, 2023 Maintainer

Uh oh!

damooo Oct 31, 2023 Author

Uh oh!

Uh oh!

sebadob Nov 8, 2023 Maintainer

Uh oh!

sebadob Nov 8, 2023 Maintainer

Uh oh!

Uh oh!

damooo Nov 8, 2023 Author

Uh oh!

sebadob Nov 8, 2023 Maintainer

Uh oh!

Uh oh!

damooo Nov 8, 2023 Author

Uh oh!

damooo Nov 8, 2023 Author

Uh oh!

Uh oh!

sebadob Nov 9, 2023 Maintainer

Uh oh!

damooo Nov 13, 2023 Author

Uh oh!

sebadob Nov 13, 2023 Maintainer

Uh oh!

Uh oh!

sebadob Nov 13, 2023 Maintainer

Uh oh!

damooo Nov 13, 2023 Author

Uh oh!

sebadob Nov 13, 2023 Maintainer

damooo
Oct 31, 2023

Replies: 4 comments 28 replies

sebadob
Oct 31, 2023
Maintainer

damooo Oct 31, 2023
Author

damooo
Oct 31, 2023
Author

sebadob Oct 31, 2023
Maintainer

damooo Oct 31, 2023
Author

sebadob Oct 31, 2023
Maintainer

damooo Oct 31, 2023
Author

sebadob
Nov 8, 2023
Maintainer

sebadob Nov 8, 2023
Maintainer

damooo Nov 8, 2023
Author

sebadob Nov 8, 2023
Maintainer

damooo Nov 8, 2023
Author

damooo Nov 8, 2023
Author

sebadob
Nov 9, 2023
Maintainer

damooo Nov 13, 2023
Author

sebadob Nov 13, 2023
Maintainer

sebadob Nov 13, 2023
Maintainer

damooo Nov 13, 2023
Author

sebadob Nov 13, 2023
Maintainer