8000 fix: role extraction from access token in keycloak oidc by Elektordi · Pull Request #1916 · oauth2-proxy/oauth2-proxy · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

fix: role extraction from access token in keycloak oidc #1916

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Apr 28, 2025

Conversation

Elektordi
Copy link
Contributor
@Elektordi Elektordi commented Nov 29, 2022

The keycloak_oidc provider uses the access_token instead of the id_token.

Description

I had 500 errors when using keycloak_oidc provider out of the box, and logs where indicating a problem with the audience:

[oauthproxy.go:830] Error creating session during OAuth2 callback: audience claims [aud] do not exist in claims: [...]

Motivation and Context

oauth2-proxy relies on the audience "aud" field, and the officiel keycloak documentation states that the access_token is not design to hold the audience of the client requesting the token:

https://www.keycloak.org/docs/latest/server_admin/#_audience_resolve

The frontend client itself is not automatically added to the access token audience, therefore allowing easy differentiation between the access token and the ID token, since the access token will not contain the client for which the token is issued as an audience.

It is the id_token which is designed for this, and this token is for the OIDC protocol. (where the access_token is for OAuth2 protocol)

It was also confirmed by Keycloak devs on their mailing list some time ago: https://lists.jboss.org/pipermail/keycloak-user/2019-August/018924.html

How Has This Been Tested?

It has been tested with a brand now keycloak and oauth2-proxy, both deployed with docker, and using traefik/whoami as a backend.
Command line: docker run --rm --name oauth2proxy -p 4180:4180 oauth2-proxy --provider=keycloak-oidc --oidc-issuer-url=http://172.17.0.1:8080/realms/test --email-domain=* --upstream=http://172.17.0.1:49156/ --reverse-proxy --http-address=0.0.0.0:4180 --client-id=dev-localhost --client-secret=xxxxxxxxxxxx --cookie-secret=xxxxxxxxxxxxxxxx --redirect-url=http://localhost:4180/oauth2/callback --skip-jwt-bearer-tokens --insecure-oidc-allow-unverified-email

It was tested with both web access, and bearer api access.

Before change: HTTP 500 error on auth2-proxy

After change : No problem.

Checklist:

  • My change requires a change to the documentation or CHANGELOG.
  • I have updated the documentation/CHANGELOG accordingly.
  • I have created a feature (non-master) branch for my PR.

(I'm not sure if I need to update "Breaking Changes" in CHANGELOG, because it may break installations where people have customized their KC to add specific stuff to their access_token, even is the specs says you should not)

@Elektordi Elektordi requested a review from a team as a code owner November 29, 2022 14:16
Elektordi added a commit to Elektordi/oauth2-proxy that referenced this pull request Nov 29, 2022
@JoelSpeed
Copy link
Member

Have you looked at the history of this provider? Is it possible the behaviour has changed in keycloak and your change makes it supported for newer versions but breaks older versions?
Would be good to get reviews and acks from some other keycloak folk before we merge.

If you can get a keycloak maintainer to comment that would help as well

@Elektordi
Copy link
Contributor Author
Elektordi commented Dec 29, 2022

One of the devs from redhat already confirmed this:
https://lists.jboss.org/pipermail/keycloak-user/2019-August/018924.html
I'll check for changes about this in history of the file..

Furthermore, access_token is from Oauth2 spec (and used in keycloak provider), and id_token is from oidc spec.

@github-actions
Copy link
Contributor

This pull request has been inactive for 60 days. If the pull request is still relevant please comment to re-activate the pull request. If no action is taken within 7 days, the pull request will be marked closed.

@github-actions github-actions bot added the Stale label Feb 28, 2023
@babs
Copy link
Contributor
babs commented Feb 28, 2023

IMHO, this is still relevant.

@JoelSpeed JoelSpeed removed the Stale label Feb 28, 2023
@JoelSpeed
Copy link
Member

This sounds like something we should fix, but I'm wondering if there's a way to avoid a breaking change here, any thoughts?

@babs
Copy link
Contributor
babs commented Apr 22, 2023

I'm not quite sure what it could break as the later adds aud claims defacto among other things if I remember correctly.
If users want to use the access token, the proper provider to use is the non oidc flavor of keycloak ;)

@yann-soubeyrand
Copy link
yann-soubeyrand commented May 2, 2023

Hello, I’ve asked Keycloak developers for some clarification:
keycloak/keycloak#20033

@babs
Copy link
Contributor
babs commented May 29, 2023

As stated by @Elektordi ,

access_token is from Oauth2 spec (and used in keycloak provider), and id_token is from oidc spec.

If you want to use access_token you can use the keycloak provider, and leave keycloak-oidc for .... oidc and therefore use the proper token :)

@github-actions
Copy link
Contributor

This pull request has been inactive for 60 days. If the pull request is still relevant please comment to re-activate the pull request. If no action is taken within 7 days, the pull request will be marked closed.

@github-actions github-actions bot added the Stale label Jul 31, 2023
@babs
Copy link
Contributor
babs commented Jul 31, 2023

Still relevant

@github-actions github-actions bot removed the Stale label Aug 1, 2023
@github-actions
Copy link
Contributor
github-actions bot commented Oct 1, 2023

This pull request has been inactive for 60 days. If the pull request is still relevant please comment to re-activate the pull request. If no action is taken within 7 days, the pull request will be marked closed.

@github-actions github-actions bot added the Stale label Oct 1, 2023
@yann-soubeyrand
Copy link

As stated by @Elektordi ,

access_token is from Oauth2 spec (and used in keycloak provider), and id_token is from oidc spec.

Yes, I know what Oauth2 and OIDC are (technically OIDC is based on Oauth2, so the access token is also and OIDC thing, but I won’t nitpick on this).

If you want to use access_token you can use the keycloak provider, and leave keycloak-oidc for .... oidc and therefore use the proper token :)

Well, if you want pure OIDC, I guess you can just use the OpenID Connect provider. I’m sure you’re aware that the Keycloak provider is already making use of the access token to get the roles and put them in the group list 😉

func (p *KeycloakOIDCProvider) getAccessClaims(ctx context.Context, s *sessions.SessionState) (*accessClaims, error) {
// HACK: This isn't an ID Token, but has similar structure & signing
token, err := p.Verifier.Verify(ctx, s.AccessToken)
if err != nil {
return nil, err
}
var claims *accessClaims
if err = token.Claims(&claims); err != nil {
return nil, err
}
return claims, nil
}

func getClientRoles(claims *accessClaims) []string {
var clientRoles []string
for clientName, access := range claims.ResourceAccess {
accessMap, ok := access.(map[string]interface{})
if !ok {
continue
}
var roles interface{}
if roles, ok = accessMap["roles"]; !ok {
continue
}
for _, role := range roles.([]interface{}) {
clientRoles = append(clientRoles, fmt.Sprintf("%s:%s", clientName, role))
}
}
return clientRoles
}

@github-actions github-actions bot removed the Stale label Oct 4, 2023
@babs
Copy link
Contributor
babs commented Oct 4, 2023

If you want to use access_token you can use the keycloak provider, and leave keycloak-oidc for .... oidc and therefore use the proper token :)

Well, if you want pure OIDC, I guess you can just use the OpenID Connect provider. I’m sure you’re aware that the Keycloak provider is already making use of the access token to get the roles and put them in the group list 😉

@yann-soubeyrand yes, I know, but in this case the aud is missing or wrong, so an extra config has to be made on KC side to make it work comply with OIDC (that, I can't emphase enought, shouldn't have to be done if proper token was used), all that because oauth2-proxy doesn't use the proper token due to legacy KC issue and a global misunderstanding.

I'd also argue: why having 2 code bases to talk to keycloak using the exact same token while the very purpose of keycloak-oidc should be to be defacto compliant with the specs.
It's not like we are talking about removing functionality here, IMHO, it's more fixing a legacy misuse and/or misunderstanding and making things right.

@yann-soubeyrand
Copy link

The problem is that, by default in Keycloak, no ID token claim contains the roles. So it seems there’s no way to avoid user configuration and to exclusively use the ID token. That’s why I asked on Keycloak discussions an advice on the way to go, but I got no answer so far.

@Elektordi
Copy link
Contributor Author
Elektordi commented Oct 5, 2023

no ID token claim contains the roles

I agree, but roles is a more specific feature, and not a needed one (you can use OIDC without roles).

why having 2 code bases to talk to keycloak using the exact same token while the very purpose of keycloak-oidc should be to be defacto compliant with the specs.

That is what I was thinking, the keycloak provider should use access token, and keycloak-oidc the id token, and things would work out of the box (without roles, indeed).

And, btw, multiple people have opened issues because of this bug.
For example, last one I found: #2117

@yann-soubeyrand
Copy link

OK, but then what’s the added value of the Keycloak OIDC provider compared to the generic OIDC provider?

@babs
Copy link
Contributor
babs commented Oct 5, 2023

good question, initially probably not that much, now there is another pull request for backend logout for example

Copy link
Contributor
github-actions bot commented Dec 5, 2023

This pull request has been inactive for 60 days. If the pull request is still relevant please comment to re-activate the pull request. If no action is taken within 7 days, the pull request will be marked closed.

@github-actions github-actions bot added the Stale label Dec 5, 2023
@babs
Copy link
Contributor
babs commented Dec 6, 2023

Not stalled!

@olivierlambert
Copy link
olivierlambert commented Jul 9, 2024

I had this temporary "fix" in Keycloak for a while (using Mappers), and now with Keycloak 25.1, I can't make it work anymore for any new client I add (or if I modify a previous client):

[oauthproxy.go:888] Error creating session during OAuth2 callback: audience from claim aud with value [xxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxx account] does not match with any of allowed audiences map[elastic:{}]

I checked 10 times, follow the doc in the exact same details and despite that, I can't make it work anymore. Note there's extra fields/boxes in Keycloak 25, that could made the configuration different:
image

Does anyone with a recent Keycloak made it work?

That would be great to get it solved via this PR, so we do not need any extra config anymore

@PaddyKe
Copy link
PaddyKe commented Jul 28, 2024

Does anyone with a recent Keycloak made it work?

That would be great to get it solved via this PR, so we do not need any extra config anymore

I just ran into the same issue. For some reason, Keycloak 25 seems to add some kind of GUID into the aud scope instead of the actual client id.
I was able to make it work by not specifying any value in the "Include Client Audience" and instead manually entering the client id in "Include Custom Audience" field.

image

@babs
Copy link
Contributor
babs commented Jul 29, 2024

@Elektordi @JoelSpeed @kvanzuijlen little up ?

@Fmstrat
Copy link
Fmstrat commented Aug 4, 2024

Could someone provide a how-to for the workaround?

@JoniJnm
Copy link
JoniJnm commented Sep 17, 2024

I "fixed" it using oidc_extra_audiences with a fixed value. Maybe aud or account, check the property aud in your jwt token

oidc_extra_audiences = [ "account" ]

@babs
Copy link
Contributor
babs commented Sep 18, 2024

@JoelSpeed ?

@franco-viotti
Copy link

@PaddyKe solution worked for me, thanks! I'm using Keycloak 25.0.1 and oauth2-proxy 7.6.0

jakubgs added a commit to status-im/infra-role-oauth-proxy that referenced this pull request Nov 28, 2024
It is a known issue with Keycloak:
oauth2-proxy/oauth2-proxy#1913
oauth2-proxy/oauth2-proxy#1916

Signed-off-by: Jakub Sokołowski <jakub@status.im>
@tuunit
Copy link
Member
tuunit commented Apr 27, 2025

Hey guys,

as this has been open for years now and I fully agree with @babs on the following:

I'd also argue: why having 2 code bases to talk to keycloak using the exact same token while the very purpose of keycloak-oidc should be to be defacto compliant with the specs.
It's not like we are talking about removing functionality here, IMHO, it's more fixing a legacy misuse and/or misunderstanding and making things right.

I investigated this whole mess in-depth!

First of all, I want to get some confusion out of the way.

OK, but then what’s the added value of the Keycloak OIDC provider compared to the generic OIDC provider?

TLDR; The Keycloak provider is deprecated and the KeycloakOIDC provider extends the generic OIDC provider and therefore offers all generic OIDC options and on top the extraction of roles from Keycloaks access token.

A couple years ago, Nick and Joel started working on generalising the code base. One of the goals was and still is, to try to have at little as possible specific provider code as possible, for this reason the generic OIDC provider was introduced and in the long run all OIDC compliant providers should only extend its functionality. The Keycloak provider was kept to stay backwards compatible and not break existing users setup. We now highlight in the docs and in the provider code that the normal Keycloak provider is therefore deprecated and people should use the KeycloakOIDC provider.

Main point which this PR was supposed to address:

Standard Compliance
Align with the OIDC specification, where the ID token is meant for authentication (containing user identity claims) and Access tokens are for authorization (OAuth2) and aren't guaranteed to contain the expected aud claim.

Breaking change
As already identified, the code was build under the assumption that the access token is used for extracting the roles:

type realmAccess struct {
Roles []string `json:"roles"`
}
type accessClaims struct {
RealmAccess realmAccess `json:"realm_access"`
ResourceAccess map[string]interface{} `json:"resource_access"`
}

func (p *KeycloakOIDCProvider) extractRoles(ctx context.Context, s *sessions.SessionState) error {
claims, err := p.getAccessClaims(ctx, s)
if err != nil {
return err
}
var roles []string
roles = append(roles, claims.RealmAccess.Roles...)
roles = append(roles, getClientRoles(claims)...)
// Add to groups list with `role:` prefix to distinguish from groups
for _, role := range roles {
s.Groups = append(s.Groups, formatRole(role))
}
return nil
}

Those aren't included in the ID Token returned by Keycloak:

ID Token:

{
  "exp": 1745780789,
  "iat": 1745780729,
  "auth_time": 1745780729,
  "jti": "76d39596-4af7-4bf5-8688-72232e064a42",
  "iss": "http://keycloak.localtest.me:9080/realms/oauth2-proxy",
  "aud": "oauth2-proxy",
  "sub": "3356c0a0-d4d5-4436-9c5a-2299c71c08ec",
  "typ": "ID",
  "azp": "oauth2-proxy",
  "sid": "8eb0bed4-de02-46e7-824f-6bb8d476a590",
  "at_hash": "pr_FFqbZrJ4q7SYld2B5Cg",
  "email_verified": true,
  "preferred_username": "admin@example.com",
  "email": "admin@example.com"
}

Access Token:

{                                                                                                                                                                                                                                               
  "exp": 1745780789,                                                                                                                                                                                                                            
  "iat": 1745780729,                                                                                                                                                                                                                            
  "auth_time": 1745780729,                                                                                                                                                                                                                      
  "jti": "af7fb1dd-143e-4d83-80f5-bb4201b2c322",                                                                                                                                                                                                
  "iss": "http://keycloak.localtest.me:9080/realms/oauth2-proxy",                                                                                                                                                                               
  "aud": [                                                                                                                                                                                                                                      
    "oauth2-proxy-realm",                                                                                                                                                                                                                       
    "account"                                                                                                                                                                                                                                   
  ],                                                                                                                                                                                                                                            
  "sub": "3356c0a0-d4d5-4436-9c5a-2299c71c08ec",                                                                                                                                                                                                
  "typ": "Bearer",                                                                                                                                                                                                                              
  "azp": "oauth2-proxy",                                                                                                                                                                                                                        
  "sid": "8eb0bed4-de02-46e7-824f-6bb8d476a590",                                                                                                                                                                                                
  "realm_access": {                                                                                                                                                                                                                             
    "roles": [                                                                                                                                                                                                                                  
      "create-realm",                                                                                                                                                                                                                           
      "offline_access",                                                                                                                                                                                                                         
      "admin",                                                                                                                                                                                                                                  
      "uma_authorization"                                                                                                                                                                       
    ]                                                                                                                                                                                           
  },                                                                                                                                                                                            
  "resource_access": {                                                                                                                                                                          
    "oauth2-proxy-realm": {                                                                                                                                                                     
      "roles": [                                                                                                                                                                                
        "view-realm",                                                                                                                                                                           
        "view-identity-providers",                                                                                                                                                              
        "manage-identity-providers",                                                                                                                                                            
        "impersonation",                                                                                                                                                                        
        "create-client",                                                                                                                                                                        
        "manage-users",                                                                                                                                                                         
        "query-realms",                                                                                                                                                                         
        "view-authorization",                                                                                                                                                                   
        "query-clients",                                                                                                                                                                        
        "query-users",                                                                                                                                                                          
        "manage-events",                                                                                                                                                                        
        "manage-realm",                                                                                                                                                                         
        "view-events",                                                                                                                                                                          
        "view-users",                                                                                                                                                                           
        "view-clients",                                                                                                                                                                         
        "manage-authorization",                                                                                                                                                                 
        "manage-clients",                                                                                                                                                                       
        "query-groups"                                                                                                                                                                          
      ]                                                                                                                                                                                         
    },                                          
    "account": {                                
      "roles": [                                
        "manage-account",                       
        "manage-account-links",                 
        "view-profile"                          
      ]                                         
    }                                           
  },                                            
  "scope": "openid profile email",              
  "email_verified": true,                       
  "preferred_username": "admin@example.com",    
  "email": "admin@example.com"                  
}

Drill down

As keycloak_oidc.go is inheriting the generic functionality from oidc.go the actual session validation and creation is happening here:

func (p *OIDCProvider) createSession(ctx context.Context, token *oauth2.Token, refresh bool) (*sessions.SessionState, error) {
_, err := p.verifyIDToken(ctx, token)
if err != nil {
switch err {
case ErrMissingIDToken:
// IDToken is mandatory in Redeem but optional in Refresh
if !refresh {
return nil, errors.New("token response did not contain an id_token")
}
default:
return nil, fmt.Errorf("could not verify id_token: %v", err)
}
}
rawIDToken := getIDToken(token)
ss, err := p.buildSessionFromClaims(rawIDToken, token.AccessToken)
if err != nil {
return nil, err
}
ss.AccessToken = token.AccessToken
ss.RefreshToken = token.RefreshToken
ss.IDToken = rawIDToken
ss.CreatedAtNow()
ss.SetExpiresOn(token.Expiry)
return ss, nil
}

func (p *OIDCProvider) ValidateSession(ctx context.Context, s *sessions.SessionState) bool {
ctx = oidc.ClientContext(ctx, requests.DefaultHTTPClient)
_, err := p.Verifier.Verify(ctx, s.IDToken)
if err != nil {
logger.Errorf("id_token verification failed: %v", err)
return false
}

And those locations are already properly using the access_token and id_token respectively.
Therefore this PR doesn't actually address the OIDC compliance as it is already in place and properly working.

So what is the problem then?
What this PR actually changed isn't the session creation or the session validation logic but instead it is just fighting a symptom of a strange implementation. In which we abuse the p.Verifier.Verify for parsing the access_token into an token interface. To be able to call the claim extraction method on it.

What really needs to be fixed is how we extract the claims from the access token of Keycloak. Which at the point of usage in getAccessClaims(...) has already been through the exchange and validation.

Therefore I propose we remove the unnecessary verification in:

func (p *KeycloakOIDCProvider) getAccessClaims(ctx context.Context, s *sessions.SessionState) (*accessClaims, error) {
// HACK: This isn't an ID Token, but has similar structure & signing
token, err := p.Verifier.Verify(ctx, s.AccessToken)
if err != nil {
return nil, err
}

and instead fix the actual problem like so:

func (p *KeycloakOIDCProvider) getAccessClaims(ctx context.Context, s *sessions.SessionState) (*accessClaims, error) {
        // extract payload from jwt
        // pseudo code
        // payload := decode(jwtParts[1])
        
	var claims *accessClaims
	if err := json.Unmarshal(payload, claims); err != nil {
		return nil, err
	}
	return claims, nil
}

@Elektordi What you correctly identified, is the fact that the tests did indeed only use the Access Token instead of the ID Token, which was an an oversight and should stay as proposed by your PR. In hindsight, this might be why the assumption was made that the KeycloakOIDC provider is using the access_token instead of the id_token for verification. Which isn't the case for the actual session creation and validation.

CC: @JoelSpeed @Elektordi @babs @yann-soubeyrand

@tuunit tuunit force-pushed the fix-keycloak-oidc branch 2 times, most recently from c729c46 to 05ab7d2 Compare April 27, 2025 21:47
@tuunit
Copy link
Member
tuunit commented Apr 27, 2025
  • I updated the branch, resolved all conflicts
  • Applied my proposed fix
  • Fixed the tests accordingly to reflect the usage of id token and access token
  • Changed the title from "Fix wrong token used in Keycloak OIDC provider" to "fix: role extraction from access token in keycloak oidc"

@tuunit tuunit force-pushed the fix-keycloak-oidc branch 2 times, most recently from 5d6835a to 20eeae5 Compare April 27, 2025 22:17
@tuunit tuunit changed the title Fix wrong token used in Keycloak OIDC provider fix: keycloak oidc role extraction from access token Apr 27, 2025
@tuunit tuunit changed the title fix: keycloak oidc role extraction from access token fix: role extraction from access token in keycloak oidc Apr 27, 2025
tuunit
tuunit previously approved these changes Apr 27, 2025
@JoelSpeed JoelSpeed merged commit 7b41c8e into oauth2-proxy:master Apr 28, 2025
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Development

Successfully merging this pull request may close these issues.

0