[Bug] OIDC not working with Authentik > 2024.2?

[badsmoke]

Is this a support request?

• This is not a support request

Is there an existing issue for this?

• I have searched the existing issues

Current Behavior

after updating to authentik 2024.4.1, the connection to headscale no longer works properly.

after a headscale restart, for example, the exit node no longer works.

I assume that this is the problem?

Starting with authentik 2024.2, applications only receive an access token. To receive a refresh token, both applications and authentik must be configured to request the offline_access scope. In authentik this can be done by selecting the offline_access Scope mapping in the provider settings.

I previously had an authentik smaller 2024, which ran without any problems

headscale_1        | 2024/04/29 13:43:32 http: response.WriteHeader on hijacked connection from github.com/juanfont/headscale/hscontrol.(*Headscale).NoiseUpgradeHandler (noise.go:84)
headscale_1        | 2024/04/29 13:43:32 http: response.Write on hijacked connection from fmt.Fprintln (print.go:305)
headscale_1        | 2024-04-29T13:43:36Z ERR noise upgrade failed error="noise handshake failed: decrypting machine key: chacha20poly1305: message authentication failed"
headscale_1        | 2024/04/29 13:43:36 http: response.WriteHeader on hijacked connection from github.com/juanfont/headscale/hscontrol.(*Headscale).NoiseUpgradeHandler (noise.go:84)
headscale_1        | 2024/04/29 13:43:36 http: response.Write on hijacked connection from fmt.Fprintln (print.go:305)

Expected Behavior

just like before :-D it should work

unfortunately I can no longer downgrade without further ado to test it further

Steps To Reproduce

1. headscale set up so that oidc runs via authentik (< 2024)

2. authentik update, which offline_access needs

3. connect to headscale, works but exit node does not work (no internet)

4. exit-node reauth, internet works again-> until the next headscale restart or a time?

Environment

- OS:ubuntu 22.04
- Headscale version: 0.23.0-alpha9
- Tailscale version: 1.64.0

Runtime environment

• Headscale is behind a (reverse) proxy
• Headscale runs in a container

Anything else?

that's just a guess, but i updated authentik at the weekend and it hasn't worked since then

[yeyeoke]

Same problem here!

[badsmoke]

are you sure? so it has something to do with the offlice_access scope.

here is an explanation

go-oidc also supports the scope

https://pkg.go.dev/github.com/coreos/go-oidc/v3/oidc

search for "offlice_access"

i have added the scope in the headscale config and authentik but it does not bring any noticeable change

[Ziomal12]

@badsmoke

are you sure? so it has something to do with the offlice_access scope.

here is an explanation

go-oidc also supports the scope

https://pkg.go.dev/github.com/coreos/go-oidc/v3/oidc

search for "offlice_access"

i have added the scope in the headscale config and authentik but it does not bring any noticeable change

Could you share your config and explain in more detail what you've done? I think I've also added that and no authentik login screen appears anyways.

[sonroyaalmerol]

Had this exact same problem. For authentik, you'd need an additional extra_params and include the offline_access scope. Make sure to actually include the scope in authentik OAuth2 provider side as well.

You would also need to set a Signing Key (you can use the self-signed one) for the OAuth2 provider of authentik to use RS256 for signing which I think is required for Headscale.

Here's a sample portion of the headscale config for the oidc:

oidc:
  ...
  scope:
    - openid
    - profile
    - email
    - offline_access
  extra_params:
    prompt: consent

[badsmoke]

everything went smoothly over the weekend, did it work for you too? @yeyeoke and @Ziomal12

then I would close this issue

[julianq]

@sonroyaalmerol @badsmoke I'm now having this exact issue.

My headscale config includes:

oidc:
  ...
  scope: ["openid", "profile", "email", "offline_access"]
  extra_params:
    prompt: consent

The OIDC provider on the Authentik side is using the self-signed certificate, not expired, and also has offline_access included in scope.

Keep getting: connect: connection refused

Any ideas?

[Ziomal12]

In docker logs headscale I see this error:

2024-05-14T16:24:40+02:00 WRN failed to set up OIDC provider, falling back to CLI based authentication error="creating OIDC provider from issuer config: Get \"https://example.com/application/o/headscale/.well-known/openid-configuration\": net/http: TLS handshake timeout"

The address is accessible through the URL with valid LE cert.

Here is my config.yaml:

oidc:
   only_start_if_oidc_is_available: false
   issuer: https://example.com/application/o/headscale/
   client_id: "headscale"
   client_secret: "*********************************"
   expiry: 0
   use_expiry_from_token: false
   scope: ["openid", "profile", "email", "offline_access"]
   extra_params:
     prompt: consent

In authentik provider config I have set singing key and the same scopes as in the config file.

[julianq]

@Ziomal12 I think you and I are having the exact same problem. I cannot figure out why it's not connecting.

[badsmoke]

i have not extra params

oidc:
#   only_start_if_oidc_is_available: true
   only_start_if_oidc_is_available: true
   issuer: "https://auth.example.io/application/o/headscale-test/"
   client_id: "123123123123213"
   client_secret: "12123123123"


   expiry: 0

   scope: ["openid", "profile", "email", "offline_access"]

[julianq]

@badsmoke extra_params seems to be mostly irrelevant, as i think the default is prompt: consent anyway. regardless, with or without extra_params I get the same error. What do you have for config on the authentik side?

[Ziomal12]

i have not extra params

oidc:
#   only_start_if_oidc_is_available: true
   only_start_if_oidc_is_available: true
   issuer: "https://auth.example.io/application/o/headscale-test/"
   client_id: "123123123123213"
   client_secret: "12123123123"


   expiry: 0

   scope: ["openid", "profile", "email", "offline_access"]

Removing extra_parms does not help. Here is my Authentik config for headscale provider:

[julianq]

@Ziomal12 exactly what I have

[Ziomal12]

@julianq What versions are you using?
I'm running Authentik 2024.4.2 and headscale v0.23.0-alpha11

[julianq]

@Ziomal12 2024.4.2 on authentik, on headscale its whatever the latest release (ie non-alpha) is. I forget the build number I'll go look it up.

ETA: 0.22.3

[badsmoke]

I have the same config except for the redirect url, which is still very open with .*

[julianq]

@Ziomal12 Okay so I seem to have fixed it by putting Headscale and Authentik on the same VLAN (they were on separate VLANs before). I have no idea how this got borked since there were no config changes before it stopped working. I also explicitly allowed all traffic between the VLANs and it still didn't work, only when they were put on the same VLAN. I'll have to do some more digging as to why this happened, but for now, I'm just happy its working.

[Ziomal12]

@julianq Glad to hear that! Unfortunately I'm unable to do that as both services run on different machines. Hopefully there's a fix soon, not a workaround.

[julianq]

@Ziomal12 Agreed.

[julianq]

For what it's worth, I removed offline_access from the scope and it's still working, so that's not a contributing factor here.

[julianq]

@Ziomal12 This is an Authentik issue. Second service having authentication issues, which were fixed by moving the service on to the same VLAN as Authentik. I'll file a bug report there.