Internal Server Error on Identity Credential Request for Identities registered with Native OIDC Flow

[aptgetgit]

Preflight checklist

• I could not find a solution in the existing issues, docs, nor discussions.
• I agree to follow this project's Code of Conduct.
• I have read and am following this repository's Contribution Guidelines.
• I have joined the Ory Community Slack.
• I am signed up to the Ory Security Patch Newsletter.

Ory Network Project

No response

Describe the bug

My setup :
Cloudflare -> Nginx -> Oathkeeper(decision api) -> Kratos.
Kratos is configured for password less code login and Google OIDC login, Self Service and registration is enabled.
Bug :
If registering a Google account via native flow using Flutter/Dart ORY client and then retrieving OIDC credential details via the admin API endpoint admin/identities/<id>?include_credential=oidc throws an error json{"error":{"code":500,"status":"Internal Server Error","reason":"Unable to decode hex encrypted string","message":"An internal server error occurred, please contact the system administrator"}} . Retrieving identity without including credential query admin/identities/<id> works perfectly.
It only happens when the account is registered using Native OIDC Flow, as using browser flow via Self-Service UI works perfectly both with query and no query.

Also registration seems to be working in the native flow cause session token is received after the registration flow.

Reproducing the bug

Version
Flutter 3.19.6
Dart SDK version: 3.3.4

Flutter Code

final res = await dio
          .get('https://api.example.com/auth/self-service/registration/api');
      final data = Map<String, dynamic>.from(res.data);
      final flowId = data['id'];
      if (credentialManager.isSupportedPlatform) {
        await credentialManager.init(
          preferImmediatelyAvailableCredentials: true,
          googleClientId: clientId,
        );
        final gCredential = await credentialManager.saveGoogleCredential(
          nonce: Nonce(nonce: "jedn23iudbuyfb"),
        );
        final idToken = gCredential!.idToken;

        var body = UpdateRegistrationFlowWithOidcMethod((b) => b
          ..idToken = idToken
          ..idTokenNonce = 'jedn23iudbuyfb'
          ..method = 'oidc'
          ..provider = 'google');

        final response = await ory.getFrontendApi().updateRegistrationFlow(
          flow: flowId,
          updateRegistrationFlowBody: UpdateRegistrationFlowBody(
              (b) => b..oneOf = OneOf.fromValue1(value: body)),
        );
        final oryData = response.data;
      }
      ```

### Relevant log output

```shell
time=2024-04-21T23:26:49Z level=info msg=started handling request http_request=map[headers:map[accept:*/* accept-encoding:gzip, br cdn-loop:cloudflare cf-connecting-ip:2401:0000:0000:0000:0000:0000:9a21:d8ae cf-ipcountry:NN cf-ray:000020003ef500a6-CDG cf-visitor:{"scheme":"https"} connection:close content-type:application/json user-agent:curl/7.81.0 x-forwarded-for:172.71.123.27 x-forwarded-proto:https x-forwarded-scheme:https x-real-ip:172.71.123.27] host:kratos.admin-domain.com method:GET path:/admin/identities/1c9fef99-8414-4395-a917-041e1d6e9e57 query:REDACTED remote:172.21.0.2:49112 scheme:http]


time=2024-04-21T23:26:49Z level=error msg=An error occurred while handling a request audience=application error=map[debug: message:An internal server error occurred, please contact the system administrator reason:Unable to decode hex encrypted string status:Internal Server Error status_code:500] http_request=map[headers:map[accept:*/* accept-encoding:gzip, br cdn-loop:cloudflare cf-connecting-ip:2401:0000:1c29:0000:0000:0000:9a21:d8ae cf-ipcountry:NN cf-ray:80000000a6-CDG cf-visitor:{"scheme":"https"} connection:close content-type:application/json user-agent:curl/7.81.0 x-forwarded-for:172.71.123.27 x-forwarded-proto:https x-forwarded-scheme:https x-real-ip:172.71.123.27] host:kratos.admin-domain.com method:GET path:/admin/identities/1c9fef99-8414-4395-a917-041e1d6e9e57 query:REDACTED remote:172.21.0.2:49112 scheme:http] http_response=map[status_code:500] service_name=Ory Kratos service_version=v1.1.0


time=2024-04-21T23:26:49Z level=info msg=completed handling request http_request=map[headers:map[accept:*/* accept-encoding:gzip, br cdn-loop:cloudflare cf-connecting-ip:2401:0000:0000:0000:0000:0000:9a21:d8ae cf-ipcountry:NN cf-ray:800000000-CDG cf-visitor:{"scheme":"https"} connection:close content-type:application/json user-agent:curl/7.81.0 x-forwarded-for:172.71.123.27 x-forwarded-proto:https x-forwarded-scheme:https x-real-ip:172.71.123.27] host:kratos.admin-domain.com method:GET path:/admin/identities/1c9fef99-8414-4395-a917-041e1d6e9e57 query:REDACTED remote:172.21.0.2:49112 scheme:http] http_response=map[headers:map[cache-control:private, no-cache, no-store, must-revalidate content-type:application/json] size:192 status:500 text_status:Internal Server Error took:4.327744ms]

Relevant configuration

**KRATOS CONFIG**

version: v1.1.0

dsn: memory # Replaced by PostgreSQL 14 via ENV

identity:
  default_schema_id: user-v1
  schemas:
    - id: user-v1
      url: file:///etc/config/kratos/user.schema.json


selfservice:
  default_browser_return_url: https://accounts.example.com
  allowed_return_urls:
    - https://accounts.example.com

  flows:
    logout:
      after:
        default_browser_return_url: https://accounts.example.com/login

    registration:
      enabled: true
      login_hints: false
      ui_url: https://accounts.example.com/registration
      lifespan: 1h
      after:
        code:
          hooks:
            - hook: session
          default_browser_return_url: https://accounts.example.com
          
        oidc:
          hooks:
            - hook: session
          default_browser_return_url: https://accounts.example.com
        default_browser_return_url: https://accounts.example.com

    login:
      ui_url: https://accounts.example.com/login
      lifespan: 1h
      after:
        oidc:
          hooks:
            - hook: require_verified_address
          default_browser_return_url: https://accounts.example.com
        code:
          hooks:
            - hook: require_verified_address
          default_browser_return_url: https://accounts.example.com
        default_browser_return_url: https://accounts.example.com

    verification:
      enabled: true
      ui_url: https://accounts.example.com/verification
      lifespan: 1h
      use: code
      notify_unknown_recipients: false
      after:
        default_browser_return_url: https://accounts.example.com
    
    recovery:
      enabled: true
      ui_url: https://accounts.example.com/recovery
      lifespan: 1h
      use: code
      notify_unknown_recipients: false
      after:
        default_browser_return_url: https://accounts.example.com

    error:
      ui_url: https://accounts.example.com/errors
    
    settings:
      ui_url: https://accounts.example.com/settings
      lifespan: 1h
      privileged_session_max_age: 15m
      required_aal: highest_available
      after:
        oidc:
          hooks:
            - hook: revoke_active_sessions
          default_browser_return_url: https://accounts.example.com/login

  methods:
    password:
      enabled: false

    code:
      passwordless_enabled: true
      mfa_enabled: false
      passwordless_login_fallback_enabled: true
      enabled: true
      config:
        lifespan: 10m

    oidc:
      enabled: true
      config:
        providers:
          - id: google
            provider: google
            client_id: "ENV"
            client_secret: "ENV"
            issuer_url: https://accounts.google.com
            auth_url: https://accounts.google.com/o/oauth2/v2/auth
            token_url: https://oauth2.googleapis.com/token
            mapper_url:  file:///etc/config/kratos/oidc.google.jsonnet
            scope:
              - email
            subject_source: userinfo
            claims_source: id_token
            requested_claims:
              id_token:
                email:
                  essential: true
                email_verified:
                  essential: true
        base_redirect_uri: https://api.example.com/auth
    
    profile:
      enabled: false

courier:
  message_retries: 2
  delivery_strategy: smtp
  smtp:
    from_address: no-reply@example.com
    from_name: Example
    connection_uri: smtp://thetoken:thetoken@smtp.postmarkapp.com:587/

preview:
  default_read_consistency_level: strong

serve:
  public:
    host: 0.0.0.0
    port: 4433
    base_url: https://api.example.com/auth
    request_log:
      disable_for_health: true
    cors:
      enabled: true
      allow_credentials: true
      options_passthrough: false
      max_age: 0
      debug: false
      allowed_methods:
        - POST
        - GET
        - PUT
        - PATCH
        - DELETE
      allowed_origins:
        - https://example.com
        - https://*.example.com
      allowed_headers:
        - Authorization
        - Cookie
        - Content-Type
      exposed_headers:
        - Content-Type
        - Set-Cookie
  admin:
    host: 0.0.0.0
    port: 4434
    base_url: https://kratos.admin-domain.com
    request_log:
      disable_for_health: true

log:
  level: info
  format: text
  leak_sensitive_values: false
  redaction_text: "REDACTED"

secrets:
  cookie:
    - ipsumipsumipsumi
  cipher:
    - ipsumipsumipsumipsumipsumipsumip
  default:
    - ipsumipsumipsumi

ciphers:
  algorithm: xchacha20-poly1305

cookies:
  same_site: Lax
  domain: example.com
  path: /

session:
  lifespan: 720h
  cookie:
    name: example_com_session
    persistent: true
    same_site: Lax
    domain: example.com
    path: /
  earliest_possible_extend: 120h
  whoami:
    required_aal: highest_available

**OIDC JSONNET**

local claims = {
  email_verified: false,
} + std.extVar('claims');

{
  identity: {
    traits: {
      [if 'email' in claims && claims.email_verified then 'email' else null]: claims.email,
    },
    verified_addresses: std.prune([
      if 'email' in claims && claims.email_verified then { via: 'email', value: claims.email },
    ]),
  },
}


**OATHKEEPER CONFIG**
FYI: Admin API is not authenticated in nginx so no rules in oathkeeper

version: v0.40.7

serve:
  proxy:
    port: 4455
    host: 0.0.0.0
    cors:
      enabled: true
      allow_credentials: true
      max_age: 0
      allowed_methods:
        - POST
        - GET
        - PUT
        - PATCH
        - DELETE
      allowed_origins:
        - https://example.com
        - https://*.example.com
      allowed_headers:
        - Authorization
        - Content-Type
      exposed_headers:
        - Content-Type
      debug: true
  api:
    port: 4456
    host: 0.0.0.0

errors:
  fallback:
    - json

  handlers:
    redirect:
      enabled: true
      config:
        to: https://accounts.example.com/login
        code: 302
        return_to_query_param: "return_to"
        when:
          - error:
              - unauthorized
              - forbidden
            request:
              header:
                accept:
                  - text/html
    json:
      enabled: true
      config:
        verbose: true

access_rules:
  matching_strategy: glob
  repositories:
    - file:///etc/config/oathkeeper/access-rules.yml

authenticators:
  anonymous:
    enabled: true
    config:
      subject: guest
  cookie_session:
    enabled: true
    config:
      check_session_url: http://kratos:4433/sessions/whoami
      preserve_query: true
      preserve_host: true
      preserve_path: true
      subject_from: "identity.id"
      only:
        - example_com_session
  noop:
    enabled: true

authorizers:
  allow:
    enabled: true

mutators:
  noop:
    enabled: true
  header:
    enabled: true
    config:
      headers:
        X-User: "{{ print .Subject }}"

log:
  level: info
  format: text
  leak_sensitive_values: false
  redaction_text: "REDACTED"

Access Rules -
- id: "ory:kratos:public"
  match:
    url: "https://api.example.com/auth/<**>"
    methods:
      - GET
      - POST
      - PUT
      - DELETE
      - PATCH
  authenticators:
    - handler: noop
  authorizer:
    handler: allow
  mutators:
    - handler: noop

- id: "ory:kratos-selfservice-ui-node:anonymous"
  match:
    url: "https://accounts.example.com/<{,self-service,registration,welcome,recovery,verification,login,error,health/{alive,ready},**.css,**.js,**.png,**.svg,**.woff*}>"
    methods:
      - GET
  authenticators:
    - handler: anonymous
  authorizer:
    handler: allow
  mutators:
    - handler: noop

- id: "ory:kratos-selfservice-ui-node:protected"
  match:
    url: "https://accounts.example.com/<{sessions,settings}>"
    methods:
      - GET
  authenticators:
    - handler: cookie_session
  authorizer:
    handler: allow
  mutators:
    - handler: noop
  errors:
    - handler: redirect
      config:
        to: https://accounts.example.com/login

**NGINX CONFIG**

ADMIN API (kratos.admin-domain.com.conf)-
location /admin/ {
    proxy_set_header Host $host;
    proxy_set_header X-Forwarded-Scheme $scheme;
    proxy_set_header X-Forwarded-Proto $scheme;
    proxy_set_header X-Forwarded-For $remote_addr;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_pass http://kratos-admin;
  }

  location /identities/ {
    proxy_set_header Host $host;
    proxy_set_header X-Forwarded-Scheme $scheme;
    proxy_set_header X-Forwarded-Proto $scheme;
    proxy_set_header X-Forwarded-For $remote_addr;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_pass http://kratos-admin;
  }

Kratos Api (api.example.com.conf) -
location /auth/ {
    auth_request /auth-decision;
    auth_request_set $next_uri $upstream_http_X_Original_URI;
    
    proxy_set_header Host $host;
    proxy_set_header X-Forwarded-Scheme $scheme;
    proxy_set_header X-Forwarded-Proto $scheme;
    proxy_set_header X-Forwarded-For $remote_addr;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_pass http://kratos-public/;
  }

Version

1.1.0

On which operating system are you observing this issue?

Other

In which environment are you deploying?

Docker Compose

Additional Context

All of the browser flow has been tested with all the configurations mentioned above, i am guessing either my native code is wrong or there might be a bug with kratos.