Failed to connect Infinispan cache in keycloak 24.0.4

[selimppc]

I am facing a problem connecting the Infinispan cache with my keycloak ( 24.0.4 ). The latest version stated we don't need to add or modify XML anymore. It has been simplified.

I have installed: the operator using:

$ kubectl apply -f https://raw.githubusercontent.com/keycloak/keycloak-k8s-resources/24.0.4/kubernetes/keycloaks.k8s.keycloak.org-v1.yml
$ kubectl apply -f https://raw.githubusercontent.com/keycloak/keycloak-k8s-resources/24.0.4/kubernetes/keycloakrealmimports.k8s.keycloak.org-v1.yml
$ kubectl apply -f https://raw.githubusercontent.com/keycloak/keycloak-k8s-resources/24.0.4/kubernetes/kubernetes.yml

and infinispan config yaml:

apiVersion: v1
kind: Secret
metadata:
  name: remote-store-secret
  namespace: default
type: Opaque
data:
  username: xxxxxxx= 
  password: xxxxxxx= 

---
apiVersion: k8s.keycloak.org/v2alpha1
kind: Keycloak
metadata:
  labels:
    app: keycloak
  name: keycloak
  namespace: default
spec:
  additionalOptions:
    - name: cache-remote-host 
      value: "18.142.142.220"
    - name: cache-remote-port 
      value: "11222"
    - name: cache-remote-username 
      secret:
        name: remote-store-secret
        key: username
    - name: cache-remote-password 
      secret:
        name: remote-store-secret
        key: password
    - name: spi-connections-infinispan-quarkus-site-name 
      value: keycloak

and keycloak deployment yaml:

apiVersion: v1
kind: Service
metadata:
  name: keycloak
  namespace: default
  labels:
    app: keycloak
spec:
  ports:
  - name: http
    port: 8080
    targetPort: 8080
  selector:
    app: keycloak
  type: ClusterIP
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: keycloak
  namespace: default
  labels:
    app: keycloak
spec:
  serviceName: "keycloak"
  replicas: 2
  selector:
    matchLabels:
      app: keycloak
  template:
    metadata:
      labels:
        app: keycloak
    spec:
      containers:
      - name: keycloak
        image: 267026814870.dkr.ecr.ap-southeast-1.amazonaws.com/keycloak:dev-9e80751d
        command: ["/opt/keycloak/bin/kc.sh"]
        args: [
            "start",
            "--proxy-headers=forwarded",
            "--optimized",
            "--db-pool-initial-size=10",
          ]
        envFrom:
         - secretRef:
              name: dev-saas-keycloak-secrets
        ports:
        - name: http
          containerPort: 8080
        readinessProbe:
          httpGet:
            path: /health/ready  # Updated path for readiness probe
            port: http
            scheme: HTTP
          initialDelaySeconds: 30
          periodSeconds: 10
          timeoutSeconds: 5
        livenessProbe:
          httpGet:
            path: /health  # Updated path for liveness probe
            port: http
            scheme: HTTP
          initialDelaySeconds: 60
          periodSeconds: 10
          timeoutSeconds: 5

my dockerfile:

ARG KEYCLOAK_VERSION=24.0.4
# Stage 1: Build the Keycloak image
FROM quay.io/keycloak/keycloak:${KEYCLOAK_VERSION} as keycloak-builder

# Enable health, metrics support, configure database vendor
ENV KC_HEALTH_ENABLED=true
ENV KC_METRICS_ENABLED=true
ENV KC_DB=postgres

WORKDIR /opt/keycloak


# Cache Infinispan
# COPY ./mvn/extensions/cache/cache-ispn.xml /opt/keycloak/conf/cache-ispn.xml

RUN /opt/keycloak/bin/kc.sh build --cache=ispn


# Stage 2: Final Keycloak image with the built JAR file
FROM quay.io/keycloak/keycloak:${KEYCLOAK_VERSION}
COPY --from=keycloak-builder /opt/keycloak/ /opt/keycloak/

ENV KC_CACHE="ispn"

RUN env
ENTRYPOINT ["/opt/keycloak/bin/kc.sh", "start", "--optimized"]

console logs:

2024-05-08 15:41:18,407 WARN  [org.keycloak.quarkus.runtime.cli.Picocli] (main) The following used options or option values are DEPRECATED and will be removed in a future release:
	- proxy: Use proxy-headers.
Consult the Release Notes for details.
2024-05-08 15:41:23,196 INFO  [org.keycloak.quarkus.runtime.hostname.DefaultHostnameProvider] (main) Hostname settings: Base URL: <unset>, Hostname: accounts.development.bongo-solutions.com, Strict HTTPS: true, Path: <request>, Strict BackChannel: false, Admin URL: https://accounts.development.bongo-solutions.com, Admin: accounts.development.bongo-solutions.com, Port: -1, Proxied: true
2024-05-08 15:41:25,096 INFO  [org.infinispan.CONTAINER] (keycloak-cache-init) ISPN000556: Starting user marshaller 'org.infinispan.jboss.marshalling.core.JBossUserMarshaller'
2024-05-08 15:41:26,132 INFO  [org.infinispan.CLUSTER] (keycloak-cache-init) ISPN000088: Unable to use any JGroups configuration mechanisms provided in properties {}. Using default JGroups configuration!
2024-05-08 15:41:26,829 INFO  [org.infinispan.CLUSTER] (keycloak-cache-init) ISPN000078: Starting JGroups channel `ISPN`
2024-05-08 15:41:26,897 INFO  [org.jgroups.JChannel] (keycloak-cache-init) local_addr: 8ec9b310-8a7f-4882-87b2-8a4c69cbf79f, name: keycloak-2-16601
2024-05-08 15:41:26,908 WARN  [org.jgroups.protocols.UDP] (keycloak-cache-init) JGRP000015: the send buffer of socket MulticastSocket was set to 1MB, but the OS only allocated 212.99KB
2024-05-08 15:41:26,908 WARN  [org.jgroups.protocols.UDP] (keycloak-cache-init) JGRP000015: the receive buffer of socket MulticastSocket was set to 20MB, but the OS only allocated 212.99KB
2024-05-08 15:41:26,909 WARN  [org.jgroups.protocols.UDP] (keycloak-cache-init) JGRP000015: the send buffer of socket MulticastSocket was set to 1MB, but the OS only allocated 212.99KB
2024-05-08 15:41:26,909 WARN  [org.jgroups.protocols.UDP] (keycloak-cache-init) JGRP000015: the receive buffer of socket MulticastSocket was set to 25MB, but the OS only allocated 212.99KB
2024-05-08 15:41:27,001 INFO  [org.jgroups.protocols.FD_SOCK2] (keycloak-cache-init) server listening on *.38205
2024-05-08 15:41:29,091 INFO  [org.jgroups.protocols.pbcast.GMS] (keycloak-cache-init) keycloak-2-16601: no members discovered after 2077 ms: creating cluster as coordinator
2024-05-08 15:41:29,117 INFO  [org.infinispan.CLUSTER] (keycloak-cache-init) ISPN000094: Received new cluster view for channel ISPN: [keycloak-2-16601|0] (1) [keycloak-2-16601]
2024-05-08 15:41:29,308 INFO  [org.infinispan.CLUSTER] (keycloak-cache-init) ISPN000079: Channel `ISPN` local address is `keycloak-2-16601`, physical addresses are `[192.194.8.108:53741]`
2024-05-08 15:41:29,413 WARN  [org.infinispan.CONFIG] (keycloak-cache-init) ISPN000569: Unable to persist Infinispan internal caches as no global state enabled
2024-05-08 15:41:47,822 INFO  [org.keycloak.broker.provider.AbstractIdentityProviderMapper] (main) Registering class org.keycloak.broker.provider.mappersync.ConfigSyncEventListener
2024-05-08 15:41:47,915 INFO  [org.keycloak.connections.infinispan.DefaultInfinispanConnectionProviderFactory] (main) Node name: keycloak-2-16601, Site name: null
2024-05-08 15:41:50,901 INFO  [io.quarkus] (main) Keycloak 24.0.4 on JVM (powered by Quarkus 3.8.4) started in 34.702s. Listening on: http://0.0.0.0:8080
2024-05-08 15:41:50,904 INFO  [io.quarkus] (main) Profile prod activated.
2024-05-08 15:41:50,904 INFO  [io.quarkus] (main) Installed features: [agroal, cdi, hibernate-orm, jdbc-postgresql, keycloak, logging-gelf, micrometer, narayana-jta, reactive-routes, resteasy-reactive, resteasy-reactive-jackson, smallrye-context-propagation, smallrye-health, vertx]

the issue is: that Keycloak is not connecting to the external cache with IP. keycloak connecting internal cache ispn.

Thanks in advance.

[codespearhead]

I saw that you created two similar discussions: #29384 and #29389.

Does this discussion mean that the other one is now resolved? If that's so, can you tell us how you solved it there and then close that other discussion?

If the same issue, then I suggest centralizing it into a single discussion to promote the unification of troubleshooting data.

[selimppc]

@codespearhead thank you - I closed one.

I did not get any solution.

[shawkins]

Can you clarify if you are using the operator, or just creating your own StatefulSet. The one you show above does not appear to be managed by the operator.

[selimppc]

@shawkins thank you.

I deployed keycloak using stateful set deployment - I got the admin panel and client login. Now I needed to connect with the external cache -- so I followed the doc and installed the operator and the YAML file for setting the external IP and port.

The issue is -- it's not connecting -- keycloak runs with its own or embedded cache. I can not figure out the issue as it has not logs or other configuration errors.

[shawkins]

The operator will need to create its own StatefulSet for the Keycloak CR - if there is an existing one in the namespace I believe if there is an exisitng one in the namespace without the appropriate owner reference, then the operator will fail to manage the old one. You can check the operator log for errors. Simply delete your initial StatefulSet and let the operator take over.