Cannot deploy sitewhere using HELM - Pods are failing with Error: SRCFG00014: Property sitewhere.config.k8s.pod.ip not found

[ssmtariq]

I am trying to deploy sitewhere using the helm chart. But 11 pods are failing with the following error:

__  ____  __  _____   ___  __ ____  ______ 
 --/ __ \/ / / / _ | / _ \/ //_/ / / / __/ 
 -/ /_/ / /_/ / __ |/ , _/ ,< / /_/ /\ \   
--\___\_\____/_/ |_/_/|_/_/|_|\____/___/   
2022-06-02 12:33:28,593 INFO  [com.sit.bat.mic.BatchOperationsMicroservice] (main) Starting microservice...
2022-06-02 12:33:28,610 INFO  [com.sit.bat.mic.BatchOperationsMicroservice] (main) Initializing Kubernetes connectivity...
2022-06-02 12:33:30,296 INFO  [com.sit.bat.mic.BatchOperationsMicroservice] (main) Kubernetes connectivity initialized.
2022-06-02 12:33:30,488 ERROR [com.sit.bat.mic.BatchOperationsMicroservice] (Service Main) Unhandled exception in microservice startup.: java.util.NoSuchElementException: SRCFG00014: Property sitewhere.config.k8s.pod.ip not found
        at io.smallrye.config.SmallRyeConfig.getValue(SmallRyeConfig.java:161)
        at io.smallrye.config.SmallRyeConfig.getValue(SmallRyeConfig.java:145)
        at io.quarkus.arc.runtime.config.IInstanceSettings_d8d9058176606bc3e6ba8bc34c7a6b329541e877_783ef3eb16d7af86a3f01a0b3278c906966ace8f.getKubernetesPodAddress(IInstanceSettings_d8d9058176606bc3e6ba8bc34c7a6b329541e877_783ef3eb16d7af86a3f01a0b3278c906966ace8f.zig:259)
        at com.sitewhere.microservice.Microservice.getHostname(Microservice.java:339)
        at com.sitewhere.batch.microservice.BatchOperationsMicroservice_ClientProxy.getHostname(BatchOperationsMicroservice_ClientProxy.zig:1564)
        at com.sitewhere.microservice.MicroserviceApplication.startMicroservice(MicroserviceApplication.java:126)
        at com.sitewhere.microservice.MicroserviceApplication$StartMicroservice.call(MicroserviceApplication.java:170)
        at com.sitewhere.microservice.MicroserviceApplication$StartMicroservice.call(MicroserviceApplication.java:162)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

2022-06-02 12:33:30,501 INFO  [com.sit.bat.mic.BatchOperationsMicroservice] (Service Main) 

!!!! Unhandled Exception !!!!

Error: SRCFG00014: Property sitewhere.config.k8s.pod.ip not found


2022-06-02 12:33:30,702 WARN  [io.fab.kub.cli.int.VersionUsageUtils] (informer-controller-SiteWhereScriptVersion) The client is using resource type 'scriptversions' with unstable version 'v1alpha4'
2022-06-02 12:33:30,704 WARN  [io.fab.kub.cli.int.VersionUsageUtils] (informer-controller-SiteWhereScript) The client is using resource type 'scripts' with unstable version 'v1alpha4'
2022-06-02 12:33:30,703 WARN  [io.fab.kub.cli.int.VersionUsageUtils] (informer-controller-SiteWhereTenantEngine) The client is using resource type 'tenantengines' with unstable version 'v1alpha4'
2022-06-02 12:33:30,703 WARN  [io.fab.kub.cli.int.VersionUsageUtils] (informer-controller-SiteWhereInstance) The client is using resource type 'instances' with unstable version 'v1alpha4'
2022-06-02 12:33:30,703 WARN  [io.fab.kub.cli.int.VersionUsageUtils] (informer-controller-SiteWhereMicroservice) The client is using resource type 'microservices' with unstable version 'v1alpha4'

Can anyone help me to solve this issue?

[derekadams]

Hi @ssmtariq it looks like you are launching the framework directly from Helm, but the new microservices framework has to be launched using https://github.com/sitewhere/swctl/blob/master/README.md .. The operator injects the correct environment variables in order to allow the microservices to run.

[ssmtariq]

Thank you very much. I got it. Now most of the services are running but still I got the pod sitewhere-keycloak-0 failed. When I described found the following. Could you please have a look

Name:         sitewhere-keycloak-0
Namespace:    sitewhere-system
Priority:     0
Node:         node-3/10.10.1.4
Start Time:   Thu, 02 Jun 2022 10:28:25 -0600
Labels:       app.kubernetes.io/instance=sitewhere
              app.kubernetes.io/name=keycloak
              controller-revision-hash=sitewhere-keycloak-7c8b8fbd54
              statefulset.kubernetes.io/pod-name=sitewhere-keycloak-0
Annotations:  checksum/config-startup: e055e8e9a2ac44f8b75847f7e552c702e6c03e6af2bdf1f59a284cb0ef2d73f7
              checksum/secrets: 44136fa355b3678a1146ad16f7e8649e94fb4fc21fe77e8310c060f61caaff8a
              cni.projectcalico.org/podIP: 192.168.45.6/32
              cni.projectcalico.org/podIPs: 192.168.45.6/32
Status:       Pending
IP:           192.168.45.6
IPs:
  IP:           192.168.45.6
Controlled By:  StatefulSet/sitewhere-keycloak
Init Containers:
  pgchecker:
    Container ID:  docker://f7ecc4e495ed308236323f5f0bb0ac32903a463a101ae237f6a7b19494cacb47
    Image:         docker.io/busybox:1.32
    Image ID:      docker-pullable://busybox@sha256:ae39a6f5c07297d7ab64dbd4f82c77c874cc6a94cea29fdec309d0992574b4f7
    Port:          <none>
    Host Port:     <none>
    Command:
      sh
      -c
      echo 'Waiting for PostgreSQL to become ready...'
      
      until printf "." && nc -z -w 2 sitewhere-postgresql 5432; do
          sleep 2;
      done;
      
      echo 'PostgreSQL OK ✓'
      
    State:          Running
      Started:      Thu, 02 Jun 2022 10:28:46 -0600
    Ready:          False
    Restart Count:  0
    Limits:
      cpu:     50m
      memory:  64Mi
    Requests:
      cpu:        50m
      memory:     64Mi
    Environment:  <none>
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from sitewhere-keycloak-token-bc7lw (ro)
Containers:
  keycloak:
    Container ID:   
    Image:          docker.io/jboss/keycloak:12.0.4
    Image ID:       
    Ports:          8080/TCP, 8443/TCP, 9990/TCP
    Host Ports:     0/TCP, 0/TCP, 0/TCP
    State:          Waiting
      Reason:       PodInitializing
    Ready:          False
    Restart Count:  0
    Liveness:       http-get http://:http/auth/ delay=300s timeout=5s period=10s #success=1 #failure=3
    Readiness:      http-get http://:http/auth/realms/master delay=30s timeout=1s period=10s #success=1 #failure=3
    Environment:
      DB_VENDOR:          postgres
      DB_ADDR:            sitewhere-postgresql
      DB_PORT:            5432
      DB_DATABASE:        keycloak
      DB_USER:            keycloak
      DB_PASSWORD:        <set to the key 'postgresql-password' in secret 'sitewhere-postgresql'>  Optional: false
      KEYCLOAK_USER:      sitewhere
      KEYCLOAK_PASSWORD:  sitewhere
    Mounts:
      /opt/jboss/startup-scripts/keycloak.cli from startup (ro,path="keycloak.cli")
      /var/run/secrets/kubernetes.io/serviceaccount from sitewhere-keycloak-token-bc7lw (ro)
Conditions:
  Type              Status
  Initialized       False 
  Ready             False 
  ContainersReady   False 
  PodScheduled      True 
Volumes:
  startup:
    Type:      ConfigMap (a volume populated by a ConfigMap)
    Name:      sitewhere-keycloak-startup
    Optional:  false
  sitewhere-keycloak-token-bc7lw:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  sitewhere-keycloak-token-bc7lw
    Optional:    false
QoS Class:       Burstable
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                 node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:          <none>