Startup order dependencies problem

[Snob1337]

Hi,

we are using the FROST-Server (v1.14.0) with kubernetes.

After investigating our last issues, it looks like the pods has to start in the correct order.

1. Database
2. HTTP
3. MessageBus
4. MQTT

If the Mqtt is starting before Database modul or MessageBus does, it failes and never try to reconnect again.
The only way to solve this issue is to restart the pod manually.

Is this a known Problem or wrong configured from our side?

Many thanks in advance.

Best regard,
Marc Beckord

[hylkevds]

Yes, it's a known issue. I've already improved it in the 2.0 branch. Maybe I can backport the improvements.
Note that the MessageBus should be on 2 (or together with the DB on 1)

[Snob1337]

Thank you for the fast reply.

to start the pods in the correct order, we need to write an kubernetes-operator by our own.
Unfortunately Kubernetes is not able to setup service-dependencies like Docker Swarm does.

Would be great if you could keep me up to date about backporting the improvements.

Best,
Marc

[phertweck]

Like mentioned by @hylkevds, this is a known issue. That's something that should be extended in the Kubernetes manifest files and the Helm chart. An operator is not needed in this case. It can be solved by using init-containers, that wait until the dependent services are available.

If you like, Pull requests are always welcome :-)

[oldluke92]

Hi!

Unfortunately init-containers only cover the initial startup - but we also observed issues where already running components (HTTP/MQTT) stopped working after others (DB/Message Bus) were restarted for example due to being moved to another node. This could be a different issue though ....

Best regards
Lukas