You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Users are allowed to define a livenessProbe on their container (for now just on the main one, with #14853 also on all sidecars). If the user defines just a livenessProbe without also defining the same check as readinessProbe we can have the following situation:
Liveness probe starts failing, readiness probe stays ok (or our check to the TCP socket)
K8s will restart the affected container (not the entire pod)
Knative (QP) does not know about this situation and tries to forward requests to the local UC-port
The user sees an error (see below)
The situation resolves itself as soon as the restarted container is up and healthy again
@ReToCode when K8s restart the affected user container, QP's readinessProbe check should also failed because it inherits the origin readinessProbe configuration of user container, then the trafffic should not go to this pod, right? so I was thinking why did what you said happen. Please help to correct me if you are available.
Yes this is correct. When your readiness-probe also fails, this is not an issue. It only happens in a scenario when the liveness and readiness probes are two different probes, then liveness fails and K8s restarts the specific container (not Pod), thus the readiness could be healthy or take longer to also start failing. During that short period until the container is back up, traffic can error out.
I suppose this problem can also occur when the LivenessProbe has a different PeriodSeconds or/and FailureThreshold from the ReadinessProbe. In this case, the Readiness probe might spot a failure at different times and not remove Endpoints from the Service quickly enough.
For sidecars, their Readiness probe PeriodSeconds/FailureThreshold are rewritten by values from the user container. So in the end the sidecar Liveness probes would have to have the same PeriodSeconds/FailureThreshold as the user container's ReadinessProbe so at to minimize this type of errors.
Description
Users are allowed to define a
livenessProbe
on their container (for now just on the main one, with #14853 also on all sidecars). If the user defines just alivenessProbe
without also defining the same check asreadinessProbe
we can have the following situation:The text was updated successfully, but these errors were encountered: