Readiness probe keeps running when the pod is ready

[Nicofuma]

Kubernetes version (use kubectl version): v1.2.0-36

Environment:

• Cloud provider or hardware configuration: OpenShift Enterprise v3

What happened:
I am using an OpenShift 3.2.0.44 platform (which is running Kubernetes master 1.2.0-36-g4a3f9c5).
On this platform I launched a POD with a single container, which has a readiness probe. My issue is that when the container is determined as being ready, I can see the process of the probe spawning regularly (according to the periodSeconds value)

Note: it seems that there is another issue with periodSeconds: I tried to set the following readiness probe and have a warning saying that my container didn't pass its readiness check. Considering the executed command, it looks like the first check will be spawned at least 3600 seconds after the creation of the container

              readinessProbe:
                exec:
                  command:
                    - "true"
                initialDelaySeconds: 30
                timeoutSeconds: 1
                periodSeconds: 3600
                successThreshold: 1
                failureThreshold: 1

What you expected to happen:
Once the pod is determined ready I am not expecting the readiness probe to keep being spawned (which eats up resources).

How to reproduce it (as minimally and precisely as possible):
Adds this readiness probe and looks at the /tmp/readiness file in the pod:

              readinessProbe:
                exec:
                  command:
                    - "bash"
                    - "-c"
                    - "echo test >> /tmp/readiness"
                initialDelaySeconds: 30
                timeoutSeconds: 1
                periodSeconds: 1
                successThreshold: 1
                failureThreshold: 1

You will see an entry appended to the file every second.

Anything else do we need to know:
I apologize if it has already been reported or fixed but I didn't see anything related in the changelog or in the issues.

[Nicofuma]

If I am reading the source code (https://github.com/kubernetes/kubernetes/blob/master/pkg/kubelet/prober/worker.go) correctly, both issues are still present in master.

There isn't any special case to stop readiness probe when the success threshold is hit, and the tickers' first tick is not immediate but respect the configured delay (the periodSeconds in our case)

[Nicofuma]

I re-read the doc and it's actually the expected behaviour so closing it.

[ticpu]

I could not find where in the doc this is expected; https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-probes/#define-readiness-probes

I was also expecting the behaviour you mentioned in your original issue.

That is what I understood from the name (and documentation) of the probes.
Readyness probe starts after initialDelaySeconds, tries again each periodSeconds for successThreshold# times and will retry if probe takes more than timeoutSeconds between periodSeconds.
Once pod is marked as ready, stop readynessProbe and engage livenessProbe until the container stops responding or is stopped.

Am I also mistaken?

[ylfforme]

I could not find where in the doc this is expected; https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-probes/#define-readiness-probes

I was also expecting the behaviour you mentioned in your original issue.

That is what I understood from the name (and documentation) of the probes.
Readyness probe starts after initialDelaySeconds, tries again each periodSeconds for successThreshold# times and will retry if probe takes more than timeoutSeconds between periodSeconds.
Once pod is marked as ready, stop readynessProbe and engage livenessProbe until the container stops responding or is stopped.

Am I also mistaken?

readinessProbe and livenessProbe all need to keep running . livenessProbe check the container is live or dead if dead restart the container. But it can't tell the service if the pod is ok to serve traffic. The readinessProbe is tell the service if the pod is ok to serve traffic. if it not ok,the pod ip will delete from service endpoints, so It needs to keep running too.

[ticpu]

Thanks for clarifying. After some tests I was able to confirm that behaviour.

The part of the documentation that mislead me was;

For example, an application might need to load large data or configuration files during startup.

since it ends by "during startup". But this is only one common example.