Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Liveness / Readiness probes for PythonWatchManager #38

Open
gabe-l-hart opened this issue Jan 22, 2024 · 0 comments
Open

Liveness / Readiness probes for PythonWatchManager #38

gabe-l-hart opened this issue Jan 22, 2024 · 0 comments

Comments

@gabe-l-hart
Copy link
Member

Is your feature request related to a problem? Please describe.

As an operator maintainer in an application, I need to be able to probe the liveness/readiness of the operator. With the AnsibleWatchManager, the native /healthz and /readyz HTTP endpoints were accessible, but there's no equivalent with PWM.

Describe the solution you'd like

Add a heartbeat_thread to the PWM's group of managed threads. This thread will periodically dump the current time to a file if enabled.

Describe alternatives you've considered

  • Use a lightweight HTTP server to expose GET endpoint(s). The downside to this is that adding an HTTP server is non-trivial to code and adds non-trivial dependencies to do in a robust way.
  • Require users to install procps in their docker builds and run a ps aux | grep ... style check to ensure the operator is running. This has the downside of requiring a potentially dangerous tool (ps can be used to probe running processes), and it is also a bad approximation for the state of the running operator code.
gabe-l-hart added a commit to gabe-l-hart/oper8 that referenced this issue Jan 22, 2024
@gabe-l-hart
PWMProbes: Add config for heartbeat thread in PWM
d0a11e3 
IBM#38
Signed-off-by: Gabe Goodhart <ghart@us.ibm.com>
gabe-l-hart added a commit to gabe-l-hart/oper8 that referenced this issue Jan 22, 2024
@gabe-l-hart
PWMProbes: Add HeartbeatThread as a child of TimerThread
972c420 
IBM#38
Signed-off-by: Gabe Goodhart <ghart@us.ibm.com>
gabe-l-hart added a commit to gabe-l-hart/oper8 that referenced this issue Jan 22, 2024
@gabe-l-hart
PWMProbes: Start/stop the heartbeat_thread in PWM
c3e0ba2 
IBM#38
Signed-off-by: Gabe Goodhart <ghart@us.ibm.com>
gabe-l-hart added a commit to gabe-l-hart/oper8 that referenced this issue Jan 22, 2024
@gabe-l-hart
PWMProbes: Add config for heartbeat thread in PWM
d61b21e 
IBM#38
Signed-off-by: Gabe Goodhart <ghart@us.ibm.com>
gabe-l-hart added a commit to gabe-l-hart/oper8 that referenced this issue Jan 22, 2024
@gabe-l-hart
PWMProbes: Add HeartbeatThread as a child of TimerThread
8c94fda 
IBM#38
Signed-off-by: Gabe Goodhart <ghart@us.ibm.com>
gabe-l-hart added a commit to gabe-l-hart/oper8 that referenced this issue Jan 22, 2024
@gabe-l-hart
PWMProbes: Start/stop the heartbeat_thread in PWM
725e79b 
IBM#38
Signed-off-by: Gabe Goodhart <ghart@us.ibm.com>
gabe-l-hart added a commit to gabe-l-hart/oper8 that referenced this issue Jan 22, 2024
@gabe-l-hart
PWMProbes: Add first (hanging) unit test for HeartbeatThread
bb42a01 
IBM#38
Signed-off-by: Gabe Goodhart <ghart@us.ibm.com>
gabe-l-hart added a commit to gabe-l-hart/oper8 that referenced this issue Jan 23, 2024
@gabe-l-hart
PWMProbes: Add config for heartbeat thread in PWM
479de90 
IBM#38
Signed-off-by: Gabe Goodhart <ghart@us.ibm.com>
gabe-l-hart added a commit to gabe-l-hart/oper8 that referenced this issue Jan 23, 2024
@gabe-l-hart
PWMProbes: Add HeartbeatThread as a child of TimerThread
39956ae 
IBM#38
Signed-off-by: Gabe Goodhart <ghart@us.ibm.com>
gabe-l-hart added a commit to gabe-l-hart/oper8 that referenced this issue Jan 23, 2024
@gabe-l-hart
PWMProbes: Start/stop the heartbeat_thread in PWM
bc8a6ce 
IBM#38
Signed-off-by: Gabe Goodhart <ghart@us.ibm.com>
gabe-l-hart added a commit to gabe-l-hart/oper8 that referenced this issue Jan 23, 2024
@gabe-l-hart
PWMProbes: Add first (hanging) unit test for HeartbeatThread
ac4165b 
IBM#38
Signed-off-by: Gabe Goodhart <ghart@us.ibm.com>
gabe-l-hart added a commit to gabe-l-hart/oper8 that referenced this issue Jan 23, 2024
@gabe-l-hart
PWMProbes: Fix lifecycle and error handling for HeartbeatThread
48fbeec 
IBM#38
Signed-off-by: Gabe Goodhart <ghart@us.ibm.com>
gabe-l-hart added a commit to gabe-l-hart/oper8 that referenced this issue Jan 23, 2024
@gabe-l-hart
PWMProbes: Unit tests for heartbeat thread
c8b3785 
IBM#38
Signed-off-by: Gabe Goodhart <ghart@us.ibm.com>
gabe-l-hart added a commit to gabe-l-hart/oper8 that referenced this issue Jan 23, 2024
@gabe-l-hart
PWMProbes: Fix Singleton to attach it to the specific cls
cba9e1f 
Previously, the hasattr call would use MRO to find _instance on the
parent so if the parent were instantiated before the child, the child
would reuse the parent's instance.

IBM#38
Signed-off-by: Gabe Goodhart <ghart@us.ibm.com>
gabe-l-hart added a commit to gabe-l-hart/oper8 that referenced this issue Jan 23, 2024
@gabe-l-hart
PWMProbes: Unit tests for heartbeat in PWM
c4c055a 
IBM#38
Signed-off-by: Gabe Goodhart <ghart@us.ibm.com>
@gabe-l-hart gabe-l-hart mentioned this issue Jan 23, 2024
Merged
3 tasks
gabe-l-hart added a commit to gabe-l-hart/oper8 that referenced this issue Jan 23, 2024
@gabe-l-hart
PWMProbes: Add check_heartbeat.sh
1536d86 
This script can be used in operator images as a exec probe

IBM#38

Signed-off-by: Gabe Goodhart <ghart@us.ibm.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@gabe-l-hart