Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Makes heartbeat more tolerant to failures. #1884

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

arianf
Copy link

@arianf arianf commented Oct 19, 2023

Makes heartbeat more tolerant to failures.

Closes #1883

Unsure if I should also be cleaning up @@all_heartbeat_threads if we detect a dead heartbeat.

@arianf arianf marked this pull request as ready for review October 24, 2023 17:57
@arianf
Copy link
Author

arianf commented Nov 1, 2023

@PatrickTulskie any chance of getting this reviewed?

@PatrickTulskie
Copy link
Contributor

@arianf sure thing. I just gave it a quick pass, but it doesn't look like there were any added tests. Also I don't really know this area of the code super well so I might need some time to digest all of this.

In the mean time, mind adding some tests for your changes?

@@ -245,6 +245,7 @@ def work(interval = 5.0, &block)

loop do
break if shutdown?
start_heartbeat if !heartbeat_alive?
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could we document inline exactly what this is protecting against?

@PatrickTulskie
Copy link
Contributor

@arianf just wanted to bump this to see if you are running this in production anywhere and if it solves the issue you were having. Also, do you think you can add a test for this update? It would be interesting if this solved #1895

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

start_heartbeat can die while worker is still running
3 participants