Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Deadlock can occur when multiple concurrent disconnects trigger a reconnect #60

Open
cpforbes opened this issue Jun 11, 2021 · 0 comments

Comments

@cpforbes
Copy link

When a an error occurs and the client is unexpectedly disconnected from the broker it will automatically reconnect.
If this occurs when responding to multiple concurrent requests each thread will trigger a reconnect via the on_disconnect MQTT callback. Thus multiple on_connect and on_disconnect callback threads will be running concurrently.

It can occur that an on_disconnect callback thread is holding the ServiceManager Rlock and is attempting to acquire the _connected_lock in DxlClient (via the connected method) while at the same time an on_connect callback thread is attempting to acquire the ServiceManager lock (via ServiceManager.on_connect called from _on_connect_run)

This only occurs when many concurrent requests all trigger the broker to disconnect the client as it does when a oversized message is received. 6-10 concurrent requests will usually trigger the deadlock.

This is related to and/or the cause of #55

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant