Call BNG using asyncio to avoid deadlock #516
Open
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
In rare situations, when BNG outputs a lot of data to STDOUT
quickly and fills the buffer, PySB could lockup due to how
subprocess handles reading pipe buffers. The only way to
avoid this would be to disable streaming output from BNG,
which is often useful for debugging.
This PR calls BNG using Python's asyncio framework.
Reading STDOUT uses a timeout to avoid deadlocking.
If a timeout occurs, the process is checked to see if
it has exited, and any remaining data in the STDOUT
buffer is processed.