Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Conversion from blocking primitives (e.g. Iterable) and deadlocks #2205

Open
Scottmitch opened this issue May 5, 2022 · 2 comments
Open

Comments

@Scottmitch
Copy link
Member

Scottmitch commented May 5, 2022

additional context: #2204

If FromIterablePublisher isn't able to unwrap to get access to the underlying Publisher deadlock is possible on consumption for the same reasons described in this PR. There are two options

Ensure the subscription and subscriber aren't using the same thread.

We can force a subscribeOn however this brings new questions:

  • offloading may not always be necessary
  • what level is the offloading applied at (e.g. do we require it during the conversion, for http we do this under the covers so we may have to hook into the execution strategy)
  • how do we infer/accept an executor (if on the concurrent.api package we don't have an executor to use, if we require one as a parameter that starts to overload operator responsibilities)
  • are there other areas in the blocking APIs with similar issues (InputStream, ..)?

Reduce default demand to 1 from PublisherAsBlockingIterable

This isn't ideal because it means demand will be very chatty and no batching is done, but will mean additional offloading isn't required.

@Scottmitch
Copy link
Member Author

From @bondolo #2203 (comment)

If it is not possible to recognize the source as potentially blocking on hasNext() then perhaps demand should be reduced to 1.

There doesn't seem to be a way to insert a subscribeOn here without using global executor which would be unacceptable for at least some customers.

are there other areas in the blocking APIs with similar issues?

I am concerned that there probably are and curious that none of our existing tests uncovered them. This problem reminds me of the problems users encounter with PipedOutputStream

@Scottmitch
Copy link
Member Author

From @bondolo 0cf2ac2#r866337445

A more detailed comment on itemsToRequestNext describing problem with blocking Iterator might be helpful.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant