MessageStream unfavourable interactions with tokio's co-operative scheduling

[tenuous-guidance]

I'll start by noting that whilst this is an issue, tokio doesn't currently support a clean solution for this, so it may not be something you want to resolve.

Problem statement

IFF:

• Run in a loop over select!
• There's a lot of data to read from kafka
• Other futures are called inside `select!

Then:

• The other futures will always return Pending, even when they could return data
• The tokio executor might starve other tasks

Explanation

tokio implements cooperative scheduling, whereby tasks will willingly return Pending when they could return a result, in order to lead to a smoother overall execution flow1. This approximately works by tokio primitives always returning Pending if the last 128 .await calls returned Ready1.

As MessageStream doesn't use any tokio primitives, it will not voluntarily pause execution. If looping over select or similar, as it is the only call that does not pause, it will continue executing, starving all other branches.

Possible solutions

1. The ideal solution would be to use consume_budget
   • Unfortunately, this is an unstable tokio feature, requiring all library consumers to enable the unstable feature
   • We could consider pushing for tokio to stabilise this or something similar
2. Implement our own co-operative solution, where we return Pending every X Readys
3. Delegate to tokio's co-operative solution by calling into a no-op function (e.g. an async read on Empty
   • This is the solution I've gone for in the wrapper I'm using to solve this issue for my code
   • It's not pretty but avoids using a custom solution and doesn't require unstable features
4. Accept the current behaviour and document it with a warning.

[duarten]

Another option is to add some code for detecting when a task is voluntarily yielding, like FuturesUnordered does.

I agree consume_budget would be ideal, but it's not cross-runtime. There's a proposal to address that, but I don't know if it has any traction.

[tenuous-guidance]

Given it was closed early-mid last year, I don't think there is much traction.

[tenuous-guidance]

I've been thinking about the approach, and I wonder if you'd be open to having a solution toggled by the tokio feature. Then if this is also noticed with other executors, if any of them are following a similar approach to co-operative scheduling their solutions could go under their own features.

I realise it's less than ideal, but would solve the problem in the short run and would hopefully be an easy lift if something like consume_budget is standardised in the future.