Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How to terminate a generator should be documented #696

Open
abeluck opened this issue Mar 13, 2020 · 0 comments
Open

How to terminate a generator should be documented #696

abeluck opened this issue Mar 13, 2020 · 0 comments

Comments

@abeluck
Copy link

abeluck commented Mar 13, 2020

The docs say this about generators:

Generators - These are functions which provide values for the Stream. They are lazy and can be infinite, they can also be asynchronous (for example, making a HTTP request). You emit values on the Stream by calling push(err, val), much like a standard Node.js callback. Once it has been called, the generator function will not be called again unless you call next(). This call to next() will signal you've finished processing the current data and allow for the generator function to be called again. If the Stream is still being consumed the generator function will then be called again.
You can also redirect a generator Stream by passing a new source Stream to read from to next. For example: next(other_stream) - then any subsequent calls will be made to the new source.

I implemented the described interface. My generator is paging through HTTP responses. Once the pages were exhausted however the final consumption functions (like collect() or toArray() or toPromise()) were not being called.

I struggled for some time, thinking the problem was with my stream pipeline somewhere.

Only later did I discover that the stream source itself was still open waiting for a call to next(). By looking at other issues in this repo I eventually discovered the highland.nil "end of stream marker" Adding push(null, highland.nil) to my generator once the pages were exhausted instantly closed the stream and my problem was resolved.

This may seem obvious to those familiar with highland, but it was totally a surprise to me. Also I realize highland.nil clearly states it is the end of stream marker, but I hadn't browsed that part of the docs. Ideally I think the generator docs should mention something about terminating itself.

Finally.. before I added highland.nil my generator actually had no dependency on highland itself, which was quite nice. Now however it does. Is there a way to terminate the generator without requiring highland in my generator module?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant