Only for pipe or for every function?

Seems like almost every utils supporting iterable instead of arrays in iter-tools could be candidate to this change.

This is such a bare minimum feature I am surprised it has not been implemented for this long... it basically makes this library unusable for working with ES Maps and Sets (which use IterableIterator for all their methods) since everything has to constantly be converted to Arrays, it seems.

ES in general is also evolving towards using Iterator-like objects in new APIs.

Oh well, time to look for yet another alternative I guess...

I agree this is a great addition to the library, I'm just not sure it's easy enough to be done without completely rewriting most of the library.

If anyone has any suggestion on how to go about this I'd be happy to discuss it.

i think the main question is how the typing would work. map right now takes an array and returns an array. would we overload map with something that takes an iterable and returns an iterableiterator? that would technically be a breaking change, but it should be possible

i also don't think that converting everything to be iterator-based would be "completely rewriting most of the library". only 25 out of the 132 exported functions are lazy, and for many of them it wouldn't be too hard to convert. the hardest would of course be pipe, but if we converted everything to iterators, then the pipe implementation would also be much simpler (as we can runtime-inspect whether something is an iterator)

i think it should be possible to write a pipe that operates on iterators:

R.iteratorPipe(
  iterator,
  R.map(x => x + 1),
  R.filter(x => x > 0),
);

the problem would be typing. either we let this return an array (which seems bad), or we want this to return an iterator

if we want this to return an iterator, then (someone check me on this) every operator in the pipe should be a LazyOp (or operate on iterators). idk how you check this on the type level. maybe you can ask everyone to explicitly use the lazy version:

R.iteratorPipe(
  iterator,
  R.map.lazy(x => x + 1),
  R.filter.lazy(x => x > 0),
);

this is worse ergonomics. (would it be against the remeda spirit? why else do we runtime purry between datafirst and datalast, rather than having separate function names?)

It's unlikely I'll be able to invest the time in researching this as we are currently handling mainly feature parity to allow people to migrate off Ramda and Lodash, and clean up the API for a a v2 release.

My concern is that if the change requires rewriting a large chunk of the existing code then it would take a long time to make sure it's a valid solution.

hm, please take a look at #520 – what do you think?

Going over the pipe and purry functions lately, I have an idea for getting closer to using iterators. The main problem with the way we iterate lazily and how iterators work is that we "push" values into transformers instead of the transformer being able to pull them. What needs to be done:

• Add an "end-of-stream" symbol that could be pushed into a pipe. This would allow us to act not only when a value is pushed into the stream, which unlocks some other enhancements.
• Expand the "single" mode in pipes so that more of our functions can work lazily.
• Remove the "indexed" variants to simplify our codebase. Indexed affects how we lazily iterate because we maintain the index within the pipe.
• We need to bump our Typescript target to at least ES2015 so we can iterate over Iterables transparently without needing to manually handle them.

After all these changes, we can try to relax the APIs for some of our functions and see if it "just works". This would allow to put any iterable at the start of a pipe at least and have its values piped. The output might still be an object/array, not a map/set, but I think that's already a pretty good API.

Side note: This sounds like a version of Clojure's transducers.

Hello.

I published remeda-compatible npm package rotery, which expands remeda with Iterators and AsyncIterators support.

https://github.com/somnicattus/rotery