Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Extending stream method inventory #17

Open
pfalcon opened this issue Sep 29, 2018 · 3 comments
Open

Extending stream method inventory #17

pfalcon opened this issue Sep 29, 2018 · 3 comments

Comments

@pfalcon
Copy link
Owner Author

pfalcon commented Sep 29, 2018

Per the latest evolution/iteration, considering to implement:

  1. stream.readline(sz=-1) to take another optional param, specifying "end of line value", i.e. stream.readline(sz=-1, end="\n"). The "end" param is not expected to be a keyword, but if it ever will be, it's named the same as print()'s argument.
  2. stream.readline_into(buf, sz=-1, end="\n")
  3. Change all "into" methods to accept not just buffer objects, but also BytesIO objects. This is an alternative to io.copy() proposal from Ideas on optimizing uasyncio micropython/micropython#2622 (comment) .

@pfalcon
Copy link
Owner Author

pfalcon commented Sep 30, 2018

So, the top-level decision point appears to be:

  1. Whether we allow for (adhoc) buffering to be built into existing stream-like objects.
  2. Or if there's a generic dual stream-buffer object which can be used even with unbuffered operations (and all current streams in uPy are unbuffered).

The current direction of thought follows choice 2, with BytesIO being such an object.

But fairly speaking, the fact that current streams in uPy are unbuffered is not ideal (won't compare favorably with benchmarks of buffered streams in other systems), so we may end up adding buffering directly to streams, and thus might rely on this buffering for some operations. But this buffering would be available only for some operations, e.g. read(), but not socket.recv(), the latter is not supposed to be buffered. But using data from recv() for buffering/stream access is still oftentimes useful, so the approach of having an external generic buffer still comes as beneficial.

(Then the next question will be how to avoid code, etc., duplication when implementing buffering for standard, e.g. file, streams - and that requires separate consideration and answers. Reusing BytesIO underlyingly can be one of the answers, unless there's a clear benefit to implement a more lightweight variant.)

@pfalcon
Copy link
Owner Author

pfalcon commented Dec 13, 2018

  1. stream.readline_into(buf, sz=-1, end="\n")
  2. Change all "into" methods to accept not just buffer objects, but also BytesIO objects. This is an alternative to io.copy() proposal from Ideas on optimizing uasyncio micropython/micropython#2622 (comment) .

Instead of that 2., and following 3., now considering to do:

stream.readinto(BytesIO(), size=-1, until="\n")

until= still can be end=, but that's less clear...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant