You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
If I want a transform stream to be able to be used as just a writer as well as a true transform, how do I stop pushing data to a possibly non-existant downstream?
#4340
Open
1mike12 opened this issue
Feb 7, 2024
· 0 comments
So I have a custom transform stream that basically just lets me accumulate data from upstream, and run a async task when it's hit a predefined batch limit. The use case is to stream files and batch insert into a database.
What I understand is that we can use a transform stream either as a true transformer, or just as a writer AKA the end of the stream pipeline.
However, if we want a stream that can behave correctly in either mode, does this just work automatically? If I push to the read side buffer but there is nothing attached on the read side, wouldn't the read side buffer of my transform grow forever? I don't get it. I can't find any conversation about this anywhere online.
Details
So I have a custom transform stream that basically just lets me accumulate data from upstream, and run a async task when it's hit a predefined batch limit. The use case is to stream files and batch insert into a database.
What I understand is that we can use a transform stream either as a true transformer, or just as a writer AKA the end of the stream pipeline.
However, if we want a stream that can behave correctly in either mode, does this just work automatically? If I push to the read side buffer but there is nothing attached on the read side, wouldn't the read side buffer of my transform grow forever? I don't get it. I can't find any conversation about this anywhere online.
Node.js version
Not applicable.
Example code
Operating system
n/a
Scope
code
Module and version
streams
The text was updated successfully, but these errors were encountered: