Switch to streamx

[phated]

streamx is a much better stream implementation and we should switch all the projects to use it.

[coreyfarrell]

I toyed with this locally, switching from through2 was easy and all tests passed. Worth noting streamx is documented as always running in both binary and object mode, so I think the only difference on require('fs-mkdirp-stream').obj is that it will pass highWaterMark: 16 to the constructor.

While I was working on the code I hit three lint parser errors:

• destructuring - const {Transform} = require('streamx')
• object method shorthand - { transform(chunk, cb) { /* code here */ } }
• object spread - {...options, /* other properties */}

Would you be open to adding "env": {"es2017": true}, "parserOptions": {"ecmaVersion": 2018} to eslint-config-gulp? Setting env.es2017 enables current globals, ecmaVersion enables current syntax. I know you just recently bumped this package to requiring node.js 10 which is why I'm suggesting ecmaVersion 2018 (2019 allows syntax that is for node.js 12+). Since ES5 is the default parser mode for eslint I'm not sure if you want to block newer syntax or if you simply haven't specified/updated it.

[phated]

Thanks for digging into this! I need to do some more digging, but I thought it wasn't required to set highWaterMark for objects.

With the update to node 10+, I'm fine supporting whichever syntax that runtime supports. I just didn't bother changing the parser since I wasn't planning to update rewrite libraries.

[coreyfarrell]

In the current release highWaterMark is set for obj only.

[phated]

Sorry, I meant that streamx is supposed to handle hwm according to the data flowing through the stream (according to the docs, but I haven't dug into the code). All of these changes are going to be batched into a major, so we can drop the .obj method.

An aside: I know you have a ton going on, but would you also be interested in joining the gulpjs core team? If not, I totally understand.

[coreyfarrell]

Looks like it unconditionally sets default hwm 16384.

Yes I'd be interested in joining the gulpjs core team. I can't promise consistent effort, my availability for open source tends to come in spurts.

[phated]

Looks like it unconditionally sets default hwm 16384.

Yeah, that's the hwm, but streamx uses the byteLength method in conjunction with hwm to determine the amount of data/objects that can go in the stream. If we don't specify
a byteLength method, the default
is used, which checks if the data is a buffer otherwise returns 1024 and 16384 / 1024 is the 16 objects we'd expect.

The docs also mention this as
"The default byteLength function returns the byte length of buffers and 1024 for any other object. This means the buffer will contain around 16 non buffers or buffers worth 16kb when full if the defaults are used."

Yes I'd be interested in joining the gulpjs core team. I can't promise consistent effort, my availability for open source tends to come in spurts.

Awesome! That's pretty much all of us too. I'll send the invite now and finish setting permissions, etc up this week.