Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Leverage node-glob stats cache #237

Closed
wants to merge 397 commits into from

Conversation

erikkemperman
Copy link
Member

@phated As discussed elsewhere, a PR that might serve as discussion piece on the merits and perils of node-glob's cache(s).

Contra and others added 30 commits February 21, 2015 18:28
Implemented `overwrite` option in `dest()`
this relies on the recent breaking changes to glob-watcher being published to npm as 2.0
switch to glob-watcher 2.0
It has been shown in gulpjs#62 this bit can be platform specific and is unreliable, testing fs.chmod is not called will be enough.
Remove realMode equality test on special bit test
export `filterSince` to `vinyl-filter-since`
@erikkemperman
Copy link
Member Author

This needs to be rebased (including the new options resolution), but I'm not sure how. My first thought is that resolve-options could somehow be made to operate across modules, and then glob-stream could make use of it too. I'll get back to this when I find more time...

@phated phated added this to Nice to Have in v4 Jun 26, 2017
@phated
Copy link
Member

phated commented Jun 29, 2017

@erikkemperman I've been running some benchmarks with your use-cache branch and it doesn't seem to improve the performance of src -> dest (I'm comparing master vs use-cache branch).

@erikkemperman
Copy link
Member Author

@phated I have a hunch the performance degradation is elsewhere, and might be sufficiently bad that it defeats measuring the impact of using this cache. Looking into that now...

@phated
Copy link
Member

phated commented Jun 29, 2017

@erikkemperman I agree that the main degradation is elsewhere, but this actually reduces the performance more (on top of that). Hopefully we can figure it out.

@erikkemperman
Copy link
Member Author

erikkemperman commented Aug 31, 2017

Rebased, would depend on (something like) gulpjs/resolve-options#3

See line 40 in package.json

@phated
Copy link
Member

phated commented Dec 5, 2017

This was closed due to me rebasing master. Please resubmit it to the updated codebase.

@phated
Copy link
Member

phated commented Dec 19, 2017

@erikkemperman just noticed this was never re-opened after my rebase. Still have interest in it?

@erikkemperman
Copy link
Member Author

erikkemperman commented Dec 20, 2017

@phated Ah, thanks for reminding me! Yeah, I looked into it briefly, trying to figure out exactly in which cases node-glob populates the stat cache.

Turns out it's only for patterns that don't have any wildcards in the last path component, e.g. **/literal, because only in that case will it do an fs.stat (actually an lstat first) to see if the file exists. Otherwise it'll do a readdir to get a list to match the wildcard patterns against and will assume those files exist and not get any stats (unless you force it to with a flag).

I'd be happy to rebase, but would need something like gulpjs/resolve-options#3 in place, if I remember correctly.

@phated phated removed this from Nice to Have in v4 Mar 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: No status
Development

Successfully merging this pull request may close these issues.

None yet