Skip to content

Commit

Permalink
Use nextTick to avoid extremely deep stack traces
Browse files Browse the repository at this point in the history
Sometimes the preprocessors and compilers call their callback
synchronously. In those cases, async.eachSeries will recurse
synchronously, and the call stack will grow very large.

Also see caolan/async#75 (comment)
and caolan/async#173 (comment).

Incidentally, this cuts initial build time in half.
  • Loading branch information
joliss committed Nov 14, 2013
1 parent 1bc1bbe commit 7b7fd51
Showing 1 changed file with 7 additions and 3 deletions.
10 changes: 7 additions & 3 deletions lib/generator.js
Expand Up @@ -110,11 +110,13 @@ Generator.prototype.preprocess = function (callback) {
async.eachSeries(paths, function (path, pathCallback) {
if (path.slice(-1) === '/') {
mkdirp.sync(self.preprocessDest + '/' + path)
pathCallback()
process.nextTick(pathCallback) // async to avoid long stack traces

This comment has been minimized.

Copy link
@stefanpenner

stefanpenner Nov 16, 2013

Contributor

unsure but we use another trick, to only enforce 1 async turn in RSVP, it may be work checking out.

https://github.com/tildeio/rsvp.js/blob/master/lib/rsvp/async.js#L57-L65

This comment has been minimized.

Copy link
@joliss

joliss Nov 16, 2013

Author Member

Hm... Is the whole coalescing thing just to squeeze out the last bit of performance? So for less sensitive stuff you can just do unconditional nextTick?

This comment has been minimized.

Copy link
@stefanpenner

stefanpenner Nov 21, 2013

Contributor

in some versions of node, it was easy to blow some sort of stack limit. Although maybe thats now just entirely gone.

} else {
var possiblePreprocessors = [].concat(package.preprocessors, self.preprocessors)
processFile(package.srcDir, path, possiblePreprocessors, function (err) {
pathCallback(err)
process.nextTick(function () { // async to avoid long stack traces
pathCallback(err)
})
})
}
}, function (err) {
Expand Down Expand Up @@ -274,7 +276,9 @@ Generator.prototype.compile = function (callback) {
this.compileDest = mktemp.createDirSync(this.dest + '/compile_dest-XXXXXX.tmp')
async.eachSeries(self.compilers, function (compiler, callback) {
compiler.run(self.preprocessDest, self.compileDest, function (err) {
callback(err)
process.nextTick(function () { // async to avoid long stack traces
callback(err)
})
})
}, function (err) {
callback(err)
Expand Down

3 comments on commit 7b7fd51

@stefanpenner
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

neat

@joliss
Copy link
Member Author

@joliss joliss commented on 7b7fd51 Nov 15, 2013

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Btw let me know if you want commit bit.

@stefanpenner
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I appreciate the offer, I am scared if I have commit bit, I will never sleep again :P

Please sign in to comment.