This is a set of basic performance tests for promise implementations. As is almost always the case, take these with the usual grains of salt. That said, they should give a reasonable ballpark comparison of the performance characteristics of common, basic operations that most promise libraries provide.
Of course, performance is not the only thing to consider in a promise library. Interoperability via a proposed standard, such as Promises/A, API convenience, safety, and even code size (for browser applications) are all important, application-specific considerations.
Right now, the tests are runnable en masse via npm test in unix-like environments, and individually via node in other envs.
- Clone the repo
npm installto install the promise implementations to be tested- Run tests:
- Run all tests:
npm test - Run a single test with node:
node <test>
- Run all tests:
when.js uses synchronous resolutions, and no longer uses Object.freeze() as of v1.6.0, to avoid this unfortunate v8-imposed performance penalty.
avow is an example Promises/A+ implementation. In its default configuration, it uses asynchronous resolutions and does not call Object.freeze. However, it can be configured to use synchronous resolutions, and/or Object.freeze. Performance tests are run using the default configuration.
Q uses asynchronous resolutions, and calls Object.freeze on its promises, and so it incurs the v8-imposed performance penalty.
RSVP uses asynchronous resolutions, and doesn't use Object.freeze.
deferred uses synchronous resolutions, and doesn't use Object.freeze.
lassez-faire uses synchronous resolutions, and doesn't use Object.freeze.
jQuery uses synchronous resolutions, and it doesn't use Object.freeze.
These tests use jQuery via jquery-browserify, with jsdom for support. This approach was taken from the Promises Test Suite, and currently, appears to be the only way to use jQuery 1.8.x in node.
jQuery Deferred is not intended to be fully Promises/A compliant in its forwarding behavior. We've done our best to design the tests so that that does not affect the performance characteristics. While this does affect the computation results of some tests, it can be ignored for most performance testing purposes.
These tests were run on a MacBook Pro Intel Core i7, 2.3Ghz, 8g RAM, 256g SSD, using Node.js v0.8.14 installed via nvm and the following library versions (npm ls):
├── avow@1.0.0
├─┬ deferred@0.6.1
│ ├── es5-ext@0.9.1
│ ├── event-emitter@0.2.1
│ └── next-tick@0.1.0
├── jquery-browserify@1.8.1
├─┬ jsdom@0.2.19
│ ├─┬ contextify@0.1.3
│ │ └── bindings@1.0.0
│ ├── cssom@0.2.5
│ ├── cssstyle@0.2.3
│ ├── htmlparser@1.7.6
│ └─┬ request@2.11.4
│ ├─┬ form-data@0.0.3
│ │ ├── async@0.1.9
│ │ └─┬ combined-stream@0.0.3
│ │ └── delayed-stream@0.0.5
│ └── mime@1.2.7
├── laissez-faire@0.1.2
├── q@0.8.9
├── rsvp@1.0.0
└── when@1.6.1
Each test is sorted from best to worst time. Times are in milliseconds, and Diff is the percentage difference from the best time (computed via ((current - best) / best) * 100)).
==========================================================
Test: promise-fulfill x 10000
----------------------------------------------------------
Name Time ms Avg ms Diff %
when 4 0.0004 -
laissez 5 0.0005 25.00
deferred 7 0.0007 75.00
avow 24 0.0024 500.00
rsvp 74 0.0074 1750.00
q 145 0.0145 3525.00
jquery 154 0.0154 3750.00
Should be empty: []
==========================================================
Test: promise-reject x 10000
----------------------------------------------------------
Name Time ms Avg ms Diff %
when 8 0.0008 -
avow 13 0.0013 62.50
rsvp 48 0.0048 500.00
laissez 82 0.0082 925.00
deferred 83 0.0083 937.50
q 158 0.0158 1875.00
jquery 196 0.0196 2350.00
==========================================================
Test: promise-sequence x 10000
----------------------------------------------------------
Name Time ms Avg ms Diff %
when 8 0.0008 -
laissez 13 0.0013 62.50
deferred 19 0.0019 137.50
avow 106 0.0106 1225.00
rsvp 242 0.0242 2925.00
jquery 462 0.0462 5675.00
q 601 0.0601 7412.50
==========================================================
Test: defer-create x 10000
----------------------------------------------------------
Name Time ms Avg ms Diff %
laissez 1 0.0001 -
when 5 0.0005 400.00
avow 18 0.0018 1700.00
deferred 28 0.0028 2700.00
rsvp 60 0.0060 5900.00
jquery 160 0.0160 15900.00
q 238 0.0238 23700.00
==========================================================
Test: defer-fulfill x 10000
----------------------------------------------------------
Name Time ms Avg ms Diff %
avow 28 0.0028 -
when 35 0.0035 25.00
laissez 42 0.0042 50.00
rsvp 183 0.0183 553.57
deferred 227 0.0227 710.71
jquery 401 0.0401 1332.14
q 817 0.0817 2817.86
Should be empty: []
==========================================================
Test: defer-reject x 10000
----------------------------------------------------------
Name Time ms Avg ms Diff %
when 27 0.0027 -
avow 31 0.0031 14.81
laissez 99 0.0099 266.67
rsvp 290 0.0290 974.07
deferred 300 0.0300 1011.11
jquery 398 0.0398 1374.07
q 889 0.0889 3192.59
==========================================================
Test: defer-sequence x 10000
----------------------------------------------------------
Name Time ms Avg ms Diff %
laissez 12 0.0012 -
when 18 0.0018 50.00
avow 97 0.0097 708.33
deferred 189 0.0189 1475.00
rsvp 306 0.0306 2450.00
jquery 411 0.0411 3325.00
q 1023 0.1023 8425.00
==========================================================
Test: map x 10000
----------------------------------------------------------
Name Time ms Avg ms Diff %
when 12 0.0012 -
deferred 29 0.0029 141.67
==========================================================
Test: reduce-small x 609
NOTE: in node v0.8.14, deferred.reduce causes a
stack overflow for an array length >= 610
----------------------------------------------------------
Name Time ms Avg ms Diff %
when 1 0.0016 -
deferred 4 0.0066 300.00
==========================================================
Test: reduce-large x 10000
NOTE: in node v0.8.14, deferred.reduce causes a
stack overflow for an array length >= 610
----------------------------------------------------------
Name Time ms Avg ms Diff %
when 13 0.0013 -
deferred [RangeError: Maximum call stack size exceeded]