Add QuickJS as a Javascript engine option #4627
Conversation
Added QuickJS CI job variants which test each Erlang version alongside Spidermonkey. All have nouveau enabled as well. |
|
looks really good. what would a NIF look like? I mentioned elsewhere that rustler would be a nice way to do that safely. I think you said 100% NIF is conditional on dropping _list/_show support? |
|
A NIF would be easier to do if we didn't have the getRow/sendRow sub-protocol for list/show, where we have to call back into Erlang, wait for a row, then return back to Javascript. As a driver it's just simpler to do with print and readline. Even then, it might still make sense to retain external driver vs NIF duality, as it is possible to apply more restrictive isolation policies (seccomp, apparmor, selinux, etc) to the OS process. With all the Rustler might improve the situation if there was a Rust-only JS engine which doesn't wrap a C or C++ one. Then, we could have a NIF and feel like we're not making as much of a safety vs speed tradeoff. I've been keeping an eye on https://github.com/Starlight-JS/Starlight and https://github.com/boa-dev/boa. Boa one looks more promising. |
|
fair enough. |
nickva
force-pushed
the
qjs
branch
3 times, most recently
from
30ff2f7
to
9d255ea
Compare
nickva
force-pushed
the
qjs
branch
4 times, most recently
from
8386088
to
331c8eb
Compare
|
Some progress update:
TODO:
|
nickva
force-pushed
the
qjs
branch
6 times, most recently
from
997c267
to
79e9f8a
Compare
nickva
force-pushed
the
qjs
branch
5 times, most recently
from
441fca1
to
d55a944
Compare
|
Hey Nick, thats really great work. I installed the MSYS2 choco package to the Win-CI and was able to compile CouchDB with QuickJS on Windows. We need an additional small change (PR) in CouchDB Glazier that this works out-of-the-box. Running the tests: |
|
Thanks for checking, Ronny! |
nickva
force-pushed
the
qjs
branch
8 times, most recently
from
2211e0a
to
cc87df1
Compare
nickva
force-pushed
the
qjs
branch
7 times, most recently
from
626d9cc
to
0810831
Compare
Some benefits over Mozilla Spidermonkey:
* Small. We're using 6 or so C files vs 700+ SM91 C++ files.
* Built with Apache CouchDB as opposed having to maintain a separate SM
package, like for RHEL9, for instance, where they dropped support for SM
already [1].
* Embedding friendly. Designed from ground-up for embedding. SM has been
updating the C++ API such that we have to keep copy-pasting new versions of
our C++ code every year or so [2].
* Easy to modify to accept Spidermonkey 1.8.5 top level functions for
map/reduce code so we don't have have to parse the JS, AST transform it, and
then re-compile it.
* Better sandboxing. Creating a whole JSRuntime takes only 300 microseconds, so
we can afford to do that on reset. JSRuntimes cannot share JS data or object
between them.
* Seems to be faster in preliminary benchmarking with small
concurrent VDU and view builds:
https://gist.github.com/nickva/ed239651114794ebb138b1f16c5f6758
Results seem promising:
- 4x faster than SM 1.8.5
- 5x faster than SM 91
- 6x reduced memory usage per couchjs process (5MB vs 30MB)
* Allows compiling JS bytecode ahead of time a C array of bytes.
QuickJS can be built alongside Spidermonkey and toggled on/off at runtime:
```
./configure --dev --js-engine=quickjs
```
This makes it the default engine. But Spidermonkey can still be set in the
config option.
```
[couchdb]
js_engine = spidermonkey | quickjs
```
To test individual views, without switching the default use the
`javascript_quickjs` language in the design docs. To keep using Spidermonkey
engine after switching the default, can use `javascript_spidermonkey` language
in design docs. However, language selection will reset the view and the view
will have to be rebuilt.
It's also possible to build without Spidermonkey support completely by using:
```
./configure --disable-spidermonkey
```
Issue: #4448
[1] #4154
[2] #4305
https://bellard.org/quickjs
Some benefits over Mozilla Spidermonkey:
Small. We're using 6 or so C files vs 700+ SM91 C++ files.
Built with Apache CouchDB as opposed having to maintain a separate SM package, like for RHEL9, for instance, where they dropped support for SM already: RHEL 9 Support for CouchDB #4154.
Embedding friendly. Designed from ground-up for embedding. SM has been updating the C++ API such that we have to keep copy-pasting new versions of our C++ code every year or so: try compiling for sm 102 #4305.
Easy to modify to accept Spidermonkey 1.8.5 top level functions for map/reduce code so we don't have have to parse the JS, AST transform it, and then re-compile it.
Better sandboxing. Creating a whole JSRuntime takes only 300 microseconds, so we can afford to do that on reset. JSRuntimes cannot share JS data or object between them.
Seems to be faster in preliminary benchmarking with small concurrent VDU and view builds:
https://gist.github.com/nickva/ed239651114794ebb138b1f16c5f6758
Allows compiling JS bytecode ahead of time a C array of bytes.
QuickJS can be built alongside Spidermonkey and toggled on/off at runtime:
This makes it the default engine. But Spidermonkey can still be set in the config option.
To test individual views, without switching the default use the
javascript_quickjslanguage in the design docs. To keep using Spidermonkey engine after switching the default, can usejavascript_spidermonkeylanguage in design docs. However, language selection will reset the view and the view will have to be rebuilt.It's also possible to build without Spidermonkey support completely by using:
Issue: #4448
Some basic benchmarking
Summary:
Setup Details
MacOS Intel, Erlang 24, ./dev/run -n1
Benchmark script: https://gist.github.com/nickva/c0cbf6a556cc2dc7dd6ee79f504f5f84.
Results measured by the zsh
timecommand as% time ./stampede_dbs_ddocs_vdus.py .... CPU usage is reported for the client not the couchjs process or Erlang VM. Some example of resource usage is provided below as btop screenshots.QuickJS
Spidermonkey 1.8.5
Spidermonkey 91
Using btop locally with one Spidermonkey run and two QuickJS runs:
Memory usage 30-32MB RSS for Spidermonkey and 4-5MB for QuickJS. The time it took to run the benchmark can seen in the CPU usage graph.