Skip to content

WebAssembly Component Model based REPL with sandboxed multi-language plugin system - unified codebase runs in CLI (Rust) and web (TypeScript)

License

Notifications You must be signed in to change notification settings

topheman/webassembly-component-model-experiments

Repository files navigation

WebAssembly Component Model Experiments

Crates.io Demo

The WebAssembly Component Model is a broad-reaching architecture for building interoperable WebAssembly libraries, applications, and environments.

It is still very early days, but it is a very promising technology. However, the examples out there are either too simple or too complex.

The goal of this project is to demonstrate the power of the WebAssembly Component Model, with more than a simple hello world.

It is a basic REPL, with a plugin system where:

  • plugins can be written in any language compiling to WebAssembly
  • plugins are sandboxed by default
  • the REPL logic is written in Rust, it also compiles to WebAssembly (you could swap it for your implementation in your own language)

There are two kinds of hosts:

  • a CLI host pluginlab, written in Rust running in a terminal
  • a web host, written in TypeScript running in a browser

Those hosts then run the same codebase which is compiled to WebAssembly:

  • the REPL logic
  • the plugins (made a few in rust, C and TypeScript)

Security model: the REPL cli implements a security model inspired by deno:

  • --allow-net: allows network access to the plugins, you can specify a list of domains comma separated (by default, no network access is allowed)
  • --allow-read: allows read access to the filesystem
  • --allow-write: allows write access to the filesystem
  • --allow-all: allows all permissions (same as all the flags above), short: -A

Plugins are sandboxed by default - they cannot access the filesystem or network unless explicitly permitted. This allows safe execution of untrusted plugins while maintaining the flexibility to grant specific permissions when needed.

Plugins like ls or cat can interact with the filesystem using the primitives of the languages they are written in.

  • on the CLI, a folder from the disk is mounted via the --dir flag
  • on the browser, a virtual filesystem is mounted, the I/O operations are forwarded via the @bytecodealliance/preview2-shim/filesystem shim, which shims the wasi:filesystem filesystem interface

Demo

Check the online demo at
topheman.github.io/webassembly-component-model-experiments

Example of running the CLI pluginlab pluginlab demo

Previous work with WebAssembly

In the last seven years I've done a few projects involving rust and WebAssembly:

Usage

pluginlab (rust) - REPL cli host

Install

# Install the pluginlab binary
cargo install pluginlab

Run

pluginlab\
  --repl-logic https://topheman.github.io/webassembly-component-model-experiments/plugins/repl_logic_guest.wasm\
  --plugins https://topheman.github.io/webassembly-component-model-experiments/plugins/plugin_greet.wasm\
  --plugins https://topheman.github.io/webassembly-component-model-experiments/plugins/plugin_ls.wasm\
  --plugins https://topheman.github.io/webassembly-component-model-experiments/plugins/plugin_echo.wasm\
  --plugins https://topheman.github.io/webassembly-component-model-experiments/plugins/plugin_weather.wasm\
  --plugins https://topheman.github.io/webassembly-component-model-experiments/plugins/plugin_cat.wasm\
  --plugins https://topheman.github.io/webassembly-component-model-experiments/plugins/plugin-echo-c.wasm\
  --allow-all

Other flags:

  • --dir: directory to be preopened (by default, the current directory)
  • --allow-net: allows network access to the plugins, you can specify a list of domains comma separated (by default, no network access is allowed)
  • --allow-read: allows read access to the filesystem
  • --allow-write: allows write access to the filesystem
  • --allow-all: allows all permissions (same as all the flags above), short: -A
  • --help: displays manual
  • --debug: run the host in debug mode (by default, the host runs in release mode)
🚀 Example of running the CLI host
pluginlab\
  --repl-logic https://topheman.github.io/webassembly-component-model-experiments/plugins/repl_logic_guest.wasm\
  --plugins https://topheman.github.io/webassembly-component-model-experiments/plugins/plugin_greet.wasm\
  --plugins https://topheman.github.io/webassembly-component-model-experiments/plugins/plugin_ls.wasm\
  --plugins https://topheman.github.io/webassembly-component-model-experiments/plugins/plugin_echo.wasm\
  --plugins https://topheman.github.io/webassembly-component-model-experiments/plugins/plugin_weather.wasm\
  --plugins https://topheman.github.io/webassembly-component-model-experiments/plugins/plugin_cat.wasm\
  --plugins https://topheman.github.io/webassembly-component-model-experiments/plugins/plugin-echo-c.wasm\
  --allow-all
[Host] Starting REPL host...
[Host] Loading REPL logic from: https://topheman.github.io/webassembly-component-model-experiments/plugins/repl_logic_guest.wasm
[Host] Loading plugin: https://topheman.github.io/webassembly-component-model-experiments/plugins/plugin_greet.wasm
[Host] Loading plugin: https://topheman.github.io/webassembly-component-model-experiments/plugins/plugin_ls.wasm
[Host] Loading plugin: https://topheman.github.io/webassembly-component-model-experiments/plugins/plugin_echo.wasm
[Host] Loading plugin: https://topheman.github.io/webassembly-component-model-experiments/plugins/plugin_weather.wasm
[Host] Loading plugin: https://topheman.github.io/webassembly-component-model-experiments/plugins/plugin_cat.wasm
repl(0)> echo foo
foo
repl(0)> echo $ROOT/$USER
/Users/Tophe
repl(0)> export FOO=toto

repl(0)> echo $FOO toto repl(0)> greet $FOO Hello, toto! repl(0)> ls wit wit/host-api.wit wit/plugin-api.wit wit/shared.wit repl(0)> weather Paris Sunny repl(0)> weather New York Partly cloudy repl(0)> azertyuiop Unknown command: azertyuiop. Try help to see available commands. repl(1)> echo $? 1 repl(0)> greet $USER Hello, Tophe! repl(0)> echo $0 Hello, Tophe! repl(0)>

web-host (typescript)

Go check topheman.github.io/webassembly-component-model-experiments online demo.

Development

Prerequisites

  • Rust 1.87+
  • Node.js 22.6.0+ (needs --experimental-strip-types flag)
  • just

Setup

# Add WebAssembly targets
rustup target add wasm32-unknown-unknown wasm32-wasip1
# Install project dependencies (web part)
npm install
# Install Playwright browsers (e2e tests for web-host)
npx playwright install

C tooling

From the WebAssembly Component Model section for C tooling

# Initialize the .env file tracking the WASI SDK version for C development
# You will be asked to update the WASI_OS and WASI_ARCH variables if needed
just init-env-file
cargo install wit-bindgen-cli@0.43.0
# Install the wasm-tools tool - you can also use cargo install wasm-tools@1.235.0 if you don't have cargo-binstall
cargo binstall wasm-tools@1.235.0
# Download the WASI SDK into ./c_deps/wasi-sdk folder
just dl-wasi-sdk

pluginlab (rust) - REPL cli host

Build

just build

This will (see justfile):

  • compile the pluginlab crate from rust to a binary file
  • compile the repl-logic-guest crate from rust to wasm
  • compile the plugin-* crates from rust to wasm
  • compile the c_modules/plugin-* C plugins to wasm

Run

./target/debug/pluginlab\
  --repl-logic ./target/wasm32-wasip1/debug/repl_logic_guest.wasm\
  --plugins ./target/wasm32-wasip1/debug/plugin_greet.wasm\
  --plugins ./target/wasm32-wasip1/debug/plugin_ls.wasm\
  --plugins ./target/wasm32-wasip1/debug/plugin_echo.wasm\
  --plugins ./target/wasm32-wasip1/debug/plugin_weather.wasm\
  --plugins ./target/wasm32-wasip1/debug/plugin_cat.wasm\
  --plugins ./c_modules/plugin-echo/plugin-echo-c.wasm\
  --allow-all

This will run the pluginlab binary which will itself:

  • load and compile the repl_logic_guest.wasm file inside the embedded wasmtime engine injecting the host-api interface
  • load and compile the plugin_*.wasm files into the engine, injecting the plugin-api interface
  • launch the REPL loop executing the code from the repl_logic_guest.wasm file which will:
    • readline from the user
    • parse the command
    • dispatch the command to the plugin(s) if needed (run the run, man functions of the plugins via the host-api interface)
    • display the result

Other example:

./target/debug/pluginlab\
  --repl-logic ./target/wasm32-wasip1/debug/repl_logic_guest.wasm\
  --plugins ./target/wasm32-wasip1/debug/plugin_ls.wasm\
  --plugins ./target/wasm32-wasip1/debug/plugin_echo.wasm\
  --dir /tmp\
  --allow-all

Test

# Runs unit tests and e2e tests on the REPL
just test
# Runs e2e tests on the REPL using the plugins from the http server
just test-e2e-pluginlab-http

Make a rust plugin

cargo component new --lib crates/plugin-hello

Publish

# Dry run
just publish-pluginlab-dry-run

Once you're happy with the changes, you can publish the pluginlab crate:

just publish-pluginlab

web-host (typescript)

Dev

npm run web-host:dev

This Will (see packages/web-host/package.json):

  • generate types from the wit files using the jco tool
  • build the plugins from rust to wasm (so that you don't have to do it manually)
  • build the repl-logic-guest from rust to wasm (so that you don't have to do it manually)
  • copy the wasm files in target/wasm32-wasip1/release to the packages/web-host/public/plugins directory (to make them available via http for the pluginlab binary)
  • transpile the wasm files to javascript using the jco tool into packages/web-host/src/wasm/generated/*/transpiled (this is the glue code wrapping the wasm files which is needed to interact with in the browser or node)
  • start the vite dev server

Go to http://localhost:5173 to see the web host.

Build

npm run web-host:build

Will do the same as the dev command, small changes:

  • the build tasks called on the rust side are just *-release (release mode)
  • it doesn't start the vite dev server, it builds the static files in the dist directory

You can then run npm run web-host:preview to preview the build.

Test

The project is configured to run e2e tests on the web-host using playwright, the test files are in packages/web-host/tests.

To run the tests against your local dev server (launched with npm run dev)

  • npm run test:e2e:all: will run all the tests in headless mode
  • npm run test:e2e:ui: will open the playwright ui to run the tests

To run the tests against a preview server (build with npm run build and launched with npm run preview):

  • npm run test:e2e:all:preview: will run all the tests in headless mode
  • npm run test:e2e:ui:preview: will open the playwright ui to run the tests

Specific to github actions:

In .github/workflows/web-host.yml, after the build step, the tests are run against the preview server.

To be sure that the preview server is up and running before running the tests, we use the webServer.command option of playwright.config.ts to run WAIT_FOR_SERVER_AT_URL=http://localhost:4173/webassembly-component-model-experiments/ npm run test:e2e:all:preview

plugins

There are currently plugins implemented in 3 languages (most of them are in rust):

Rust

You can write plugins in rust in crates/plugin-*.

C

You can write plugins in C in c_modules/plugin-*, thanks to wit-bindgen (based on wit-bindgen).

TypeScript

You can also write plugins in TypeScript in packages/plugin-*, thanks to jco componentize (based on componentize-js).

The downsides of writing plugins in TypeScript is mostly that your .wasm file will be much larger than the one compiled from rust or C:

  • ~100KB of wasm for the rust plugin
  • 11MB of wasm for the TypeScript plugin

The reason is that a JavaScript runtime needs to be embedded in the .wasm file, which is not the case for the rust plugin.

More about the SpiderMonkey runtime embedding.

Other languages

Coming.

Developer experience

Formating and linting

  • I use biome for formating and linting the TypeScript code
  • I use cargo fmt for formating the rust code
  • They are both configured to run on save in the editor

Git hooks

  • I use husky to run lint-staged on pre-commit
  • I use lint-staged to run linting and formating on the changed files - the following are automatically run on pre-commit:
    • formating / linting the TypeScript code
    • formating the rust code
    • typechecking the TypeScript code

Resources

Optional tools

Those are optional tools that are handy for WebAssembly development:

# latest versions
cargo binstall cargo-component wasm-tools wasm-opt
# specific versions I used for this project
cargo binstall cargo-component@0.21.1 wasm-tools@1.235.0 wasm-opt@116

C tooling