Explore: Bun for bundling, testing, developing (instead of Vite, vitest, esbuild, rollup, multiple configs, ...)

[adamziel]

https://bun.sh/ is amazing – let's explore moving the entire Playground to it.

Bun "just works"

Bun just does the right thing. I can point it to a file deep in a monorepo and it will resolve all the imports etc. without a single config f

It seems like https://bun.sh/docs/bundler has na upper hand over Vite:

• It's a single tool while Vite is rollup + esbuild and requires writing plugins for both build tools
• It can handle browser libraries and server libraries without sweat (while Vite requires trickery)
• It almost "just works" with zero config. No more paths issues or Vite plugins issues
• The documentation seems nicer than Vite
• It is fast – bun packages/playground/cli/src/main.ts just works, while in Vite we don't have a corresponding command so we need to build everything before running.
• It's the same tool we'd use for the Playground server (A Playground server package #1056)
• Maybe it would help us ditch nx?
• ...there's more

Here's how Bun performs at specific tasks.

Development run

This single command runs the CLI version of PHP.wasm:

$ bun --watch ./packages/php-wasm/cli/src/main.ts
Usage: php [options] [-f] <file> [--] [args...]
   php [options] -r <code> [--] [args...]

Here's what I didn't do with Bun but had to do with Vite:

• Write a config file
• cd into the package directory first – I'm running this from the repo root!
• Install a watcher or struggled to get it to work on my OS
• Spend a week looking for the right set of plugins to get this setup to work
• Run into module resolution issues around CJS, ESM, monorepo setup etc.
• Run into any TypeScript issues

Bundling

I ran this command and Bun just bundled the @php-wasm/cli package:

bun build ./packages/php-wasm/cli/src/main.ts --outdir tmp --target=node

Here's what I didn't do with Bun but had to do with Vite:

• Wait for all the dependent nx builds – bun build took only 22ms!
• Write a build config that's separate from the dev config
• Worry about ESM vs CJS of my code or the installed dependencies
• Give up and write a separate esbuild setup because Vite couldn't do something I needed for Node.js

Testing

I ran this command and Bun just ran the tests:

bun test ./packages/php-wasm/node/src/test/php-vars.spec.ts                                                                                                                                  1 ↵
bun test v1.1.3 (2615dc74)

packages/php-wasm/node/src/test/php-vars.spec.ts:
✓ phpVar > Encodes Hello, World [17.39ms]
✓ phpVar > Encodes This is `Playground's` "test suite" [11.34ms]
✓ phpVar > Encodes This is \\`Playground's\` \"test suite\"\ [11.06ms]
✓ phpVar > Encodes $variables \$variables $$variables [11.12ms]
✓ phpVar > Encodes
	 [10.75ms]
✓ phpVar > Encodes こんにちは [12.09ms]
✓ phpVar > Encodes %s [9.96ms]

Here's what I didn't do with Bun but had to do with Vite:

• Install a separate testing tool
• Write a test config that's separate from the build or dev config
• Worry about ESM vs CJS of my code or the installed dependencies

New features

Single-file executable

Bun can build a TypeScript project into a single executable. We could use it to distribute a Playground CLI tool without worrying about Node versions, node_modules etc.

Related Resources

• Playground CLI #1289
• A Playground server package #1056

Concerns

• Building, testing, and developing with Bun would mean we're not running Playground code on Node.js on a daily basis. Instead, Node would become a build target, an afterthought. How can we ensure the won't start breaking on Node?

Notes

• Bun can produce Node.js-compatible code. Node is a valid build target.
• Bun can only produce ESM. We could either use rollup to also ship CJS version of our modules, or drop CJS support.

Let's use #1056 as an excuse to explore switching to Bun for bundling packages

[adamziel]

cc @bgrgicak @dmsnell @brandonpayton @sejas @wojtekn for context and thoughts.

[dmsnell]

this all makes sense to me; I'm glad to see the exploration going as far as it is.
node has always been a fundamental issue with wp-now and I like having a static binary that doesn't interact with system-installed node versions and npm packages.

[bgrgicak]

After reading the docs a bit Bun sounds like magic especially because it does a lot of things well.

If #1056 is a success let's explore where would we benefit from Bun the most and start replacing these parts.

[brandonpayton]

So far, this looks amazing.

As an aside, I love how the use of Jest-like APIs apparently lets us switch between various test runners.

After reading the docs a bit Bun sounds like magic especially because it does a lot of things well.

Magic is great but always makes me think, "What can we do if the magic fails?". And I guess there is always a last resort of going back to tools that require more config and integration work. It certain seems worth trying to use something that requires much less work and offers the upside of producing standalone native executables.

I took a quick look at the Bun codebase to get a feel for the project, and IMO, it is pretty readable at a glance. The Zig language is interesting. And the project seems disciplined and well thought out.

[adamziel]

I've been playing with Bun today. Here's what I've learned.

Running tests

Bun fails where we use Vitest-specific syntax like vi, vitest.fn(). It supports the corresponding Jest syntax, though, like jest.fn(). I was able to run a lot of Playground spec files with minor or no adjustments just like this:

bun test packages/php-wasm/node/src/test/php.spec.ts

The tests were running fast and Bun provided really good visual feedback throughout the test runs.

That being said, I saw some failures that didn't happen with Node.js. They could be bugs in Bun or pointers towards a deeper portability issue with PHP.wasm (I believe Bun runs WebKit, not v8):

PHP 8.2 > proc_open() > echo "WordPress"; stdin=file (empty), stdout=pipe, stderr=pipe, stream_get_contents [18.03ms]
402 |           headers: PHPResponse['headers'];
403 |           httpStatusCode: number;
404 |   } {
405 |           const headersFilePath = '/internal/headers.json';
406 |           if (!this.fileExists(headersFilePath)) {
407 |                   throw new Error(
               ^
error: SAPI Error: Could not find response headers file.

This log message talks about PHP failure and I assume it's logged by one of our console.error() lines because the test the stacktrace points to actually passed earlier:

✓ PHP 8.3 > Exit codes > After failure, returns the correct exit code on subsequent runs [11.11ms]
... some output ...
error: PHP.run() failed with exit code 255 and the following output: PHP Fatal error:  Uncaught Exception in php-wasm run script:1
Stack trace:
#0 {main}
  thrown in php-wasm run script on line 1

      at new PHPExecutionFailureError (/Users/cloudnik/www/Automattic/core/plugins/playground/packages/php-wasm/universal/src/lib/base-php.ts:39:3)
      at /Users/cloudnik/www/Automattic/core/plugins/playground/packages/php-wasm/universal/src/lib/base-php.ts:293:19
      at /Users/cloudnik/www/Automattic/core/plugins/playground/packages/php-wasm/node/src/test/php.spec.ts:964:10
      at /Users/cloudnik/www/Automattic/core/plugins/playground/packages/php-wasm/node/src/test/php.spec.ts:959:73

I also saw some timeout errors which I fixed by increasing the timeout.

Building

Bun builds extremely fast. I pointed it to the Blueprints package and it gave me a .js output in 22ms – it takes Vite+nx about 5 minutes.

The output file contained all the module dependencies, including code from other packages. This is perfect for the Playground website, but it's not useful for publishing npm packages. We'd have to figure out how to bundle only a single module. The --splitting option didn't work for me, but I sort of got there by explicitly listing all the dependencies as --external – I don't want to maintain that list, though.

Also, Bun produced an ESM output but we're shipping Playground packages also as CJS. We could either transpile that output using another tool, or drop CJS support. The latter will be more difficult as there's plenty of execution context where you need CJS, e.g. the VS code extension AFAIR.

For some inputs, Bun crashed without providing a good error:

$ bun build packages/php-wasm/node/src/index.ts --outdir=logger --target=node          

uh-oh: Internal error: missing asset file
bun will crash now 😭😭😭

----- bun meta -----
Bun v1.1.3 (2615dc74) macOS Silicon 23.2.0
BuildCommand: tsconfig_paths(5) tsconfig(10) bunfig 
Elapsed: 42ms | User: 78ms | Sys: 39ms
RSS: 0.23GB | Peak: 0.23GB | Commit: 0.33GB | Faults: 11
----- bun meta -----

0   0x10519ce34
1   ???
2   ???

Search GitHub issues https://bun.sh/issues or ask for #help in https://bun.sh/discord

thread 6196728 panic: Internal error: missing asset file
[1]    34045 abort      bun build packages/php-wasm/node/src/index.ts --outdir=logger --target=node

We could maybe figure out which asset was missing, but definitely less than ideal. That being said, it's still a far better error message that many messages I saw coming from Vite or its plugins.

[brandonpayton]

@adamziel those are wonderfully detailed notes!

That being said, I saw some failures that didn't happen with Node.js. They could be bugs in Bun or pointers towards a deeper portability issue with PHP.wasm (I believe Bun runs WebKit, not v8):

🤔 It's pretty neat to use different but nearly-equivalent platforms to shake out bugs.

This log message talks about PHP failure and I assume it's logged by one of our console.error() lines because the test the stacktrace points to actually passed earlier:

✓ PHP 8.3 > Exit codes > After failure, returns the correct exit code on subsequent runs [11.11ms]
... some output ...
error: PHP.run() failed with exit code 255 and the following output: PHP Fatal error:  Uncaught Exception in php-wasm run script:1
Stack trace:
#0 {main}
  thrown in php-wasm run script on line 1

It might be related to this line.

await expect(promise1).rejects.toThrow(
	'PHP.run() failed with exit code 255'
);

This may be a known issue since last year:
oven-sh/bun#1546 (comment)
But there is a workaround mentioned in the next comment:
oven-sh/bun#1546 (comment)

For some inputs, Bun crashed without providing a good error:

$ bun build packages/php-wasm/node/src/index.ts --outdir=logger --target=node          

uh-oh: Internal error: missing asset file
bun will crash now 😭😭😭

Unfortunately, there is some indirection going on in Bun which results in that error:
https://github.com/oven-sh/bun/blob/e3689e7e83507ae9c63dac7d785e41884daead66/src/bundler/bundle_v2.zig#L11280-L11282

const files = additional_files[index];
if (!(files.len > 0)) {
    Output.panic("Internal error: missing asset file", .{});
}

So to find out which file, it seems the missing file needs to be noticed earlier in bundling.

[brandonpayton]

A couple other thoughts:

• Since Bun uses JavaScriptCore instead of V8, do we care that it might take longer to get JSPI support?
• Will releasing executables require more vigilance for releasing updates in response to security vulnerabilities?

[adamziel]

Since Bun uses JavaScriptCore instead of V8, do we care that it might take longer to get JSPI support?

It would have to run the Asyncify version until JSPI is more widely supported

Will releasing executables require more vigilance for releasing updates in response to security vulnerabilities?

Do you mean fixes in Bun itself? Most likely, yes. Otherwise that executable is no different than the source code itself, it just happens to bundle the runtime.

[brandonpayton]

Will releasing executables require more vigilance for releasing updates in response to security vulnerabilities?

Do you mean fixes in Bun itself? Most likely, yes. Otherwise that executable is no different than the source code itself, it just happens to bundle the runtime.

I think you're right that it is no different. If we didn't include a package-lock.json, it might be different. In that case, the source code might have a chance of getting security updates for dependencies for new installs, depending on the versions allowed by package.json.