Support Browser

[vincent-herlemont]

• Explore different ways
  • Implement a test using wasmer.js as indicated by @GregoryConrad in his comment.
  • explore no-std way Support no-std #66
  • etc.
• Fix various issues.

[misaim]

Hey!

Thought I'd drop some more info on my use case. I essentially would like to use an embedded in-memory database within WASM (Web Assembly).

While it's possible to use alternatives (i.e Sqlite3 has a WASM build), cross-compatibitlity between C -> WASM and rust -> WASM isn't great due to the ABI, and the fact that we're in a web environment.

native_db looks to solve this problem - a pure rust implementation should have no issues, and as far as I know the only real requirements to compile to wasm are no-std (As browser has no file system support yet).

Happy to help test/implement @vincent-herlemont : If you can get a no-std build let me know and i'll happily put together a WASM example.

[vincent-herlemont]

@misaim Thank you for your message. Currently, Redb has opened support for adding new backends cberner/redb#707, and the reason for this support is specifically linked to WASM with a technology called JavaScript OPFS.

It appears that the API might be present in the web_sys crate here: web_sys::FileSystem

It could be a good path to explore, let's see!

I'll keep you updated on the progress; a PR should be coming soon.

[vincent-herlemont]

Persistence and Browser

Currently, I am unsure how to correctly integrate support for persistence in browsers.

The challenge is that a synchronous interface is needed for the redb/StorageBackend trait, and most JavaScript interactions occur asynchronously. So far, I have not found any solution other than resolving futures via wasm_bindgen_futures::spawn_local.

I have explored how SQLite is implemented in WebAssembly. A potential solution could involve using a proxy like sqlite3-opfs-async-proxy.js that could bypass JavaScript's asynchronicity. However, this requires substantial development and seems more like a workaround than a long-term, maintainable solution.

Related information sqlite/wasm/building.

Here's a summary of the implementations source: The Current State of SQLite Persistence on the Web

Implementation	Storage	Asyncify?	In main page?	In Web worker?	Concurrent reads?	File system transparency?	Needs COOP and COEP headers?	Chrome (1)	Safari	Firefox
sqlite-wasm - opfs	OPFS	No	No	Yes	No	Yes	Yes	102+	17+	111+
sqlite-wasm - opfs-sahpool	OPFS	No	No	Yes	No	No	No	108+	16.4+	111+
wa-sqlite - IDBBatchAtomicVFS	IndexedDB	Yes	Yes	Yes	Yes	No	No	69+	15.4+	96+
wa-sqlite - OPFS	OPFS	Yes	No	Yes	Yes	Yes	No	102+	15.4+	111+
wa-sqlite - AccessHandlePoolVFS	OPFS	No	No	Yes	No	No	No	108+	16.4+	111+
sql.js - absurd-sql	IndexedDB	No	No	Yes	Not confirmed	No	Yes	91+	15.2+	79+

---

PS: Hello @DouglasDwyer, do you by any chance have any advice since you started this job with cberner/redb#707?

[DouglasDwyer]

Hey @vincent-herlemont, I haven't gotten around to implementing a web backend for redb yet. But my personal plan was to use wasm_thread to spawn a web worker, and then ferry all database reads/writes to the web worker using a spinlock. The web worker would use the synchronous OPFS API to read and write to a JavaScript file. I think that it is the best approach and it ensures that the API remains synchronous. Let me know if you have interest in such a backend. Due to the fact that the solution is a bit cursed I was not planning on open-sourcing it (just using it in my own work) but I can publish it if there is interest when I code it.

[vincent-herlemont]

Thank @DouglasDwyer. I wasn't familiar with the wasm_thread library. I like your approach, and I imagine you could solve the entire problem using Rust. In any case, I would be very happy to see your work, and if you would allow me to draw inspiration from it and reference it for an implementation in native_db.

On my side, I will try to make a draft in the coming months if I have some time. You can follow the progress in this issue.

[GregoryConrad]

I think that it is the best approach and it ensures that the API remains synchronous.

just my two cents: if you compile to WASI you can just use a WASI-JS polyfill and get browser support with the sync/async issue taken care of for you and not need something like a spinlock. However, that also:

• won’t be pure-Rust (your website would require some brief JS to load the WASI module)
• May require nightly (not 100% sure on this one)

That being said, please @ me in any issue/PR you may make with regard to web support here or in redb! I could likely benefit from it over in Mimir