Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

any plans to support persistence for Browser like leveldb #213

Open
Volland opened this issue Dec 10, 2023 · 5 comments
Open

any plans to support persistence for Browser like leveldb #213

Volland opened this issue Dec 10, 2023 · 5 comments

Comments

@Volland
Copy link

Volland commented Dec 10, 2023

Cozo wasm module is amazing but lacks persistence for in-browser apps .
Support of leveldb or any browser-based persistence could open a lot of usecases

@chuanqisun
Copy link
Contributor

In the interim, I've been using json-based export/import relations against browser's origin private filesystem. It isn't the fastest solution but provides the functionality. Ideally I'd love to have the ability to load and save data in binary.

@simonwh
Copy link

simonwh commented Jan 9, 2024

@chuanqisun We've done something similar, with in-memory cozo + localstorage (instead of origin private filesystem). We find it's working-ish. It's not super stable, and definitely not a long-term solution :) Would be interest to hear how you implemented it. Do you export/write file after every cozo operation? Or periodically? And then import on page load?

@chuanqisun
Copy link
Contributor

chuanqisun commented Jan 10, 2024

@simonwh, my technique isn't ideal either. Exporting on every transaction causes performance issue. My workaround:

  • Run the cozo wasm from a worker thread, so all the database I/O is isolated from the UI thread
  • Export only upon user request, leading to data loss when user closes the app without saving.

In my opinion, a good continuous export implementation is a balancing act between the inevitable tradeoff between performance and data integrity. A few ideas I find useful:

  • Use requestIdleCallback to let the browser pick a best moment to export but that won't work in Safari.
  • Use some throttle and debounce techniques to reduce the export frequency
  • Compare the stringified values and perform the export only when the value changes.

@94bryanr
Copy link

I am also interested in this. A few thoughts:

  • Enabling the sqlite backend for wasm targets may be possible. See this PR to do so in rusqlite as an example.
  • Could there be an addition to the current in-memory API so that you could optionally pass an impl of e.g. std::io::Write as an alternative location to write to? Then persisting to OPFS or localstorage would be relatively easy.

@davidhmays
Copy link

davidhmays commented Apr 26, 2024

I would love browser based persistentence for local first applications.

I wonder if saving to ElectricSQL would be possible, enabling automatic sync to a back end as well?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants