Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Proposal] Adding an optional serialization dependency. #519

Open
limarta opened this issue Feb 16, 2024 · 0 comments
Open

[Proposal] Adding an optional serialization dependency. #519

limarta opened this issue Feb 16, 2024 · 0 comments

Comments

@limarta
Copy link
Contributor

limarta commented Feb 16, 2024

GenSerialization.jl serves to serialize the common trace types offered by Gen.

Why?
Direct serialization using Julia's native serialize function does not properly save ephemeral (e.g. generative function pointers) as pointed out in #129. Users who intend to save out traces for long term storage will necessarily restart the Julia runtime, so there is no guarantee that runtime data will properly be read back in. Given a saved out trace, there are two ways to read data back in. The first requires the generative function to reconstruct the function pointers, and #369 takes a similar strategy and implicitly invokes the generative function while reading. The second amortizes the generative function call over calls to the GFI such as update and regenerate. In the package, we distinguish the two by realize and deserialize respectively.

Goals:

  • Provide out of the box serialization for Dynamic, Static, and combinator traces.
  • Support programs with untraced randomness.

Caveats:

  • Currently supports Dynamic and Map traces. The remaining combinators aren't yet supported, but they are essentially copies of Map.
  • The format, which is opaque to the user, assumes the machine used to read a trace has the same endianness as the machine used to write the trace.
  • This is a Julia-only solution, so it does not provide support to serialize between different Gen implementations.

WIP

  • Finish support of combinators
  • Finish Static DSL traces.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant