Wasmer 3 upgrade makes Module caching harder. Any ideas?

[webmaster128]

In Wasmer 2 we were able to build an in-memory Module cache using something like HashMap<[u8; 32], Module> where the key is the checksum of the .wasm file. The use case for doing that is being able to instantiate Wasm files that are used very often very fast in a blockchain setup. Compared to on-disk Module caching it saves us (a) loading from disk and (b) the deserialization into memory. This has shown significant performance improvements.

The following diagram shows the caching strategy:

Highlighted in yellow are the places where we clone the Module in order to be able to use it in different threads and store to other caches.

---

During the upgrade to Wasmer 3 we realized that we have to cache the (Store, Module) pair in order to achieve something similar. However, the Store is not cloneable such that we have to use HashMap::remove to obtain an element from the cache. This also means we can't load modules from cache and then execute them in parallel because they are gone from the cache.

Are there some ideas how to address this problem? Are we doing something conceptually wrong?

Some options I see

1. Wrap the Store in an Arc<Mutex<Store>> and do constant locking/unlocking from many places
2. Give up direct Module caching and cache serialized modules in cache
3. Find a way to clone Artifacts and construct Modules from them using a new Store without going through full artifact serialization
4. ...

But I'm all ear for other suggestions

[theduke]

Can you expand a bit more on your use case?

You can cache Modules just fine.
A Store is the execution context for a concrete instantiation of a module.

Creating new stores from a module should be fast and cheap.

[webmaster128]

So you are saying I can create a Module with one Strore (e.g. via unsafe { Module::deserialize_from_file(store, &file_path) }), put it in the cache and when use a new Store later on? That would be ideal since I can have 1 Store per instance and run the instances in parallel.

I tried this approach now again but it creates a series of low level errors:

panicked at 'unexpected error, RuntimeErr { msg: "Wasmer runtime error: RuntimeError: out of bounds memory access" }', packages/vm/src/cache.rs:1097:18

panicked at 'assertion failed: prev.start > max', ~/.cargo/registry/src/github.com-1ecc6299db9ec823/wasmer-compiler-3.2.1/src/engine/trap/frame_info.rs:224:9

panicked at 'called `Result::unwrap()` on an `Err` value: PoisonError { .. }', ~/.cargo/registry/src/github.com-1ecc6299db9ec823/wasmer-compiler-3.2.1/src/engine/trap/frame_info.rs:220:39

(many of these)

This made me assume you cannot swap out the Store for a given Module when instantiating it.

[webmaster128]

Can you expand a bit more on your use case?

I added a diagram showing our caching strategy to the main post. The primary motivation for the setup is avoiding serializations of Modules if possible.

[syrusakbary]

Hey @webmaster128, you can add the Engine to wasmer-cache now, not the store. So that should make things easier I believe

[webmaster128]

Thank you @syrusakbary.

So the bytes -> Module constructors (deserialize, deserialize_from_file) can take an Engine or new Store. This we already use in the FileSystemCache. But after the Mosule is in memory, the pair (Module, Store) seems to be fixed. I can't clone the Module in a meaningful way because I can't clone or replace the Store.

In Wasmer 2 this was the case already as well but the Store was only needed at instantiation time, not during the runtime of the Instance. Now in Wasmer 3 the single owned Store is a blocker for multiple instances of one Module because I have to pass mutable references all the time.

[webmaster128]

Trying to rephrase the problem:

• Before there was a 1:1 relationship between Module and Store. Wasmer code models this as Module owning a Store and both can be cloned (which I assume still preserves the same instances under the hood). Once instantiated we did not need the Store anymore.
• Now there seems to be a 1:1 relationship between Instance and Store since for basically all operations such as memory access and function calls and mutable Store reference is needed. I could solve my type problems by putting the Store in an Arc<Mutex<Store>> but this does not seem to allow parallel execution then.

This brings us back to an old question: is there a way to do in-memory Artifact caching that does not require the serialization/deserialization step? This would allow us to create a Module and Store on demand (one per instance).

[theduke]

As previously stated:

Module just represents the (compiled) Webassembly code, and is not tied to a store / instance at all.

An Instance / Store pair represent the runtime/execution state of a particular instance.
So each instance needs it's own store.

You need a Module to create a Store, but after that the Module is not tied to the store at all, and can be re-used to crate as many additional stores/instances as you want.

So if you want to cache in memory, then caching Modules is the way to go.

Constructing new stores and instances should be plenty fast enough for most use cases.

If you need every last bit of performance you can also look into reusing instance/store pairs, but then you also accumulate runtime state (memory changes etc), which may or may not be acceptable for your use case.

[webmaster128]

Thank you for elborating on the expected behaviour. This is very helpful.

In https://gist.github.com/webmaster128/3528145a49ec4ff20bafc64336be4e3c I applied the concept to examples in the Wasmer repo and for those simple cases I can confirm that a new store per instance works.

We are now trying to implement the same idea in our codebase and hunt down the remailing memory problems we saw when first trying that.

[jost-s]

@webmaster128

I'm using the exact same method for caching and it works perfectly, efficient and simple implementation.
I'm having trouble using metering though:

Metering::transform_module_info: Attempting to use a `Metering` middleware from multiple modules.

(How) are you handling metering with only one engine?

[webmaster128]

The metering middleware is only needed for compilation. Then it is compiled into the module and just works. We create a new engine every time we compile a contract (called "compiling engine" in our codebase). This compiling engine gets a new metering middleware instance.
https://github.com/CosmWasm/cosmwasm/blob/v1.5.0/packages/vm/src/wasm_backend/engine.rs#L45-L67

Now as discussed in the other ticket, all instances of a module need to be run with the same engine. In our case we run all instances in the process with a single runtime engine. So we take the compiled module and serialize+deserialize it in order to switch from the compiling engine to the runtime engine:
https://github.com/CosmWasm/cosmwasm/blob/v1.5.0/packages/vm/src/cache.rs#L392-L401

[jost-s]

Fantastic, makes sense and it works beautifully. I'm very grateful for your help!

[kwonoj]

We have same issue. SWC used to store Module in-memory then clone it, using simpliest hashmap (https://github.com/swc-project/swc/blob/e9f3371c5ff029555ead23f10039b02830e86c64/crates/swc_plugin_runner/src/cache.rs#L156-L165) but since Wasmer@3, those cloned module explicitly caused unexpected heap_oob from the execution, which is contrary to the statement So if you want to cache in memory, then caching Modules is the way to go.

[webmaster128]

We found this as part of our Wasmer 2 -> 3 upgrade work: #3890. Smells like the same problem you are experiencing.

[kwonoj]

Yes, looks like store was the culprit. We workaround this by storing store and pseudo-clone it by creating new store from the engine: https://github.com/swc-project/swc/blob/ed9d316833dd77c84c2419b9388206109d69679c/crates/swc_plugin_runner/src/cache.rs#L140

And seems it's working so far.