IT Mini-MVP?

[aardappel]

I see IT as crucial puzzle piece for the Wasm eco system, and I keep seeing new proposals and ideas that could benefit from it being available. Yet at the same time it seems to become a more and more complex beast (usefully so, in the long run) that is going to take much longer to land.

What are thoughts on cutting off the absolute simplest slice of this system (with maybe just "unicode strings" and "byte buffers"), with very limited tooling, limited adapter functions etc such that this can get started to be put to use, and avoid less well thought out solutions to proliferate? Maybe something like a usable proof of concept that will attract more people to want to push it further?

[lukewagner]

Great question! I totally feel you on wanting to get something out the door so we can start iterating and getting feedback.

First, three observations:

1. This proposal is kindof weird in that it's not clearly derived from some preexisting feature elsewhere. Consequently, the proposal has morphed significantly as we've collected use cases and, more recently, as we've worked through the details of all the value types. Even just recently, working through variants, strings-as-list-abbreviations and destructors (in the most recent PR) has led to significant changes. And I fully expect that, in the process of working through the remaining TODOs), things will morph further. Thus, I'm a little scared to ship anything until we have this clique of features worked out and tried out, b/c the probability of painting ourselves into a corner is higher than usual.
2. Using a adapter fusion, it should be natural to create a polyfill that compiles a linked DAG of adapter modules into core wasm + JS glue code. It wouldn't derive all the optimization benefits partially motivating Interface Types, but it'll still be comparable to current glue-code techniques so I think build-time-polyfilling will be a reasonable way for devs to start using this feature (for its non-performance benefits) well ahead of native engine implementations (in the same way as JS features get used ahead of engine support via tools like Babel).
3. I expect the work required to support just a "Mini-MVP" is not significantly less than that needed to support the full MVP as proposed (even with the TODOs). Once you have all the linking and fusing infrastructure, the extra types/instructions should be just a few more cases in the compiler. An exception may be the JS/WebIDL binding: it's possible supporting the full set of types will require non-trivial work beyond an Mini-MVP set and so it might make sense to initially only support a subset at JS/WebIDL boundaries (as we did with i64-vs-i64-to-BigInt).

Putting these together, I think the right next step (in parallel to fleshing out the aforementioned TODOs) is to start work on a tool for doing the fusion and (optionally, for JS hosts) JS glue code generation. With such a tool, there wouldn't need to be a hard Mini-MVP release cut but, rather, an incremental set of improvements to the tool over time that devs could pick up incrementally.

The prereq for doing that is to pick a tentative binary encoding to bootstrap the producers/consumers. And to do that we need a tentative binary encoding for Module Linking (upon which Interface Types is (proposed to be) based), which @alexcrichton has, thankfully, done. I think @eqrion may have a draft of a binary format for interface types in some state (but maybe pre-linking-rebase?)?

[fgmccabe]

Concur!
I would say that the current PR does represent a mini-MVP anyway.

[aardappel]

@lukewagner thanks for the perspective. I can see that its a tough balance, and had we moved forward earlier then a lot of these improvements would now be a lot harder.

[tlively]

I think the right next step (in parallel to fleshing out the aforementioned TODOs) is to start work on a tool for doing the fusion and (optionally, for JS hosts) JS glue code generation.

Out of curiosity, has anyone started working on such a tool or announced plans to do so?

[alexcrichton]

I've recently started working on this, and my goal is basically as @lukewagner gisted above. My intention is that the end-state is that you use *.witx files to describe functions (both imported and exported) and the same files are used for:

• Generating glue code in the wasm module itself for, say, importing wasi
• Generating glue code in the wasm module for exporting signatures of the right type
• Generating glue code on a host (e.g. JS) for hooking up imports to JS types
• Generating glue code on a host (doesn't have to be JS, could be something like Wasmtime) for calling exports with the "correct type"

This started in WebAssembly/WASI#395 but still has some distance to go. I'm talking with others still about particular type representations and the grammar of possible types, and I think there's still more witx changes to come, but I am at least very actively working on this right now!

[tlively]

That's super cool, thanks for the update @alexcrichton!

[Hywan]

@alexcrichton Thanks for tackling this. Can we help? I've dedicated time to work on WIT.

[alexcrichton]

Unfortunately it's still relatively early so there's a lot of churn, but I'll try make sure to post here when I've got something concrete to show.

[J0eCool]

For what it's worth the work I did at attempting to polyfill this all lives over at https://github.com/jgravelle-google/it-tools

An interesting result of that which had a similar effect to prototyping lift+lower combining was the IT->wasm polyfill, for use in non-JS environments https://github.com/jgravelle-google/it-tools/tree/master/examples/05%20-%20Self%20Hosting%20(generated)

Not sure how bitrotted it is by this point, but figured I should leave a pointer to it just in case