- Where: zoom.us
- When: July 18, 16:00-17:00 UTC
- Location: link on calendar invite
- Contact:
- Name: Dan Gohman
- Email: sunfish@mozilla.com
None required if you've attended before. Email Dan Gohman to sign up if it's your first time. The meeting is open to CG members only.
The meeting will be on a zoom.us video conference. Installation is required, see the calendar invite.
-
Opening, welcome and roll call
- Opening of the meeting
- Introduction of attendees
-
Find volunteers for note taking (acting chair to volunteer)
-
Adoption of the agenda
-
Proposals and discussions
- Import names
- WebAssembly/design#1286
- There's a lot of big-picture design work to be done here.
- To unblock modularization and general design work, can we adopt a new temporary scheme, still containing "wasi_unstable"?
- Weak Imports
- IDL
- WatIDL: WebAssembly#64
- What other blockers do we have before we can start designing new "wasi_unstable" APIs?
- Import names
-
Closure
None
None
2019-07-18 WebAssembly CG WASI Subgroup Video Meeting Notes
Agenda: https://github.com/WebAssembly/WASI/blob/master/meetings/2019/WASI-07-18.md
Attendees:
Dan Gohman Martin Becze Mark McCaskey Alex Crichton Andrew Brown Sam Clegg Yury Delendik Arun Purushan Pat Hickey Jacob Gravelle Luke Wagner Till Schneidereit Luke Imhoff
Meeting notes:
Pat - agenda seconded
DG: Import resolution outside of the scope of WASI.
Pick naming convention that involves the name wasi_unstable + a possible additional identifier, to allow us to continue designing APIs.
SC: wasi_unstable implies a bulk rename in the future
DG: agreeing but bringing up additional prefix to clarify. Does anyone have an opinion. Suggested wasi_unstable/<file | network | …>
_ : Is there any reason to use a /
DG: We want to split the existing wasi_unstable into multiple modules, :, $, the specific character doesn’t matter.
DG: is there a problem with /
_ : Agreeing that separator doesn’t matter, but we should move everything into namespaces
DG: it will allow us to start design work on other APIs…
DG: wasi_unstable/identifier is the current proposal. We’ll call that a decision, we can use that to unblock things and start modularizing things. We’ll start queueing up those issues for the next meetings. Let’s have those issues and we’ll start tagging them. Part of that work will be deciding what goes in what modules
DG: next agenda item. Weak imports
SC: I got as far as implementing the custom section and realized that there’s quite a lot of redundancy, we’ll have a global corresponding to each global import. All we need is a way of finding that list of globals, we’ll probably use name mangling, so all we need to do is specify the suffix and the runtime can find all imports matching that pattern. Name mangling just for is_present. It would just be a simple custom section saying “is present”
DG: that strikes me as overkill
SC: (something)… we can find all weak imports by looking for that suffix
SC: someone asked why we weren’t going forward with the official Wasm spec proposal of weak imports
DG: one possibility is that we can take weak imports as being in the same bucket as snowman bindings. It’s a way of describing an interface to Wasm, so maybe we should put it in the snowman bindings custom section. If that’s the case, we can roll it into the snowman bindings proposal. I think it aligns pretty well with the snowman bindings things, because it’s a custom section, and (justification regarding name mangling)
PH: if you have a whole bunch of weak imports, you can map them to the same global if you want to. So that’s an advantage of a custom section
SC: what does that mean, if anyone of them is missing
PH: that’s useful if you’re importing an entire module of things
SC: that’s useful for all-or-none situations
SC: yeah, I like it
DG: alright, so with that, I’ll put that to the group, is this a good way to go forward, put it in a custom section and roll it into snowman bindings?
(Luke allegedly nods head)
SC: I don’t see how it fits into snowman bindings DG: it’s part of a custom section that is the interpretation of the module; it’s not quite the same thing but it’s in the same category. It would be in a different part of the custom section, and eventually things like specifying the entry point
LW: (missed)... this is in the same kind of layer
SC: that makes sense when you put it like that
DG: does anyone want to comment on the issue then? Someone in the WASI repo brought up the question of using a custom section
J: if that is the case then we should put it in the webidl bindings repo
LW: the webidl bindings repo is itself a layer of ….
J: should add assuming it does exist (snowman bindings)
PH: there’s a large overlap in the people working on both, so it’s probably a non-issue
J: do we want to use weak imports for something else? Mentioning it there will get more eyes on it. Weak imports in contexts outside of WASI
…
SC: should I keep going with specing that in the WASI repo?
DG: yeah, for now.. We’ll figure it out as it goes
DG: next agenda item: WAT IDL proposal
M: So where we got stuck last week is with conflating two things, the way watidl was written made the interface an object, that’s a mistake because an interface is a Wasm module. I rewrote it and I’ll push up the changes today. We should take a key from webidl we can add the extra field that this is a method and then we know that the bindings need to bind some sort of context, so I’ll introduce a method field. It’s also important to remember what the whole point of this was which was partially virtualization, ideally we should be able to have an IDL that Wasm binaries can bind to in two different ways, one using aztrack? Data ADT where all the functions are imported, the other way that would be easy to virtualize, I’m also trying to, I wrote up a little doc on how to do virtualization which I’ll throw up today, so that’s where we’re at with watidl.
LW: what does virtualization specifically mean? You’ve requested to import this fn but I’ll give you a virtualized one instead
M: the ability for a Wasm module that a Wasm module can implement a given interface and another module can use that, there’s 2 types of virt. Static: another module importing fd_close, open, sync, which another imports. Dynamic: a module being able to on the fly create a file descriptor or implement it to hand to another module, which is what I meant
LW: do you mean as a reference to a struct that contains fn refs?
M: yeah, exactly. SO in that case, the WASI interface, ref to structs of fns, you would just import types at that point. It would be nice to have an IDL that can describe both ways of interacting.
LW: and who would use the IDL?
M: used to describe interface and then you’d write bindings.
LW: so this would be the interface of WASI?
M: yeah
LW: would this be equivalent to a list of Wasm function signatures that would be allowed to use snowman binding types in their signatures along with module and field name
M: list of functions of types, it’s pretty basic
LW: If we go the snowman bindings format and have a Wasm style and there were fn sigs that could use these types instead of core Wasm types. What if I make a module that just describes the interface… just a subset of the text format.
M: that’s pretty much what this is + the addition of whether a function can be virtualized or not
LW: that has some pretty significant runtime and compile time implications
M: We don’t even have structs or fields, we need partial GC and func ref before we can do dynamic dispatch properly anyways, so we’re looking ahead.
LW: thanks for explaining that
M: it’s the text format - bodies, just types and function signatures
MM: Wasm interfaces are overlapping, we should consolidate these or figure out what’s going on SC: virtualization isn’t needed for what we need right now, so maybe we shouldn’t push on that too hard if we don’t need it right now.
M: agreement/partial agreement
M: I think virt. Is important in a context where you have multiple untrusted modules working together. As WASI is now, it’s generally a single module, they can add object capabilities to each other. In that context it’s not as useful, once we have func refs, …. Then the capab to virt interfaces is more important
TS: I thought that was a different layer, instead of the runtime built-in you use this instead. I don’t see how that would interact with the IDL, can you explain that more?
SC: you’re talking about interposition vs method calls, interposition is like intercepting a method call.
M: Are you all familiar with ADT style? RIght now everything is ADT style, so you can’t really virtualize ADT style. A module can’t really implement those functions, it can’t generate them on the fly. So to be able to generate or implement a file descriptor, that’s /udev/random , so a module can do this and generate a FD on the file when requested by creating a struct and hand it off to the requester, why this matters at an IDL level is, a module may only want to use OO interface and in that scenario, you’d only implement types and the entrypoint fn would only receive capabilities, references to structs which point to functions. One use case for an IDL here is the host system would know that the module wants OO type vs ADT style, does that make sense?
TS: I think so, thanks for the clarification
M: that said, since we don’t have, since it’s not given we don’t have GC or func ref, (func ref?) looks more promising, it might not be worthwhile to worry about this. It might not be worth worrying about this yet and just focus on ADT style
DG: as far as next steps, martin you mentioned that you’ll post an updated version of the proposal?
M: yep
DG: we’ll iterate from there. Anything else we want to cover in this meeting?
M: I also wrote up some stuff about virtualization, should I add that to the repo?
DG: Sure, make a PR and we can discuss it and decide if we want to incorporate it
DG: that’s a good point, virt. Is an interesting enough point that we should document what we’re going to do in this space
DG: next on the agenda, with the theme of setting up the wasi_unstable namespace, what are the blockers that we have before we can split up wasi_unstable into modules and working on them. Having an IDL nicer than a C header file or markdown is good, are there other blockers?
LW: do you think we should hold off until we can make use of reference types and type imports? Or do we want to do it later
DG: I think that’s something we can deal with later. When we have full snowman bindings, we’ll want to convert them into that form, and emulating lower level concepts with our higher level types. We need to figure out what is a file and that doesn’t need to wait for (those things)
TS: maybe we should say that the changes we make going forward should take these concepts into mind, (describes using indirection of fds in a table of anyref to make transition easier)
DG: that seems reasonable and I think that will somewhat naturally fallout given API design. We want a vocab to talk about things until we have an actual IDL, we can do API design with C headers but it’s not ideal, so we should figure out what to do there. MAking it easier to migrate to future bindings systems seems great
TS: two advantages, we can do API design now knowing thta API design can map well, later on we have a straight forward way to make it easy to consume in C/CPP/Rust. Two APIs with the AnyRef being the fundamental and an indice-only one on top of it
DG: how much do we want to do now vs waiting for snowman bindings? If we just design with this in mind, it will be pretty straight forward to retro fit this, that’s my gut feeling here.
TS: All I’m proposing is making that explicit that we want to have this 1-to-1 relationship, making that relationship straight-forward
DG: should we have a document about “How to design a WASI API”
M: that sounds like a good idea
DG: martin or till do you want to start a document like that?
TS: I can start that document by writing what I just said and then we can flesh it out with more content
DG: it can evolve as we get snowman bindings and other tools
LW: it might be good to have a “future intended steps” section. The intent to move to ref types and binding types are only in our own heads and we should document that somewhere
DG: looking for someone to document OCAP vision and put it in a repo, if someone wants to do that, that’d be great. It’s been discussed in various places, but we don’t have a document in the repo describing the plan. IF someone could digest that down and start that, that’s what we’re looking for
LW: I can help there
DG: we have the docs directory in the WASI repo. Alright, I’m trying to push forward to the point where we can do API design. Are there any other blockers? We’ll have a document, (somethinG) in progress, and a plan … for a module naming system, temporary one, pending discussion about import naming schemes. That’s the end of the agenda. Is there anything else?
SC: presumably we’ll need an IDL to header file conversion after, DG: yes (..) + rust interface generation
M: … LW: … TS: s-expressions as type definitions minus the body is the obvious way to define the functions. WHat’s missing is how to fit the binding expressions in there
LW: importantly, binding expressions don’t fit in there, interface is just the types
M: to generate a header file, you’d need the IDL and the bindings and they do need to be separate because different languages want different bindings
LW: I’m not quite sure what you mean by supplying the bindings. You could generate for say, C, there’d be policy choices like what to do with strings, but it’s possible. Sounds like a cool tool though
DG: that sounds like the end of the meeting, see you all in 2 weeks