continuation parentage and return values are the same thing

[disruptek]

Some definitions we should probably standardize on:

• app -- the "user" code that makes use of continuations and need not understand CPS per se
• library -- the "dispatcher" which is written by someone who understands CPS surface

Consider Zevv's channels:

https://github.com/disruptek/cps/blob/master/stash/channels.nim

proc sink(ch: Channel) {.cps:Cont.} =
  while true:
    ch.recv()
    echo ch.getVal()

What he wants is to

proc sink(ch: Channel) {.cps:Cont.} =
  while true:
    let v = ch.recv()
    echo v

...wherein recv is a CPS call.

We talked about this (what else, it's all talk) and the general take-away is that we should be comfortable with bulking up the responsibilities of the dispatcher provided by the library author, but that the app author should still be able to use CPS without exposing the continuation in the procedure signature or environment.

To restate, CPS should, uh, continue to focus on the most rudimentary elements of control-flow and we can make the library author write .cpsMagic and such in order to access the variable predefined in the continuation type.

The best (read: only) idea for doing this, so far, and without involving the dispatcher, is to have a parent continuation which we will resume when the current one is complete. CPS could probably manage this entirely, but if we instead push it into dispatcher-space, then that might be fine, too.

Pushing it to the dispatcher means a little more boilerplate for the library author, because they need to handle the rendezvous themselves. Leaving it in CPS might mean less implicit overhead but more API surface because dispatchers need to know about unwinding the continuation stack.

@zevv is that about right?

[alaviss]

Is there any reason why we can't just go wait ch.recv() and have wait be implemented by the dispatcher (ie. push all responsibilities to the dispatcher)?

[disruptek]

It can work that way, yes, but along the same lines of not exposing the continuation input parameter and return type in the code the user creates, we might prefer to make assignment work more naturally if possible. It's only the library author that we want to expose the machinery to. This is largely my interpretation of @zevv's position on this, though.

I personally think we should make let x = recv() work because that's what I want to write -- what else?

[zevv]

@disruptek That basically sums it up. I think our conclusion also was that it is pretty much ok if CPS itself is a pain to use - the pain is felt by library implementors only, because end-users will typically not build on top of CPS directly. If this pain leads to greater control and flexibility, it should be worth it.

[disruptek]

CPS can do the parentage dance itself; the dispatcher doesn't need to know about it. This is what you want as a library author because you cannot fathom the garbage your users will write, and you really have nothing to gain from trying to second-guess CPS when it comes to control-flow. That said, the dispatcher can perform parentage hackery via magic/voodoo, so it's just as accessible as ever. This feature necessarily starts with correctly handling the control-flow, then adds in return values, and finally ends with the shims for scoped resolution of the result.

[disruptek]

These are the stages, basically.

• impl spawn
• move spawn into new bootstrap (or be lazy and leave it outside)
• rename cps proc and annotate it
• rewrite cps calls as statements
• rewrite cps calls as expressions
• impl jit-continuation

This is the rationale:

"""
proc foo()
  -> foo_234234(): Continuation
"""
  when cps:
    foo()                # control-flow sugar (we rewrite it)
    echo foo()           # fails sem, what else
    let x = foo()        # fails sem, what else
  else:
    foo()                # we trampoline you in a bootstrap
    spawn bar()          # bootstrap, no trampoline
    resume foo():        # just-in-time continuation
      ...


"""
proc foo(): T
  -> foo_234234(): Continuation
"""
  when cps:
    foo()                # fails sem, what else
    let x = foo()        # control-flow sugar (we rewrite it)
    echo foo()           # control-flow sugar (we rewrite it)
    spawn bar()          # works normally; bar is already sem'd
  else:
    let x = foo()        # we trampoline you in a bootstrap
    spawn bar()          # bootstrap, no trampoline
    resume foo() as x:   # just-in-time continuation
      ...


"combine the two `when cps:` blocks together"
    foo()                # control-flow sugar (we rewrite it)
    echo foo()           # fails sem, what else
    let x = foo()        # fails sem, what else
    foo()                # fails sem, what else
    let x = foo()        # control-flow sugar (we rewrite it)
    echo foo()           # control-flow sugar (we rewrite it)
    spawn bar()          # works normally; bar is already sem'd

"throw out the shit that doesn't work because it didn't pass sem anyway"
    foo()                # control-flow sugar (we rewrite it)
    let x = foo()        # control-flow sugar (we rewrite it)
    echo foo()           # control-flow sugar (we rewrite it)
    spawn bar()          # works normally; bar is already sem'd

[zevv]

For those who do not know what this is about:

https://github.com/disruptek/cps/blob/master/stash/lua_coroutines.nim#L48