[Feature Request] PlutoObservable - for intermediate outputs

[schlichtanders]

Motivation

I just thought about how Optimizations.jl and callbacks could be used inside Pluto and realized that this is difficult.

Concretely is this an example of a loop during which updates to a plot should be made automatically. In Jupyter some update mechanism like this is possible as far as I remember.

Options

I see two ways for Pluto to support this (if it is not already supported):

1. have plots which rather work like a Ref in that they can be updated from other cells without actually triggering Pluto's reactivity. I think something like this is anyway needed to support Makie.jl
2. have a special function put_pluto_intermediate_output(...) or similar which can be called inside the loop to make the frontend already render an output despite the cell is not yet done with its computation.

I find 2. cool, a bit more Plutonic than 1., but think that 1. is ultimately more powerful and will win.

The simplest way for 1. would actually be to support a kind of PlutoObservable wrapper, which will be rerendered the wrapped element (e.g. a plot) changed. I like it.

Other considered alternatives

Note that this is currently not implementable via PlutoHooks, as they simply put a cellid onto the stack of future cellid runs. Hence this wouldn't work for intermediate outputs, as the current cell is still running.

Call for action

What do others think?

[schlichtanders]

Actually I think Hooks are quite good for this. It is just that the loop itself needs to go inside a BackgroundTask so that Pluto's cell execution is not blocked.

(This should also be a way to deal with Makie interactivity)

[schlichtanders]

Unfortunately this approach only works from Julia, as PythonCall/RCall segfault in such a multi task setup...
It speaks in favor of option 2. put_pluto_intermediate_output(...) so that there is a way to do this from PythonCall, too.

I know this more like my personal problem ;-), but at least it also happens in current Pluto if someone uses PythonCall respectively.

[disberd]

I think you could potentially use the functionality added in #2726 for this? (Still not on any released Pluto version, but will likely appear as experimental feature in the next release)

[schlichtanders]

This looks great!