Live programming ideas

[josevalim]

This is a meta issue to describe what we want to implement in relation to live programming:

• Line counting: the idea is to have a counter with how many times a line has been executed in the gutter. This is useful because it gives us immediate feedback on hot-paths, dead code, etc. This will require an extension to Elixir to allow us to annotate expanded code before evaluation (or we add a line callback to Elixir itself)
• Process line coloring: sometimes it is hard to build the intuition which lines of code run in which process. We could use different backgrounds or a gutter to annotate code executed by different processes. This requires pretty much the same infrastructure as line counting
• Variable explorer: this is a feature known in many notebook platforms but those platforms have global state. A variable explorer in Livebook would need to be per cell. This probably requires a side-pane or similar alongside the cell to list variables with previews of their content (perhaps even a good opportunity to add a general search/autocomplete tied to the cell). See previous issue Add a variable explorer #179.
• Module explorer: we can show which modules each section defines under the Sections pane in the sidebar
• Example-live programming: the canonical introduction, "Usable Live Programming", introduces two concepts: probes and traces. In Elixir, we could translate those to doctests and dbg. The idea is to automatically run doctests and have their output in a specific frame with a re-run button.
• dbg time slicing: currently dbg only keeps the first/last value. We want to allow it to register several values but, not only that, have a frame that shows have the values of multiple dbgs changed over time, in order. For example with you have a loop with a nested loop, we should have all dbg in order and control how they were emitted. See proof of concept here: https://szymonkaliski.com/projects/live-coding-livebook

---

References on probing/example-based live programming:

• Usable live programming
• Example-based live programming for everyone: building language-agnostic tools for live programming with LSP and GraalVM
• Log-it: visual debugging (added Apr/2023)

[derpycoder]

I loved Projection-boxes, I had to watch the slowed down version to understand what's going on and it blew my mind.

The approach is awesome, and it's seems like a great tool for live programming.

Going through other links.

[derpycoder]

The Microsoft paper, Usable live programming, is interesting but looks ugly in comparison to Projection Boxes.

I wish there was a tool like, Kernel.dbg, for Recursive code, so I don't have to trace it manually or have a mental model for it.

The paper had links to two YouTube videos, which made some sense compared to the paper itself 😅:
sqrt
fib

[derpycoder]

Example-based live programming for everyone: building language-agnostic tools for live programming with LSP and GraalVM

This is really awesome, I liked the usage of Emoji, which made it lot less confusing than the Projection Boxes method.
I liked how it allows inspection of the Recursive code as well.

It's like the stepping through code and debug is happening simultaneously while coding, so no more separate code, run and debug cycle.

It also has Polyglot thing, so Elixir & Erlang both can benefit I guess.

Elixir will be even more developer friendly.

[derpycoder]

Jupyter Notebook also uses the GraalVM for live feedback: Ipolyglot

[Adzz]

This is a great recent talk in that space too:
https://youtu.be/8Ab3ArE8W3s

[derpycoder]

I liked how he said: "Stuff it in a docker container & then ship it out to the cloud for a CI job, wait for it to compile, why do we tolerate this, Docker shouldn't exist, it exists only because everything else is terribly complicated that they added another layer of complexity to make it work, it's like they thought if deployment is bad, we should make development bad too!"

---

No one will say no to Live Programming.

A self documenting, always running code, that removes the mental burden about the state of the code while writing, it will be a game changer.

I think Elixir will have the best implementation.

[100phlecs]

Here's a related talk that explores "an immediate connection to your creation" with programming. Has some interesting examples.

[derpycoder]

This is a much fleshed out video, with lots of working examples and filled with ideas to the brim, thank you.

It points out all the pain points in programming.

I really liked Swift Playground because of REPL and the immediate feedback while coding.

Mapping between code and the output was next level!! The way he used that magnifying glass and it pointed out which code made that, is something I want badly.

[thelastinuit]

LiveView feels like the perfect tech to do Bret Victor's Media for Thinking the Unthinkable!

[derpycoder]

Yup, @100phlecs also pointed out a talk by him:

Bret Victor - Inventing on Principle

---

It's like Serendipity: "The occurrence and development of events by chance in a happy or beneficial way."

[derpycoder]

@josevalim,

I was blown away by the the demo when Swift was introduced, back when I wanted to make games. The demo felt extraordinary!!

WWDC 2014 - Swift Demo

[RichMorin]

At this point, there are about a dozen code notebooks. Several of these are support multiple programming languages and a couple even support Elixir.

This begs the question "what can (should) Livebook do that the others can't (or won't)". One possible answer is that Livebook can support the growing variety of BEAM-based languages. In particular, it could support Erlang.

I even have a minor use case to suggest for this. There is an Erlang test suite that I'd like to play with. If Livebook supported Erlang, I could incorporate these tests as executable code.

[josevalim]

@RichMorin there has been a discussion for it: #190 - I would suggest moving it there to keep this issue focused.

[hugobarauna]

Related to

Module explorer: we can show which modules each section defines under the Sections pane in the sidebar

It would be useful to have a way to navigate to a module using a keyboard shortcut, similar to what symbol navigation looks like in a code editor.

[bt-maps]

ZeroBrane Studio Lua IDE has some nice live coding features

https://youtu.be/FpxIfCHKGpQ?si=IdG7abN7w5BhEPAp&t=86
http://notebook.kulchenko.com/zerobrane/live-coding-in-lua-bret-victor-style