Grabbing multiple outputs in swank:eval-and-grab-output

[akater]

The suggestion had previously been discussed in PR #551 but I proposed an incompatible change then. Patches suggested here ensure everything is backwards-compatible.

swank:eval-and-grab-output now returns alist with many possible outputs, if they were requested in the call. (Return values are provided in that alist too.) Outputs are requested in an optional argument. If optional argument is not supplied, we infer that caller uses old interface and return standard output and values as they are returned now. It thus doesn't break anything.

The check for old interface can certainly be dropped when Emacs 27 is not supported anymore, and can very likely be dropped when Org-mode adopts new interface (Org-mode is likely the only user of swank:eval-and-grab-output in the present day Emacs; I did check). I have a corresponding patch ready for Org-mode but it's difficult to test until SLIME merges this.

My main motivation was to make traces and errors accessible to Org-Babel. I use the feature very often (mostly traces).

My Org-mode patch also ensures backwards compatibility with stable SLIME releases. I presume this feature can be used since SLIME 2.25. Please correct me on this if needed.

The only Elisp functions affected are slime-eval-print and slime-eval-save. They behave exactly as they do now.

[akater]

One question from @luismbo I left unanswered concerns *debug-io*. I did not include support for bidirectional streams as there is no way that I know of to test this feature applied to such streams, in contrast to output streams. E.g. org-mode does not support any kind of interactive evaluation at the moment.

[akater]

Could we return to this please? Everything is backward-compatible now, and the feature needs to be merged into slime first, then into Org.

[luismbo]

@akater I have some refactoring suggestions, but before getting to that, I'm wondering to you think about my question regarding interleaving output in #551 (comment)

[akater]

Yes, it makes sense. However, it should not happen in `eval-and-grab-output`: it should be up to higher level interfaces how to combine the outputs. I guess elisp's `slime-eval-save` does count as such but I don't use it so I hold no opinions on what it should do. I only have a clear vision for the interface of `eval-and-grab-output`.

[luismbo]

@akater but if eval-and-grab-output segregates output per stream and doesn't store the order then higher level interfaces won't be able to guess the order.

[akater]

Let's provide an additional combined stream with SLIME's custom tag, e.g. `slime:debug-and-trace-output`, — but keep the original ones. Having use cases from my practice in mind, I would not want to see anything irregular in trace output simply because it could break structural editing modes like paredit or lispy.

[luismbo]

@akater can you give me an example where's you'd prefer to see output segregated by stream rather than interleaved?

[akater]

An example with trace-output: https://gitlab.com/akater/cl-serere/-/blob/master/serere.org#L1506 A general (non-trace) example would be literally any block in that repository.

[luismbo]

That example only has trace output, it doesn't show output from other streams does it?

[akater]

Exactly. Looks like you expected a more elaborate answer but I don't know what else to say — it only exemplifies my point. I evaluate that form to get the trace.

[luismbo]

@akater What I mean is, given

(defun foo (x)
  (print x))

(trace foo)

what output do you expect to grab with swank:eval-and-grab-output? And assuming you'd want to grab both standard output and trace output, what do you expect the result to look like?

[akater]

given [...] what output do you expect to grab with swank:eval-and-grab-output?

All of it. swank:eval-and-grab-output should always grab all output.

what do you expect the result to look like?

It depends on :results header-arg in Org. Usually it'd be :results ‹kind›, and contents of the corresponding stream would be inserted into Org buffer as usual. So far I don't have any support for emitting multiple results, mostly because I'd like to take small steps and get feedback from practitioners. One variation I have in mind is

#+begin_src lisp :results output trace :wrap example
(f x)
#+end_src

#+RESULTS:
#+begin_example fundamental
this came from standard output
#+end_example
#+begin_example lisp
0: (F X)
  1: (G X Y)
  (G X Y) returned NIL
(F X) returned NIL
#+end_example

but it presumes some guessing by org-mode so I'm not certain. I'd rather discuss this in emacs-orgmode mailing list, and I'd rather do it when diverse multiple stream contents are actually available in org-mode to combine.