Embedding executable code in LaTeX documents

[nokome]

Over in Encoda @fkmendes and I are working on improving Stencila's support for LaTeX. We have had a LaTeX converter for some time (based on Pandoc) but it had had little attention.

One of the things that we are, of course, wanting to have in LaTeX documents is executable code. So this is just to kick start some discussion on that. In particular, I'm keen to get the input of people like @finlay, @edwardabraham and @richardmansfield (apolgies for the poke :) who have been using LaTeX for executable documents for a long time. I'd love to know what conventions / syntax you all use for executable code within LaTeX.

Using (abusing?) existing LaTeX commands

I've created a text fixture in that PR who's main purpose is to see how we can represent various types of Code nodes from Stencila's Schema in LaTeX (i.e what LaTeX command Pandoc parsers and how it parses them).

See https://github.com/stencila/schema/tree/master/schema for the definitions of Stencila Code nodes. See https://github.com/jgm/pandoc/blob/master/src/Text/Pandoc/Readers/LaTeX.hs for how Pandoc parses LaTeX.

I have looked at the Pandoc JSON output from parsing all the following examples and, based on that, think we can use all of the following LaTeX encodings.

A CodeFragment with no programmingLanguage can be represented using \verb.

Some inline \verb|code|.

A CodeFragment with a programmingLanguage can be represented using \lstinline

Some inline Java \lstinline[language=Java]{public class MyClass}.

A CodeBlock with no programmingLanguage can be represented using a verbatim environment.

\begin{verbatim}
This is like the HTML <pre> element
\end{verbatim}

A CodeBlock with a programmingLanguage can be represented using a lstlisting environment.

\begin{lstlisting}[language=js]
const foo = 'bar'
console.log(foo)
\end{lstlisting}

The above are all "static" code nodes with standard approaches for inclusion in LaTeX. The following a "dynamic" code nodes - where we want to find was to represent them in LaTeX using modifications of syntaxes that Pandoc already knows about.

A CodeExpression could be represented by adding exec to the \lstinline language. (It would be preferable to use a comma between the language and the "exec" (i.e. a different "attr" on the element) but at the time of writing Pandoc v2.12 only seems to pass language and not other attributes.)

An evaluated Python expression \lstinline[language=python exec]{pow(2,2)}.

A CodeChunk could be represented by adding the exec class to \lstlisting language. In this case, Pandoc v2.12 does parse "exec" as a separate attribute on the element (in contrast to lstinline) but for consistency we don't use the comma here.

\begin{lstlisting}[language=r exec]
plot(y~x, data)
\end{lstlisting}

A CodeChunk with id and caption properties can be represented using a lstlisting as well (but might require a second parsing of the caption string)

\begin{lstlisting}[language=r exec, caption={
    A plot of the data which hasd a mean of \lstinline[language=r exec]{ mean(data) }
}, label=fig2]
plot(data)
\end{lstlisting}

Using existing syntax for executable code in LaTeX

noweb is a very old, but seemingly not widely used, syntax for literate programming allowing embedding executable code within LaTeX:

- https://www.cs.tufts.edu/~nr/noweb/
- https://github.com/nrnrnr/noweb
- https://www.cs.tufts.edu/~nr/noweb/johnson-lj.pdf
- https://rpubs.com/YaRrr/SweaveIntro 
- https://stat.ethz.ch/R-manual/R-devel/library/utils/doc/Sweave.pdf

A CodeChunk is represented in noweb like this:

<<>>=
library(foobar)
@

A CodeExpression is equivalent to a "Mini-code chunk" (in Sweave parlance) and designated by \Sexpr{} (for S expression, S being R's precursor of course):

The mean age of participants was \Sexpr{mean(data$age)} and there were \Sexpr{sum(data$sex == “f”)} females

• As a more language agnostic alternative we are also considering supporting the \expr{} command.
• We could allow for different languages withing the same document by using a similar syntax as above: \expr[language=python]
• We could use file name extensions to indicate the default language for the file (so the author does not need to repeat it all the time) e.g. rnw, pynw, jsnw`
• In Encoda, we would create a new nw "codec" that would parse the noweb syntax and convert it into pure LaTeX as defined in the previous section, which would then be parsed by the existing latex codec for conversion to Stencila Schema. This is analogous to how our xmd codec (for Rmarkdown-like files) works.

[fkmendes]

When wanting to display non-executable code, I have also used and seen others just using \texttt{} as a way to highlight that a string is to be read as code (not colorized). But the most obvious, general way seems to be using lstlisting, which I have also used myself.

I guess just to clarify: this particular usage is aimed at people wanting to share a block of code that is not linked to any data (?). Say, maybe a chunk of pseudocode to describe an algorithm, or even language-specific code to just illustrate how a procedure is done (rather than to produce a figure or plot).

As for executable code chunks, it might be also possible to define the global programming language at the preamble of a LaTeX file (which could be genererically named .nw). Whenever a second language is used, then the use specifies it in the <<>>=@ block. (I like the idea of supporting \expr{} by the way.)

[nokome]

A little more googling lead me to python.sty circa 2008 which adds a python environment e.g.

\documentclass{article}
\usepackage{graphicx}
\usepackage{python}
\begin{document}

\begin{figure}
\centering
\begin{python}
#
from pyx import *

g = graph.graphxy(width=8)
g.plot(graph.data.function("y(x)=sin(x)/x", min=-15, max=15))
g.writePDFfile("function")
print r'\includegraphics{function}'
\end{python}
\caption{$y(x)=\frac{\sin(x)}{x}$}
\end{figure}

\end{document}

but more importantly the well documented, and actively developed and distributed pythonlatex:

• https://ctan.org/pkg/pythontex?lang=en
• https://iopscience.iop.org/article/10.1088/1749-4699/8/1/014010/meta
• https://www.12000.org/my_notes/python_in_latex/index.htm

that adds the following inline commands:

• \py{<expression>}: prints value of expression
• \pyc{<code>}: executes code, output goes to STDOUT
• \pys{<code>}: supports variable and expression substitution
• \pyb{<code>}: execute code and pretty print result
• \pyv{<code>}: pretty print code

and the following LaTeX environments

• pycode: executed, but not typeset
• pysub: variable and expression substitution
• pyverbatim: typeset, but not executed
• pyblock: executed and typeset
• pyconsole: simulates an interactive Python-console

Given this prior art, we should support at least some of these (although some variants are nor applicable / redundant given Stencila's execution semantics or can be accommodated via options). For example, \py{expression} could be shorthand \expr{expression} where language is Python.

[rgieseke]

LuaTeX's syntax for embedded Lua might also be worth a look

$2 + 5 \neq \ctxlua{context(3+5)}$, but is equal to \ctxlua{context(2+5)}.
This is \ctxlua{context(string.upper("absolutely"))} true.

\startluacode
-- include the file my-lua-lib.lua
require("my-lua-lib")
\stopluacode

https://wiki.contextgarden.net/Programming_in_LuaTeX

[nokome]

Thanks @rgieseke, that's useful!