filters_pandoc incompatible with panflute >= 2.0.3 --- TypeError: 'Caption' object is not iterable

[phelps-sg]

Bug Report

Describe the bug

The panflute filters implemented in filters_pandoc seem to assume that panflute version 1.x is installed. It looks as if panflute versions 2.x introduced a new class Caption to represent table captions, whereas in previous versions table captions were represented as lists. The ipypublish filters treat captions as lists and attempt to use e.g. iteration and subscripting directly on captions, resulting in

TypeError: 'Caption' object is not iterable

see below for full trace.

To Reproduce

Steps to reproduce the behavior:

1. Set up the conda environment below noting panflute 2.0.3.
2. Convert notebook(s) containing tables using nbconvert.

Runtime Information

  - _libgcc_mutex=0.1=main
  - alabaster=0.7.12=pyhd3eb1b0_0
  - async_generator=1.10=pyhd3eb1b0_0
  - attrs=20.3.0=pyhd3eb1b0_0
  - babel=2.9.0=pyhd3eb1b0_0
  - backcall=0.2.0=pyhd3eb1b0_0
  - blas=1.0=mkl
  - bleach=3.3.0=pyhd3eb1b0_0
  - brotlipy=0.7.0=py37h27cfd23_1003
  - ca-certificates=2021.4.13=h06a4308_1
  - certifi=2020.12.5=py37h06a4308_0
  - cffi=1.14.5=py37h261ae71_0
  - chardet=4.0.0=py37h06a4308_1003
  - colorama=0.4.4=pyhd3eb1b0_0
  - cryptography=3.4.7=py37hd23ed53_0
  - cycler=0.10.0=py37_0
  - dataclasses=0.8=pyh6d0b6a4_7
  - dbus=1.13.18=hb2f20db_0
  - decorator=5.0.6=pyhd3eb1b0_0
  - defusedxml=0.7.1=pyhd3eb1b0_0
  - docutils=0.17=py37h06a4308_1
  - entrypoints=0.3=py37_0
  - expat=2.3.0=h2531618_2
  - fontconfig=2.13.1=h6c09931_0
  - freetype=2.10.4=h5ab3b9f_0
  - future=0.18.2=py37_1
  - glib=2.68.1=h36276a3_0
  - greenlet=1.0.0=py37h2531618_2
  - gst-plugins-base=1.14.0=h8213a91_2
  - gstreamer=1.14.0=h28cd5cc_2
  - icu=58.2=he6710b0_3
  - idna=2.10=pyhd3eb1b0_0
  - ijson=3.1.4=pyhd3eb1b0_0
  - imagesize=1.2.0=pyhd3eb1b0_0
  - importlib-metadata=3.10.0=py37h06a4308_0
  - importlib_metadata=3.10.0=hd3eb1b0_0
  - importlib_resources=5.1.2=py37h06a4308_1
  - intel-openmp=2021.2.0=h06a4308_610
  - ipykernel=5.3.4=py37h5ca1d4c_0
  - ipython=7.22.0=py37hb070fc8_0
  - ipython_genutils=0.2.0=pyhd3eb1b0_1
  - ipywidgets=7.6.3=pyhd3eb1b0_1
  - jedi=0.17.0=py37_0
  - jinja2=2.11.3=pyhd3eb1b0_0
  - jpeg=9b=h024ee3a_2
  - jsonextended=0.7.11=py_0
  - jsonschema=3.2.0=py_2
  - jupyter=1.0.0=py37_7
  - jupyter_client=6.1.12=pyhd3eb1b0_0
  - jupyter_console=6.4.0=pyhd3eb1b0_0
  - jupyter_contrib_core=0.3.3=py_2
  - jupyter_contrib_nbextensions=0.5.1=pyhd8ed1ab_2
  - jupyter_core=4.7.1=py37h06a4308_0
  - jupyter_highlight_selected_word=0.2.0=py37h89c1867_1002
  - jupyter_latex_envs=1.4.6=pyhd8ed1ab_1002
  - jupyter_nbextensions_configurator=0.4.1=pyhd8ed1ab_2
  - jupyterlab_pygments=0.1.2=py_0
  - jupyterlab_widgets=1.0.0=pyhd3eb1b0_1
  - jupytext=1.11.2=pyh6c4a22f_0
  - kiwisolver=1.3.1=py37h2531618_0
  - krb5=1.17.1=h173b8e3_0
  - latexcodec=2.0.1=pyh9f0ad1d_0
  - lcms2=2.12=h3be6417_0
  - ld_impl_linux-64=2.33.1=h53a641e_7
  - libedit=3.1.20210216=h27cfd23_1
  - libffi=3.3=he6710b0_2
  - libgcc-ng=9.1.0=hdf63c60_0
  - libgfortran-ng=7.3.0=hdf63c60_0
  - libpng=1.6.37=hbc83047_0
  - libpq=12.2=h20c2e04_0
  - libsodium=1.0.18=h7b6447c_0
  - libstdcxx-ng=9.1.0=hdf63c60_0
  - libtiff=4.1.0=h2733197_1
  - libuuid=1.0.3=h1bed415_2
  - libxcb=1.14=h7b6447c_0
  - libxml2=2.9.10=hb55368b_3
  - libxslt=1.1.34=hc22bd24_0
  - lxml=4.6.3=py37h9120a33_0
  - lz4-c=1.9.3=h2531618_0
  - markdown-it-py=1.1.0=pyhd8ed1ab_0
  - markupsafe=1.1.1=py37h14c3975_1
  - matplotlib=3.3.4=py37h06a4308_0
  - matplotlib-base=3.3.4=py37h62a2d02_0
  - mistune=0.8.4=py37h14c3975_1001
  - mkl=2021.2.0=h06a4308_296
  - mkl-service=2.3.0=py37h27cfd23_1
  - mkl_fft=1.3.0=py37h42c9631_2
  - mkl_random=1.2.1=py37ha9443f7_2
  - nbconvert=5.6.1=py37_1
  - nbformat=5.1.3=pyhd3eb1b0_0
  - ncurses=6.2=he6710b0_1
  - nest-asyncio=1.5.1=pyhd3eb1b0_0
  - notebook=6.0.3=py37_0
  - numpy=1.20.1=py37h93e21f0_0
  - numpy-base=1.20.1=py37h7d8b39e_0
  - olefile=0.46=py_0
  - openssl=1.1.1k=h27cfd23_0
  - ordered-set=4.0.2=py_0
  - packaging=20.9=pyhd3eb1b0_0
  - pandas=1.2.4=py37h2531618_0
  - pandoc=2.11.2=h36c2ea0_0
  - pandocfilters=1.4.3=py37h06a4308_1
  - panflute=2.0.5=pyhd8ed1ab_1
  - parso=0.8.2=pyhd3eb1b0_0
  - pathlib2=2.3.5=py37h06a4308_2
  - pcre=8.44=he6710b0_0
  - pexpect=4.8.0=pyhd3eb1b0_3
  - pickleshare=0.7.5=pyhd3eb1b0_1003
  - pillow=8.2.0=py37he98fc37_0
  - pint=0.17=pyhd8ed1ab_0
  - pip=21.0.1=py37h06a4308_0
  - prometheus_client=0.10.1=pyhd3eb1b0_0
  - prompt-toolkit=3.0.17=pyh06a4308_0
  - prompt_toolkit=3.0.17=hd3eb1b0_0
  - psycopg2=2.8.6=py37h3c74f83_1
  - ptyprocess=0.7.0=pyhd3eb1b0_2
  - pybtex=0.24.0=py37h89c1867_0
  - pybtex-docutils=1.0.0=py37h89c1867_0
  - pycparser=2.20=py_2
  - pygments=2.8.1=pyhd3eb1b0_0
  - pyopenssl=20.0.1=pyhd3eb1b0_1
  - pyparsing=2.4.7=pyhd3eb1b0_0
  - pyqt=5.9.2=py37h05f1152_2
  - pyrsistent=0.17.3=py37h7b6447c_0
  - pysocks=1.7.1=py37_1
  - python=3.7.10=hdb3f193_0
  - python-dateutil=2.8.1=pyhd3eb1b0_0
  - python_abi=3.7=1_cp37m
  - pytz=2021.1=pyhd3eb1b0_0
  - pyzmq=20.0.0=py37h2531618_1
  - qt=5.9.7=h5867ecd_1
  - qtconsole=5.0.3=pyhd3eb1b0_0
  - qtpy=1.9.0=py_0
  - readline=8.1=h27cfd23_0
  - requests=2.25.1=pyhd3eb1b0_0
  - ruamel=1.0=py37_2
  - ruamel.yaml=0.16.5=py37h516909a_0
  - ruamel.yaml.clib=0.2.2=py37h7b6447c_0
  - scipy=1.6.2=py37had2a1c9_1
  - send2trash=1.5.0=pyhd3eb1b0_1
  - setuptools=52.0.0=py37h06a4308_0
  - shutilwhich=1.1.0=py_1
  - sip=4.19.8=py37hf484d3e_0
  - six=1.15.0=py37h06a4308_0
  - snowballstemmer=2.1.0=pyhd3eb1b0_0
  - sphinx=4.0.1=pyhd3eb1b0_0
  - sphinxcontrib-applehelp=1.0.2=pyhd3eb1b0_0
  - sphinxcontrib-bibtex=2.2.0=pyhd8ed1ab_0
  - sphinxcontrib-devhelp=1.0.2=pyhd3eb1b0_0
  - sphinxcontrib-htmlhelp=1.0.3=pyhd3eb1b0_0
  - sphinxcontrib-jsmath=1.0.1=pyhd3eb1b0_0
  - sphinxcontrib-qthelp=1.0.3=pyhd3eb1b0_0
  - sphinxcontrib-serializinghtml=1.1.4=pyhd3eb1b0_0
  - sqlalchemy=1.4.7=py37h27cfd23_0
  - sqlite=3.35.4=hdfb4753_0
  - terminado=0.9.4=py37h06a4308_0
  - testpath=0.4.4=pyhd3eb1b0_0
  - texsoup=0.1.4=py_0
  - tk=8.6.10=hbc83047_0
  - toml=0.10.2=pyhd3eb1b0_0
  - tornado=6.1=py37h27cfd23_0
  - traitlets=5.0.5=pyhd3eb1b0_0
  - typing_extensions=3.7.4.3=pyha847dfd_0
  - urllib3=1.26.4=pyhd3eb1b0_0
  - wcwidth=0.2.5=py_0
  - webencodings=0.5.1=py37_1
  - wheel=0.36.2=pyhd3eb1b0_0
  - widgetsnbextension=3.5.1=py37_0
  - xz=5.2.5=h7b6447c_0
  - yaml=0.2.5=h7b6447c_0
  - zeromq=4.3.4=h2531618_0
  - zipp=3.4.1=pyhd3eb1b0_0
  - zlib=1.2.11=h7b6447c_3
  - zstd=1.4.9=haebb681_0

Additional context

Add any other context about the problem here.

Logging

INFO:ipypublish:finding conversion configuration: latex_ipypublish_main
INFO:ipypublish:loading conversion configuration
INFO:ipypublish:creating exporter
INFO:ipypublish:creating template and loading filters
DEBUG:template:overwrite keys: []
DEBUG:template:overwrite keys: []
DEBUG:template:overwrite keys: []
DEBUG:template:overwrite keys: []
DEBUG:template:overwrite keys: []
DEBUG:template:overwrite keys: []
DEBUG:template:overwrite keys: []
DEBUG:template:overwrite keys: []
INFO:ipypublish:creating process configuration
INFO:ipypublish:running nbconvert
DEBUG:traitlets:Applying preprocessor: TagRemovePreprocessor
DEBUG:traitlets:Applying preprocessor: RegexRemovePreprocessor
DEBUG:traitlets:Applying preprocessor: coalesce_streams
DEBUG:traitlets:Applying preprocessor: SVG2PDFPreprocessor
DEBUG:traitlets:Applying preprocessor: LatexPreprocessor
DEBUG:traitlets:Applying preprocessor: HighlightMagicsPreprocessor
DEBUG:traitlets:Applying preprocessor: ExtractOutputPreprocessor
DEBUG:traitlets:Applying preprocessor: SplitOutputs
INFO:root:splitting outputs into separate cells
DEBUG:traitlets:Applying preprocessor: LatexDocLinks
INFO:resolve_links:resolving external file paths in ipub metadata to: slides/all/1_Overview_of_Python.ipynb
DEBUG:traitlets:Applying preprocessor: LatexCaptions
INFO:captions:extracting caption cells
DEBUG:traitlets:Attempting to load template template_file
DEBUG:traitlets:    template_path: .
Run Failed: 'Caption' object is not iterable
Traceback (most recent call last):
  File "/home/sphelps/anaconda3/envs/fitch-python/bin/nbpublish", line 11, in <module>
    sys.exit(run())
  File "/home/sphelps/anaconda3/envs/fitch-python/lib/python3.7/site-packages/ipypublish/frontend/nbpublish.py", line 100, in run
    outcode = nbpublish(filepath, **options)
  File "/home/sphelps/anaconda3/envs/fitch-python/lib/python3.7/site-packages/ipypublish/frontend/nbpublish.py", line 83, in nbpublish
    publish(ipynb_path)
  File "/home/sphelps/anaconda3/envs/fitch-python/lib/python3.7/site-packages/ipypublish/convert/main.py", line 354, in __call__
    return self.publish(ipynb_path, nb_node)
  File "/home/sphelps/anaconda3/envs/fitch-python/lib/python3.7/site-packages/ipypublish/convert/main.py", line 487, in publish
    final_nb, exporter_cls, econfig, jinja_template
  File "/home/sphelps/anaconda3/envs/fitch-python/lib/python3.7/site-packages/ipypublish/convert/main.py", line 615, in export_notebook
    body, resources = exporter.from_notebook_node(final_nb)
  File "/home/sphelps/anaconda3/envs/fitch-python/lib/python3.7/site-packages/nbconvert/exporters/latex.py", line 88, in from_notebook_node
    return super(LatexExporter, self).from_notebook_node(nb, resources, **kw)
  File "/home/sphelps/anaconda3/envs/fitch-python/lib/python3.7/site-packages/nbconvert/exporters/templateexporter.py", line 322, in from_notebook_node
    output = self.template.render(nb=nb_copy, resources=resources)
  File "/home/sphelps/anaconda3/envs/fitch-python/lib/python3.7/site-packages/jinja2/environment.py", line 1090, in render
    self.environment.handle_exception()
  File "/home/sphelps/anaconda3/envs/fitch-python/lib/python3.7/site-packages/jinja2/environment.py", line 832, in handle_exception
    reraise(*rewrite_traceback_stack(source=source))
  File "/home/sphelps/anaconda3/envs/fitch-python/lib/python3.7/site-packages/jinja2/_compat.py", line 28, in reraise
    raise value.with_traceback(tb)
  File "<template>", line 919, in top-level template code
  File "/home/sphelps/anaconda3/envs/fitch-python/lib/python3.7/site-packages/nbconvert/exporters/../templates/latex/skeleton/display_priority.tplx", line 5, in top-level template code
    ((*- extends 'null.tplx' -*))
  File "/home/sphelps/anaconda3/envs/fitch-python/lib/python3.7/site-packages/nbconvert/exporters/../templates/latex/skeleton/null.tplx", line 30, in top-level template code
    ((*- block body -*))
  File "<template>", line 575, in block "body"
  File "/home/sphelps/anaconda3/envs/fitch-python/lib/python3.7/site-packages/nbconvert/exporters/../templates/latex/skeleton/null.tplx", line 32, in block "body"
    ((*- block any_cell scoped -*))
  File "<template>", line 624, in block "any_cell"
  File "/home/sphelps/anaconda3/envs/fitch-python/lib/python3.7/site-packages/nbconvert/exporters/../templates/latex/skeleton/null.tplx", line 85, in block "any_cell"
    ((*- block markdowncell scoped-*)) ((*- endblock markdowncell -*))
  File "<template>", line 651, in block "markdowncell"
  File "/home/sphelps/anaconda3/envs/fitch-python/lib/python3.7/site-packages/ipypublish/filters_pandoc/main.py", line 214, in jinja_filter
    strip_meta=bool(strip_meta),
  File "/home/sphelps/anaconda3/envs/fitch-python/lib/python3.7/site-packages/ipypublish/filters_pandoc/utils.py", line 114, in apply_filter
    out_doc = func(out_doc, **kwargs)  # type: Doc
  File "/home/sphelps/anaconda3/envs/fitch-python/lib/python3.7/site-packages/ipypublish/filters_pandoc/prepare_cites.py", line 131, in main
    return pf.run_filter(process_citations, prepare, finalize, doc=doc)
  File "/home/sphelps/anaconda3/envs/fitch-python/lib/python3.7/site-packages/panflute/io.py", line 224, in run_filter
    return run_filters([action], *args, **kwargs)
  File "/home/sphelps/anaconda3/envs/fitch-python/lib/python3.7/site-packages/panflute/io.py", line 205, in run_filters
    doc = doc.walk(action, doc)
  File "/home/sphelps/anaconda3/envs/fitch-python/lib/python3.7/site-packages/panflute/base.py", line 264, in walk
    ans = list(chain.from_iterable(ans))
  File "/home/sphelps/anaconda3/envs/fitch-python/lib/python3.7/site-packages/panflute/base.py", line 262, in <genexpr>
    ans = ((item,) if type(item) != list else item for item in ans)
  File "/home/sphelps/anaconda3/envs/fitch-python/lib/python3.7/site-packages/panflute/base.py", line 259, in <genexpr>
    ans = (item.walk(action, doc) for item in obj)
  File "/home/sphelps/anaconda3/envs/fitch-python/lib/python3.7/site-packages/panflute/base.py", line 275, in walk
    altered = action(self, doc)
  File "/home/sphelps/anaconda3/envs/fitch-python/lib/python3.7/site-packages/ipypublish/filters_pandoc/prepare_cites.py", line 62, in process_citations
    for subel in initial_content:
TypeError: 'Caption' object is not iterable