The Manubot Python package prepares scholarly manuscripts for Pandoc consumption. It automates and scripts several aspects of manuscript creation, including fetching bibliographic metadata for citations.
This program is designed to be used with clones of Manubot Rootstock, which perform Pandoc conversion and continuous deployment. See the Manubot Rootstock usage guide for more information.
To cite the Manubot project or for more information on its design and history, see:
Open collaborative writing with Manubot
Daniel S. Himmelstein, Vincent Rubinetti, David R. Slochower, Dongbo Hu, Venkat S. Malladi, Casey S. Greene, Anthony Gitter
Manubot Preprint (2019) https://greenelab.github.io/meta-review/
If you are using the manubot Python package as part of a manuscript repository, installation of this package is handled though the Rootstock's environment specification.
For other use cases, this package can be installed via pip.
Install the latest release version from PyPI:
pip install --upgrade manubotOr install from the source code on GitHub, using the version specified by a commit hash:
COMMIT=d2160151e52750895571079a6e257beb6e0b1278
pip install --upgrade git+https://github.com/manubot/manubot@$COMMITThe --upgrade argument ensures pip updates an existing manubot installation if present.
Installing the python package creates the manubot command line program.
Here is the usage information as per manubot --help:
usage: manubot [-h] [--version] {process,cite} ...
Manubot: the manuscript bot for scholarly writing
optional arguments:
-h, --help show this help message and exit
--version show program's version number and exit
subcommands:
All operations are done through subcommands:
{process,cite}
process process manuscript content
cite citation to CSL command line utility
Note that all operations are done through the following sub-commands.
The manubot process program is the primary interface to using Manubot.
There are two required arguments: --content-directory and --output-directory, which specify the respective paths to the content and output directories.
The content directory stores the manuscript source files.
Files generated by Manubot are saved to the output directory.
One common setup is to create a directory for a manuscript that contains both the content and output directory.
Under this setup, you can run the Manubot using:
manubot process \
--content-directory=content \
--output-directory=outputSee manubot process --help for documentation of all command line arguments:
usage: manubot process [-h] --content-directory CONTENT_DIRECTORY
--output-directory OUTPUT_DIRECTORY
[--template-variables-path TEMPLATE_VARIABLES_PATH]
[--cache-directory CACHE_DIRECTORY]
[--clear-requests-cache]
[--log-level {DEBUG,INFO,WARNING,ERROR,CRITICAL}]
Process manuscript content to create outputs for Pandoc consumption. Performs
bibliographic processing and templating.
optional arguments:
-h, --help show this help message and exit
--content-directory CONTENT_DIRECTORY
Directory where manuscript content files are located.
--output-directory OUTPUT_DIRECTORY
Directory to output files generated by this script.
--template-variables-path TEMPLATE_VARIABLES_PATH
Path or URL of a JSON file containing template
variables for jinja2. Specify this argument multiple
times to read multiple files. Variables can be applied
to a namespace (i.e. stored under a dictionary key)
like `--template-variables-
path=namespace=path_or_url`. Namespaces must match the
regex `[a-zA-Z_][a-zA-Z0-9_]*`.
--cache-directory CACHE_DIRECTORY
Custom cache directory. If not specified, caches to
output-directory.
--clear-requests-cache
--log-level {DEBUG,INFO,WARNING,ERROR,CRITICAL}
Set the logging level for stderr logging
Manubot has the ability to rely on user-provided reference metadata rather than generating it.
manubot process searches the content directory for files containing manually-provided reference metadata that match the glob manual-references*.*.
If a manual reference filename ends with .json or .yaml, it's assumed to contain CSL Data (i.e. Citation Style Language JSON).
Otherwise, the format is inferred from the extension and converted to CSL JSON using the pandoc-citeproc --bib2json utility.
The standard citation for manual references is inferred from the CSL JSON id or note field.
When no prefix is provided, such as doi:, url:, or raw:, a raw: prefix is automatically added.
If multiple manual reference files load metadata for the same standard citation id, precedence is assigned according to descending filename order.
manubot cite is a command line utility to create CSL JSON items for one or more citations.
Citations should be in the format source:identifier.
For example, the following example generates CSL JSON for four references:
manubot cite doi:10.1098/rsif.2017.0387 pmid:29424689 pmcid:PMC5640425 arxiv:1806.05726The following terminal recording demonstrates the main features of manubot cite:
Additional usage information is available from manubot cite --help:
usage: manubot cite [-h] [--render] [--csl CSL]
[--format {plain,markdown,docx,html,jats}]
[--output OUTPUT] [--allow-invalid-csl-data]
[--log-level {DEBUG,INFO,WARNING,ERROR,CRITICAL}]
citations [citations ...]
Retrieve bibliographic metadata for one or more citation identifiers.
positional arguments:
citations One or more (space separated) citations to produce CSL
for.
optional arguments:
-h, --help show this help message and exit
--render Whether to render CSL Data into a formatted reference
list using Pandoc. Pandoc version 2.0 or higher is
required for complete support of available output
formats.
--csl CSL When --render, specify an XML CSL definition to style
references (i.e. Pandoc's --csl option). Defaults to
Manubot's style.
--format {plain,markdown,docx,html,jats}
When --render, format to use for output file. If not
specified, attempt to infer this from filename
extension. Otherwise, default to plain.
--output OUTPUT Specify a file to write output, otherwise default to
stdout.
--allow-invalid-csl-data
Allow CSL Items that do not conform to the JSON
Schema. Skips CSL pruning.
--log-level {DEBUG,INFO,WARNING,ERROR,CRITICAL}
Set the logging level for stderr logging
Create a development environment using:
conda create --name manubot-dev --channel conda-forge \
python=3.6 jinja2 pandas pytest pandoc
conda activate manubot-dev # assumes conda >= 4.4
pip install --editable .Inside this environment, use pytest to run the test suite.
You can also use the manubot CLI to build manuscripts.
For example:
manubot process \
--content-directory=tests/manuscripts/example/content \
--output-directory=tests/manuscripts/example/output \
--log-level=DEBUG
This section is only relevant for project maintainers.
Travis CI deployments are used to upload releases to PyPI.
To create a new release, bump the __version__ in manubot/__init__.py.
Then run the following commands:
TAG=v`python setup.py --version`
# Commit updated __version__ info
git add manubot/__init__.py release-notes/$TAG.md
git commit --message="Prepare $TAG release"
git push
# Create & push tag (assuming upstream is greenelab remote)
git tag --annotate $TAG --file=release-notes/$TAG.md
git push upstream $TAGOur goal is to create scholarly infrastructure that encourages open science and assists reproducibility.
Accordingly, we hope for the Manubot software and philosophy to be adopted widely, by both academic and commercial entities.
As such, Manubot is free/libre and open source software (see LICENSE.md).
We would like to thank the contributors and funders whose support makes this project possible. Specifically, Manubot development has been financially supported by:
- the Alfred P. Sloan Foundation in Grant G-2018-11163 to @dhimmel.
- the Gordon & Betty Moore Foundation (@DDD-Moore) in Grant GBMF4552 to @cgreene.