Custom Builders

Customizing the Build System

Since the release on March 13, 2014 (v3.1.0), LaTeXTools has had support for custom build systems, in addition to the default build system, called the "traditional" builder. Details on how to customize the traditional builder are documented in the README. If none of the available options meet your needs you can also create a completely custom builder which should be able to support just about anything you can imagine. Let me know if you are interested in writing a custom builder!

Custom builders are small Python scripts that interact with the LaTeXTools build system. In order to write a basic builder it is a good idea to have some basic familiarity with the Python language. Python aims to be easy to understand, but to get started, you could refer either to the Python tutorial or any of the resources Python suggests for non-programmers or those familiar with other programming languages.

LaTeXTools comes packaged with a small sample builder to demonstrate the basics of the builder system, called SimpleBuilder which can be used as a reference for what builders can do.

Note that if you are interested in creating your own builder, please pay attention to the Caveats section below.

The Basics

A really simple Builder

Every builder consists of a single Python class called "WhateverBuilder" (so, for example, "TraditionalBuilder", "SimpleBuilder", etc.) which is a sub-class of a class called "PdfBuilder". Note that the class name should be unique (i.e., it can't share a name with any of the built-in builders) and it must end with "Builder". Each builder class implements a single generator function called commands() which is responsible for generating a list of commands to be run.

Below is a really simple builder that does nothing to demonstrate the basic structure of a builder:

# the pdfBuilder module will always be available on the PYTHONPATH
from pdfBuilder import PdfBuilder

class ReallySimpleBuilder(PdfBuilder):
    # for now, we are ignoring all of the arguments passed to the
    # builder
    def __init__(self, *args):
        # call the __init__ method of PdfBuilder
        # this does some basic initialization, which we'll discuss
        # in more detail later
        super(ReallySimpleBuilder, self).__init__(*args)
        
        # now we do the initialization for this builder
        # the only thing that must be set here is the builder name
        self.name = "Really Simple Builder"
    
    # commands is a generator function that yields the commands to
    # be run
    def commands(self):
        # display a message in the build output console
        self.display("\n\nReallySimpleBuilder")
        
        # yield is how we pass the command to be run back to
        # LaTeXTools
        #
        # each yield should yield a tuple consisting of the command
        # to be run and a message to be displayed, if any
        #
        # for the ReallySimpleBuilder, we yield ("", None) which
        # tells LaTeXTools there is no command to be run and no
        # message to be displayed
        yield ("", None)

To use this, save it to a file called "reallySimpleBuilder.py" in a subfolder of your Sublime Text User package (you can find this folder by selecting Preferences|Browse Packages... or running the Preferences: Browse Packages command). Call the subfolder "LaTeXTools Builders" (the name doesn't matter). Then, in your LaTeXTools preferences, change the "builder" setting to "reallySimple" and change the "builder_path" setting to "User/LaTeXTools Builders". Try compiling a document. You should see the following:

[Compiling ...]

ReallySimpleBuilder

Notice how the message we set using self.display() gets displayed.

Also notice how the name of the Python file matches the name of the builder (except with the first letter lower-cased) and the name given to the setting. These must match in order for LaTeXTools to be able to find and execute your builder.

Generating Basic Commands

The PdfBuilder base class provides access to some very basic information about the tex document being compiled, which can be gathered from the following variables:

Variable	Description
`self.tex_root`	the full path of the main tex document, e.g. C:\path\to\tex_root.tex
`self.tex_dir`	the full path to the directory containing the main tex document, e.g. C:\path\to
`self.tex_name`	the name of the main tex document, e.g. tex_root.tex
`self.base_name`	the name of the main tex document without the extension, e.g. tex_root
`self.tex_ext`	the extension of the main tex document, e.g. tex

(Note that all of these refer to the main tex document, specified using the %!TEX root directive or the "TEXroot" setting)

With this is mind, we can now write a builder that actually does something useful. Below is a sample builder that simply runs the standard pdflatex, bibtex, pdflatex, pdflatex pattern.

from pdfBuilder import PdfBuilder

# here we define the commands to be used
# commands are passed to subprocess.Popen which prefers a list of
# arguments to a string
PDFLATEX = ["pdflatex", "-interaction=nonstopmode", "-synctex=1"]
BIBTEX = ["bibtex"]

class BasicBuilder(PdfBuilder):
    def __init__(self, *args):
        super(BasicBuilder, self).__init__(*args)
        
        # now we do the initialization for this builder
        self.name = "Basic Builder"

    def commands(self):
        self.display("\n\nBasicBuilder: ")

        # first run of pdflatex
        # this tells LaTeXTools to run:
        #  pdflatex -interaction=nonstopmode -synctex=1 tex_root
        # note that we append the base_name of the file to the
        # command here
        yield(PDFLATEX + [self.base_name], "Running pdflatex...")

        # LaTeXTools has run pdflatex and returned control to the
        # builder
        # here we just add text saying the step is done, to give
        # some feedback
        self.display("done.\n")

        # now run bibtex
        yield(BIBTEX + [self.base_name], "Running bibtex...")

        self.display("done.\n")

        # second run of pdflatex
        yield(
            PDFLATEX + [self.base_name],
            "Running pdflatex again..."
        )

        self.display("done.\n")

        # third run of pdflatex
        yield(
            PDFLATEX + [self.base_name],
            "Running pdflatex for the last time..."
        )

        self.display("done.\n")

To use this, save it to a file called "basicBuilder.py" then change your "builder" setting to "basic". When you compile a document, you should see the following output:

[Compiling ...]

BasicBuilder: Running pdflatex...done.
Running bibtex...done.
Running pdflatex again...done.
Running pdflatex for the last time...done.

...

Since this builder actually does a build, there may be additional messages displayed after the last message from our builder. These usually come from LaTeXTools log-parsing code which is run after the build completes.

Interacting with Output

Of course, sometimes it is necessary not just to run a series of commands, but also to react to the output of those commands to determine the next step in the process. This is what the SimpleBuilder does to determine whether or not to run BibTeX, searching the output for a particular pattern that pdflatex generates to determine whether or not to run BibTeX.

The output of the previously run command is available after LaTeXTools returns control to the builder in the variable self.out. This consists of anything written to STDOUT and STDERR, i.e., all the messages you would see if running the command from the terminal / command line.

Building on our previous example, here's a builder that checks to see if BibTeX (only) needs to be run. This example makes use of Python's re library, which provides operations for dealing with regular expressions, a way of matching patterns in strings.

from pdfBuilder import PdfBuilder

import re

PDFLATEX = ["pdflatex", "-interaction=nonstopmode", "-synctex=1"]
BIBTEX = ["bibtex"]

# here we define a regular expression to match the output expected
# if we need to run bibtex
# this matches any lines like:
#  Warning: Citation: `aristotle:ethics' on page 2 undefined
CITATIONS_REGEX = re.compile(
    r"Warning: Citation `.+' on page \d+ undefined"
)

class BibTeXBuilder(PdfBuilder):
    def __init__(self, *args):
        super(BibTeXBuilder, self).__init__(*args)
        
        # now we do the initialization for this builder
        self.name = "BibTeX Builder"

    def commands(self):
        self.display("\n\nBibTeXBuilder: ")

        # we always run pdflatex
        yield(PDFLATEX + [self.base_name], "Running pdflatex...")

        # here control has returned to the builder from LaTeXTools
        # we display the same message as last time...
        self.display("done.\n")

        # and now we check the output to see if bibtex needs to be
        # run
        # search will scan the entire output for any match of the
        # pattern we defined above
        if CITATIONS_REGEX.search(self.out):
            # if a matching bit of text is found, we need to run
            # bibtex
            # now run bibtex
            yield(BIBTEX + [self.base_name], "Running bibtex...")

            self.display("done.\n")

            # we only need to run the second and third runs of
            # pdflatex if we actually ran bibtex, so these remain
            # inside the same `if` block code to run bibtex

            # second run of pdflatex
            yield(
                PDFLATEX + [self.base_name],
                "Running pdflatex again..."
            )

            self.display("done.\n")

            # third run of pdflatex
            yield(
                PDFLATEX + [self.base_name],
                "Running pdflatex for the last time..."
            )

            self.display("done.\n")

To use this builder, save it to a file called "bibTeXBuilder.py" and change the "builder" setting to "bibTeX". When using this builder, you should see that it only runs bibtex and the final two pdflatex commands if you have any citations.

More Advanced Topics

The following sections deal with more advanced concepts in creating builders or with features that need careful handling for one reason or another.

Allowing the user to set options

Sometimes having a static series of commands to build a document is not enough and you'd want to give the user an opportunity to, for example, tell the builder what command to run to generate a bibliography. LaTeXTools provides your builder with access to the settings in the build_settings block of your LaTeXTools preferences through the variable self.builder_settings. Note that when allowing the use of settings it is important to verify that values you get make sense and that if no value is supplied, you provide a sane default.

Our builder from the previous example could be modified to support either bibtex or biber as the bibliography program depending on a user setting like so:

from pdfBuilder import PdfBuilder

import re
import sublime

PDFLATEX = ["pdflatex", "-interaction=nonstopmode", "-synctex=1"]
# notice that we do not define the bibliography command here, since
# it will depend on settings that can only be known when our builder
# is initialized

CITATIONS_REGEX = re.compile(
    r"Warning: Citation `.+' on page \d+ undefined"
)
# BibLaTeX outputs a different message from BibTeX, so we must catch
# that too
BIBLATEX_REGEX = re.compile(
    r"Package biblatex Warning: Please \(re\)run \S*"
)

class BibBuilder(PdfBuilder):
    def __init__(self, *args):
        super(BibBuilder, self).__init__(*args)

        # now we do the initialization for this builder
        self.name = "Bibliography Builder"

        # here we get which bibliography command to use from the
        # builder_settings
        # notice that we draw this setting from the
        # platform-specific portion of the builder_settings block
        # this allows the setting to be changed for each platform
        self.bibtex = self.builder_settings.get(
            sublime.platform(), {}).get('bibtex') or 'bibtex'
        # notice that or clause here ensures that self.bibtex will
        # be set to 'bibtex' if the 'bibtex' setting is unset or
        # blank.

    def commands(self):
        self.display("\n\nBibBuilder: ")

        # we always run pdflatex
        yield(PDFLATEX + [self.base_name], "Running pdflatex...")

        # here control has returned to the builder from LaTeXTools
        self.display("done.\n")

        # and now we check the output to see if bibtex / biber needs
        # to be run
        if (
            CITATIONS_REGEX.search(self.out) or 
            BIBLATEX_REGEX.search(self.out)
        ):
            # if a matching bit of text is found, we need to run the
            # configured bibtex command
            # note that we wrap this value in a list to ensure that
            # a list is yielded to LaTeXTools
            yield(
                [self.bibtex] + [self.base_name],
                "Running " + self.bibtex + "..."
            )

            self.display("done.\n")

            # second run of pdflatex
            yield(
                PDFLATEX + [self.base_name],
                "Running pdflatex again..."
            )

            self.display("done.\n")

            # third run of pdflatex
            yield(
                PDFLATEX + [self.base_name],
                "Running pdflatex for the last time..."
            )

            self.display("done.\n")

To use this builder, save it to a file called "bibBuilder.py" and change the "builder" setting to "bib". You should be able to change what command gets run when a bibliography is needed by changing the "bibtex" setting in "builder_settings".

Assuming you are on OS X, you might use something like this to run biber instead of bibtex with this builder:

{
    "builder_settings": {
        "osx": {
            "bibtex": "biber"
        }
    }
}

Important Notice that all interaction with the settings occurred in builder's __init__() function. This is to ensure that the builder works on ST2 as well as ST3. For more on this, see the section on Interacting with the Sublime API below.

Working with Output and Auxiliary Directories

Starting with v3.8.0 (released June 18, 2016), LaTeXTools provides support for the --output-directory, --aux-directory and --jobname parameters for pdflatex and friends. The selected values of these options are exposed as simple variables to the builder:

Variable	Description
`self.output_directory`	the output directory to use
`self.aux_directory`	the auxiliary directory to use
`self.jobname`	the jobname to use

The can be passed on to the underlying build engine using the appropriate flags, as is done in the built-in basic builder. But note that there are some caveats:

If unset, any of these values will be set to the Python value of None. If they are None the option should not be passed to pdflatex as it will cause an error.
Your builder will need to make the output_directory and aux_directory if they do not exist. More importantly, it will need to make any subdirectories for files from subdirectories that are input using the \include command. This is because \include opens a per-file .aux file with the same directory structure in the output directory. If you are wrapping another LaTeX build tool, it may already handle this, but if you are using pdflatex and friends directly, they do not handle this.

Passing `Popen` Objects

With v3.6.2 (released January 18, 2016), the main execution loop has been modified to support accepting not only strings and lists as we have seen above, but also to allow a builder to yield Popen objects as well. This might be useful if you need more fine-grained control over the exact way a command is executed, perhaps using a custom environment or using the user's shell. For the most part, it is recommended that you avoid creating custom Popen objects unless you cannot get the standard tools to do what you need.

In principle, using Popen objects is pretty much the same as using strings or lists of commands, you simply yield them, and LaTeXTools allows them to run before returning to your code. However, some caveats must be observed.

Caveats:

It is important to ensure that your Popen objects are always created with stdout set to subprocess.PIPE. Otherwise, you will not get any output to respond to. It is recommended that you set stderr to subprocess.STDOUT so that you also get any error output.
On OS X and Linux, it is important to set the preexec_fn to os.setsid. Otherwise, your builder could fail to kill any spawned or forked processes that are started as part of the build.
If you override the env, please be sure that you properly copy the PATH variable from os.environ to your new environment, especially on OS X or otherwise take account of the user's configured texpath setting. Otherwise, your builder may be unable to find LaTeX and friends. This is especially true on more recent versions of OS X.
In addition to the above, please make sure you have read through and understood the other caveats in the Caveats section of this document.

An example where this could be helpful is in selecting the bibliography engine to run using the MiKTeX-only command texify. As the documentation states, the bibliography processing program can be selected by setting the %BIBTEX% environment variable. Expanding on our previous example, here is a builder that uses texify to run a standard latex build with the bibliography processor selected by a user-configurable option:

from pdfBuilder import PdfBuilder

import copy
import os
import sublime
# we need to import subprocess as we will be creating one of our own
import subprocess


class MikbibBuilder(PdfBuilder):
    def __init__(self, *args):
        super(BibBuilder, self).__init__(*args)
        
        # now we do the initialization for this builder
        self.name = "MiKTeX Bibliography Builder"

        self.bibtex = self.builder_settings.get(
            sublime.platform(), {}).get('bibtex') or 'bibtex'

    def commands(self):
        self.display("\n\nMiKTeX BibliographyBuilder: ")

        # MiKTeX is Windows-specific, so it doesn't make sense to
        # try to run it on a non-Windows machine
        if sublime.platform() != 'windows':
            yield(
                "",
                "The MiKTeX Bibliography Builder can only be used"
                " on Windows"
            )
            return

        # we create this startupinfo object to ensure that the
        # Windows console does not appear
        startupinfo = subprocess.STARTUPINFO()
        startupinfo.dwFlags |= subprocess.STARTF_USESHOWWINDOW

        # now we construct the environment
        # first creating a copy of the current environment
        env = copy.copy(os.environ)
        # then setting the %BIBTEX% variable to the user-configured
        # setting
        env['BIBTEX'] = self.bibtex
        # note that LaTeXTools ensures that the %PATH% variable
        # contains the `texpath` setting by default

        # here we construct our Popen object
        p = subprocess.Popen(
            ['texify', '-b', '-p', '--texoption="--synctex=1"'],
            # here we use the startupinfo object created above
            startupinfo=startupinfo,
            # we redirect any output to stderr to stdout
            stderr=subprocess.STDOUT,
            # ensure that the output to stdout is available to
            # LaTeXTools
            stdout=subprocess.PIPE,
            # finally, we pass in the environment that should be
            # used including our modified %BIBTEX% value
            env=env
        )

        # now we yield the Popen object to LaTeXTools to be run
        yield(p, "Running texify...")

        # here control has returned to the builder from LaTeXTools
        self.display("done.\n")
        # because texify runs the whole pdflatex, bibtex, pdflatex,
        # pdflatex cycle, we don't need to do anything else.

To use this builder, save it to a file called "mikbibBuilder.py" and change the "builder" setting to "mikbib". You should be able to change what command gets run when a bibliography is needed by changing the "bibtex" setting in "builder_settings" as with the previous builder.

Caveats

LaTeXTools makes some assumptions that should be adhered to or else things won't work as expected:

the final product is a PDF which will be written to the output directory or the same directory as the main file and named $file_base_name.pdf
the LaTeX log will be written to the output directory or the same directory as the main file and named $file_base_name.log
if you change the PATH in the environment (by using the env setting), you need to ensure that the PATH is still sane, e.g., that it contains the path for the TeX executables and other command line resources that may be necessary.

In addition, to ensure that forward and backward sync work, you need to ensure that the -synctex=1 flag is set for your latex command. Again, don't forget the -interaction=nonstopmode flag (or whatever is needed for your tex programs not to expect user input in case of error).

Interacting with the Sublime API

Sublime Text provides a very rich API that could be of use to builders. However, it is advisable that any interactions with the Sublime Text API happen in the __init__() function, if possible. This is because the __init__() function is run on the main Sublime thread whereas the commands() function is called from a separate thread which cannot safely interact with the Sublime API on ST2. commands() is run on a separate thread.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Custom Builders

Customizing the Build System

The Basics

A really simple Builder

Generating Basic Commands

Interacting with Output

More Advanced Topics

Allowing the user to set options

Working with Output and Auxiliary Directories

Passing `Popen` Objects

Caveats

Interacting with the Sublime API

Clone this wiki locally

Custom Builders

Customizing the Build System

The Basics

A really simple Builder

Generating Basic Commands

Interacting with Output

More Advanced Topics

Allowing the user to set options

Working with Output and Auxiliary Directories

Passing Popen Objects

Caveats

Interacting with the Sublime API

Clone this wiki locally

Passing `Popen` Objects