Skip to content

thomasw/testtube

Repository files navigation

Testtube

Build Status Coverage Status Latest Version

Spare your alt and tab keys by automatically running your project's test suite whenever files change.

Installation

pip install testtube

testtube is tested with Python 2.6, 2.7, 3.2, 3.3 and 3.4, 3.5 and pypy.

Usage

1. Configure testtube

The simplest way to configure testtube is to place a tube.py file in whatever directory testtube's watch command (stir) will be executed in (this is typically a project's root directory). The tube.py file needs to define an iterable named PATTERNS that contains tuples which 1. specify a regular expression to test the paths of changed files and 2. an iterable containing a list of tests to run when a path matches the corresponding regular expression.

Here's an example tube.py file:

In the example above, PATTERNS contains a series of patterns, coupled with a list of callable tests. The second test group, which calls Flake8 and Frosted, includes an optional test group configuration.

An iterable named IGNORE_PATTERNS is also specified. Any paths that match these patterns will be ignored regardless of whether or not they also match a test group defined in PATTERNS.

A test, at its simplest, is just a callable that returns True or False after being passed the path to a changed file and a regular expression match object for the path's match against the corresponding test group's regular expression. The example uses several helpers that ship with testtube. These helpers are callable objects that can be configured in various ways when they are instantiated.

Testtube comes with a number of such helpers, which can be found in helpers.py. They are designed to save consumers from specifying their own tests as much as is possible. If they are insufficient for a specific project, please see Writing custom tests.

Included helpers:

  • Pep8
  • Pyflakes
  • Frosted
  • Pep257
  • Nosetests
  • PythonSetupPyTest (runs python setup.py when matching files change)
  • ClearScreen (clears the screen)

Helpers typically accept the following arguments when instantiated:

  • all_files: run the test against the entire source directory instead of just the changed file (which is the default behavior)
  • fail_fast: Abort running the rest of the test group if the test fails.
  • bells: On failure, testtube will audibly notify the user 3 times unless otherwise specified
  • name: The name of the test in test report output

The following generates a pep8 test configured to run against all files, abort processing of its test group on failure, alert the user 5 times audibly, and show up as "follow pep8 dude" in test report output:

Note that helpers, once instantiated, are just callables that return True or False:

And here's that same example fully incorporated into a tube.py file:

The behavior of helpers can be customized as necessary by overriding specific methods. See helpers.py for further information.

In addition to configuring helpers, test groups can also be configured:

  • fail_fast: abort processing of subsequent test groups if all tests in the configured group did not pass.

In the first example tube.py file, the second test group is configured to abort the rest of the test suite if either Flake8 or Frosted fail.

2. Stir it

Once a tube.py file is in place, tell testtube to watch the project for changes:

$ stir
testtube is now watching /Path/to/CWD/ for changes...

By default, stir will watch the current working directory and configure itself with a settings module named tube (tube.py). If the tube.py file was placed in the project root directory, then one shouldn't need to specify any parameters assuming stir is executed from that same directory. If paths need to be customized a bit, stir -h will light the way:

$ stir -h
usage: stir [-h] [--src_dir SRC_DIR] [--settings SETTINGS]

Watch a directory and run a custom set of tests whenever a file changes.

optional arguments:
  -h, --help           show this help message and exit
  --src_dir SRC_DIR    The directory to watch for changes. (Defaults to CWD)
  --settings SETTINGS  Path to a testtube settings file that defines which
                       tests to run (Defaults to "tube.py" - your settings
                       file must be importable and the path must be relative
                       to your CWD)

Writing custom tests

If the included helpers don't meet the specific needs of a project, custom tests can be defined directly in tube.py. Simply define a callable that accepts two arguments and add it to the PATTERNS list:

If a custom test needs to be configurable like the builtin helpers or if it needs to make system calls, extending the base helper class (testtube.helpers.Helper) and customizing the behavior as is necessary is usually the simplest approach. The following is a tube.py file which defines a configureable test that outputs the file tree for the entire project each time a python file changes:

Note that this example requires tree to be installed on the system ($ brew install tree for OS X users).

Caveats

  • The distinction between r'.*\.py' and r'.*\.py$' is significant. Without the trailing $, testtube will run tests every time pyc files change. That's very likely to not be useful.
  • testtube doesn't currently reload its own configuration when it changes. If tube.py is modified, testtube will need to be restarted.

Local development

Install the development requirements using the included requirements.txt file:

pip install -r requirements.txt

It is often useful to use to use the checkout of testtube that's currently under development to monitor itself using its included tube.py file. Use testtube to help build testtube. This can be achieved by installing the checkout as an editable. Execute the following from the project root and then use the stir command as one usually would:

pip install -e ./

Note that testtube will need to be restarted for code changes to take effect.

Everything else

Copyright (c) Thomas Welfley. See LICENSE for details.