Documentation and Examples

Tuun is a toolkit for efficient hyperparameter tuning via uncertainty modeling, with a focus on flexible model choice, scalability, and use in distributed settings.

Installation

Tuun requires Python 3.6+. To install all dependencies for development, clone and cd into this repo, and run:

$ pip install -r requirements/requirements_dev.txt

For the full functionality of Tuun, a Stan model must also be compiled (this takes roughly 1 minute) by running:

$ python tuun/probo/models/stan/compile_models.py -m gp_distmat_fixedsig

Quick Start

Note: Tuun is still in the early stages and the following API and functionality may undergo changes.

Here is a minimal working example, which uses Tuun to optimize a function over a one-dimensional Euclidean search space with bounds [-5, 5].

from tuun.main import Tuun

# instantiate Tuun
tu = Tuun()

# set search space
search_space = ('real', [-5, 5])
tu.set_config_from_list(search_space)

# define function to optimize
f = lambda x: x[0] ** 4 - x[0] ** 2 + 0.1 * x[0]

# minimize function over search space
result = tu.minimize_function(f, 20)

This should find a minima at roughly: 𝑥*=[−0.73], where 𝑓(𝑥*)=−0.32. See this docs page for more details on defining different search spaces for Tuun.

Tuun also allows for fine-grained configuration of individual components and search spaces, as shown in the example below.

from tuun.main import Tuun

config = {
    # configure tuning backend
    'backend': 'probo',

    # configure model
    'model_config': {'name': 'simplegp', 'ls': 3.0, 'alpha': 1.5, 'sigma': 1e-5},

    # configure acquisition function
    'acqfunction_config': {'name': 'default', 'acq_str': 'ei'},

    # configure acquisition optimizer
    'acqoptimizer_config': {'name': 'default', 'max_iter': 200},

    # configure domain
    'domain_config': ('real', [-5, 5]),
}
tu = Tuun(config)

f = lambda x: x[0] ** 4 - x[0] ** 2 + 0.1 * x[0]
result = tu.minimize_function(f, 20)

This should also find a minima at roughly: 𝑥*=[−0.73], where 𝑓(𝑥*)=−0.32. See this docs page for more details on possible configurations.

See here for a larger set of quick start examples for Tuun.

Use in NNI

Tuun can be used as a custom tuner in NNI (neural network intelligence), which allows for visualization and experiment management. See this docs page for more details, and this directory for a minimal example.

Examples

See a few examples of Tuun here, including the setup for various benchmark functions, different types of search spaces, and use within NNI.

The plots below show a couple examples of Tuun, along with other tuning algorithms, on benchmark functions.

   

Affiliations

Tuun is part of the CASL project.

Companies and universities using and developing Tuun.