SysGym

A suite of real-world systems and interface to compare iterative optimization algorithms.

Installation

SysGym relies on using Docker to simplify executing a wide variety of real-world environments and benchmarks. Follow the official Docker's documentation to install it.

Once docker is installed, clone this repository and then you can either run pip install . all to install all environments and their dependencies, or pip install .[name_of_env] for example pip install .[postgres] to install

Once installed, you will need to configure the docker instance, e.g., pulling the right image and setup the dev workspace, we provide convienent scripts for you to run. For example to prepare gem5-Aladdin you would run sh scripts/gem5_dockersetup/aladdin_setup.sh. Other scripts available in the scripts directory

Quick start

Run a single optimization loop of a given environment

class SweepCacheBandwidth(ParamsSpace):
    # you can define as many parameters you want to optimize
    cache_bandwidth: DiscreteBox = DiscreteBox(lower_bound=2,
                                               upper_bound=18,
                                               default=4)
# Create the env
env = Gem5(env_cfg=get_gem5_configurations())
# Create a dictionary helper that can translate from system 
# specifications to numpy values (check EnvParamsDict API doc)
params_dict = EnvParamsDict(param_space=param_space)
# Your optimization loop
for i in range(20): 
    # update the parameters of the env and run it
    # this would be the values proposed by your optimizer
    params_dict["cache_bandwidth"] = i
    # run the system with default param space values
    env_measures = env.run(params=params_dict)
    # obser the env_measures
    print(env_measures)  
    # this would be what your optimizer want to observe
    # Run and observe the results
    env_measures = env.run(params_dict)
    print(env_measures)

Supported libraries

You can use this interface and these following extensions to run optimizations using these following methods:

Single-Objective Optimization

For optimizing a single objective (e.g. latency only, throughput only, etc...)

• BoTorch, Turbo, or DeepGP-BO: Use the SysmGym-BoTorch extension.
• NNI suite: Use the SysmGym-NNI extension.

Multi-Objective Optimization

You can optimize multiple objectives using only the following:

• BoGraph (supports sysgym out othe box)
• BoTorch-sysgym

Supported environments

• gem5-Aladdin: System Documentation, SysGYM documentation

• PostgresSQL

• RocksDB: System Documentation, SysGYM documentation

Feel free to contribute your own environment, please check CONTRIBUTION as well as Instruction on adding new environment.