Advanced Controls Test Bed

The Advanced Controls Test Bed (ACTB) is a virtual buildings test bed that interfaces external controllers to high-fidelity Spawn of EnergyPlus models. The ACTB has two interfaces to Python control libraries:

• one interface for model predictive controllers (MPC) based on do-mpc
• one interface for reinforcement learning controllers (RLC) based on OpenAI Gym

Spawn of EnergyPlus is a model-exchange framework that allows the simulation of building envelope and internal gains models in EnergyPlus, and their HVAC systems and controls in Modelica.

This makes the ACTB a flexible and user-friendly framework for developing and evaluating advanced controls using high-fidelity building models.

The ACTB is based on the BOPTEST framework available here and the BOPTEST OpenAI Gym interface.

The ACTB is currently in its first release version and might undergo changes, contain broken modules, or function unexpectedly. Please report all issues in the Issues tab at the top of the page.

Install

Operating systems

The ACTB works with Linux and macOS out of the box, but requires an additional software layer for working on Windows. If you are a Windows user, please install the Windows Subsystem for Linux (WSL) by following the instructions here. You will simply need to start an instance of the WSL first, then run the commands found in the various instructions of this repository in the Linux terminal that will be launched.

Docker

The ACTB is packaged as a Docker container, please install Docker by following the instructions here.

Python

While the Docker container comes with Python and all the dependencies installed, if you are running a controller script externally you will need to install Python and a few libraries. For Python, please install Python 3.9 by following these links for Linux, macOS, or Windows.

Note: If you have several versions of Python installed, make sure you are running Python 3.9 by replacing python commands with python3 whenever the command is present in the instructions.

Libraries

We strongly recommend installing and using pip to manage Python libraries. Follow the instructions here to install pip.

With pip installed, just open a terminal window, cd at the root of the ACTB and run:

pip install -r requirements.txt

If you have several versions of Python installed, you may need to replace pip by pip3 in the above command to make sure you are installing these libraries for your Python 3.

Quick-Start to Run Test Cases

1. Make sure the Docker daemon is running.
2. Build the Docker containers by running $ make build
3. Run the Docker containers by running $ make run
4. Run an example test controller in a separate terminal:

• $ cd examples/python/MPC-spawnrefsmalloffice && python main.py to test a MPC controller.
• $ cd examples/python/RLC-spawnrefsmalloffice && python DQN_Test_run.py to test a RLC controller.
• $ cd examples/python/RBC-spawnrefsmalloffice && python supervisory_example.py to test the RBC controller.

4. Shutdown a test case container by selecting the container terminal window, Ctrl+C to close port, and make stop to stop the Docker container.
5. Remove the test case Docker image by $ make remove-image.

Please refer to the ReadMe file present in the actb_client directory for more details on the API usage.

Architecture

The ACTB is based on BOPTEST-service, a merge between BOPTEST and Alfalfa. It is supplemented by a library of high-fidelity Spawn models and two advanced controller interfaces. A metamodeling framework allows the generation of reduced order models from Spawn data, in order to provide computationally-efficient models for MPC planning models and RLC pre-training (see the RLC guide here).

Interfaces

Two advanced controller interfaces are currently available for the ACTB.

• the do-mpc interface for MPC is available under /interfaces/dompc, along with a ReadMe file and examples of applications. It is used by the MPC example found in /examples/python/MPC-spawnrefsmalloffice.
• the OpenAI Gym interface for RLC is available under /interfaces/openai-gym, along with a ReadMe file and examples of applications. It is used by the RLC example found in /examples/python/RLC-spawnrefsmalloffice

Test cases

Testcases are found in the /boptest-service/boptest/testcases directory. Example controllers to go with these test cases are found under the /examples directory.

For the moment, one Spawn test case is available. It represents the U.S. Department of Energy's Small Office Building. It is provided with a documentation, found under /boptest-service/boptest/testcases/spawnrefsmalloffice/docs.

Known Issues

Currently, the ACTB has some issues that we are aware of and are working towards solving. These are:

• the metamodeling framework produces models which temperature prediction is shifted by approximately 20 K.
• the Spawn test cases cannot be initialized again after the simulation completed. You currently need to stop the Docker container and start it again. This will be fxed

Structure

• /boptest-service contains a copy of the boptest-service repo.
  • /boptest-service/boptest/testcases contains Spawn of EnergyPlus test cases, including docs, models, and configuration settings.
• /examples contains examples of MPC and RLC Python controllers that interact with Spawn models.
• /interfaces contains the clients for interfacing do-mpc and OpenAI Gym to the ACTB
• /metamodeling contains prototype code for the metamodeling framework
• /templates contains template Modelica code for a test case emulator model.

Acknowledgements

We gratefully acknowledge funding by the U.S. Department of Energy under Project 3.2.6.80, titled Multi-Objective Deep Reinforcement Learning Controls.

To develop the ACTB, we rely on the following software:

• the Buildings Operation Performance TEST, developed under IBPSA Project 1
• the BOPTEST OpenAI Gym interface, developed under IBPSA Project 1
• Spawn of EnergyPlus
• the System Identification Package for Python
• the do-mpc package
• the OpenAI Gym package

Authors

This project is led by Professor Gregor Henze, at the University of Colorado Boulder. It is developed and maintained by Dr. Thibault Marzullo, Sourav Dey, and Nicholas Long, at the University of Colorado Boulder.

Former project members:

• Developer (2021-2021): José Angel Leiva Vilaplana, Masters candidate, Universitat Politecnica de Catalunya.