Implementing Tube-Based Robust MPC Controller

[Federico-PizarroBejarano]

Implementing the Robust Tube Model Predictive Control based on:
D. Mayne, M. Seron and S. Raković "Robust model predictive control of constrained linear systems with bounded disturbance," in Automatica 41(2): 219–224. 2005. doi: https://doi.org/10.1016/ j.automatica.2004.08.019

[Federico-PizarroBejarano]

Non TubeMPC Changes:

• Changed iterations to max_steps in the PID controller and its experiments to match the variable name in the MPC controllers
• Put YAML files in pid experiment in their own folder to match structure of other experiments
• Updated pytope to version 0.0.5 in the toml file
• Did a large restructuring of the LQR code to update it to current controller standards. For example, it accepted all the arguments to create its own environment rather than getting an env_func as is standard for other controllers. Also, it did not keep track of the data at each iteration (obs, info, done, reward, and action) so this was added. I would recommend completely stripping down LQR to the basic functionality since it incorporates a bunch of extra code that does not seem to be the current standard controller architecture and isn't useful (like why does it run multiple evaluations? this is part of the experiment design, not the controller).
• Switched use_prev_start to warmstart everywhere since this is the current standard used in most of the MPC implementations
• Added two components to linearMPC:

1. terminal ingredients in the form of a terminal constraint and a terminal cost which allowed the linearMPC to actually function on the cartpole environment. These can be turned on or off using the terminal ingredients flag
2. A "backup" controller where, if the linearMPC is infeasible, it switches to solving the MPC problem without state or input constraints to ideally find a feasible solution. This can be turned on and off using the use_backup flag. This allowed us to test the linearMPC in challenging scenarios without it becoming infeasible and quitting early.

• Extended MPC's reset to reset more if a different environment is passed to the run function. Also updated the run function to quit once "done" is sent by the environment as this seems to be the intended behaviour of "done".

[Federico-PizarroBejarano]

Execution Instructions

• Set the number of tests to run using the current configuration, num_tests in mpc_experiment.py
• Turn external disturbances on and off in the respective environment configuration files cartpole_stabilization.yaml and cartpole_tracking.yaml.
• Determine whether to train the TubeMPC controller using the disturbed or undisturbed environment by selecting the correct train_env in mpc_experiment.py
• Set the correct task (stabilization or tracking) and the correct controller (linear_mpc, tube_mpc, lqr) in the executable file mpc_experiment.sh.
• Run the experiment by calling the executable in the command line ./mpc_experiment.sh.

Expected Output

The submitted results were for 14 scenarios, each with 100 trials and took approximately 8hrs to run. The results after each trial are printed to the terminal and the aggregated results are exported as a pickle. Here are the expected results for undisturbed scenarios with 1 trial:

LQR Stabilization:

NUM SUCCESSES: 1
NUM VIOLATIONS: 3
AVG ITERATIONS: 38.0
RMSE: 0.6202432706102726

Linear MPC Stabilization

NUM SUCCESSES: 1
NUM VIOLATIONS: 0
AVG ITERATIONS: 43.0
RMSE: 0.5957017873213217

Tube MPC Stabilization

NUM SUCCESSES: 1
NUM VIOLATIONS: 5
AVG ITERATIONS: 34.0
RMSE: 0.6639842760497662

LQR Tracking

NUM SUCCESSES: 1
NUM VIOLATIONS: 0
AVG ITERATIONS: 201.0
RMSE: 0.0435026456247442

Linear MPC Tracking

NUM SUCCESSES: 0
NUM VIOLATIONS: 0
AVG ITERATIONS: 201.0
RMSE: 0.1492713725180259

Tube MPC Tracking

NUM SUCCESSES: 0
NUM VIOLATIONS: 0
AVG ITERATIONS: 106.0
RMSE: inf

[Federico-PizarroBejarano]

I have an even leaner version of LQR coded up that removes all the extra logging and debugging that the other controllers don't have and leaves only the core LQR. I prefer this. Let me know if you want me to add this simpler LQR to this PR since I am already cutting down LQR in this PR