Reinforcement Learning is a hot field right now, with exciting applications in robotics and self-driving cars.
I present a rigorous mathematical derivation of a reinforcement learning algorithm that learns to beat a computer at Pong.
This AI does not rely on hand-engineered rules or features;
instead, it masters the environment by looking at raw pixels and learning from experience, just as humans do.
A full description is contained in the Jupyter notebook analysis.ipynb.
To see the videos embedded in the notebook, please see the rendered version of the notebook.
On Mac OS X 10.11.6 and 10.12.6, one can set up an environment to simulate Pong by issuing the commands
brew install cmake boost boost-python sdl2 swig wget
conda create -n openai python=2 ipython-notebook --yes
source activate openai
pip install 'gym[all]'
I use Jake Vanderplas' JSAnimation package to render movies in the browser. Install the package in the currently active conda environment as follows:
git clone https://github.com/jakevdp/JSAnimation
cd JSAnimation
python setup.py install
Additionally, you may need to do
conda install matplotlib
conda install pandas
in the openai conda environment.
Inspired by Andrej Karpathy's blog post.