chapter1.tex

\chapter{Introduction and Motivations}
\label{chapter1}

\vspace{0.5cm}

\noindent This document has been created as final report for the semester project of Eurecom Spring 2017 semester. The project has been done with supervision of Professor Pietro Michiardi whom I infinitely thank for his patience and help in following a correct path in the discovery of the world of reinforcement learning.

\section{Why reinforcement learning?}
The question that everyone might ask is: \textit{Why reinforcement learning?} Well, there are many answers that one could give. First of all, it's getting popular and it's going to become the new hype around machine learning soon, so it's better to be prepared for this new wave around this field. Second, reinforcement learning represents the future and the world itself. Try to imagine a baby that is learning his first steps. He tries to stand up but falls and so he tries again. He tries again, again and again until he finally manages to stand up. And then from there he tries to lift one foot and does a step but he falls again. And so he will stand up again and try again until he will finally manage to make some steps. The same happens in nature. We humans, initially, were undeveloped and at a primal stage. With time, evolution comes into act and during the years we developed new abilities and evolved according to our needs. And reinforcement learning is exactly this. It's the model of learning and evolving only from previous experiences in order to improve. Reinforcement learning tries to reproduce nature and aims to apply human learning abilities to a machine. It's the \textit{future}.

\section{The journey to this report}
The initial goal of this project was to play with openAI's Gym environments, reproduce results of the agent in openAI's gitHub page and use that as a starter point to make tests and solve other environments. Studying the model and playing with it, I realized that the methodology and the theory behind reinforcement learning were absolutely unknown arguments to me. This is why, together with the project supervisor, it has been decided to study what reinforcement learning was, how it worked and what were its fundamentals. This report wants to be a written handout of everything it has been learned in these last months of research.

\section{Report outline}
The structure of this report is the following.

In \autoref{chapter2}, an overview of Markov Decision Processes is given, underlying basis of reinforcement learning. 

In \autoref{chapter3}, it is presented the concept of reinforcement learning, focusing on the different functions and main components. The last part of this chapter is dedicated to present the main algorithms studied during the semester project period.

In \autoref{chapter4}, openAI Gym and openAI Universe are introduced and the work done with these environments is reported.

In \autoref{chapter5} some conclusions and future works are reported while \autoref{chapter6}, last one, wants to be a basket full of links, papers, books and other type of interesting materials to study or give a look to get to know more about reinforcement learning.