Implemented a new neural network architecture called "dueling architecture" on top of Double Deep Q-network in the Space Invaders games in Atari 2600 domain, given only raw pixel observations and game rewards.
Most approaches for improving deep reinforcement learning performance focus on the learning algorithm itself while using standard neural networks not necessarily designed for reinforcement learning. In this project, we study and implement a potential improvement to the standard neural networks used in Deep Q-learning applications, known as the dueling architecture. Given only raw pixel observations and game rewards, we implement the dueling architecture on top of the Double Deep Q-Network in the Space Invaders games on the Atari environment. We show that the dueling architecture outperforms the stand-alone implementation of the Double Deep Q-Network algorithm on the Atari 2600 domain.
This project has been implemented in Python using Tensorflow, OpenAI-Gym, and OpenAI-baselines libraries. We have learned to play the Space Invaders game using the Dueling DQN and Double DQN algorithms. OpenAI-Gym provides two different versions of the game: SpaceInvaders-ram-v0 represents every instance of the game as 128 length feature vector and SpaceInvaders-v0 represents every instance of the game as a 210x160x3 image. In the case of SpaceInvaders-ram-v0, for input we used the 128 length feature vector as is whereas for SpaceInvaders-v0, we pre-processed the images by gray scaling, down-sampling, and cropping to obtain an image of size 84x84. The RL agent takes as input the current pre-processed image and the history of past three pre-processed images to estimate the direction of movement/change in various objects in the image.