Multiagent Deep Q-learning

This code is implemented in Tensorflow-2 and Python 3.6

Requirments

Latest version of Tensorflow-2 can be downloaded from the official website. Tensorflow-2 requires CUDA 11
Install other dependencies using the following line of code

pip install --user -r requirements.txt

This package uses openai/multiagent-particle-envs, to install it use the following code

pip install -e /mape/

Getting started

For running a single agent DQN, use the following code*

python main.py --net_mode 'DQN'

For multiagent indpendent DQN**,

python main.py --net_mode 'DQN' --independent

For single agent DRQN,

python main.py --net_mode 'DQN' --recurrent

For single agent dueling,

python main.py --net_mode 'Dueling' --recurrent

* Environment is set at multiagent by default. Change this line of /mape/multiagent/scenarios/simple_spread.py
** Multiagent DQN will use recurrent network by default

Instructions on modifying the game

Changing environment

Import the desired environment class and change code this line in main.py \

No further changes needed.

Increasing agents

For increasing the number of agents follow the steps given below:

1. /mape/multiagent/scenarios/simple_spread.py: Change the number of agents in this line\

Other less relevant changes which can be made is reducing agent and obstacle size as the evironment could clutter.

2. /deeprl_prj/idqn_keras.py: Network size scales with increasing agents. Therefore it is necessary to modify this line and this line, moreover the depth of policy layers can be increased in case of large policy size or number of agents.
   Add addtional networks in these lines. Subseqeuent changes are to be made in calc_q_values, select_action and update_policy function. Correct these lines depending on the policy size.

3. /deeprl_prj/core.py: Change the policy shape on this line. A similar change is to be made in /deeprl_prj/preprocesors.py, this line.

4. While running the code use the flag '-num_agents' which is set default at 2.

Acknowledgements

/deeprl_prj folder is inspired by the works of Yilun Chen.
OpenAI's Multi-Agent Particle Environment is used for training.

Author

Varun Pawar