Tennis Unity Environment
In this project, I used DDPG model to train 2 agents playing tennis. This environment has 24 states per agent. And each agent has 2 continuous actions or degrees of freedom. The environment is episodic, and to solve it, one of the agents must attain a average score of +0.5 over 100 consecutive episodes.
These instructions will get you a copy of the project up and running on your local machine for development and testing purposes.
I have tested this repo with Python 3.9 and 3.10. To continue, install either of these versions on your local machine. With Python installed, I suggest you create a virtual environment to install required libraries:
python -m venv desired_path_for_envActivate this environment before moving to next step. For addirional help, check Python documentation here.
The required packages for this project are listed in requirements file. To install these libraries, from the repo folder, run the following command in your virtual env:
python -m pip install -r requirements.txtThe already built Unity environment for this project is accessible from following links:
Linux: click here
Mac OSX: click here
Windows (32-bit): click here
Windows (64-bit): click here
Decompress (unzip) the downloaded file and copy it to the repo folder.
The training and testing scripts are located in scripts folder.
To train the model, use train_agent.py script. This script accepts the following arguments:
- Path to downloaded Unity App: --unity-app
- Target Score to save trained model: --target-score
cd scripts
python train_agent.py --unity-app Tennis.app --target-score 0.5On my machine, the environment was solved in 722 episodes:
Episode 100 Average Score: 0.00400
Episode 200 Average Score: 0.00780
Episode 300 Average Score: 0.00600
Episode 400 Average Score: 0.02550
Episode 500 Average Score: 0.04760
Episode 600 Average Score: 0.12620
Episode 700 Average Score: 0.31120
Environment solved in 722 episodes! Average Score: 0.50260
Trained model weights saved to: checkpoint_722.pth
Saved Trained Actor Checkpoint
Saved Trained Critic Checkpoint
To compare a trained agent with a untrained one, use [test_agent.py] script. This script accepts the following arguments:
- Path to downloaded Unity App: --unity-app
- Path to saved actor model checkpoint: --actor-checkpoint-file
- Path to saved critic model checkpoint: --critic-checkpoint-file
cd scripts
python test_agent.py --unity-app Tennis.app --actor-checkpoint-file ../checkpoints/actor_checkpoint_722.pth --critic-checkpoint-file ../checkpoints/critic_checkpoint_722.pthOutput:
[>] Try untrained Tennis agents.
[-] Score: 0.2700000088661909
[>] Try a trained DDPG agent to play tennis.
[-] Score: 0.7650000127032399
Click on below GIF animation to open youtube video:
- Sina Fathi-Kazerooni - Website
This project is open source under MIT License and free to use. It is for educational purposes only and provided as is.
I have used parts of DDPG_Pendulum scripts in Udacity DRL repo under MIT License. Scripts in mlagents are based on Udacity DRL repo with minor modifications.