Deep Reinforcement Learning for Playing 2.5D Fighting Games

Official Tensorflow implemenation of the paper. we create an OpenAI-gym-like gaming environment with the game of Little Fighter 2 (LF2), and present a novel A3C+ network for learning RL agents based on A3C. The introduced model includes a Recurrent Info network, which utilizes game-related info features with recurrent layers to observe combo skills for fighting.

Demo

Our agent plays Firen (red) against Freeze (blue).

For the enviromnent of LF2 Gym, please refer here.

Paper

Deep Reinforcement Learning for Playing 2.5D Fighting Games
Hsin-Yu Chang*, Yu-Jhe Li*, Yu-Jing Lin, Po-Wei Wu and Yu-Chiang Frank Wang.
IEEE International Conference on Image Processing (ICIP), 2018 (* equal contribution)
[arxiv]

Citing

If you find the work is useful in your research, please consider citing:

@inproceedings{li2018deep
  title={Deep Reinforcement Learning for Playing 2.5 D Fighting Games},
  author={Li, Yu-Jhe and Chang, Hsin-Yu and Lin, Yu-Jing and Wu, Po-Wei and FrankWang, Yu-Chiang},
  booktitle={2018 25th IEEE International Conference on Image Processing (ICIP)}
  year={2018}
}

Usage

Requirements

1. Python 3.4 or higher
2. TensorFlow 1.3.0

Download the Web Drivers

1. Download PhantomJS
2. Put to webdriver/phantomjsdriver_2.1.1_linux64/phantomjs

Installing PhantomJS in Ubuntu: https://gist.github.com/julionc/7476620

Setup LF2 GYM

1. Clone the env first

git clone https://github.com/elvisyjlin/lf2gym.git

Then follow the step to prepare LF2 game environment

2. Put a3c*.py in this repo beside the file lf2gym.py

Train an agent

1. Start a LF2Server first. For more information, please refer here.
2. python a3c.py --train --savepath [/path/to/your/model]

Test with the pre-trained model

1. Download the models from coming soon!
2. python a3c.py --test --loadpath [/path/to/your/model]
3. It would save 10 testing video clips under the current working directory

Example:

env = lf2gym.make(
        startServer=True,
        difficulty=Difficulty.Challengar,
        characters=[Character.Firen, Character.Freeze],
        port=[YOURPORT])

Control and play with pre-trained agent

1. python a3c_play.py --loadpath [/path/to/your/model]
2. Chrome browser will be opened and have fun with it!

Keyboard control setting is described here.

Modifications to original game

In order to train the agent better, we remove the background of 'HK Coliseum' and the image of 'pause'. The frame rate is also raised to 180fps.

Reference

1. Project F.
2. OpenAI Gym.

Code inspired from:

1. https://github.com/coreylynch/async-rl
2. https://medium.com/emergent-future/simple-reinforcement-learning-with-tensorflow-part-8-asynchronous-actor-critic-agents-a3c-c88f72a5e9f2
3. https://github.com/papoudakis/a3c-tensorflow