Skip to content

NeuroCSUT/intentions

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

41 Commits
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Do deep reinforcement learning agents model intentions?

This is the code for implementing the intention reading and generalization experiments from the paper Do deep reinforcement learning agents model intentions?. It is using the simple_spread environment from the Multi-Agent Particle Environments (MPE).

Installation

  • To install, cd into the root directory and type pip install -e .

  • Known dependencies: OpenAI gym, tensorflow, numpy, also scikit-learn and matplotlib for plotting.

Re-running the experiments

  • Download and install the MPE code here by following the README.

  • To run the code, cd into the experiments directory and run:

    • for basic MADDPG agents:
      ./experiment.sh coop_navi_0
    • for MADDPG + shared scheme, all agents use shared model:
      ./experiment.sh coop_navi_shared_0 --shared
    • for MADDPG + shuffle scheme, agents are shuffled for each episode:
      ./experiment.sh coop_navi_shuffle_episode_0 --shuffle episode
    • for MADDPG + ensemble scheme, agents are sampled for each episode:
      ./experiment_ensemble.sh coop_navi_ensemble_episode_0 --ensemble-choice episode

Individual scripts

  • train.py - basic training script, also used for evaluation
  • ensemble.py - ensemble training script, also used for evaluation
  • learning_curve.py - plots learning curve of an experiment
  • statistics.py - collects basic benchmark data from evaluation
  • prepare.py - simplifies evaluation data for further processing
  • prepare_ensemble.py - simplifies evaluation data for further processing, for ensemble results
  • accuracy.py - calculates per-timestep target prediction accuracies
  • figure.py - plots target prediction accuracies for all agents
  • sheldon.py - runs evaluation against Sheldon agents (agents with fixed targets)
  • sheldon_ensemble.py - runs evaluation against Sheldon agents, for ensemble results

For usage details refer to experiment.sh, experiment_ensemble.sh and individual files.

Paper citation

If you used this code for your experiments or found it helpful, consider citing the following paper:

@article{matiisen2018do,
  title={Do deep reinforcement learning agents model intentions?},
  author={Matiisen, Tambet and Labash, Aqeel and Majoral, Daniel and Aru, Jaan and Vicente, Raul},
  journal={arXiv preprint arXiv:1805.06020},
  year={2018}
}

Thanks

Thanks to OpenAI for the original paper and for releasing the code.

About

Do deep reinforcement learning agents model intentions?

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Jupyter Notebook 71.1%
  • Python 27.7%
  • Shell 1.2%