Attack on Meta-Learning Pytorch Implementation

Original implementation by Tristan Deleu based on Chelsea Finn et al. in Pytorch.

Table of Contents

• Background
• Install
• Usage
• References
• Maintainers
• Contributing
  • Contributors
• License

Background

2d-navigation

2d-navigation without attack	2d-navigation with attack

Halfcheetah-vel

Halfcheetah-vel without attack	Halfcheetah-vel with attack

Meta learning, as a learning how to learn method, has become the focus of researchers in recent years because of its great prospect in the field of artificial intelligence. During the meta training, the meta learner can develop a common learning strategy, such as the learning experience memory modules or a fast adaptation initial value, for not only the computer vision tasks but also the reinforcement learning tasks. Despite its approvingly efficient performance, meta learning is under the suspicion of security. The dependability and robustness of meta learning are doubtful, especially in the reinforcement learning area when given an extremely complex task environment. Even though meta reinforcement learning has not been extensively applied, it would be too late to consider the potential threat when attacks occur. Therefore, in our paper, we create an adversarial attacking method on the sampling process of meta reinforcement learning.

Install

To avoid any conflict with your existing Python setup, and to keep this project self-contained, it is suggested to work in a conda environment with Anaconda. To create a conda environment:

conda create -n your_env_name python=3.8

Activate it and install the requirements in requirements.txt.

conda activate your_env_name
pip install -r requirements.txt

Requirements

• Python 3.8 or above
• PyTorch 1.9.1
• MuJoCo 200
• mujoco-py 2.0.2.13

Usage

Learner Training

python train.py --config configs/maml/2d-navigation.yaml --output-folder 2d_navigation --seed 1 --num-workers 8 --use-cuda

Learner Testing

python test.py --config 2d_navigation/config.json --policy 2d_navigation/policy.th --output 2d_navigation/results.npz --num-batches 10 --meta-batch-size 20 --num-workers 12 --use-cuda

Learner Result

python draw.py --resultpath 2d_navigation --output_folder 2d_navigation/returns --num-batches 10 --num-traj 20 

Intermittent Attacker Training

python train_intermittent.py --config configs/maml/2d-navigation.yaml --output-folder 2d_navigation_inter --seed 1 --num-workers 8 --use-cuda

Persistent Attacker Training

python train_persistent.py --config configs/maml/2d-navigation.yaml --output-folder 2d_navigation_pers --seed 1 --num-workers 8 --use-cuda

References

This project contains experiments implementation of

Li, Tao, Haozhe Lei, and Quanyan Zhu. "Sampling Attacks on Meta Reinforcement Learning: A Minimax Formulation and Complexity Analysis." arXiv preprint arXiv:2208.00081 (2022).

If you want to cite the paper

@article{li2022sampling,
  title={Sampling Attacks on Meta Reinforcement Learning: A Minimax Formulation and Complexity Analysis},
  author={Li, Tao and Lei, Haozhe and Zhu, Quanyan},
  journal={arXiv preprint arXiv:2208.00081},
  year={2022}
}

Maintainers

@Haozhe Lei.

Contributing

Feel free to dive in! Open an issue or submit PRs.

Standard Readme follows the Contributor Covenant Code of Conduct.

Contributors ✨

Thanks goes to these wonderful people (emoji key):

Haozhe Lei 💻 🔣 📖 🤔 🚧 📆 💬 👀 🎨

Tao Li 🎨 📋 🤔 🔣 🖋 💬

This project follows the all-contributors specification. Contributions of any kind welcome!

License

MIT © Haozhe Lei