Skip to content

Official implementation of ACL 2023 paper "Dynamic Transformers Provide a False Sense of Efficiency"

Notifications You must be signed in to change notification settings

MatthewCYM/SAME

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

3 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

SAME

Official implementation of ACL 2023 paper "Dynamic Transformers Provide a False Sense of Efficiency"

Overview

We propose a simple yet effective energy-oriented attacking framework, SAME, a Slowdown Attack framework on Multi-Exit models.

Requirements

To run our code, please install all the dependency packages by using the following command:

pip install -r requirements.txt

and also download and prepare the glue data by using the following command:

python tools/download_glue.py

Example Usage

We upload two trained multi-exit models to huggingface hub. More models can be trained using the official repo of DeeBERT and PABEE.

To use SAME to attack a entropy-based multi-exit model:

python main.py \
        --early_exit_entropy 0.19 \
        --model_name_or_path mattymchen/deebert-base-sst2 \
        --model_type deebert \
        --data_dir glue_data/SST-2 \
        --task_name SST-2 \
        --do_lower_case \
        --lam 0.8 \
        --top_n 100 \
        --beam_width 5 \
        --per_size 10 \
        --output_dir results/deebert-base-sst2

To use SAME to attack a patience-based multi-exit model:

python main.py \
        --early_exit_patience 4 \
        --model_name_or_path mattymchen/pabee-bert-base-sst2 \
        --model_type pabeebert \
        --data_dir glue_data/SST-2 \
        --task_name SST-2 \
        --do_lower_case  \
        --lam 0.8 \
        --top_n 100 \
        --beam_width 5 \
        --per_size 10 \
        --output_dir results/pabee-bert-base-sst2

Citation

Please cite our paper if you are inspired by SAME in your work:

@inproceedings{chen2023same,
  title={Dynamic Transformers Provide a False Sense of Efficiency},
  author={Chen, Yiming and Chen, Simin and Li, Zexin and Yang, Wei and Liu, Cong and Tan, Robby and Li, Haizhou},
  booktitle={Annual Meeting of the Association for Computational Linguistics (ACL)},
  year={2023}
}

Acknowledgement

Code is implemented based on TextAttack, DeeBERT, and PABEE. We would like thank the authors for making their code public.

About

Official implementation of ACL 2023 paper "Dynamic Transformers Provide a False Sense of Efficiency"

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages