Skip to content

DecompX: Explaining Transformers Decisions by Propagating Token Decomposition

Notifications You must be signed in to change notification settings

mohsenfayyaz/DecompX

Repository files navigation

DecompX

[ACL 2023] DecompX:
Explaining Transformers Decisions by Propagating Token Decomposition

Paper
Gradio Demo Colab Demo
Video Slides Poster

Online Demos

Demo Link
Gradio Demo Check out our online Gradio demo on HuggingFace Spaces
Colab Demo Check out our Colab demo on Google Colab

Abstract

An emerging solution for explaining Transformer-based models is to use vector-based analysis on how the representations are formed. However, providing a faithful vector-based explanation for a multi-layer model could be challenging in three aspects: (1) Incorporating all components into the analysis, (2) Aggregating the layer dynamics to determine the information flow and mixture throughout the entire model, and (3) Identifying the connection between the vector-based analysis and the model's predictions. In this paper, we present DecompX to tackle these challenges. DecompX is based on the construction of decomposed token representations and their successive propagation throughout the model without mixing them in between layers. Additionally, our proposal provides multiple advantages over existing solutions for its inclusion of all encoder components (especially nonlinear feed-forward networks) and the classification head. The former allows acquiring precise vectors while the latter transforms the decomposition into meaningful prediction-based values, eliminating the need for norm- or summation-based vector aggregation. According to the standard faithfulness evaluations, DecompX consistently outperforms existing gradient-based and vector-based approaches on various datasets.

DecompX

Citation

If you found this work useful, please consider citing our paper:

@inproceedings{modarressi-etal-2023-decompx,
    title = "{D}ecomp{X}: Explaining Transformers Decisions by Propagating Token Decomposition",
    author = "Modarressi, Ali  and
      Fayyaz, Mohsen  and
      Aghazadeh, Ehsan  and
      Yaghoobzadeh, Yadollah  and
      Pilehvar, Mohammad Taher",
    booktitle = "Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)",
    month = jul,
    year = "2023",
    address = "Toronto, Canada",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2023.acl-long.149",
    pages = "2649--2664",
    abstract = "An emerging solution for explaining Transformer-based models is to use vector-based analysis on how the representations are formed. However, providing a faithful vector-based explanation for a multi-layer model could be challenging in three aspects: (1) Incorporating all components into the analysis, (2) Aggregating the layer dynamics to determine the information flow and mixture throughout the entire model, and (3) Identifying the connection between the vector-based analysis and the model{'}s predictions. In this paper, we present DecompX to tackle these challenges. DecompX is based on the construction of decomposed token representations and their successive propagation throughout the model without mixing them in between layers. Additionally, our proposal provides multiple advantages over existing solutions for its inclusion of all encoder components (especially nonlinear feed-forward networks) and the classification head. The former allows acquiring precise vectors while the latter transforms the decomposition into meaningful prediction-based values, eliminating the need for norm- or summation-based vector aggregation. According to the standard faithfulness evaluations, DecompX consistently outperforms existing gradient-based and vector-based approaches on various datasets.Our code is available at https://github.com/mohsenfayyaz/DecompX.",
}

About

DecompX: Explaining Transformers Decisions by Propagating Token Decomposition

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published