Some examples on how to apply zero-shot on text classification using the 🤗 transformers library.
zero_shot_example.py showcases some examples, while zero_shot_gnad10.py and zero_shot_mlsum.py applies the idea
on to German topic classification datasets. Performance evaluation can be found below.
| Dataset | Accuracy @top 1 | Accuracy @top 3 |
|---|---|---|
| Gnad10 (https://huggingface.co/datasets/gnad10) | 42 - 44% | 47 - 50% |
| MLSUM (https://huggingface.co/datasets/mlsum) | 58% | 82 - 85% |
See the full blog post on www.statworx.com or in this file.