hsemotion-onnx

AffectNet SOTA 모델인 Effectnet 기반 멀티 태스크 러닝 모델(Hsemotion) 데모 및 ONNX 구현

악간의 변경점이 있습니다. Output을 4가지로 변형하였습니다.

두 가지 버전의 임베딩 벡터가 다릅니다. hsemotion_1280.onnx의 경우 최종 임베딩 벡터에 GAP를 적용하여 출력하는 버젼이고,
hsemotion_1280_7_7.onnx의 경우 GAP 적용 전의 피쳐맵을 출력합니다.

• embedding vector : (1, 1280) or (1, 1280, 7, 7)
• expression(emotion) : (1, 8)
• valence : (1, )
• arousal : (1, )

hsemotion link : https://github.com/HSE-asavchenko/face-emotion-recognition

Type

Weight : enet_b0_8_va_mtl.pt
Backbone : Effectnet_B0
Input Image Shape(Face Croped) : 224, 224
Output : embedding vector, emotion(8 category), valence, arousal

Demo

# Run
python3 demo.py

Research papers

If you use our models, please cite the following papers:

@inproceedings{savchenko2021facial,
  title={Facial expression and attributes recognition based on multi-task learning of lightweight neural networks},
  author={Savchenko, Andrey V.},
  booktitle={Proceedings of the 19th International Symposium on Intelligent Systems and Informatics (SISY)},
  pages={119--124},
  year={2021},
  organization={IEEE},
  url={https://arxiv.org/abs/2103.17107}
}

@inproceedings{Savchenko_2022_CVPRW,
  author    = {Savchenko, Andrey V.},
  title     = {Video-Based Frame-Level Facial Analysis of Affective Behavior on Mobile Devices Using EfficientNets},
  booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops},
  month     = {June},
  year      = {2022},
  pages     = {2359-2366},
  url={https://arxiv.org/abs/2103.17107}
}

@inproceedings{Savchenko_2022_ECCVW,
  author    = {Savchenko, Andrey V.},
  title     = {{MT-EmotiEffNet} for Multi-task Human Affective Behavior Analysis and Learning from Synthetic Data},
  booktitle = {Proceedings of the European Conference on Computer Vision (ECCV 2022) Workshops},
  pages={45--59},
  year={2023},
  organization={Springer},
  url={https://arxiv.org/abs/2207.09508}
}

@article{savchenko2022classifying,
  title={Classifying emotions and engagement in online learning based on a single facial expression recognition neural network},
  author={Savchenko, Andrey V and Savchenko, Lyudmila V and Makarov, Ilya},
  journal={IEEE Transactions on Affective Computing},
  year={2022},
  publisher={IEEE},
  url={https://ieeexplore.ieee.org/document/9815154}
}