Mother and Son

We provide the modelling of a somatic facial motor space. The variants proposed are conceived to be part of a larger system for dealing with simulation-based face emotion analysis along dual interactions.

Dependencies

After cloning the repository manually download:

• [2]. and extract the aasthana_cvpr2013_code_version_2.0.zip under libraries/landmarks extraction/AA/ folder.
• [4] and extract the toolbox-master.zip under libraries/toolbox/ folder.

If you intend to use OpenFace as landmark extractor, install the library following [5] and place the bin folder under libraries/landmarks extraction/OF/.

Demo

Run mainMotherSon, choose mother(s) data and the example son.jpg image as son's neutral image.

Features done

• Included support to mother data acquired via images.
• Included support to multiple "mothers".
• Included four landmark extraction methods (Zhu-Ramanan [1], Aashtana [2], TCDCN [3] and OpenFace [5]).
• Included webcam modality for son acquisition.
• Inserted option to handle color images.
• Code refactored using MATLAB classes.
• Faster face texture reconstruction.
• Corrected action parameters.

Features to do

• Include mother acquisition via webcam.

References

If you use this code or data, please cite the paper:

@article{boccignone2018deep,
  title={Deep construction of an affective latent space via multimodal enactment},
  author={Boccignone, Giuseppe and Conte, Donatello and Cuculo, Vittorio and D’Amelio, Alessandro and Grossi, Giuliano and Lanzarotti, Raffaella},
  journal={IEEE Transactions on Cognitive and Developmental Systems},
  volume={10},
  number={4},
  pages={865--880},
  year={2018},
  publisher={IEEE}
}

@inproceedings{d2017note,
  title={A note on modelling a somatic motor space for affective facial expressions},
  author={D’Amelio, Alessandro and Cuculo, Vittorio and Grossi, Giuliano and Lanzarotti, Raffaella and Lin, Jianyi},
  booktitle={International Conference on Image Analysis and Processing},
  pages={181--188},
  year={2017},
  organization={Springer, Cham}
}

[1]: http://www.ics.uci.edu/~xzhu/face/ "X. Zhu, D. Ramanan. "Face detection, pose estimation and landmark localization in the wild" Computer Vision and Pattern Recognition (CVPR) Providence, Rhode Island, June 2012."
[2]: http://ibug.doc.ic.ac.uk/resources/drmf-matlab-code-cvpr-2013/ "A. Asthana, S. Zafeiriou, S. Cheng and M. Pantic. Robust Discriminative Response Map Fitting with Constrained Local Models. In CVPR 2013."
[3]: http://mmlab.ie.cuhk.edu.hk/projects/TCDCN.html "Zhanpeng Zhang, Ping Luo, Chen Change Loy, Xiaoou Tang. Facial Landmark Detection by Deep Multi-task Learning, in Proceedings of European Conference on Computer Vision (ECCV), 2014"
[4]: https://pdollar.github.io/toolbox/
[5]: https://github.com/TadasBaltrusaitis/OpenFace/wiki "Baltrušaitis, T., Robinson, P., & Morency, L. P. (2016, March). Openface: an open source facial behavior analysis toolkit. In Applications of Computer Vision (WACV), 2016 IEEE Winter Conference on (pp. 1-10). IEEE."

Acknowledgement

The son demo picture /Data/images/son.jpg is by Muhammad Mahdi Karim (www.micro2macro.net) Facebook Youtube - Own work, CC BY-SA 2.5-2.0-1.0, https://commons.wikimedia.org/w/index.php?curid=4155050