Facial detection
Use case: face detection on heritage images
Annotation of visages on heritage material can be useful for information retrieval scenario, digital humanities research objectives, etc.
Facial recognition, a technology capable of matching a human face from a digital image, is not covered here.
Out of the box and easy to use systems are available for face detection.
The OpenCV/dnn module can be used to try some pretrained neural network models imported from frameworks as Caffe or Tensorflow.
This Python 3 script uses dnn to call a ResNet SSD network (see this post or this notebook for details). The model can be easily downloaded from the web.
The images of a Gallica document are first loaded thanks to the IIIF protocol. The detection then occurs and annotated images are generated, as well as the CSV data. Users may play with the confidence score value and look for the impact on the detection process. A basic filter on very large (and improbable) detections is implemented.
Display the Jupyter notebook with nbviewer
Launch the notebook with Binder
A final stage of evaluation of the quality of the detection is carried out, using the intersection over union (IOU) method. A basic Python implementation is performed, processing only images that include a single face. For an effective method, see this implementation. Ground truth images are annotated thanks to the VGG Image Annotator.
The Google Cloud Vision API may be used to perform face and gender detection.
The Perl script described here calls the API to perform visual recognition of content or human faces.
First, we have to build a JSON request:
{
"requests": [{
"image": {
"content": "binary image content"
},
"features": [{
"type": "FACE_DETECTION",
"maxResults": "30"
}],
"imageContext": {
"languageHints": ["fr"]
}
}]
}
Then the API endpoint is simply called with a curl command:
curl --max-time 10 -v -s -H "Content-Type: application/json" https://vision.googleapis.com/v1/images:annotate?key=your_key --data-binary @/tmp/request.json
Note: IBM Watson no longer offers facial detection.
This script is a Python3 implementation of the MTCNN face detector for TensorFlow, based on the paper Zhang, K et al. (2016). It requires OpenCV>=3.2 and Tensorflow>=1.4.0.
See it in action on school years photos.
These deep learning systems are well known for facial detection or recognition:
-
Information Retrieval:
- GallicaPix, faces in the 1915 issues of L'Excelsior (BnF)
-
Digital humanities:
- Data analysis of newspapers front page regarding human faces use: Excelsior (1910-1920) (BnF)
- Averaging of faces: A Century of Portraits: A Visual Historical Record of American High School Yearbooks (UC Berkeley). Face detection is the first step of an averaging pipeline, which also implies detection of the facial features.
- Exploring the records of the White Australia Policy: The People Inside, The real face of White Australia (University of Canberra)
-
Digital mediation:
- Averaging of faces, WW1 faces (BnF)
- Face replacement using vintage photos: The Vintage Face Depot (Tim Sherratt)
-
Facial recognition: As averaging of faces, facial recognition generally starts with a face detection step.
- Face Recognition in the Archives (The Slovak University of Technology)
- Face detection with Google Cloud Vision
- Kaggle faces dataset
- Face detection with Detectron2
- Introduction to deep learning for face recognition
- “Deep Face Recognition: A Survey"
- Wei Liu, Dragomir Anguelov, Dumitru Erhan, Christian Szegedy, Scott Reed, Cheng-Yang Fu: “SSD: Single Shot MultiBox Detector”, 2016; arXiv:1512.02325
- Dang Ha The Hien. A guide to receptive field arithmetic for Convolutional Neural Networks
- Howard Jeremy. Lesson 9: Deep Learning Part 2 2018 - Multi-object detection
- Evaluation of object detection systems