Nodoface

C++ Addon for fast and efficient facial analytics on Nodejs/TypeScript.

CCExtractor + Google Summer of Code = 💜

Nodoface is the backbone of Poor Man's Rekognition being developed under Google Summer of Code 2019 for CCExtractor Development.

Features

This Addon provides bindings to all OpenFace 2.0 features except the visualization utilities and FaceAnalyser class. Face detection can be easily run in realtime on any modern CPU. The Addon is built using node-addon-api.

Available classes from OpenFace

• ImageCapture
• SequenceCapture
• FaceDetectorMTCNN
• FaceDetectorHaar
• FaceDetectorHOG
• CLNF (LandmarkDetectorModel in OpenFace)
• LandmarkModelConfig (FaceModelParameters in OpenFace)
• GazeAnalyser (GazeAnalysis in OpenFace) [Does not work]

CLNF also provides HOG-based and Haar Cascades face detector. GazeAnalysis is a namespace with static classes in OpenFace but GazeAnalyser is a class in Nodoface.

Bindings to some OpenCV types and functions

• Image (cv::Mat)
• IntImage (cv::Mat_<int>)
• FloatImage (cv::Mat_<float>)
• drawRect() (cv::rectangle())
• drawText() (cv::putText()) (not working)
• showImage() (cv::imshow())
• readImage() (cv::imread())
• saveImage() (cv::imwrite)
• destroyWindow() (cv::destroyWindow())
• destroyAllWindows() (cv::destroyAllWindows())
• waitkey() (cv::waitKey())
• VideoCapture (cv::VideoCapture)
• VideoWriter (cv::VideoWriter) Note: All methods return RGB image instead of BGR unlike OpenCV OpenCV classes are partial bindings just sufficient for passing underlying Mat to/from nodejs and C++.

Utility functions

• Image.fromBase64() (decodes base64 string into cv::Mat)
• drawDetections() (Draws bounding boxes and texts on image)

Installation

Dependencies

• OpenFace >= 2.1.0
• OpenCV >= 3.4.0
• dlib >= 19.13
  All above libs must be compiled as a shared library
• node-addon-api >= 1.6.3

Step 1: Install OpenCV 3.4.x.
If opencv is already installed, ensure it is recognised by pkg-config by executing:

$ pkg-config --modversion opencv

Step 2: Install dlib 19.13 or greater as a shared library (default is static). Check this answer by DavisKing for compiling it as shared library.
Ensure it is recognised by pkg-config:

$ pkg-config --modversion dlib

Step 3: Install OpenFace. The original repo has some issues. It can only compile as static library. So use my OpenFace fork instead. I have modified CMakelists.txt to compile OpenFace as shared lib.

$ git clone https://github.com/sziraqui/OpenFace.git && cd OpenFace
$ git checkout dynamic-compile
$ ./download_models.sh
$ mkdir build && cd build
$ cmake -D CMAKE_BUILD_TYPE=RELEASE CMAKE_CXX_FLAGS="-std=c++11" -D CMAKE_EXE_LINKER_FLAGS="-std=c++11" 
$ make -j2
$ sudo make install

Step 4: Compile the bindings.

$ git clone https://github.com/sziraqui/nodoface.git && cd nodoface
$ npm install

Step 5 (Optional): Run tests

$ npm test

This will execute mocha tests. Some tests generate image outputs which can be checked in ./tests/outputs directory To clear all files generated by tests, execute npm run clean:test

If you encounter errors, see my gsoc notes for some assistance. The addon is exported from ./api/nodoface.js

Examples

Run some samples to test if everything is working.

$ node examples/mtcnn_on_video.js path/to/a/video/file <OpenFace_REPO_DIR>/lib/local/LandmarkDetector/model/mtcnn_detector/MTCNN_detector.txt

Here's sample run:

Mentors

• Carlos Fernandez
• Johannes Lochter

License

GNU General Public License 3.0 (GPL-v3.0)
Please carefully read OpenFace 2.0 license. You must also respect opencv and dlib license.