This repo contains the datasets and the experiments done in the paper:
- Complexity-Aware Assignment of Latent Values in Discriminative Models for Accurate Gesture Recognition
Presented at:
- 2016 29th SIBGRAPI Conference on Graphics, Patterns and Images
Abstract. Many of the state-of-the-art algorithms for gesture recognition are based on Conditional Random Fields (CRFs). Successful approaches, such as the Latent-Dynamic CRFs, extend the CRF by ncorporating latent variables, whose values are mapped to the values of the labels. In this paper we propose a novel methodology to set the latent values according to the gesture complexity. We use an heuristic that iterates through the samples associated with each label value, estimating their complexity. We then use it to assign the latent values to the label values. We evaluate our method on the task of recognizing human gestures from video streams. The experiments were performed in binary datasets, generated by grouping different labels. Our results demonstrate that our approach outperforms the arbitrary one in many cases, increasing the accuracy by up to 10%.
For any doubts or if you want to discuss the results, feel free to contact me at manoelribeiro at dcc . ufmg . br.
The scripts run in Python 2.7, and you need pystruct, cvxopt, cython, pyqpbo and scikit-learn (which requires numpy and scipy). In Ubuntu you can install these via apt:
sudo apt-get install build-essential python-dev python-setuptools python-numpy \\
python-scipy libatlas-dev libatlas3gf-base python-cvxopt
And then install scikit-learn, pyqpbo and pystruct using pip:
pip install --user --upgrade scikit-learn pystruct pyqpbo
If you have any problems pystruct gives more detailed instructions, including for other operating systems.
The ./Dataset folders contain two datasets, that can be found here:
- The NATOPS dataset
This dataset contains three pairs of body-hand gestures used when handling aircraft on the deck of an aircraft carrier. The observation features include automatically tracked 3D body postures and hand shapes. The body feature includes 3D joint velocities for left/right elbows and wrists, and represented as a 12D feature vector. The hand feature includes probability estimates of five predefined hand shapes - opened/closed palm, thumb up/down, and "no hand". The fifth shape, no hand, was dropped in the final representation, resulting in an 8D feature vector. The dataset was sampled at 20 FPS.
- The ArmGesture dataset
This dataset includes the six arm gestures. Observation features include automatically tracked 2D joint angles and 3D euclidean coordinates for left/right shoulders and elbows; each observation is represented as a 20D feature vector. The dataset was collected from 13 participants with an average of 120 samples per class (exact sample counts per class are
[88, 117, 118, 132, 179, 90])."
Inside each of the dataset's folder, the AG.mat and the NT.mat files correspond to the original data-sets. The other files in the format X_Y{c,d}.mat contain the binarized datasets, where all the labels in X became a new label 0 and all the labels in Y, become a new label 1.