Output Format

The two projects that process data are FaceLandmarkImg and FeatureExtraction and their output is explained in detail in this page. FaceLandmarkImg is intended for individual images and FeatureExtraction for sequence (image sequence/webcam/video) analysis that contain a single face, while FaceLandmarkVidMulti is intended for sequences containing multiple faces. Their output format is the same.

CSV file

Part of OpenFace processing outputs a comma separated value file of the following format:

frame, timestamp, confidence, success, gaze_0_x, gaze_0_y, gaze_0_z, gaze_1_x, gaze_1_y, gaze_1_z, pose_Tx, pose_Ty, pose_Tz, pose_Rx, pose_Ry, pose_Rz, x_0, x_1, x_2, x_3, x_4, x_5, x_6, x_7, x_8, x_9, x_10, x_11, x_12, x_13, x_14, x_15, x_16, x_17, x_18, x_19, x_20, x_21, x_22, x_23, x_24, x_25, x_26, x_27, x_28, x_29, x_30, x_31, x_32, x_33, x_34, x_35, x_36, x_37, x_38, x_39, x_40, x_41, x_42, x_43, x_44, x_45, x_46, x_47, x_48, x_49, x_50, x_51, x_52, x_53, x_54, x_55, x_56, x_57, x_58, x_59, x_60, x_61, x_62, x_63, x_64, x_65, x_66, x_67, y_0, y_1, y_2, y_3, y_4, y_5, y_6, y_7, y_8, y_9, y_10, y_11, y_12, y_13, y_14, y_15, y_16, y_17, y_18, y_19, y_20, y_21, y_22, y_23, y_24, y_25, y_26, y_27, y_28, y_29, y_30, y_31, y_32, y_33, y_34, y_35, y_36, y_37, y_38, y_39, y_40, y_41, y_42, y_43, y_44, y_45, y_46, y_47, y_48, y_49, y_50, y_51, y_52, y_53, y_54, y_55, y_56, y_57, y_58, y_59, y_60, y_61, y_62, y_63, y_64, y_65, y_66, y_67, X_0, X_1, X_2, X_3, X_4, X_5, X_6, X_7, X_8, X_9, X_10, X_11, X_12, X_13, X_14, X_15, X_16, X_17, X_18, X_19, X_20, X_21, X_22, X_23, X_24, X_25, X_26, X_27, X_28, X_29, X_30, X_31, X_32, X_33, X_34, X_35, X_36, X_37, X_38, X_39, X_40, X_41, X_42, X_43, X_44, X_45, X_46, X_47, X_48, X_49, X_50, X_51, X_52, X_53, X_54, X_55, X_56, X_57, X_58, X_59, X_60, X_61, X_62, X_63, X_64, X_65, X_66, X_67, Y_0, Y_1, Y_2, Y_3, Y_4, Y_5, Y_6, Y_7, Y_8, Y_9, Y_10, Y_11, Y_12, Y_13, Y_14, Y_15, Y_16, Y_17, Y_18, Y_19, Y_20, Y_21, Y_22, Y_23, Y_24, Y_25, Y_26, Y_27, Y_28, Y_29, Y_30, Y_31, Y_32, Y_33, Y_34, Y_35, Y_36, Y_37, Y_38, Y_39, Y_40, Y_41, Y_42, Y_43, Y_44, Y_45, Y_46, Y_47, Y_48, Y_49, Y_50, Y_51, Y_52, Y_53, Y_54, Y_55, Y_56, Y_57, Y_58, Y_59, Y_60, Y_61, Y_62, Y_63, Y_64, Y_65, Y_66, Y_67, Z_0, Z_1, Z_2, Z_3, Z_4, Z_5, Z_6, Z_7, Z_8, Z_9, Z_10, Z_11, Z_12, Z_13, Z_14, Z_15, Z_16, Z_17, Z_18, Z_19, Z_20, Z_21, Z_22, Z_23, Z_24, Z_25, Z_26, Z_27, Z_28, Z_29, Z_30, Z_31, Z_32, Z_33, Z_34, Z_35, Z_36, Z_37, Z_38, Z_39, Z_40, Z_41, Z_42, Z_43, Z_44, Z_45, Z_46, Z_47, Z_48, Z_49, Z_50, Z_51, Z_52, Z_53, Z_54, Z_55, Z_56, Z_57, Z_58, Z_59, Z_60, Z_61, Z_62, Z_63, Z_64, Z_65, Z_66, Z_67, p_scale, p_rx, p_ry, p_rz, p_tx, p_ty, p_0, p_1, p_2, p_3, p_4, p_5, p_6, p_7, p_8, p_9, p_10, p_11, p_12, p_13, p_14, p_15, p_16, p_17, p_18, p_19, p_20, p_21, p_22, p_23, p_24, p_25, p_26, p_27, p_28, p_29, p_30, p_31, p_32, p_33, AU01_r, AU02_r, AU04_r, AU05_r, AU06_r, AU09_r, AU10_r, AU12_r, AU14_r, AU15_r, AU17_r, AU20_r, AU25_r, AU26_r, AU04_c, AU12_c, AU15_c, AU23_c, AU28_c, AU45_c
1, 0, 0.94848, 1, 0.10917, 0.147619, -0.983001, -0.166114, 0.136956, -0.97655, 21.184, -4.04434, 422.08, 0.0528253, -0.0474321, 0.0169401, 129.32, 129.568, 130.737, 132.621, 136.572, 143.131, 151.472, 161.39, 172.219, 182.459, 191.094, 198.564, 204.427, 208.15, 210.221, 211.544, 212.038, 137.679, 143.604, 151.613, 159.659, 166.93, 180.113, 187.491, 194.915, 202.123, 206.965, 173.757, 173.999, 174.255, 174.513, 164.595, 169.096, 173.678, 178.155, 182.141, 146.832, 151.662, 157.183, 161.887, 156.979, 151.443, 183.759, 188.823, 194.114, 198.327, 194.444, 189.226, 155.318, 162.665, 169.103, 173.317, 178.143, 183.948, 189.568, 183.938, 178.302, 173.212, 168.623, 162.257, 157.973, 168.889, 173.214, 178.095, 187.128, 178.053, 173.151, 168.76, 96.3795, 107.458, 118.622, 129.446, 139.556, 148.479, 155.831, 161.454, 163.038, 161.598, 156.413, 149.47, 140.829, 130.685, 119.815, 108.843, 97.7375, 88.3902, 83.379, 82.0184, 83.1791, 86.0683, 85.8879, 83.1019, 82.3679, 84.0608, 89.0695, 93.5734, 100.678, 107.762, 115.118, 120.167, 121.755, 123.095, 121.833, 120.453, 94.8765, 92.2265, 92.3677, 95.7136, 96.5624, 96.597, 95.9942, 92.5987, 92.7628, 95.4758, 96.9835, 96.9262, 133.507, 131.346, 130.399, 131.494, 130.532, 131.774, 133.82, 138.237, 140.029, 140.445, 140.003, 138.113, 133.83, 133.706, 134.202, 133.716, 134.111, 135.027, 135.524, 135.051, -58.3457, -57.8686, -55.7015, -52.0705, -44.2213, -31.3873, -15.4704, 2.43908, 21.2727, 39.7591, 57.0255, 72.6207, 84.8269, 92.2786, 96.0504, 98.3322, 99.4596, -39.2106, -28.2826, -14.2539, -0.571417, 11.4866, 33.4085, 46.1934, 59.6174, 73.1567, 83.0303, 22.8051, 22.7966, 22.7769, 22.7809, 7.55478, 14.8233, 22.1575, 29.6144, 36.4299, -22.7817, -14.3064, -4.82294, 3.22773, -5.15778, -14.6417, 40.7882, 49.5531, 58.8734, 66.6969, 59.3376, 50.1574, -8.06112, 4.43767, 14.947, 21.8374, 29.8506, 40.1354, 51.1408, 40.0907, 30.0283, 21.5851, 14.1216, 3.76024, -3.46833, 14.6581, 21.7447, 29.9099, 46.5786, 29.7515, 21.5704, 14.4138, -44.9202, -23.849, -2.62326, 17.9654, 36.9141, 52.9898, 65.0022, 72.7413, 74.9274, 73.6421, 66.7804, 55.4954, 39.7695, 20.4769, -0.353364, -21.2845, -42.5497, -55.527, -63.1692, -64.5493, -61.6804, -56.2403, -56.6602, -62.0003, -64.2568, -62.4176, -54.6825, -43.8063, -31.4651, -19.5541, -7.66358, 0.274791, 2.86038, 5.01436, 2.98958, 0.745632, -43.4648, -47.6557, -47.3021, -41.5453, -40.0151, -40.0447, -41.2118, -47.1081, -47.0056, -42.6776, -39.6515, -39.5986, 23.2559, 18.8953, 17.0763, 18.8488, 17.3286, 19.7329, 23.9027, 30.5422, 32.8614, 33.4028, 32.7583, 30.1806, 23.664, 22.6006, 23.3707, 22.6715, 24.2276, 24.7644, 25.4641, 24.7658, 475.436, 475.394, 475.868, 475.46, 471.895, 465.17, 453.538, 438.689, 435.239, 442.584, 458.493, 470.782, 477.338, 479.125, 478.136, 476.929, 477.818, 439.159, 431.236, 424.872, 418.787, 414.364, 415.25, 420.077, 426.875, 434.189, 441.978, 414.415, 407.107, 399.446, 392.419, 411.019, 407.402, 404.993, 407.801, 411.345, 432.512, 428.968, 427.96, 427.66, 426.826, 427.773, 429.186, 429.799, 431.446, 435.057, 430.685, 429.042, 430.446, 416.351, 410.516, 409.959, 411.321, 418.989, 432.401, 418.688, 410.179, 408.44, 409.407, 416.563, 427.753, 412.239, 411.386, 413.232, 429.244, 411.998, 410.066, 411.369, 0.579649, 0.0810897, -0.0438177, 0.0165988, 172.634, 117.523, 6.75643, 3.98344, -0.913317, 0.0951292, -7.52669, 9.41583, -0.982634, -0.479738, 1.05969, -0.755182, -1.9985, -0.206534, -2.59672, -0.208004, 0.222584, 0.116356, 0.133241, 0.994562, 0.0731596, -1.20472, -0.238291, -0.344064, -0.00294826, 0.0520344, -0.0141846, 0.030905, -0.269735, -0.0120041, 0.00161123, -0.00588343, 0.00292991, -0.0331269, -0.0246774, -0.0311158,0,0,0,0,0,0,0,0,0.552199,0,0.781291,0,0,0,0,1,0,0,0,0
2, 0.04, 0.953299, 1, 0.107503, 0.150627, -0.982728, -0.165798, 0.140331, -0.976124, 21.3134, -4.05283, 421.827, 0.0619525, -0.0490535, 0.0168232, 129.336, 129.567, 130.75, 132.65, 136.563, 143.075, 151.445, 161.462, 172.38, 182.676, 191.325, 198.773, 204.549, 208.192, 210.265, 211.618, 212.133, 137.856, 143.729, 151.727, 159.777, 167.044, 180.122, 187.493, 194.902, 202.075, 206.89, 173.806, 174.052, 174.313, 174.572, 164.818, 169.246, 173.749, 178.158, 182.088, 146.974, 151.765, 157.282, 161.952, 157.09, 151.557, 183.829, 188.841, 194.121, 198.291, 194.443, 189.236, 155.974, 163.107, 169.344, 173.436, 178.093, 183.736, 189.175, 183.746, 178.285, 173.366, 168.913, 162.75, 158.641, 169.142, 173.336, 178.049, 186.737, 178.019, 173.289, 169.038, 95.9927, 107.028, 118.141, 128.947, 139.12, 148.149, 155.628, 161.367, 162.976, 161.47, 156.173, 149.133, 140.443, 130.307, 119.476, 108.535, 97.4481, 88.2293, 83.2484, 81.9124, 83.1303, 86.0446, 85.8819, 83.0908, 82.3074, 83.9467, 88.8832, 93.5991, 100.657, 107.693, 115.004, 120.21, 121.774, 123.079, 121.855, 120.476, 94.8033, 92.1472, 92.2812, 95.6349, 96.5423, 96.5816, 95.9065, 92.5106, 92.6841, 95.3944, 96.9555, 96.896, 134.007, 131.723, 130.671, 131.765, 130.799, 132.118, 134.269, 138.531, 140.243, 140.649, 140.224, 138.43, 134.267, 134.042, 134.509, 134.044, 134.517, 135.211, 135.696, 135.234, -58.2086, -57.8092, -55.6541, -52.0168, -44.2455, -31.5006, -15.5284, 2.56799, 21.5799, 40.1942, 57.4893, 73.0478, 85.0984, 92.3773, 96.1133, 98.3753, 99.4681, -38.8064, -27.9958, -14.0245, -0.372126, 11.6562, 33.38, 46.1293, 59.5017, 72.95, 82.7599, 22.8533, 22.8575, 22.851, 22.8607, 7.91679, 15.0663, 22.2783, 29.6165, 36.3292, -22.4956, -14.1035, -4.64386, 3.33442, -4.96069, -14.4253, 40.8605, 49.5092, 58.8087, 66.5768, 59.2829, 50.121, -6.91503, 5.16541, 15.3299, 22.019, 29.7463, 39.7302, 50.3622, 39.7334, 30.011, 21.8481, 14.6001, 4.57635, -2.319, 15.0647, 21.9356, 29.8184, 45.8087, 29.6861, 21.7936, 14.8643, -45.5718, -24.6405, -3.53652, 17.0162, 36.0957, 52.39, 64.6725, 72.6753, 74.9137, 73.5076, 66.387, 54.8869, 39.0508, 19.7564, -1.00258, -21.8502, -43.0288, -55.6755, -63.236, -64.5682, -61.6266, -56.187, -56.5968, -61.9274, -64.2598, -62.5092, -54.9206, -43.7003, -31.4643, -19.6496, -7.83844, 0.344816, 2.89131, 4.98971, 3.02566, 0.783229, -43.5146, -47.6997, -47.3576, -41.6108, -39.989, -40.0125, -41.3143, -47.1893, -47.0799, -42.7818, -39.6634, -39.608, 24.056, 19.4857, 17.5062, 19.2801, 17.7546, 20.2844, 24.6317, 31.007, 33.2248, 33.7511, 33.1268, 30.6736, 24.3459, 23.1395, 23.8642, 23.2014, 24.872, 25.0598, 25.7411, 25.0555, 474.563, 474.886, 475.676, 475.469, 471.954, 465.285, 453.8, 439.211, 435.789, 443.135, 458.818, 470.999, 477.556, 479.211, 478.032, 476.455, 476.996, 438.105, 430.158, 423.814, 417.867, 413.682, 414.712, 419.458, 426.209, 433.45, 441.246, 413.815, 406.66, 399.141, 392.209, 410.76, 407.373, 405.085, 407.758, 411.182, 431.749, 428.141, 427.126, 426.951, 426.182, 427.149, 428.686, 429.158, 430.884, 434.676, 430.292, 428.583, 429.35, 415.56, 410.144, 409.697, 411.012, 418.464, 431.546, 418.321, 410.322, 408.639, 409.504, 416.087, 426.62, 411.968, 411.198, 413.014, 428.331, 411.867, 409.997, 411.171, 0.580147, 0.0901969, -0.0454037, 0.0164686, 172.717, 117.509, 7.10181, 4.50381, -4.01826, 1.54338, -8.07686, 7.79325, 0.242849, -0.430568, 2.11593, -0.569905, -1.95438, -0.186302, -2.62639, -0.66344, 0.287365, 0.0960486, 0.060306, 1.01928, 0.0387424, -1.19855, -0.197185, -0.298684, -0.0910158, -0.125318, -0.0463423, 0.0533237, -0.265746, -0.001024, 0.000431798, -0.0062599, -0.00415493, -0.0304479, -0.0233709, -0.0273382,0,0,0,0,0,0,0,0,0.915043,0,0.829461,0,0,0,1,1,0,0,0,0
3, 0.08, 0.955743, 1, 0.10642, 0.153098, -0.982464, -0.165396, 0.142754, -0.975841, 20.9377, -3.801, 420.152, 0.077066, -0.0420931, 0.0169737, 129.401, 129.594, 130.767, 132.676, 136.57, 143.019, 151.292, 161.226, 172.174, 182.559, 191.292, 198.798, 204.585, 208.236, 210.338, 211.708, 212.235, 137.757, 143.558, 151.557, 159.649, 166.934, 179.846, 187.29, 194.788, 202.024, 206.814, 173.596, 173.791, 174.003, 174.215, 164.574, 168.959, 173.436, 177.864, 181.811, 146.849, 151.61, 157.125, 161.785, 156.941, 151.406, 183.701, 188.703, 193.995, 198.159, 194.303, 189.084, 155.877, 162.898, 169.084, 173.14, 177.757, 183.416, 188.87, 183.418, 177.926, 173.048, 168.628, 162.526, 158.533, 168.889, 173.044, 177.718, 186.426, 177.674, 172.983, 168.774, 95.6202, 106.721, 117.895, 128.763, 139.03, 148.158, 155.763, 161.625, 163.27, 161.7, 156.224, 148.998, 140.187, 130.005, 119.171, 108.22, 97.0928, 88.0227, 83.0285, 81.6829, 82.9415, 85.9184, 85.7633, 82.9075, 82.0903, 83.7578, 88.7132, 93.5322, 100.648, 107.74, 115.111, 120.397, 121.981, 123.297, 122.083, 120.68, 94.6611, 91.9945, 92.1351, 95.5051, 96.4325, 96.4753, 95.7805, 92.3638, 92.5401, 95.2671, 96.8642, 96.7984, 134.514, 132.31, 131.265, 132.377, 131.409, 132.715, 134.797, 139.187, 140.946, 141.35, 140.907, 139.043, 134.785, 134.619, 135.093, 134.633, 135.065, 135.898, 136.382, 135.898, -57.7963, -57.5128, -55.4276, -51.8225, -44.1074, -31.5218, -15.7691, 2.1492, 21.1698, 39.8704, 57.2123, 72.7664, 84.7293, 91.9476, 95.6769, 97.8994, 98.9726, -38.7815, -28.1283, -14.2234, -0.582882, 11.3971, 32.7018, 45.4661, 58.8913, 72.3502, 82.0745, 22.3586, 22.2893, 22.2157, 22.161, 7.47888, 14.5276, 21.6631, 28.9891, 35.6869, -22.5903, -14.2881, -4.88292, 3.03171, -5.18665, -14.6055, 40.3788, 48.9382, 58.1852, 65.9022, 58.6493, 49.5365, -7.05484, 4.79916, 14.8453, 21.4504, 29.0745, 39.0269, 49.6099, 39.047, 29.3435, 21.2744, 14.0979, 4.19211, -2.49391, 14.595, 21.3746, 29.1584, 45.0794, 29.0195, 21.2248, 14.3849, -46.049, -25.1181, -3.99074, 16.6187, 35.8241, 52.2684, 64.7628, 72.9841, 75.2452, 73.6986, 66.2311, 54.3871, 38.3621, 19.0716, -1.57547, -22.3033, -43.4037, -55.7537, -63.2509, -64.5494, -61.5294, -56.0224, -56.4147, -61.7967, -64.1762, -62.3953, -54.8522, -43.5264, -31.2778, -19.4503, -7.62195, 0.649278, 3.21165, 5.31534, 3.37995, 1.11266, -43.5276, -47.692, -47.3305, -41.5955, -39.9576, -39.9819, -41.2621, -47.1186, -46.9998, -42.7146, -39.5565, -39.5174, 24.8362, 20.3861, 18.4102, 20.2054, 18.6809, 21.1924, 25.4271, 31.9927, 34.2885, 34.8108, 34.1632, 31.6067, 25.1329, 24.0016, 24.7326, 24.0811, 25.6989, 26.1037, 26.7825, 26.0654, 472.205, 472.875, 474.009, 474.141, 470.63, 464.069, 452.717, 438.346, 434.746, 441.841, 457.091, 468.883, 475.095, 476.549, 475.169, 473.324, 473.69, 435.885, 427.701, 421.153, 415.082, 410.943, 411.946, 416.504, 423.218, 430.406, 438.301, 411.126, 404.058, 396.624, 389.759, 408.75, 405.374, 403.087, 405.693, 409.045, 429.454, 425.738, 424.643, 424.533, 423.864, 424.892, 425.918, 426.24, 427.894, 431.76, 427.438, 425.804, 427.791, 414.009, 408.56, 408.111, 409.332, 416.673, 429.592, 416.845, 409.241, 407.624, 408.511, 414.931, 424.986, 410.466, 409.675, 411.426, 426.47, 410.485, 408.706, 409.877, 0.58296, 0.104298, -0.037298, 0.0164906, 172.547, 117.629, 5.15587, 4.96086, -4.71681, 3.75421, -7.39546, 7.4205, 0.815089, -0.930302, 3.05524, 0.635019, -1.79681, -0.262913, -2.66956, -1.19047, -0.168866, 0.441451, 0.0106802, 0.836421, -0.0732082, -1.31846, -0.009313, -0.427927, -0.152835, -0.169329, 0.00582317, -0.00231911, -0.302436, -0.00821701, -0.00528087, 0.00686356, -0.0275763, -0.0160663, -0.0206939, -0.0243734,0,0,0,0.585777,0,0,0,0,0.897447,0,0,0,0,0,0,1,0,0,0,0
4, 0.12, 0.963871, 1, 0.106936, 0.154229, -0.982231, -0.164536, 0.145976, -0.97551, 20.4166, -3.87284, 418.836, 0.0831857, -0.0378122, 0.0119711, 129.144, 129.391, 130.663, 132.677, 136.624, 143.065, 151.28, 161.147, 172.084, 182.496, 191.261, 198.78, 204.543, 208.15, 210.23, 211.575, 212.05, 137.348, 143.094, 151.069, 159.147, 166.416, 179.388, 186.845, 194.366, 201.622, 206.425, 173.146, 173.35, 173.57, 173.794, 164.306, 168.655, 173.097, 177.508, 181.441, 146.413, 151.144, 156.68, 161.364, 156.515, 150.961, 183.329, 188.301, 193.62, 197.804, 193.941, 188.694, 155.989, 162.766, 168.859, 172.893, 177.457, 183.089, 188.434, 183.124, 177.679, 172.837, 168.435, 162.43, 158.656, 168.69, 172.813, 177.439, 185.967, 177.405, 172.753, 168.574, 95.3783, 106.561, 117.817, 128.737, 139.072, 148.242, 155.899, 161.8, 163.445, 161.821, 156.208, 148.816, 139.888, 129.629, 118.745, 107.729, 96.5486, 87.8055, 82.8152, 81.471, 82.7331, 85.6975, 85.5142, 82.6243, 81.7406, 83.3467, 88.2501, 93.3341, 100.469, 107.574, 114.96, 120.346, 121.903, 123.189, 121.972, 120.548, 94.4962, 91.7979, 91.9157, 95.2878, 96.2555, 96.3212, 95.4683, 92.021, 92.1838, 94.8937, 96.5396, 96.4907, 134.93, 132.47, 131.274, 132.386, 131.388, 132.787, 135.047, 139.58, 141.426, 141.852, 141.42, 139.522, 135.15, 134.745, 135.186, 134.732, 135.304, 136.217, 136.719, 136.244, -58.0438, -57.7087, -55.4857, -51.7169, -43.9191, -31.3727, -15.7612, 2.0087, 20.9785, 39.6772, 56.9978, 72.5033, 84.3436, 91.4036, 95.0215, 97.1298, 98.0419, -39.3322, -28.8012, -14.9813, -1.41038, 10.5049, 31.8226, 44.5371, 57.9182, 71.3196, 80.9994, 21.5338, 21.4946, 21.453, 21.4328, 7.01797, 13.9888, 21.0505, 28.315, 34.9537, -23.2538, -15.0221, -5.6169, 2.30756, -5.88643, -15.3085, 39.5821, 48.0401, 57.2876, 65.0072, 57.7844, 48.6685, -6.83714, 4.56332, 14.4231, 20.9665, 28.4698, 38.3203, 48.6254, 38.432, 28.8791, 20.8921, 13.7578, 4.02238, -2.27471, 14.221, 20.9252, 28.5985, 44.0923, 28.4991, 20.7979, 14.0224, -46.3159, -25.3367, -4.128, 16.5375, 35.8332, 52.3181, 64.8855, 73.184, 75.4207, 73.7597, 66.0177, 53.8737, 37.6595, 18.2782, -2.37357, -23.1091, -44.1732, -55.9018, -63.3495, -64.6288, -61.615, -56.161, -56.6028, -62.0088, -64.4808, -62.8062, -55.3957, -43.6786, -31.4468, -19.644, -7.83114, 0.563514, 3.07607, 5.12585, 3.18886, 0.89298, -43.6502, -47.8403, -47.5113, -41.7962, -40.1093, -40.1018, -41.6224, -47.4935, -47.3978, -43.1723, -39.9416, -39.8749, 25.4465, 20.5698, 18.3556, 20.1422, 18.5721, 21.2223, 25.733, 32.5414, 35.0005, 35.5642, 34.9361, 32.3131, 25.6484, 24.13, 24.8013, 24.1595, 25.9861, 26.5539, 27.2651, 26.5642, 470.274, 471.331, 472.833, 473.199, 469.711, 463.123, 451.86, 437.701, 433.999, 440.928, 455.82, 467.397, 473.388, 474.575, 472.933, 470.82, 470.901, 434.094, 425.91, 419.352, 413.336, 409.306, 410.335, 414.766, 421.339, 428.38, 436.188, 409.5, 402.532, 395.218, 388.432, 407.42, 404.087, 401.832, 404.316, 407.564, 427.879, 424.085, 422.935, 422.83, 422.301, 423.394, 424.17, 424.368, 425.991, 429.895, 425.628, 424.033, 426.107, 412.374, 407.018, 406.555, 407.71, 414.922, 427.534, 415.492, 408.386, 406.883, 407.754, 413.801, 423.254, 409.126, 408.293, 409.981, 424.509, 409.361, 407.693, 408.843, 0.585041, 0.110759, -0.0323022, 0.0113965, 172.276, 117.569, 3.39147, 5.48401, -5.82276, 5.7464, -7.71578, 5.88891, 1.78466, -0.592173, 3.80212, 0.150893, -1.38067, -0.384341, -2.18357, -1.15011, -0.0751731, 0.465033, 0.123919, 0.467066, -0.0636545, -1.50489, 0.0495619, -0.382291, -0.102015, -0.140306, 0.0301354, 0.0119626, -0.296017, 0.00944414, -0.00441807, 0.00994939, -0.03512, -0.0213233, -0.0229672, -0.0172276,0,0,0,0.761611,0,0,0,0,0.859176,0,0,0,0,0,1,1,0,0,0,1
...
...
204, 8.12, 0.952574, 1, 0.121502, 0.17169, -0.97763, -0.143931, 0.155402, -0.97731, 12.8349, -5.59191, 410.328, 0.107749, -0.058971, 0.0502874, 124.423, 124.206, 125.189, 127.019, 130.81, 136.986, 144.871, 154.67, 165.76, 176.364, 185.205, 192.765, 198.453, 202.191, 204.701, 206.547, 207.38, 134.016, 140.133, 148.326, 156.506, 163.684, 176.924, 184.35, 191.732, 198.696, 202.84, 170.352, 170.333, 170.369, 170.413, 160.385, 164.655, 169.032, 173.437, 177.298, 142.986, 147.962, 153.561, 158.117, 153.19, 147.555, 179.885, 185.047, 190.369, 194.303, 190.481, 185.208, 151.2, 157.83, 164.439, 168.367, 172.788, 178.289, 182.76, 177.82, 172.299, 167.523, 163.095, 156.868, 153.929, 164.094, 168.118, 172.56, 180.245, 172.316, 167.75, 163.613, 90.7048, 102.714, 114.894, 126.637, 137.949, 148.032, 156.631, 163.26, 165.364, 163.631, 157.274, 148.879, 139.271, 128.646, 117.732, 106.652, 95.4064, 84.1519, 79.2084, 78.1078, 79.7568, 83.1749, 83.5542, 80.9658, 80.2812, 82.0667, 87.1396, 91.0586, 98.2875, 105.457, 112.959, 118.162, 119.885, 121.381, 120.315, 118.953, 91.1784, 88.6154, 88.9671, 92.5478, 93.3091, 93.1254, 93.5461, 90.3313, 90.7103, 93.5923, 95.0917, 94.802, 134.062, 130.64, 129.237, 130.557, 129.695, 131.766, 135.318, 141.588, 144.028, 144.359, 143.74, 140.793, 134.273, 133.083, 133.62, 133.43, 135.495, 138.078, 138.506, 137.853, -64.8277, -65.5949, -64.161, -60.9109, -53.5648, -41.7385, -26.877, -9.2042, 9.87829, 28.5026, 45.2353, 60.0876, 71.1772, 78.048, 82.2624, 85.2714, 86.7794, -43.6395, -32.8012, -19.0088, -5.62194, 5.88637, 27.2162, 39.6153, 52.4579, 65.0991, 73.379, 16.5598, 16.2532, 16.0267, 15.8265, 0.612278, 7.35013, 14.1969, 21.2636, 27.6129, -28.2466, -19.8299, -10.5916, -3.09777, -11.1908, -20.4795, 33.0152, 41.5994, 50.6684, 57.7878, 50.8295, 41.8479, -14.6597, -3.49515, 7.05118, 13.2859, 20.3825, 29.6938, 38.1083, 29.1565, 19.8406, 12.0906, 4.97844, -5.09143, -10.0578, 6.55086, 12.9695, 20.1658, 33.6984, 19.8808, 12.4544, 5.81502, -53.3805, -31.6783, -9.41097, 12.2573, 32.9367, 50.8391, 65.0743, 74.7071, 77.7971, 75.9963, 66.8945, 52.9602, 35.6703, 15.9935, -4.17446, -24.4526, -45.0452, -60.2068, -67.3474, -68.2115, -64.7472, -58.8435, -58.6091, -63.5065, -65.6617, -63.8166, -56.2858, -46.296, -34.1534, -22.4783, -10.7005, -2.92296, -0.18142, 2.17019, 0.499203, -1.67149, -47.8494, -51.6986, -51.0428, -45.1676, -43.8579, -44.2264, -43.9206, -49.2762, -48.8674, -44.4865, -41.5367, -41.8321, 23.4261, 17.136, 14.6729, 16.7639, 15.4535, 19.1032, 25.6484, 35.3208, 38.7598, 39.147, 38.1877, 33.8, 23.6442, 20.9363, 21.7583, 21.5635, 25.7923, 29.1829, 29.7385, 28.7325, 455.539, 458.139, 460.781, 461.714, 458.764, 453.398, 444.118, 431.729, 428.738, 435.451, 448.667, 458.469, 462.749, 462.473, 460.067, 457.984, 457.895, 419.875, 412.753, 407.066, 402.224, 399.48, 402.029, 406.735, 413.292, 420.584, 428.219, 399.911, 393.246, 386.418, 379.956, 397.658, 394.762, 392.961, 395.618, 399.086, 415.048, 411.815, 411.2, 411.329, 410.794, 411.415, 415.066, 415.22, 417.104, 421.152, 416.896, 415.034, 416.489, 402.647, 397.135, 396.991, 398.476, 405.901, 418.595, 409.035, 403.281, 401.767, 402.138, 406.381, 414.142, 400.053, 399.388, 401.396, 416.129, 403.568, 401.731, 402.343, 0.598208, 0.136351, -0.053165, 0.0496707, 167.876, 116.389, -13.2689, 13.6493, -1.02892, 14.3784, -10.5895, -2.9426, 6.2489, -2.82006, 5.56716, -3.2069, 0.360861, -0.513787, -1.28229, -0.941657, 1.25147, 0.0729805, 1.28252, 0.780729, -0.533899, -0.698888, 0.809073, -0.755656, 0.0321429, 0.0375847, 0.10858, 0.308759, 0.0772421, -0.0557184, -0.0312446, -0.00364208, -0.0493718, -0.00840708, -0.013717, 0.00441885,0,0,0,0,0,0.583841,0,0,0,0,0,0,2.76711,0.790546,0,0,0,1,0,0
205, 8.16, 0.943502, 1, 0.119735, 0.171356, -0.977906, -0.141987, 0.154965, -0.977663, 13.3425, -6.40826, 410.029, 0.0806764, -0.0697097, 0.0482961, 124.486, 124.252, 125.217, 127.038, 130.833, 137.022, 144.961, 154.903, 166.055, 176.638, 185.385, 192.861, 198.514, 202.261, 204.773, 206.598, 207.407, 134.295, 140.436, 148.657, 156.836, 163.983, 177.144, 184.535, 191.864, 198.783, 202.935, 170.605, 170.671, 170.785, 170.901, 160.762, 165.072, 169.495, 173.902, 177.752, 143.257, 148.209, 153.793, 158.343, 153.444, 147.829, 180.137, 185.263, 190.544, 194.433, 190.682, 185.455, 151.681, 158.442, 164.953, 168.876, 173.295, 178.673, 183.105, 178.271, 172.891, 168.138, 163.741, 157.567, 154.428, 164.603, 168.626, 173.064, 180.638, 172.853, 168.306, 164.182, 91.0214, 102.987, 115.135, 126.847, 138.101, 148.116, 156.591, 163.054, 165.089, 163.481, 157.366, 149.133, 139.554, 128.863, 117.816, 106.614, 95.2744, 83.855, 78.8175, 77.6544, 79.2258, 82.5482, 82.8669, 80.3027, 79.6619, 81.5165, 86.6856, 90.3551, 97.574, 104.74, 112.233, 117.593, 119.307, 120.781, 119.707, 118.348, 90.8022, 88.1684, 88.4913, 92.027, 92.8203, 92.6803, 92.9791, 89.7532, 90.1293, 93.0418, 94.4966, 94.2085, 133.686, 130.224, 128.796, 130.083, 129.227, 131.271, 134.808, 140.566, 142.765, 143.083, 142.495, 139.836, 133.868, 132.578, 133.092, 132.893, 134.958, 137.01, 137.427, 136.802, -64.7382, -65.4541, -63.9829, -60.708, -53.3381, -41.5021, -26.5929, -8.75591, 10.328, 28.8486, 45.416, 60.1852, 71.3281, 78.3458, 82.6714, 85.6934, 87.2226, -43.2659, -32.3908, -18.5333, -5.10962, 6.39056, 27.6794, 40.0855, 52.9093, 65.5315, 73.8434, 17.0189, 16.8272, 16.6984, 16.5816, 1.21236, 8.00656, 14.921, 22.0026, 28.3581, -27.8489, -19.4695, -10.2356, -2.73267, -10.7948, -20.0668, 33.5316, 42.1028, 51.1382, 58.1974, 51.3232, 42.3829, -13.8222, -2.50517, 7.8642, 14.0885, 21.1879, 30.3096, 38.654, 29.8141, 20.7193, 13.0256, 5.99305, -3.93975, -9.20443, 7.35589, 13.7624, 20.9548, 34.3077, 20.6938, 13.3089, 6.71097, -52.8242, -31.1508, -8.94923, 12.6099, 33.1017, 50.7809, 64.7007, 73.9557, 76.911, 75.3902, 66.8523, 53.3569, 36.2136, 16.4308, -4.03298, -24.6165, -45.4914, -60.8382, -68.1848, -69.1889, -65.8564, -60.0833, -59.9537, -64.8587, -66.9801, -65.0251, -57.2969, -47.5757, -35.3623, -23.6268, -11.8157, -3.8292, -1.09356, 1.22785, -0.464457, -2.63968, -48.5654, -52.5601, -51.9563, -46.1358, -44.7531, -45.0424, -44.9936, -50.408, -50.0112, -45.564, -42.6612, -42.9431, 22.7395, 16.4402, 13.9663, 16.0044, 14.7051, 18.2949, 24.7743, 33.5577, 36.5908, 36.9477, 36.0386, 32.118, 22.9081, 20.099, 20.889, 20.6815, 24.865, 27.3873, 27.9229, 26.9612, 455.718, 457.749, 459.875, 460.438, 457.173, 451.532, 442.055, 429.433, 426.435, 433.468, 447.274, 457.881, 463.006, 463.463, 461.618, 459.752, 459.962, 420.793, 413.919, 408.478, 403.787, 401.071, 403.641, 408.458, 415.117, 422.422, 429.971, 401.213, 394.212, 387.066, 380.294, 397.679, 394.668, 392.847, 395.674, 399.36, 415.831, 412.798, 412.237, 412.325, 411.641, 412.179, 416.285, 416.64, 418.564, 422.543, 418.192, 416.253, 415.375, 402.005, 396.972, 396.835, 398.429, 405.789, 418.246, 407.933, 401.833, 400.164, 400.515, 404.794, 412.97, 399.495, 398.885, 401.007, 415.585, 402.523, 400.559, 401.158, 0.598085, 0.11082, -0.0664424, 0.0479842, 168.181, 115.939, -11.9401, 12.1435, -2.53836, 14.045, -11.2765, -0.732664, 6.17934, -3.12388, 5.21925, -3.14636, 0.102961, -0.0954346, -1.78584, -1.12418, 1.58407, -0.0888273, 0.826367, 0.700256, -0.351087, -0.623023, 0.66073, -0.450211, -0.0178976, -0.175885, 0.205149, 0.308204, 0.0731635, -0.0274786, -0.0243863, -0.00930196, -0.0267284, -0.0130051, -0.0129202, 0.000738565,0.720321,0,0,0.522199,0,0.992203,0,0,0,0,0,0,3.04877,0.744715,1,0,0,1,0,0

The header specifies the meaning of each column. The explanation of each:

Basic

frame the number of the frame (in case of sequences)

face_id the face id (in case of multiple faces), there is no guarantee that this is consistent across frames in case of FaceLandmarkVidMulti, especially in longer sequences

timestamp the timer of video being processed in seconds (in case of sequences)

confidence how confident is the tracker in current landmark detection estimage

success is the track successful (is there a face in the frame or do we think we tracked it well)

Gaze related

gaze_0_x, gaze_0_y, gaze_0_z Eye gaze direction vector in world coordinates for eye 0 (normalized), eye 0 is the leftmost eye in the image (think of it as a ray going from the left eye in the image in the direction of the eye gaze)

gaze_1_x, gaze_1_y, gaze_1_z Eye gaze direction vector in world coordinates for eye 1 (normalized), eye 1 is the rightmost eye in the image (think of it as a ray going from the right eye in the image in the direction of the eye gaze)

gaze_angle_x, gaze_angle_y Eye gaze direction in radians in world coordinates averaged for both eyes and converted into more easy to use format than gaze vectors. If a person is looking left-right this will results in the change of gaze_angle_x (from positive to negative) and, if a person is looking up-down this will result in change of gaze_angle_y (from negative to positive), if a person is looking straight ahead both of the angles will be close to 0 (within measurement error).

eye_lmk_x_0, eye_lmk_x_1,... eye_lmk_x55, eye_lmk_y_1,... eye_lmk_y_55 location of 2D eye region landmarks in pixels. The landmark index can be found below

eye_lmk_X_0, eye_lmk_X_1,... eye_lmk_X55, eye_lmk_Y_0,... eye_lmk_Z_55 location of 3D eye region landmarks in millimeters. The landmark index can be found below

Eye landmark markup

Pose

pose_Tx, pose_Ty, pose_Tz the location of the head with respect to camera in millimeters (positive Z is away from the camera)

pose_Rx, pose_Ry, pose_Rz Rotation is in radians around X,Y,Z axes with the convention R = Rx * Ry * Rz, left-handed positive sign. This can be seen as pitch (Rx), yaw (Ry), and roll (Rz). The rotation is in world coordinates with camera being the origin.

Lines in au intensities and au occurrences correspond to predicted Action Unit presence and intensities respectively. For more details see here

Where the landmarks are no longer in pixel values but in millimetres and we also report head pose and gaze (this however needs accurate estimates of fx,fy,cx,cy. This functionality is useful for batch image processing where the camera is the same and we want to know pose and gaze.

Landmarks locations in 2D

x_0, x_1, ... x_66, x_67, y_0,...y_67 location of 2D landmarks in pixels, the landmark index can be seen below

Landmarks locations in 3D

X_0, ... X_67, Y_0,...Y_67, Z_0,...Z_67 location of 3D landmarks in millimetres, the landmark index can be seen below. For this to be accurate need to have good estimates for fx,fy,cx,cy

Rigid and non-rigid shape parameters

Parameters of a point distribution model (PDM) that describe the rigid face shape (location, scale and rotation) and non-rigid face shape (deformation due to expression and identity). For more details look at chapter 4.2 of my Thesis for more details.

p_scale, p_rx, p_ry, p_rz, p_tx, p_ty - scale, rotation and translation terms of the PDM

p_0, p_1, ... p_33 - non-rigid shape parameters

Facial Action Units

Facial Action Units (AUs) are a way to describe human facial expression, more details on Action Units can be found here

Note that AUs are most accurate in videos of one person if there is a range of expressions observed, they are not as accurate with FaceLandmarkImg and FaceLandmarkVidMulti

The system can detect the intensity (from 0 to 5) of 17 AUs:

AU01_r, AU02_r, AU04_r, AU05_r, AU06_r, AU07_r, AU09_r, AU10_r, AU12_r, AU14_r, AU15_r, AU17_r, AU20_r, AU23_r, AU25_r, AU26_r, AU45_r

And the presense (0 absent, 1 present) of 18 AUs:

AU01_c, AU02_c, AU04_c, AU05_c, AU06_c, AU07_c, AU09_c, AU10_c, AU12_c, AU14_c, AU15_c, AU17_c, AU20_c, AU23_c, AU25_c, AU26_c, AU28_c, AU45_c

Similarity aligned face images and HOG binary files

Sample aligned face and HOG image

The executables also allow for extraction of aligned faces (left) and HOG features (right). The HOG features are written as a binary file (for space consideration) and can be read in using the ./matlab_runners/Demos/Read_HOG_files.m script to Matlab, see ./matlab_runners/Demos/feature_extraction_demo_vid.m for an example. Once the script is finished, the HOG features are stored in hog_data variable

It is also possible to control the size of the similarity aligned faces, using the -simsize <pixel width and height> and -simscale <scale> command line arguments, you can get the following results:

Default parameters - size 112 and scale 0.7

aligned face default