Machine Learning model mapping Action Units to arkit-compatible blendshapes

This repository contains model checkpoints and data for the mapping of OpenFace action units to blendshapes directly extracted with Arkit Liveface

model.ipyng : the code, training and testing of the models
pytorch models : folder containing multiple models
training.md : file to track the performances of the models
image_for_md : image folder for training.md
au_bs_lagMeasure.ipynb : notebook that recorded the delay between action unit scripts and blendshape scripts
AU26_r_JawOpen-lagMeasure.csv : file generated by the above notebook. Each au script is associated with its bs script, the framerate for both files, the lag between both files, and the number of frames for both files
preprocessing_and_giga_dataset.ipynb : notebook to preprocess data (not available yet) and create the dataset used for my models
outliers : folder containing plotting of Action Unit 26 VS JawOpen blendshape of scripts that have a too large lag to be included in the final dataset
JawOpen_AU26_plots : folder of images containing the plot of Action Unit 26 VS JawOpen blendshape of all scripts

Name		Name	Last commit message	Last commit date
Latest commit History 13 Commits
GRUNetPack_RMSELoss_whole		GRUNetPack_RMSELoss_whole
JawOpen_AU26_corrected		JawOpen_AU26_corrected
JawOpen_AU26_plots		JawOpen_AU26_plots
bs_loss		bs_loss
image_for_md		image_for_md
loss		loss
model comparison		model comparison
outliers		outliers
pytorch models		pytorch models
whole_test_dataset		whole_test_dataset
.gitignore		.gitignore
AU26_r-JawOpen-lagMeasure.csv		AU26_r-JawOpen-lagMeasure.csv
README.md		README.md
au_bs_lagMeasure.ipynb		au_bs_lagMeasure.ipynb
convnet.ipynb		convnet.ipynb
emotalk_vs_grunet.ipynb		emotalk_vs_grunet.ipynb
high.csv		high.csv
install.md		install.md
low.csv		low.csv
model.ipynb		model.ipynb
model.py		model.py
muffled.csv		muffled.csv
normal.csv		normal.csv
openface.sh		openface.sh
plot_data.ipynb		plot_data.ipynb
preprocessing_and_giga_dataset.ipynb		preprocessing_and_giga_dataset.ipynb
requirements.txt		requirements.txt
training.md		training.md

tony-wolff/AU_BS_model