We are looking to create a network that can identify how "right" a squat is. Initially, we considered doing multiple exercises, but in an effort not to bite off more than we can chew we decided to stick with just squats. We chose a squat because it is a common exercise, easy to mess up, and there are negative impacts for doing one improperly. The neural network will take images as input and output a single number between 0 and 1, guessing how good the squat was. The network will only be comparing the input state against the final, correct state of a squat. To simplify the problem, we will not measure the entire motion from start to end state. We hope that any confidence interval between 0.8 and 0.95 would be identifying an attempt at a squat, and any interval between 0.95 and 1 would be a good squat. The numbers and theory will need to be tested, but the general idea is to create a network that would be able to "guide" less experienced gym-goers or exercisers.
Despite looking through resources like Kaggle, ImageNet, and other data repos there seem to not be any open datasets containing what we need. Google has images that we can use but extracting each by hand would be incredibly time-consuming. I've found code for a node-based image scraper and another python-based one. If we do end up collecting images with a scraper, we will likely have to do a good bit of image processing. (3) There are several libraries we can look at using to streamline this, like pillow, Keras, SciPy, or Scikit-image. If we have a relatively small scope, for instance, one or two exercises, I don't see this being too difficult. Given that I haven't been able to find any other datasets even close to what we need, this might be option one of two. The second option would be to create our dataset from frames in videos that we take ourselves. Both options present their own set of challenges and results. Using scraped images might produce a more adaptable, less accurate model while the created dataset would likely produce the opposite.
Our modeling will employ a neural network of inputs of either picture scraping or data from a motion capture system of human motions and or points such as joints during the workout. The proposed neural network's measure of success would be weighting our set of inputs against certain frequencies or thresholds to determine if the squatter's form is correct. As there may be a lot of variation in data, inputs, and different accepted forms in the workout (squats) we plan to measure against thresholds such as 80% to 95% accuracy to ensure proper form.
In most research projects that process sequential time-series data, recurrent units are most often used in the implementation of these models. This project will include time-series data, but there has been recent work that suggested it might not be the best way to go with this project. (1) Researchers have recently done something very similar to what we are working on. In their testing, they used a few different architectures to find the level of correctness that an individual is performing a physical rehabilitation exercise. The one that performed the best was a convolutional neural network. Since we will be using images of independent exercise states, we will be using a convolutional neural network.