P2 Multimodal Information Processing and Analysis project - MSc in Data Science @ NCSR Demokritos
Find the most similar song in database
~ Create a 10 genre list
~ Implement a script for:
- Downloading top 50 youtube videos per music genre
- Gathering metadata for each video on spotify
- Saving video metadata and file attributes in a Postgres database
- Split videos into 10sec segments
- Extract audio features using pyAudioAnalysis
- Extract video features using Tyiannak modules
- Save audio & video features in database
- Train a NN LSTM autoencoder (44 timesteps x 422 features) for encoding videos into a fixed length vector (40 dimensions)
- Train a NN classifier on database video 40D encodings for predicting new video genre (10 target classes)
- Implement KNN on SQL evaluating distance weights along each feature
- Use Euclidian and Cosine similarity metrics
- Classify the new video by the pre trained classifier, boost same genre videos on similarity
The app is built with Flask framework. A Postgres container serves the database, therefore Docker and the psycopg2 driver are required to be installed.
- Download and extract database data into the project root directory.
- Install the project requirements:
$ pip3 install -r requirements.txt
- Run the app by sourcing run.sh
$ source ./run.sh
- Open your browser and connect to http://localhost:5000