If you want to understand what an LSTM is and how it works, I suggest this great article.
Completing this LSTM was the practical exercise accompanying the Deep Learning & Neural Networks course in the summer term of 2020 at KIT. The template for the LSTM was written by our instructor, Ngoc Quan Pham.
We implemented the forward pass and backpropagation and also applied some additional modifications to the LSTM, like changing it's sequence rate over time. This way, we hoped, it would first learn basic words and then go on to learn bigger structures, like sentences, later on. From what we could observe, that was a useful change, as the network produced way less random (non-)words and formed (mostly) nicely structured sentences. We also changed how characters are sampled from the probability distribution that is generated, which helped with reducing the amount of random (non-)words as well.
- Get an API key from Genius API and add it to
downloadlyrics.py. - Then start the training with
./train.sh ARTIST-NAMEand enjoy the show.