VaPar Synth - A Variational Parametric Model for Audio Synthesis

Krishna Subramani¹, Preeti Rao¹, Alexandre D'Hooge²

¹IIT Bombay, ²ENS Paris-Saclay

Paper / Accompanying Webpage / BibTeX

This repository contains the code for VaPar Synth, a Conditional Variational Autoencoder trained on a source-filter inspired parametric representation.

For the necessary libraries/prerequisites, please use conda/anaconda to create an environment (from the environment.yml file in this repository) with the command

conda env create -f environment.yml

Also install SMS-Tools in the same environment. With these, all the code in the repository can be run inside this environment by activating it.

The repository contains code to do the following (One directory for each)-

1. Data Loading: We use the good-sounds dataset.
2. Parametric: Obtaining the parametric representation of the audio.
   1. The parametric representation used utilizes the True Amplitude Envelope Estimation (TAE) algorithm on top of the Harmonic plus Residual model.
   2. We have implemented the code for TAE using the algorithm described here . To the best of our knowledge, there is no open-source implementation (in Python) for this algorithm, so we make our code available to the community to use it and build upon/improve it. The algorithm is present in the func_envs.py file under the extra_dependencies directory. We have also demonstrated the algorithms usage in a Jupyter Notebook in the same directory.
3. Network: PyTorch code for the Conditional VAE/AE. Also contains code to evaluate the models. There is also code to play around with the network and synthesize/generate audio with specified pitch contours.