This repository contains a video demo of IEEE International Conference on Acoustics, Speech and Signal Processing submitted paper titled "Lip-to-Speech Synthesis in the Wild with Multi-task Learning."
The official code is now available in here.
A demo video contains the original speech, the generated speech from previous state-of-the-art work [1], and the generated speech from the proposed method from three different speakers on both LRS2 and LRS3 datasets, respectively, and six speakers on LRW dataset. The video demo is located in demo-video folder in our repository, and it is also available in Youtube:
- LRS2 and LRS3 [Demo Video]
- LRW [Demo Video]
[1] Rodrigo Mira, Alexandros Haliassos, Stavros Petridis, Bj̈orn W Schuller, and Maja Pantic, “Svts: Scalable video-to-speech synthesis,” arXiv preprint arXiv:2205.02058, 2022.