Speech Recognition with the caffe deep learning framework
UPDATE: We are migrating to tensorflow
This project is quite fresh and only the first of three milestones is accomplished: Even now it might be useful if you just want to train a handful of commands/options (1,2,3..yes/no/cancel/...)
- training spoken numbers:
- get spectogram training images from http://pannous.net/spoken_numbers.tar (470 MB)
- start ./train.sh
- test with
ipython notebook test-speech-recognition.ipynborcaffe test ...or<caffe-root>/python/classify.py - 99% accuracy, nice!
- online recognition and learning with
./recognition-server.pyand./record.pyscripts
Sample spectrogram, Karen uttering 'zero' with 160 words per minute.
- training words:
- 4GB of training data
- net topology: work in progress ...
- todo: use upcoming new caffe LSTM layers etc
- UPDATE LSTMs get rolling, still not merged
- UPDATE since the caffe project leaders have a hindering merging policy and this pull request was shifted many times without ever being merged, we are migrating to tensorflow
- todo: add extra categories for a) silence b) common noises like typing, achoo c) ALL other noises
- training speech:
- todo!
- 100GB of training data here: http://www.openslr.org/12/
- TIMIT dataset $27,000.00 membership fee or $250 for non-members+$2400 under research-only license?
- combine with google n-grams
Theoretical background: papers
A. Graves and N. Jaitly. Towards end-to-end speech recognition with recurrent neural networks. In ICML, 2014
O. Vinyals, S. V. Ravuri, and D. Povey. Revisiting recurrent neural networks for robust ASR. In ICASSP, 2012
Schmidhuber et al using new 'ClockWork-RNNs'
The book: Automatic Speech Recognition: A Deep Learning Approach (Signals and Communication Technology) Hardcover – November 11, 2014 by Dong Yu (Author) and Li Deng (Author)
Related work
Also see the Kaldi project, which seems a bit messy but already uses deep learning with LSTM Another experimental LSTM network, which works out-of-the-box: Currennt