Acoustic Word Embeddings and Phonological Similarity 🍵

This is the code base for the acoustic word embedding models, training experiments, and evaluation scripts for the experiments reported in our INTERSPEECH 2021 paper

📝 Do Acoustic Word Embeddings Capture Phonological Similarity? An Empirical Study

Dependencies 🧬

python 3.8, pytorch 1.1, numpy, scipy, faiss, pickle, pandas, yaml

Speech Data 💬 🗨️

The data in our study is drawn from the Multilingual GlobalPhone speech database for German 🇩🇪 and Czech 🇨🇿. Because the data is distributed under a research license by Appen Butler Hill Pty Ltd., we cannot re-distribute the raw speech data. However, if you have already access to the GlobalPhone speech database and you would like to access to our word-alignment annotations, train/test splits, and word-level IPA transcriptions, please contact the first author.

Working with the code 🐍

To run a training experiment, write down all hyperparameters and other info in the config file config_file_train_awe_bigru_seq2seq.yml

Then ...

>>> cd AWEs_phon_sim
>>> python nn_train_seq2seq_embeddings.py config_files/config_file_train_awe_bigru_seq2seq.yml

To evaluate the model on the acoustic word discrimination task, make sure the path to the pre-trained model is in this config file config_file_eval_awe_bigru_seq2seq.yml

Then ...

>>> cd AWEs_phon_sim
>>> python nn_eval/nn_eval_seq2seq_embeddings.py config_files/config_file_eval_awe_bigru_seq2seq.yml

The code is fairly documented and the vectorization logic, as well as the code for the models, should be useful for other speech technology tasks. If you use our code and encounter problems, please create an issue or contact the first author.

If you use our code in a work that leads to a publication, please cite our paper as

@inproceedings{Abdullah2021DoAW,
  title={Do Acoustic Word Embeddings Capture Phonological Similarity? An Empirical Study},
  author={Badr M. Abdullah and Marius Mosbach and Iuliia Zaitova and Bernd Möbius and Dietrich Klakow},
  booktitle={Proc. Interspeech},
  year={2021}
}