cleverSpeech

Code to generate adversarial examples for Mozilla DeepSpeech. Began as a modified version of Carlini and Wagner's attack. Has slowly turned into a bit of a behemoth.

This is the build repo. If you want to see the package in action, grab a docker image or install the package using the steps outlined below.

run the code

Docker images are available on DockerHub. Each docker image contains the necessary audio examples, transcripts and model checkpoints etc. to get up and running with minimal fussing about.

To start running some experiments with docker:

1. Install the latest version of docker (at least version 19.03).
2. Install and configure the NVIDIA container runtime.
3. Run the container (the image itself will be pulled automatically):

docker run \
    -it \
    --rm \
    --name cleverspeech \
    --gpus all \
    dijksterhuis/cleverspeech:latest

4. Run one of the scripts from ./cleverspeech/scripts

python3 ./cleverspeech/scripts/ctc_attacks.py \
  --max_examples 1 \  # number of adversarial examples to generate
  --attack_graph cgd \  # clipped gradient descent
  --loss ctc  # tensorflow provides two ctc loss implementations

If you want to run the container as your user and group ID you'll need to some extra arguments so that the container can change the deepspeech checkpoint, language model, trie etc. file permissions:

# You will need to wait a few minutes for file permissions to propagate
docker run \
    -it \
    --rm \
    --name cleverspeech \
    --gpus all \
    -e LOCAL_UID=$(id -u ${USER}) \
    -e LOCAL_GID=$(id -g ${USER}) \
    -v path/to/output/dir:/home/cleverspeech/cleverSpeech/adv:rw \
    dijksterhuis/cleverspeech:latest

Note: Using --user with docker run will not work as the container must start as root then switch users after start up (otherwise you can't chown the model checkpoints/scorer files).

don't like docker?

Run:

git clone --recurse-submodules --shallow-submodules \
  https://github.com/dijksterhuis/cleverSpeech.git \
  && cd ./cleverSpeech/ \
  && ./bin/downloads.sh \
  && python3 -m pip install -e .

Then run an experiment as before and have fun fixing undocumented/unexpected dependency errors (there shouldn't be any on ubuntu >= 18.04).