Interpretable Neural Dialog Generation via Discrete Sentence Representation Learning

Codebase for Unsupervised Discrete Sentence Representation Learning for Interpretable Neural Dialog Generation, published as a long paper in ACL 2018. Presentation slides here.

The bibtex are listed below:

@article{zhao2018unsupervised,
  title={Unsupervised Discrete Sentence Representation Learning for Interpretable Neural Dialog Generation},
  author={Zhao, Tiancheng and Lee, Kyusong and Eskenazi, Maxine},
  journal={arXiv preprint arXiv:1804.08069},
  year={2018}
}

Requirements

python 3.7
pytorch
numpy
nltk

Datasets

The data folder contains three datasets:

• PennTree Bank: sentence data
• Daily Dialog: human-human open domain chatting.
• Stanford Multi-domain Dialog: human-woz task-oriented dialogs.

Run Models

The first two scripts are sentence models (DI-VAE/DI-VST) that learn discrete sentence representations from either auto-encoding or context-predicting.

Discrete Info Variational Autoencoder (DI-VAE)

The following command will train a DI-VAE on the PTB dataset. To run on different datasets, follows the pattern in PTB dataloader and corpus reader and implement your own data interface.

python ptb-utt.py

Discrete info Variational Skip-thought (DI-VST)

The following command will train a DI-VST on the Daily Dialog corpus.

python dailydialog-utt-skip.py

The next two train a latent-action encoder decoder with either DI-VAE or DI-VST.

DI-VAE + Encoder Decoder (AE-ED)

The following command will first train a DI-VAE on the Stanford multi domain dialog dataset, and then train a hierarchical encoder decoder (HRED) model with the latent code from the DI-VAE.

python stanford-ae.py

DI-VST + Encoder Decoder (ST-ED)

The following command will first train a DI-VST on the Stanford multi domain dialog dataset, and then train a hierarchical encoder decoder (HRED) model with the latent code from the DI-VST.

python stanford-skip.py

Change Configurations

Change model parameters

Generally all the parameters are defined at the top of each script. You can either passed a different value in the command line or change the default value of each parameters. Some key parameters are explained below:

• y_size: the number of discrete latent variable
• k: the number of classes for each discrete latent variable
• use_reg_kl: whether or not use KL regulization on the latetn space. If False, the model becomes normal autoencoder or skip thought.
• use_mutual: whether or not use Batch Prior Regulization (BPR) proposed in our work or the standard ELBO setup.

Extra essential parameters for LA-ED or ST-ED:

• use_attribute: whether or not use the attribute forcing loss in Eq 10.
• freeze_step: the number of batch we train DI-VAE/VST before we freeze latent action and training encoder-decoders.

Test a existing model

All trained models and log files are saved to the log folder. To run a existing model, you can:

• Set the forward_only argument to be True
• Set the load_sess argument to te the path to the model folder in log
• Run the script