Response Selection Models

This repo collect the models for response selection and merge them into one framework.

This repo contains the following models:

• SMN: Sequential Matching Network: A New Architecture for Multi-turn Response Selection in Retrieval-based Chatbots. pdf
• DAM: Multi-Turn Response Selection for Chatbots with Deep Attention Matching Network. pdf
• IOI: One Time of Interaction May Not Be Enough: Go Deep with an Interaction-over-Interaction Network for Response Selection in Dialogues. pdf
• ESIM: Sequential Matching Model for End-to-end Multi-turn Response Selection. pdf
• MSN: Multi-hop Selector Network for Multi-turn Response Selection in Retrieval-based Chatbots. pdf

Note: I am still working on reproduce the results of MSN. The results pf MSN is still quite low in this repo. Welcome to pull requests to optimize MSN in this repo.

Copyright

• models are all copied from their official released code.
• The dataset is from MSN.
• The data processing and code structure is from DAM.

Dependency

• python 3
• tensorflow 1.10
• scipy
• sklearn

Data Preparation

• The dataset is from the data released by MSN.
• The dataset can be downloaded from

cd utils
python parse_data.py ../ResponseSelection/ubuntu_data ../data/ubuntu_write
python compose_data.py ../data/ubuntu_write

Train Model

The configuration is located in main.py.

python main.py ubuntu train SMN

Prediction

CHECKPOINT_NAME=output/**
python main.py ubuntu test SMN CHECKPOINT_NAME

Results

Official results from Leaderboards:

Ubuntu Dialogue Corpus V1

Model	R_2@1	R_10@1	R_10@2	R_10@5
SMN	0.926	0.726	0.847	0.961
DAM	0.938	0.767	0.874	0.969
ESIM	-	0.796	0.894	0.975
IoI	0.947	0.796	0.894	0.974
MSN	-	0.800	0.899	0.978

Douban Conversation Corpus

Model	MAP	MRR	P@1	R_10@1	R_10@2	R_10@5
SMN	0.529	0.569	0.397	0.233	0.396	0.724
DAM	0.550	0.601	0.427	0.254	0.410	0.757
ESIM	-	-	-	-	-	-
IoI	0.573	0.621	0.444	0.269	0.451	0.786
MSN	0.587	0.632	0.470	0.295	0.452	0.788

E-commerce Corpus

Model	R_10@1	R_10@2	R_10@5
SMN	0.453	0.654	0.886
ESIM	0.570	0.767	0.948
IoI	0.563	0.768	0.950
MSN	0.606	0.770	0.937

The results of this repo:

Ubuntu Dialogue Corpus V1

Model	R_2@1	R_10@1	R_10@2	R_10@5
SMN	0.944	0.782	0.885	0.973
DAM	0.947	0.790	0.890	0.975
ESIM	0.948	0.786	0.893	0.977
IoI	0.954	0.814	0.904	0.978

Douban Conversation Corpus

Model	MAP	MRR	P@1	R_10@1	R_10@2	R_10@5
SMN	0.550	0.589	0.407	0.253	0.411	0.769
DAM	0.562	0.607	0.428	0.256	0.427	0.784
ESIM	0.569	0.614	0.437	0.270	0.441	0.772
IoI	0.575	0.622	0.446	0.266	0.451	0.796

E-commerce Corpus

Model	R_10@1	R_10@2	R_10@5
SMN	0.530	0.703	0.928
DAM	0.595	0.786	0.958
ESIM	0.551	0.749	0.945
IoI	0.601	0.817	0.960

TODO

1. Upgrade tendorflow to newer version
2. Optimizing MSN
3. For ESIM, the data processing is not standard from their original paper.

Reference

We refer theese papers and repos to build this repo:

• SMN Code: https://github.com/MarkWuNLP/MultiTurnResponseSelection
• DAM Code: https://github.com/baidu/Dialogue
• IOI Code: https://github.com/chongyangtao/IOI
• ESIM Code: https://github.com/alibaba/esim-response-selection
• MSN Code: https://github.com/chunyuanY/Dialogue

I am Zhenxin Fu from Peking University.