Skip to content

Sentential paraphrase datasets and BERT-based paraphrase detection models for the Armenian language.

License

Notifications You must be signed in to change notification settings

ivannikov-lab/arpa-paraphrase-corpus

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

7 Commits
 
 
 
 
 
 
 
 
 
 

Repository files navigation

ARPA: Armenian Paraphrase Detection Corpus and Models

We provide sentential paraphrase detection train, test datasets as well as BERT-based models for the Armenian language.

To sentences in the dataset are taken from Hetq and Panarmenian news articles.

To generate paraphrase for the sentences, we used back translation from Armenian to English. We repeated the step twice, after which the generated paraphrases were manually reviewed. Invalid sentences were filtered out, while the rest were labelled as either paraphrase, near paraphrase or non-paraphrase (with 1, 0, -1 labels respectively). The sentences are divided into train and test sets. Test examples were reviewed by 3 different annotators. In addition, to increase the number of non-paraphrase pairs, we padded the dataset with automatically generated negative examples, including pairs of consecutive sentences and random pairs.

Number of examples Total Paraphrase Non-paraphrase (near paraphrase)
Train 4233 1339 2684 (210)
Test 1682 1021 449 (212)

We finetuned Multilingual BERT on several training sets, including the proposed ARPA dataset, and evaluated their performance on our test set. During training and evaluation, near paraphrase and non-paraphrase pairs were combined into one class. The results are provided below:

BERT Model Train set F1 Acc.
Multilingual BERT ARPA train set 84.27 78.06
Multilingual BERT Paraphraser.ru train set machine-translated into Armenian 83.81 77.09
Multilingual BERT MRPC train set machine-translated into Armenian 80.07 69.87
Multilingual BERT All of the above combined 84 77.6

The model trained on ARPA is available for use, and can be downloaded using this link.

For more details about the models and dataset construction, refer to the paper.

About

Sentential paraphrase datasets and BERT-based paraphrase detection models for the Armenian language.

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published