NLP_PEMDC

NLP Predtrained Embeddings, Models and Datasets Collections(NLP_PEMDC)

The pretrained word embeddings and datasets for NLP. The collection will keep updating. The purpose of these pre-trained word vectors and datasets is for learning and research purposes only.

不断收集我遇到的各种NLP预训练词向量、模型和数据集。这些预训练词向量和数据集的目的仅用来学习和研究。

The rankings are in no particular order, only in the order I added them. The data set belongs to the original author, thanks! If there is any infringement, please email me and let me know.

排名不分先后，仅按我添加的先后顺序。数据集所有权均属于原作者，感谢！若有侵权，请电邮我告知删除。

Pretrained Chinese Word Vectors(embeddings):

Word2vec

1. 100+ Chinese Word Vectors 上百种预训练中文词向量

   Github

2. Tencent AI Lab Embedding Corpus for Chinese Words and Phrases

   URL

GloVe

TODO

Chinese Pre-trained Models

1. Chinese-BERT

   Github

2. Chinese-BERT-wwm

   Github

3. Chinese-XLNet

   Github1

   Github2

4. Chinese-RoBERTa

   Github

5. Chinese-ALBERT

   Github1

   Github2

Chinese Courpus:

1. [集合]大规模中文自然语言处理语料 Large Scale Chinese Corpus for NLP

   Github

2. [集合]搜狗实验室语料集合

   语料数据

3. [集合]ChineseNlpCorpus

   Github

4. [集合]ChineseGLUE

   Github

   目前包含：

   1. LCQMC 口语化描述的语义相似度任务 Semantic Similarity Task COLING 2018

   2. XNLI 语言推断任务 Natural Language Inference EMNLP 2015

   3. TNEWS 今日头条中文新闻（短文本）分类 Short Text Classificaiton for News

   4. INEWS 互联网情感分析任务 Sentiment Analysis for Internet News

   5. THUCNEWS 长文本分类 Long Text classification

   6. iFLYTEK 长文本分类 Long Text classification

   7. DRCD 繁体阅读理解任务 Reading Comprehension for Traditional Chinese

   8. CMRC2018 简体中文阅读理解任务 Reading Comprehension for Simplified Chinese

   9. BQ 智能客服问句匹配 Question Matching for Customer Service EMNLP 2018 Download

   10. MSRANER 命名实体识别 Name Entity Recognition

   11. CHID 成语阅读理解填空 Chinese IDiom Dataset for Cloze Test

   12. CMNLI 语言推理任务 Chinese Multi-Genre NLI

5. LCSTS: A Large Scale Chinese Short Text Summarization Dataset

   大规模中文短文本摘要数据集

   arXiv

   Download

6. chinese-poetry: 最全中文诗歌古典文集数据库

   Github

7. SentiBridge: 中文实体情感知识库

   Github

English Corpus:

1. [collections]GLUE

   Download

   Including:

   1. The Corpus of Linguistic Acceptability
   2. The Stanford Sentiment Treebank
   3. Microsoft Research Paraphrase Corpus
   4. Semantic Textual Similarity Benchmark
   5. Quora Question Pairs
   6. MultiNLI Matched
   7. MultiNLI Mismatched
   8. Question NLI
   9. Recognizing Textual Entailment
   10. Winograd NLI
   11. Diagnostics Main

2. [collections]SuperGLUE

   Download

   Including:

   1. Broadcoverage Diagnostics
   2. CommitmentBank
   3. Choice of Plausible Alternatives
   4. Multi-Sentence Reading Comprehension
   5. Recognizing Textual Entailment
   6. Words in Context
   7. The Winograd Schema Challenge
   8. BoolQ
   9. Reading Comprehension with Commonsense Reasoning
   10. Winogender Schema Diagnostics

3. IMDB Large Movie Review Dataset

   This is a dataset for binary sentiment classification containing substantially more data than previous benchmark datasets. We provide a set of 25,000 highly polar movie reviews for training, and 25,000 for testing. There is additional unlabeled data for use as well.

   Download

4. SQuAD2.0

   The Stanford Question Answering Dataset

   Website