Skip to content

makcedward/nlp

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 

Repository files navigation

NLP - Tutorial

Repository to show how NLP can tacke real problem. Including the source code, dataset, state-of-the art in NLP

Data Augmentation

General

Text Preprocessing

Section Sub-Section Description Story
Tokenization Subword Tokenization Medium
Tokenization Word Tokenization Medium Github
Tokenization Sentence Tokenization Medium Github
Part of Speech Medium Github
Lemmatization Medium Github
Stemming Medium Github
Stop Words Medium Github
Phrase Word Recognition
Spell Checking Lexicon-based Peter Norvig algorithm Medium Github
Lexicon-based Symspell Medium Github
Machine Translation Statistical Machine Translation Medium
Machine Translation Attention Medium
String Matching Fuzzywuzzy Medium Github

Text Representation

Section Sub-Section Research Lab Story Source
Traditional Method Bag-of-words (BoW) Medium Github
Latent Semantic Analysis (LSA) and Latent Dirichlet Allocation (LDA) Medium Github
Character Level Character Embedding NYU Medium Github Paper
Word Level Negative Sampling and Hierarchical Softmax Medium
Word2Vec, GloVe, fastText Medium Github
Contextualized Word Vectors (CoVe) Salesforce Medium Github Paper Code
Misspelling Oblivious (word) Embeddings Facebook Medium Paper
Embeddings from Language Models (ELMo) AI2 Medium Github Paper Code
Contextual String Embeddings Zalando Research Medium Paper Code
Sentence Level Skip-thoughts Medium Github Paper Code
InferSent Medium Github Paper Code
Quick-Thoughts Google Medium Paper Code
General Purpose Sentence (GenSen) Medium Paper Code
Bidirectional Encoder Representations from Transformers (BERT) Google Medium Paper(2019) Code
Generative Pre-Training (GPT) OpenAI Medium Paper(2019) Code
Self-Governing Neural Networks (SGNN) Google Medium Paper
Multi-Task Deep Neural Networks (MT-DNN) Microsoft Medium Paper(2019)
Generative Pre-Training-2 (GPT-2) OpenAI Medium Paper(2019) Code
Universal Language Model Fine-tuning (ULMFiT) OpenAI Medium Paper Code
BERT in Science Domain Medium Paper(2019) Paper(2019)
BERT in Clinical Domain NYU/PU Medium Paper(2019) Paper(2019)
RoBERTa UW/Facebook Medium Paper(2019) Paper
Unified Language Model for NLP and NLU (UNILM) Microsoft Medium Paper(2019)
Cross-lingual Language Model (XLMs) Facebook Medium Paper(2019)
Transformer-XL CMU/Google Medium Paper(2019)
XLNet CMU/Google Medium Paper(2019)
CTRL Salesforce Medium Paper(2019)
ALBERT Google/Toyota Medium Paper(2019)
T5 Googles Medium Paper(2019)
MultiFiT Medium Paper(2019)
XTREME Medium Paper(2020)
REALM Medium Paper(2020)

| Document Level | lda2vec | | Medium | Paper | | | doc2vec | Google | Medium Github | Paper |

NLP Problem

Section Sub-Section Description Research Lab Story Paper & Code
Named Entity Recognition (NER) Pattern-based Recognition Medium
Lexicon-based Recognition Medium
spaCy Pre-trained NER Medium Github
Optical Character Recognition (OCR) Printed Text Google Cloud Vision API Google Medium Paper
Handwriting LSTM Google Medium Paper
Text Summarization Extractive Approach Medium Github
Abstractive Approach Medium
Emotion Recognition Audio, Text, Visual 3 Multimodals for Emotion Recognition Medium

Acoustic Problem

Section Sub-Section Description Research Lab Story Paper & Code
Feature Representation Unsupervised Learning Introduction to Audio Feature Learning Medium Paper 1 Paper 2 Paper 3
Feature Representation Unsupervised Learning Speech2Vec and Sentence Level Embeddings Medium Paper 1 Paper 2
Feature Representation Unsupervised Learning Wav2vec Medium Paper
Speech-to-text Introduction to Speeh-to-text Medium

Text Distance Measurement

Section Sub-Section Description Research Lab Story Paper & Code
Euclidean Distance, Cosine Similarity and Jaccard Similarity Medium Github
Edit Distance Levenshtein Distance Medium Github
Word Moving Distance (WMD) Medium Github
Supervised Word Moving Distance (S-WMD) Medium
Manhattan LSTM Medium Paper

Model Interpretation

Section Sub-Section Description Research Lab Story Paper & Code
ELI5, LIME and Skater Medium Github
SHapley Additive exPlanations (SHAP) Medium Github
Anchors Medium Github

Graph

Section Sub-Section Description Research Lab Story Paper & Code
Embeddings TransE, RESCAL, DistMult, ComplEx, PyTorch BigGraph Medium RESCAL(2011) TransE(2013) DistMult(2015) ComplEx(2016) PyTorch BigGraph(2019)
Embeddings DeepWalk, node2vec, LINE, GraphSAGE Medium DeepWalk(2014) node2vec(2015) LINE(2015) GraphSAGE(2018)
Embeddings WLG, GCN, GAT, GIN Medium WLG(2011) GCN2017) GAT(2017) GraphSAGE(2018)
Embeddings PinSAGE(2018) Pinterest Medium
Embeddings HoIE(2015), SimpIE(2018) Medium
Embeddings ContE(2017), ETE(2017) Medium

Meta-Learning

Section Sub-Section Description Story
Introduction Matching Nets(2016) MANN(2016) LSTM-based meta-learner(2017) Prototypical Networks(2017) ARC(2017) MAML(2017) MetaNet(2017) Medium
NLP Dialog Generation DAML(2019), PAML(2019), NTMS(2019) Medium
Classification Intent Embeddings(2016) LEOPARD(2019) Medium
CV Unsupervised Learning CACTUs(2018) Medium
General Siamese Network(1994), Triplet Network(2015) Medium
MAML+(2018) Medium

Image

Section Sub-Section Description Research Lab Story Paper & Code
Object Detection R-CNN Medium Paper(2013)
Object Detection Fast R-CNN Medium Paper(2015)
Object Detection Faster R-CNN Medium Paper(2015)
Object Detection VGGNet Medium Paper(2014)
Instance Segmentation Mask R-CNN FAIR Medium Paper(2017)
Image Classification ResNet(2015) Microsoft Medium
Image Classification ResNeXt(2016) Medium

Evaluation

Section Sub-Section Description Story
Introduction Medium
Classification Confusion Matrix, ROC, AUC Medium
Regression MAE, MSE, RMSE, MAPE, WMAPE Medium
Textual Perplexity, BLEU, GER, WER, GLUE Medium

Source Code

Section Sub-Section Description Link
Spellcheck Github
InferSent Github