Skip to content

Mcamin/Nlp-Tutorial

 
 

Repository files navigation

nlp-tutorial

nlp-tutorial is a tutorial for who is studying NLP(Natural Language Processing) using TensorFlow and Pytorch. Most of the models in NLP were implemented with less than 100 lines of code.(except comments or blank lines)

Curriculum - (Example Purpose)

1. Basic Embedding Model

2. CNN(Convolutional Neural Network)

3. RNN(Recurrent Neural Network)

4. Attention Mechanism

5. Model based on Transformer

Model Example Framework Lines(torch/tensor)
NNLM Predict Next Word Torch, Tensor 67/83
Word2Vec(Softmax) Embedding Words and Show Graph Torch, Tensor 77/94
TextCNN Sentence Classification Torch, Tensor 94/99
TextRNN Predict Next Step Torch, Tensor 70/88
TextLSTM Autocomplete Torch, Tensor 73/78
Bi-LSTM Predict Next Word in Long Sentence Torch, Tensor 73/78
Seq2Seq Change Word Torch, Tensor 93/111
Seq2Seq with Attention Translate Torch, Tensor 108/118
Bi-LSTM with Attention Binary Sentiment Classification Torch, Tensor 92/104
Transformer Translate Torch 222/0
Greedy Decoder Transformer Translate Torch 246/0
BERT how to train Torch 242/0

Dependencies

  • Python 3.5+
  • Tensorflow 1.12.0+
  • Pytorch 0.4.1+
  • Plan to add Keras Version

Author