Natural Language Processing: DIY

Implement well-known NLP models from scratch with high-level APIs.

All implementations are in PyTorch and/or Tensorflow.

• NLP_with_PyTorch

  • 1_basics_(tutorial)
    : Basic ideas and introduction to NLP. Contents from PyTorch tutorials.
  • 2_word_embedding
    : Word embedding models. Contents mainly from research articles and Lee (2019)¹.
    • Neural Probabilistic Language Model²: [notebook] [blog]
    • Word2Vec (skip gram)^3,4: [notebook] [blog]
    • FastText⁵: [notebook] [blog]
    • Latent Dirichlet Allocation^6,7,8,: notebook[blog 1, 2, 3, 4, 5]
  • 3_document-embedding
    : Sentence/document-level embedding models. Contents mainly from research articles and Lee (2019)¹.

• 4_sentiment_analysis: TBD.

• NLP_with_TensorFlow
  : tensorflow port of NLP_with_PyTorch

  • 1_basics_(tutorial)
  • 2_word_embedding
    • Neural Probabilistic Language Model²: [notebook] [blog]
    • Word2Vec: TBD.

---

1: Lee. 2019. 한국어 임베딩 (Embedding Korean). 에이콘 출판사.
2: Bengio et al. 2003. A neural probabilistic language model. The journal of machine learning research, 3, 1137-1155.
3: Mikolov et al. 2013. Efficient Estimation of Word Representations in Vector Space. International Conference on Learning Representations.
4: Mikolov et al. 2013. Distributed Representations of Words and Phrases and their Compositionality. Advances in Neural Information Processing Systems, 26.
5: Bojanowski et al. 2017. Enriching Word Vectors with Subword Information. Transactions of the Association for Computational Linguistics, 5, 135-146.
6: Pritchard, Stephens, Donnelly. 2000. Inference of Population Structure Using Multilocus Genotype Data. Genetics. 155: 945–959.
7: Blei, Ng, Jordan. 2003. Latent Dirichlet Allocation. Journal of Machine Learning Research. 3 (4–5): 993–1022.
8: Griffiths, Steyvers. 2004. Finding scientific topics. Proceedings of the National Academy of Sciencess of the United States of America. 101: 5228-5235.