Natural Language Processing Specialization

Table of Contents

1. Introduction
2. Instructions to use the repository
3. My Learnings from the Specialization
4. Programming Assignments
5. Results
6. Disclaimer

Introduction

This Specialization is designed and taught by two experts in NLP, machine learning, and deep learning. Younes Bensouda Mourri is an Instructor of AI at Stanford University who also helped build the Deep Learning Specialization. Łukasz Kaiser is a Staff Research Scientist at Google Brain and the co-author of Tensorflow, the Tensor2Tensor and Trax libraries, and the Transformer paper.

Instructions to use the repository

Using this repository is straight forward. Clone this repository to use. This repository contains all my work for this specialization. All the code base, quiz questions, screenshot, and images, are taken from, unless specified, Natural Language Processing Specialization on Coursera.

My Learnings from the Specialization

In this four course series, I learned about logistic regression and naïve Bayes algorithms and implemented them from scratch in Python, used word vectors to implement sentiment analysis model, complete analogies (King - Man + Woman = Queen), and translate words (English to French), and used locality sensitive hashing for approximate nearest neighbors.

I used dynamic programming, hidden Markov models, and word embeddings to autocorrect misspelled words, autocomplete partial sentences, and identify part-of-speech tags for words.

I learned about dense and recurrent neural networks, LSTMs, GRUs, and Siamese networks in TensorFlow and Trax and used these to perform advanced sentiment analysis, text generation, named entity recognition, and to identify duplicate questions on Quora.

I implemented encoder-decoder (Seq2seq model), causal, and self-attention to perform advanced machine translation of complete sentences, text summarization, question-answering and to build chatbots. State of the art models were taught that includes T5, BERT, transformer, reformer, and more!

Programming Assignments

1. Course 1: Natural Language Processing with Classification and Vector Spaces

   • Week 1: Logistic Regression for Sentiment Analysis of Tweets

     • Classify positive or negative sentiment in tweets using Logistic Regression

   • Week 2: Naïve Bayes for Sentiment Analysis of Tweets

     • Classify positive or negative sentiment in tweets using more advanced model

   • Week 3: Vector Space Models

     • Vector space models capture semantic meaning and relationships between words. Use them to discover relationships between words, then visualize their relationships in two dimensions using PCA (dimensionality reduction technique).

   • Week 4: Word Embeddings and Locality Sensitive Hashing for Machine Translation

     • Write a simple English-to-French translation algorithm using pre-computed word embeddings and locality sensitive hashing to relate words via approximate k-nearest neighbors search.

2. Course 2: Natural Language Processing with Probabilistic Models

   • Week 1: Auto-correct using Minimum Edit Distance

     • Create a simple auto-correct algorithm using minimum edit distance and dynamic programming

   • Week 2: Part-of-Speech (POS) Tagging

     • Learn about Markov chains and Hidden Markov models, then use them to create part-of-speech tags for a Wall Street Journal text corpus. Apply the Viterbi algorithm for POS tagging, which is important for computational linguistics

   • Week 3: N-gram Language Models

     • Learn about how N-gram language models work by calculating sequence probabilities, then build your own autocomplete language model using a text corpus from Twitter (similar models are used for translation, determining the author of a text, and speech recognition)

   • Week 4: Word2Vec and Stochastic Gradient Descent

     • Learn about how word embeddings carry the semantic meaning of words, which makes them much more powerful for NLP tasks, then build your own Continuous bag-of-words model that uses a neural network to compute word embeddings from Shakespeare text.

3. Course 3: Natural Language Processing with Sequence Models**

   • Week 1: Neural Networks for Sentiment Analysis
     • Train a neural network with GLoVe word embeddings to perform sentiment analysis on tweets

• Week 2: Recurrent Neural Networks for Language Modeling

  • Learn about the limitations of traditional language models and see how RNNs and GRUs use sequential data for text prediction. Build your own next-word generator using a simple RNN on Shakespeare text data.

  • Week 3: LSTMs and Named Entity Recognition

    • Learn about how long short-term memory units (LSTMs) solve the vanishing gradient problem, and how Named Entity Recognition systems quickly extract important information from text. Build your own Named Entity Recognition system using an LSTM and data from Kaggle.

  • Week 4: Siamese Networks

    • Learn about Siamese networks, a special type of neural network made of two identical networks that are eventually merged together. Build your own Siamese network that identifies question duplicates in a dataset from Quora.

4. Course 4: Natural Language Processing with Attention Models

   • Week 1: Neural Machine Translation with Attention

     • Discover some of the shortcomings of a traditional seq2seq model and how to solve for them by adding an attention mechanism. Build a Neural Machine Translation model with Attention that translates English sentences into German.

   • Week 2: Text Summarization with Transformer Models

     • Compare RNNs and other sequential models to the more modern Transformer architecture. Build a transformer model that generates text summaries.

   • Week 3: Question-Answering with Transformer Models

     • Explore transfer learning with state-of-the-art models like T5 and BERT, then build a model that can perform question answering task.

   • Week 4: Chatbots with a Reformer Model

     • Examine some unique challenges Transformer models face and their solutions, then build a chatbot using a Reformer model.

Results

Some results from the programming assignments of this specialization

• Neural Machine Translation English to German

• Word Embeddings projection in 2D space

• Named Entity Recognition results

• Auto Suggestion results

• Duplicate questions identification

• Text Summarization using Transformers

• ChatBot results

Disclaimer

The solutions uploaded in this repository are only for reference when you got stuck somewhere. Please don't use these solutions to pass programming assignments.