sentiment_analysis_classical_explainer.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*Copyright (c) Microsoft Corporation. All rights reserved.*\n",
    "\n",
    "*Licensed under the MIT License.*\n",
    "\n",
    "# Sentiment Analysis using Classical Explainer "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "_**This notebook showcases how to use the interpret-text repo to implement an interpretable module using feature importances and bag of words representation.**_\n",
    "\n",
    "\n",
    "## Contents\n",
    "1. [Introduction](#Introduction)\n",
    "2. [Setup](#Setup)\n",
    "3. [Training](#Training)\n",
    "4. [Results](#Results)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import sys\n",
    "sys.path.append(\"../..\")\n",
    "import os\n",
    "\n",
    "import pandas as pd\n",
    "import nlp\n",
    "\n",
    "from sklearn.preprocessing import LabelEncoder\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.metrics import precision_recall_fscore_support\n",
    "\n",
    "from interpret_text.experimental.classical import ClassicalTextExplainer\n",
    "\n",
    "pd.set_option('display.max_columns', None)\n",
    "pd.set_option('max_colwidth', None)\n",
    "pd.set_option(\"max_rows\", None)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. Introduction\n",
    "This notebook illustrates how to locally use interpret-text to help interpret text classification using a logisitic regression baseline and bag of words encoding. It demonstrates the API calls needed to obtain the feature importances along with a visualization dashbard.\n",
    "\n",
    "###### Note:\n",
    "* *Although we use logistic regression, any model that follows sklearn's classifier API should be supported natively or with minimal tweaking.*\n",
    "* *The interpreter supports interpretations using either coefficients associated with linear models or feature importances associated with ensemble models.*\n",
    "* *The classifier relies on scipy's sparse representations to keep the dataset in memory.*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. Setup\n",
    "\n",
    "The notebook is built on features made available by [scikit-learn](https://scikit-learn.org/stable/) and [spacy](https://spacy.io/) for easier compatibiltiy with popular tookits."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Configuration parameters\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "EMOTION_COL = \"emotion\"\n",
    "LABEL_COL = \"label\"\n",
    "TEXT_COL = \"text\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Load data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "train, test = nlp.load_dataset(\"emo\", split = [\"train\", \"test\"])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "id2label = {0: 'others', 1: 'happy', 2: 'sad', 3: 'angry'}\n",
    "labels=list(id2label.values())\n",
    "label2id = {}\n",
    "for i,label in enumerate(labels):\n",
    "    label2id[label]=i"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "train_data={TEXT_COL:[],\n",
    "     EMOTION_COL:[]}\n",
    "for val in train:\n",
    "    if id2label[val[LABEL_COL]]!='others':\n",
    "        train_data[TEXT_COL].append(val[TEXT_COL])\n",
    "        train_data[EMOTION_COL].append(id2label[val[LABEL_COL]])\n",
    "        \n",
    "train_data = pd.DataFrame(train_data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "test_data={TEXT_COL:[],\n",
    "     EMOTION_COL:[]}\n",
    "for val in test:\n",
    "    if id2label[val[LABEL_COL]]!='others':\n",
    "        test_data[TEXT_COL].append(val[TEXT_COL])\n",
    "        test_data[EMOTION_COL].append(id2label[val[LABEL_COL]])\n",
    "        \n",
    "test_data = pd.DataFrame(test_data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_str = train_data[TEXT_COL]\n",
    "ylabels = train_data[EMOTION_COL]\n",
    "\n",
    "X_str_test = test_data[TEXT_COL]\n",
    "ylabels_test = test_data[EMOTION_COL]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Create Explainer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create explainer object that contains default glassbox classifier and explanation methods\n",
    "explainer = ClassicalTextExplainer()\n",
    "label_encoder = LabelEncoder()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Training\n",
    "\n",
    "###### Note: Vocabulary\n",
    "\n",
    "* *The vocabulary is compiled from the training set. Any word that does not appear in the training data split, will not appear in the vocabulary.*\n",
    "* *The word must appear one or more times to be considered part of the vocabulary.*\n",
    "* *However, the sklearn countvectorizer allows the addition of a custom vocabulary as an input parameter.*\n",
    "\n",
    "### Configure training setup\n",
    "This step will cast the training data and labels into the correct format\n",
    "\n",
    "1. Data is alreadt split. Otherwise, split data into train and test using a random shuffle\n",
    "2. Load desired classifier. In this case, Logistic Regression is set as default.\n",
    "3. Setup grid search for hyperparameter optimization and train model. Edit the hyper parameter range to search over as per your model.\n",
    "4. Fit models to train set"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_train, X_test, y_train, y_test = X_str, X_str_test, ylabels,  ylabels_test\n",
    "y_train = label_encoder.fit_transform(y_train)\n",
    "y_test = label_encoder.transform(y_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"X_train shape =\" + str(X_train.shape))\n",
    "print(\"y_train shape =\" + str(y_train.shape))\n",
    "print(\"X_train data structure = \" + str(type(X_train)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Model Overview\n",
    "\n",
    "The 1-gram [Bag of Words](https://en.wikipedia.org/wiki/Bag-of-words_model) allows a 1:1 mapping from individual words to their respective frequencies in the [document-term matrix](https://en.wikipedia.org/wiki/Document-term_matrix). "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "classifier, best_params = explainer.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Results\n",
    "\n",
    "###### Notes for default Logistic Regression classifier:\n",
    "* *The parameters are set using cross-validation*\n",
    "* *Below listed hyperparamters are selected by searching over a larger space.*\n",
    "* *These apply specifically to this instance of the logistic regression model and mnli dataset.*\n",
    "* *'Multinomial' setup was found to be better than 'one-vs-all' across the board*\n",
    "* *Default 'liblinear' solver is not supported for 'multinomial' model setup*\n",
    "* *For a different model or dataset, set the range as appropriate using the hyperparam_range argument in the train method* "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# obtain best classifier and hyper params\n",
    "print(\"best classifier: \" + str(best_params))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Performance Metrics"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "mean_accuracy = classifier.score(X_test, y_test, sample_weight=None)\n",
    "print(\"accuracy = \" + str(mean_accuracy * 100) + \"%\")\n",
    "y_pred = classifier.predict(X_test)\n",
    "[precision, recall, fscore, support] = precision_recall_fscore_support(y_test, y_pred,average='macro')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Enter any document or a document and label pair that needs to be interpreted\n",
    "document = \"There is no limit to what we, as women can acomplish\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Obtain the top feature ids for the selected class label\n",
    "explainer.preprocessor.labelEncoder = label_encoder"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Explain Model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "local_explanation = explainer.explain_local(document)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "y = classifier.predict(document)\n",
    "predicted_label = label_encoder.inverse_transform(y)\n",
    "local_explanation = explainer.explain_local(document, predicted_label)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Visualize Explanations"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from interpret_text.experimental.widget import ExplanationDashboard"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ExplanationDashboard(local_explanation)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}