day_13.ipynb

{
 "metadata": {
  "name": "",
  "signature": "sha256:6b7a9871a5faf5bb4795616b0e9bafe863b6355645a9f59f61d21de111aba08a"
 },
 "nbformat": 3,
 "nbformat_minor": 0,
 "worksheets": [
  {
   "cells": [
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "### Classifying emails\n",
      "\n",
      "Has there been a shift over time in the emails written to Democratic donors between 2012 and 2014? Below I develop a very simple model.  Your mission is to blend your data science and social science skills together to come up with the best fitting model. At a miminium, you will want to use the tools like `pipeline` and `grid_search` from the [previous assignment](http://nbviewer.ipython.org/github/nealcaren/quant-text-fall-2014/blob/master/day_12.ipynb).\n",
      "\n",
      "If you want to press forward with other tools, you might want explore using other estimators, such as [support vector machines](http://scikit-learn.org/stable/modules/svm.html) or random [random forests](http://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomForestClassifier.html). Make sure you understand these estimators before using them! Note, if you are going to use one of these, you'll probably want to use a different vectorizer, such as the [term frequency-inverse document frequency vectorizer](http://scikit-learn.org/stable/modules/generated/sklearn.feature_extraction.text.TfidfVectorizer.html).\n",
      "\n",
      "If you want, feel free to try and model the sender to see if there's a difference depending on who is sending the emails."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "import requests\n",
      "from collections import Counter\n"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 95
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "# http://stackoverflow.com/questions/11116697/how-to-get-most-informative-features-for-scikit-learn-classifiers\n",
      "\n",
      "def show_most_informative_features_2_classes(vectorizer, clf, n=20):\n",
      "    feature_names = vectorizer.get_feature_names()\n",
      "    coefs_with_fns = sorted(zip(clf.coef_[0], feature_names))\n",
      "    top = zip(coefs_with_fns[:n], coefs_with_fns[:-(n + 1):-1])\n",
      "    for (coef_1, fn_1), (coef_2, fn_2) in top:\n",
      "        print \"\\t%.4f\\t%-15s\\t\\t%.4f\\t%-15s\" % (coef_1, fn_1, coef_2, fn_2)\n",
      "        \n",
      "\n",
      "def show_most_informative_features_n_classes(vectorizer, clf, class_labels):\n",
      "    \"\"\"Prints features with the highest coefficient values, per class\"\"\"\n",
      "    feature_names = vectorizer.get_feature_names()\n",
      "    for i, class_label in enumerate(class_labels):\n",
      "        top10 = np.argsort(clf.coef_[i])[-20:]\n",
      "        print(\"%s: %s\" % (class_label,\n",
      "              \" \".join(feature_names[j] for j in top10)))\n",
      "        "
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 96
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "url = 'https://raw.githubusercontent.com/nealcaren/quant-text-fall-2014/master/emails.json'\n",
      "emails = requests.get(url).json()\n",
      "print len(emails)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "2735\n"
       ]
      }
     ],
     "prompt_number": 97
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "years = []\n",
      "texts = []\n",
      "senders = []\n",
      "\n",
      "for email in emails:\n",
      "    text = email['copy']\n",
      "    year = int(email['date'].split()[2])\n",
      "    sender = email['sender'].split('@')[-1].split('>')[0].replace('email.','')\n",
      "    years.append(year)\n",
      "    texts.append(text)\n",
      "    senders.append(sender)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 99
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "print Counter(senders)\n",
      "print Counter(years)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "Counter({u'barackobama.com': 721, u'emilyslist.org': 512, u'thehousemajoritypac.com': 473, u'democrats.org': 438, u'dccc.org': 386, u'patrickmurphyforcongress.com': 186, u'loisfrankelforcongress.com': 15, u'loisfrankel.com': 4})\n",
        "Counter({2014: 1584, 2013: 654, 2012: 497})\n"
       ]
      }
     ],
     "prompt_number": 100
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "from sklearn.feature_extraction.text import CountVectorizer\n",
      "from sklearn.naive_bayes import MultinomialNB\n",
      "from sklearn import metrics\n",
      "import numpy as np"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 101
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "vectorizer = CountVectorizer(stop_words='english')"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 102
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "vectorizer.fit(texts)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 103,
       "text": [
        "CountVectorizer(analyzer=u'word', binary=False, charset=None,\n",
        "        charset_error=None, decode_error=u'strict',\n",
        "        dtype=<type 'numpy.int64'>, encoding=u'utf-8', input=u'content',\n",
        "        lowercase=True, max_df=1.0, max_features=None, min_df=1,\n",
        "        ngram_range=(1, 1), preprocessor=None, stop_words='english',\n",
        "        strip_accents=None, token_pattern=u'(?u)\\\\b\\\\w\\\\w+\\\\b',\n",
        "        tokenizer=None, vocabulary=None)"
       ]
      }
     ],
     "prompt_number": 103
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "X = vectorizer.transform(texts)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 104
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "y = years"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 105
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "clf = MultinomialNB()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 106
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "clf.fit(X,y)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 107,
       "text": [
        "MultinomialNB(alpha=1.0, class_prior=None, fit_prior=True)"
       ]
      }
     ],
     "prompt_number": 107
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "metrics.confusion_matrix(y, clf.predict(X))"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 108,
       "text": [
        "array([[ 426,   66,    5],\n",
        "       [   7,  555,   92],\n",
        "       [   0,  248, 1336]])"
       ]
      }
     ],
     "prompt_number": 108
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "print metrics.classification_report(y, clf.predict(X) )"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "             precision    recall  f1-score   support\n",
        "\n",
        "       2012       0.98      0.86      0.92       497\n",
        "       2013       0.64      0.85      0.73       654\n",
        "       2014       0.93      0.84      0.89      1584\n",
        "\n",
        "avg / total       0.87      0.85      0.85      2735\n",
        "\n"
       ]
      }
     ],
     "prompt_number": 109
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "show_most_informative_features_n_classes(vectorizer, clf, [2012,2013,2014])"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "2012: gifts update gmail bassetti donate tax list make unsubscribe like election sent click president com victoria america campaign email obama\n",
        "2013: house update information gmail bassetti like organizing unsubscribe paid women committee sent candidate action click com emily victoria list email\n",
        "2014: paid sent gmail unsubscribe information campaign obama democratic bassetti donate democrats president list like com click committee candidate victoria email\n"
       ]
      }
     ],
     "prompt_number": 110
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "# One possible useful thing from\n",
      "# http://stackoverflow.com/questions/24386489/adding-words-to-scikit-learns-countvectorizers-stop-list\n",
      "\n",
      "from sklearn.feature_extraction import text\n",
      "stop_words = text.ENGLISH_STOP_WORDS.union(['victoria','bassetti','gmail','unsubscribe','email'])\n",
      "vectorizer = CountVectorizer(stop_words=stop_words)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 117
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [],
     "language": "python",
     "metadata": {},
     "outputs": []
    }
   ],
   "metadata": {}
  }
 ]
}