W3-5 Solo Work.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 32,
   "id": "approximate-origin",
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import nltk\n",
    "import sklearn\n",
    "from sklearn.metrics import cohen_kappa_score"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "rocky-savannah",
   "metadata": {},
   "source": [
    "## The Adult Data Set\n",
    "\n",
    "In this workbook, you'll work with data extracted from the 1994 census. I got it from [here](http://archive.ics.uci.edu/ml/datasets/Adult). (Yes, the title of the this corpus is not very helpful.)\n",
    "\n",
    "Essentially, this data set has some information about some adults, plus whether their income was more or less thatn 50k. The classification task is to use the other variables to predict whether someone made more or less than 50k.\n",
    "\n",
    "The data is in a weird format - an `arff` file. I'll deal with reading it in for you."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "greenhouse-fashion",
   "metadata": {},
   "outputs": [],
   "source": [
    "from scipy.io import arff\n",
    "data, meta = arff.loadarff(\"corpora/adult/adult.arff\")\n",
    "df = pd.DataFrame(data)\n",
    "dlist = df.to_dict('records')\n",
    "for row in dlist:\n",
    "    for k, v in row.items():\n",
    "        if isinstance(v, bytes):\n",
    "            row[k] = str(v, \"utf-8\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "powerful-fetish",
   "metadata": {},
   "source": [
    "Now the data is in the same form we used in the preceding notebooks; namely it's a list of dictionaries.\n",
    "\n",
    "You should probably poke around in it a bit to get a sense for what's there."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "suitable-conference",
   "metadata": {},
   "source": [
    "## Your task\n",
    "\n",
    "Your task here is to create the best classifier that you can - one that predicts whether someone makes more or less than 50k given the other variables.\n",
    "\n",
    "Although constructing a classifier can be cool, in this case we're probably more interested in what the performance of our classifier tells us about the data itself."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "excessive-member",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "silver-defensive",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}