explore.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Breast Cancer Data Challenge\n",
    "\n",
    "Your task is to develop a model that predicts whether a biopsied breast cell is benign (not harmful) or malignant (cancerous), given a set of attributes about the cell.\n",
    "\n",
    "* Classification or regression: Classification\n",
    "* Supervised or unsupervised: Supervised"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Data Description\n",
    "The dataset consists of 699 cells for which you have the following features:\n",
    "1. Sample code number: id number\n",
    "2. Clump Thickness: 1 - 10\n",
    "3. Uniformity of Cell Size: 1 - 10\n",
    "4. Uniformity of Cell Shape: 1 - 10\n",
    "5. Marginal Adhesion: 1 - 10\n",
    "6. Single Epithelial Cell Size: 1 - 10\n",
    "7. Bare Nuclei: 1 - 10\n",
    "8. Bland Chromatin: 1 - 10\n",
    "9. Normal Nucleoli: 1 - 10\n",
    "10. Mitoses: 1 - 10\n",
    "11. Class: (2 for benign, 4 for malignant)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "import seaborn as sns\n",
    "import matplotlib.pyplot as plt\n",
    "import itertools\n",
    "\n",
    "from collections import Counter\n",
    "from imblearn.over_sampling import SMOTE \n",
    "from scipy.stats import randint as sp_randint\n",
    "from sklearn.decomposition import PCA\n",
    "from sklearn.feature_selection import RFECV\n",
    "from sklearn.ensemble import RandomForestClassifier\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.metrics import classification_report, confusion_matrix\n",
    "\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Read in txt file as data frame\n",
    "df = pd.read_csv('./breast-cancer-wisconsin.data.txt', header = None)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Name the columns\n",
    "cols = ['id', 'thickness', 'uniformity size', 'uniformity shape', \n",
    "        'marginal adhesion', 'epithelial cell size', 'bare nuclei', 'bland chromatin', \n",
    "        'normal nucleoli', 'mitoses', 'class'] \n",
    "df.columns = cols"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Print first five rows\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Define class, features, IDs\n",
    "classes = df['class']                 # just classes\n",
    "features = df.drop(['class','id'], 1) # just the features\n",
    "feat_class = df.drop('id', 1)         # features and classes\n",
    "ids = df['id']                        # just IDs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Plot how many data points there are for each class\n",
    "sns.set(font_scale=1.5)              # make the font bigger\n",
    "fig = sns.countplot(classes)\n",
    "fig.set(xticklabels=['benign', 'malignant'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is an unbalanced dataset where benign tumors occur ~2x more than malignant ones. I'll talk about how to address this later."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Missing Values\n",
    "\n",
    "I want to find the missing values and determine the best method for imputing or removing them. The most important questions are: How many missing values? Where are they in the data? What is the nature of the data where the values are missing?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Are there any null values? NaN are represented by '?' in data\n",
    "features = features.replace('?', np.nan)        # Replace ? with NaN\n",
    "print(f'Number of null values: {features.isnull().sum().sum()}')            # How many NaN in data\n",
    "\n",
    "# Show where the null values occur\n",
    "df_null = features.isnull().unstack()\n",
    "print(df_null[df_null])\n",
    "\n",
    "# Find indices of null values\n",
    "null_ind = [df_null[df_null].index[i][1] for i in np.arange(df_null[df_null].shape[0])]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(null_ind)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# See how many of each number are represented\n",
    "# Need to use float because int produces error with NaN\n",
    "fig = sns.countplot(features['bare nuclei'].astype(float))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There are only 16 missing values, all in the bare nuclei. After looking up how bare nuclei are used in cancer detection, I believe setting the value of bare nuclei to the mean, median, or mode isn't desired. With more time, I would model the bare nuclei to impute the missing values. But given the time constraint and how few missing values there are, I'm going to drop the missing values."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Impute missing values with mode of column\n",
    "X = features.dropna()\n",
    "X = X.astype(int)      # Convert to integer, the bare nuclei column is string"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Drop null rows\n",
    "classes = classes.drop(null_ind, axis=0)\n",
    "feat_class = feat_class.drop(null_ind, axis=0)\n",
    "ids = ids.drop(null_ind, axis=0)\n",
    "\n",
    "# Make the targets array with benign = 2 --> 0, malignant = 4 --> 1\n",
    "y = classes.replace(2, 0)\n",
    "y = y.replace(4, 1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Visualizing Features"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Make a pairwise plot of the feature set\n",
    "sns.pairplot(feat_class.astype(int), hue = 'class')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Unbalanced data\n",
    "\n",
    "There are twice as many benign points as malignant, so the data are unbalanced. Since we have a relatively small number of data points, we'll use SMOTE to balance the data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print('Original dataset shape {}'.format(Counter(y)))\n",
    "sm = SMOTE()\n",
    "X_res, y_res = sm.fit_resample(X, y)\n",
    "print('Resampled dataset shape {}'.format(Counter(y_res)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Running and interpreting the model\n",
    "\n",
    "I'm choosing the Random Forest Classifier. Given the time constraint, I wanted a model that didn't need hyperparameter tuning, was relatively fast, and was intuitive. Random Forests also work well with discrete value features and don't assume linear behavior.\n",
    "\n",
    "I'm doing a train/test split since there isn't a separate testing set. For performance metrics I could have done k-fold cross-validation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Train/test split: want to test the model on testing data it has never seen\n",
    "X_train, X_test, y_train, y_test = train_test_split(X_res, y_res, test_size=0.2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "# Define the classifier, fit on training data\n",
    "rf = RandomForestClassifier()\n",
    "rf.fit(X_train, y_train)\n",
    "y_pred = rf.predict(X_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Show feature importances\n",
    "importances = rf.feature_importances_\n",
    "print(importances)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Classification report\n",
    "target_names = ['benign', 'malignant']\n",
    "print(classification_report(y_test, y_pred, target_names=target_names))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Define confusion matrix plot - from sklearn example\n",
    "def plot_confusion_matrix(cm, classes,\n",
    "                          normalize=False,\n",
    "                          title='Confusion matrix',\n",
    "                          cmap=plt.cm.Blues):\n",
    "    \"\"\"\n",
    "    This function prints and plots the confusion matrix.\n",
    "    Normalization can be applied by setting `normalize=True`.\n",
    "    \"\"\"\n",
    "    plt.imshow(cm, interpolation='nearest', cmap=cmap)\n",
    "    plt.title(title)\n",
    "    plt.colorbar()\n",
    "    tick_marks = np.arange(len(classes))\n",
    "    plt.xticks(tick_marks, classes, rotation=45)\n",
    "    plt.yticks(tick_marks, classes)\n",
    "\n",
    "    if normalize:\n",
    "        cm = cm.astype('float') / cm.sum(axis=1)[:, np.newaxis]\n",
    "        print(\"Normalized confusion matrix\")\n",
    "    else:\n",
    "        print('Confusion matrix, without normalization')\n",
    "\n",
    "    print(cm)\n",
    "\n",
    "    thresh = cm.max() / 2.\n",
    "    for i, j in itertools.product(range(cm.shape[0]), range(cm.shape[1])):\n",
    "        plt.text(j, i, cm[i, j],\n",
    "                 horizontalalignment=\"center\",\n",
    "                 color=\"white\" if cm[i, j] > thresh else \"black\")\n",
    "\n",
    "    plt.tight_layout()\n",
    "    plt.ylabel('True label')\n",
    "    plt.xlabel('Predicted label')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Compute confusion matrix\n",
    "cnf_matrix = confusion_matrix(y_test, y_pred)\n",
    "np.set_printoptions(precision=2)\n",
    "\n",
    "# Plot non-normalized confusion matrix\n",
    "plt.figure()\n",
    "plot_confusion_matrix(cnf_matrix, classes=target_names,\n",
    "                      title='Confusion matrix')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Plot feature importances\n",
    "plt.figure()\n",
    "plt.title(\"Feature importances\")\n",
    "plt.bar(range(X_test.shape[1]), importances,\n",
    "       color=\"g\", align=\"center\")\n",
    "plt.xticks(np.arange(X_test.shape[1]), ('thickness', 'uniformity size', 'uniformity shape', \n",
    "            'marginal adhesion', 'epithelial cell size', 'bare nuclei', 'bland chromatin', \n",
    "            'normal nucleoli', 'mitoses'), rotation=90)\n",
    "plt.xlim([-1, X_test.shape[1]])\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Conclusions\n",
    "\n",
    "* Cancer detection is an inherently unbalanced problem, there are usually more benign data points than malignant. To address this I used SMOTE.\n",
    "* Random Forest does a good job of modeling this particular problem in a short period of time.\n",
    "* Uniformity of size is the most important feature for this model. Note: Feature importances are not stable in random forests.\n",
    "* Additional steps to improve this model could include: Feature selection & dimensionality reduction, comparison with other models, cross-validation, and missing value imputation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}