Credit Risk Modeling.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Consumer loans are the most typical retail product where credit risk modeling is applied."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "PD (Probability of default): Logistic regression\n",
    "\n",
    "LGD (Loss given default): Beta regression\n",
    "\n",
    "EAD (Exposure at default): Beta regression"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "PD needs a flag of whether borrower defaulted or not (loan_status column helps).\n",
    "\n",
    "LGD: How much of the loan was recovered after borrower had defaulted (recoveries column helps)\n",
    "\n",
    "EAD: Total exposure the moment the borrower defaulted compared to total exosure in the past (total_rec_prncp column helps)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Grade is that of the external agency given as letters from A to G.\n",
    "\n",
    "DTI is debt-to-income ratio"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "PD model: All independent features need to be categorical. Grouping multiple categories into one to reduce the number of categories. We need to make continuous variables (Annual income, number of credit inquiries in the last 6 months) discrete using dummy variables."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`Fine classing:` Dividing the data into finite intervals (Ex: number of months since the loan has been granted can be grouped as less than 1, 1 to 3, 4 to 6...). The grouping of the variables will be based on if the adjacent category discriminates between deafulted and non=defaulted borrowers. If they don't discriminate, they are merged into one."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`Coarse classing:` After grouping based on discrimination between defaulted and non-defaulted, categories are obatained. There is no need for intervals to be equal."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Data Preparation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Import libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Import Data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "loan_data_backup = pd.read_csv('loan.csv')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It's always a good practice to store a copy of our data before making any changes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "loan_data = loan_data_backup.copy()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Explore Data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "loan_data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`loan_data` just displays a few columns."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "pd.options.display.max_columns = None"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Hereafter, pandas displays all the columns of all objects."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "loan_data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "TO view the first 5 records of the dataframe:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "loan_data.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To view the last 5 records of a dataframe:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "loan_data.tail()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To see the names of all columns:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "loan_data.columns.values"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To see the datatype of all columns:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "loan_data.info()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Object datatype is for text strings."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# General Preprocessing"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Preprocessing few continuous variables"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Term and emp_length are strings rather than numeric. Let's correct them"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#To see the values that emp_length takes\n",
    "loan_data['emp_length'].unique()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Because there is the word 'years', the datatype is string. We need to get rid of this word and '+', '<'."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "loan_data['emp_length_int'] = loan_data['emp_length'].str.replace('\\+ years', '')\n",
    "loan_data['emp_length_int'] = loan_data['emp_length_int'].str.replace('< 1 year', str(0))\n",
    "loan_data['emp_length_int'] = loan_data['emp_length_int'].str.replace('n/a', str(0))\n",
    "loan_data['emp_length_int'] = loan_data['emp_length_int'].str.replace(' years', '')\n",
    "loan_data['emp_length_int'] = loan_data['emp_length_int'].str.replace(' year', '')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "type(loan_data['emp_length_int'][0]) #The type is still string"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "loan_data['emp_length_int'] = pd.to_numeric(loan_data['emp_length_int']) #Converts a series into numeric type"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "type(loan_data['emp_length_int'][0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Preprocessing term variable"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "loan_data['term'].unique()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "loan_data['term_int'] = loan_data['term'].str.replace(' months', '')\n",
    "loan_data['term_int'] = loan_data['term_int'].str.replace(' ', '')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "type(loan_data['term_int'][0])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "loan_data['term_int'] = pd.to_numeric(loan_data['term_int'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "type(loan_data['term_int'][0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Preprocessing date variables (earliest_cr_line)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Earliest credit line should be of type date but is an object instead."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "loan_data['earliest_cr_line']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Convert this into number of months that has passed since the given month and year."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "loan_data['earliest_cr_line_date'] = pd.to_datetime(loan_data['earliest_cr_line'], format = '%b-%y')\n",
    "#%b indicates first 3 letters of the month, %y indicates last 2 digits of the year"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "type(loan_data['earliest_cr_line_date'][0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We need to provide a reference date to calculate the number of months that has passed. Taking today's date is a standard practice. Since this is older data, let's take December 1, 2017 as the reference."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "pd.to_datetime('2017-12-01') - loan_data['earliest_cr_line_date']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We prefer working with months, we can just divide by 30. Better approach is to get the difference in months."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Take the difference and divide by `np.deltatime(1, 'M')` to get in months and round to get a whole number."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "loan_data['mths_since_earliest_cr_line'] = round(pd.to_numeric((pd.to_datetime('2017-12-01') - loan_data['earliest_cr_line_date']) / np.timedelta64(1, 'M')))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Descriptive statistics\n",
    "loan_data['mths_since_earliest_cr_line'].describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The time difference cannot be negative. Let's find out what's wrong."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "loan_data.loc[: , ['earliest_cr_line', 'earliest_cr_line_date', 'mths_since_earliest_cr_line']][loan_data['mths_since_earliest_cr_line'] < 0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The problem is while converting mth-year initially, it interpreted as 2065, 2067... instead of 1965, 1967. The issue arose in the first place because the origin of the built-in time scale starts after 1970. Removing these will not impact our conclusion. Intead of removing, we will impute the values (substitute the maximum observed difference, as they are in the distant past, somewhere in the 60s). "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "loan_data['mths_since_earliest_cr_line'][loan_data['mths_since_earliest_cr_line'] < 0] = loan_data['mths_since_earliest_cr_line'].max()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "min(loan_data['mths_since_earliest_cr_line'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Preprocessing date variable (issue_date)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "loan_data['issue_d']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "loan_data['issue_d_date'] = pd.to_datetime(loan_data['issue_d'], format = '%b-%y')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "type(loan_data['issue_d_date'][0])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "pd.to_datetime('2017-12-01') - loan_data['issue_d_date']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "loan_data['mths_since_issue_d_date'] = round(pd.to_numeric((pd.to_datetime('2017-12-01') - loan_data['issue_d_date']) / np.timedelta64(1, 'M')))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "loan_data['mths_since_issue_d_date'].describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Preprocessing few discrete variables"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Dummy variables have to be created which are binary indicators: 1 if an observation belongs to a category, else 0.\n",
    "For example, gender. Dummy variables would be Male (1 for Male, 0 for Female) and Female (1 for Female, 0 for Male). The other dummy variable is redundant. \n",
    "\n",
    "`Conclusion:` 1 dummy variable is enough to represent 2 categories. k - 1 dummy variables for k categories"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will create a new dataframe for dummy variables. Concatenate this with `loan_data` dataframe"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Grade column"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "pd.get_dummies(loan_data['grade'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`get_dummies` gets number of dummy variables equal to number of categories. The names of the dummy variables are same as the categories."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#To make the names more descriptive:\n",
    "pd.get_dummies(loan_data['grade'], prefix = 'Grade', prefix_sep = ' : ')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "loan_data_dummies = [pd.get_dummies(loan_data['grade'], prefix = 'Grade', prefix_sep = ' : '),\n",
    "                     pd.get_dummies(loan_data['sub_grade'], prefix = 'sub_grade', prefix_sep = ' : '),\n",
    "                     pd.get_dummies(loan_data['home_ownership'], prefix = 'home_ownership', prefix_sep = ' : '),\n",
    "                     pd.get_dummies(loan_data['verification_status'], prefix = 'verification_status', prefix_sep = ' : '),\n",
    "                     pd.get_dummies(loan_data['loan_status'], prefix = 'loan_status', prefix_sep = ' : '),\n",
    "                     pd.get_dummies(loan_data['purpose'], prefix = 'purpose', prefix_sep = ' : '),\n",
    "                     pd.get_dummies(loan_data['addr_state'], prefix = 'addr_state', prefix_sep = ' : '),\n",
    "                     pd.get_dummies(loan_data['initial_list_status'], prefix = 'initial_list_status', prefix_sep = ' : ')\n",
    "                    ]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Converting list into a dataframe\n",
    "loan_data_dummies = pd.concat(loan_data_dummies, axis = 1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "type(loan_data_dummies)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Appending this dataframe to the original one\n",
    "loan_data = pd.concat([loan_data, loan_data_dummies], axis = 1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "loan_data.columns.values"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Check for missing values and clean"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "loan_data.isnull()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#number of missing values in all columns\n",
    "pd.options.display.max_rows = None\n",
    "loan_data.isnull().sum()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#filling missing values with the funded amount in the same place (variable)\n",
    "loan_data['total_rev_hi_lim'].fillna(loan_data['funded_amnt'], inplace = True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "loan_data['total_rev_hi_lim'].isnull().sum()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#df_inputs_prepr['total_rev_hi_lim'].unique()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#df_inputs_prepr['total_rev_hi_lim_factor'] = pd.cut(df_inputs_prepr['total_rev_hi_lim'], 50)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#df_temp = woe_ordered_continuous(df_inputs_prepr, df_targets_prepr)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Preprocessing annual income"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "loan_data['annual_inc'].isnull().sum()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "mean_annual_income = loan_data['annual_inc'].mean()\n",
    "loan_data['annual_inc'].fillna(mean_annual_income, inplace = True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "loan_data['annual_inc'].isnull().sum()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "loan_data['mths_since_earliest_cr_line'].fillna(0, inplace = True)\n",
    "loan_data['acc_now_delinq'].fillna(0, inplace = True)\n",
    "loan_data['total_acc'].fillna(0, inplace = True)\n",
    "loan_data['pub_rec'].fillna(0, inplace = True)\n",
    "loan_data['open_acc'].fillna(0, inplace = True)\n",
    "loan_data['inq_last_6mths'].fillna(0, inplace = True)\n",
    "loan_data['delinq_2yrs'].fillna(0, inplace = True)\n",
    "loan_data['emp_length_int'].fillna(0, inplace = True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "loan_data['mths_since_earliest_cr_line'].isnull().sum()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "loan_data['acc_now_delinq'].isnull().sum()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "loan_data['total_acc'].isnull().sum()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "loan_data['pub_rec'].isnull().sum()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "loan_data['open_acc'].isnull().sum()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "loan_data['inq_last_6mths'].isnull().sum()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "loan_data['delinq_2yrs'].isnull().sum()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "loan_data['emp_length_int'].isnull().sum()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Credit Risk calculation means calculating expected loss.\n",
    "\n",
    "`Expected loss = Probability of default * Loss given default * Exposure at default`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Deafult: 0, Non-default: 1\n",
    "Many definitions of default: 90 days overdue payment, committed a fraud."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "PD model: Logisitic regression (LR). Dependent variable (output variable) ranges between 0 (default) and 1 (non-default). "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "LR estimates the relationship between ln(odds) of outcome variable and a linear combination of independent variables. \n",
    "ln(odds) = ln(non-defaults / defaults)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "PD model should be easy to use and understand. So, all independent variables in PD model should be dummy variables. Discrete variables have already been converted into dummy variables. Continuous variables too should be converted. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# PD model "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Data Preparation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "loan_data['loan_status'].unique()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`Charged off:` Borrower declaring that it's highly unlikely to pay debt.\n",
    "\n",
    "`Grace period:` A set length of time after the due date during which payment may be made without penalty."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Number of people with each loan_status\n",
    "loan_data['loan_status'].value_counts()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We need to find the coefficients of the independent variables (logistic regression)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Ratio of each counts\n",
    "loan_data['loan_status'].value_counts() / loan_data['loan_status'].count()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Applying loan default and non-default definition\n",
    "#Where works like if, else\n",
    "#isin checks if values are in a list\n",
    "#2nd arg (0): if condition is True, returns 0, else 1 (3rd arg)\n",
    "loan_data['good_bad'] = np.where(loan_data['loan_status'].isin(['Charged Off', 'Default', 'Does not meet the credit policy. Status:Charged Off', \n",
    "                                                                'Late (31-120 days)']), 0, 1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "pd.options.display.max_rows = 10\n",
    "loan_data['good_bad']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Weight of evidence = ln(%good <sub>i</sub> / %bad <sub>i</sub> )\n",
    "\n",
    "%good <sub>i</sub> is the ratio of good in a category to the total number of goods.\n",
    "\n",
    "%bad <sub>i</sub> is the ratio of bad in a category to the total number of bads"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "    Weight of evidence: To what extent does an independent variable would predict a dependent variable. WOE is used to group multiple categories. Categories with similar WOE are bundled together. Further away from 0 WOE is, is better in differentiating categories."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Information value: How much informaiton an ind. variable brings in explaining the dependent variable. It helps for pre-selection of features. Range of IV is 0 - 1.  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`Information value = sum of all ((%good <sub>i</sub> - %bad <sub>i</sub>) * WOE)`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Scale of information value: (PP means predictive power)\n",
    "\n",
    "    IV<0.02: No PP\n",
    "    0.02<IV<0.1: Weak PP\n",
    "    0.1<IV<0.3: Medium PP\n",
    "    0.3<IV<0.5: Strong PP\n",
    "    IV>=0.5: Suspiciously high, too good to be true"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Splitting data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "loan_data_inputs_train, loan_data_inputs_test, loan_data_targets_train, loan_data_targets_test = train_test_split(loan_data.drop('good_bad', axis = 1), loan_data['good_bad'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "loan_data_inputs_train.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "loan_data_targets_train.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "loan_data_inputs_test.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "loan_data_targets_test.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "By default, sklearn splits train: test = 75%:25% (349713:116572). To get 80%:20% split, use `test_size`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "loan_data_inputs_train, loan_data_inputs_test, loan_data_targets_train, loan_data_targets_test = train_test_split(\n",
    "    loan_data.drop('good_bad', axis = 1), loan_data['good_bad'], test_size = 0.2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "By default, shuffling (boolean var) = True. It means that sklearn shuffles the data. So, every run gives a different split, loss and accuracy. To get the same split everytime we run, use `random_state`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "loan_data_inputs_train, loan_data_inputs_test, loan_data_targets_train, loan_data_targets_test = train_test_split(\n",
    "    loan_data.drop('good_bad', axis = 1), loan_data['good_bad'], test_size = 0.2, random_state = 42)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "loan_data_inputs_train.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "loan_data_targets_train.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "loan_data_targets_test.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "loan_data_inputs_test.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Data Preparation: An Example"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_inputs_prepr = loan_data_inputs_train\n",
    "df_targets_prepr = loan_data_targets_train\n",
    "\n",
    "#Assigning test dataset for test dataset preprocessing\n",
    "#df_inputs_prepr = loan_data_inputs_test\n",
    "#df_targets_prepr = loan_data_targets_test"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_inputs_prepr.columns.values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_inputs_prepr['grade'].unique()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "pd.options.display.max_columns = 5\n",
    "df1 = pd.concat([df_inputs_prepr['grade'], df_targets_prepr], axis = 1)\n",
    "df1.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A = Highest credit worthiness,  G = Lowest credit worthiness"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Count of each grade\n",
    "df1.groupby(df1.columns.values[0], as_index=False)[df1.columns.values[1]].count()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Proportion of good borrowers = 1 - Proportion of bad borrowers"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Proportion of good for each grade\n",
    "df1.groupby(df1.columns.values[0], as_index=False)[df1.columns.values[1]].mean()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Mergigng both the above outputs into 1 dataframe\n",
    "df1 = pd.concat([df1.groupby(df1.columns.values[0], as_index=False)[df1.columns.values[1]].count(),\n",
    "                df1.groupby(df1.columns.values[0], as_index=False)[df1.columns.values[1]].mean()], axis = 1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can be certain that the concatenation has happened correctly, since the grade column matches for every row. Let's get rid of the the second `grade` column using df1.iloc[:, [0,1,3]]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Renaming column names\n",
    "df1 = df1.iloc[:, [0,1,3]]\n",
    "df1.columns = [df1.columns.values[0], 'n_obs', 'prop_good']\n",
    "df1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Calculating proportion of observations\n",
    "df1['prop_n_obs'] = df1['n_obs'] / df1['n_obs'].sum()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Calculating number of good and bad borrowers for each grade\n",
    "pd.options.display.max_columns = None\n",
    "df1['n_good'] = df1['prop_good'] * df1['n_obs']\n",
    "df1['n_bad'] = (1 - df1['prop_good']) * df1['n_obs']\n",
    "df1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Calculating proportion of good and bad\n",
    "df1['prop_n_good'] = df1['n_good'] / df1['n_good'].sum()\n",
    "df1['prop_n_bad'] = df1['n_bad'] / df1['n_bad'].sum()\n",
    "df1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Calculating Weight of evidence\n",
    "df1['WOE'] = np.log(df1['prop_n_good'] / df1['prop_n_bad'])\n",
    "df1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df1 = df1.sort_values(['WOE'])\n",
    "df1 = df1.reset_index(drop=True)\n",
    "df1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Calculating the difference between rows (above - below)\n",
    "df1['diff_prop_good'] = df1['prop_good'].diff().abs()\n",
    "df1['diff_WOE'] = df1['WOE'].diff().abs()\n",
    "df1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Calculating Information Value (IV)\n",
    "df1['IV'] = (df1['prop_n_good'] - df1['prop_n_bad']) * df1['WOE']\n",
    "df1['IV'] = df1['IV'].sum()\n",
    "df1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "IV is the same for all rows, as it's the value for grade overall."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Preprocessing discrete variables: Automating calculations"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def woe_discrete(df, discrete_variable_name, good_bad_variable_df):\n",
    "    df = pd.concat([df[discrete_variable_name], good_bad_variable_df], axis=1)\n",
    "    df = pd.concat([df.groupby(df.columns.values[0], as_index=False)[df.columns.values[1]].count(),\n",
    "                   df.groupby(df.columns.values[0], as_index=False)[df.columns.values[1]].mean()], axis=1)\n",
    "    df = df.iloc[:, [0,1,3]]\n",
    "    df.columns = [df.columns.values[0], 'n_obs', 'prop_good']\n",
    "    df['prop_n_obs'] = df['n_obs'] / df['n_obs'].sum()\n",
    "    df['n_good'] = df['prop_good'] * df['n_obs']\n",
    "    df['n_bad'] = (1 - df['prop_good']) * df['n_obs']\n",
    "    df['prop_n_good'] = df['n_good'] / df['n_good'].sum()\n",
    "    df['prop_n_bad'] = df['n_bad'] / df['n_bad'].sum()\n",
    "    df['WOE'] = np.log(df['prop_n_good'] / df['prop_n_bad'])\n",
    "    df = df.sort_values(['WOE'])\n",
    "    df = df.reset_index(drop=True)\n",
    "    df['diff_prop_good'] = df['prop_good'].diff().abs()\n",
    "    df['diff_WOE'] = df['WOE'].diff().abs()\n",
    "    df['IV'] = (df['prop_n_good'] - df['prop_n_bad']) * df['WOE']\n",
    "    df['IV'] = df['IV'].sum()\n",
    "    return df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_temp = woe_discrete(df_inputs_prepr, 'grade', df_targets_prepr)\n",
    "df_temp"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Preprocessing Discrete variables: Visualizing Results"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "sns.set()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def plot_by_woe(df_WOE, rotation_of_x_axis_labels = 0):\n",
    "    #converting the independent variable categories into strings and making an array\n",
    "    x = np.array(df_WOE.iloc[:,0].apply(str))\n",
    "    y = df_WOE['WOE']\n",
    "    #width = 18 inches, height = 6 inches\n",
    "    plt.figure(figsize = (18,6))\n",
    "    plt.plot(x, y, marker = 'o', linestyle = '--', color = 'k') #marker='o': displays a dot for each point, dashed lines, black color\n",
    "    plt.xlabel(df_WOE.columns[0])\n",
    "    plt.ylabel('Weight of Evidence')\n",
    "    plt.title(str('Weight of Evidence by ' + df_WOE.columns[0]))\n",
    "    plt.xticks(rotation = rotation_of_x_axis_labels)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plot_by_woe(df_temp)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The category with the lowest weight of evidence is the reference category."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_temp = woe_discrete(df_inputs_prepr, 'home_ownership', df_targets_prepr)\n",
    "df_temp"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plot_by_woe(df_temp)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Preprocessing discrete variables: Creating dummy variables"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Home ownership variable"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Since OTHER, NONE, OWN, ANY have such a low number of observations, we will combine them into 1 dummy variable"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_inputs_prepr['home_ownership : RENT_OTHER_NONE_ANY'] = sum([df_inputs_prepr['home_ownership : RENT'], \n",
    "df_inputs_prepr['home_ownership : OTHER'], df_inputs_prepr['home_ownership : NONE'], df_inputs_prepr['home_ownership : ANY']])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Addr_state variable"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_inputs_prepr['addr_state'].unique()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "pd.options.display.max_rows = None\n",
    "df_temp = woe_discrete(df_inputs_prepr, 'addr_state', df_targets_prepr)\n",
    "df_temp"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plot_by_woe(df_temp)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "North Dakota (ND) is not present in the plot on the x-axis since there are no borrowers from that state."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#set all values of the state ND to 0 if there is no such column, else pass\n",
    "if ['addr_state : ND'] in df_inputs_prepr.columns.values:\n",
    "    pass\n",
    "else:\n",
    "    df_inputs_prepr['addr_state : ND'] = 0"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Nebraska (NE) and Iowa (IA) have the lowest WOE, WOE of Maine (ME) and Idaho (ID) couldn't be calculated as there are no bad borrowers from that state (denominato = 0, => WOE = inf)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "NE, IA, ME, ID have low number of observations. This might be the reason for their extreme WOE values. The graph says that for the other 46 states, WOE is more or less the same (except first 2 and last 2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Plotting without first 2 and last 2 states\n",
    "plot_by_woe(df_temp.iloc[2:-2, :])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We were misguided that for the 46 states, WOE is almost the same. Now, we see it from a different perspective. We can group first 6 + ND (the state with information is assumed in the WORST category), last 6 in one category. Let's plot the remaining ones:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plot_by_woe(df_temp.iloc[6:-6, :])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Categories:\n",
    "* NE to AL + ND\n",
    "* NM, VA\n",
    "* NY (2nd highest)\n",
    "* OK to NC\n",
    "* CA (highest number of borrowers)\n",
    "* UT to NJ\n",
    "* AR to MN\n",
    "* RI to IN\n",
    "* GA to OR\n",
    "* WI, MT\n",
    "* TX (3rd highest)\n",
    "* IL, CT\n",
    "* KS to MS\n",
    "* WV to ID (low number of observations)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We just need to create dummy variables if we are combining more than 1 state into a category. Since, the single state dummy variables are already present."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Creating the actual dummy variables\n",
    "df_inputs_prepr['addr_state : ND_NE_IA_NV_FL_HI_AL'] = sum([df_inputs_prepr['addr_state : ND'], df_inputs_prepr['addr_state : NE'],\n",
    "df_inputs_prepr['addr_state : IA'], df_inputs_prepr['addr_state : NV'], df_inputs_prepr['addr_state : FL'], \n",
    "df_inputs_prepr['addr_state : HI'], df_inputs_prepr['addr_state : AL']])\n",
    "\n",
    "df_inputs_prepr['addr_state : NM_VA'] = sum([df_inputs_prepr['addr_state : NM'], df_inputs_prepr['addr_state : VA']])\n",
    "\n",
    "df_inputs_prepr['addr_state : OK_TN_MO_LA_MD_NC'] = sum([df_inputs_prepr['addr_state : OK'], \n",
    "df_inputs_prepr['addr_state : TN'], df_inputs_prepr['addr_state : MO'], df_inputs_prepr['addr_state : LA'],\n",
    "df_inputs_prepr['addr_state : MD'], df_inputs_prepr['addr_state : NC']])\n",
    "\n",
    "df_inputs_prepr['addr_state : UT_KY_AZ_NJ'] = sum([df_inputs_prepr['addr_state : UT'], df_inputs_prepr['addr_state : KY'],\n",
    "df_inputs_prepr['addr_state : AZ'], df_inputs_prepr['addr_state : NJ']])\n",
    "\n",
    "df_inputs_prepr['addr_state : AR_MI_PA_OH_MN'] = sum([df_inputs_prepr['addr_state : AR'], df_inputs_prepr['addr_state : MI'],\n",
    "df_inputs_prepr['addr_state : PA'], df_inputs_prepr['addr_state : OH'], df_inputs_prepr['addr_state : MN']])\n",
    "\n",
    "df_inputs_prepr['addr_state : RI_MA_DE_SD_IN'] = sum([df_inputs_prepr['addr_state : RI'], df_inputs_prepr['addr_state : MA'],\n",
    "df_inputs_prepr['addr_state : DE'], df_inputs_prepr['addr_state : SD'], df_inputs_prepr['addr_state : IN']])\n",
    "\n",
    "df_inputs_prepr['addr_state : GA_WA_OR'] = sum([df_inputs_prepr['addr_state : GA'], df_inputs_prepr['addr_state : WA'], \n",
    "df_inputs_prepr['addr_state : OR']])\n",
    "\n",
    "df_inputs_prepr['addr_state : WI_MT'] = sum([df_inputs_prepr['addr_state : WI'], df_inputs_prepr['addr_state : MT']])\n",
    "\n",
    "df_inputs_prepr['addr_state : IL_CT'] = sum([df_inputs_prepr['addr_state : IL'], df_inputs_prepr['addr_state : CT']])\n",
    "\n",
    "df_inputs_prepr['addr_state : KS_SC_CO_VT_AK_MS'] = sum([df_inputs_prepr['addr_state : KS'], \n",
    "df_inputs_prepr['addr_state : SC'], df_inputs_prepr['addr_state : CO'], df_inputs_prepr['addr_state : VT'],\n",
    "df_inputs_prepr['addr_state : AK'], df_inputs_prepr['addr_state : MS']])\n",
    "\n",
    "df_inputs_prepr['addr_state : WV_NH_WY_DC_ME_ID'] = sum([df_inputs_prepr['addr_state : WV'], \n",
    "df_inputs_prepr['addr_state : NH'], df_inputs_prepr['addr_state : WY'], df_inputs_prepr['addr_state : DC'],\n",
    "df_inputs_prepr['addr_state : ME'], df_inputs_prepr['addr_state : ID']])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Verification_status variable"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_temp = woe_discrete(df_inputs_prepr, 'verification_status', df_targets_prepr)\n",
    "df_temp"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plot_by_woe(df_temp)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`Verification_status` should not be used in the PD model as it's IV is only approx. 0.02 which means it has no predictive power"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Purpose variable"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_temp = woe_discrete(df_inputs_prepr, 'purpose', df_targets_prepr)\n",
    "df_temp"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plot_by_woe(df_temp, rotation_of_x_axis_labels = 90)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`Purpose` should not be used in the PD model since it has weak predictive power since it's IV falls in the second category"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_inputs_prepr['purpose : SB_ED'] = sum([df_inputs_prepr['purpose : small_business'], df_inputs_prepr['purpose : educational']])\n",
    "\n",
    "df_inputs_prepr['purpose : HO_OT_RE_ME'] = sum([df_inputs_prepr['purpose : house'], df_inputs_prepr['purpose : other'],\n",
    "df_inputs_prepr['purpose : renewable_energy'], df_inputs_prepr['purpose : medical']])\n",
    "\n",
    "df_inputs_prepr['purpose : WE_VA_DC'] = sum([df_inputs_prepr['purpose : wedding'], df_inputs_prepr['purpose : vacation'],\n",
    "df_inputs_prepr['purpose : debt_consolidation']])\n",
    "\n",
    "df_inputs_prepr['purpose : HI_MP_CA_CC'] = sum([df_inputs_prepr['purpose : home_improvement'], df_inputs_prepr['purpose : major_purchase'],\n",
    "df_inputs_prepr['purpose : car'], df_inputs_prepr['purpose : credit_card']])\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## initial_list_status variable"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_temp = woe_discrete(df_inputs_prepr, 'initial_list_status', df_targets_prepr)\n",
    "df_temp"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plot_by_woe(df_temp)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`initial_list_status` should not be used in the PD model since it has no predictive power as it's IV is approx. 0.02"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Preprocessing continuous variables: Automating calculations and visualizing results"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The same function used for discrete variables can be used for continuous variables. Discrete variables don't have qualitative comparison (so, sorted them by WOE). But continuous variables have a quantitative comparison (so,leave them in natural order). For example, all values in $80k - $90k  are less than $90k - $100k"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def woe_ordered_continuous(df, discrete_variable_name, good_bad_variable_df):\n",
    "    df = pd.concat([df[discrete_variable_name], good_bad_variable_df], axis=1)\n",
    "    df = pd.concat([df.groupby(df.columns.values[0], as_index=False)[df.columns.values[1]].count(),\n",
    "                   df.groupby(df.columns.values[0], as_index=False)[df.columns.values[1]].mean()], axis=1)\n",
    "    df = df.iloc[:, [0,1,3]]\n",
    "    df.columns = [df.columns.values[0], 'n_obs', 'prop_good']\n",
    "    df['prop_n_obs'] = df['n_obs'] / df['n_obs'].sum()\n",
    "    df['n_good'] = df['prop_good'] * df['n_obs']\n",
    "    df['n_bad'] = (1 - df['prop_good']) * df['n_obs']\n",
    "    df['prop_n_good'] = df['n_good'] / df['n_good'].sum()\n",
    "    df['prop_n_bad'] = df['n_bad'] / df['n_bad'].sum()\n",
    "    df['WOE'] = np.log(df['prop_n_good'] / df['prop_n_bad'])\n",
    "    df['diff_prop_good'] = df['prop_good'].diff().abs()\n",
    "    df['diff_WOE'] = df['WOE'].diff().abs()\n",
    "    df['IV'] = (df['prop_n_good'] - df['prop_n_bad']) * df['WOE']\n",
    "    df['IV'] = df['IV'].sum()\n",
    "    return df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_temp = woe_ordered_continuous(df_inputs_prepr, 'term_int', df_targets_prepr)\n",
    "df_temp"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plot_by_woe(df_temp)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "60 months loan are much riskier. We will give all 36 months term, value 1. Else, 0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_inputs_prepr['term : 36'] = np.where((df_inputs_prepr['term_int'] == 36), 1, 0)\n",
    "df_inputs_prepr['term : 60'] = np.where((df_inputs_prepr['term_int'] == 60), 1, 0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## emp_length variable"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_inputs_prepr['emp_length_int'].unique()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_temp = woe_ordered_continuous(df_inputs_prepr, 'emp_length_int', df_targets_prepr)\n",
    "df_temp"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plot_by_woe(df_temp)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_inputs_prepr['emp_length : 0'] = np.where(df_inputs_prepr['emp_length_int'].isin([0]), 1, 0)\n",
    "df_inputs_prepr['emp_length : 1'] = np.where(df_inputs_prepr['emp_length_int'].isin([1]), 1, 0)\n",
    "df_inputs_prepr['emp_length : 2-4'] = np.where(df_inputs_prepr['emp_length_int'].isin(range(2,5)), 1, 0)\n",
    "df_inputs_prepr['emp_length : 5-6'] = np.where(df_inputs_prepr['emp_length_int'].isin(range(5,7)), 1, 0)\n",
    "df_inputs_prepr['emp_length : 7-9'] = np.where(df_inputs_prepr['emp_length_int'].isin(range(7,10)), 1, 0)\n",
    "df_inputs_prepr['emp_length : 10'] = np.where(df_inputs_prepr['emp_length_int'].isin([10]), 1, 0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## mths_since_issue_date variable"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_inputs_prepr['mths_since_issue_d_date'].unique()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#We want to divide the above values into 50 categories since it's easy to work with < 50 categories\n",
    "#Fine classing\n",
    "df_inputs_prepr['mths_since_issue_d_date_factor'] = pd.cut(df_inputs_prepr['mths_since_issue_d_date'], 50)\n",
    "df_inputs_prepr['mths_since_issue_d_date_factor']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Each interval is from greater than 1st no. and less than or equal to 2nd no."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_temp = woe_ordered_continuous(df_inputs_prepr, 'mths_since_issue_d_date_factor', df_targets_prepr)\n",
    "df_temp"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plot_by_woe(df_temp, rotation_of_x_axis_labels=90)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Since the first 3 values have a high WOE than rest of the categories, let's plot without these 3 categories"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plot_by_woe(df_temp.iloc[3:, :], rotation_of_x_axis_labels=90)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "    Surely check the no. of observations if the plot goes up and down."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_inputs_prepr['mths_since_issue_d_date_factor : <38'] = np.where(df_inputs_prepr['mths_since_issue_d_date_factor'].isin(range(38)), 1, 0)\n",
    "df_inputs_prepr['mths_since_issue_d_date_factor : 38-39'] = np.where(df_inputs_prepr['mths_since_issue_d_date_factor'].isin(range(38,40)), 1, 0)\n",
    "df_inputs_prepr['mths_since_issue_d_date_factor : 40-41'] = np.where(df_inputs_prepr['mths_since_issue_d_date_factor'].isin(range(40,42)), 1, 0)\n",
    "df_inputs_prepr['mths_since_issue_d_date_factor : 42-48'] = np.where(df_inputs_prepr['mths_since_issue_d_date_factor'].isin(range(42,49)), 1, 0)\n",
    "df_inputs_prepr['mths_since_issue_d_date_factor : 49-52'] = np.where(df_inputs_prepr['mths_since_issue_d_date_factor'].isin(range(49,53)), 1, 0)\n",
    "df_inputs_prepr['mths_since_issue_d_date_factor : 53-64'] = np.where(df_inputs_prepr['mths_since_issue_d_date_factor'].isin(range(53,65)), 1, 0)\n",
    "df_inputs_prepr['mths_since_issue_d_date_factor : 65-84'] = np.where(df_inputs_prepr['mths_since_issue_d_date_factor'].isin(range(65,85)), 1, 0)\n",
    "df_inputs_prepr['mths_since_issue_d_date_factor : >84'] = np.where(df_inputs_prepr['mths_since_issue_d_date_factor'].isin(range(85, int(df_inputs_prepr['mths_since_issue_d_date'].max()))), 1, 0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Interest rate variable"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_inputs_prepr['int_rate_factor'] = pd.cut(df_inputs_prepr['int_rate'], 50)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_temp = woe_ordered_continuous(df_inputs_prepr, 'int_rate_factor' , df_targets_prepr)\n",
    "df_temp"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plot_by_woe(df_temp, rotation_of_x_axis_labels=90)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Greater the interest rate, higher WOE and greater probability of default."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_inputs_prepr['int_rate : <9.548'] = np.where((df_inputs_prepr['int_rate'] <= 9.548), 1, 0)\n",
    "df_inputs_prepr['int_rate : 9.548-12.025'] = np.where((df_inputs_prepr['int_rate'] > 9.548) & (df_inputs_prepr['int_rate'] <= 12.025), 1, 0)\n",
    "df_inputs_prepr['int_rate : 12.025-15.74'] = np.where((df_inputs_prepr['int_rate'] > 12.025) & (df_inputs_prepr['int_rate'] <= 15.74), 1, 0)\n",
    "df_inputs_prepr['int_rate : 15.74-20.281'] = np.where((df_inputs_prepr['int_rate'] > 15.74) & (df_inputs_prepr['int_rate'] <= 20.281), 1, 0)\n",
    "df_inputs_prepr['int_rate : >20.281'] = np.where((df_inputs_prepr['int_rate'] > 20.281), 1, 0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## funded_amt_factor variable"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_inputs_prepr['funded_amnt_factor'] = pd.cut(df_inputs_prepr['funded_amnt'], 50)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "df_temp = woe_ordered_continuous(df_inputs_prepr, 'funded_amnt_factor', df_targets_prepr)\n",
    "df_temp"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plot_by_woe(df_temp, 90)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`funded_amnt_factor` varies greatly. There seems to be no association between WOE and `funded_amt_factor`.So, we won't use this variable in our PD model."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## mths_since_earliest_cr_line variable"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "df_inputs_prepr['mths_since_earliest_cr_line'].unique()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_inputs_prepr['mths_since_earliest_cr_line_factor'] = pd.cut(df_inputs_prepr['mths_since_earliest_cr_line'], 50)\n",
    "df_inputs_prepr['mths_since_earliest_cr_line_factor']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "df_temp = woe_ordered_continuous(df_inputs_prepr, 'mths_since_earliest_cr_line_factor', df_targets_prepr)\n",
    "df_temp"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plot_by_woe(df_temp, rotation_of_x_axis_labels=90)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_inputs_prepr['mths_since_earliest_cr_line_factor : <70'] = np.where(df_inputs_prepr['mths_since_earliest_cr_line_factor'].isin(range(70)), 1, 0)\n",
    "df_inputs_prepr['mths_since_earliest_cr_line_factor : 70-93'] = np.where(df_inputs_prepr['mths_since_earliest_cr_line_factor'].isin(range(70,93)), 1, 0)\n",
    "df_inputs_prepr['mths_since_earliest_cr_line_factor : 94-140'] = np.where(df_inputs_prepr['mths_since_earliest_cr_line_factor'].isin(range(94,140)), 1, 0)\n",
    "df_inputs_prepr['mths_since_earliest_cr_line_factor : 141-270'] = np.where(df_inputs_prepr['mths_since_earliest_cr_line_factor'].isin(range(141-270)), 1, 0)\n",
    "df_inputs_prepr['mths_since_earliest_cr_line_factor : 271-352'] = np.where(df_inputs_prepr['mths_since_earliest_cr_line_factor'].isin(range(271,352)), 1, 0)\n",
    "df_inputs_prepr['mths_since_earliest_cr_line_factor : 353-410'] = np.where(df_inputs_prepr['mths_since_earliest_cr_line_factor'].isin(range(353,410)), 1, 0)\n",
    "df_inputs_prepr['mths_since_earliest_cr_line_factor : 411-563'] = np.where(df_inputs_prepr['mths_since_earliest_cr_line_factor'].isin(range(411,563)), 1, 0)\n",
    "df_inputs_prepr['mths_since_earliest_cr_line_factor : >563'] = np.where(df_inputs_prepr['mths_since_earliest_cr_line_factor'].isin(range(563, int(df_inputs_prepr['mths_since_earliest_cr_line'].max()))), 1, 0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Installment variable"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "unique_installment = []\n",
    "for value in df_inputs_prepr['installment']:\n",
    "    if value not in unique_installment:\n",
    "        unique_installment.append(value)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "len(unique_installment)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When I used .unique() method, it didn't display all values in the list (it used ellipsis). To display all values I used a for loop and an if statement. To find the number of unique values, use len()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_inputs_prepr['installment_factor'] = pd.cut(df_inputs_prepr['installment'], 50)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "df_temp = woe_ordered_continuous(df_inputs_prepr, 'installment_factor', df_targets_prepr)\n",
    "df_temp"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plot_by_woe(df_temp, rotation_of_x_axis_labels=90)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_inputs_prepr['installment_factor : <183'] = np.where(df_inputs_prepr['installment_factor'].isin(range(183)), 1, 0)\n",
    "df_inputs_prepr['installment_factor : 183-266'] = np.where(df_inputs_prepr['installment_factor'].isin(range(183,266)), 1, 0)\n",
    "df_inputs_prepr['installment_factor : 267-517'] = np.where(df_inputs_prepr['installment_factor'].isin(range(267,517)), 1, 0)\n",
    "df_inputs_prepr['installment_factor : 518-601'] = np.where(df_inputs_prepr['installment_factor'].isin(range(518,601)), 1, 0)\n",
    "df_inputs_prepr['installment_factor : 602-880'] = np.where(df_inputs_prepr['installment_factor'].isin(range(602,880)), 1, 0)\n",
    "df_inputs_prepr['installment_factor : 881-963'] = np.where(df_inputs_prepr['installment_factor'].isin(range(881,963)), 1, 0)\n",
    "df_inputs_prepr['installment_factor : 964-1075'] = np.where(df_inputs_prepr['installment_factor'].isin(range(964,1075)), 1, 0)\n",
    "df_inputs_prepr['installment_factor : 1076-1242'] = np.where(df_inputs_prepr['installment_factor'].isin(range(1076,1242)), 1, 0)\n",
    "df_inputs_prepr['installment_factor : >1242'] = np.where(df_inputs_prepr['installment_factor'].isin(range(1242, int(df_inputs_prepr['installment'].max()))), 1, 0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## delinq_2yrs variable"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_inputs_prepr['delinq_2yrs'].unique()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Since there are less number of categories, there is no need to use `pd.cut()`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "df_temp = woe_ordered_continuous(df_inputs_prepr, 'delinq_2yrs', df_targets_prepr)\n",
    "df_temp"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plot_by_woe(df_temp.iloc[:15,:])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To get rid of rows with WOE `inf`:\n",
    "    \n",
    "    df_temp = df_temp.iloc[[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,17,18,20], :]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_temp"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plot_by_woe(df_temp)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_inputs_prepr['delinq_2yrs : <4'] = np.where(df_inputs_prepr['delinq_2yrs'].isin(range(0,3)), 1, 0)\n",
    "df_inputs_prepr['delinq_2yrs : >=4'] = np.where(df_inputs_prepr['delinq_2yrs'].isin(range(4,21)), 1, 0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## inq_last_6mths variable"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_inputs_prepr['inq_last_6mths'].unique()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "df_temp = woe_ordered_continuous(df_inputs_prepr, 'inq_last_6mths', df_targets_prepr)\n",
    "df_temp"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There are rows with `inf` or `-inf` as WOE. To get rid of those, do as below: (the above way is a lengthy one as it requires explicitly naming row indices)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_temp = df_temp[(df_temp['WOE'] != float('inf')) & (df_temp['WOE'] != float('-inf'))]\n",
    "df_temp"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plot_by_woe(df_temp)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_inputs_prepr['inq_last_6mths : 0-3'] = np.where(df_inputs_prepr['inq_last_6mths'].isin(range(0,4)), 1, 0)\n",
    "df_inputs_prepr['inq_last_6mths : 4-6'] = np.where(df_inputs_prepr['inq_last_6mths'].isin(range(4,7)), 1, 0)\n",
    "df_inputs_prepr['inq_last_6mths : >=6'] = np.where(df_inputs_prepr['inq_last_6mths'].isin(range(6,18)), 1, 0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## open_acc variable"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_inputs_prepr['open_acc'].unique()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_inputs_prepr['open_acc_factor'] = pd.cut(df_inputs_prepr['open_acc'], 50)\n",
    "df_inputs_prepr['open_acc_factor']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "df_temp = woe_ordered_continuous(df_inputs_prepr, 'open_acc_factor', df_targets_prepr)\n",
    "df_temp"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plot_by_woe(df_temp, rotation_of_x_axis_labels=90)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There is no need to remove `inf`, `-inf` and `NaN` as the plot function itself ignored those observations"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_inputs_prepr['open_acc_factor : 0'] = np.where(df_inputs_prepr['open_acc_factor'].isin(range(0)), 1, 0)\n",
    "df_inputs_prepr['open_acc_factor : 1-3'] = np.where(df_inputs_prepr['open_acc_factor'].isin(range(1,3)), 1, 0)\n",
    "df_inputs_prepr['open_acc_factor : 3-11'] = np.where(df_inputs_prepr['open_acc_factor'].isin(range(3,12)), 1, 0)\n",
    "df_inputs_prepr['open_acc_factor : 12-25'] = np.where(df_inputs_prepr['open_acc_factor'].isin(range(12,26)), 1, 0)\n",
    "df_inputs_prepr['open_acc_factor : 26-33'] = np.where(df_inputs_prepr['open_acc_factor'].isin(range(26,34)), 1, 0)\n",
    "df_inputs_prepr['open_acc_factor : >34'] = np.where(df_inputs_prepr['open_acc_factor'].isin(range(34, int(df_inputs_prepr['open_acc'].max()))), 1, 0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## pub_rec variable"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_inputs_prepr['pub_rec'].unique()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_temp = woe_ordered_continuous(df_inputs_prepr, 'pub_rec', df_targets_prepr) \n",
    "df_temp"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plot_by_woe(df_temp)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#removing all inf values\n",
    "df_temp = df_temp[df_temp['WOE'] != float('inf')]\n",
    "df_temp"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plot_by_woe(df_temp)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_inputs_prepr['pub_rec : 0-2'] = np.where(df_inputs_prepr['pub_rec'].isin(range(0,3)), 1, 0)\n",
    "df_inputs_prepr['pub_rec : 3-4'] = np.where(df_inputs_prepr['pub_rec'].isin(range(3,5)), 1, 0)\n",
    "df_inputs_prepr['pub_rec : 5-7'] = np.where(df_inputs_prepr['pub_rec'].isin(range(5,7)), 1, 0)\n",
    "df_inputs_prepr['pub_rec : >7'] = np.where(df_inputs_prepr['pub_rec'].isin(range(7,int(df_inputs_prepr['pub_rec'].max()))), 1, 0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## total_acc variable"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_inputs_prepr['total_acc'].unique()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_inputs_prepr['total_acc_factor'] = pd.cut(df_inputs_prepr['total_acc'], 50)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "df_temp = woe_ordered_continuous(df_inputs_prepr, 'total_acc_factor', df_targets_prepr)\n",
    "df_temp"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plot_by_woe(df_temp, rotation_of_x_axis_labels=90)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_inputs_prepr['total_acc_factor : <=27'] = np.where(df_inputs_prepr['total_acc_factor'].isin(range(0,28)), 1, 0)\n",
    "df_inputs_prepr['total_acc_factor : 28-30'] = np.where(df_inputs_prepr['total_acc_factor'].isin(range(18,30)), 1, 0)\n",
    "df_inputs_prepr['total_acc_factor : 30-45'] = np.where(df_inputs_prepr['total_acc_factor'].isin(range(30,45)), 1, 0)\n",
    "df_inputs_prepr['total_acc_factor : 45-60'] = np.where(df_inputs_prepr['total_acc_factor'].isin(range(45,60)), 1, 0)\n",
    "df_inputs_prepr['total_acc_factor : 60-72'] = np.where(df_inputs_prepr['total_acc_factor'].isin(range(60,72)), 1, 0)\n",
    "df_inputs_prepr['total_acc_factor : >72'] = np.where(df_inputs_prepr['total_acc_factor'].isin(range(72, int(df_inputs_prepr['total_acc'].max()))), 1, 0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## acc_now_delinq variable"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_inputs_prepr['acc_now_delinq'].unique()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_temp = woe_ordered_continuous(df_inputs_prepr, 'acc_now_delinq', df_targets_prepr)\n",
    "df_temp"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#df_temp = df_temp.iloc[[0,1,2,3,5], :]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plot_by_woe(df_temp, rotation_of_x_axis_labels=90)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_inputs_prepr['acc_now_delinq : 0'] = np.where(df_inputs_prepr['acc_now_delinq'].isin(range(0)), 1, 0)\n",
    "df_inputs_prepr['acc_now_delinq : 1-5'] = np.where(df_inputs_prepr['acc_now_delinq'].isin(range(1, 5)), 1, 0) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_inputs_prepr.columns.values"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Annual income variable"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_inputs_prepr['annual_inc_factor'] = pd.cut(df_inputs_prepr['annual_inc'], 50)\n",
    "df_inputs_prepr['annual_inc_factor']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "df_temp = woe_ordered_continuous(df_inputs_prepr, 'annual_inc_factor', df_targets_prepr)\n",
    "df_temp"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "94% of the observations are in one category. It seems that 50 categories weren't enough to split our data well. Let's split into 100 categories."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "df_inputs_prepr['annual_inc_factor'] = pd.cut(df_inputs_prepr['annual_inc'], 100)\n",
    "df_temp = woe_ordered_continuous(df_inputs_prepr, 'annual_inc_factor', df_targets_prepr)\n",
    "df_temp"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This makes sense since there are very few people with high incomes but a lot with low incomes. Let's set a threshold of $140,000 (first 2 categories have a lot of obs.). Above threshold: high income, Below threshold: low income"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#low income dataframe\n",
    "df_inputs_prepr_temp = df_inputs_prepr.loc[df_inputs_prepr['annual_inc'] <= 140000, :]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "df_inputs_prepr_temp['annual_inc_factor'] = pd.cut(df_inputs_prepr_temp['annual_inc'], 50)\n",
    "df_temp = woe_ordered_continuous(df_inputs_prepr_temp, 'annual_inc_factor', df_targets_prepr.loc[df_inputs_prepr_temp.index])\n",
    "df_temp"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plot_by_woe(df_temp, rotation_of_x_axis_labels=90)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "    We will split with a width of $10000, but first and last category will have low no. of values.So, let's make sure first and last 2 categories have a width of $20000. The intervals from $20k to $100k will have a width of $10k."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_inputs_prepr['annual_inc : <20k'] = np.where((df_inputs_prepr['annual_inc'] <= 20000), 1, 0)\n",
    "df_inputs_prepr['annual_inc : 20k-30k'] = np.where((df_inputs_prepr['annual_inc'] > 20000) & (df_inputs_prepr['annual_inc'] <= 30000), 1, 0)\n",
    "df_inputs_prepr['annual_inc : 30k-40k'] = np.where((df_inputs_prepr['annual_inc'] > 30000) & (df_inputs_prepr['annual_inc'] <= 40000), 1, 0)\n",
    "df_inputs_prepr['annual_inc : 40k-50k'] = np.where((df_inputs_prepr['annual_inc'] > 40000) & (df_inputs_prepr['annual_inc'] <= 50000), 1, 0)\n",
    "df_inputs_prepr['annual_inc : 50k-60k'] = np.where((df_inputs_prepr['annual_inc'] > 50000) & (df_inputs_prepr['annual_inc'] <= 60000), 1, 0)\n",
    "df_inputs_prepr['annual_inc : 60k-70k'] = np.where((df_inputs_prepr['annual_inc'] > 60000) & (df_inputs_prepr['annual_inc'] <= 70000), 1, 0)\n",
    "df_inputs_prepr['annual_inc : 70k-80k'] = np.where((df_inputs_prepr['annual_inc'] > 70000) & (df_inputs_prepr['annual_inc'] <= 80000), 1, 0)\n",
    "df_inputs_prepr['annual_inc : 80k-90k'] = np.where((df_inputs_prepr['annual_inc'] > 80000) & (df_inputs_prepr['annual_inc'] <= 90000), 1, 0)\n",
    "df_inputs_prepr['annual_inc : 90k-100k'] = np.where((df_inputs_prepr['annual_inc'] > 90000) & (df_inputs_prepr['annual_inc'] <= 100000), 1, 0)\n",
    "df_inputs_prepr['annual_inc : 100k-120k'] = np.where((df_inputs_prepr['annual_inc'] > 100000) & (df_inputs_prepr['annual_inc'] <= 120000), 1, 0)\n",
    "df_inputs_prepr['annual_inc : 120k-140k'] = np.where((df_inputs_prepr['annual_inc'] > 120000) & (df_inputs_prepr['annual_inc'] <= 140000), 1, 0)\n",
    "df_inputs_prepr['annual_inc : >140k'] = np.where((df_inputs_prepr['annual_inc'] > 140000), 1, 0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## mths_since_last_delinq variable"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_inputs_prepr['mths_since_last_delinq'].isnull().sum()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There are a lot of missing values in this variable. So, we will create a dummy variable which is 1: when value is missing, 0: value isn't missing\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_inputs_prepr_temp = df_inputs_prepr[pd.notnull(df_inputs_prepr['mths_since_last_delinq'])]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "df_inputs_prepr_temp['mths_since_last_delinq_factor'] = pd.cut(df_inputs_prepr_temp['mths_since_last_delinq'], 50)\n",
    "df_temp = woe_ordered_continuous(df_inputs_prepr_temp, 'mths_since_last_delinq_factor', df_targets_prepr[df_inputs_prepr_temp.index])\n",
    "df_temp"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plot_by_woe(df_temp, rotation_of_x_axis_labels=90)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_inputs_prepr['mths_since_last_delinq : Missing'] = np.where(df_inputs_prepr['mths_since_last_delinq'].isnull(), 1, 0)\n",
    "df_inputs_prepr['mths_since_last_delinq : 0-3'] = np.where((df_inputs_prepr['mths_since_last_delinq'] >= 0) & (df_inputs_prepr['mths_since_last_delinq'] < 3), 1, 0)\n",
    "df_inputs_prepr['mths_since_last_delinq : 4-30'] = np.where((df_inputs_prepr['mths_since_last_delinq'] >= 4) & (df_inputs_prepr['mths_since_last_delinq'] < 30), 1, 0)\n",
    "df_inputs_prepr['mths_since_last_delinq : 31-56'] = np.where((df_inputs_prepr['mths_since_last_delinq'] >= 31) & (df_inputs_prepr['mths_since_last_delinq'] < 56), 1, 0)\n",
    "df_inputs_prepr['mths_since_last_delinq : >=57'] = np.where((df_inputs_prepr['mths_since_last_delinq'] >= 57), 1, 0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## DTI(debt-to-income) variable"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "df_inputs_prepr['dti_factor'] = pd.cut(df_inputs_prepr['dti'], 50)\n",
    "df_temp = woe_ordered_continuous(df_inputs_prepr, 'dti_factor', df_targets_prepr)\n",
    "df_temp"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plot_by_woe(df_temp, rotation_of_x_axis_labels=90)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_inputs_prepr['dti : <=1.6'] = np.where(df_inputs_prepr['dti'] <= 1.7, 1, 0)\n",
    "df_inputs_prepr['dti : 1.7-4.1'] = np.where((df_inputs_prepr['dti'] > 1.7) & (df_inputs_prepr['dti'] <= 4.1), 1, 0)\n",
    "df_inputs_prepr['dti : 4.1-8.9'] = np.where((df_inputs_prepr['dti'] > 4.1) & (df_inputs_prepr['dti'] <= 8.9), 1, 0)\n",
    "df_inputs_prepr['dti : 8.9-14.4'] = np.where((df_inputs_prepr['dti'] > 8.9) & (df_inputs_prepr['dti'] <= 14.4), 1, 0)\n",
    "df_inputs_prepr['dti : 14.4-16.8'] = np.where((df_inputs_prepr['dti'] > 14.4) & (df_inputs_prepr['dti'] <= 16.8), 1, 0)\n",
    "df_inputs_prepr['dti : 16.8-24'] = np.where((df_inputs_prepr['dti'] > 16.8) & (df_inputs_prepr['dti'] <= 24.0), 1, 0)\n",
    "df_inputs_prepr['dti : 24-35.9'] = np.where((df_inputs_prepr['dti'] > 24.0) & (df_inputs_prepr['dti'] <= 35.9), 1, 0)\n",
    "df_inputs_prepr['dti : >35.9'] = np.where(df_inputs_prepr['dti'] > 35.9, 1, 0) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_inputs_prepr.columns.values"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## mths_since_last_record variable"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_inputs_prepr['mths_since_last_record'].isnull().sum()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Taking only non-null values\n",
    "df_inputs_prepr_temp = df_inputs_prepr[pd.notnull(df_inputs_prepr['mths_since_last_record'])]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "df_inputs_prepr_temp['mths_since_last_record_factor'] = pd.cut(df_inputs_prepr_temp['mths_since_last_record'], 50)\n",
    "df_temp = woe_ordered_continuous(df_inputs_prepr_temp, 'mths_since_last_record_factor', df_targets_prepr[df_inputs_prepr_temp.index])\n",
    "df_temp"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plot_by_woe(df_temp, rotation_of_x_axis_labels=90)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_inputs_prepr['mths_since_last_record : Missing'] = np.where(df_inputs_prepr['mths_since_last_record'].isnull(), 1, 0)\n",
    "df_inputs_prepr['mths_since_last_record : 0-2'] = np.where(df_inputs_prepr['mths_since_last_record'].isin(range(0,3)), 1, 0)\n",
    "df_inputs_prepr['mths_since_last_record : 3-20'] = np.where(df_inputs_prepr['mths_since_last_record'].isin(range(3,21)), 1, 0)\n",
    "df_inputs_prepr['mths_since_last_record : 21-40'] = np.where(df_inputs_prepr['mths_since_last_record'].isin(range(21,41)), 1, 0)\n",
    "df_inputs_prepr['mths_since_last_record : 41-65'] = np.where(df_inputs_prepr['mths_since_last_record'].isin(range(41,66)), 1, 0)\n",
    "df_inputs_prepr['mths_since_last_record : 66-84'] = np.where(df_inputs_prepr['mths_since_last_record'].isin(range(66,85)), 1, 0)\n",
    "df_inputs_prepr['mths_since_last_record : 85-96'] = np.where(df_inputs_prepr['mths_since_last_record'].isin(range(85,97)), 1, 0)\n",
    "df_inputs_prepr['mths_since_last_record : >=97'] = np.where(df_inputs_prepr['mths_since_last_record'].isin(range(97,int(df_inputs_prepr['mths_since_last_record'].max()))), 1, 0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Preprocessing test dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#loan_data_inputs_train = df_inputs_prepr"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "        After executing the code from reassigning test_input and test_target to df_inputs_prep and df_targets_prep, now df_inputs_prep contains data for test dataset. Let's reassign this to loan_data_inputs_test"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#loan_data_inputs_test = df_inputs_prepr"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# #Exporting the final datasets to CSV files\n",
    "# loan_data_inputs_train.to_csv('loan_data_inputs_train.csv')\n",
    "# loan_data_targets_train.to_csv('loan_data_targets_train.csv')\n",
    "# loan_data_inputs_test.to_csv('loan_data_inputs_test.csv')\n",
    "# loan_data_targets_test.to_csv('loan_data_targets_test.csv')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Linear regression: output = linear combination of predictors\n",
    "Logistic regression: P(Y = 1) = $\\frac{\\exp (linear combi)}{1 + \\exp(linear combi)}$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$\\frac{P(Y=1)}{P(Y=0)}$ [This ratio is called odds] = $\\exp(linear combi)$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Taking ln on both sides: ln($\\frac{P(Y=1)}{P(Y=0)}$) = linear combi. So, linear and logistic regression are equivalent."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "ln($\\frac{P(Y=1 | X1= 1)}{P(Y=0 | X1 = 1)}$) - ln($\\frac{P(Y=1 | X1= 0)}{P(Y=0 | X1 = 0)}$ = Beta1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Which implies,\n",
    "ln($\\frac{odds(Y = 1 | X1 = 1)}{odds(Y = 1 | X1 = 0)}$) = Beta1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Also, $\\frac{odds(Y = 1 | X1 = 1)}{odds(Y = 1 | X1 = 0)}$ = $\\exp(Beta1)$"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}