01-02_interaction-effects_trade-effect-growth-paper.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This notebook uses the `InteractionEncoder` to replicate the regression output from an economics paper by **Frankel & Romer (1999)**: [Does Trade Cause Growth?](https://www.aeaweb.org/articles?id=10.1257/aer.89.3.379) (paper aer.89.3.379).  The data files are sourced from that webpage.  The `tablelist` folder contains the relevant data sources and files that specify how to produce the models from the paper.\n",
    "\n",
    "**Table 1 from the paper is replicated in this notebook.**  The model finds that the presence of a common border (variable `adj`) between two countries (`i` & `j`) changes the effects of other factors on the amount of trade that country `i` does with country `j`.\n",
    "\n",
    "The table has the interactions between `adj` (the common border variable) with many other geographical variables that influence trade, e.g. distance between countries and size of countries, so it is a nice example of how the Appelpy encoder can produce interaction effects relatively easily."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "import requests\n",
    "import io\n",
    "import zipfile\n",
    "\n",
    "# Appelpy imports\n",
    "from appelpy.utils import InteractionEncoder, get_dataframe_columns_diff\n",
    "from appelpy.eda import statistical_moments\n",
    "from appelpy.linear_model import OLS\n",
    "\n",
    "# Hide Numpy warnings from Statsmodels\n",
    "import warnings\n",
    "warnings.filterwarnings('ignore')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Load data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The data files are from an old version of Stata, which is not supported by the Pandas `read_stata` function.  They can be read by the `read_csv` method, via some arguments and pre-processing.\n",
    "\n",
    "There are multiple datasets to be combined into one dataframe.  They are in the `tablelist` ZIP file from the data sources provided on the American Economic Association webpage.\n",
    "\n",
    "Each dataset has 3782 observations and a common key, so the files can be simply concatenated."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "def load_source_zip():\n",
    "    req = requests.get('https://www.aeaweb.org/aer/data/89-3-379_Data.zip', stream=True)\n",
    "\n",
    "    with zipfile.ZipFile(io.BytesIO(req.content)) as z1:\n",
    "        main_zip = z1.open('tablelist.zip')\n",
    "    return main_zip"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "source_zip = load_source_zip()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The source ZIP file is `89-3-379_Data.zip`.\n",
    "\n",
    "Access files from the inner zip file (`tablelist.zip`)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "def extract_data(source_zip, file):\n",
    "    with zipfile.ZipFile(source_zip) as z2:\n",
    "        inner_zip = z2.open(file)\n",
    "    return pd.read_csv(inner_zip, delim_whitespace=True, header=None)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>codesj</th>\n",
       "      <th>trade</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>3777</th>\n",
       "      <td>6459</td>\n",
       "      <td>405.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3778</th>\n",
       "      <td>6460</td>\n",
       "      <td>1512.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3779</th>\n",
       "      <td>6461</td>\n",
       "      <td>685.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3780</th>\n",
       "      <td>6462</td>\n",
       "      <td>166.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3781</th>\n",
       "      <td>6463</td>\n",
       "      <td>522.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      codesj   trade\n",
       "3777    6459   405.0\n",
       "3778    6460  1512.0\n",
       "3779    6461   685.0\n",
       "3780    6462   166.0\n",
       "3781    6463   522.0"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "trade62 = extract_data(source_zip, 'trade62.dat')\n",
    "trade62.columns = ['codesj', 'trade']\n",
    "trade62.tail()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>codesj</th>\n",
       "      <th>dist</th>\n",
       "      <th>adj</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>3777</th>\n",
       "      <td>6459</td>\n",
       "      <td>1840.548</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3778</th>\n",
       "      <td>6460</td>\n",
       "      <td>3798.440</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3779</th>\n",
       "      <td>6461</td>\n",
       "      <td>2876.819</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3780</th>\n",
       "      <td>6462</td>\n",
       "      <td>8353.153</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3781</th>\n",
       "      <td>6463</td>\n",
       "      <td>7978.150</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      codesj      dist  adj\n",
       "3777    6459  1840.548    0\n",
       "3778    6460  3798.440    0\n",
       "3779    6461  2876.819    0\n",
       "3780    6462  8353.153    0\n",
       "3781    6463  7978.150    0"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dist62 = extract_data(source_zip, 'dist62.dat')\n",
    "dist62.columns = ['codesj', 'dist', 'adj']\n",
    "dist62.tail()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>codei</th>\n",
       "      <th>gdpi</th>\n",
       "      <th>locki</th>\n",
       "      <th>worki</th>\n",
       "      <th>areai</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>3777</th>\n",
       "      <td>62.0</td>\n",
       "      <td>1.326380e+09</td>\n",
       "      <td>0.0</td>\n",
       "      <td>612363.1</td>\n",
       "      <td>3689631.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3778</th>\n",
       "      <td>62.0</td>\n",
       "      <td>1.326380e+09</td>\n",
       "      <td>0.0</td>\n",
       "      <td>612363.1</td>\n",
       "      <td>3689631.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3779</th>\n",
       "      <td>62.0</td>\n",
       "      <td>1.326380e+09</td>\n",
       "      <td>0.0</td>\n",
       "      <td>612363.1</td>\n",
       "      <td>3689631.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3780</th>\n",
       "      <td>62.0</td>\n",
       "      <td>1.326380e+09</td>\n",
       "      <td>0.0</td>\n",
       "      <td>612363.1</td>\n",
       "      <td>3689631.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3781</th>\n",
       "      <td>62.0</td>\n",
       "      <td>1.326380e+09</td>\n",
       "      <td>0.0</td>\n",
       "      <td>612363.1</td>\n",
       "      <td>3689631.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      codei          gdpi  locki     worki      areai\n",
       "3777   62.0  1.326380e+09    0.0  612363.1  3689631.0\n",
       "3778   62.0  1.326380e+09    0.0  612363.1  3689631.0\n",
       "3779   62.0  1.326380e+09    0.0  612363.1  3689631.0\n",
       "3780   62.0  1.326380e+09    0.0  612363.1  3689631.0\n",
       "3781   62.0  1.326380e+09    0.0  612363.1  3689631.0"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "var62i = extract_data(source_zip, 'var62i.dat')\n",
    "var62i.columns = ['codei', 'gdpi', 'locki', 'worki', 'areai']\n",
    "var62i.tail()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>codej</th>\n",
       "      <th>gdpj</th>\n",
       "      <th>lockj</th>\n",
       "      <th>workj</th>\n",
       "      <th>areaj</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>3777</th>\n",
       "      <td>57</td>\n",
       "      <td>84347400.0</td>\n",
       "      <td>0</td>\n",
       "      <td>19944.99805</td>\n",
       "      <td>115830</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3778</th>\n",
       "      <td>58</td>\n",
       "      <td>21393500.0</td>\n",
       "      <td>0</td>\n",
       "      <td>1189.45447</td>\n",
       "      <td>220</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3779</th>\n",
       "      <td>59</td>\n",
       "      <td>127295000.0</td>\n",
       "      <td>0</td>\n",
       "      <td>26793.35547</td>\n",
       "      <td>198455</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3780</th>\n",
       "      <td>60</td>\n",
       "      <td>56247600.0</td>\n",
       "      <td>1</td>\n",
       "      <td>5195.12744</td>\n",
       "      <td>35920</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3781</th>\n",
       "      <td>61</td>\n",
       "      <td>155397000.0</td>\n",
       "      <td>0</td>\n",
       "      <td>19234.67383</td>\n",
       "      <td>120728</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      codej         gdpj  lockj        workj   areaj\n",
       "3777     57   84347400.0      0  19944.99805  115830\n",
       "3778     58   21393500.0      0   1189.45447     220\n",
       "3779     59  127295000.0      0  26793.35547  198455\n",
       "3780     60   56247600.0      1   5195.12744   35920\n",
       "3781     61  155397000.0      0  19234.67383  120728"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "var62j = extract_data(source_zip, 'var62j.dat')\n",
    "var62j.columns = ['codej', 'gdpj', 'lockj', 'workj', 'areaj']\n",
    "\n",
    "var62j['gdpj'] = var62j['gdpj'].str.replace('D', 'E').astype(float)\n",
    "\n",
    "var62j.tail()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>codesj</th>\n",
       "      <th>trade</th>\n",
       "      <th>dist</th>\n",
       "      <th>adj</th>\n",
       "      <th>codei</th>\n",
       "      <th>gdpi</th>\n",
       "      <th>locki</th>\n",
       "      <th>worki</th>\n",
       "      <th>areai</th>\n",
       "      <th>codej</th>\n",
       "      <th>gdpj</th>\n",
       "      <th>lockj</th>\n",
       "      <th>workj</th>\n",
       "      <th>areaj</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>102</td>\n",
       "      <td>1610.0</td>\n",
       "      <td>5652.806</td>\n",
       "      <td>0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>392297000.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>12595.02</td>\n",
       "      <td>3851809.0</td>\n",
       "      <td>2</td>\n",
       "      <td>6.734050e+08</td>\n",
       "      <td>0</td>\n",
       "      <td>24881.94727</td>\n",
       "      <td>211208</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>103</td>\n",
       "      <td>2761.0</td>\n",
       "      <td>5856.608</td>\n",
       "      <td>0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>392297000.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>12595.02</td>\n",
       "      <td>3851809.0</td>\n",
       "      <td>3</td>\n",
       "      <td>7.653620e+08</td>\n",
       "      <td>0</td>\n",
       "      <td>28084.61914</td>\n",
       "      <td>96010</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>104</td>\n",
       "      <td>1382.0</td>\n",
       "      <td>6735.185</td>\n",
       "      <td>0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>392297000.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>12595.02</td>\n",
       "      <td>3851809.0</td>\n",
       "      <td>4</td>\n",
       "      <td>6.175800e+08</td>\n",
       "      <td>0</td>\n",
       "      <td>22714.33008</td>\n",
       "      <td>116500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>105</td>\n",
       "      <td>8875.0</td>\n",
       "      <td>10327.390</td>\n",
       "      <td>0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>392297000.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>12595.02</td>\n",
       "      <td>3851809.0</td>\n",
       "      <td>5</td>\n",
       "      <td>1.421400e+09</td>\n",
       "      <td>0</td>\n",
       "      <td>75525.78906</td>\n",
       "      <td>143574</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>106</td>\n",
       "      <td>3831.0</td>\n",
       "      <td>5367.578</td>\n",
       "      <td>0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>392297000.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>12595.02</td>\n",
       "      <td>3851809.0</td>\n",
       "      <td>6</td>\n",
       "      <td>6.362160e+08</td>\n",
       "      <td>0</td>\n",
       "      <td>27684.45508</td>\n",
       "      <td>94247</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   codesj   trade       dist  adj  codei         gdpi  locki     worki  \\\n",
       "0     102  1610.0   5652.806    0    1.0  392297000.0    0.0  12595.02   \n",
       "1     103  2761.0   5856.608    0    1.0  392297000.0    0.0  12595.02   \n",
       "2     104  1382.0   6735.185    0    1.0  392297000.0    0.0  12595.02   \n",
       "3     105  8875.0  10327.390    0    1.0  392297000.0    0.0  12595.02   \n",
       "4     106  3831.0   5367.578    0    1.0  392297000.0    0.0  12595.02   \n",
       "\n",
       "       areai  codej          gdpj  lockj        workj   areaj  \n",
       "0  3851809.0      2  6.734050e+08      0  24881.94727  211208  \n",
       "1  3851809.0      3  7.653620e+08      0  28084.61914   96010  \n",
       "2  3851809.0      4  6.175800e+08      0  22714.33008  116500  \n",
       "3  3851809.0      5  1.421400e+09      0  75525.78906  143574  \n",
       "4  3851809.0      6  6.362160e+08      0  27684.45508   94247  "
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_pre = pd.concat([trade62, dist62.drop(columns=['codesj']), var62i, var62j], axis='columns')\n",
    "df_pre.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Initial feature engineering"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Create a copy of the dataframe.\n",
    "\n",
    "More feature engineering will be done before modelling."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = df_pre.copy()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**The dependent variable for the model is `ltrdsh`.**\n",
    "\n",
    "That is the logarithm of country `i`'s trade with country `j` as a proportion of country `i`'s GDP."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "df['trdsh'] = df['trade'] / df['gdpi']\n",
    "df['ltrdsh'] = np.where(df['trdsh'] == 0, np.NaN, np.log(df['trdsh']))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Log transformation of independent variables."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Log transformations of many variables\n",
    "for col in ['worki', 'workj', 'areai', 'areaj', 'dist']:\n",
    "    df['l' + col] = np.log(df[col])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Variable to represent how many of the countries i & j are landlocked (between 0 and 2)\n",
    "df['lockij'] = df['locki'] + df['lockj']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>mean</th>\n",
       "      <th>var</th>\n",
       "      <th>skew</th>\n",
       "      <th>kurtosis</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>codesj</th>\n",
       "      <td>3279.24</td>\n",
       "      <td>3.50875e+06</td>\n",
       "      <td>0.00453871</td>\n",
       "      <td>-1.2432</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>trade</th>\n",
       "      <td>813.376</td>\n",
       "      <td>1.94226e+07</td>\n",
       "      <td>17.6183</td>\n",
       "      <td>413.533</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dist</th>\n",
       "      <td>7922.44</td>\n",
       "      <td>2.27986e+07</td>\n",
       "      <td>0.353816</td>\n",
       "      <td>-0.567253</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>adj</th>\n",
       "      <td>0.0333157</td>\n",
       "      <td>0.0322143</td>\n",
       "      <td>5.20099</td>\n",
       "      <td>25.0503</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>codei</th>\n",
       "      <td>31.5</td>\n",
       "      <td>320.335</td>\n",
       "      <td>0</td>\n",
       "      <td>-1.20062</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>gdpi</th>\n",
       "      <td>2.55534e+08</td>\n",
       "      <td>3.08583e+17</td>\n",
       "      <td>5.08283</td>\n",
       "      <td>29.6711</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>locki</th>\n",
       "      <td>0.0806452</td>\n",
       "      <td>0.0741611</td>\n",
       "      <td>3.08021</td>\n",
       "      <td>7.48772</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>worki</th>\n",
       "      <td>28243.5</td>\n",
       "      <td>7.22914e+09</td>\n",
       "      <td>5.74072</td>\n",
       "      <td>34.5483</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>areai</th>\n",
       "      <td>516431</td>\n",
       "      <td>8.53234e+11</td>\n",
       "      <td>2.66406</td>\n",
       "      <td>5.99213</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>codej</th>\n",
       "      <td>31.5</td>\n",
       "      <td>320.335</td>\n",
       "      <td>0</td>\n",
       "      <td>-1.20062</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>gdpj</th>\n",
       "      <td>2.55534e+08</td>\n",
       "      <td>3.08583e+17</td>\n",
       "      <td>5.08283</td>\n",
       "      <td>29.6711</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>lockj</th>\n",
       "      <td>0.0806452</td>\n",
       "      <td>0.0741611</td>\n",
       "      <td>3.08021</td>\n",
       "      <td>7.48772</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>workj</th>\n",
       "      <td>28243.5</td>\n",
       "      <td>7.22914e+09</td>\n",
       "      <td>5.74072</td>\n",
       "      <td>34.5483</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>areaj</th>\n",
       "      <td>516431</td>\n",
       "      <td>8.53234e+11</td>\n",
       "      <td>2.66406</td>\n",
       "      <td>5.99213</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>trdsh</th>\n",
       "      <td>4.79286e-06</td>\n",
       "      <td>3.58577e-10</td>\n",
       "      <td>11.013</td>\n",
       "      <td>166.591</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ltrdsh</th>\n",
       "      <td>-14.035</td>\n",
       "      <td>4.20763</td>\n",
       "      <td>-0.0507283</td>\n",
       "      <td>-0.0847643</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>lworki</th>\n",
       "      <td>8.97814</td>\n",
       "      <td>2.04221</td>\n",
       "      <td>0.316759</td>\n",
       "      <td>1.14408</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>lworkj</th>\n",
       "      <td>8.97814</td>\n",
       "      <td>2.04221</td>\n",
       "      <td>0.316759</td>\n",
       "      <td>1.14408</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>lareai</th>\n",
       "      <td>11.8455</td>\n",
       "      <td>3.64826</td>\n",
       "      <td>-0.853428</td>\n",
       "      <td>1.68396</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>lareaj</th>\n",
       "      <td>11.8455</td>\n",
       "      <td>3.64826</td>\n",
       "      <td>-0.853428</td>\n",
       "      <td>1.68396</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ldist</th>\n",
       "      <td>8.70542</td>\n",
       "      <td>0.73488</td>\n",
       "      <td>-1.1105</td>\n",
       "      <td>0.83213</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>lockij</th>\n",
       "      <td>0.16129</td>\n",
       "      <td>0.145891</td>\n",
       "      <td>2.12291</td>\n",
       "      <td>3.37396</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "               mean          var        skew   kurtosis\n",
       "codesj      3279.24  3.50875e+06  0.00453871    -1.2432\n",
       "trade       813.376  1.94226e+07     17.6183    413.533\n",
       "dist        7922.44  2.27986e+07    0.353816  -0.567253\n",
       "adj       0.0333157    0.0322143     5.20099    25.0503\n",
       "codei          31.5      320.335           0   -1.20062\n",
       "gdpi    2.55534e+08  3.08583e+17     5.08283    29.6711\n",
       "locki     0.0806452    0.0741611     3.08021    7.48772\n",
       "worki       28243.5  7.22914e+09     5.74072    34.5483\n",
       "areai        516431  8.53234e+11     2.66406    5.99213\n",
       "codej          31.5      320.335           0   -1.20062\n",
       "gdpj    2.55534e+08  3.08583e+17     5.08283    29.6711\n",
       "lockj     0.0806452    0.0741611     3.08021    7.48772\n",
       "workj       28243.5  7.22914e+09     5.74072    34.5483\n",
       "areaj        516431  8.53234e+11     2.66406    5.99213\n",
       "trdsh   4.79286e-06  3.58577e-10      11.013    166.591\n",
       "ltrdsh      -14.035      4.20763  -0.0507283 -0.0847643\n",
       "lworki      8.97814      2.04221    0.316759    1.14408\n",
       "lworkj      8.97814      2.04221    0.316759    1.14408\n",
       "lareai      11.8455      3.64826   -0.853428    1.68396\n",
       "lareaj      11.8455      3.64826   -0.853428    1.68396\n",
       "ldist       8.70542      0.73488     -1.1105    0.83213\n",
       "lockij      0.16129     0.145891     2.12291    3.37396"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "statistical_moments(df)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Model with interaction terms"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Via `InteractionEncoder`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Make sure there are no NaN values for the regressors used in the model (as of Appelpy 0.2)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Set up the encoder object.\n",
    "\n",
    "Encode the interaction effects and store them in `df_model`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_model = df.dropna()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_model = (InteractionEncoder(df_model,\n",
    "                              {'adj': ['lockj', 'ldist', 'lockij', 'lworki',\n",
    "                                       'lworkj', 'lareai', 'lareaj']})\n",
    "            .transform())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Columns added: ['adj#lockj', 'adj#lworkj', 'adj#lareai', 'adj#lworki', 'adj#lareaj', 'adj#ldist', 'adj#lockij']\n"
     ]
    }
   ],
   "source": [
    "print(f\"Columns added: {get_dataframe_columns_diff(df_model, df)}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The encoder uses the nullable integer dtype (introduced in Pandas 0.24) for producing dummy columns.  Notice how `adj#lockj` is `Int64` (i.e. `pd.Int64Dtype()`.\n",
    "\n",
    "However nullable integer dtype is not supported in Statsmodels regression objects, so those will be converted to `int64`.  After all, dataframes used in Statsmodels regression must not have any null values."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "adj          int64\n",
       "lockj        int64\n",
       "adj#lockj    Int64\n",
       "dtype: object"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_model.dtypes.loc[['adj', 'lockj', 'adj#lockj']]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "for col in df_model.select_dtypes(pd.Int64Dtype()):\n",
    "    df_model[col] = df_model[col].astype('int64')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>codesj</th>\n",
       "      <th>trade</th>\n",
       "      <th>dist</th>\n",
       "      <th>adj</th>\n",
       "      <th>codei</th>\n",
       "      <th>gdpi</th>\n",
       "      <th>locki</th>\n",
       "      <th>worki</th>\n",
       "      <th>areai</th>\n",
       "      <th>codej</th>\n",
       "      <th>...</th>\n",
       "      <th>lareaj</th>\n",
       "      <th>ldist</th>\n",
       "      <th>lockij</th>\n",
       "      <th>adj#lockj</th>\n",
       "      <th>adj#ldist</th>\n",
       "      <th>adj#lockij</th>\n",
       "      <th>adj#lworki</th>\n",
       "      <th>adj#lworkj</th>\n",
       "      <th>adj#lareai</th>\n",
       "      <th>adj#lareaj</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>102</td>\n",
       "      <td>1610.0</td>\n",
       "      <td>5652.806</td>\n",
       "      <td>0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>392297000.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>12595.02</td>\n",
       "      <td>3851809.0</td>\n",
       "      <td>2</td>\n",
       "      <td>...</td>\n",
       "      <td>12.260599</td>\n",
       "      <td>8.639907</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>103</td>\n",
       "      <td>2761.0</td>\n",
       "      <td>5856.608</td>\n",
       "      <td>0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>392297000.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>12595.02</td>\n",
       "      <td>3851809.0</td>\n",
       "      <td>3</td>\n",
       "      <td>...</td>\n",
       "      <td>11.472208</td>\n",
       "      <td>8.675326</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>104</td>\n",
       "      <td>1382.0</td>\n",
       "      <td>6735.185</td>\n",
       "      <td>0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>392297000.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>12595.02</td>\n",
       "      <td>3851809.0</td>\n",
       "      <td>4</td>\n",
       "      <td>...</td>\n",
       "      <td>11.665647</td>\n",
       "      <td>8.815101</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>105</td>\n",
       "      <td>8875.0</td>\n",
       "      <td>10327.390</td>\n",
       "      <td>0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>392297000.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>12595.02</td>\n",
       "      <td>3851809.0</td>\n",
       "      <td>5</td>\n",
       "      <td>...</td>\n",
       "      <td>11.874606</td>\n",
       "      <td>9.242555</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>106</td>\n",
       "      <td>3831.0</td>\n",
       "      <td>5367.578</td>\n",
       "      <td>0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>392297000.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>12595.02</td>\n",
       "      <td>3851809.0</td>\n",
       "      <td>6</td>\n",
       "      <td>...</td>\n",
       "      <td>11.453674</td>\n",
       "      <td>8.588132</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>107</td>\n",
       "      <td>122257.0</td>\n",
       "      <td>1037.087</td>\n",
       "      <td>1</td>\n",
       "      <td>1.0</td>\n",
       "      <td>392297000.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>12595.02</td>\n",
       "      <td>3851809.0</td>\n",
       "      <td>7</td>\n",
       "      <td>...</td>\n",
       "      <td>15.079903</td>\n",
       "      <td>6.944171</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>6.944171</td>\n",
       "      <td>0.0</td>\n",
       "      <td>9.441057</td>\n",
       "      <td>11.673022</td>\n",
       "      <td>15.164053</td>\n",
       "      <td>15.079903</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>108</td>\n",
       "      <td>187.0</td>\n",
       "      <td>6573.839</td>\n",
       "      <td>0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>392297000.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>12595.02</td>\n",
       "      <td>3851809.0</td>\n",
       "      <td>8</td>\n",
       "      <td>...</td>\n",
       "      <td>10.385142</td>\n",
       "      <td>8.790853</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>109</td>\n",
       "      <td>834.0</td>\n",
       "      <td>5679.494</td>\n",
       "      <td>0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>392297000.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>12595.02</td>\n",
       "      <td>3851809.0</td>\n",
       "      <td>9</td>\n",
       "      <td>...</td>\n",
       "      <td>9.374243</td>\n",
       "      <td>8.644617</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>110</td>\n",
       "      <td>223.0</td>\n",
       "      <td>5913.385</td>\n",
       "      <td>0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>392297000.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>12595.02</td>\n",
       "      <td>3851809.0</td>\n",
       "      <td>10</td>\n",
       "      <td>...</td>\n",
       "      <td>9.719024</td>\n",
       "      <td>8.684974</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>111</td>\n",
       "      <td>214.0</td>\n",
       "      <td>6278.729</td>\n",
       "      <td>0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>392297000.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>12595.02</td>\n",
       "      <td>3851809.0</td>\n",
       "      <td>11</td>\n",
       "      <td>...</td>\n",
       "      <td>11.776205</td>\n",
       "      <td>8.744923</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>10 rows × 29 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   codesj     trade       dist  adj  codei         gdpi  locki     worki  \\\n",
       "0     102    1610.0   5652.806    0    1.0  392297000.0    0.0  12595.02   \n",
       "1     103    2761.0   5856.608    0    1.0  392297000.0    0.0  12595.02   \n",
       "2     104    1382.0   6735.185    0    1.0  392297000.0    0.0  12595.02   \n",
       "3     105    8875.0  10327.390    0    1.0  392297000.0    0.0  12595.02   \n",
       "4     106    3831.0   5367.578    0    1.0  392297000.0    0.0  12595.02   \n",
       "5     107  122257.0   1037.087    1    1.0  392297000.0    0.0  12595.02   \n",
       "6     108     187.0   6573.839    0    1.0  392297000.0    0.0  12595.02   \n",
       "7     109     834.0   5679.494    0    1.0  392297000.0    0.0  12595.02   \n",
       "8     110     223.0   5913.385    0    1.0  392297000.0    0.0  12595.02   \n",
       "9     111     214.0   6278.729    0    1.0  392297000.0    0.0  12595.02   \n",
       "\n",
       "       areai  codej  ...     lareaj     ldist  lockij  adj#lockj  adj#ldist  \\\n",
       "0  3851809.0      2  ...  12.260599  8.639907     0.0          0   0.000000   \n",
       "1  3851809.0      3  ...  11.472208  8.675326     0.0          0   0.000000   \n",
       "2  3851809.0      4  ...  11.665647  8.815101     0.0          0   0.000000   \n",
       "3  3851809.0      5  ...  11.874606  9.242555     0.0          0   0.000000   \n",
       "4  3851809.0      6  ...  11.453674  8.588132     0.0          0   0.000000   \n",
       "5  3851809.0      7  ...  15.079903  6.944171     0.0          0   6.944171   \n",
       "6  3851809.0      8  ...  10.385142  8.790853     1.0          0   0.000000   \n",
       "7  3851809.0      9  ...   9.374243  8.644617     0.0          0   0.000000   \n",
       "8  3851809.0     10  ...   9.719024  8.684974     0.0          0   0.000000   \n",
       "9  3851809.0     11  ...  11.776205  8.744923     0.0          0   0.000000   \n",
       "\n",
       "   adj#lockij  adj#lworki  adj#lworkj  adj#lareai  adj#lareaj  \n",
       "0         0.0    0.000000    0.000000    0.000000    0.000000  \n",
       "1         0.0    0.000000    0.000000    0.000000    0.000000  \n",
       "2         0.0    0.000000    0.000000    0.000000    0.000000  \n",
       "3         0.0    0.000000    0.000000    0.000000    0.000000  \n",
       "4         0.0    0.000000    0.000000    0.000000    0.000000  \n",
       "5         0.0    9.441057   11.673022   15.164053   15.079903  \n",
       "6         0.0    0.000000    0.000000    0.000000    0.000000  \n",
       "7         0.0    0.000000    0.000000    0.000000    0.000000  \n",
       "8         0.0    0.000000    0.000000    0.000000    0.000000  \n",
       "9         0.0    0.000000    0.000000    0.000000    0.000000  \n",
       "\n",
       "[10 rows x 29 columns]"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_model.head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Gather the independent variables in a list `X_cols` and fit a regression model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_cols = ['ldist', 'lworki', 'lareai', 'lworkj', 'lareaj', 'lockij', \n",
    "          'adj', 'adj#ldist', 'adj#lworki', 'adj#lareai', 'adj#lworkj', 'adj#lareaj', 'adj#lockij']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "model = OLS(df_model, ['ltrdsh'], X_cols).fit()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"simpletable\">\n",
       "<caption>OLS Regression Results</caption>\n",
       "<tr>\n",
       "  <th>Dep. Variable:</th>         <td>ltrdsh</td>      <th>  R-squared:         </th> <td>   0.361</td> \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Model:</th>                   <td>OLS</td>       <th>  Adj. R-squared:    </th> <td>   0.358</td> \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Method:</th>             <td>Least Squares</td>  <th>  F-statistic:       </th> <td>   139.1</td> \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Date:</th>             <td>Sun, 17 Nov 2019</td> <th>  Prob (F-statistic):</th> <td>2.05e-299</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Time:</th>                 <td>22:50:43</td>     <th>  Log-Likelihood:    </th> <td> -6161.7</td> \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>No. Observations:</th>      <td>  3220</td>      <th>  AIC:               </th> <td>1.235e+04</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Df Residuals:</th>          <td>  3206</td>      <th>  BIC:               </th> <td>1.244e+04</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Df Model:</th>              <td>    13</td>      <th>                     </th>     <td> </td>    \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Covariance Type:</th>      <td>nonrobust</td>    <th>                     </th>     <td> </td>    \n",
       "</tr>\n",
       "</table>\n",
       "<table class=\"simpletable\">\n",
       "<tr>\n",
       "       <td></td>         <th>coef</th>     <th>std err</th>      <th>t</th>      <th>P>|t|</th>  <th>[0.025</th>    <th>0.975]</th>  \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>const</th>      <td>   -6.3777</td> <td>    0.415</td> <td>  -15.361</td> <td> 0.000</td> <td>   -7.192</td> <td>   -5.564</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>ldist</th>      <td>   -0.8521</td> <td>    0.038</td> <td>  -22.653</td> <td> 0.000</td> <td>   -0.926</td> <td>   -0.778</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>lworki</th>     <td>   -0.2393</td> <td>    0.026</td> <td>   -9.326</td> <td> 0.000</td> <td>   -0.290</td> <td>   -0.189</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>lareai</th>     <td>   -0.1206</td> <td>    0.018</td> <td>   -6.588</td> <td> 0.000</td> <td>   -0.156</td> <td>   -0.085</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>lworkj</th>     <td>    0.6093</td> <td>    0.026</td> <td>   23.745</td> <td> 0.000</td> <td>    0.559</td> <td>    0.660</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>lareaj</th>     <td>   -0.1851</td> <td>    0.018</td> <td>  -10.114</td> <td> 0.000</td> <td>   -0.221</td> <td>   -0.149</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>lockij</th>     <td>   -0.3609</td> <td>    0.084</td> <td>   -4.314</td> <td> 0.000</td> <td>   -0.525</td> <td>   -0.197</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>adj</th>        <td>    5.0952</td> <td>    1.780</td> <td>    2.863</td> <td> 0.004</td> <td>    1.606</td> <td>    8.585</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>adj#ldist</th>  <td>    0.1507</td> <td>    0.303</td> <td>    0.498</td> <td> 0.619</td> <td>   -0.443</td> <td>    0.744</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>adj#lworki</th> <td>   -0.2924</td> <td>    0.181</td> <td>   -1.619</td> <td> 0.106</td> <td>   -0.646</td> <td>    0.062</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>adj#lareai</th> <td>   -0.0572</td> <td>    0.151</td> <td>   -0.378</td> <td> 0.705</td> <td>   -0.354</td> <td>    0.240</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>adj#lworkj</th> <td>   -0.1365</td> <td>    0.181</td> <td>   -0.756</td> <td> 0.450</td> <td>   -0.491</td> <td>    0.218</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>adj#lareaj</th> <td>   -0.0691</td> <td>    0.151</td> <td>   -0.456</td> <td> 0.648</td> <td>   -0.366</td> <td>    0.228</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>adj#lockij</th> <td>    0.3262</td> <td>    0.331</td> <td>    0.986</td> <td> 0.324</td> <td>   -0.322</td> <td>    0.975</td>\n",
       "</tr>\n",
       "</table>\n",
       "<table class=\"simpletable\">\n",
       "<tr>\n",
       "  <th>Omnibus:</th>       <td>48.172</td> <th>  Durbin-Watson:     </th> <td>   1.153</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Prob(Omnibus):</th> <td> 0.000</td> <th>  Jarque-Bera (JB):  </th> <td>  50.111</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Skew:</th>          <td>-0.306</td> <th>  Prob(JB):          </th> <td>1.31e-11</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Kurtosis:</th>      <td> 3.000</td> <th>  Cond. No.          </th> <td>1.41e+03</td>\n",
       "</tr>\n",
       "</table><br/><br/>Warnings:<br/>[1] Standard Errors assume that the covariance matrix of the errors is correctly specified.<br/>[2] The condition number is large, 1.41e+03. This might indicate that there are<br/>strong multicollinearity or other numerical problems."
      ],
      "text/plain": [
       "<class 'statsmodels.iolib.summary.Summary'>\n",
       "\"\"\"\n",
       "                            OLS Regression Results                            \n",
       "==============================================================================\n",
       "Dep. Variable:                 ltrdsh   R-squared:                       0.361\n",
       "Model:                            OLS   Adj. R-squared:                  0.358\n",
       "Method:                 Least Squares   F-statistic:                     139.1\n",
       "Date:                Sun, 17 Nov 2019   Prob (F-statistic):          2.05e-299\n",
       "Time:                        22:50:43   Log-Likelihood:                -6161.7\n",
       "No. Observations:                3220   AIC:                         1.235e+04\n",
       "Df Residuals:                    3206   BIC:                         1.244e+04\n",
       "Df Model:                          13                                         \n",
       "Covariance Type:            nonrobust                                         \n",
       "==============================================================================\n",
       "                 coef    std err          t      P>|t|      [0.025      0.975]\n",
       "------------------------------------------------------------------------------\n",
       "const         -6.3777      0.415    -15.361      0.000      -7.192      -5.564\n",
       "ldist         -0.8521      0.038    -22.653      0.000      -0.926      -0.778\n",
       "lworki        -0.2393      0.026     -9.326      0.000      -0.290      -0.189\n",
       "lareai        -0.1206      0.018     -6.588      0.000      -0.156      -0.085\n",
       "lworkj         0.6093      0.026     23.745      0.000       0.559       0.660\n",
       "lareaj        -0.1851      0.018    -10.114      0.000      -0.221      -0.149\n",
       "lockij        -0.3609      0.084     -4.314      0.000      -0.525      -0.197\n",
       "adj            5.0952      1.780      2.863      0.004       1.606       8.585\n",
       "adj#ldist      0.1507      0.303      0.498      0.619      -0.443       0.744\n",
       "adj#lworki    -0.2924      0.181     -1.619      0.106      -0.646       0.062\n",
       "adj#lareai    -0.0572      0.151     -0.378      0.705      -0.354       0.240\n",
       "adj#lworkj    -0.1365      0.181     -0.756      0.450      -0.491       0.218\n",
       "adj#lareaj    -0.0691      0.151     -0.456      0.648      -0.366       0.228\n",
       "adj#lockij     0.3262      0.331      0.986      0.324      -0.322       0.975\n",
       "==============================================================================\n",
       "Omnibus:                       48.172   Durbin-Watson:                   1.153\n",
       "Prob(Omnibus):                  0.000   Jarque-Bera (JB):               50.111\n",
       "Skew:                          -0.306   Prob(JB):                     1.31e-11\n",
       "Kurtosis:                       3.000   Cond. No.                     1.41e+03\n",
       "==============================================================================\n",
       "\n",
       "Warnings:\n",
       "[1] Standard Errors assume that the covariance matrix of the errors is correctly specified.\n",
       "[2] The condition number is large, 1.41e+03. This might indicate that there are\n",
       "strong multicollinearity or other numerical problems.\n",
       "\"\"\""
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.results_output"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**The output above replicates Table 1 in the paper.**\n",
    "\n",
    "This is one of the key sentences in the paper about the table:\n",
    "\n",
    "\"The estimates also imply that the presence of a common border alters the effects of the other variables substantially.\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Via manually coded interactions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This section does manual coding of the interaction effects.  The variable names, e.g. `adjlij`, are all from the data files used by the paper's authors.\n",
    "\n",
    "Even with simpler approaches with less code, e.g. loops, there is scope for error.  The `InteractionEncoder` is more practical for a pipeline of interaction effect generation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_model['adjdist'] = df_model['adj'] * df_model['ldist']\n",
    "df_model['adjlij'] = df_model['adj'] * df_model['lockij']\n",
    "df_model['adjworki'] = df_model['adj'] * df_model['lworki']\n",
    "df_model['adjworkj'] = df_model['adj'] * df_model['lworkj']\n",
    "df_model['adjareai'] = df_model['adj'] * df_model['lareai']\n",
    "df_model['adjareaj'] = df_model['adj'] * df_model['lareaj']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_cols_manual = ['ldist', 'lworki', 'lareai', 'lworkj', 'lareaj', 'lockij', \n",
    "                 'adj', 'adjdist', 'adjworki', 'adjareai', 'adjworkj', 'adjareaj', 'adjlij']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [],
   "source": [
    "model_manual = OLS(df_model, ['ltrdsh'], X_cols_manual).fit()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"simpletable\">\n",
       "<caption>OLS Regression Results</caption>\n",
       "<tr>\n",
       "  <th>Dep. Variable:</th>         <td>ltrdsh</td>      <th>  R-squared:         </th> <td>   0.361</td> \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Model:</th>                   <td>OLS</td>       <th>  Adj. R-squared:    </th> <td>   0.358</td> \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Method:</th>             <td>Least Squares</td>  <th>  F-statistic:       </th> <td>   139.1</td> \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Date:</th>             <td>Sun, 17 Nov 2019</td> <th>  Prob (F-statistic):</th> <td>2.05e-299</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Time:</th>                 <td>22:50:46</td>     <th>  Log-Likelihood:    </th> <td> -6161.7</td> \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>No. Observations:</th>      <td>  3220</td>      <th>  AIC:               </th> <td>1.235e+04</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Df Residuals:</th>          <td>  3206</td>      <th>  BIC:               </th> <td>1.244e+04</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Df Model:</th>              <td>    13</td>      <th>                     </th>     <td> </td>    \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Covariance Type:</th>      <td>nonrobust</td>    <th>                     </th>     <td> </td>    \n",
       "</tr>\n",
       "</table>\n",
       "<table class=\"simpletable\">\n",
       "<tr>\n",
       "      <td></td>        <th>coef</th>     <th>std err</th>      <th>t</th>      <th>P>|t|</th>  <th>[0.025</th>    <th>0.975]</th>  \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>const</th>    <td>   -6.3777</td> <td>    0.415</td> <td>  -15.361</td> <td> 0.000</td> <td>   -7.192</td> <td>   -5.564</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>ldist</th>    <td>   -0.8521</td> <td>    0.038</td> <td>  -22.653</td> <td> 0.000</td> <td>   -0.926</td> <td>   -0.778</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>lworki</th>   <td>   -0.2393</td> <td>    0.026</td> <td>   -9.326</td> <td> 0.000</td> <td>   -0.290</td> <td>   -0.189</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>lareai</th>   <td>   -0.1206</td> <td>    0.018</td> <td>   -6.588</td> <td> 0.000</td> <td>   -0.156</td> <td>   -0.085</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>lworkj</th>   <td>    0.6093</td> <td>    0.026</td> <td>   23.745</td> <td> 0.000</td> <td>    0.559</td> <td>    0.660</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>lareaj</th>   <td>   -0.1851</td> <td>    0.018</td> <td>  -10.114</td> <td> 0.000</td> <td>   -0.221</td> <td>   -0.149</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>lockij</th>   <td>   -0.3609</td> <td>    0.084</td> <td>   -4.314</td> <td> 0.000</td> <td>   -0.525</td> <td>   -0.197</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>adj</th>      <td>    5.0952</td> <td>    1.780</td> <td>    2.863</td> <td> 0.004</td> <td>    1.606</td> <td>    8.585</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>adjdist</th>  <td>    0.1507</td> <td>    0.303</td> <td>    0.498</td> <td> 0.619</td> <td>   -0.443</td> <td>    0.744</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>adjworki</th> <td>   -0.2924</td> <td>    0.181</td> <td>   -1.619</td> <td> 0.106</td> <td>   -0.646</td> <td>    0.062</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>adjareai</th> <td>   -0.0572</td> <td>    0.151</td> <td>   -0.378</td> <td> 0.705</td> <td>   -0.354</td> <td>    0.240</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>adjworkj</th> <td>   -0.1365</td> <td>    0.181</td> <td>   -0.756</td> <td> 0.450</td> <td>   -0.491</td> <td>    0.218</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>adjareaj</th> <td>   -0.0691</td> <td>    0.151</td> <td>   -0.456</td> <td> 0.648</td> <td>   -0.366</td> <td>    0.228</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>adjlij</th>   <td>    0.3262</td> <td>    0.331</td> <td>    0.986</td> <td> 0.324</td> <td>   -0.322</td> <td>    0.975</td>\n",
       "</tr>\n",
       "</table>\n",
       "<table class=\"simpletable\">\n",
       "<tr>\n",
       "  <th>Omnibus:</th>       <td>48.172</td> <th>  Durbin-Watson:     </th> <td>   1.153</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Prob(Omnibus):</th> <td> 0.000</td> <th>  Jarque-Bera (JB):  </th> <td>  50.111</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Skew:</th>          <td>-0.306</td> <th>  Prob(JB):          </th> <td>1.31e-11</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Kurtosis:</th>      <td> 3.000</td> <th>  Cond. No.          </th> <td>1.41e+03</td>\n",
       "</tr>\n",
       "</table><br/><br/>Warnings:<br/>[1] Standard Errors assume that the covariance matrix of the errors is correctly specified.<br/>[2] The condition number is large, 1.41e+03. This might indicate that there are<br/>strong multicollinearity or other numerical problems."
      ],
      "text/plain": [
       "<class 'statsmodels.iolib.summary.Summary'>\n",
       "\"\"\"\n",
       "                            OLS Regression Results                            \n",
       "==============================================================================\n",
       "Dep. Variable:                 ltrdsh   R-squared:                       0.361\n",
       "Model:                            OLS   Adj. R-squared:                  0.358\n",
       "Method:                 Least Squares   F-statistic:                     139.1\n",
       "Date:                Sun, 17 Nov 2019   Prob (F-statistic):          2.05e-299\n",
       "Time:                        22:50:46   Log-Likelihood:                -6161.7\n",
       "No. Observations:                3220   AIC:                         1.235e+04\n",
       "Df Residuals:                    3206   BIC:                         1.244e+04\n",
       "Df Model:                          13                                         \n",
       "Covariance Type:            nonrobust                                         \n",
       "==============================================================================\n",
       "                 coef    std err          t      P>|t|      [0.025      0.975]\n",
       "------------------------------------------------------------------------------\n",
       "const         -6.3777      0.415    -15.361      0.000      -7.192      -5.564\n",
       "ldist         -0.8521      0.038    -22.653      0.000      -0.926      -0.778\n",
       "lworki        -0.2393      0.026     -9.326      0.000      -0.290      -0.189\n",
       "lareai        -0.1206      0.018     -6.588      0.000      -0.156      -0.085\n",
       "lworkj         0.6093      0.026     23.745      0.000       0.559       0.660\n",
       "lareaj        -0.1851      0.018    -10.114      0.000      -0.221      -0.149\n",
       "lockij        -0.3609      0.084     -4.314      0.000      -0.525      -0.197\n",
       "adj            5.0952      1.780      2.863      0.004       1.606       8.585\n",
       "adjdist        0.1507      0.303      0.498      0.619      -0.443       0.744\n",
       "adjworki      -0.2924      0.181     -1.619      0.106      -0.646       0.062\n",
       "adjareai      -0.0572      0.151     -0.378      0.705      -0.354       0.240\n",
       "adjworkj      -0.1365      0.181     -0.756      0.450      -0.491       0.218\n",
       "adjareaj      -0.0691      0.151     -0.456      0.648      -0.366       0.228\n",
       "adjlij         0.3262      0.331      0.986      0.324      -0.322       0.975\n",
       "==============================================================================\n",
       "Omnibus:                       48.172   Durbin-Watson:                   1.153\n",
       "Prob(Omnibus):                  0.000   Jarque-Bera (JB):               50.111\n",
       "Skew:                          -0.306   Prob(JB):                     1.31e-11\n",
       "Kurtosis:                       3.000   Cond. No.                     1.41e+03\n",
       "==============================================================================\n",
       "\n",
       "Warnings:\n",
       "[1] Standard Errors assume that the covariance matrix of the errors is correctly specified.\n",
       "[2] The condition number is large, 1.41e+03. This might indicate that there are\n",
       "strong multicollinearity or other numerical problems.\n",
       "\"\"\""
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model_manual.results_output"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.4"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": true
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}