run_gerrymandering_metrics.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Compute gerrymandering metrics\n",
    "\n",
    "This notebook demonstrates how to load in election results, creating 3 different DataFrames that you might be interested in:\n",
    "1. Election results\n",
    "2. Metrics\n",
    "3. Percentiles for metrics\n",
    "\n",
    "First, we will load in our data, both for Congressional elections and state legislative (lower house) elections.\n",
    "\n",
    "Then we will compute the tests and generate percentile rankings, using a few parameters that we set below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "State Legislative\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 48/48 [00:01<00:00, 24.23it/s]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Congressional\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      " 89%|████████▉ | 32/36 [00:02<00:00, 14.28it/s]"
     ]
    }
   ],
   "source": [
    "import gerrymetrics as g\n",
    "import IPython.display as ipd\n",
    "\n",
    "from collections import defaultdict\n",
    "\n",
    "# impute uncontested races at a voteshare of 0 or 1; in other words, don't impute them\n",
    "impute_val = 1\n",
    "\n",
    "# only consider races after 1972\n",
    "min_year = 1972\n",
    "\n",
    "# when identifying the worst gerrymanders:\n",
    "# only examine races where D voteshare is between .45 and .55\n",
    "competitiveness_threshold = .55 \n",
    "\n",
    "# only examine races in states with at least 7 districts\n",
    "min_districts = 7\n",
    "\n",
    "\n",
    "\n",
    "chambers = defaultdict(lambda: defaultdict(list))\n",
    "chambers['State Legislative']['filepath'] = 'https://raw.githubusercontent.com/PrincetonUniversity/historic_state_legislative_election_results/2bf28f2ac1a74636b09dfb700eef08a4324d2650/state_legislative_election_results_post1971.csv'\n",
    "chambers['Congressional']['filepath'] = 'election_data/congressional_election_results_post1948.csv'\n",
    "\n",
    "metric_dict = {'t_test_diff':            g.t_test_diff,\n",
    "#                'mean_median_diff':       g.mean_median,\n",
    "               'declination':            g.declination,\n",
    "               'declination_buffered':   g.bdec,\n",
    "               'efficiency_gap':         g.EG,\n",
    "               'loss_gap':               g.EG_loss_only,\n",
    "               'difference_gap':         g.EG_difference,\n",
    "               'surplus_gap':            g.EG_surplus_only,\n",
    "               'vote_centric_gap':       g.EG_vote_centric,\n",
    "               'vote_centric_gap_two':   g.EG_vote_centric_two,\n",
    "               'partisan_bias':          g.partisan_bias,\n",
    "               'equal_vote_weight_bias': g.equal_vote_weight}\n",
    "\n",
    "for chamber in chambers:\n",
    "    print(chamber)\n",
    "    chambers[chamber]['elections_df'] = g.parse_results(chambers[chamber]['filepath'])\n",
    "    chambers[chamber]['tests_df'] = g.tests_df(g.run_all_tests(\n",
    "        chambers[chamber]['elections_df'],\n",
    "        impute_val=impute_val,\n",
    "        metrics=metric_dict))\n",
    "    chambers[chamber]['percentile_df'] = g.generate_percentiles(chambers[chamber]['tests_df'],\n",
    "        metric_dict.keys(),\n",
    "        competitiveness_threshold=competitiveness_threshold,\n",
    "        min_districts=min_districts,\n",
    "        min_year=min_year)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. Election results\n",
    "The first dataframe is just election results. Let's look at a few 2016 Congressional elections:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "chambers['Congressional']['elections_df'].loc[2016].head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. Metrics\n",
    "Then we have the output from the metrics that we specified in the first cell. Let's look at those same elections again:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "chambers['Congressional']['tests_df'].loc[2016].head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. Percentiles for metrics\n",
    "Then we have the percentile rankings for all of those metrics, using the parameters specified in the first cell."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "chambers['Congressional']['percentile_df'].loc[2016].head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "All of the above DataFrames are MultiIndex, so you can specify any state and year like so:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "chambers['Congressional']['percentile_df'].loc[2012, 'VA']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Which elections in the ongoing cycle seem to be most gerrymandered according to these metrics?\n",
    "\n",
    "Finally, we might be interested in which elections seem particularly gerrymandered in the ongoing 2012-2021 cycle."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "min_percentile = 95\n",
    "min_n_tests = 3\n",
    "cycle_start_year = 2012\n",
    "\n",
    "print(\n",
    "'''\n",
    "Shown below are elections since {start} that rank, for\n",
    "at least {min_tests} of {total_tests} metrics, in at least the {pctile}th\n",
    "percentile of all elections since {min_year_overall}.\n",
    "\n",
    "Only showing elections for states that have at least {n_districts} districts,\n",
    "and for which the statewide total voteshare was competitive,\n",
    "i.e., between {comp_lo:.2} and {comp_hi:.2}.\n",
    "'''.format(start=cycle_start_year,\n",
    "    min_tests=min_n_tests,\n",
    "    total_tests=len(metric_dict),\n",
    "    pctile=min_percentile,\n",
    "    min_year_overall=min_year,\n",
    "    n_districts=min_districts,\n",
    "    comp_lo=1-competitiveness_threshold,\n",
    "    comp_hi=competitiveness_threshold))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for chamber in chambers:\n",
    "    print('\\n' + chamber + ' elections:')\n",
    "    df = chambers[chamber]['percentile_df']\n",
    "    cut = df[(df.loc[:, metric_dict.keys()] > min_percentile).sum(axis=1) >= min_n_tests]\n",
    "    ipd.display(cut.loc[cycle_start_year:])"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}