notebooks/05/population_permutation.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## A problem of populations\n",
    "\n",
    "As in the [Brexit analysis\n",
    "exercise]({{ site.baseurl }}/exercises/brexiteering.zip),\n",
    "we analyze the [Brexit survey]({{ site.baseurl }}/data/brexit_survey).\n",
    "\n",
    "As you will see in the link above, the data are from a survey of the UK\n",
    "population.  Each row in the survey corresponds to one person answering.  One\n",
    "of the questions, named `cut15` is how the person voted in the Brexit\n",
    "referendum.  Another, `numage` is the age of the person in years.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Array library.\n",
    "import numpy as np\n",
    "\n",
    "# Data frame library.\n",
    "import pandas as pd\n",
    "\n",
    "# Plotting\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "\n",
    "# Fancy plots\n",
    "plt.style.use('fivethirtyeight')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you are running on your laptop, first [download the data file]({{ site.baseurl }}/data/audit_of_political_engagement_14_2017.tab) to the same directory as this notebook.\n",
    "\n",
    "We load the data:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Load the data frame, and put it in the variable \"audit_data\".\n",
    "# The values are separated by tab characters, written as \"\\t\" in Python\n",
    "# strings.\n",
    "audit_data = pd.read_csv('audit_of_political_engagement_14_2017.tab', sep='\\t')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now get the ages for the Leavers and the Remainers.\n",
    "\n",
    "A small number of ages are recorded as 0, meaning we do not have the correct age for that person / row.  First we drop rows with ages recorded as 0, then select the remaining rows corresponding to people who voted to remain (`cut15` value of 1) and leave (`cut15` value of 2):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Drop rows where age is 0\n",
    "good_data = audit_data[audit_data['numage'] != 0]\n",
    "# Get data frames for leavers and remainers\n",
    "remain_ages = good_data[good_data['cut15'] == 1]['numage']\n",
    "leave_ages = good_data[good_data['cut15'] == 2]['numage']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Show the age distributions for the two groups:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "774"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZwAAAD1CAYAAABkzUMfAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAPTElEQVR4nO3dccxddX3H8fdnVNSiawG3pmtZ6MYzDTEDkTAMzjjqJjojZHEGY7amIfMfpqAmils24x9LNDEqSxaTxY51i0MR2SDEqKxiNpasThAVqOx2qNCmULbZusxlE/fdH/dUn1V4Cvfc+7vn3r5fyZN7z++ce863J+feT3+/c+65qSokSZq1n5h3AZKkk4OBI0lqwsCRJDVh4EiSmjBwJElNrJvHRo8ePeqlcZK0xDZs2JDj2+zhSJKaMHAkSU0sfeCMRqN5lzAR625rUeuGxa3dutsaQt1LHziSpGEwcCRJTRg4kqQmDBxJUhMGjiSpCQNHktSEgSNJamIut7aZlo03HHwaS62Hu57OcpM7snPLTNcvScvAHo4kqQkDR5LUhIEjSWrCwJEkNWHgSJKaMHAkSU0YOJKkJgwcSVITBo4kqQkDR5LUhIEjSWrCwJEkNWHgSJKaMHAkSU0YOJKkJgwcSVITC/0DbPr/nt4P0s3aeo6szLsGSUNkD0eS1MQJAyfJnyU5nOS+VW1nJLkjyah7PL1rT5I/TrI/ydeSXDDL4iVJi+Pp9HD+HLjsuLbrgD1VtQLs6aYBXgOsdH9vAT46nTIlSYvuhIFTVX8H/PtxzZcDu7vnu4ErVrX/RY39I7AxyeZpFStJWlyTXjSwqaoOdc8fBTZ1z7cAj6xa7kDXdoinMBqNJiwBYH2P105Pv3/DNNe73Ptj1ha1bljc2q27rVnXvbKy9hVDva9Sq6pKUpO+/kQFrumuIVyV1fPf8BRGo9EzX+8S749Zm2h/D8Si1m7dbQ2h7kmvUnvs2FBZ93i4az8InLVqua1dmyTpJDdp4NwG7Oie7wBuXdX+293VahcDR1cNvUmSTmInHFJLciPwSuAFSQ4A7wXeD9yU5Crg28Abu8U/A7wW2A98D9g5g5olSQvohIFTVW96ilnbn2TZAq7uW5Qkafl4pwFJUhMGjiSpCW/eKZ1EhnGDVziyc8u8S9Ac2MORJDVh4EiSmnBITZqx2Q9jrR/MXSaktdjDkSQ1YeBIkpowcCRJTRg4kqQmDBxJUhMGjiSpCQNHktSE38OZgtl8z8LvVkhaLvZwJElN2MPR0npmPU97lNKs2cORJDVh4EiSmjBwJElNGDiSpCYMHElSEwaOJKkJA0eS1ISBI0lqwsCRJDVh4EiSmvDWNpq62dzMVNKis4cjSWqiV+AkeXuS+5Pcl+TGJM9Jsi3J3iT7k3wyyanTKlaStLgmDpwkW4C3ARdW1YuBU4ArgQ8AH66qc4DvAFdNo1BJ0mLrO6S2DnhuknXAeuAQcClwczd/N3BFz21IkpbAxBcNVNXBJB8EHgb+C/g8cDdwpKqe6BY7AGxZaz2j0WjSEhhnnKRFc+x93+/9Pz/W/eRWVlbWnD9x4CQ5Hbgc2AYcAT4FXPZM13OiAtfkD2ZJC2llZYXRaNTv/T8n1j25PkNqrwK+WVWPV9X3gVuAS4CN3RAbwFbAVJAk9Qqch4GLk6xPEmA78ABwJ/CGbpkdwK39SpQkLYOJA6eq9jK+OOAe4Ovduv4UeDfwjiT7gTOBXVOoU5K04HrdaaCq3gu897jmh4CL+qxXkrR8vNOAJKkJA0eS1ISBI0lqwsCRJDVh4EiSmjBwJElNGDiSpCYMHElSEwaOJKkJA0eS1ISBI0lqwsCRJDVh4EiSmjBwJElNGDiSpCYMHElSEwaOJKkJA0eS1ISBI0lqwsCRJDVh4EiSmjBwJElNrJt3AZJOPhtvOAish7sOzrWOIzu3zHX7Jxt7OJKkJgwcSVITBo4kqQkDR5LUhIEjSWqiV+Ak2Zjk5iTfSLIvycuSnJHkjiSj7vH0aRUrSVpcfXs41wOfraoXAecB+4DrgD1VtQLs6aYlSSe5iQMnyQbgFcAugKr6n6o6AlwO7O4W2w1c0bdISdLi6/PFz23A48ANSc4D7gauATZV1aFumUeBTWutZDQa9ShhfY/XSjrZTfr50+9za35mXffKysqa8/sEzjrgAuCtVbU3yfUcN3xWVZWk+hS4pjl/S1nSYpvk82c0GvX73JqTIdTd5xzOAeBAVe3tpm9mHECPJdkM0D0e7leiJGkZTBw4VfUo8EiSF3ZN24EHgNuAHV3bDuDWXhVKkpZC35t3vhX4eJJTgYeAnYxD7KYkVwHfBt7YcxuSpCXQK3Cq6l7gwieZtb3PeiVJy8c7DUiSmjBwJElNGDiSpCYMHElSEwaOJKkJA0eS1ISBI0lqwsCRJDVh4EiSmjBwJElNGDiSpCYMHElSEwaOJKkJA0eS1ISBI0lqwsCRJDVh4EiSmjBwJElNGDiSpCYMHElSEwaOJKkJA0eS1ISBI0lqwsCRJDVh4EiSmjBwJElNGDiSpCYMHElSE70DJ8kpSb6S5PZueluSvUn2J/lkklP7lylJWnTT6OFcA+xbNf0B4MNVdQ7wHeCqKWxDkrTgegVOkq3ArwMf66YDXArc3C2yG7iizzYkScthXc/XfwR4F/D8bvpM4EhVPdFNHwC2rLWC0WjUY/Pre7xW0slu0s+ffp9b8zPruldWVtacP3HgJHkdcLiq7k7yyknXc6IC13TXwclfK+mkN8nnz2g06ve5NSdDqLtPD+cS4PVJXgs8B/hJ4HpgY5J1XS9nK2AqSJImP4dTVe+pqq1VdTZwJfCFqnozcCfwhm6xHcCtvauUJC28WXwP593AO5LsZ3xOZ9cMtiFJWjB9LxoAoKq+CHyxe/4QcNE01itJWh7eaUCS1ISBI0lqwsCRJDVh4EiSmjBwJElNGDiSpCYMHElSEwaOJKkJA0eS1ISBI0lqwsCRJDUxlXupSdIi2njDJL+esn7qv8V1ZOeav1O5NOzhSJKaMHAkSU0YOJKkJgwcSVITBo4kqQkDR5LUhIEjSWrCwJEkNWHgSJKaMHAkSU0YOJKkJgwcSVITBo4kqQkDR5LUhIEjSWrCwJEkNTFx4CQ5K8mdSR5Icn+Sa7r2M5LckWTUPZ4+vXIlSYuqTw/nCeCdVXUucDFwdZJzgeuAPVW1AuzppiVJJ7mJA6eqDlXVPd3z/wD2AVuAy4Hd3WK7gSv6FilJWnzrprGSJGcDLwH2Apuq6lA361Fg01qvHY1GPba8vsdrJWkY+n0ODmc7Kysra87vHThJngd8Gri2qr6b5IfzqqqSVJ8C13TXwclfK0kD0etz8GkajUZNtrOWXlepJXkW47D5eFXd0jU/lmRzN38zcLhfiZKkZdDnKrUAu4B9VfWhVbNuA3Z0z3cAt05eniRpWfQZUrsE+C3g60nu7dp+D3g/cFOSq4BvA2/sV6IkaRlMHDhVdReQp5i9fdL1SpKWk3cakCQ1YeBIkpowcCRJTRg4kqQmDBxJUhMGjiSpCQNHktSEgSNJasLAkSQ1YeBIkpowcCRJTaRqzZ+rmYmjR49OZaMbb/D3cCRpWo7s3DK1dW3YsOHH7rVpD0eS1ISBI0lqwsCRJDVh4EiSmjBwJElNGDiSpCYMHElSEwaOJKkJA0eS1ISBI0lqwsCRJDVh4EiSmjBwJElNGDiSpCYMHElSEwaOJKmJmQROksuSPJhkf5LrZrENSdJimXrgJDkF+BPgNcC5wJuSnDvt7UiSFsu6GazzImB/VT0EkOQTwOXAA9Pe0DR/DlWSNFuzGFLbAjyyavpA1yZJOol50YAkqYlZDKkdBM5aNb21a/uhDRs2ZAbblSQN2Cx6OP8ErCTZluRU4ErgthlsR5K0QKYeOFX1BPC7wOeAfcBNVXX/tLdzvCRnJbkzyQNJ7k9yTdd+RpI7koy6x9NnXcszleQ5Sb6U5Ktd7e/r2rcl2dtdXv7JLsAHJckpSb6S5PZuevA1AyT5VpKvJ7k3yZe7tkU4VjYmuTnJN5LsS/Kyoded5IXdfj72990k1w69boAkb+/ek/clubF7rw7+GE9yTVfz/Umu7drmvr9ncg6nqj5TVb9QVT9fVX80i208iSeAd1bVucDFwNXd5djXAXuqagXY000PzX8Dl1bVecD5wGVJLgY+AHy4qs4BvgNcNccan8o1jP9jccwi1HzMr1TV+VV1YTe9CMfK9cBnq+pFwHmM9/2g666qB7v9fD7wUuB7wF8z8LqTbAHeBlxYVS8GTmE8YjPoYzzJi4HfYXzF8HnA65KcwxD2d1Ut5R9wK/CrwIPA5q5tM/DgvGs7Qd3rgXuAXwL+FVjXtb8M+Ny86zuu1q2MD9xLgduBDL3mVbV/C3jBcW2DPlaADcA3gSxS3cfV+mvAPyxC3fzoitszGJ/vvh149dCPceA3gV2rpv8AeNcQ9vdSXqWW5GzgJcBeYFNVHepmPQpsmlNZa+qGpu4FDgN3AP8CHKnxECUM8/LyjzA+kP+3mz6T4dd8TAGfT3J3krd0bUM/VrYBjwM3dMOYH0tyGsOve7UrgRu754Ouu6oOAh8EHgYOAUeBuxn+MX4f8MtJzkyyHngt4wu55r6/ly5wkjwP+DRwbVV9d/W8Gkd7zaWwE6iqH9R4yGEr467wi+Zc0pqSvA44XFV3z7uWCb28qi5gfEeMq5O8YvXMgR4r64ALgI9W1UuA/+S4YZGB1g1Ad67j9cCnjp83xLq7cxyXMw76nwFOAy6ba1FPQ1XtYzzs93ngs8C9wA+OW2Yu+3upAifJsxiHzcer6pau+bEkm7v5mxn3IAarqo4AdzLuqm9McuzS9R+7vHzOLgFen+RbwCcYD6tdz7Br/qHuf69U1WHG5xMuYvjHygHgQFXt7aZvZhxAQ6/7mNcA91TVY9300Ot+FfDNqnq8qr4P3ML4uB/8MV5Vu6rqpVX1Csbnmf6ZAezvpQmcJAF2Afuq6kOrZt0G7Oie72B8bmdQkvxUko3d8+cyPve0j3HwvKFbbFC1V9V7qmprVZ3NeJjkC1X1ZgZc8zFJTkvy/GPPGZ9XuI+BHytV9SjwSJIXdk3bGd8yatB1r/ImfjScBsOv+2Hg4iTru8+XY/t7EY7xn+4efxb4DeCvGML+nvcJrimeKHs54y7i1xh3Ie9lPHZ5JuMT2yPgb4Ez5l3rk9T+i8BXutrvA/6wa/854EvAfsbDEM+ed61PUf8rgdsXpeauxq92f/cDv9+1L8Kxcj7w5e5Y+Rvg9AWp+zTg34ANq9oWoe73Ad/o3pd/CTx7QY7xv2ccjl8Ftg9lf6crRJKkmVqaITVJ0rAZOJKkJgwcSVITBo4kqQkDR5LUhIEjSWrCwJEkNWHgSJKa+D/oiAh6SvA5UwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "remain_ages.hist()\n",
    "len(remain_ages)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "541"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZYAAAD1CAYAAABz79PWAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAOOElEQVR4nO3dfaxkdX3H8fenrIgX7V2gLdnu0kDLrYaYgooUAzUWrEVrhDTUYEyzIaT9x+qiJoo2rfEPE00alT+aJsYNIY31CWkhpFUpYtJt0lV5sAIrnS0+seHBNuza1KYV++0fc3bdbty9l3u/s3Nmeb+SmzvnnJlzPwznzmfP73dmbqoKSZK6/My8A0iSTiwWiySplcUiSWplsUiSWlkskqRWm2a14wMHDni5mSSd4JaXl3PkOs9YJEmtLBZJUqsTplgmk8m8I6yLuY8vcx9/i5rd3Ot3whSLJGkcLBZJUiuLRZLUymKRJLWyWCRJrSwWSVIri0WS1GpmH+kiPdtsvmnfDPe+BLvWtv/9126dYQ5pdZ6xSJJaWSySpFYWiySplcUiSWplsUiSWlkskqRWFoskqZXFIklqZbFIklpZLJKkVhaLJKmVxSJJamWxSJJaWSySpFYWiySplcUiSWplsUiSWlkskqRWFoskqdWaiiXJ25M8mOSBJJ9MckqSc5LsTrI3yaeTnDzrsJKk8Vu1WJJsBd4GXFhVLwZOAq4BPgR8pKrOBZ4CrptlUEnSYljrUNgm4HlJNgFLwGPAZcAtw/abgav640mSFk2qavU7JTuADwD/BXwR2AH803C2QpKzgL8bzmgAOHDgwKEdTyaT5tjS+Lx819K8IwDw1Ut/OO8IOsGtrKwcur28vJwjt29abQdJTgOuBM4B9gOfBa5Yb4hZmUwmx+XndDP38TXT3Lv2zWa/z9DY/r94rBxfY8i9lqGwVwPfqqrvV9WPgFuBS4DNw9AYwDZgHL9VkqS5WkuxfBe4OMlSkgCXAw8BdwNXD/fZDtw2m4iSpEWyarFU1W6mk/T3At8YHvMx4N3AO5LsBc4Ads4wpyRpQaw6xwJQVe8D3nfE6keAi9oTSZIWmu+8lyS1slgkSa0sFklSK4tFktTKYpEktbJYJEmtLBZJUiuLRZLUymKRJLWyWCRJrSwWSVIri0WS1MpikSS1slgkSa0sFklSK4tFktTKYpEktbJYJEmtLBZJUiuLRZLUymKRJLWyWCRJrTbNO4CkXptv2jfvCIfsv3brvCNoDjxjkSS1slgkSa0sFklSK+dYJM3MdL5nCXbNd97HuZ7jyzMWSVIri0WS1MpikSS1slgkSa0sFklSK4tFktTKy421bmP66JC1m/+lr9KJzjMWSVIri0WS1MpikSS1WlOxJNmc5JYk30yyJ8krkpye5M4kk+H7abMOK0kav7VO3t8IfL6qrk5yMrAEvBe4q6o+mOQG4Abg3TPKqcOMY9J8ad4BJI3UqmcsSZaBVwI7Aarqf6pqP3AlcPNwt5uBq2YVUpK0OFJVx75DcgHwMeAh4HzgHmAHsK+qNg/3CfDUwWWAAwcOHNrxZDLpT/4s9vJdni1Iz8RXL/3hvCOcUFZWVg7dXl5ezpHb1zIUtgl4KfDWqtqd5Eamw16HVFUlOWpDHR5iViaTyXH5Od3Wldv3YUjPyHpeG55VrynN1jJ5/yjwaFXtHpZvYVo0TyTZAjB8f3I2ESVJi2TVYqmqx4HvJXnhsOpypsNitwPbh3XbgdtmklCStFDWelXYW4FPDFeEPQJcy7SUPpPkOuA7wBtnE1GStEjWVCxVdT9w4U/ZdHlvHEnSovOd95KkVhaLJKmVxSJJamWxSJJaWSySpFYWiySplcUiSWplsUiSWlkskqRWFoskqZXFIklqZbFIklpZLJKkVhaLJKmVxSJJamWxSJJaWSySpFYWiySplcUiSWplsUiSWm2ad4BFsvmmfTPY6xLsmsV+JWk+PGORJLWyWCRJrSwWSVKr0c+xrH1ew7kKSRoDz1gkSa0sFklSK4tFktTKYpEktbJYJEmtLBZJUiuLRZLUymKRJLWyWCRJrSwWSVIri0WS1MpikSS1WnOxJDkpyX1J7hiWz0myO8neJJ9OcvLsYkqSFsUzOWPZAew5bPlDwEeq6lzgKeC6zmCSpMW0pmJJsg34HeDjw3KAy4BbhrvcDFw1i4CSpMWy1r/H8lHgXcALhuUzgP1V9fSw/Ciw9WgPnkwm6w4ISxt4rCSt/zVoY69d8zPr3CsrK8fcvmqxJHk98GRV3ZPkVbMIcUz+8S5JG7Se16DJZLKx1645GUPutZyxXAK8IcnrgFOAnwVuBDYn2TSctWwDbABJ0upzLFX1nqraVlVnA9cAX6qqNwN3A1cPd9sO3DazlJKkhbGR97G8G3hHkr1M51x29kSSJC2ytU7eA1BVXwa+PNx+BLioP5IkaZH5zntJUiuLRZLUymKRJLWyWCRJrSwWSVIri0WS1MpikSS1slgkSa0sFklSK4tFktTKYpEktbJYJEmtLBZJUiuLRZLUymKRJLWyWCRJrSwWSVIri0WS1MpikSS1slgkSa0sFklSK4tFktTKYpEktbJYJEmtLBZJUiuLRZLUatO8A0jSrG2+ad86HrUEu9bzuKPbf+3W1v2NlWcskqRWFoskqZXFIklqZbFIklpZLJKkVhaLJKmVxSJJamWxSJJaWSySpFYWiySp1arFkuSsJHcneSjJg0l2DOtPT3Jnksnw/bTZx5Ukjd1azlieBt5ZVecBFwNvSXIecANwV1WtAHcNy5KkZ7lVi6WqHquqe4fb/wHsAbYCVwI3D3e7GbhqViElSYvjGX26cZKzgZcAu4Ezq+qxYdPjwJlHe9xkMllnPIClDTxWksZjY6+F4/k5Kysrx9y+5mJJ8nzgc8D1VfWDJIe2VVUlqfWGOKbmj62WpHnZ0GvhGk0mk+Pyc45lTVeFJXkO01L5RFXdOqx+IsmWYfsW4MnZRJQkLZK1XBUWYCewp6o+fNim24Htw+3twG398SRJi2YtQ2GXAL8PfCPJ/cO69wIfBD6T5DrgO8AbZxNRkrRIVi2WqtoF5CibL++NI0ladL7zXpLUymKRJLWyWCRJrSwWSVIri0WS1MpikSS1slgkSa0sFklSK4tFktTKYpEktbJYJEmtLBZJUiuLRZLUymKRJLWyWCRJrSwWSVIri0WS1MpikSS1slgkSa0sFklSK4tFktTKYpEktUpVzWTHBw4caNnx5pv2dexGkjTYf+3Wtn0tLy/nyHWesUiSWlkskqRWFoskqZXFIklqZbFIklpZLJKkVhaLJKmVxSJJamWxSJJaWSySpFYWiySplcUiSWplsUiSWlkskqRWGyqWJFckeTjJ3iQ3dIWSJC2udRdLkpOAPwdeC5wHvCnJeV3BJEmLadMGHnsRsLeqHgFI8ingSuChjmAHdf5BGknS7G1kKGwr8L3Dlh8d1kmSnsWcvJcktdrIUNg+4KzDlrcN64Cf/neQJUknvo2csXwVWElyTpKTgWuA23tiSZIW1bqLpaqeBv4I+AKwB/hMVT3YFexokpyV5O4kDyV5MMmOYf3pSe5MMhm+nzbrLM9EklOSfCXJ14fc7x/Wn5Nk93DJ9qeHkh6dJCcluS/JHcPyouT+dpJvJLk/ydeGdaM+VgCSbE5yS5JvJtmT5BVjz53khcPzfPDrB0muH3tugCRvH34vH0jyyeH3dfTHeJIdQ+YHk1w/rJv7872hOZaq+tuq+tWq+pWq+kBXqFU8Dbyzqs4DLgbeMlzmfANwV1WtAHcNy2Py38BlVXU+cAFwRZKLgQ8BH6mqc4GngOvmmPFYdjD9B8RBi5Ib4Der6oKqunBYHvuxAnAj8PmqehFwPtPnftS5q+rh4Xm+AHgZ8EPgrxl57iRbgbcBF1bVi4GTmI7AjPoYT/Ji4A+YXqF7PvD6JOcyhue7qhb6C7gN+C3gYWDLsG4L8PC8sx0j8xJwL/DrwL8Bm4b1rwC+MO98PyXvNqYH6GXAHUAWIfeQ7dvAzx2xbtTHCrAMfAvIIuU+IutrgH9chNz85ArX05nOO98B/PbYj3Hg94Cdhy3/CfCuMTzfC31VWJKzgZcAu4Ezq+qxYdPjwJlzinVUw3DS/cCTwJ3AvwL7azqsCOO9ZPujTA/Y/x2Wz2AxcgMU8MUk9yT5w2Hd2I+Vc4DvAzcNw48fT3Iq4899uGuATw63R527qvYBfwZ8F3gMOADcw/iP8QeA30hyRpIl4HVML6ia+/O9sMWS5PnA54Drq+oHh2+raVXXXIIdQ1X9uKbDBNuYnr6+aM6RVpXk9cCTVXXPvLOs06VV9VKmnxDxliSvPHzjSI+VTcBLgb+oqpcA/8kRwxkjzQ3AMBfxBuCzR24bY+5hDuJKpoX+i8CpwBVzDbUGVbWH6XDdF4HPA/cDPz7iPnN5vheyWJI8h2mpfKKqbh1WP5Fky7B9C9OzglGqqv3A3UxPrzcnOXjZ9/+7ZHskLgHekOTbwKeYDofdyPhzA4f+NUpVPcl0vP8ixn+sPAo8WlW7h+VbmBbN2HMf9Frg3qp6Ylgee+5XA9+qqu9X1Y+AW5ke96M/xqtqZ1W9rKpeyXQe6F8YwfO9cMWSJMBOYE9VffiwTbcD24fb25nOvYxGkp9Psnm4/Tym80J7mBbM1cPdRpe7qt5TVduq6mymwxtfqqo3M/LcAElOTfKCg7eZjvs/wMiPlap6HPhekhcOqy5n+lFJo859mDfxk2EwGH/u7wIXJ1kaXl8OPt+LcIz/wvD9l4DfBf6KMTzf856AWseE1aVMT+3+memp3/1MxxbPYDrBPAH+Hjh93lmPyP1rwH1D7geAPx3W/zLwFWAv06GD58476zH+G14F3LEouYeMXx++HgT+eFg/6mNlyHgB8LXhePkb4LQFyX0q8O/A8mHrFiH3+4FvDr+bfwk8d0GO8X9gWoJfBy4fy/OdIYgkSS0WbihMkjRuFoskqZXFIklqZbFIklpZLJKkVhaLJKmVxSJJamWxSJJa/R+gJ38iQtCgjwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "leave_ages.hist()\n",
    "len(leave_ages)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "These certainly look like different distributions."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We might summarize the difference, by looking at the difference in means:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "51.715341959334566"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "leave_mean = np.mean(leave_ages)\n",
    "leave_mean"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "48.01550387596899"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "remain_mean = np.mean(remain_ages)\n",
    "remain_mean"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3.6998380833655773"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "difference = leave_mean - remain_mean\n",
    "difference"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The distributions do look different.\n",
    "\n",
    "They have a mean difference of nearly 4 years.\n",
    "\n",
    "Could this be due to sampling error?\n",
    "\n",
    "If we took two random samples of 774 and 541 voters, *from the same\n",
    "population*, we would expect to see some difference, just by chance.\n",
    "\n",
    "*By chance* means, because random samples vary."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "What is the population, in this case?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It is not exactly the whole UK population, because the survey only sampled\n",
    "people who were eligible to vote."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It might not even be the whole UK population, who are eligible to vote.\n",
    "Perhaps the survey company got a not-representative range of ages, for some\n",
    "reason.  We are not interested in that question, only the question of whether\n",
    "the Leave and Remain voters could come from the same population, where the\n",
    "population is, people selected by the survey company."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "How do we find this population, to do our simulation?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Population by permutation\n",
    "\n",
    "Here comes a nice trick.   We can use the data that we already have, to\n",
    "simulate the effect of drawing lots of random samples, from the underlying\n",
    "population."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let us assume that the Leave voters and the Remain voters are in fact samples\n",
    "from the same underlying population.\n",
    "\n",
    "If that is the case, we can throw the Leave and Remain voters into one big pool\n",
    "of 774 + 541 == 1315 voters.\n",
    "\n",
    "Then we can take split this new mixed sample into two groups, at\n",
    "random, one with 774 voters, and the other with 541.  The new groups\n",
    "have a random mix of the original Leave and Remain voters.  Then we\n",
    "calculate the difference in means between these two new, fake groups."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([37, 55, 37, ..., 20, 40, 31])"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pooled = np.append(remain_ages, leave_ages)\n",
    "pooled"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1315"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(pooled)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We mix the two samples together, using `np.random.shuffle`.  It works like\n",
    "this:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array(['cat', 'dog', 'rabbit'], dtype='<U6')"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pets = np.array(['cat', 'dog', 'rabbit'])\n",
    "pets"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array(['cat', 'rabbit', 'dog'], dtype='<U6')"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.random.shuffle(pets)\n",
    "pets"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array(['cat', 'rabbit', 'dog'], dtype='<U6')"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.random.shuffle(pets)\n",
    "pets"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now to mix up ages of the Leavers and Remainers:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([54, 50, 19, ..., 52, 80, 92])"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.random.shuffle(pooled)\n",
    "pooled"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We split the newly mixed group into 774 simulated Remain voters and 541\n",
    "simulated Leave voters, where each group is a random mix of the original Leave\n",
    "and Remain ages."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "541"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# The first 774 values\n",
    "fake_remainers = pooled[:774]\n",
    "# The rest\n",
    "fake_leavers = pooled[774:]\n",
    "len(fake_leavers)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we can calculate the mean difference.  This is our first simulation:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.11033973835417754"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "fake_difference = np.mean(fake_leavers) - np.mean(fake_remainers)\n",
    "fake_difference"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "That looks a lot smaller than the difference we saw.   We want to keep doing\n",
    "this, to collect more simulations.  We need to mix up the ages again, to give\n",
    "us new random samples of fake Remainers and fake Leavers."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "-1.5329493186605347"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.random.shuffle(pooled)\n",
    "fake_difference_2 = np.mean(pooled[:774]) - np.mean(pooled[774:])\n",
    "fake_difference_2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We want to keep doing this - and that calls for a `for` loop."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}