Randomizing building attributes in exposure databases.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Randomizing building attributes in the exposure database\n",
    "\n",
    "To explore uncertainty in modelling the impact of TCs, we would like to randomly attribute building vulnerability to buildings within an exposure database. By doing so, we can quantify the maximum possible range of impact outcomes based solely on the known building stock. \n",
    "\n",
    "A second step would be to randomize the vulnerability on a per-suburb basis. This would have the effect of constraining the range of building types within each suburb, which each have different age profiles (age is a major contributor to vulnerability). "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "from __future__ import division, print_function\n",
    "import warnings\n",
    "warnings.filterwarnings(\"ignore\", category=FutureWarning)\n",
    "\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import seaborn as sns\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "from mpl_toolkits.basemap import Basemap\n",
    "\n",
    "sns.set_context(\"poster\")\n",
    "sns.set_style('ticks')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Start by loading the base exposure database file into a pandas ``DataFrame``. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "filename = \"C:/WorkSpace/data/exposure/CAIRNS_Residential_Wind_Exposure_201510_M4.csv\"\n",
    "df = pd.read_csv(filename, sep=\",\", header=0, index_col=0, skipinitialspace=True)\n",
    "df.info()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This function assigns a classification to each building based on the value of the \"M4\" field in the loaded data frame. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def buildingClass(df):\n",
    "    thresholds = [0.0, 0.8278, 0.973, 1.147]\n",
    "    classes = ['C1', 'C2', 'C3', 'C4']\n",
    "    for thres, cls in zip(thresholds, classes):\n",
    "        idx = np.where(df['M4'] >= thres)[0]\n",
    "        df['AS4055_CLASS'][idx] = cls\n",
    "        \n",
    "    return df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This function assigns a building vulnerability curve based on the given AS 4055 class (which is assigned by the previous function). It ignores all buildings that are indicated as older than 1982, because this standard only came into being in the early 1980's."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def vulnCurve(df, default='dw1'):\n",
    "    classes = ['C1', 'C2', 'C3', 'C4']\n",
    "    curves = ['dw3', 'dw4', 'dw5', 'dw6']\n",
    "    # Set all to be default curve to begin with\n",
    "    df['WIND_VULNERABILITY_FUNCTION_ID'] = default\n",
    "    filter = df['YEAR_BUILT'].map(lambda x: x not in ['1982 - 1996', '1997 - present'])\n",
    "    for cls, curve in zip(classes, curves):\n",
    "        idx = np.where(df['AS4055_CLASS'] == cls)[0]\n",
    "        df['WIND_VULNERABILITY_FUNCTION_ID'][idx] = curve\n",
    "\n",
    "    df['WIND_VULNERABILITY_FUNCTION_ID'][filter] = default\n",
    "    return df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Perform the classification (based on M4 value), and assign an appropriate wind vulnerability curve. The default curve corresponds to the pre-code curve."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "df = buildingClass(df)\n",
    "df = vulnCurve(df, 'dw1')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here's where the magic happens. Using the ``groupby`` method in combination with the ``transform`` method, we take random permutations of the attribute we want to shuffle. We can group by any sensible attribute (e.g. LGA, suburb, meshblock) and randomly permutate any attribute (e.g. the wind vulnerability function). It returns a new dataframe, so the original data remains untouched. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "collapsed": false,
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "def randomize(df, byfield, attribute):\n",
    "    newdf = df.copy()\n",
    "    newdf[attribute] = newdf.groupby(byfield)[attribute].transform(np.random.permutation)\n",
    "    return newdf"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "newdf = randomize(df, 'SUBURB_2015', 'WIND_VULNERABILITY_FUNCTION_ID')\n",
    "\n",
    "fig, (ax0, ax1) = plt.subplots(1, 2)\n",
    "\n",
    "colors = {'dw1':'r', 'dw3':'k', 'dw4':'b', 'dw5':'g', 'dw6':'y'}\n",
    "minLon = newdf.LONGITUDE.min()\n",
    "maxLon = newdf.LONGITUDE.max()\n",
    "minLat = newdf.LATITUDE.min()\n",
    "maxLat = newdf.LATITUDE.max()\n",
    "\n",
    "\n",
    "m0 = Basemap(projection='cyl', llcrnrlon=minLon, llcrnrlat=minLat, \n",
    "            urcrnrlon=maxLon, urcrnrlat=maxLat, resolution='h', ax=ax0)\n",
    "m0.drawcoastlines()\n",
    "m0.drawstates()\n",
    "m0.drawcountries()\n",
    "m0.scatter(df.LONGITUDE, df.LATITUDE, \n",
    "           c=df['WIND_VULNERABILITY_FUNCTION_ID'].apply(lambda x: colors[x]), \n",
    "           alpha=0.25, edgecolors=None, s=4)\n",
    "\n",
    "m1 = Basemap(projection='cyl', llcrnrlon=minLon, llcrnrlat=minLat, \n",
    "            urcrnrlon=maxLon, urcrnrlat=maxLat, resolution='h', ax=ax1)\n",
    "m1.drawcoastlines()\n",
    "m1.drawstates()\n",
    "m1.drawcountries()\n",
    "m1.scatter(newdf.LONGITUDE, newdf.LATITUDE, \n",
    "           c=newdf['WIND_VULNERABILITY_FUNCTION_ID'].apply(lambda x: colors[x]), \n",
    "           alpha=0.25, edgecolors=None, s=4)\n",
    "\n",
    "ax0.set_title(\"Original exposure\")\n",
    "ax1.set_title(\"Randomized exposure (by suburb)\")\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "outputfile = \"C:/WorkSpace/data/exposure/cairns_retrofit_classified_shuffled.csv\"\n",
    "df.to_csv(outputfile, header=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "datadict = newdf.to_dict('list')\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}