1-reading-writing-files.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Data Analysis Using Python: A Beginner’s Guide Featuring NYC Open Data\n",
    "## Part 1: Reading and Writing Files in Python\n",
    "\n",
    "Author: Mark Bauer"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Table of Contents\n",
    "=================\n",
    "\n",
    "   Getting Started\n",
    " \n",
    "   1. Reading In Data from Local Folder\n",
    "       * 1.1 Reading in a CSV file  \n",
    "       * 1.2 Reading in a JSON file  \n",
    "       * 1.3 Reading in an Excel file  \n",
    "       * 1.4 Reading in a Shapefile  \n",
    "       * 1.5 Reading in a GeoJSON file   \n",
    "   2. Reading In Data from a URL         \n",
    "       * 2.1 Unzipping and reading in data as csv to local folders  \n",
    "       * 2.2 Unzipping and reading in data as csv from local folder  \n",
    "       * 2.3 Unzipping and reading in data as csv in memory  \n",
    "   3. Reading In Data from NYC Open Data  \n",
    "       * 3.1 Reading in data as CSV in static form  \n",
    "       * 3.2 Reading in data as JSON in static form  \n",
    "       * 3.3 Reading in Shapefile data  \n",
    "       * 3.4 Reading in data from Socrata Open Data API (SODA)    \n",
    "   4. Writing Out Data  \n",
    "       * 4.1 Writing to a CSV file  \n",
    "       * 4.2 Writing to a Excel (xlsx) file  \n",
    "       * 4.3 Writing to a JSON file  \n",
    "       * 4.4 Writing to a Shapefile  \n",
    "   5. Conclusion"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Goal:** In this notebook, we will review various ways to read (load) and write (save) data from NYC Open Data. Specifically, we will focus on reading our data into a pandas dataframe.\n",
    "\n",
    "**Main Library:** [pandas](https://pandas.pydata.org/) is a fast, powerful, flexible and easy to use open source data analysis and manipulation tool, built on top of the Python programming language."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "# importing libraries\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import geopandas as gpd\n",
    "from fiona.crs import from_epsg\n",
    "import matplotlib.pyplot as plt\n",
    "import os\n",
    "from pathlib import Path\n",
    "import urllib\n",
    "import json\n",
    "import requests\n",
    "from io import BytesIO\n",
    "from sodapy import Socrata\n",
    "import zipfile\n",
    "from zipfile import ZipFile\n",
    "from os.path import basename\n",
    "from openpyxl import Workbook"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Python implementation: CPython\n",
      "Python version       : 3.8.13\n",
      "IPython version      : 8.4.0\n",
      "\n",
      "numpy     : 1.23.1\n",
      "pandas    : 1.4.3\n",
      "geopandas : 0.11.1\n",
      "matplotlib: 3.5.2\n",
      "json      : 2.0.9\n",
      "requests  : 2.28.1\n",
      "sodapy    : 2.1.1\n",
      "\n"
     ]
    }
   ],
   "source": [
    "'''\n",
    "Printing verions of Python modules and packages with **watermark** - the IPython magic extension.\n",
    "Documention for installing watermark: https://github.com/rasbt/watermark\n",
    "'''\n",
    "\n",
    "%reload_ext watermark\n",
    "%watermark -v -p numpy,pandas,geopandas,matplotlib,json,requests,sodapy"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Getting Started: Accessing the Building Footprints Dataset\n",
    "\n",
    "I've provided sample data for you in the [data folder](https://github.com/mebauer/data-analysis-using-python/tree/master/1-reading-writing-files/data), but follow these steps to access data used in these tutorials. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. Search NYC Open Data in Google\n",
    "![building_footprints](images/1.nyc-open-data-google.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. Search \"Building Footprints\" in NYC Open Data search bar\n",
    "![building_footprints](images/2.building-footprints-opendata-search.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. Select \"Building Footprints\" Dataset\n",
    "![building_footprints](images/3.building-footprints-dataset-link.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4. The Building Footprints Dataset Page\n",
    "![building_footprints](images/4.data-homepage.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Dataset Link: \n",
    "https://data.cityofnewyork.us/Housing-Development/Building-Footprints/nqwf-w8eh\n",
    "\n",
    "Documentation/Metadata: \n",
    "https://github.com/CityOfNewYork/nyc-geo-metadata/blob/master/Metadata/Metadata_BuildingFootprints.md"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 1. Reading In Data from Local Folder\n",
    "\n",
    "Before we dive into retrieving data on NYC Open Data, let's practice reading data into a pandas dataframe from a local folder. If you'd like to see how I exported this sample data, please visit the [data-wrangling notebook](https://github.com/mebauer/data-analysis-using-python/blob/main/wrangling/sample-data.ipynb). "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1.1 Reading in a CSV file"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>the_geom</th>\n",
       "      <th>NAME</th>\n",
       "      <th>BIN</th>\n",
       "      <th>CNSTRCT_YR</th>\n",
       "      <th>LSTMODDATE</th>\n",
       "      <th>LSTSTATYPE</th>\n",
       "      <th>DOITT_ID</th>\n",
       "      <th>HEIGHTROOF</th>\n",
       "      <th>FEAT_CODE</th>\n",
       "      <th>GROUNDELEV</th>\n",
       "      <th>SHAPE_AREA</th>\n",
       "      <th>SHAPE_LEN</th>\n",
       "      <th>BASE_BBL</th>\n",
       "      <th>MPLUTO_BBL</th>\n",
       "      <th>GEOMSOURCE</th>\n",
       "      <th>GLOBALID</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>MULTIPOLYGON (((-73.87172426474349 40.65519420...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>3394834</td>\n",
       "      <td>2011.0</td>\n",
       "      <td>08/22/2017 12:00:00 AM</td>\n",
       "      <td>Constructed</td>\n",
       "      <td>1250314</td>\n",
       "      <td>26.000000</td>\n",
       "      <td>2100</td>\n",
       "      <td>15.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>3044520924</td>\n",
       "      <td>3044520924</td>\n",
       "      <td>Other (Man</td>\n",
       "      <td>{C045C815-79DB-4644-AD9D-C34AC03D1AB4}</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>MULTIPOLYGON (((-73.86650099829305 40.74236058...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>4540159</td>\n",
       "      <td>2010.0</td>\n",
       "      <td>08/22/2017 12:00:00 AM</td>\n",
       "      <td>Constructed</td>\n",
       "      <td>201366</td>\n",
       "      <td>28.000000</td>\n",
       "      <td>2100</td>\n",
       "      <td>37.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>4018780115</td>\n",
       "      <td>4018780115</td>\n",
       "      <td>Other (Man</td>\n",
       "      <td>{FDF673E7-FF92-4A7A-AF6D-C49D77343C47}</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>MULTIPOLYGON (((-73.87805078807256 40.71475698...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>4540051</td>\n",
       "      <td>2010.0</td>\n",
       "      <td>08/22/2017 12:00:00 AM</td>\n",
       "      <td>Constructed</td>\n",
       "      <td>1171655</td>\n",
       "      <td>28.330225</td>\n",
       "      <td>2100</td>\n",
       "      <td>112.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>4030600139</td>\n",
       "      <td>4030600139</td>\n",
       "      <td>Photogramm</td>\n",
       "      <td>{788E5D72-46C1-443F-8BC9-6B97F329BFED}</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>MULTIPOLYGON (((-73.81520745135124 40.73053646...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>4545453</td>\n",
       "      <td>2012.0</td>\n",
       "      <td>08/22/2017 12:00:00 AM</td>\n",
       "      <td>Constructed</td>\n",
       "      <td>1118502</td>\n",
       "      <td>16.640000</td>\n",
       "      <td>2100</td>\n",
       "      <td>74.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>4066560052</td>\n",
       "      <td>4066560052</td>\n",
       "      <td>Photogramm</td>\n",
       "      <td>{789A5A51-5B12-46DC-AE85-B06820A3225E}</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>MULTIPOLYGON (((-73.84769179857282 40.87911947...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2118998</td>\n",
       "      <td>2012.0</td>\n",
       "      <td>08/22/2017 12:00:00 AM</td>\n",
       "      <td>Constructed</td>\n",
       "      <td>1254551</td>\n",
       "      <td>33.000000</td>\n",
       "      <td>2100</td>\n",
       "      <td>154.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2047220003</td>\n",
       "      <td>2047220003</td>\n",
       "      <td>Other (Man</td>\n",
       "      <td>{BB58FD7B-CC22-4896-901D-F8BAFF4AC129}</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                            the_geom NAME      BIN  \\\n",
       "0  MULTIPOLYGON (((-73.87172426474349 40.65519420...  NaN  3394834   \n",
       "1  MULTIPOLYGON (((-73.86650099829305 40.74236058...  NaN  4540159   \n",
       "2  MULTIPOLYGON (((-73.87805078807256 40.71475698...  NaN  4540051   \n",
       "3  MULTIPOLYGON (((-73.81520745135124 40.73053646...  NaN  4545453   \n",
       "4  MULTIPOLYGON (((-73.84769179857282 40.87911947...  NaN  2118998   \n",
       "\n",
       "   CNSTRCT_YR              LSTMODDATE   LSTSTATYPE  DOITT_ID  HEIGHTROOF  \\\n",
       "0      2011.0  08/22/2017 12:00:00 AM  Constructed   1250314   26.000000   \n",
       "1      2010.0  08/22/2017 12:00:00 AM  Constructed    201366   28.000000   \n",
       "2      2010.0  08/22/2017 12:00:00 AM  Constructed   1171655   28.330225   \n",
       "3      2012.0  08/22/2017 12:00:00 AM  Constructed   1118502   16.640000   \n",
       "4      2012.0  08/22/2017 12:00:00 AM  Constructed   1254551   33.000000   \n",
       "\n",
       "   FEAT_CODE  GROUNDELEV  SHAPE_AREA  SHAPE_LEN    BASE_BBL  MPLUTO_BBL  \\\n",
       "0       2100        15.0           0          0  3044520924  3044520924   \n",
       "1       2100        37.0           0          0  4018780115  4018780115   \n",
       "2       2100       112.0           0          0  4030600139  4030600139   \n",
       "3       2100        74.0           0          0  4066560052  4066560052   \n",
       "4       2100       154.0           0          0  2047220003  2047220003   \n",
       "\n",
       "   GEOMSOURCE                                GLOBALID  \n",
       "0  Other (Man  {C045C815-79DB-4644-AD9D-C34AC03D1AB4}  \n",
       "1  Other (Man  {FDF673E7-FF92-4A7A-AF6D-C49D77343C47}  \n",
       "2  Photogramm  {788E5D72-46C1-443F-8BC9-6B97F329BFED}  \n",
       "3  Photogramm  {789A5A51-5B12-46DC-AE85-B06820A3225E}  \n",
       "4  Other (Man  {BB58FD7B-CC22-4896-901D-F8BAFF4AC129}  "
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# read data as a dataframe\n",
    "df = pd.read_csv('data/sample-data.csv')\n",
    "\n",
    "# previewing first five rows in data\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "This dataset has 16,308 rows and 16 columns.\n"
     ]
    }
   ],
   "source": [
    "rows, columns = df.shape\n",
    "print('This dataset has {:,} rows and {:,} columns.'.format(rows, columns))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Sanity check**\n",
    "\n",
    "We use pandas `.head()` method to preview the first five rows of the dataframe.\n",
    "\n",
    "We use pandas `.shape` method to print the dimensions of the dataframe (i.e. number of rows, number of columns).\n",
    "\n",
    "We will use these two methods throughout the examples."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1.2 Reading in a JSON file"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>the_geom</th>\n",
       "      <th>NAME</th>\n",
       "      <th>BIN</th>\n",
       "      <th>CNSTRCT_YR</th>\n",
       "      <th>LSTMODDATE</th>\n",
       "      <th>LSTSTATYPE</th>\n",
       "      <th>DOITT_ID</th>\n",
       "      <th>HEIGHTROOF</th>\n",
       "      <th>FEAT_CODE</th>\n",
       "      <th>GROUNDELEV</th>\n",
       "      <th>SHAPE_AREA</th>\n",
       "      <th>SHAPE_LEN</th>\n",
       "      <th>BASE_BBL</th>\n",
       "      <th>MPLUTO_BBL</th>\n",
       "      <th>GEOMSOURCE</th>\n",
       "      <th>GLOBALID</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>MULTIPOLYGON (((-73.87172426474349 40.65519420...</td>\n",
       "      <td>None</td>\n",
       "      <td>3394834</td>\n",
       "      <td>2011</td>\n",
       "      <td>08/22/2017 12:00:00 AM</td>\n",
       "      <td>Constructed</td>\n",
       "      <td>1250314</td>\n",
       "      <td>26.000000</td>\n",
       "      <td>2100</td>\n",
       "      <td>15.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>3044520924</td>\n",
       "      <td>3044520924</td>\n",
       "      <td>Other (Man</td>\n",
       "      <td>{C045C815-79DB-4644-AD9D-C34AC03D1AB4}</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>MULTIPOLYGON (((-73.86650099829305 40.74236058...</td>\n",
       "      <td>None</td>\n",
       "      <td>4540159</td>\n",
       "      <td>2010</td>\n",
       "      <td>08/22/2017 12:00:00 AM</td>\n",
       "      <td>Constructed</td>\n",
       "      <td>201366</td>\n",
       "      <td>28.000000</td>\n",
       "      <td>2100</td>\n",
       "      <td>37.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>4018780115</td>\n",
       "      <td>4018780115</td>\n",
       "      <td>Other (Man</td>\n",
       "      <td>{FDF673E7-FF92-4A7A-AF6D-C49D77343C47}</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>MULTIPOLYGON (((-73.87805078807256 40.71475698...</td>\n",
       "      <td>None</td>\n",
       "      <td>4540051</td>\n",
       "      <td>2010</td>\n",
       "      <td>08/22/2017 12:00:00 AM</td>\n",
       "      <td>Constructed</td>\n",
       "      <td>1171655</td>\n",
       "      <td>28.330225</td>\n",
       "      <td>2100</td>\n",
       "      <td>112.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>4030600139</td>\n",
       "      <td>4030600139</td>\n",
       "      <td>Photogramm</td>\n",
       "      <td>{788E5D72-46C1-443F-8BC9-6B97F329BFED}</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>MULTIPOLYGON (((-73.81520745135124 40.73053646...</td>\n",
       "      <td>None</td>\n",
       "      <td>4545453</td>\n",
       "      <td>2012</td>\n",
       "      <td>08/22/2017 12:00:00 AM</td>\n",
       "      <td>Constructed</td>\n",
       "      <td>1118502</td>\n",
       "      <td>16.640000</td>\n",
       "      <td>2100</td>\n",
       "      <td>74.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>4066560052</td>\n",
       "      <td>4066560052</td>\n",
       "      <td>Photogramm</td>\n",
       "      <td>{789A5A51-5B12-46DC-AE85-B06820A3225E}</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>MULTIPOLYGON (((-73.84769179857282 40.87911947...</td>\n",
       "      <td>None</td>\n",
       "      <td>2118998</td>\n",
       "      <td>2012</td>\n",
       "      <td>08/22/2017 12:00:00 AM</td>\n",
       "      <td>Constructed</td>\n",
       "      <td>1254551</td>\n",
       "      <td>33.000000</td>\n",
       "      <td>2100</td>\n",
       "      <td>154.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2047220003</td>\n",
       "      <td>2047220003</td>\n",
       "      <td>Other (Man</td>\n",
       "      <td>{BB58FD7B-CC22-4896-901D-F8BAFF4AC129}</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                            the_geom  NAME      BIN  \\\n",
       "0  MULTIPOLYGON (((-73.87172426474349 40.65519420...  None  3394834   \n",
       "1  MULTIPOLYGON (((-73.86650099829305 40.74236058...  None  4540159   \n",
       "2  MULTIPOLYGON (((-73.87805078807256 40.71475698...  None  4540051   \n",
       "3  MULTIPOLYGON (((-73.81520745135124 40.73053646...  None  4545453   \n",
       "4  MULTIPOLYGON (((-73.84769179857282 40.87911947...  None  2118998   \n",
       "\n",
       "   CNSTRCT_YR              LSTMODDATE   LSTSTATYPE  DOITT_ID  HEIGHTROOF  \\\n",
       "0        2011  08/22/2017 12:00:00 AM  Constructed   1250314   26.000000   \n",
       "1        2010  08/22/2017 12:00:00 AM  Constructed    201366   28.000000   \n",
       "2        2010  08/22/2017 12:00:00 AM  Constructed   1171655   28.330225   \n",
       "3        2012  08/22/2017 12:00:00 AM  Constructed   1118502   16.640000   \n",
       "4        2012  08/22/2017 12:00:00 AM  Constructed   1254551   33.000000   \n",
       "\n",
       "   FEAT_CODE  GROUNDELEV  SHAPE_AREA  SHAPE_LEN    BASE_BBL  MPLUTO_BBL  \\\n",
       "0       2100        15.0           0          0  3044520924  3044520924   \n",
       "1       2100        37.0           0          0  4018780115  4018780115   \n",
       "2       2100       112.0           0          0  4030600139  4030600139   \n",
       "3       2100        74.0           0          0  4066560052  4066560052   \n",
       "4       2100       154.0           0          0  2047220003  2047220003   \n",
       "\n",
       "   GEOMSOURCE                                GLOBALID  \n",
       "0  Other (Man  {C045C815-79DB-4644-AD9D-C34AC03D1AB4}  \n",
       "1  Other (Man  {FDF673E7-FF92-4A7A-AF6D-C49D77343C47}  \n",
       "2  Photogramm  {788E5D72-46C1-443F-8BC9-6B97F329BFED}  \n",
       "3  Photogramm  {789A5A51-5B12-46DC-AE85-B06820A3225E}  \n",
       "4  Other (Man  {BB58FD7B-CC22-4896-901D-F8BAFF4AC129}  "
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# read data as a dataframe\n",
    "df = pd.read_json('data/sample-data.json')\n",
    "\n",
    "# previewing first five rows in data\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "This dataset has 16,308 rows and 16 columns.\n"
     ]
    }
   ],
   "source": [
    "rows, columns = df.shape\n",
    "print('This dataset has {:,} rows and {:,} columns.'.format(rows, columns))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1.3 Reading in an Excel file"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>the_geom</th>\n",
       "      <th>NAME</th>\n",
       "      <th>BIN</th>\n",
       "      <th>CNSTRCT_YR</th>\n",
       "      <th>LSTMODDATE</th>\n",
       "      <th>LSTSTATYPE</th>\n",
       "      <th>DOITT_ID</th>\n",
       "      <th>HEIGHTROOF</th>\n",
       "      <th>FEAT_CODE</th>\n",
       "      <th>GROUNDELEV</th>\n",
       "      <th>SHAPE_AREA</th>\n",
       "      <th>SHAPE_LEN</th>\n",
       "      <th>BASE_BBL</th>\n",
       "      <th>MPLUTO_BBL</th>\n",
       "      <th>GEOMSOURCE</th>\n",
       "      <th>GLOBALID</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>MULTIPOLYGON (((-73.87172426474349 40.65519420...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>3394834</td>\n",
       "      <td>2011</td>\n",
       "      <td>08/22/2017 12:00:00 AM</td>\n",
       "      <td>Constructed</td>\n",
       "      <td>1250314</td>\n",
       "      <td>26.000000</td>\n",
       "      <td>2100</td>\n",
       "      <td>15.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>3044520924</td>\n",
       "      <td>3044520924</td>\n",
       "      <td>Other (Man</td>\n",
       "      <td>{C045C815-79DB-4644-AD9D-C34AC03D1AB4}</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>MULTIPOLYGON (((-73.86650099829305 40.74236058...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>4540159</td>\n",
       "      <td>2010</td>\n",
       "      <td>08/22/2017 12:00:00 AM</td>\n",
       "      <td>Constructed</td>\n",
       "      <td>201366</td>\n",
       "      <td>28.000000</td>\n",
       "      <td>2100</td>\n",
       "      <td>37.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>4018780115</td>\n",
       "      <td>4018780115</td>\n",
       "      <td>Other (Man</td>\n",
       "      <td>{FDF673E7-FF92-4A7A-AF6D-C49D77343C47}</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>MULTIPOLYGON (((-73.87805078807256 40.71475698...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>4540051</td>\n",
       "      <td>2010</td>\n",
       "      <td>08/22/2017 12:00:00 AM</td>\n",
       "      <td>Constructed</td>\n",
       "      <td>1171655</td>\n",
       "      <td>28.330225</td>\n",
       "      <td>2100</td>\n",
       "      <td>112.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>4030600139</td>\n",
       "      <td>4030600139</td>\n",
       "      <td>Photogramm</td>\n",
       "      <td>{788E5D72-46C1-443F-8BC9-6B97F329BFED}</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>MULTIPOLYGON (((-73.81520745135124 40.73053646...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>4545453</td>\n",
       "      <td>2012</td>\n",
       "      <td>08/22/2017 12:00:00 AM</td>\n",
       "      <td>Constructed</td>\n",
       "      <td>1118502</td>\n",
       "      <td>16.640000</td>\n",
       "      <td>2100</td>\n",
       "      <td>74.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>4066560052</td>\n",
       "      <td>4066560052</td>\n",
       "      <td>Photogramm</td>\n",
       "      <td>{789A5A51-5B12-46DC-AE85-B06820A3225E}</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>MULTIPOLYGON (((-73.84769179857282 40.87911947...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2118998</td>\n",
       "      <td>2012</td>\n",
       "      <td>08/22/2017 12:00:00 AM</td>\n",
       "      <td>Constructed</td>\n",
       "      <td>1254551</td>\n",
       "      <td>33.000000</td>\n",
       "      <td>2100</td>\n",
       "      <td>154.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2047220003</td>\n",
       "      <td>2047220003</td>\n",
       "      <td>Other (Man</td>\n",
       "      <td>{BB58FD7B-CC22-4896-901D-F8BAFF4AC129}</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                            the_geom NAME      BIN  \\\n",
       "0  MULTIPOLYGON (((-73.87172426474349 40.65519420...  NaN  3394834   \n",
       "1  MULTIPOLYGON (((-73.86650099829305 40.74236058...  NaN  4540159   \n",
       "2  MULTIPOLYGON (((-73.87805078807256 40.71475698...  NaN  4540051   \n",
       "3  MULTIPOLYGON (((-73.81520745135124 40.73053646...  NaN  4545453   \n",
       "4  MULTIPOLYGON (((-73.84769179857282 40.87911947...  NaN  2118998   \n",
       "\n",
       "   CNSTRCT_YR              LSTMODDATE   LSTSTATYPE  DOITT_ID  HEIGHTROOF  \\\n",
       "0        2011  08/22/2017 12:00:00 AM  Constructed   1250314   26.000000   \n",
       "1        2010  08/22/2017 12:00:00 AM  Constructed    201366   28.000000   \n",
       "2        2010  08/22/2017 12:00:00 AM  Constructed   1171655   28.330225   \n",
       "3        2012  08/22/2017 12:00:00 AM  Constructed   1118502   16.640000   \n",
       "4        2012  08/22/2017 12:00:00 AM  Constructed   1254551   33.000000   \n",
       "\n",
       "   FEAT_CODE  GROUNDELEV  SHAPE_AREA  SHAPE_LEN    BASE_BBL  MPLUTO_BBL  \\\n",
       "0       2100        15.0           0          0  3044520924  3044520924   \n",
       "1       2100        37.0           0          0  4018780115  4018780115   \n",
       "2       2100       112.0           0          0  4030600139  4030600139   \n",
       "3       2100        74.0           0          0  4066560052  4066560052   \n",
       "4       2100       154.0           0          0  2047220003  2047220003   \n",
       "\n",
       "   GEOMSOURCE                                GLOBALID  \n",
       "0  Other (Man  {C045C815-79DB-4644-AD9D-C34AC03D1AB4}  \n",
       "1  Other (Man  {FDF673E7-FF92-4A7A-AF6D-C49D77343C47}  \n",
       "2  Photogramm  {788E5D72-46C1-443F-8BC9-6B97F329BFED}  \n",
       "3  Photogramm  {789A5A51-5B12-46DC-AE85-B06820A3225E}  \n",
       "4  Other (Man  {BB58FD7B-CC22-4896-901D-F8BAFF4AC129}  "
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# read data as a dataframe\n",
    "df = pd.read_excel('data/sample-data.xlsx')\n",
    "\n",
    "# previewing first five rows in data\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "This dataset has 16,308 rows and 16 columns.\n"
     ]
    }
   ],
   "source": [
    "rows, columns = df.shape\n",
    "print('This dataset has {:,} rows and {:,} columns.'.format(rows, columns))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1.4 Reading in a Shapefile"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>the_geom</th>\n",
       "      <th>NAME</th>\n",
       "      <th>BIN</th>\n",
       "      <th>CNSTRCT_YR</th>\n",
       "      <th>LSTMODDATE</th>\n",
       "      <th>LSTSTATYPE</th>\n",
       "      <th>DOITT_ID</th>\n",
       "      <th>HEIGHTROOF</th>\n",
       "      <th>FEAT_CODE</th>\n",
       "      <th>GROUNDELEV</th>\n",
       "      <th>SHAPE_AREA</th>\n",
       "      <th>SHAPE_LEN</th>\n",
       "      <th>BASE_BBL</th>\n",
       "      <th>MPLUTO_BBL</th>\n",
       "      <th>GEOMSOURCE</th>\n",
       "      <th>GLOBALID</th>\n",
       "      <th>geometry</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>MULTIPOLYGON (((-73.87172426474349 40.65519420...</td>\n",
       "      <td>None</td>\n",
       "      <td>3394834</td>\n",
       "      <td>2011.0</td>\n",
       "      <td>08/22/2017 12:00:00 AM</td>\n",
       "      <td>Constructed</td>\n",
       "      <td>1250314</td>\n",
       "      <td>26.000000</td>\n",
       "      <td>2100</td>\n",
       "      <td>15.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>3044520924</td>\n",
       "      <td>3044520924</td>\n",
       "      <td>Other (Man</td>\n",
       "      <td>{C045C815-79DB-4644-AD9D-C34AC03D1AB4}</td>\n",
       "      <td>POLYGON ((-73.87172 40.65519, -73.87179 40.655...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>MULTIPOLYGON (((-73.86650099829305 40.74236058...</td>\n",
       "      <td>None</td>\n",
       "      <td>4540159</td>\n",
       "      <td>2010.0</td>\n",
       "      <td>08/22/2017 12:00:00 AM</td>\n",
       "      <td>Constructed</td>\n",
       "      <td>201366</td>\n",
       "      <td>28.000000</td>\n",
       "      <td>2100</td>\n",
       "      <td>37.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>4018780115</td>\n",
       "      <td>4018780115</td>\n",
       "      <td>Other (Man</td>\n",
       "      <td>{FDF673E7-FF92-4A7A-AF6D-C49D77343C47}</td>\n",
       "      <td>POLYGON ((-73.86650 40.74236, -73.86645 40.742...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>MULTIPOLYGON (((-73.87805078807256 40.71475698...</td>\n",
       "      <td>None</td>\n",
       "      <td>4540051</td>\n",
       "      <td>2010.0</td>\n",
       "      <td>08/22/2017 12:00:00 AM</td>\n",
       "      <td>Constructed</td>\n",
       "      <td>1171655</td>\n",
       "      <td>28.330225</td>\n",
       "      <td>2100</td>\n",
       "      <td>112.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>4030600139</td>\n",
       "      <td>4030600139</td>\n",
       "      <td>Photogramm</td>\n",
       "      <td>{788E5D72-46C1-443F-8BC9-6B97F329BFED}</td>\n",
       "      <td>POLYGON ((-73.87805 40.71476, -73.87787 40.714...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>MULTIPOLYGON (((-73.81520745135124 40.73053646...</td>\n",
       "      <td>None</td>\n",
       "      <td>4545453</td>\n",
       "      <td>2012.0</td>\n",
       "      <td>08/22/2017 12:00:00 AM</td>\n",
       "      <td>Constructed</td>\n",
       "      <td>1118502</td>\n",
       "      <td>16.640000</td>\n",
       "      <td>2100</td>\n",
       "      <td>74.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>4066560052</td>\n",
       "      <td>4066560052</td>\n",
       "      <td>Photogramm</td>\n",
       "      <td>{789A5A51-5B12-46DC-AE85-B06820A3225E}</td>\n",
       "      <td>POLYGON ((-73.81521 40.73054, -73.81546 40.730...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>MULTIPOLYGON (((-73.84769179857282 40.87911947...</td>\n",
       "      <td>None</td>\n",
       "      <td>2118998</td>\n",
       "      <td>2012.0</td>\n",
       "      <td>08/22/2017 12:00:00 AM</td>\n",
       "      <td>Constructed</td>\n",
       "      <td>1254551</td>\n",
       "      <td>33.000000</td>\n",
       "      <td>2100</td>\n",
       "      <td>154.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2047220003</td>\n",
       "      <td>2047220003</td>\n",
       "      <td>Other (Man</td>\n",
       "      <td>{BB58FD7B-CC22-4896-901D-F8BAFF4AC129}</td>\n",
       "      <td>POLYGON ((-73.84769 40.87912, -73.84784 40.879...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                            the_geom  NAME      BIN  \\\n",
       "0  MULTIPOLYGON (((-73.87172426474349 40.65519420...  None  3394834   \n",
       "1  MULTIPOLYGON (((-73.86650099829305 40.74236058...  None  4540159   \n",
       "2  MULTIPOLYGON (((-73.87805078807256 40.71475698...  None  4540051   \n",
       "3  MULTIPOLYGON (((-73.81520745135124 40.73053646...  None  4545453   \n",
       "4  MULTIPOLYGON (((-73.84769179857282 40.87911947...  None  2118998   \n",
       "\n",
       "   CNSTRCT_YR              LSTMODDATE   LSTSTATYPE  DOITT_ID  HEIGHTROOF  \\\n",
       "0      2011.0  08/22/2017 12:00:00 AM  Constructed   1250314   26.000000   \n",
       "1      2010.0  08/22/2017 12:00:00 AM  Constructed    201366   28.000000   \n",
       "2      2010.0  08/22/2017 12:00:00 AM  Constructed   1171655   28.330225   \n",
       "3      2012.0  08/22/2017 12:00:00 AM  Constructed   1118502   16.640000   \n",
       "4      2012.0  08/22/2017 12:00:00 AM  Constructed   1254551   33.000000   \n",
       "\n",
       "   FEAT_CODE  GROUNDELEV  SHAPE_AREA  SHAPE_LEN    BASE_BBL  MPLUTO_BBL  \\\n",
       "0       2100        15.0           0          0  3044520924  3044520924   \n",
       "1       2100        37.0           0          0  4018780115  4018780115   \n",
       "2       2100       112.0           0          0  4030600139  4030600139   \n",
       "3       2100        74.0           0          0  4066560052  4066560052   \n",
       "4       2100       154.0           0          0  2047220003  2047220003   \n",
       "\n",
       "   GEOMSOURCE                                GLOBALID  \\\n",
       "0  Other (Man  {C045C815-79DB-4644-AD9D-C34AC03D1AB4}   \n",
       "1  Other (Man  {FDF673E7-FF92-4A7A-AF6D-C49D77343C47}   \n",
       "2  Photogramm  {788E5D72-46C1-443F-8BC9-6B97F329BFED}   \n",
       "3  Photogramm  {789A5A51-5B12-46DC-AE85-B06820A3225E}   \n",
       "4  Other (Man  {BB58FD7B-CC22-4896-901D-F8BAFF4AC129}   \n",
       "\n",
       "                                            geometry  \n",
       "0  POLYGON ((-73.87172 40.65519, -73.87179 40.655...  \n",
       "1  POLYGON ((-73.86650 40.74236, -73.86645 40.742...  \n",
       "2  POLYGON ((-73.87805 40.71476, -73.87787 40.714...  \n",
       "3  POLYGON ((-73.81521 40.73054, -73.81546 40.730...  \n",
       "4  POLYGON ((-73.84769 40.87912, -73.84784 40.879...  "
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# read data as a geodataframe\n",
    "path = 'data/shapefile/sample-data.shp'\n",
    "gdf = gpd.read_file(path)\n",
    "\n",
    "# previewing first five rows in data\n",
    "gdf.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "This dataset has 16,308 rows and 17 columns.\n"
     ]
    }
   ],
   "source": [
    "rows, columns = gdf.shape\n",
    "print('This dataset has {:,} rows and {:,} columns.'.format(rows, columns))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1.5 Reading in a GeoJSON file"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>the_geom</th>\n",
       "      <th>NAME</th>\n",
       "      <th>BIN</th>\n",
       "      <th>CNSTRCT_YR</th>\n",
       "      <th>LSTMODDATE</th>\n",
       "      <th>LSTSTATYPE</th>\n",
       "      <th>DOITT_ID</th>\n",
       "      <th>HEIGHTROOF</th>\n",
       "      <th>FEAT_CODE</th>\n",
       "      <th>GROUNDELEV</th>\n",
       "      <th>SHAPE_AREA</th>\n",
       "      <th>SHAPE_LEN</th>\n",
       "      <th>BASE_BBL</th>\n",
       "      <th>MPLUTO_BBL</th>\n",
       "      <th>GEOMSOURCE</th>\n",
       "      <th>GLOBALID</th>\n",
       "      <th>geometry</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>MULTIPOLYGON (((-73.87172426474349 40.65519420...</td>\n",
       "      <td>None</td>\n",
       "      <td>3394834</td>\n",
       "      <td>2011.0</td>\n",
       "      <td>08/22/2017 12:00:00 AM</td>\n",
       "      <td>Constructed</td>\n",
       "      <td>1250314</td>\n",
       "      <td>26.000000</td>\n",
       "      <td>2100</td>\n",
       "      <td>15.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>3044520924</td>\n",
       "      <td>3044520924</td>\n",
       "      <td>Other (Man</td>\n",
       "      <td>{C045C815-79DB-4644-AD9D-C34AC03D1AB4}</td>\n",
       "      <td>MULTIPOLYGON (((-73.87172 40.65519, -73.87179 ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>MULTIPOLYGON (((-73.86650099829305 40.74236058...</td>\n",
       "      <td>None</td>\n",
       "      <td>4540159</td>\n",
       "      <td>2010.0</td>\n",
       "      <td>08/22/2017 12:00:00 AM</td>\n",
       "      <td>Constructed</td>\n",
       "      <td>201366</td>\n",
       "      <td>28.000000</td>\n",
       "      <td>2100</td>\n",
       "      <td>37.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>4018780115</td>\n",
       "      <td>4018780115</td>\n",
       "      <td>Other (Man</td>\n",
       "      <td>{FDF673E7-FF92-4A7A-AF6D-C49D77343C47}</td>\n",
       "      <td>MULTIPOLYGON (((-73.86650 40.74236, -73.86645 ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>MULTIPOLYGON (((-73.87805078807256 40.71475698...</td>\n",
       "      <td>None</td>\n",
       "      <td>4540051</td>\n",
       "      <td>2010.0</td>\n",
       "      <td>08/22/2017 12:00:00 AM</td>\n",
       "      <td>Constructed</td>\n",
       "      <td>1171655</td>\n",
       "      <td>28.330225</td>\n",
       "      <td>2100</td>\n",
       "      <td>112.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>4030600139</td>\n",
       "      <td>4030600139</td>\n",
       "      <td>Photogramm</td>\n",
       "      <td>{788E5D72-46C1-443F-8BC9-6B97F329BFED}</td>\n",
       "      <td>MULTIPOLYGON (((-73.87805 40.71476, -73.87787 ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>MULTIPOLYGON (((-73.81520745135124 40.73053646...</td>\n",
       "      <td>None</td>\n",
       "      <td>4545453</td>\n",
       "      <td>2012.0</td>\n",
       "      <td>08/22/2017 12:00:00 AM</td>\n",
       "      <td>Constructed</td>\n",
       "      <td>1118502</td>\n",
       "      <td>16.640000</td>\n",
       "      <td>2100</td>\n",
       "      <td>74.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>4066560052</td>\n",
       "      <td>4066560052</td>\n",
       "      <td>Photogramm</td>\n",
       "      <td>{789A5A51-5B12-46DC-AE85-B06820A3225E}</td>\n",
       "      <td>MULTIPOLYGON (((-73.81521 40.73054, -73.81546 ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>MULTIPOLYGON (((-73.84769179857282 40.87911947...</td>\n",
       "      <td>None</td>\n",
       "      <td>2118998</td>\n",
       "      <td>2012.0</td>\n",
       "      <td>08/22/2017 12:00:00 AM</td>\n",
       "      <td>Constructed</td>\n",
       "      <td>1254551</td>\n",
       "      <td>33.000000</td>\n",
       "      <td>2100</td>\n",
       "      <td>154.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2047220003</td>\n",
       "      <td>2047220003</td>\n",
       "      <td>Other (Man</td>\n",
       "      <td>{BB58FD7B-CC22-4896-901D-F8BAFF4AC129}</td>\n",
       "      <td>MULTIPOLYGON (((-73.84769 40.87912, -73.84784 ...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                            the_geom  NAME      BIN  \\\n",
       "0  MULTIPOLYGON (((-73.87172426474349 40.65519420...  None  3394834   \n",
       "1  MULTIPOLYGON (((-73.86650099829305 40.74236058...  None  4540159   \n",
       "2  MULTIPOLYGON (((-73.87805078807256 40.71475698...  None  4540051   \n",
       "3  MULTIPOLYGON (((-73.81520745135124 40.73053646...  None  4545453   \n",
       "4  MULTIPOLYGON (((-73.84769179857282 40.87911947...  None  2118998   \n",
       "\n",
       "   CNSTRCT_YR              LSTMODDATE   LSTSTATYPE  DOITT_ID  HEIGHTROOF  \\\n",
       "0      2011.0  08/22/2017 12:00:00 AM  Constructed   1250314   26.000000   \n",
       "1      2010.0  08/22/2017 12:00:00 AM  Constructed    201366   28.000000   \n",
       "2      2010.0  08/22/2017 12:00:00 AM  Constructed   1171655   28.330225   \n",
       "3      2012.0  08/22/2017 12:00:00 AM  Constructed   1118502   16.640000   \n",
       "4      2012.0  08/22/2017 12:00:00 AM  Constructed   1254551   33.000000   \n",
       "\n",
       "   FEAT_CODE  GROUNDELEV  SHAPE_AREA  SHAPE_LEN    BASE_BBL  MPLUTO_BBL  \\\n",
       "0       2100        15.0           0          0  3044520924  3044520924   \n",
       "1       2100        37.0           0          0  4018780115  4018780115   \n",
       "2       2100       112.0           0          0  4030600139  4030600139   \n",
       "3       2100        74.0           0          0  4066560052  4066560052   \n",
       "4       2100       154.0           0          0  2047220003  2047220003   \n",
       "\n",
       "   GEOMSOURCE                                GLOBALID  \\\n",
       "0  Other (Man  {C045C815-79DB-4644-AD9D-C34AC03D1AB4}   \n",
       "1  Other (Man  {FDF673E7-FF92-4A7A-AF6D-C49D77343C47}   \n",
       "2  Photogramm  {788E5D72-46C1-443F-8BC9-6B97F329BFED}   \n",
       "3  Photogramm  {789A5A51-5B12-46DC-AE85-B06820A3225E}   \n",
       "4  Other (Man  {BB58FD7B-CC22-4896-901D-F8BAFF4AC129}   \n",
       "\n",
       "                                            geometry  \n",
       "0  MULTIPOLYGON (((-73.87172 40.65519, -73.87179 ...  \n",
       "1  MULTIPOLYGON (((-73.86650 40.74236, -73.86645 ...  \n",
       "2  MULTIPOLYGON (((-73.87805 40.71476, -73.87787 ...  \n",
       "3  MULTIPOLYGON (((-73.81521 40.73054, -73.81546 ...  \n",
       "4  MULTIPOLYGON (((-73.84769 40.87912, -73.84784 ...  "
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "path = 'data/sample-data.geojson'\n",
    "gdf = gpd.read_file(path)\n",
    "\n",
    "# previewing first five rows in data\n",
    "gdf.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "This dataset has 16,308 rows and 17 columns.\n"
     ]
    }
   ],
   "source": [
    "rows, columns = gdf.shape\n",
    "print('This dataset has {:,} rows and {:,} columns.'.format(rows, columns))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 2. Reading in Data from a URL"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Another popular dataset is NYC's PLUTO dataset. We will use this one because it comes in a ZIP file.\n",
    "\n",
    "- **Description**: Extensive land use and geographic data at the tax lot level in comma–separated values (CSV) file format. The PLUTO files contain more than seventy fields derived from data maintained by city agencies.\n",
    "- **Dataset Link**: https://www1.nyc.gov/site/planning/data-maps/open-data/dwn-pluto-mappluto.page\n",
    "- **Data Dictionary**: https://www1.nyc.gov/assets/planning/download/pdf/data-maps/open-data/pluto_datadictionary.pdf?v=20v1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![building_footprints_csv](images/pluto-csv.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.1 Unzipping and reading in data as CSV to local folder\n",
    "\n",
    "We will retrieve, unzip and read in data in our downloads folder.  \n",
    "**Note: right-click the Download icon and copy the link address as the url.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "('/Users/geribauer/Downloads/PLUTO.gz',\n",
       " <http.client.HTTPMessage at 0x17606a280>)"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# download url of dataset\n",
    "url = 'https://www1.nyc.gov/assets/planning/download/zip/data-maps/open-data/nyc_pluto_20v1_csv.zip'\n",
    "\n",
    "# a path to our downloads folder \n",
    "downloads_path = str(Path.home() / \"Downloads\")\n",
    "\n",
    "# a path to our file from our downloads path\n",
    "fullfilename = os.path.join(downloads_path, 'PLUTO.gz')\n",
    "\n",
    "# retrieving data \n",
    "urllib.request.urlretrieve(url, fullfilename)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['pluto_20v1.csv', 'PLUTODD20v1.pdf', 'PlutoReadme20v1.pdf']\n"
     ]
    }
   ],
   "source": [
    "# open zipfile and saving items in our zipfolder\n",
    "items = zipfile.ZipFile(fullfilename)\n",
    "\n",
    "# available files in the container\n",
    "print(items.namelist())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "# opening zipfile using 'with' keyword in read mode\n",
    "with zipfile.ZipFile(fullfilename, 'r') as file:\n",
    "    \n",
    "    # extract all files inside the zip file\n",
    "    file.extractall(downloads_path)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>borough</th>\n",
       "      <th>block</th>\n",
       "      <th>lot</th>\n",
       "      <th>cd</th>\n",
       "      <th>ct2010</th>\n",
       "      <th>cb2010</th>\n",
       "      <th>schooldist</th>\n",
       "      <th>council</th>\n",
       "      <th>zipcode</th>\n",
       "      <th>firecomp</th>\n",
       "      <th>...</th>\n",
       "      <th>dcasdate</th>\n",
       "      <th>zoningdate</th>\n",
       "      <th>landmkdate</th>\n",
       "      <th>basempdate</th>\n",
       "      <th>masdate</th>\n",
       "      <th>polidate</th>\n",
       "      <th>edesigdate</th>\n",
       "      <th>geom</th>\n",
       "      <th>dcpedited</th>\n",
       "      <th>notes</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>BK</td>\n",
       "      <td>834</td>\n",
       "      <td>46</td>\n",
       "      <td>307.0</td>\n",
       "      <td>106.0</td>\n",
       "      <td>2001.0</td>\n",
       "      <td>20.0</td>\n",
       "      <td>38.0</td>\n",
       "      <td>11220.0</td>\n",
       "      <td>L114</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0106000020E61000000100000001030000000100000005...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>QN</td>\n",
       "      <td>4042</td>\n",
       "      <td>106</td>\n",
       "      <td>407.0</td>\n",
       "      <td>929.0</td>\n",
       "      <td>3000.0</td>\n",
       "      <td>25.0</td>\n",
       "      <td>19.0</td>\n",
       "      <td>11356.0</td>\n",
       "      <td>E297</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0106000020E61000000100000001030000000100000007...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>BK</td>\n",
       "      <td>4679</td>\n",
       "      <td>17</td>\n",
       "      <td>317.0</td>\n",
       "      <td>866.0</td>\n",
       "      <td>3002.0</td>\n",
       "      <td>18.0</td>\n",
       "      <td>41.0</td>\n",
       "      <td>11203.0</td>\n",
       "      <td>L174</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0106000020E61000000100000001030000000100000006...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>BK</td>\n",
       "      <td>7831</td>\n",
       "      <td>6</td>\n",
       "      <td>318.0</td>\n",
       "      <td>676.0</td>\n",
       "      <td>1002.0</td>\n",
       "      <td>22.0</td>\n",
       "      <td>46.0</td>\n",
       "      <td>11234.0</td>\n",
       "      <td>L159</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0106000020E61000000100000001030000000100000005...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>BK</td>\n",
       "      <td>7831</td>\n",
       "      <td>7</td>\n",
       "      <td>318.0</td>\n",
       "      <td>676.0</td>\n",
       "      <td>1002.0</td>\n",
       "      <td>22.0</td>\n",
       "      <td>46.0</td>\n",
       "      <td>11234.0</td>\n",
       "      <td>L159</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0106000020E61000000100000001030000000100000005...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 99 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "  borough  block  lot     cd  ct2010  cb2010  schooldist  council  zipcode  \\\n",
       "0      BK    834   46  307.0   106.0  2001.0        20.0     38.0  11220.0   \n",
       "1      QN   4042  106  407.0   929.0  3000.0        25.0     19.0  11356.0   \n",
       "2      BK   4679   17  317.0   866.0  3002.0        18.0     41.0  11203.0   \n",
       "3      BK   7831    6  318.0   676.0  1002.0        22.0     46.0  11234.0   \n",
       "4      BK   7831    7  318.0   676.0  1002.0        22.0     46.0  11234.0   \n",
       "\n",
       "  firecomp  ...  dcasdate  zoningdate  landmkdate basempdate masdate polidate  \\\n",
       "0     L114  ...       NaN         NaN         NaN        NaN     NaN      NaN   \n",
       "1     E297  ...       NaN         NaN         NaN        NaN     NaN      NaN   \n",
       "2     L174  ...       NaN         NaN         NaN        NaN     NaN      NaN   \n",
       "3     L159  ...       NaN         NaN         NaN        NaN     NaN      NaN   \n",
       "4     L159  ...       NaN         NaN         NaN        NaN     NaN      NaN   \n",
       "\n",
       "  edesigdate                                               geom dcpedited  \\\n",
       "0        NaN  0106000020E61000000100000001030000000100000005...       NaN   \n",
       "1        NaN  0106000020E61000000100000001030000000100000007...       NaN   \n",
       "2        NaN  0106000020E61000000100000001030000000100000006...       NaN   \n",
       "3        NaN  0106000020E61000000100000001030000000100000005...       NaN   \n",
       "4        NaN  0106000020E61000000100000001030000000100000005...       NaN   \n",
       "\n",
       "  notes  \n",
       "0   NaN  \n",
       "1   NaN  \n",
       "2   NaN  \n",
       "3   NaN  \n",
       "4   NaN  \n",
       "\n",
       "[5 rows x 99 columns]"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# read our data into a dataframe from our downloads path\n",
    "file = 'pluto_20v1.csv'\n",
    "path = os.path.join(downloads_path, file)\n",
    "\n",
    "pluto_data = pd.read_csv(path, low_memory=False)\n",
    "\n",
    "pluto_data.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "This dataset has 859,172 rows and 99 columns.\n"
     ]
    }
   ],
   "source": [
    "rows, columns = pluto_data.shape\n",
    "print('This dataset has {:,} rows and {:,} columns.'.format(rows, columns))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.2 Unzipping and reading in data as CSV from local folder"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "# note: ive already placed data into this zip file\n",
    "file_name = 'data/sample-buildings.zip'\n",
    "\n",
    "# opening zip using 'with' keyword in read mode\n",
    "with zipfile.ZipFile(file_name, 'r') as file:\n",
    "    \n",
    "    # extracing all items in our zipfile\n",
    "    file.extractall('data/unzipped-data/')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "sample-data.csv\r\n"
     ]
    }
   ],
   "source": [
    "# list files in this file path\n",
    "%ls data/unzipped-data/"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>the_geom</th>\n",
       "      <th>NAME</th>\n",
       "      <th>BIN</th>\n",
       "      <th>CNSTRCT_YR</th>\n",
       "      <th>LSTMODDATE</th>\n",
       "      <th>LSTSTATYPE</th>\n",
       "      <th>DOITT_ID</th>\n",
       "      <th>HEIGHTROOF</th>\n",
       "      <th>FEAT_CODE</th>\n",
       "      <th>GROUNDELEV</th>\n",
       "      <th>SHAPE_AREA</th>\n",
       "      <th>SHAPE_LEN</th>\n",
       "      <th>BASE_BBL</th>\n",
       "      <th>MPLUTO_BBL</th>\n",
       "      <th>GEOMSOURCE</th>\n",
       "      <th>GLOBALID</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>MULTIPOLYGON (((-73.87172426474349 40.65519420...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>3394834</td>\n",
       "      <td>2011.0</td>\n",
       "      <td>08/22/2017 12:00:00 AM</td>\n",
       "      <td>Constructed</td>\n",
       "      <td>1250314</td>\n",
       "      <td>26.000000</td>\n",
       "      <td>2100</td>\n",
       "      <td>15.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>3044520924</td>\n",
       "      <td>3044520924</td>\n",
       "      <td>Other (Man</td>\n",
       "      <td>{C045C815-79DB-4644-AD9D-C34AC03D1AB4}</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>MULTIPOLYGON (((-73.86650099829305 40.74236058...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>4540159</td>\n",
       "      <td>2010.0</td>\n",
       "      <td>08/22/2017 12:00:00 AM</td>\n",
       "      <td>Constructed</td>\n",
       "      <td>201366</td>\n",
       "      <td>28.000000</td>\n",
       "      <td>2100</td>\n",
       "      <td>37.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>4018780115</td>\n",
       "      <td>4018780115</td>\n",
       "      <td>Other (Man</td>\n",
       "      <td>{FDF673E7-FF92-4A7A-AF6D-C49D77343C47}</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>MULTIPOLYGON (((-73.87805078807256 40.71475698...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>4540051</td>\n",
       "      <td>2010.0</td>\n",
       "      <td>08/22/2017 12:00:00 AM</td>\n",
       "      <td>Constructed</td>\n",
       "      <td>1171655</td>\n",
       "      <td>28.330225</td>\n",
       "      <td>2100</td>\n",
       "      <td>112.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>4030600139</td>\n",
       "      <td>4030600139</td>\n",
       "      <td>Photogramm</td>\n",
       "      <td>{788E5D72-46C1-443F-8BC9-6B97F329BFED}</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>MULTIPOLYGON (((-73.81520745135124 40.73053646...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>4545453</td>\n",
       "      <td>2012.0</td>\n",
       "      <td>08/22/2017 12:00:00 AM</td>\n",
       "      <td>Constructed</td>\n",
       "      <td>1118502</td>\n",
       "      <td>16.640000</td>\n",
       "      <td>2100</td>\n",
       "      <td>74.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>4066560052</td>\n",
       "      <td>4066560052</td>\n",
       "      <td>Photogramm</td>\n",
       "      <td>{789A5A51-5B12-46DC-AE85-B06820A3225E}</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>MULTIPOLYGON (((-73.84769179857282 40.87911947...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2118998</td>\n",
       "      <td>2012.0</td>\n",
       "      <td>08/22/2017 12:00:00 AM</td>\n",
       "      <td>Constructed</td>\n",
       "      <td>1254551</td>\n",
       "      <td>33.000000</td>\n",
       "      <td>2100</td>\n",
       "      <td>154.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2047220003</td>\n",
       "      <td>2047220003</td>\n",
       "      <td>Other (Man</td>\n",
       "      <td>{BB58FD7B-CC22-4896-901D-F8BAFF4AC129}</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                            the_geom NAME      BIN  \\\n",
       "0  MULTIPOLYGON (((-73.87172426474349 40.65519420...  NaN  3394834   \n",
       "1  MULTIPOLYGON (((-73.86650099829305 40.74236058...  NaN  4540159   \n",
       "2  MULTIPOLYGON (((-73.87805078807256 40.71475698...  NaN  4540051   \n",
       "3  MULTIPOLYGON (((-73.81520745135124 40.73053646...  NaN  4545453   \n",
       "4  MULTIPOLYGON (((-73.84769179857282 40.87911947...  NaN  2118998   \n",
       "\n",
       "   CNSTRCT_YR              LSTMODDATE   LSTSTATYPE  DOITT_ID  HEIGHTROOF  \\\n",
       "0      2011.0  08/22/2017 12:00:00 AM  Constructed   1250314   26.000000   \n",
       "1      2010.0  08/22/2017 12:00:00 AM  Constructed    201366   28.000000   \n",
       "2      2010.0  08/22/2017 12:00:00 AM  Constructed   1171655   28.330225   \n",
       "3      2012.0  08/22/2017 12:00:00 AM  Constructed   1118502   16.640000   \n",
       "4      2012.0  08/22/2017 12:00:00 AM  Constructed   1254551   33.000000   \n",
       "\n",
       "   FEAT_CODE  GROUNDELEV  SHAPE_AREA  SHAPE_LEN    BASE_BBL  MPLUTO_BBL  \\\n",
       "0       2100        15.0           0          0  3044520924  3044520924   \n",
       "1       2100        37.0           0          0  4018780115  4018780115   \n",
       "2       2100       112.0           0          0  4030600139  4030600139   \n",
       "3       2100        74.0           0          0  4066560052  4066560052   \n",
       "4       2100       154.0           0          0  2047220003  2047220003   \n",
       "\n",
       "   GEOMSOURCE                                GLOBALID  \n",
       "0  Other (Man  {C045C815-79DB-4644-AD9D-C34AC03D1AB4}  \n",
       "1  Other (Man  {FDF673E7-FF92-4A7A-AF6D-C49D77343C47}  \n",
       "2  Photogramm  {788E5D72-46C1-443F-8BC9-6B97F329BFED}  \n",
       "3  Photogramm  {789A5A51-5B12-46DC-AE85-B06820A3225E}  \n",
       "4  Other (Man  {BB58FD7B-CC22-4896-901D-F8BAFF4AC129}  "
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# read data as a dataframe\n",
    "sample_buidlings = pd.read_csv('data/unzipped-data/sample-data.csv')\n",
    "\n",
    "# previewing first five rows of data\n",
    "sample_buidlings.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "This dataset has 16,308 rows and 16 columns.\n"
     ]
    }
   ],
   "source": [
    "rows, columns = sample_buidlings.shape\n",
    "print('This dataset has {:,} rows and {:,} columns.'.format(rows, columns))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.3 Unzipping and reading in data as a CSV in memory"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "File in zip: pluto_20v1.csv\n",
      "File in zip: PLUTODD20v1.pdf\n",
      "File in zip: PlutoReadme20v1.pdf\n"
     ]
    }
   ],
   "source": [
    "# our download link\n",
    "url = 'https://www1.nyc.gov/assets/planning/download/zip/data-maps/open-data/nyc_pluto_20v1_csv.zip'\n",
    "\n",
    "# reading in our zipfile data in-memory\n",
    "content = requests.get(url)\n",
    "zf = ZipFile(BytesIO(content.content))\n",
    "\n",
    "# printing files in our zipfile\n",
    "for item in zf.namelist():\n",
    "    print(\"File in zip: {}\".format(item))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>borough</th>\n",
       "      <th>block</th>\n",
       "      <th>lot</th>\n",
       "      <th>cd</th>\n",
       "      <th>ct2010</th>\n",
       "      <th>cb2010</th>\n",
       "      <th>schooldist</th>\n",
       "      <th>council</th>\n",
       "      <th>zipcode</th>\n",
       "      <th>firecomp</th>\n",
       "      <th>...</th>\n",
       "      <th>dcasdate</th>\n",
       "      <th>zoningdate</th>\n",
       "      <th>landmkdate</th>\n",
       "      <th>basempdate</th>\n",
       "      <th>masdate</th>\n",
       "      <th>polidate</th>\n",
       "      <th>edesigdate</th>\n",
       "      <th>geom</th>\n",
       "      <th>dcpedited</th>\n",
       "      <th>notes</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>BK</td>\n",
       "      <td>834</td>\n",
       "      <td>46</td>\n",
       "      <td>307.0</td>\n",
       "      <td>106.0</td>\n",
       "      <td>2001.0</td>\n",
       "      <td>20.0</td>\n",
       "      <td>38.0</td>\n",
       "      <td>11220.0</td>\n",
       "      <td>L114</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0106000020E61000000100000001030000000100000005...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>QN</td>\n",
       "      <td>4042</td>\n",
       "      <td>106</td>\n",
       "      <td>407.0</td>\n",
       "      <td>929.0</td>\n",
       "      <td>3000.0</td>\n",
       "      <td>25.0</td>\n",
       "      <td>19.0</td>\n",
       "      <td>11356.0</td>\n",
       "      <td>E297</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0106000020E61000000100000001030000000100000007...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>BK</td>\n",
       "      <td>4679</td>\n",
       "      <td>17</td>\n",
       "      <td>317.0</td>\n",
       "      <td>866.0</td>\n",
       "      <td>3002.0</td>\n",
       "      <td>18.0</td>\n",
       "      <td>41.0</td>\n",
       "      <td>11203.0</td>\n",
       "      <td>L174</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0106000020E61000000100000001030000000100000006...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>BK</td>\n",
       "      <td>7831</td>\n",
       "      <td>6</td>\n",
       "      <td>318.0</td>\n",
       "      <td>676.0</td>\n",
       "      <td>1002.0</td>\n",
       "      <td>22.0</td>\n",
       "      <td>46.0</td>\n",
       "      <td>11234.0</td>\n",
       "      <td>L159</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0106000020E61000000100000001030000000100000005...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>BK</td>\n",
       "      <td>7831</td>\n",
       "      <td>7</td>\n",
       "      <td>318.0</td>\n",
       "      <td>676.0</td>\n",
       "      <td>1002.0</td>\n",
       "      <td>22.0</td>\n",
       "      <td>46.0</td>\n",
       "      <td>11234.0</td>\n",
       "      <td>L159</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0106000020E61000000100000001030000000100000005...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 99 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "  borough  block  lot     cd  ct2010  cb2010  schooldist  council  zipcode  \\\n",
       "0      BK    834   46  307.0   106.0  2001.0        20.0     38.0  11220.0   \n",
       "1      QN   4042  106  407.0   929.0  3000.0        25.0     19.0  11356.0   \n",
       "2      BK   4679   17  317.0   866.0  3002.0        18.0     41.0  11203.0   \n",
       "3      BK   7831    6  318.0   676.0  1002.0        22.0     46.0  11234.0   \n",
       "4      BK   7831    7  318.0   676.0  1002.0        22.0     46.0  11234.0   \n",
       "\n",
       "  firecomp  ...  dcasdate  zoningdate  landmkdate basempdate masdate polidate  \\\n",
       "0     L114  ...       NaN         NaN         NaN        NaN     NaN      NaN   \n",
       "1     E297  ...       NaN         NaN         NaN        NaN     NaN      NaN   \n",
       "2     L174  ...       NaN         NaN         NaN        NaN     NaN      NaN   \n",
       "3     L159  ...       NaN         NaN         NaN        NaN     NaN      NaN   \n",
       "4     L159  ...       NaN         NaN         NaN        NaN     NaN      NaN   \n",
       "\n",
       "  edesigdate                                               geom dcpedited  \\\n",
       "0        NaN  0106000020E61000000100000001030000000100000005...       NaN   \n",
       "1        NaN  0106000020E61000000100000001030000000100000007...       NaN   \n",
       "2        NaN  0106000020E61000000100000001030000000100000006...       NaN   \n",
       "3        NaN  0106000020E61000000100000001030000000100000005...       NaN   \n",
       "4        NaN  0106000020E61000000100000001030000000100000005...       NaN   \n",
       "\n",
       "  notes  \n",
       "0   NaN  \n",
       "1   NaN  \n",
       "2   NaN  \n",
       "3   NaN  \n",
       "4   NaN  \n",
       "\n",
       "[5 rows x 99 columns]"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# read our csv data into a dataframe from our zipfile\n",
    "file = 'pluto_20v1.csv'\n",
    "pluto_data = pd.read_csv(zf.open(file), low_memory=False)\n",
    "\n",
    "# previewing the first five rows of data\n",
    "pluto_data.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "This dataset has 859,172 rows and 99 columns.\n"
     ]
    }
   ],
   "source": [
    "rows, columns = pluto_data.shape\n",
    "print('This dataset has {:,} rows and {:,} columns.'.format(rows, columns))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 3. Reading in data from NYC Open Data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3.1 Reading in data as CSV in static form"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![building_footprints_csv](images/building-footprints-csv.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>the_geom</th>\n",
       "      <th>NAME</th>\n",
       "      <th>BIN</th>\n",
       "      <th>CNSTRCT_YR</th>\n",
       "      <th>LSTMODDATE</th>\n",
       "      <th>LSTSTATYPE</th>\n",
       "      <th>DOITT_ID</th>\n",
       "      <th>HEIGHTROOF</th>\n",
       "      <th>FEAT_CODE</th>\n",
       "      <th>GROUNDELEV</th>\n",
       "      <th>SHAPE_AREA</th>\n",
       "      <th>SHAPE_LEN</th>\n",
       "      <th>BASE_BBL</th>\n",
       "      <th>MPLUTO_BBL</th>\n",
       "      <th>GEOMSOURCE</th>\n",
       "      <th>GLOBALID</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>MULTIPOLYGON (((-73.96664570466969 40.62599676...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>3170958</td>\n",
       "      <td>1925.0</td>\n",
       "      <td>08/22/2017 12:00:00 AM</td>\n",
       "      <td>Constructed</td>\n",
       "      <td>96807</td>\n",
       "      <td>29.749853</td>\n",
       "      <td>2100</td>\n",
       "      <td>40.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>3065220021</td>\n",
       "      <td>3065220021</td>\n",
       "      <td>Photogramm</td>\n",
       "      <td>{31298F86-3088-4F53-B3DB-71A9EFA6FA1F}</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>MULTIPOLYGON (((-74.16790202462265 40.63936048...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>5028452</td>\n",
       "      <td>1965.0</td>\n",
       "      <td>08/22/2017 12:00:00 AM</td>\n",
       "      <td>Constructed</td>\n",
       "      <td>326368</td>\n",
       "      <td>22.630000</td>\n",
       "      <td>2100</td>\n",
       "      <td>39.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>5012640036</td>\n",
       "      <td>5012640036</td>\n",
       "      <td>Photogramm</td>\n",
       "      <td>{F5F8CDA5-69E2-46F8-8F69-BA95C025B520}</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>MULTIPOLYGON (((-74.19510813278613 40.55610681...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>5078368</td>\n",
       "      <td>1970.0</td>\n",
       "      <td>08/22/2017 12:00:00 AM</td>\n",
       "      <td>Constructed</td>\n",
       "      <td>746627</td>\n",
       "      <td>35.760000</td>\n",
       "      <td>2100</td>\n",
       "      <td>51.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>5060190091</td>\n",
       "      <td>5060190091</td>\n",
       "      <td>Photogramm</td>\n",
       "      <td>{9F644794-F72C-4582-9E5E-B337E2B97068}</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>MULTIPOLYGON (((-73.96113466505085 40.57743931...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>3245111</td>\n",
       "      <td>1928.0</td>\n",
       "      <td>08/22/2017 12:00:00 AM</td>\n",
       "      <td>Constructed</td>\n",
       "      <td>786626</td>\n",
       "      <td>37.500000</td>\n",
       "      <td>2100</td>\n",
       "      <td>6.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>3086910048</td>\n",
       "      <td>3086910048</td>\n",
       "      <td>Photogramm</td>\n",
       "      <td>{F916B22D-E25B-44AE-9FA9-2A51191B9CDF}</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>MULTIPOLYGON (((-73.75421559146166 40.75591276...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>4161096</td>\n",
       "      <td>1950.0</td>\n",
       "      <td>08/22/2017 12:00:00 AM</td>\n",
       "      <td>Constructed</td>\n",
       "      <td>746409</td>\n",
       "      <td>18.015113</td>\n",
       "      <td>2100</td>\n",
       "      <td>93.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>4075020005</td>\n",
       "      <td>4075020005</td>\n",
       "      <td>Photogramm</td>\n",
       "      <td>{525F2C24-616B-4F29-98A3-8FEA5D4B1A7D}</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                            the_geom NAME      BIN  \\\n",
       "0  MULTIPOLYGON (((-73.96664570466969 40.62599676...  NaN  3170958   \n",
       "1  MULTIPOLYGON (((-74.16790202462265 40.63936048...  NaN  5028452   \n",
       "2  MULTIPOLYGON (((-74.19510813278613 40.55610681...  NaN  5078368   \n",
       "3  MULTIPOLYGON (((-73.96113466505085 40.57743931...  NaN  3245111   \n",
       "4  MULTIPOLYGON (((-73.75421559146166 40.75591276...  NaN  4161096   \n",
       "\n",
       "   CNSTRCT_YR              LSTMODDATE   LSTSTATYPE  DOITT_ID  HEIGHTROOF  \\\n",
       "0      1925.0  08/22/2017 12:00:00 AM  Constructed     96807   29.749853   \n",
       "1      1965.0  08/22/2017 12:00:00 AM  Constructed    326368   22.630000   \n",
       "2      1970.0  08/22/2017 12:00:00 AM  Constructed    746627   35.760000   \n",
       "3      1928.0  08/22/2017 12:00:00 AM  Constructed    786626   37.500000   \n",
       "4      1950.0  08/22/2017 12:00:00 AM  Constructed    746409   18.015113   \n",
       "\n",
       "   FEAT_CODE  GROUNDELEV  SHAPE_AREA  SHAPE_LEN    BASE_BBL  MPLUTO_BBL  \\\n",
       "0       2100        40.0           0          0  3065220021  3065220021   \n",
       "1       2100        39.0           0          0  5012640036  5012640036   \n",
       "2       2100        51.0           0          0  5060190091  5060190091   \n",
       "3       2100         6.0           0          0  3086910048  3086910048   \n",
       "4       2100        93.0           0          0  4075020005  4075020005   \n",
       "\n",
       "   GEOMSOURCE                                GLOBALID  \n",
       "0  Photogramm  {31298F86-3088-4F53-B3DB-71A9EFA6FA1F}  \n",
       "1  Photogramm  {F5F8CDA5-69E2-46F8-8F69-BA95C025B520}  \n",
       "2  Photogramm  {9F644794-F72C-4582-9E5E-B337E2B97068}  \n",
       "3  Photogramm  {F916B22D-E25B-44AE-9FA9-2A51191B9CDF}  \n",
       "4  Photogramm  {525F2C24-616B-4F29-98A3-8FEA5D4B1A7D}  "
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# reading in data as a url from NYC Open Data\n",
    "url = 'https://data.cityofnewyork.us/api/views/qb5r-6dgf/rows.csv?accessType=DOWNLOAD'\n",
    "\n",
    "# saving data as a pandas dataframe named 'df_csv'\n",
    "df_csv = pd.read_csv(url)\n",
    "\n",
    "# previewing the first five rows \n",
    "df_csv.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "This dataset has 1,083,339 rows and 16 columns.\n"
     ]
    }
   ],
   "source": [
    "rows, columns = df_csv.shape\n",
    "print('This dataset has {:,} rows and {:,} columns.'.format(rows, columns))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3.2 Reading in data as JSON in static form\n",
    "![building_footprints_csv](images/building-footprints-json.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "type: <class 'dict'>\n",
      "keys: dict_keys(['meta', 'data'])\n"
     ]
    }
   ],
   "source": [
    "url = 'https://data.cityofnewyork.us/api/views/qb5r-6dgf/rows.json?accessType=DOWNLOAD'\n",
    "\n",
    "# loads a json object as a python object\n",
    "with urllib.request.urlopen(url) as url:\n",
    "    data = json.loads(url.read().decode())\n",
    "    \n",
    "# identifying type of python object\n",
    "print('type: {}'.format(type(data)))\n",
    "\n",
    "# reviewing the dictionairy's keys\n",
    "print('keys: {}'.format(data.keys()))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['row-2mv5.r2xm.f5nt',\n",
       " '00000000-0000-0000-B60C-C89FFFC2D248',\n",
       " 0,\n",
       " 1713819867,\n",
       " None,\n",
       " 1713819867,\n",
       " None,\n",
       " '{ }',\n",
       " 'MULTIPOLYGON (((-73.96664570466969 40.62599676998366, -73.96684846176461 40.625977490862574, -73.96685938726297 40.62604419372411, -73.96661621040211 40.62606731716107, -73.96660638332114 40.626007324369795, -73.96664680403327 40.626003480977275, -73.96664570466969 40.62599676998366)))',\n",
       " None,\n",
       " '3170958',\n",
       " '1925',\n",
       " '2017-08-22T00:00:00',\n",
       " 'Constructed',\n",
       " '96807',\n",
       " '29.74985318',\n",
       " '2100',\n",
       " '40',\n",
       " '0.0',\n",
       " '0.0',\n",
       " '3065220021',\n",
       " '3065220021',\n",
       " 'Photogramm',\n",
       " '{31298F86-3088-4F53-B3DB-71A9EFA6FA1F}']"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# previewing the first row of our data\n",
    "data['data'][0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "dict_keys(['view'])"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Retrieving column names from the meta data.\n",
    "data['meta'].keys()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "id\n",
      "name\n",
      "assetType\n",
      "averageRating\n",
      "createdAt\n",
      "displayType\n",
      "downloadCount\n",
      "hideFromCatalog\n",
      "hideFromDataJson\n",
      "newBackend\n",
      "numberOfComments\n",
      "oid\n",
      "provenance\n",
      "publicationAppendEnabled\n",
      "publicationDate\n",
      "publicationGroup\n",
      "publicationStage\n",
      "rowsUpdatedAt\n",
      "rowsUpdatedBy\n",
      "tableId\n",
      "totalTimesRated\n",
      "viewCount\n",
      "viewLastModified\n",
      "viewType\n",
      "approvals\n",
      "clientContext\n",
      "columns\n",
      "grants\n",
      "metadata\n",
      "owner\n",
      "query\n",
      "rights\n",
      "tableAuthor\n",
      "flags\n"
     ]
    }
   ],
   "source": [
    "# preview view keys\n",
    "keys = data['meta']['view'].keys()\n",
    "\n",
    "for key in keys:\n",
    "    print(key)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "length of columns: 24\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "[{'id': -1,\n",
       "  'name': 'sid',\n",
       "  'dataTypeName': 'meta_data',\n",
       "  'fieldName': ':sid',\n",
       "  'position': 0,\n",
       "  'renderTypeName': 'meta_data',\n",
       "  'format': {},\n",
       "  'flags': ['hidden']},\n",
       " {'id': -1,\n",
       "  'name': 'id',\n",
       "  'dataTypeName': 'meta_data',\n",
       "  'fieldName': ':id',\n",
       "  'position': 0,\n",
       "  'renderTypeName': 'meta_data',\n",
       "  'format': {},\n",
       "  'flags': ['hidden']},\n",
       " {'id': -1,\n",
       "  'name': 'position',\n",
       "  'dataTypeName': 'meta_data',\n",
       "  'fieldName': ':position',\n",
       "  'position': 0,\n",
       "  'renderTypeName': 'meta_data',\n",
       "  'format': {},\n",
       "  'flags': ['hidden']},\n",
       " {'id': -1,\n",
       "  'name': 'created_at',\n",
       "  'dataTypeName': 'meta_data',\n",
       "  'fieldName': ':created_at',\n",
       "  'position': 0,\n",
       "  'renderTypeName': 'meta_data',\n",
       "  'format': {},\n",
       "  'flags': ['hidden']},\n",
       " {'id': -1,\n",
       "  'name': 'created_meta',\n",
       "  'dataTypeName': 'meta_data',\n",
       "  'fieldName': ':created_meta',\n",
       "  'position': 0,\n",
       "  'renderTypeName': 'meta_data',\n",
       "  'format': {},\n",
       "  'flags': ['hidden']}]"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# locating our columns (i.e. field names) and saving as a new variable called 'cols'\n",
    "cols = data['meta']['view']['columns']\n",
    "print('length of columns: {}'.format(len(cols)))\n",
    "\n",
    "# previewing first five\n",
    "cols[:5]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      ":sid\n",
      ":id\n",
      ":position\n",
      ":created_at\n",
      ":created_meta\n",
      ":updated_at\n",
      ":updated_meta\n",
      ":meta\n",
      "the_geom\n",
      "name\n",
      "bin\n",
      "cnstrct_yr\n",
      "lstmoddate\n",
      "lststatype\n",
      "doitt_id\n",
      "heightroof\n",
      "feat_code\n",
      "groundelev\n",
      "shape_area\n",
      "shape_len\n",
      "base_bbl\n",
      "mpluto_bbl\n",
      "geomsource\n",
      "globalid\n"
     ]
    }
   ],
   "source": [
    "col_names = []\n",
    "for col in cols:\n",
    "    print(col['fieldName'])\n",
    "    col_names.append(col['fieldName'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>:sid</th>\n",
       "      <th>:id</th>\n",
       "      <th>:position</th>\n",
       "      <th>:created_at</th>\n",
       "      <th>:created_meta</th>\n",
       "      <th>:updated_at</th>\n",
       "      <th>:updated_meta</th>\n",
       "      <th>:meta</th>\n",
       "      <th>the_geom</th>\n",
       "      <th>name</th>\n",
       "      <th>...</th>\n",
       "      <th>doitt_id</th>\n",
       "      <th>heightroof</th>\n",
       "      <th>feat_code</th>\n",
       "      <th>groundelev</th>\n",
       "      <th>shape_area</th>\n",
       "      <th>shape_len</th>\n",
       "      <th>base_bbl</th>\n",
       "      <th>mpluto_bbl</th>\n",
       "      <th>geomsource</th>\n",
       "      <th>globalid</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>row-2mv5.r2xm.f5nt</td>\n",
       "      <td>00000000-0000-0000-B60C-C89FFFC2D248</td>\n",
       "      <td>0</td>\n",
       "      <td>1713819867</td>\n",
       "      <td>None</td>\n",
       "      <td>1713819867</td>\n",
       "      <td>None</td>\n",
       "      <td>{ }</td>\n",
       "      <td>MULTIPOLYGON (((-73.96664570466969 40.62599676...</td>\n",
       "      <td>None</td>\n",
       "      <td>...</td>\n",
       "      <td>96807</td>\n",
       "      <td>29.74985318</td>\n",
       "      <td>2100</td>\n",
       "      <td>40</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>3065220021</td>\n",
       "      <td>3065220021</td>\n",
       "      <td>Photogramm</td>\n",
       "      <td>{31298F86-3088-4F53-B3DB-71A9EFA6FA1F}</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>row-pm73.muej.qxpz</td>\n",
       "      <td>00000000-0000-0000-598B-3FA74A14D320</td>\n",
       "      <td>0</td>\n",
       "      <td>1713819867</td>\n",
       "      <td>None</td>\n",
       "      <td>1713819867</td>\n",
       "      <td>None</td>\n",
       "      <td>{ }</td>\n",
       "      <td>MULTIPOLYGON (((-74.16790202462265 40.63936048...</td>\n",
       "      <td>None</td>\n",
       "      <td>...</td>\n",
       "      <td>326368</td>\n",
       "      <td>22.63</td>\n",
       "      <td>2100</td>\n",
       "      <td>39</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>5012640036</td>\n",
       "      <td>5012640036</td>\n",
       "      <td>Photogramm</td>\n",
       "      <td>{F5F8CDA5-69E2-46F8-8F69-BA95C025B520}</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>row-394q~zh7d-ats9</td>\n",
       "      <td>00000000-0000-0000-28E5-2A08E546599A</td>\n",
       "      <td>0</td>\n",
       "      <td>1713819867</td>\n",
       "      <td>None</td>\n",
       "      <td>1713819867</td>\n",
       "      <td>None</td>\n",
       "      <td>{ }</td>\n",
       "      <td>MULTIPOLYGON (((-74.19510813278613 40.55610681...</td>\n",
       "      <td>None</td>\n",
       "      <td>...</td>\n",
       "      <td>746627</td>\n",
       "      <td>35.76</td>\n",
       "      <td>2100</td>\n",
       "      <td>51</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>5060190091</td>\n",
       "      <td>5060190091</td>\n",
       "      <td>Photogramm</td>\n",
       "      <td>{9F644794-F72C-4582-9E5E-B337E2B97068}</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>row-jwfr~cxvn-3vdk</td>\n",
       "      <td>00000000-0000-0000-E608-6C044DBB5C66</td>\n",
       "      <td>0</td>\n",
       "      <td>1713819867</td>\n",
       "      <td>None</td>\n",
       "      <td>1713819867</td>\n",
       "      <td>None</td>\n",
       "      <td>{ }</td>\n",
       "      <td>MULTIPOLYGON (((-73.96113466505085 40.57743931...</td>\n",
       "      <td>None</td>\n",
       "      <td>...</td>\n",
       "      <td>786626</td>\n",
       "      <td>37.5</td>\n",
       "      <td>2100</td>\n",
       "      <td>6</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>3086910048</td>\n",
       "      <td>3086910048</td>\n",
       "      <td>Photogramm</td>\n",
       "      <td>{F916B22D-E25B-44AE-9FA9-2A51191B9CDF}</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>row-msin.i62d_4bf6</td>\n",
       "      <td>00000000-0000-0000-42C1-0BAF51DBB1A4</td>\n",
       "      <td>0</td>\n",
       "      <td>1713819867</td>\n",
       "      <td>None</td>\n",
       "      <td>1713819867</td>\n",
       "      <td>None</td>\n",
       "      <td>{ }</td>\n",
       "      <td>MULTIPOLYGON (((-73.75421559146166 40.75591276...</td>\n",
       "      <td>None</td>\n",
       "      <td>...</td>\n",
       "      <td>746409</td>\n",
       "      <td>18.01511294</td>\n",
       "      <td>2100</td>\n",
       "      <td>93</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>4075020005</td>\n",
       "      <td>4075020005</td>\n",
       "      <td>Photogramm</td>\n",
       "      <td>{525F2C24-616B-4F29-98A3-8FEA5D4B1A7D}</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 24 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                 :sid                                   :id  :position  \\\n",
       "0  row-2mv5.r2xm.f5nt  00000000-0000-0000-B60C-C89FFFC2D248          0   \n",
       "1  row-pm73.muej.qxpz  00000000-0000-0000-598B-3FA74A14D320          0   \n",
       "2  row-394q~zh7d-ats9  00000000-0000-0000-28E5-2A08E546599A          0   \n",
       "3  row-jwfr~cxvn-3vdk  00000000-0000-0000-E608-6C044DBB5C66          0   \n",
       "4  row-msin.i62d_4bf6  00000000-0000-0000-42C1-0BAF51DBB1A4          0   \n",
       "\n",
       "   :created_at :created_meta  :updated_at :updated_meta :meta  \\\n",
       "0   1713819867          None   1713819867          None   { }   \n",
       "1   1713819867          None   1713819867          None   { }   \n",
       "2   1713819867          None   1713819867          None   { }   \n",
       "3   1713819867          None   1713819867          None   { }   \n",
       "4   1713819867          None   1713819867          None   { }   \n",
       "\n",
       "                                            the_geom  name  ... doitt_id  \\\n",
       "0  MULTIPOLYGON (((-73.96664570466969 40.62599676...  None  ...    96807   \n",
       "1  MULTIPOLYGON (((-74.16790202462265 40.63936048...  None  ...   326368   \n",
       "2  MULTIPOLYGON (((-74.19510813278613 40.55610681...  None  ...   746627   \n",
       "3  MULTIPOLYGON (((-73.96113466505085 40.57743931...  None  ...   786626   \n",
       "4  MULTIPOLYGON (((-73.75421559146166 40.75591276...  None  ...   746409   \n",
       "\n",
       "    heightroof feat_code groundelev shape_area shape_len    base_bbl  \\\n",
       "0  29.74985318      2100         40        0.0       0.0  3065220021   \n",
       "1        22.63      2100         39        0.0       0.0  5012640036   \n",
       "2        35.76      2100         51        0.0       0.0  5060190091   \n",
       "3         37.5      2100          6        0.0       0.0  3086910048   \n",
       "4  18.01511294      2100         93        0.0       0.0  4075020005   \n",
       "\n",
       "   mpluto_bbl  geomsource                                globalid  \n",
       "0  3065220021  Photogramm  {31298F86-3088-4F53-B3DB-71A9EFA6FA1F}  \n",
       "1  5012640036  Photogramm  {F5F8CDA5-69E2-46F8-8F69-BA95C025B520}  \n",
       "2  5060190091  Photogramm  {9F644794-F72C-4582-9E5E-B337E2B97068}  \n",
       "3  3086910048  Photogramm  {F916B22D-E25B-44AE-9FA9-2A51191B9CDF}  \n",
       "4  4075020005  Photogramm  {525F2C24-616B-4F29-98A3-8FEA5D4B1A7D}  \n",
       "\n",
       "[5 rows x 24 columns]"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# constructing a dataframe\n",
    "df_json = pd.DataFrame(\n",
    "    data['data'],\n",
    "    columns=col_names\n",
    ")\n",
    "\n",
    "df_json.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>the_geom</th>\n",
       "      <th>name</th>\n",
       "      <th>bin</th>\n",
       "      <th>cnstrct_yr</th>\n",
       "      <th>lstmoddate</th>\n",
       "      <th>lststatype</th>\n",
       "      <th>doitt_id</th>\n",
       "      <th>heightroof</th>\n",
       "      <th>feat_code</th>\n",
       "      <th>groundelev</th>\n",
       "      <th>base_bbl</th>\n",
       "      <th>mpluto_bbl</th>\n",
       "      <th>geomsource</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>MULTIPOLYGON (((-73.96664570466969 40.62599676...</td>\n",
       "      <td>None</td>\n",
       "      <td>3170958</td>\n",
       "      <td>1925</td>\n",
       "      <td>2017-08-22T00:00:00</td>\n",
       "      <td>Constructed</td>\n",
       "      <td>96807</td>\n",
       "      <td>29.74985318</td>\n",
       "      <td>2100</td>\n",
       "      <td>40</td>\n",
       "      <td>3065220021</td>\n",
       "      <td>3065220021</td>\n",
       "      <td>Photogramm</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>MULTIPOLYGON (((-74.16790202462265 40.63936048...</td>\n",
       "      <td>None</td>\n",
       "      <td>5028452</td>\n",
       "      <td>1965</td>\n",
       "      <td>2017-08-22T00:00:00</td>\n",
       "      <td>Constructed</td>\n",
       "      <td>326368</td>\n",
       "      <td>22.63</td>\n",
       "      <td>2100</td>\n",
       "      <td>39</td>\n",
       "      <td>5012640036</td>\n",
       "      <td>5012640036</td>\n",
       "      <td>Photogramm</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>MULTIPOLYGON (((-74.19510813278613 40.55610681...</td>\n",
       "      <td>None</td>\n",
       "      <td>5078368</td>\n",
       "      <td>1970</td>\n",
       "      <td>2017-08-22T00:00:00</td>\n",
       "      <td>Constructed</td>\n",
       "      <td>746627</td>\n",
       "      <td>35.76</td>\n",
       "      <td>2100</td>\n",
       "      <td>51</td>\n",
       "      <td>5060190091</td>\n",
       "      <td>5060190091</td>\n",
       "      <td>Photogramm</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>MULTIPOLYGON (((-73.96113466505085 40.57743931...</td>\n",
       "      <td>None</td>\n",
       "      <td>3245111</td>\n",
       "      <td>1928</td>\n",
       "      <td>2017-08-22T00:00:00</td>\n",
       "      <td>Constructed</td>\n",
       "      <td>786626</td>\n",
       "      <td>37.5</td>\n",
       "      <td>2100</td>\n",
       "      <td>6</td>\n",
       "      <td>3086910048</td>\n",
       "      <td>3086910048</td>\n",
       "      <td>Photogramm</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>MULTIPOLYGON (((-73.75421559146166 40.75591276...</td>\n",
       "      <td>None</td>\n",
       "      <td>4161096</td>\n",
       "      <td>1950</td>\n",
       "      <td>2017-08-22T00:00:00</td>\n",
       "      <td>Constructed</td>\n",
       "      <td>746409</td>\n",
       "      <td>18.01511294</td>\n",
       "      <td>2100</td>\n",
       "      <td>93</td>\n",
       "      <td>4075020005</td>\n",
       "      <td>4075020005</td>\n",
       "      <td>Photogramm</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                            the_geom  name      bin  \\\n",
       "0  MULTIPOLYGON (((-73.96664570466969 40.62599676...  None  3170958   \n",
       "1  MULTIPOLYGON (((-74.16790202462265 40.63936048...  None  5028452   \n",
       "2  MULTIPOLYGON (((-74.19510813278613 40.55610681...  None  5078368   \n",
       "3  MULTIPOLYGON (((-73.96113466505085 40.57743931...  None  3245111   \n",
       "4  MULTIPOLYGON (((-73.75421559146166 40.75591276...  None  4161096   \n",
       "\n",
       "  cnstrct_yr           lstmoddate   lststatype doitt_id   heightroof  \\\n",
       "0       1925  2017-08-22T00:00:00  Constructed    96807  29.74985318   \n",
       "1       1965  2017-08-22T00:00:00  Constructed   326368        22.63   \n",
       "2       1970  2017-08-22T00:00:00  Constructed   746627        35.76   \n",
       "3       1928  2017-08-22T00:00:00  Constructed   786626         37.5   \n",
       "4       1950  2017-08-22T00:00:00  Constructed   746409  18.01511294   \n",
       "\n",
       "  feat_code groundelev    base_bbl  mpluto_bbl  geomsource  \n",
       "0      2100         40  3065220021  3065220021  Photogramm  \n",
       "1      2100         39  5012640036  5012640036  Photogramm  \n",
       "2      2100         51  5060190091  5060190091  Photogramm  \n",
       "3      2100          6  3086910048  3086910048  Photogramm  \n",
       "4      2100         93  4075020005  4075020005  Photogramm  "
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# removing unwanted columns\n",
    "cols = [\n",
    "    'the_geom', 'name', 'bin', \n",
    "    'cnstrct_yr', 'lstmoddate', 'lststatype',\n",
    "    'doitt_id', 'heightroof', 'feat_code', \n",
    "    'groundelev', 'base_bbl', 'mpluto_bbl', 'geomsource'\n",
    "]\n",
    "\n",
    "df_json = df_json.loc[:, cols]\n",
    "\n",
    "df_json.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 1083339 entries, 0 to 1083338\n",
      "Data columns (total 13 columns):\n",
      " #   Column      Non-Null Count    Dtype \n",
      "---  ------      --------------    ----- \n",
      " 0   the_geom    1083339 non-null  object\n",
      " 1   name        2254 non-null     object\n",
      " 2   bin         1083339 non-null  object\n",
      " 3   cnstrct_yr  1072979 non-null  object\n",
      " 4   lstmoddate  1083339 non-null  object\n",
      " 5   lststatype  1083016 non-null  object\n",
      " 6   doitt_id    1083339 non-null  object\n",
      " 7   heightroof  1080692 non-null  object\n",
      " 8   feat_code   1083339 non-null  object\n",
      " 9   groundelev  1082774 non-null  object\n",
      " 10  base_bbl    1083339 non-null  object\n",
      " 11  mpluto_bbl  1083339 non-null  object\n",
      " 12  geomsource  1083020 non-null  object\n",
      "dtypes: object(13)\n",
      "memory usage: 107.4+ MB\n"
     ]
    }
   ],
   "source": [
    "#sanity check\n",
    "df_json.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "This dataset has 1,083,339 rows and 13 columns.\n"
     ]
    }
   ],
   "source": [
    "# printing dimensions of data\n",
    "rows, columns = df_json.shape\n",
    "print('This dataset has {:,} rows and {:,} columns.'.format(rows, columns))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3.3 Reading in Shapefile data \n",
    "![borough-boundaries-shp](images/borough-boundaries-shp.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>boro_code</th>\n",
       "      <th>boro_name</th>\n",
       "      <th>shape_area</th>\n",
       "      <th>shape_leng</th>\n",
       "      <th>geometry</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>5.0</td>\n",
       "      <td>Staten Island</td>\n",
       "      <td>1.623621e+09</td>\n",
       "      <td>325917.353950</td>\n",
       "      <td>MULTIPOLYGON (((-74.05051 40.56642, -74.05047 ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1.0</td>\n",
       "      <td>Manhattan</td>\n",
       "      <td>6.365205e+08</td>\n",
       "      <td>357713.308162</td>\n",
       "      <td>MULTIPOLYGON (((-74.01093 40.68449, -74.01193 ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2.0</td>\n",
       "      <td>Bronx</td>\n",
       "      <td>1.187175e+09</td>\n",
       "      <td>463180.579449</td>\n",
       "      <td>MULTIPOLYGON (((-73.89681 40.79581, -73.89694 ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3.0</td>\n",
       "      <td>Brooklyn</td>\n",
       "      <td>1.934138e+09</td>\n",
       "      <td>728146.574928</td>\n",
       "      <td>MULTIPOLYGON (((-73.86327 40.58388, -73.86381 ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>4.0</td>\n",
       "      <td>Queens</td>\n",
       "      <td>3.041419e+09</td>\n",
       "      <td>888199.731385</td>\n",
       "      <td>MULTIPOLYGON (((-73.82645 40.59053, -73.82642 ...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   boro_code      boro_name    shape_area     shape_leng  \\\n",
       "0        5.0  Staten Island  1.623621e+09  325917.353950   \n",
       "1        1.0      Manhattan  6.365205e+08  357713.308162   \n",
       "2        2.0          Bronx  1.187175e+09  463180.579449   \n",
       "3        3.0       Brooklyn  1.934138e+09  728146.574928   \n",
       "4        4.0         Queens  3.041419e+09  888199.731385   \n",
       "\n",
       "                                            geometry  \n",
       "0  MULTIPOLYGON (((-74.05051 40.56642, -74.05047 ...  \n",
       "1  MULTIPOLYGON (((-74.01093 40.68449, -74.01193 ...  \n",
       "2  MULTIPOLYGON (((-73.89681 40.79581, -73.89694 ...  \n",
       "3  MULTIPOLYGON (((-73.86327 40.58388, -73.86381 ...  \n",
       "4  MULTIPOLYGON (((-73.82645 40.59053, -73.82642 ...  "
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "url = 'https://data.cityofnewyork.us/api/geospatial/tqmj-j8zm?method=export&format=Shapefile'\n",
    "\n",
    "# reading in data as a geodataframe\n",
    "gdf = gpd.read_file(url)\n",
    "\n",
    "# printing the firt five rows\n",
    "gdf.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "This dataset has 5 rows and 5 columns.\n"
     ]
    }
   ],
   "source": [
    "rows, columns = gdf.shape\n",
    "print('This dataset has {:,} rows and {:,} columns.'.format(rows, columns))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<AxesSubplot:>"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAQsAAAD4CAYAAAD7JMNRAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAAsTAAALEwEAmpwYAABH7klEQVR4nO2dd3ib1fm/7yPJ8t57O3EcZ09nAQlJIIwkJJSyoc23zG4KXVBW6frR0pZRWiizjELDDjuQQCAJWc52huPYcbz3Xprn94cUxUO25SnZPvd1+bL0vue872NZ+uiMZwgpJQqFQtEbGncboFAoRgZKLBQKhUsosVAoFC6hxEKhULiEEguFQuESOncb4IyIiAiZkpLibjMUilHN3r17q6SUka6290ixSElJITMz091mKBSjGiHE6b60V9MQhULhEkosFAqFSyixUCgULqHEQqFQuIQSC4VC4RJKLBQKhUsosVAoFC6hxEKhULiEEguFx7P3dC2Pfn6CioY2d5sypvFID06Foj3vHyjmpR2neWpLLqtnxLJqRixL06PQaoS7TRtTKLFQeDwFNS0AGC1W3tlfzDv7i4kI8ObqjASumZdIcrh/n695sqKR1MgAhFCC4ypqGqLwaEwWK4eL67scr2oy8K8tuZz/yBau+fcO3t5bRJvJ4tI1T1Y08cnhMiUUfcRlsRBCaIUQ+4UQH9qfhwkhPhdC5Nh/h3bT7w4hRJYQ4ogQ4meDZLdijJBd1khVk7HHNrtO1fDzNw+y6P9t5qEPjnC0pKHbtlar5I3MQj48VMrVT+/gg4MlGM3WwTZ7VNKXkcUdwLF2z+8GNksp04DN9ucdEEJMA24F5gMzgdVCiLT+m6sYa7x/sMTltrUtJl7cns/KJ7Zy2T+28fKOfOpbTB3aFNa28NI3+WSXN7I7v4afvL6fv3x6nL2nawfb9FGHS2IhhEgAVgHPtTu8FnjJ/vgl4HInXScDO6WULVJKM/AV8K1+W6sYc3zYB7Foz+Hieh7YcIT5f9rEXW8cYEduNVJK/vTxMQydRhLPbTvFc1vzBsPcUY2rC5yPAb8CAtsdi5ZSlgJIKUuFEFFO+mUBfxRChAOtwEpAJapQuERNs5HSAW6XGsxW3tlXzDv7iokL9qGq2fmUZvPxCorrWokP8R3Q/UYzvY4shBCrgQop5d6+XlxKeQz4M/A58ClwEDB3c5/bhBCZQojMysrKvt5KMQrZdrKKwSxrU1Lf1u36hNFspbHN5PScwoYr05BzgTVCiHzgf8ByIcSrQLkQIhbA/rvCWWcp5fNSyjlSyiVADZDTTbtnpJQZUsqMyEiXM30pRjHbcob3SyO/qmVY7zfS6FUspJT3SCkTpJQpwLXAF1LKG4H3gXX2ZuuADc76n5meCCGSgCuA1wfBbsUox2qVfHHc6ffPkPHYphN92oIdawzEz+JhYIUQIgdYYX+OECJOCPFxu3ZvCyGOAh8AP5JSqmVnRa8cLW3odct0sDle1sjP3zxIxh82cf97WRwrbcBksZJb2TSsdngqwhNrnWZkZEiVsHds84/NOfzt8xPuNoNJMYHkVTUzLS6IK+YkcOm0GMIDvN1t1qAghNgrpcxwtb1y91Z4JF9kD+8UpDuOlzUCsK+gjn0FdTywIYsF48JZPTOWi6fGEDFKhMMVlFgoPI7Cmhb2F9S52wynWCXsyKtmR141972XxfyUMFbPjGPlKBpxdIeKDVF4HH3x2nQnUtpcze9/L4t5f9zE3z7LdrdJQ4oSC4VHIaXk3f3F7jajzwghWDIxkr2na6hobKOmyYDJMrpiTpRYKDyK6mYjU+OCOhzLSA4lKczPTRa5RoivF4U1Lax7YQ8vbs/no8OlaEdZVKsSC4XH8MnhUn72vwMsSYvg2nmJaDWCjJRQMk/XUt9qJMjHc5fYrp2fSHFtK00GM09tyeXNzCKqmwzuNmtQ8dxXXzGmKG9o4471BzCarcxJDmVGQghL0sIprjMQ4uvF5mMV+Ht75tv1vAkRfO/ccTy2ybbVqxHwwGVT0GnPjiwa2kwE+Xi5y8RBwTNffcWY49mv8xxxGy9sO0WovxeFNa0AXDE7nivnJvDm3iJ3mtgt962ezJNfnOQtu30+Xlo2Hinj0umxhPrbdkiCfLyobzUR7DtyBUNNQxRux2qVbGi3A9JkMDuEAqC2xUh9q2cFeV08NRp/vZafXZjGgYI6/vNNPm0mm9i1GC38b3choX76Dn1GslCAGlkoPACjxUplY/fz+yUTI3nyi5PDaFHvbM2p4qcXpPGt2fFc9fQOx/HVM2J5YPUU9DoNIZ3E4gxSyhGZ0k+JhcLtfJJV2u25MH89UYE+VHeTh8JdtBgt/POLk5yqbHYkFAa4+9JJRAX5dNvPbN9Obb+eMVJQ0xCFWzGarTy+yWnWAjQCnrx+Nn/ZeHyYreqe8RFnM4k3GsyszyzscN5Pb/v+tVolFmvXuCudVoNOOzI/diPTasWo4cXtp8ivdp5H4qZzU3h+2ylOd3N+uFk1I5aNdy7hlvPGATArMaRLm1NVtgjVVpOFZsPZPE+jIexdTUMUbqOuxci/v+6a+zIu2IekcD9mJobw3Lb84TesGz46VMrEqEB+eUk6sSG+6LWCA4V1jvP3XDqJuclhlNS18tquAn5xcbrjnLdOw9Nf5XLFnHiiArufpngyamShcBtPfZVLjZO1CIPZyn2rpvCLNw+5waqe2ZpTibdOy83njWPeuLAO5xaMC+NfW05yoryRn180scM5IQQbj5Tx4IYjeGJaCFdQYqFwC5WNBl7cnu/03HXzErn3vawuWbg9gbp2W7j+9vUJP72W129dyKSYIDbsL6G0vg0hBCV1LV3yen6SVcanWWXDavNgocRC4Rae/irXafLcUD8vgvy8ONhueO8pPLRmKk/fOMfx/Izvx/hIfxaMC0OnFUyNC2LjkTKklMSF+BFo99q0WCVaIfjZhWlcOj3WLfYPFCUWimGloc3ET17fz/PbTjk9f9vi8Ty1JXeYrXKNpDA/JkSdrYYxLT6Y1TNiySlvwmzf+bh6XiJbsiu55aXMDmKoERDi50V0D9uqno4SC8WwUlbfxgfd5KuIDvJGqxXUtniWt+YZ/ruroMuxqzMSeWjNVPQ625Zond32L7MrWJ9Z6FifEEKwYko0oX4j14tTiYViWAnoIRjse+em8Fg3PheeQJCPjjaTxeFYBTbv0mvmJTqebzpWDtgyam06Wt7B10Ig8PHSDp/Bg4wSC8WwEhPk41Qw4kN8Ka5ro8Xomf4IGmHLivXKjtNoNR29L9u7bt+yeBxnTq/qtDZxVUbCiI4PUWKhGFZyK5toMnQtSnfx1Gg+Oey5uwRWCUsmRvCtOfEIIWhoM7HhQDGtncQtNTKABePCAWg2mjG2G4UIIZidFDqsdg8mSiwUw8rLO047PT4pNpAqD08W8/WJKsL9bcFhXhoNAd46yjvVYvXSalh3TgoAL27Px0c3cqcdnVFioRg2rFbJh4e6Lm6ekxrONyer3WDRWXy8NEyODeqxzZS4IEftVV+9lgsmR5PSLlbkDJdMi+E7C5PRCBiBwaXdosRCMWzkVTU53elYNT2WDw51H3k6HLSZrPjpex4FTI0LQqNx7dP/k+UTePL6OSMyFL07VGyIYtj4Jtf56EGv0ziN0BxudL0IQZ2LW7qnq5tJDvfvMVR9JKJGFophwWyx8p9v8p2eq21xf66KJRMjOVra0GObg0V1Ll0rObzr1GQ0oMRCMSxsOFBCXmWz03Nebs7vEOBt859obOu6S9Oe/QV17Mxz79qKO1FioRhytp+s4u89FDnWuHle32ayuFwQ6Jmv8yiua+294ShEiYViSDlYWMcNz+3q8QPm7hRzZqskrJt8mZ1ZMSUaAbw3AqumDRS1wKkYUj5xIRzbE/YLGtpMxAT5UNbJb6IzL2w7xe8/PIqPl5bLZsZ18eYczaiRhWJI6W2HAXBEbLqTfQV1zE3u3bsyp6KJFqOFmmYjp6udr8GMVpRYKIaMIyX1PLO1a9q8znR2mXYHSydG9rnPJ1llWD1A6IYLJRaKIaHNZOGu9QedJrjpjLurjes0gvtXT+Gzo67HpnhpBY9szCbjj5t4ftspt/8Nw4ESC8WQ8OzXeWSXN7rUVj/IW6feOg03nzeOK2bHOz0vhM0R7AwX2t22Y4Jdd6LSazVcOTeBmmYjh4vqaDG4f3Q01KgFTsWgU9di5FkXph9nGOyRvMFs5fltp1g7K47r5ifx+u6OSWu8tBqSwvxICffjy+xK1syKo6rJQGld18VNvVbD4rQIFqdFMCEqkLyqJg4X1bPhQAkzE4LRCsFv10zFtxdX8dGAEgvFoPPs1jwaenFwak+Az9B80DYcKGH9bQvJq2wit7LZEdVqNFs5WdFEekwgv74knQsnR2O2WvH10tJoD5+/YnY8K6ZEMyc5tEMqvPPSIgA4d0IEP3/zIK/funBMCAUosVAMMrXNRl7+xnkYendohMBLKzBZBn+xcMuJStbfvgiAguoWHvrgCJuPVwCQHh3IbUtSAdCj4fqFSfz7qzziQ3x55KqZPW6LXj47niBfHXvya/jDR0e5am4C1y9IHtVbqWrNQjGo/P3zE45vZ1f5764C7rggrU99UiP9WTIxkvGR/j2GgbdPu58U7tfh3JNfnqShXar+ldNi8dIKDGYrrnzml0+K5kfLJvDnb8/gg0OlrHpiKy90k4h4NODyyEIIoQUygWIp5WohRBiwHkgB8oGrpZS1TvrdCdwCSOAw8D0pZc+eL4oRSU55I//d1bdRBcCRkgYumBTNvJRQ9uR3eQt1YHFaBP93TgrnpUXgrdNitljZnlvN91/ZS6uTEoGdFzl/u2Yqc1NCqW4yEhXoTavRQpA9Xf/MxBB23HMBO3KrsUpw1bF0cmwQb9y+iL9/lt0lGc5oQrhaHUkIcReQAQTZxeIvQI2U8mEhxN1AqJTy1536xAPbgClSylYhxBvAx1LK//R0r4yMDJmZmdmPP0fhTm59OZPPj5b3q2+4v563vr+If3+dR15lE7udiMZ5EyJ46ab5NLaZqG0xMa5d4pkvj1fwRmYhMxND2HikjP0FdcSH+LL55+eP6CS5Q4kQYq+UMsPV9i6NLIQQCcAq4I/AXfbDa4Gl9scvAVuAX3fua7+HrxDCBPgBzvPAK0Y0e0/X9FsoAKqbjdS0mNh4pAyj2WpP4Hs2niQiQM+j18xCqxFszaniUFEd966a4ji/bFIUyyZFAbb0/DXNBoJ99YMiFN/kVnG0pIGbzxs3qpLZ9BVXpyGPAb8CAtsdi5ZSlgJIKUuFEFGdO0kpi4UQfwUKgFbgMynlZ85uIIS4DbgNICkpyeU/QOF+pJT8dWP3UaWuklt5NpNWqD+cPzGSbSer0ArB366eRWSgN2Dzi1hpz5xd32rik8OlpEUHMDfZVns0zF9PmL9rgWFnMFmsWKyyg7jUtRg5VtrID17dR32riW9yq/nJ8gmkxwTipx97ewO9/sVCiNVAhZRyrxBiaV8uLoQIxTYCGQfUAW8KIW6UUr7aua2U8hngGbBNQ/pyH4V72ZpTxY4B5nkQAqbHnc2BWVTbSlFtK49fM4up8UEdKoEJAe/uL+bKuQk8tukEL27Px9dLy9PfmcuStAiXv/3bTBbeP1DChVOiWb+nkJK6Vn5/+TTHue88v5vDxfWO9l8cr2BLdgVeWg3zUsKYEBVAs8HMhKgArpmXSEinyNXGNhN///wEbSYrv1s71e15OwaKK/J4LrBGCLES8AGChBCvAuVCiFj7qCIWqHDS90LglJSyEkAI8Q5wDtBFLBQjE6tV8udPjw/oGnqthvtWTybHSXKcA0V1rO20SOnjpeXKuQkA7DttW9toNVlY98JuQvy8+NbseH59yaQepyB/+PAo8aG+PPTBUfw/0NJstBDgreO+1ZPx1mnx1mmIDvLpIBZgcyAzmK1sO1nFtpNVjuOPbMzm5Zvnc05qhOOY2SL54GApVU0GogK9uXNFx8rqI41epU5KeY+UMkFKmQJcC3whpbwReB9YZ2+2DtjgpHsBsFAI4Sdscn8BcGxQLFd4BO8fLOFISc/p6HoiwFvHa7cuYM3MOKoaDQS2K0AU7OvF/9nT6ne3EG/pdLyuxcSL2/O59eVMCmtaur3v4eJ6HvrgKADN9kA2g9lCRYMBq1Xy5t6iPmXFMlslccG+judWq6S+1YTRbLv2a7sLqPOA9IEDYSATr4eBN4QQN2MThasAhBBxwHNSypVSyl1CiLeAfYAZ2I99qqEY+RjN1h4zYLnCj5dPIC06kNX/2EpJO3frIB8d/7ttIbHBvjz6+Qle2XmapemRLBgXxrJJUUQF2rwqf7d2Gute2N0lJd7WnCpueSmTjXcu6XLPoyUN7C+o63Jcr9WQXdaIwWzhV28d6vPfsiW7gpr9JnbmVWMwWcgqaXAkIq5sNPDZ0XKuzkjs5SqeS5/EQkq5BduuB1LKamwjhc5tSoCV7Z4/CDw4ECMVnsn6zEIKevj27g1fLy3XzUvi4U+OU1jTMZPWA5dNZXJsEPe9d5hXd9piO97ZV8w7+4qJDvLm5ZsWkB4TyJykUK5fkMRL3+TTZuoY+dlqr0uq1QjHOkZdi5HPj5YT6KOjurnjN32z0cItL2f2WI+1J35rH6l0RiMgIdSPpj64wHsiY29JVzEo1LeYeHxT/0cVGgG/XTOFkvrWLoFey9Ij+faceL48XuG0cnl5g4FbX87kq18uRQjBPZdO5u5LJpFb2cSuUzVUNBhICPXFapUcL2tkWnywo+8v3zrU6xavs/KKA8EqYXJsIDedN25QrzvcjOzlWYXbeG13AVVN/ZuDCwGv37qQa+YloddpmJkQ7AhT1whYd04KO/Kq+eVbB3G2VJEa6c/1C5KoaDQ48kgIIZgQFcj185O4bn4Sl0yL4dFNOWSXdQyT//75qb1WHhsKNh4pZ1MPItVqtHS7LuMpuOzBOZwoD07PpsVo5pyHv3C56E5nIgL0ZN63gtPVzcQE++Ct01LVZOAfm3O4ZfF4KpsMfPf53U6/4VMjbV6bhTWteHtpePzaWSyfFA3YfCVyK5u45t87iQ/xZVZSCH9YO61LFbHyhjZ8vLS8tqtgwDs5fSHY14sPf3IeiWEdY1QsVslFj37F5bPi+fHyCcPm+NVXD041slD0CYtV8s6+4n4LBUBVk5GvT1QSHWQTCoCIAG8eWjuN6mYj33luV7dTgdzKZnIrmzFarDS2mTlgX6gsq28j/b5PuOSxrdS3mjha2sDbe4t4ftspzJ2yWEUH+RDgrXMaSzKU1Lea+N2HR7tUX9t0rJzcymYe35zDW3uLqG/1zF0TJRaKPlHR2MZjA1irOMPR0gZ8vLR8dKiUKQ98yt8/P8E/vzzJ5f/c7tjKdIX86hbyKpsID9B3CQ83mK388eNjXP6v7eRWNnU4p9UIzpsQwXCzLaeqixBuya4EbNuv92/I4uJHt/LVicpht603lFgoXMZilby4Pb/faxXtmWFfdPx/nxyjxWjhic05PLIxu8/X+ehwKWUNbbSaLN2m58sqbuDSx7byxOYcDOazQuRKftDBptVk4dWdHSNzF44PczxuM1kpa2hj3Qu7+eeXJz1qHUOJhcIlTBYrGw4U85/t+QO+1nXzk0gM86OqyUBypxwTfcVildy5/gAzfvtZjyMSo8XmE7Ly8a3syqtGSkmJmyqLPbUll5MVZxdeU7qpjfrIxmx++N99tBg9Y8tViYWiV6SU5FY2sflYBcYBZrGekxTC7UvG8e2nvqHNZOHl780n1M9rQNcsbzC43La62YhV2nZPpsQN/64I2LZmv/v8bkfui8c353Tb9pOsMq59ZqfbhK09SiwUvVLbYqK8wcDHWaUDuo5eq+HvV8/kvveO8KNlEzhd3cLaf213RJoOB1PjgpiZaJsCRQf5uC0NXkl9Gz/87z6klCSF9Ty6OlRUz5ont7P3dM0wWeccJRaKXimpa2Xv6VqnPg994Y4L0zhQWI9epyHUX88Nz+0iq7j/cSV9QacRPHXDHJ6+ca4jvDwy0JsfLZtAVKA3K6fHkBDq28tVBpe9p2u56T97aHNhV6aqycC1z+zs4sA2nCixUPRIeUMbp6tbeHtv0YCuMzMxhDWzYnlicw53rpjIve8eHiQLXcNslezOr8FPr+uwaHjXiol8+rMlHC9tdEt19C+zK/noUCmxLtQsMVkk97xzmDvXH3BJYAYbJRaKHjGarfz50+MD+iAFeOv421UzueftLG5ePI7XdhV0CfwaDl7deZpNx856UUopKapt4cqnviGvqnnAI6f+cibBcUSAawl73t1fzPXP7qS6yfW1msFAxYYouuVYaQMbDpQMKFhMI+Af183mUFEdvnotE6MCuffdrEG00nVMFsntr+xlcVoEl06LpdVk4emvcqlsHN4PnTNK69sI8Na5XBJhX0Edl/9rO898J2PY3NeVWCi6JcBbxzNf5w7oGj+/KJ0JUQFc+8xO3v/Judz43O5Bsq7/bM2pYmtOVe8Nh5kmgxl/vRaTxbUpRmFNK99+6hv+cd1sLpgcPcTWqWmIogfe3lc0oNKC0+ODueW8cdz3XhYPXDaFDftLOFY6PAuaI5W+eK8CtBgt3PpyJk9/lTvkFd2VWCic0may8OzXrtcr7YyfXsu/bpjDf3cVMH9cGNPig3l0gIlyFM6xSnj4k+Pc/ureQQ+vb48SC4VT3j9Y0udvufbcu2oy/t46yhva+MH5qdy1/kCfK5Up+sbnR8t56Zv8Ibu+EgtFF9pMFr445iz/smuckxrO9fOTMJgt/PLidDYeKWPXKfc6FI0VKoawIpoSC0UXapqNaPr5ztBqBL9bOxWLVRIb7Etmfi13vXGQKHvND8XQUjGEOztqN0TRBZ1W9LlIzxl+vGxChxofm4+X02qyDHvuiLHKUPqvqJGFogtWK/1yUAr01vH981Mdz4+W1PO/PYXdtj+T9UoxeNQ0D13iHCUWii5YpOzXcHZSbCC++rOFfTJP13b7TRfoowa1Q0FF49CtWaj/mKILVqtkWz+cltpXNQc4UFDHrYvH0Ww0IyX46XVsya6wxWcghy2IbCwR7OuFlHJI8ngqsVB0ISHUF2sf5yFajSC00zrHb9dO5fInt5NXdbYs4YWTo1k4Poy39w0sME3hnDtXTByyhL9qGqLoQnWzEUMfU85ZrJLvLkrpcOzRz090EAqAQ0V1/PHjYySE+g446Y2iI+Mj/bl0WuyQXV+JhaIDRrO1xxqhPdF+j19K6TQDeEWjASlh07EKlkyM7Lediq78ZPmEIU3mo8RC0YGi2hZe2XG694ZOeH7bKcdjKWHhuLBu20oJzQazI3GvYmCMj/Bnzcz43hsOACUWig54aTVdivK4ykeHSx0VwjQaQVhAz45Ym45V0Nhm4pp5idxy3jjSogL6dd+xTqifF7+8OH3IUwSqBU5FB6wu5ITsjmBfL7zapeO3SklkoHeP+SJOVbdwqto27bl18XhK69uGNBhqNBAd5M1lM+JYOD6c9JhA4kN8+y3wfUGJhaIDyeH+/f6G6jwy2HS0nGBfL5eTy+wrqOW5dRk8tukEBwvrlddnJ1ZOj+HGBcksGB/ulkTDSiwUHSisaWF3P4O+2mdsKqpu5tMjZX1yP957upbyhjZ25qmgs848cd1s1syMc6sNas1C4UBKSWObuUupP1eZYhcLi1Xy4PtH+hWn8PWJSoapLvCI4mRF//4ng4kSC4WDQ0X13PSfPZTV989leL5992NLdgWbs/tWq1MIuGBSFDXNRpanR/Xr/qMZd9cMATUNUbQjv7qZsn7mQ0gJ93O4e7+Z2XfvzIzkUPKrm5FSkhjm53Li2rHC4aL6IXPjdhU1slA4iOxlq7MnVs+IQwiB2WLtcwVwL60g0MeL3MpmCmtaqWsxKaHoREObeUBZ1gcDJRYKBxsOlPS5jxA2h6DrFyQBcLCo77sYXloNzfbtUpNVUt1sdFtZQU/maIl7A+/UNETh4GBRXbfnxkf6c96ECKbFBTM+0p+9p2tJCPVjZmIwCaE2v4yKxjaOl/X9Dd1itFBSf7aIUUObiRBfL6qHMDfDSKRlADlRBwMlFgoADGZLt7sXV8yJ58/fntHB4Soj5awrt5QSo8VKVnG903gQVyisaScWrWbC/PRMignkeFljv643GjFbB1bBfqC4LBZCCC2QCRRLKVcLIcKA9UAKkA9cLaWs7dQn3d7mDOOBB6SUjw3MbMVgoxUCva7jrNRPr+X2JaksnhiBl1ZDVWMb2eVNVDYa+DqnktzKZgwmC+enR/KT5WmcKG/iYGEdEQF6qpoGNiqoaTFS12pkfkoou/Nre+8wBjD2MRJ4sOnLyOIO4BhwxvPmbmCzlPJhIcTd9ue/bt9BSpkNzAKH2BQD7w7QZsUQoNNquPvSSdz+yl4A4kN8+ecNc5iVGALYKqn/v4+P8cGh0i5986ubWZYexdzkUL44XjFow2WrhN35tcSH+lJcO/xFiz2NvqYNGGxcWuAUQiQAq4Dn2h1eC7xkf/wScHkvl7kAyJVS9i+kUTEg2kwWXttV0OO308VTY/jrVTO5fcl41t++kOn2iFCj2cqx0gY2HXdeHqDNZOWBDVk8sTkHKeWgz60TQnwH9XojlWbDyFizeAz4FRDY7li0lLIUQEpZKoTozZPmWuD17k4KIW4DbgNISkpy0SyFqzz6+QkqmwyckxpOSkT3iXKvnJvQ5di+glreO1BMaw8icKK8iRPlQ+NleLysgenxQRwe42n43B0r0+vIQgixGqiQUu7t702EEHpgDfBmd22klM9IKTOklBmRkSopymBz9bxEfnXxpB6FwhkWq2RcuF+/cnIOFvWtZg4XN7Cgh/wYYwGzxfPXLM4F1gghVgI+QJAQ4lWgXAgRax9VxAI9lbC6FNgnpSwfuMmK/pAa2XuuCItVohF08BI8XFTHqzsLqO3nLsdg0pfQ9dRIf8L89Viskn0FdUNn1DBidLNY9DqykFLeI6VMkFKmYJtKfCGlvBF4H1hnb7YO2NDDZa6jhymIwv28siOf1N98zEWPfk12u+3KQ8X1ZJd7xvD/SEkDc5NCem23YFwY+VXN7MmvdbvX42Di7t2QgXhwPgysEELkACvszxFCxAkhPj7TSAjhZz//zkAMVfSPNpOFK5/6pseoxaMlDfzx42MA5FQ08dUJ2yDxaGkDtc1GwHO8KQ8U1hHs232i34zkUHadquGMt3hVk7FDLZORjLvFok9OWVLKLcAW++NqbDscnduUACvbPW8BwgdipKL/+Hhpefa7GV18KM5Q02zk1pczaTNZ0WoE81PCWDvLlsvx6+xK3tlXzGkP+na2SEgK8+NwcX2Xc/NTwtid3zU6MzHUd8gWX4cTd1ehVx6cY4DO9Tza88jG4xTXtTI7KYTfr51GekwgWiF4cdspPjpc6lFCcYYzeT69dRoMZith/npSI/3Z40QogB5HIiOJejevGymxGOPcung8C8eHs2p6LDqthhajmV0Fdbyy6zR5lc29X8ANHC9rZF5KKFVNRoJ9vDhR3sCeHrw8PWcSNTDcvcCpxGKMMz4ygPH2nZLKRgM/f/MgK6fFeKxQnKEncehMXlUzc5ND2Xt6ZLuNy/5Uqx5ElFgoAGgxmrniqe0U1rTydR/zUXg6VU1GogK90QibC/lIZUS4eytGP20mq9tX24eSo6WNxIeObLfxSTGBvTcaQpRYKAAI89czNzkUH6/R+5YYSCYwT+DCKdFuvf/ofWco+szSiVFcNCWm223WkY5OO7L/rmlx7i31OLJfPcWgYbVKwvz1/OzCNEcMRkSAN15a4UidN9LJr2pmXkqou83oNz1tgQ8HSiwUgK02aWl9K0lhfvztqpksS49k5fQYAH66PG1U7D9WNBo45uY8lv3FW6ch0Nu9+xFqN0RxFiHQaTXsOlXDP2+YQ5vJyq2Lx/Obdw97/FaqqzQZLcQE+/S7Noq78NNrh6WeaU+okYXCwbL0SN7aW0R4gB4/vY7KRgMnyhtZtyjF3aYNGgIId/Nwvj/UtZpobFMenAoPISHUj7WzfKhqMvCbdw8T6K1je24VNQPMp+lJzB8Xxq5uarl2F1sCtm3LykaD2zKO6zSiQ8Jkt9jg1rsrPIryhjbe3V9MTbOR6fHBPLAha1QV+wnw1tLTXxMe0HXEEeit46bzxrEnv4bxkf746XVs7GPB58HAZJFszalihRu3T5VYKACbK3FhTQtbcyrJzK91u7fgYBMb7MOi1HDe2VfcbZsz3tTjIvypajTQaDBz7oQIPj5cSo4HFCZ+e2+RW8VCrVkoAJsr8fGyRrLLGkedUIAt8rS3tHxWu1poBA5vz/SYQI8QCui5CNRwoMRCQWObiQc3HOGzo+UDrvfhbvz1WqKDunpqHi9r5Jmv83r0s2g1WdDrNPjqtRTX2UoPNBnMeHuIk1ppfVufUgsONp7xKijcRkldKzvzatidXzMqAsiajRbC/J27dedWNnNeWgQZyaHodRpm2muinCGnvImJ0QEU1rSSHm2Lw8gqrme2C6n8hhqdRrB2VpxbhUutWYwQztTj8B9ExxwpJdVNtkxZI5VgXy+sUnZYcCysaSE9OpDs8q6lD3fn1bD+9kVUNhq48fldHc6VNbQxIyGYrOIGVs+IpbLJQKifnonRAezMc75LMpQEeOu4OiORafFBnJMaQUywz7Db0B4lFiOAIyX1/P7Do+zJryU53I+EUD/uWzWZidEDi0I8WmrzZvT10rq9JkV/aTNZuGvFRP7fJ8cdx5oMZkL8nGfHmpEYglYj+OpEhdO8pBdOiebS6TGsmBLDtfNt9WssVsnOUzXs7mbLdaDMSAimoKalQ53YVdNj+cPl09zu4t0eJRa9UNHQhl6nIcRPz9GSBqbEBfXeqY8YzBbe3ltMSV0rJouVmGAfIgK8KW9oY2tOFV+1mx7kVTaTV9nMVYV1PHHdbBZPiOiXZ19Vk4G4YF++/+reESsUYFuYffqrXJLD/ThdfTYF4P6COn6+YiKPb87B3C6JRXmDzXNz0fgIIgO9qWw0oBG2UgkzE0NICfdnvpOF0Ge+M5f3D5bwry9zKWsYXO/Pw8X1XDsvkSMlDRwqsuUW/cHSVI8SClBi0S0tRjN//OgYmfaMTL+8OJ071x/gv7cuYEZCCFJKhBD8+LV93LAgmUWpfc9JbLVKCmtb+MGr+xzf8q5S32pi3Qu7eeeH5zAnyfXgKCkl92/IYkduNSV1bSNaKM5Q22IiOsiH125ZwPf+sweD2YrRYmVGYgjLJkXx+dFy9DoNS9IiHJGbSeF+fHrHYqqajMSF+BDo03OezhA/Pd9dlMINC5J5cfsp/rUll5pBctCSEl7fXcj1C5Ioqm2lptmIj5fnZSQX7k7V5YyMjAyZmem+ebTBbOGWlzLZ6qQK17gIf3y8tMxKDOaaeUnUthhZND68yz9XSsnp6hZ2n6rh82PlhPh68d1FKUyICqDRYOKVHafJKW/iy+yKfm9VzkwI5tFrZjnS4rlCQXULSx75sl/383RmJgSzdlY8v/vwKGDzuvz5Renctf4A//7uXM5JjRi0e1U1GfjH5hxe3VWAZZDSb2k1givnJLA+s5DM+y4kYojzbwgh9kopM1xtr0YWnTBbrNzx+gGnQgFwqsoWUHWstIFNxyr45/VzANu8trHNREFNC1tzqiiqbeH13YUd+r65twg/vW19YDA02t9b1yehkFKyv3Bk56HsiYNF9cxOCmXFlGg+P1rO8bJGaluMbP31MkL8nA/p86uau5R0PFRUx6SYIPQ6DXvya5iX0nVaEhHgzUNrp3HjwmTufS9r0NYzNBoI8fOivKFtyMWiryix6MQfPjrGp0fKXGpb2Wjgumd34q3TMH9cGNVNRnIrm3qsIj6YFcYjArwd0yFXEELwyWHX/raRyn++yefBy6YwOSaQkvo2psYFdSsUTQYzq57Yyn9vXcisdtuoW7IrefiT4zy/bh5HiuuZFBPY7TQlLTqQN25fxJ78Gu5553CPxZx6Iz06kKlxwdx9yWTe3lfEVDcnu+mMEot2nKxo5LXdBX3qY7HatjS3ZA+/j0J9q8lloQCoazGy+fjoLzfrpdVw10Xpvbb7KruSeePCmBDVcXR2weQo/v75Ceb/cRMGs5W06EDOndDzFGZeShif37mEO9cf4P2DJX1ODBwX7MOPl09g5fRY6ltMRAZ61qgClFg4yC5r5OdvHhhRSWu9tH3bBfn4cNmICgwL9Nb1qQqXl1Zw/+op3Lgwuds2+wtqeXZrHlNig9BpNfxm5WQC2vmuNLSZmBIb5NgpAThUVN+rWFisEquUPHrNLC6dHsu972ZR1WTo1eZxEf7csngcV8xOcJRZDPbz4rKZca78ycOKEgs792/IIqt4ZGVRamgz02Qwd3iz98SGA90HUXkiK6fHMik2kL9/fsKlKM+7L53Md3vJvfFNbjVl9W2snhHHJVNjumw7C6C62egQCoDsMtv74mRFIynh/l1yeVqtEgGOEPKLp8awKDWcpY9s6XbHJD7Elx8tm8DVGQkjJjeoEgtsjj2Z3eQx8GTK6ttoM1kwW6zdzsvPIKV0Wh/Uk1mfWUhSmB+/WTmZg4V1rM8sdLowHBGg5zsLU9B1429yuKieafFBCCH40bIJ/GjZBMD2ITdbrOi0GsfaT6CPF18fKu3Qf/PxCpoNZvYV1OGr1xEf0rGkgDM/l0BvHSF+XtQ0G9FqRIcdkzsvnMjt54/3yO3RnhgZkjbENBvMI7L4jNliJSLAu1ehAFuQ1GAurg4XBTUt3PPOYYrrWvnrlTPJSO7oU+Kv17L+9kXccWEa685J6XCuzWThN+8e5rIntzn9htdohONbfX9hHW12n5MlEyM6xI00tpm58fldXDkngROdXMibDGY+PFTSZfv0X1tyyats5k/fmk7Wby/mHLsfTkKoLz9ePmHECQUosQBApxmZL0NpQxul9a0utR0sByJ3sTWnirvfOcRlM+N4/NpZxAT5IAT85II0Up1sH5+saGTtk9t5bVcBF02JxmixFVE6Iwgmi9Xx897+YuYkhTo+wFlF9bQaO057Supa2ZNfQ055o6Mw8+Zj5ax6YmuXxe1Ps8r462fZTI0L4tp5ifjqtTx141zSogK4cm4CWjfn0uwvahoCNBrcm9uwv0gJf914gr9dPbPXtq6ua3gyJovkDx8d5d6Vk/nLlTOIDvIh3UmVrsz8Gm76zx4a7Osc+dXNRAX6oNUIpJRkFdej0woe35TDUzfO5fyJkY6+ZfVtfJNbTXWnUH1vnS3L1tUZiXjZpy0/fX0/zUYLF02JRiPgN+8exmKRvHugGCnhNysnO6Yogd46blyYTFK439C9QEPMyPxKHWTiQ3xH7Ifp7X1FPPr5Ccy9VNj299Z5TF6GgWCySH77wVFueSnTaTnC+lYT33vxrFAAnK5uwWy1vT7NRgvv7i/GaLby0NqpQMd6HIW1LWw6Vt4l12ZBTQsfHSrt4G+REOpHaqQ/yyZFsb+wjtd2FbA+sxCj2crFU6M77KBIICrQm4KakZslfeS/ewYB28LWyBQLgMc35/Cfb/J7bOOl1XTxJxhJdP7/xIb44K/vOu/PrWzqsN2aEu7HPZdOwltnaxvgreP+1VOYkRBCVGDXkO9mg5k2J9vnGgH3rZ7smEIIIbhxYRIxwT6ckxrBm5lFHdrXtnQcrWo1ggsmR/P4ppOOqdBIY+R+QgYRg9kyovwrnPGHj44RHeTT4/58WlQAR0ZYkR0hbNOtHyxN5ePDpY7t7WaDBauEzq4mU2KD+NmFaVitkiUTI5mbHOqy41qbycLz205hscouYftWCaV1bR1cw9fMisfbS8uhojre299xW3pPfg0bDhSzZmac4/4VjW3MTQ6lptlIXMjIK9KsRhbAu/uK3ZbifTD5zTuHaTF2749wVUbiMFozcCZGB3CefSj/l0+zWT4pmii7Z2NVk4Ef/XcfH3Xa5vTx0vKzCydy10XpZKSE9SoUVqvkpW/yaTGaqWsx8cS1s3jrB4vYc9+F/P7yaR12X3734VHqW8+OGIJ9vfj2nATeyCzsEr0rJTz4/hEaWm3/j9zKJp784iRWq+SNzI4xQyMFJRYw6PkJ3EWjwcy/v8rr9vy5EyL44dLUYbRoYMSH+HbIOfnE5hx+uDTVUcZve24V//4615EvszPdRVQ3GcycrGjijT2FWKXkyS9Pcv2zu7jk8a+paDQSFehDgLeO7yxM5teXTmJqXBDBvl746bX4dZr6aARMjg1iUruFVo2Aa+cl8tFPFxNsT8JTUNPCzrxqNh+v4OUdp/nzp8c5ZE/AK6Xkm5NV3PXGAR7ZeBxPRYkFZxOijAae3ZrXxRegPf93bsqIqciVkRLG/oK6Dsfe2V/MhfZ0+I1tZqbFBxMb5Dzd3O8+PIrVasXayQfihW2n+MGre0kM86PZaKGy0UBWcT3nT4wkPSaQb05W0WI0s/d0DbMTQ1i3yPaafZVd2cXxy2SRTI0L5vFrZ3HzeeOIDfbh1ZsX8PC3ZxAf4usQAX+9jvd/ch4Z9mnIU1tyuek/mZz78BfM/cMmrn9uF+/sK+afX+ay97RnRgarNQvA12v0vAwtRgt//+wE/7h+ttMKVlGBPjx6zSz+78XdHu2INjMhmMh2IdqBPjpajBYqGgwOf4gwfz3fX5LabaawSTGBfHi4lHNTIwhvd63LZsZxxZx4/PU6/vLpcfRaDRNjAnhg9RSOlzXw3Rd2863Z8WSVNPD0jXMI8fMir6qZlHC/DtMao9nK7z48Sk55IxoheP22hXz//FRHEFh2WSM3v5RJq8nCeRMimJcSSnbZWSHvLnbkxe2nmJvsekKj4cLlT4kQQgtkAsVSytVCiDBgPZAC5ANXSym7SKIQIgR4DpiGbQfpJinljgFbPohcOCWKF7afcrcZg8anR8r408fHePCyqU7PL5kYyXPrMvjrxhOcrGjC2Mu2qzs4WFTPOROamZ0Uwv6COp64djbeOg3Fda1MjA7kdE0LKyZH46vX8s3JKnbn15AeHcil02Md17hoSgwWKQnrNJIaZ1+kNJosRAbouX5+EmEBesIDvCmoaeEHS1NZlh7F1zmVnK5u4ZGN2QDcs3IyjW0mx/apyWLlk8MlBPp4cfls28Jy+2jRn76+37GWER/iS3WzkaYe1pTO8OGhUhalnuaGBd0HxLmDvnyl3gEcA84kobwb2CylfFgIcbf9+a+d9Hsc+FRKeaUQQg94nFfKOakR3HzeOJ7fNnoEI6e857wKyydFs3xSNGaLldM1LezKq+G13afJq2x2i1v4Y9fM4i+fHqekXXXzp7bk8puVk/jh0glMig10VD5/ecdpQvy8OFJSz9GSBkeiohsXJnUQi1B/PS1GM8V1rSSE+lFY00J8iK9jJKL30nLb+amYLVaCfG2CMjsplNn2NIVzkkMxmq3UthjRCNtoZ+GfNnPvqilcvyAJf28dmfetAOgQs/LZkTLOnRCB3u7XkhjmS0ZKGCV1rS4nPbr33SzSowPJcJJ4x124JBZCiARgFfBH4C774bXAUvvjl4AtdBILIUQQsAT4PwAppRHwyG2H288fzwvbTw1KBitPoLyhzaXEODqthtTIAFIjA7h+QRJtJpvT0l8+Pd7FV2AoefqrXB64bAoPf3KcX1ycTmldG09szuFPH9sW/Pz02l5F7GBhx0A5k8VKcW0radGBZBXXs+bJbQBEB/lw76rJrJ4Rh5++54/A/oJajGYrGSlhRAX6MCEqgE+ySrl+gS3z95nXVwhbrM5/dxVQ1WRgZ14Nh4vrWTAujCUTI/k6p5KmPtZH9bSFd1cXOB8DfgW0H69GSylLAey/o5z0Gw9UAi8KIfYLIZ4TQvg7aYcQ4jYhRKYQIrOycvgTyUQF+jAj3rMyEw2Ea+cn9WtNwsdLy3Xzk/j4jsVMix/8TObdcbyskR+/tp87V0wkIcSX9w4U88Blk0mwe2m6Mto5XFzfYWvTZLE66qyEB+gJ9dPb/CXq2/jTR8ccMR6dsVglrUYLXxwvZ2tOFY0GMxOjA9BoBH++cgaXTrONXoxmK3/7LJvvv7KXq/+9g8V/+ZJPs8qICvTmhe2nmBAVwCs3L2BnXjUfHCzBS6shuQ/u3u6umt6ZXkcWQojVQIWUcq8QYmk/rj8H+ImUcpcQ4nFs05X7OzeUUj4DPAO2hL19vM+AsVplnxKteDK3nz+em88bN6BrxAb78sbti/i/F/cMWb2Mzpitkl+8eZB/XDebFqOFB98/yj2XTuL+DUdcvsbe0zUsn2TbLfHT6zCYbIIQG+zL366eyZNfnCTET8+PlqV2+DD++6tcsssbuW/VFG59OZPDRfWYrVas0rYVetti25bzpJggJsXYRPStvUX844uTHe5/7bwknrNPZ09WNHHJ41+TV2lz8e6pKHNnpsQGEdRLxvHhxpVpyLnAGiHESsAHCBJCvAqUCyFipZSlQohYoMJJ3yKgSEp5pvTTW9jEwuP47Gi54586klmcFsGvLp40KNfy0+t44trZLP3rl7SZhm4RNNBbx8rpsazPLLTFfrx/lF9eks5L3+Tz6ZEyJscGcczFUgl5lc0sb/fnt4/7WJoexdL0rgPgg4V1jiJFLUYL4yL8WTAujMQwP8wWK3qdpkMAmJQ2x6qdedXMSAh21PoA+CSrlIqGs7scfXlPaTWCNTPjmBoXRKvR0sWnw930KhZSynuAewDsI4tfSClvFEI8AqwDHrb/3uCkb5kQolAIkS6lzAYuAI4OnvmDQ35VMw++n+VuMwbMrMQQ/v2duYMaAh0T7MM/rpvD/e9lDdkcutFgdkw3wDZX//Vbh3jwsil8eKiUxFBfl8ViT34Ntywe3+FYZaOBMH99t6/LlLggMpJDyTxdy6dZZfh6adl3/wqMFis+XhpHXAnYhOLVXQV8fKgUnVZ0mR4dL+vex6U9gd46vHQaR+qAiABv/vO9eUzz4KnwQBwMHgbeEELcDBQAVwEIIeKA56SUK+3tfgL8174Tkgd8bwD3HHTqW03cvyGL8obe8yV6MoE+Ov55w5xeF+z6w+K0CC6eGs1LO04P+rXPkFVS7/B3yCpuwGyVPPHFSV6/dQHPb8t3+Tpfn6jCaLY6diIOFdVxw7O7SI0K4N0fnuN0wddLq+F/ty3krb1F/OGjY0yPD0av03CkpL7DbsSJ8kZ+/sZBR8YxjaDXdaGkMD8ykkOJDfEhJsiHuBBfksP9GB9hWwMpb2jjVFUzk2NtXqKeTJ/eWVLKLdh2PZBSVmMbKXRuUwKsbPf8AOByIZPh5nR1c7c1QkYSty8Z3yXd22AhBEyNC+a7i5J5eYgEQ0p4dl0G617Y7TgWF+wDCBrsi5bjI/ypbDJwbmpEt+UaWk0WKhrbSAj1w2KVfP+VvTQazBworGNLdiXLJjlbh7ftCl07P4kr5yZgtkq0GkFGShht9hov/9tTwEMfdBwUOxOKUD8vpsYFc15aBJdOiyE53Ol6voPoIB+iu/FA9TRGj+tiPxmJ6c06891FyY68kkOBwWzly+wKPskaupoje0/X4qXT8MOlqVQ1GQj08WJchD8fHirh/PRIrpufxJzkEHQaDV5awdp/bu+wVtCeV3ac5p6Vk6lrMXbw23CldqhOq+HMrOPvn2Xz0o7TWKyyQ4xKe0L9vDh3QgQLx4czf1wYaVEBfSrPMJIY82LROSPSSGPBuDDuXz1lSN+gJyua2Gj/Jr9oSjSHi+spre/b+oVeqyE+1JfIAG9MVitS2qaAZquV8noDEltdE3+9liaDma05VVQ3GRECdv3mgi55RtfMjOtWLAprbQWSwwO8CfPXU9di5LuLUpiZ0Pt6QJvJwqZj5byzr5gvsyu6+N3MSQohNTKAKXFBnJMaQWpk12zfo5UxLxb7CjwzaMcV/PVaHrhsypDsx+dXNfPu/mLiQ31JDPXjH9fNIdTfi9hgX5b/bQtRgd5UNLq+zjMlLoj4EF8Sw/xYlBrO5JhALFJitkgMZgsJoX4dRnknKxrZllNFY5uZoyUNnNOpbkdP1ex/uPTsKOu+VZMJ9dezrNMuyNacSj44WEJMkA8JoX40tJkc+TI6u7+nRvqzakYc35od73AVH4uMebHYfGxkVujSagT/unFutyXu/rXlZIcPTV94M7OQ37x7uENBohA/Ly6YFM3S9Ej+ePl0PjtaRkUfqrAdKKzjQGEdYPPW7Mzuey/oIBYTogKZENU1v+YZ9E4EMjLQm58un9BhR+GKOQkd2jS0mfjthiO8s79nnwe9TsOKKdHcuCCZheN7z4sxFhjTYmG1SopqXcuO7Wn8fu00zrWnl3fGQIrqvrOvuEvlsroWE2/vK+LtfUXd9BoYhwrruXCK6wt9MxJCmJEQzOK0CCbFBBEV6M2spJAO25yd2ZlXzc/fONht/guA5HA/frI8jVXTYx0VwhQ2xrRY5Fc392ko7QloNYLfrpnqiE3ojqsHkBVrRmIwO/Kq+92/O7x1GibFBHLxtBhajRYsVltEaIvR4jT5bk/odRre//F5LrU1mG1h+89szes29icxzJebzh3H9QuSehScscyYFotsFx1oPInfrZ065KHLP7tgIrMTQ9lfUMsbmYUuB5QF+uhIjQxgVmIICaG+xIX4Eh3kQ0KoLzqNINRP323uiaFiX0Etd799iBPdROFOignkh8smsGp67Iit5zFcjGmxGGnJa392Ydqw5Djw1Wu5ZFoMl0yL4ebF41j1hC1a02i24q/XkhTux4SoAGKDfYkP8SXUX0+CfSFUpxHDLgjOaDGa+evGE7z4jfNI4lmJIfxgaSorJkd7hL0jgTEtFptG0OLmhZOj+enytGG/b1SgDzvuXt7hW/fMYp/JYnVrZKTZYqWgpoWUcH/yqprw1es4UdZIdbORxzadcLoetTgtgjsuSPOoPBEjhTErFrmVTS778bubdYuSuadddStXqWk2EuSjG7AfQHf93R1CfdcbB3n/YAneOg0Gs9VRNqAzscE+LE2P4vJZcSwY3/2isKJnxqxYfHCwxN0muERGcigPXja1X0NlrUbQbLAQ7DcynYaklHydU8XWE5X8aNkEhwemlJL/7Snko8O2MgAGe82XzkIxKzGEWxaP4+KpMW4XttHAmBWLkxU9p53zBL41O54fLZvQ7zm1pwcmOaPZYKakrpWCmhbW7ynks6O2qeJ7B4q5cWEyxbWtVDUZ+LIbHw8h4LIZcfx4+YRR7XrtDsasWLS6Ic9kXwjz1/O7tVM71NYc7XyaVcad6w90KdgDUNVk5LFNOd32TQj15ZqMRK6YmzBkAXVjnTEpFm0mC5keWpsBbGHNL/xfxpgRCqtV8tRXuY4s2q4S6KPj4qkxXJ2RyNzkULX1OcSMarGobzXx0aFSQv28mJ0UipdWYJGSX791qEOuRk9Cr9Xwo2WpPbo6jxZqm4388q1D7DpVTaMLyWyXTIxk1fQYhBCkRQUwOTZoVEQNjxRGrVg0Gcxc8LctVLWLKk2N9Ke4rnVIU8QNhKXpkcxJCuWaeT17Z44WHv7kuEvb18vSI7lrRTrTXYgaVQwdo1Is/rXlJO8fKOkgFAC5Hpxjc9WMWJ68bvaoKUXgCr1F/C4aH86vLkl31PFQuJdRJxaFNS28saeQ/OoWd5viMj5eGh6056QYS4v3ft7O334Lx4fx42VpnDshXO1meBCjRiwKa1poM1n4/qt7R4xQRAd5c+m0WBaODyNqhKRWG0w6Fyw+JzWcX1yczhw1kvBIRoVYFNe1cvk/t1Pd7HlZr9bMjOOOC9MI8vHiz58e5629thDvWYkhPPzt6Y4aFGORulbb/2vR+HB+sDSVxWkRaiThwYwKsXhua55HCsWi8eH84VvTHMVi/vSt6ezMqyYjOZRHr5k15j8YaVGBPHndHGYmhrjbFIULjAof2HWLUtB52B67ViP45w1zOlSV0us0TI0L4qE108a8UEgpefrGuUooRhCjQixSIvyZFOtZfgkWq8RoPrtFa7VKXth2iiUTIwn2GxvOVj0hhHDU9lCMDEbNfysj2bNCji+YFIW/t81hSErJxiNlrN9TyDUDyGClULiTUbFmAXDHBWl8mlXm9jL10UHe/PSCNK6YncDh4nq+PlFJm8nC67sLeOK62WMmbbxi9DFqxCLUX09GSigfHip1mw3xIb48ty6DybG2HQ4h4MkvbVW2r5gdzwWTo91mm0IxUEaNWABujTacFh/Ea7cu7LCgmZEcSkKoLQ/l7y6f5jbbFIrBYFSJxVcnXK9jMZhcMSee366Z2kEowLaI9/YPziHQRzckBYsViuFk1LyDmw1mssuHL02eEHDbkvFcOi2WWT1s/42UorcKRW+MGrGobTGiFQLzMERixYf48s8b5vQoEgrFaGPUiEVCqB9h/vohLRq0cHwYv1s7jeggnxGZsk6hGAijRiwAHr1mFute2I3ZOjijC61GcMXseMZF+nOkuIFHr5mlHIkUY5YRLxbSPu146qtcNmaVDZpQ6DSCF783j8VpkY77jHUXbcXYZsSLxbHSRl7bfZpXdxYM6nUfvGyKQygAJRSKMc+IFoui2ha+8/yuQYk49dIKR+Xw6xck8Z1FKQO+pkIxmhjRYvHaroIBCcW18xK5YHI0c5JCeGdfMX/8+BjT4oN48LIpg2ilQjE6GLFiYbZY2ZFX3e/+3zs3hQfsqewAbl0ynrToAFIjA/DWqYzRCkVnRqxYvL6nkP0Fdf3qmxzux10rJnZZh1iaHjUIlikUoxOXxUIIoQUygWIp5WohRBiwHkgB8oGrpZRd0jULIfKBRsACmKWUGQM3G+cVcF1g0fhwXvzePFVvQqHoI31xGrgDONbu+d3AZillGrDZ/rw7lkkpZw2aUADjIwP61D422Iflk6L4y5UzlFAoFP3ApZGFECIBWAX8EbjLfngtsNT++CVgC/DrwTWve85JDSctKoAcJwWOw/31rJ4Ry7Xzk0gJ96ekvpXxEf5q+1OhGACujiweA34FtC/lFS2lLAWw/+5uwi+Bz4QQe4UQt3V3AyHEbUKITCFEZmVl79GjQggevWYWgT4d9W7h+DC2/HIpD62dxuTYIHz1WlIjVTVthWKg9DqyEEKsBiqklHuFEEv7cY9zpZQlQogo4HMhxHEp5dedG0kpnwGeAcjIyHBpQWJafDD3XDqZF7af4hcXpRMZqGdidOCYKSisUAwnrkxDzgXWCCFWAj5AkBDiVaBcCBErpSwVQsQCFc46SylL7L8rhBDvAvOBLmLRX65fkMT1C8ZGbVCFwp30Og2RUt4jpUyQUqYA1wJfSClvBN4H1tmbrQM2dO4rhPAXQgSeeQxcBGQNku0KhWIYGUgI5cPACiFEDrDC/hwhRJwQ4mN7m2hgmxDiILAb+EhK+elADFYoFO6hT05ZUsot2HY9kFJWAxc4aVMCrLQ/zgNmDtRIhULhflRyBoVC4RJKLBQKhUsosVAoFC6hxEKhULiEEguFQuESSiwUCoVLCDkMdTb6ihCiEjg9zLeNAKqG+Z49oezpGU+yx5NsAdftSZZSRvbezIZHioU7EEJkDmYI/UBR9vSMJ9njSbbA0NmjpiEKhcIllFgoFAqXUGJxlmfcbUAnlD0940n2eJItMET2qDULhULhEmpkoVAoXEKJhUKhcIkxJRZCiPVCiAP2n3whxIFO55OEEE1CiF900/8RIcRxIcQhIcS7QogQN9tzlRDiiBDCKoQY8FbZINgTJoT4XAiRY/8dOti2CCHmtzt+UAjxrW76zxRC7BBCHBZCfCCECOqvLYNkzywhxE57u0whxHw329Pj/9opUsox+QP8DXig07G3gTeBX3TT5yJAZ3/8Z+DPbrZnMpCOLcdIhge8Pn8B7rY/vnuwXp/2tgB+7f4HZ9I56pz02QOcb398E/D7oXht+mDPZ8Cl9scrgS3utKe3/7WznxFbkWwgCFuq76uB5e2OXQ7kAc3d9ZNSftbu6U7gSjfbc8zedjDMGLA9DEF5iM62SClb2p32wZY93hnpnM31+jmwEbh/ILYM0B4JnBndBAMlA7VlgPY47d8TY2oa0o7FQLmUMgcc+UF/DTzUh2vcBHziQfYMJv21x9XyEP22xW7PAiHEEeAw8H0ppdlJvyxgjf3xVUDiINgyEHt+BjwihCgE/grc42Z7uu3fHaNuZCGE2ATEODl1r5TyTFLh64DX2517CHhUStnkyre0EOJewAz81xPs6QueZE8/bUFKuQuYKoSYDLwkhPhEStnW6Ro3AU8IIR7Allza6GZ7fgDcKaV8WwhxNfA8cKEb7TlDl/7dMljzppHyg00gy4GEdse2YqvXmg/UATXAj7vpvw7YAfh5gj329lsYpDWLgdgDZAOx9sexQPZg2+KkzZe9/e3ARGD3ULw2rtoD1HPWr0kADe60x9X+7X9G3cjCBS4Ejkspi84ckFIuPvNYCPFboElK+WTnjkKIS7ANx8+XHeeGbrFniBiIPWfKQzxMN+UhBmqLEGIcUCilNAshkrGtTeR37iiEiJK2WjUa4D7g6QHaMiB7sK1RnI9N2JcDvQ77h9gep/17YiyuWVyLq8MuQAjxXLttySeBQGyV1Q4IIQbjDdhve4QQ3xJCFAGLgI+EEBvdaQ/dlIcYZFvOAw7at/reBX4opaxyYst1QogTwHFsH9QXB2jLQO25FfibsJXF+BPQbSnPYbKnu/7doty9FQqFS4zFkYVCoegHSiwUCoVLKLFQKBQuocRCoVC4hBILhULhEkosFAqFSyixUCgULvH/AXfdDUuQQj64AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# plotting one building footprint centroid\n",
    "gdf.plot()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3.4 Reading in data from Socrata Open Data API (SODA)\n",
    "**Note: If you haven't signed up for an app token, there might be a 1,000 rows limit.**\n",
    "\n",
    "![building_footprints_csv](images/building-footprints-soda-api.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:root:Requests made without an app_token will be subject to strict throttling limits.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "client.__dict__:\n",
      "{'domain': 'data.cityofnewyork.us', 'session': <requests.sessions.Session object at 0x17414f100>, 'uri_prefix': 'https://', 'timeout': 10}\n"
     ]
    }
   ],
   "source": [
    "# nyc open data domain\n",
    "socrata_domain = 'data.cityofnewyork.us' \n",
    "\n",
    "# building footprints dataset identifier\n",
    "socrata_dataset_identifier = 'qb5r-6dgf' \n",
    "\n",
    "# The main class that interacts with the SODA API.     \n",
    "client = Socrata(socrata_domain, None)\n",
    "\n",
    "print('client.__dict__:\\n{}'.format(client.__dict__))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We are setting the **limit** at **1,000 rows** (i.e. the full data set)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>the_geom</th>\n",
       "      <th>bin</th>\n",
       "      <th>cnstrct_yr</th>\n",
       "      <th>lstmoddate</th>\n",
       "      <th>lststatype</th>\n",
       "      <th>doitt_id</th>\n",
       "      <th>heightroof</th>\n",
       "      <th>feat_code</th>\n",
       "      <th>groundelev</th>\n",
       "      <th>shape_area</th>\n",
       "      <th>shape_len</th>\n",
       "      <th>base_bbl</th>\n",
       "      <th>mpluto_bbl</th>\n",
       "      <th>geomsource</th>\n",
       "      <th>globalid</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>{'type': 'MultiPolygon', 'coordinates': [[[[-7...</td>\n",
       "      <td>3170958</td>\n",
       "      <td>1925</td>\n",
       "      <td>2017-08-22T00:00:00.000</td>\n",
       "      <td>Constructed</td>\n",
       "      <td>96807</td>\n",
       "      <td>29.74985318</td>\n",
       "      <td>2100</td>\n",
       "      <td>40</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>3065220021</td>\n",
       "      <td>3065220021</td>\n",
       "      <td>Photogramm</td>\n",
       "      <td>{31298F86-3088-4F53-B3DB-71A9EFA6FA1F}</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>{'type': 'MultiPolygon', 'coordinates': [[[[-7...</td>\n",
       "      <td>5028452</td>\n",
       "      <td>1965</td>\n",
       "      <td>2017-08-22T00:00:00.000</td>\n",
       "      <td>Constructed</td>\n",
       "      <td>326368</td>\n",
       "      <td>22.63</td>\n",
       "      <td>2100</td>\n",
       "      <td>39</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>5012640036</td>\n",
       "      <td>5012640036</td>\n",
       "      <td>Photogramm</td>\n",
       "      <td>{F5F8CDA5-69E2-46F8-8F69-BA95C025B520}</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>{'type': 'MultiPolygon', 'coordinates': [[[[-7...</td>\n",
       "      <td>5078368</td>\n",
       "      <td>1970</td>\n",
       "      <td>2017-08-22T00:00:00.000</td>\n",
       "      <td>Constructed</td>\n",
       "      <td>746627</td>\n",
       "      <td>35.76</td>\n",
       "      <td>2100</td>\n",
       "      <td>51</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>5060190091</td>\n",
       "      <td>5060190091</td>\n",
       "      <td>Photogramm</td>\n",
       "      <td>{9F644794-F72C-4582-9E5E-B337E2B97068}</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>{'type': 'MultiPolygon', 'coordinates': [[[[-7...</td>\n",
       "      <td>3245111</td>\n",
       "      <td>1928</td>\n",
       "      <td>2017-08-22T00:00:00.000</td>\n",
       "      <td>Constructed</td>\n",
       "      <td>786626</td>\n",
       "      <td>37.5</td>\n",
       "      <td>2100</td>\n",
       "      <td>6</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>3086910048</td>\n",
       "      <td>3086910048</td>\n",
       "      <td>Photogramm</td>\n",
       "      <td>{F916B22D-E25B-44AE-9FA9-2A51191B9CDF}</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>{'type': 'MultiPolygon', 'coordinates': [[[[-7...</td>\n",
       "      <td>4161096</td>\n",
       "      <td>1950</td>\n",
       "      <td>2017-08-22T00:00:00.000</td>\n",
       "      <td>Constructed</td>\n",
       "      <td>746409</td>\n",
       "      <td>18.01511294</td>\n",
       "      <td>2100</td>\n",
       "      <td>93</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>4075020005</td>\n",
       "      <td>4075020005</td>\n",
       "      <td>Photogramm</td>\n",
       "      <td>{525F2C24-616B-4F29-98A3-8FEA5D4B1A7D}</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                            the_geom      bin cnstrct_yr  \\\n",
       "0  {'type': 'MultiPolygon', 'coordinates': [[[[-7...  3170958       1925   \n",
       "1  {'type': 'MultiPolygon', 'coordinates': [[[[-7...  5028452       1965   \n",
       "2  {'type': 'MultiPolygon', 'coordinates': [[[[-7...  5078368       1970   \n",
       "3  {'type': 'MultiPolygon', 'coordinates': [[[[-7...  3245111       1928   \n",
       "4  {'type': 'MultiPolygon', 'coordinates': [[[[-7...  4161096       1950   \n",
       "\n",
       "                lstmoddate   lststatype doitt_id   heightroof feat_code  \\\n",
       "0  2017-08-22T00:00:00.000  Constructed    96807  29.74985318      2100   \n",
       "1  2017-08-22T00:00:00.000  Constructed   326368        22.63      2100   \n",
       "2  2017-08-22T00:00:00.000  Constructed   746627        35.76      2100   \n",
       "3  2017-08-22T00:00:00.000  Constructed   786626         37.5      2100   \n",
       "4  2017-08-22T00:00:00.000  Constructed   746409  18.01511294      2100   \n",
       "\n",
       "  groundelev shape_area shape_len    base_bbl  mpluto_bbl  geomsource  \\\n",
       "0         40        0.0       0.0  3065220021  3065220021  Photogramm   \n",
       "1         39        0.0       0.0  5012640036  5012640036  Photogramm   \n",
       "2         51        0.0       0.0  5060190091  5060190091  Photogramm   \n",
       "3          6        0.0       0.0  3086910048  3086910048  Photogramm   \n",
       "4         93        0.0       0.0  4075020005  4075020005  Photogramm   \n",
       "\n",
       "                                 globalid  \n",
       "0  {31298F86-3088-4F53-B3DB-71A9EFA6FA1F}  \n",
       "1  {F5F8CDA5-69E2-46F8-8F69-BA95C025B520}  \n",
       "2  {9F644794-F72C-4582-9E5E-B337E2B97068}  \n",
       "3  {F916B22D-E25B-44AE-9FA9-2A51191B9CDF}  \n",
       "4  {525F2C24-616B-4F29-98A3-8FEA5D4B1A7D}  "
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# retrieving data as a dictionary \n",
    "results = client.get(\n",
    "    socrata_dataset_identifier,\n",
    "    limit=1000\n",
    ")\n",
    "\n",
    "# creating a dataframe from our dictionary\n",
    "df_api = pd.DataFrame.from_dict(results)\n",
    "\n",
    "# ending our API request\n",
    "client.close()\n",
    "\n",
    "# printing first five rows of data\n",
    "df_api.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "This dataset has 1,000 rows and 15 columns.\n"
     ]
    }
   ],
   "source": [
    "rows, columns = df_api.shape\n",
    "print('This dataset has {:,} rows and {:,} columns.'.format(rows, columns))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Useful resources:**\n",
    "    \n",
    "- API Docs: https://dev.socrata.com/foundry/data.cityofnewyork.us/i62d-kjv8\n",
    "- Sign up for app token: https://data.cityofnewyork.us/profile/edit/developer_settings\n",
    "- Python client for the Socrata Open Data API: https://github.com/xmunoz/sodapy\n",
    "- Examples: https://github.com/xmunoz/sodapy/tree/master/examples"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 4. Writing out data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>the_geom</th>\n",
       "      <th>NAME</th>\n",
       "      <th>BIN</th>\n",
       "      <th>CNSTRCT_YR</th>\n",
       "      <th>LSTMODDATE</th>\n",
       "      <th>LSTSTATYPE</th>\n",
       "      <th>DOITT_ID</th>\n",
       "      <th>HEIGHTROOF</th>\n",
       "      <th>FEAT_CODE</th>\n",
       "      <th>GROUNDELEV</th>\n",
       "      <th>SHAPE_AREA</th>\n",
       "      <th>SHAPE_LEN</th>\n",
       "      <th>BASE_BBL</th>\n",
       "      <th>MPLUTO_BBL</th>\n",
       "      <th>GEOMSOURCE</th>\n",
       "      <th>GLOBALID</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>MULTIPOLYGON (((-73.87172426474349 40.65519420...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>3394834</td>\n",
       "      <td>2011.0</td>\n",
       "      <td>08/22/2017 12:00:00 AM</td>\n",
       "      <td>Constructed</td>\n",
       "      <td>1250314</td>\n",
       "      <td>26.000000</td>\n",
       "      <td>2100</td>\n",
       "      <td>15.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>3044520924</td>\n",
       "      <td>3044520924</td>\n",
       "      <td>Other (Man</td>\n",
       "      <td>{C045C815-79DB-4644-AD9D-C34AC03D1AB4}</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>MULTIPOLYGON (((-73.86650099829305 40.74236058...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>4540159</td>\n",
       "      <td>2010.0</td>\n",
       "      <td>08/22/2017 12:00:00 AM</td>\n",
       "      <td>Constructed</td>\n",
       "      <td>201366</td>\n",
       "      <td>28.000000</td>\n",
       "      <td>2100</td>\n",
       "      <td>37.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>4018780115</td>\n",
       "      <td>4018780115</td>\n",
       "      <td>Other (Man</td>\n",
       "      <td>{FDF673E7-FF92-4A7A-AF6D-C49D77343C47}</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>MULTIPOLYGON (((-73.87805078807256 40.71475698...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>4540051</td>\n",
       "      <td>2010.0</td>\n",
       "      <td>08/22/2017 12:00:00 AM</td>\n",
       "      <td>Constructed</td>\n",
       "      <td>1171655</td>\n",
       "      <td>28.330225</td>\n",
       "      <td>2100</td>\n",
       "      <td>112.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>4030600139</td>\n",
       "      <td>4030600139</td>\n",
       "      <td>Photogramm</td>\n",
       "      <td>{788E5D72-46C1-443F-8BC9-6B97F329BFED}</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>MULTIPOLYGON (((-73.81520745135124 40.73053646...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>4545453</td>\n",
       "      <td>2012.0</td>\n",
       "      <td>08/22/2017 12:00:00 AM</td>\n",
       "      <td>Constructed</td>\n",
       "      <td>1118502</td>\n",
       "      <td>16.640000</td>\n",
       "      <td>2100</td>\n",
       "      <td>74.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>4066560052</td>\n",
       "      <td>4066560052</td>\n",
       "      <td>Photogramm</td>\n",
       "      <td>{789A5A51-5B12-46DC-AE85-B06820A3225E}</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>MULTIPOLYGON (((-73.84769179857282 40.87911947...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2118998</td>\n",
       "      <td>2012.0</td>\n",
       "      <td>08/22/2017 12:00:00 AM</td>\n",
       "      <td>Constructed</td>\n",
       "      <td>1254551</td>\n",
       "      <td>33.000000</td>\n",
       "      <td>2100</td>\n",
       "      <td>154.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2047220003</td>\n",
       "      <td>2047220003</td>\n",
       "      <td>Other (Man</td>\n",
       "      <td>{BB58FD7B-CC22-4896-901D-F8BAFF4AC129}</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                            the_geom NAME      BIN  \\\n",
       "0  MULTIPOLYGON (((-73.87172426474349 40.65519420...  NaN  3394834   \n",
       "1  MULTIPOLYGON (((-73.86650099829305 40.74236058...  NaN  4540159   \n",
       "2  MULTIPOLYGON (((-73.87805078807256 40.71475698...  NaN  4540051   \n",
       "3  MULTIPOLYGON (((-73.81520745135124 40.73053646...  NaN  4545453   \n",
       "4  MULTIPOLYGON (((-73.84769179857282 40.87911947...  NaN  2118998   \n",
       "\n",
       "   CNSTRCT_YR              LSTMODDATE   LSTSTATYPE  DOITT_ID  HEIGHTROOF  \\\n",
       "0      2011.0  08/22/2017 12:00:00 AM  Constructed   1250314   26.000000   \n",
       "1      2010.0  08/22/2017 12:00:00 AM  Constructed    201366   28.000000   \n",
       "2      2010.0  08/22/2017 12:00:00 AM  Constructed   1171655   28.330225   \n",
       "3      2012.0  08/22/2017 12:00:00 AM  Constructed   1118502   16.640000   \n",
       "4      2012.0  08/22/2017 12:00:00 AM  Constructed   1254551   33.000000   \n",
       "\n",
       "   FEAT_CODE  GROUNDELEV  SHAPE_AREA  SHAPE_LEN    BASE_BBL  MPLUTO_BBL  \\\n",
       "0       2100        15.0           0          0  3044520924  3044520924   \n",
       "1       2100        37.0           0          0  4018780115  4018780115   \n",
       "2       2100       112.0           0          0  4030600139  4030600139   \n",
       "3       2100        74.0           0          0  4066560052  4066560052   \n",
       "4       2100       154.0           0          0  2047220003  2047220003   \n",
       "\n",
       "   GEOMSOURCE                                GLOBALID  \n",
       "0  Other (Man  {C045C815-79DB-4644-AD9D-C34AC03D1AB4}  \n",
       "1  Other (Man  {FDF673E7-FF92-4A7A-AF6D-C49D77343C47}  \n",
       "2  Photogramm  {788E5D72-46C1-443F-8BC9-6B97F329BFED}  \n",
       "3  Photogramm  {789A5A51-5B12-46DC-AE85-B06820A3225E}  \n",
       "4  Other (Man  {BB58FD7B-CC22-4896-901D-F8BAFF4AC129}  "
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# read data as a dataframe\n",
    "df = pd.read_csv('data/sample-data.csv')\n",
    "\n",
    "# previewing first five rows in data\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4.1 Writing to a CSV file"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "README.md                      nta_shape.shp\r\n",
      "building-footprints-pluto.csv  nta_shape.shx\r\n",
      "nta-shape.cpg                  output.csv\r\n",
      "nta-shape.dbf                  sample-buildings.zip\r\n",
      "nta-shape.geojson              sample-data.csv\r\n",
      "nta-shape.prj                  sample-data.geojson\r\n",
      "nta-shape.shp                  sample-data.gpkg\r\n",
      "nta-shape.shx                  sample-data.json\r\n",
      "nta_shape.cpg                  sample-data.xlsx\r\n",
      "nta_shape.dbf                  \u001b[34mshapefile\u001b[m\u001b[m/\r\n",
      "nta_shape.geojson              \u001b[34munzipped-data\u001b[m\u001b[m/\r\n",
      "nta_shape.prj\r\n"
     ]
    }
   ],
   "source": [
    "# writing files as a csv\n",
    "df.to_csv('data/output.csv', index=False)\n",
    "\n",
    "# listing items in data folder\n",
    "%ls data/"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4.2 Writing to an Excel (xlsx) file"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "README.md                      nta_shape.shp\r\n",
      "building-footprints-pluto.csv  nta_shape.shx\r\n",
      "nta-shape.cpg                  output.csv\r\n",
      "nta-shape.dbf                  output.xlsx\r\n",
      "nta-shape.geojson              sample-buildings.zip\r\n",
      "nta-shape.prj                  sample-data.csv\r\n",
      "nta-shape.shp                  sample-data.geojson\r\n",
      "nta-shape.shx                  sample-data.gpkg\r\n",
      "nta_shape.cpg                  sample-data.json\r\n",
      "nta_shape.dbf                  sample-data.xlsx\r\n",
      "nta_shape.geojson              \u001b[34mshapefile\u001b[m\u001b[m/\r\n",
      "nta_shape.prj                  \u001b[34munzipped-data\u001b[m\u001b[m/\r\n"
     ]
    }
   ],
   "source": [
    "# writing files as an excel file\n",
    "df.to_excel('data/output.xlsx', index=False)\n",
    "\n",
    "# listing items in data folder\n",
    "%ls data/"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4.3 Writing to a JSON file"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "README.md                      nta_shape.shx\r\n",
      "building-footprints-pluto.csv  output.csv\r\n",
      "nta-shape.cpg                  output.json\r\n",
      "nta-shape.dbf                  output.xlsx\r\n",
      "nta-shape.geojson              sample-buildings.zip\r\n",
      "nta-shape.prj                  sample-data.csv\r\n",
      "nta-shape.shp                  sample-data.geojson\r\n",
      "nta-shape.shx                  sample-data.gpkg\r\n",
      "nta_shape.cpg                  sample-data.json\r\n",
      "nta_shape.dbf                  sample-data.xlsx\r\n",
      "nta_shape.geojson              \u001b[34mshapefile\u001b[m\u001b[m/\r\n",
      "nta_shape.prj                  \u001b[34munzipped-data\u001b[m\u001b[m/\r\n",
      "nta_shape.shp\r\n"
     ]
    }
   ],
   "source": [
    "# writing files as json\n",
    "df.to_json('data/output.json')\n",
    "\n",
    "# listing items in data folder\n",
    "%ls data/"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4.4 Writing to a Shapefile"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>the_geom</th>\n",
       "      <th>NAME</th>\n",
       "      <th>BIN</th>\n",
       "      <th>CNSTRCT_YR</th>\n",
       "      <th>LSTMODDATE</th>\n",
       "      <th>LSTSTATYPE</th>\n",
       "      <th>DOITT_ID</th>\n",
       "      <th>HEIGHTROOF</th>\n",
       "      <th>FEAT_CODE</th>\n",
       "      <th>GROUNDELEV</th>\n",
       "      <th>SHAPE_AREA</th>\n",
       "      <th>SHAPE_LEN</th>\n",
       "      <th>BASE_BBL</th>\n",
       "      <th>MPLUTO_BBL</th>\n",
       "      <th>GEOMSOURCE</th>\n",
       "      <th>GLOBALID</th>\n",
       "      <th>geometry</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>MULTIPOLYGON (((-73.87172426474349 40.65519420...</td>\n",
       "      <td>None</td>\n",
       "      <td>3394834</td>\n",
       "      <td>2011.0</td>\n",
       "      <td>08/22/2017 12:00:00 AM</td>\n",
       "      <td>Constructed</td>\n",
       "      <td>1250314</td>\n",
       "      <td>26.000000</td>\n",
       "      <td>2100</td>\n",
       "      <td>15.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>3044520924</td>\n",
       "      <td>3044520924</td>\n",
       "      <td>Other (Man</td>\n",
       "      <td>{C045C815-79DB-4644-AD9D-C34AC03D1AB4}</td>\n",
       "      <td>POLYGON ((-73.87172 40.65519, -73.87179 40.655...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>MULTIPOLYGON (((-73.86650099829305 40.74236058...</td>\n",
       "      <td>None</td>\n",
       "      <td>4540159</td>\n",
       "      <td>2010.0</td>\n",
       "      <td>08/22/2017 12:00:00 AM</td>\n",
       "      <td>Constructed</td>\n",
       "      <td>201366</td>\n",
       "      <td>28.000000</td>\n",
       "      <td>2100</td>\n",
       "      <td>37.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>4018780115</td>\n",
       "      <td>4018780115</td>\n",
       "      <td>Other (Man</td>\n",
       "      <td>{FDF673E7-FF92-4A7A-AF6D-C49D77343C47}</td>\n",
       "      <td>POLYGON ((-73.86650 40.74236, -73.86645 40.742...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>MULTIPOLYGON (((-73.87805078807256 40.71475698...</td>\n",
       "      <td>None</td>\n",
       "      <td>4540051</td>\n",
       "      <td>2010.0</td>\n",
       "      <td>08/22/2017 12:00:00 AM</td>\n",
       "      <td>Constructed</td>\n",
       "      <td>1171655</td>\n",
       "      <td>28.330225</td>\n",
       "      <td>2100</td>\n",
       "      <td>112.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>4030600139</td>\n",
       "      <td>4030600139</td>\n",
       "      <td>Photogramm</td>\n",
       "      <td>{788E5D72-46C1-443F-8BC9-6B97F329BFED}</td>\n",
       "      <td>POLYGON ((-73.87805 40.71476, -73.87787 40.714...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>MULTIPOLYGON (((-73.81520745135124 40.73053646...</td>\n",
       "      <td>None</td>\n",
       "      <td>4545453</td>\n",
       "      <td>2012.0</td>\n",
       "      <td>08/22/2017 12:00:00 AM</td>\n",
       "      <td>Constructed</td>\n",
       "      <td>1118502</td>\n",
       "      <td>16.640000</td>\n",
       "      <td>2100</td>\n",
       "      <td>74.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>4066560052</td>\n",
       "      <td>4066560052</td>\n",
       "      <td>Photogramm</td>\n",
       "      <td>{789A5A51-5B12-46DC-AE85-B06820A3225E}</td>\n",
       "      <td>POLYGON ((-73.81521 40.73054, -73.81546 40.730...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>MULTIPOLYGON (((-73.84769179857282 40.87911947...</td>\n",
       "      <td>None</td>\n",
       "      <td>2118998</td>\n",
       "      <td>2012.0</td>\n",
       "      <td>08/22/2017 12:00:00 AM</td>\n",
       "      <td>Constructed</td>\n",
       "      <td>1254551</td>\n",
       "      <td>33.000000</td>\n",
       "      <td>2100</td>\n",
       "      <td>154.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2047220003</td>\n",
       "      <td>2047220003</td>\n",
       "      <td>Other (Man</td>\n",
       "      <td>{BB58FD7B-CC22-4896-901D-F8BAFF4AC129}</td>\n",
       "      <td>POLYGON ((-73.84769 40.87912, -73.84784 40.879...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                            the_geom  NAME      BIN  \\\n",
       "0  MULTIPOLYGON (((-73.87172426474349 40.65519420...  None  3394834   \n",
       "1  MULTIPOLYGON (((-73.86650099829305 40.74236058...  None  4540159   \n",
       "2  MULTIPOLYGON (((-73.87805078807256 40.71475698...  None  4540051   \n",
       "3  MULTIPOLYGON (((-73.81520745135124 40.73053646...  None  4545453   \n",
       "4  MULTIPOLYGON (((-73.84769179857282 40.87911947...  None  2118998   \n",
       "\n",
       "   CNSTRCT_YR              LSTMODDATE   LSTSTATYPE  DOITT_ID  HEIGHTROOF  \\\n",
       "0      2011.0  08/22/2017 12:00:00 AM  Constructed   1250314   26.000000   \n",
       "1      2010.0  08/22/2017 12:00:00 AM  Constructed    201366   28.000000   \n",
       "2      2010.0  08/22/2017 12:00:00 AM  Constructed   1171655   28.330225   \n",
       "3      2012.0  08/22/2017 12:00:00 AM  Constructed   1118502   16.640000   \n",
       "4      2012.0  08/22/2017 12:00:00 AM  Constructed   1254551   33.000000   \n",
       "\n",
       "   FEAT_CODE  GROUNDELEV  SHAPE_AREA  SHAPE_LEN    BASE_BBL  MPLUTO_BBL  \\\n",
       "0       2100        15.0           0          0  3044520924  3044520924   \n",
       "1       2100        37.0           0          0  4018780115  4018780115   \n",
       "2       2100       112.0           0          0  4030600139  4030600139   \n",
       "3       2100        74.0           0          0  4066560052  4066560052   \n",
       "4       2100       154.0           0          0  2047220003  2047220003   \n",
       "\n",
       "   GEOMSOURCE                                GLOBALID  \\\n",
       "0  Other (Man  {C045C815-79DB-4644-AD9D-C34AC03D1AB4}   \n",
       "1  Other (Man  {FDF673E7-FF92-4A7A-AF6D-C49D77343C47}   \n",
       "2  Photogramm  {788E5D72-46C1-443F-8BC9-6B97F329BFED}   \n",
       "3  Photogramm  {789A5A51-5B12-46DC-AE85-B06820A3225E}   \n",
       "4  Other (Man  {BB58FD7B-CC22-4896-901D-F8BAFF4AC129}   \n",
       "\n",
       "                                            geometry  \n",
       "0  POLYGON ((-73.87172 40.65519, -73.87179 40.655...  \n",
       "1  POLYGON ((-73.86650 40.74236, -73.86645 40.742...  \n",
       "2  POLYGON ((-73.87805 40.71476, -73.87787 40.714...  \n",
       "3  POLYGON ((-73.81521 40.73054, -73.81546 40.730...  \n",
       "4  POLYGON ((-73.84769 40.87912, -73.84784 40.879...  "
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# read data as a geodataframe\n",
    "gdf = gpd.read_file('data/shapefile/sample-data.shp')\n",
    "\n",
    "# previewing first five rows in data\n",
    "gdf.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "nta-shape.cpg    nta-shape.shx    output.shp       sample-data.prj\r\n",
      "nta-shape.dbf    output.cpg       output.shx       sample-data.shp\r\n",
      "nta-shape.prj    output.dbf       sample-data.cpg  sample-data.shx\r\n",
      "nta-shape.shp    output.prj       sample-data.dbf\r\n"
     ]
    }
   ],
   "source": [
    "gdf.to_file('data/shapefile/output.shp', index=False)\n",
    "\n",
    "# listing items in data folder\n",
    "%ls data/shapefile/"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 5. Conclusion"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this notebook, we reviewed various ways to read (load) and write (save) data from NYC Open Data. Specifically, we focused on reading our data into a pandas dataframe. We also went over common file formats that you might encounter - CSV, JSON, Shapefiles, and ZIP files. In Part II, we will focus on basic data inspection and wrangling techniques in the data analysis workflow."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}