An Overview of Python's Datatable package.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The datatable is an open-source library for tabular data manipulation. It is a younger sibling of R data.table, and attempts to mimic its API and some of the core algorithms. The development of datatable is sponsored by H2O.ai."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Installation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Mac OSX\n",
    "# !pip install datatable\n",
    "\n",
    "\n",
    "# Linux\n",
    "# Python 3.5\n",
    "# !pip install https://s3.amazonaws.com/h2o-release/datatable/stable/datatable-0.8.0/datatable-0.8.0-cp35-cp35m-linux_x86_64.whl\n",
    "\n",
    "# Python 3.6\n",
    "# !pip install https://s3.amazonaws.com/h2o-release/datatable/stable/datatable-0.8.0/datatable-0.8.0-cp36-cp36m-linux_x86_64.whl\n",
    "\n",
    "# Python 3.7\n",
    "# !pip install https://s3.amazonaws.com/h2o-release/datatable/stable/datata\n",
    "\n",
    "\n",
    "# Windows\n",
    "# Currently, datatable does not work on Windows."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Importing Necessary Libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "import datatable as dt\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Dataset**: https://www.kaggle.com/wendykan/lending-club-loan-data/downloads/loan.csv/1\n",
    "\n",
    "The dataset being used has been taken from Kaggle and belongs to the Lending Club Loan Data Dataset. The dataset consists of complete loan data for all loans issued through the 2007–2015, including the current loan status (Current, Late, Fully Paid, etc.) and latest payment information. The file consists of 2.26 Million and 145 columns. The data size is ideal to demonstrate the capabilities of the datatable library."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Reading Data with datatable\n",
    "\n",
    "The fundamental unit of analysis in datatable is a Frame. It is the same notion as a pandas DataFrame or SQL table: data arranged in a two-dimensional array with rows and columns."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 33.7 s, sys: 5.16 s, total: 38.9 s                              \u001b[m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\u001b[38m\n",
      "Wall time: 23.6 s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "dt_df = dt.fread(\"data.csv\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `fread()` function shown above is both powerful and extremely fast. It can automatically detect parse parameters for the majority of text files, load data from .zip archives or URLs, read Excel files, and much more.\n",
    "\n",
    "The datatable parser\n",
    "\n",
    "* Automatically detects separators, headers, column types, quoting rules, etc.\n",
    "* Reads from file, URL, shell, raw text, archives, glob\n",
    "* Provides multi-threaded file reading for maximum speed\n",
    "* Includes a progress indicator when reading large files\n",
    "* Reads both RFC4180-compliant and non-compliant files"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Reading Data with Pandas\n",
    "\n",
    "Now, let us calculate the time taken by pandas to read in the same file."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "<string>:2: DtypeWarning: Columns (19,47,55,112,123,124,125,128,129,130,133,139,140,141) have mixed types. Specify dtype option on import or set low_memory=False.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 47.5 s, sys: 12.1 s, total: 59.6 s\n",
      "Wall time: 1min 4s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "pd_df= pd.read_csv(\"data.csv\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Converting Frame to pandas Dataframe.\n",
    "We can also convert an existing Frame into a pandas DataFrame, "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 17.1 s, sys: 4 s, total: 21.1 s\n",
      "Wall time: 21.4 s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "pandas_df = dt_df.to_pandas()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "type(dt_pd)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This shows that reading a file as datatable frame and then converting it into a pandas dataframe consumes less time."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Basic Frame Properties\n",
    "\n",
    "Let's look at some of the basic properties of frame:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(2260668, 145)\n",
      "('id', 'member_id', 'loan_amnt', 'funded_amnt', 'funded_amnt_inv')\n",
      "(stype.bool8, stype.bool8, stype.int32, stype.int32, stype.float64)\n"
     ]
    }
   ],
   "source": [
    "print(dt_df.shape)       # (nrows, ncols)\n",
    "print(dt_df.names[:5])   # top 5 column names\n",
    "print(dt_df.stypes[:5])  # column types(top 5)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<style type='text/css'>\n",
       ".datatable table.frame { margin-bottom: 0; }\n",
       ".datatable table.frame thead { border-bottom: none; }\n",
       ".datatable table.frame tr.coltypes td {  color: #FFFFFF;  line-height: 6px;  padding: 0 0.5em;}\n",
       ".datatable .bool { background: #DDDD99; }\n",
       ".datatable .obj  { background: #565656; }\n",
       ".datatable .int  { background: #5D9E5D; }\n",
       ".datatable .real { background: #4040CC; }\n",
       ".datatable .str  { background: #CC4040; }\n",
       ".datatable .row_index {  background: var(--jp-border-color3);  border-right: 1px solid var(--jp-border-color0);  color: var(--jp-ui-font-color3);  font-size: 9px;}\n",
       ".datatable .frame tr.coltypes .row_index {  background: var(--jp-border-color0);}\n",
       ".datatable th:nth-child(2) { padding-left: 12px; }\n",
       ".datatable .hellipsis {  color: var(--jp-cell-editor-border-color);}\n",
       ".datatable .vellipsis {  background: var(--jp-layout-color0);  color: var(--jp-cell-editor-border-color);}\n",
       ".datatable .na {  color: var(--jp-cell-editor-border-color);  font-size: 80%;}\n",
       ".datatable .footer { font-size: 9px; }\n",
       ".datatable .frame_dimensions {  background: var(--jp-border-color3);  border-top: 1px solid var(--jp-border-color0);  color: var(--jp-ui-font-color3);  display: inline-block;  opacity: 0.6;  padding: 1px 10px 1px 5px;}\n",
       "</style>\n",
       "<div class='datatable'>\n",
       "  <table class='frame'>\n",
       "  <thead>\n",
       "    <tr class='colnames'><td class='row_index'></td><th>id</th><th>member_id</th><th>loan_amnt</th><th>funded_amnt</th><th>funded_amnt_inv</th><th>term</th><th>int_rate</th><th>installment</th><th>grade</th><th>sub_grade</th><th class='vellipsis'>&hellip;</th><th>settlement_status</th><th>settlement_date</th><th>settlement_amount</th><th>settlement_percentage</th><th>settlement_term</th></tr>\n",
       "    <tr class='coltypes'><td class='row_index'></td><td class='bool' title='bool8'>&#x25AA;</td><td class='bool' title='bool8'>&#x25AA;</td><td class='int' title='int32'>&#x25AA;&#x25AA;&#x25AA;&#x25AA;</td><td class='int' title='int32'>&#x25AA;&#x25AA;&#x25AA;&#x25AA;</td><td class='real' title='float64'>&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;</td><td class='str' title='str32'>&#x25AA;&#x25AA;&#x25AA;&#x25AA;</td><td class='real' title='float64'>&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;</td><td class='real' title='float64'>&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;</td><td class='str' title='str32'>&#x25AA;&#x25AA;&#x25AA;&#x25AA;</td><td class='str' title='str32'>&#x25AA;&#x25AA;&#x25AA;&#x25AA;</td><th class='vellipsis'>&hellip;</th><td class='str' title='str32'>&#x25AA;&#x25AA;&#x25AA;&#x25AA;</td><td class='str' title='str32'>&#x25AA;&#x25AA;&#x25AA;&#x25AA;</td><td class='real' title='float64'>&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;</td><td class='real' title='float64'>&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;</td><td class='int' title='int32'>&#x25AA;&#x25AA;&#x25AA;&#x25AA;</td></tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr><td class='row_index'>0</td><td><span class=na>NA</span></td><td><span class=na>NA</span></td><td>2500</td><td>2500</td><td>2500</td><td>36 months</td><td>13.56</td><td>84.92</td><td>C</td><td>C1</td><td class=vellipsis>&hellip;</td><td></td><td></td><td><span class=na>NA</span></td><td><span class=na>NA</span></td><td><span class=na>NA</span></td></tr>\n",
       "    <tr><td class='row_index'>1</td><td><span class=na>NA</span></td><td><span class=na>NA</span></td><td>30000</td><td>30000</td><td>30000</td><td>60 months</td><td>18.94</td><td>777.23</td><td>D</td><td>D2</td><td class=vellipsis>&hellip;</td><td></td><td></td><td><span class=na>NA</span></td><td><span class=na>NA</span></td><td><span class=na>NA</span></td></tr>\n",
       "    <tr><td class='row_index'>2</td><td><span class=na>NA</span></td><td><span class=na>NA</span></td><td>5000</td><td>5000</td><td>5000</td><td>36 months</td><td>17.97</td><td>180.69</td><td>D</td><td>D1</td><td class=vellipsis>&hellip;</td><td></td><td></td><td><span class=na>NA</span></td><td><span class=na>NA</span></td><td><span class=na>NA</span></td></tr>\n",
       "    <tr><td class='row_index'>3</td><td><span class=na>NA</span></td><td><span class=na>NA</span></td><td>4000</td><td>4000</td><td>4000</td><td>36 months</td><td>18.94</td><td>146.51</td><td>D</td><td>D2</td><td class=vellipsis>&hellip;</td><td></td><td></td><td><span class=na>NA</span></td><td><span class=na>NA</span></td><td><span class=na>NA</span></td></tr>\n",
       "    <tr><td class='row_index'>4</td><td><span class=na>NA</span></td><td><span class=na>NA</span></td><td>30000</td><td>30000</td><td>30000</td><td>60 months</td><td>16.14</td><td>731.78</td><td>C</td><td>C4</td><td class=vellipsis>&hellip;</td><td></td><td></td><td><span class=na>NA</span></td><td><span class=na>NA</span></td><td><span class=na>NA</span></td></tr>\n",
       "    <tr><td class='row_index'>5</td><td><span class=na>NA</span></td><td><span class=na>NA</span></td><td>5550</td><td>5550</td><td>5550</td><td>36 months</td><td>15.02</td><td>192.45</td><td>C</td><td>C3</td><td class=vellipsis>&hellip;</td><td></td><td></td><td><span class=na>NA</span></td><td><span class=na>NA</span></td><td><span class=na>NA</span></td></tr>\n",
       "    <tr><td class='row_index'>6</td><td><span class=na>NA</span></td><td><span class=na>NA</span></td><td>2000</td><td>2000</td><td>2000</td><td>36 months</td><td>17.97</td><td>72.28</td><td>D</td><td>D1</td><td class=vellipsis>&hellip;</td><td></td><td></td><td><span class=na>NA</span></td><td><span class=na>NA</span></td><td><span class=na>NA</span></td></tr>\n",
       "    <tr><td class='row_index'>7</td><td><span class=na>NA</span></td><td><span class=na>NA</span></td><td>6000</td><td>6000</td><td>6000</td><td>36 months</td><td>13.56</td><td>203.79</td><td>C</td><td>C1</td><td class=vellipsis>&hellip;</td><td></td><td></td><td><span class=na>NA</span></td><td><span class=na>NA</span></td><td><span class=na>NA</span></td></tr>\n",
       "    <tr><td class='row_index'>8</td><td><span class=na>NA</span></td><td><span class=na>NA</span></td><td>5000</td><td>5000</td><td>5000</td><td>36 months</td><td>17.97</td><td>180.69</td><td>D</td><td>D1</td><td class=vellipsis>&hellip;</td><td></td><td></td><td><span class=na>NA</span></td><td><span class=na>NA</span></td><td><span class=na>NA</span></td></tr>\n",
       "    <tr><td class='row_index'>9</td><td><span class=na>NA</span></td><td><span class=na>NA</span></td><td>6000</td><td>6000</td><td>6000</td><td>36 months</td><td>14.47</td><td>206.44</td><td>C</td><td>C2</td><td class=vellipsis>&hellip;</td><td></td><td></td><td><span class=na>NA</span></td><td><span class=na>NA</span></td><td><span class=na>NA</span></td></tr>\n",
       "  </tbody>\n",
       "  </table>\n",
       "  <div class='footer'>\n",
       "    <div class='frame_dimensions'>10 rows &times; 145 columns</div>\n",
       "  </div>\n",
       "</div>\n"
      ],
      "text/plain": []
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dt_df.head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Summary Stats\n",
    "Calculating the summary stats in pandas is a memory consuming process but not anymore with datatable. We can compute the following per-column summary stats using datatable:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### mean of all the columns\n",
    "\n",
    "#### Using datatable"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 5.11 s, sys: 51.8 ms, total: 5.16 s\n",
      "Wall time: 1.43 s\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div class='datatable'>\n",
       "  <table class='frame'>\n",
       "  <thead>\n",
       "    <tr class='colnames'><td class='row_index'></td><th>id</th><th>member_id</th><th>loan_amnt</th><th>funded_amnt</th><th>funded_amnt_inv</th><th>term</th><th>int_rate</th><th>installment</th><th>grade</th><th>sub_grade</th><th class='vellipsis'>&hellip;</th><th>settlement_status</th><th>settlement_date</th><th>settlement_amount</th><th>settlement_percentage</th><th>settlement_term</th></tr>\n",
       "    <tr class='coltypes'><td class='row_index'></td><td class='real' title='float64'>&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;</td><td class='real' title='float64'>&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;</td><td class='real' title='float64'>&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;</td><td class='real' title='float64'>&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;</td><td class='real' title='float64'>&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;</td><td class='real' title='float64'>&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;</td><td class='real' title='float64'>&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;</td><td class='real' title='float64'>&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;</td><td class='real' title='float64'>&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;</td><td class='real' title='float64'>&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;</td><th class='vellipsis'>&hellip;</th><td class='real' title='float64'>&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;</td><td class='real' title='float64'>&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;</td><td class='real' title='float64'>&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;</td><td class='real' title='float64'>&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;</td><td class='real' title='float64'>&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;</td></tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr><td class='row_index'>0</td><td><span class=na>NA</span></td><td><span class=na>NA</span></td><td>15046.9</td><td>15041.7</td><td>15023.4</td><td><span class=na>NA</span></td><td>13.0929</td><td>445.808</td><td><span class=na>NA</span></td><td><span class=na>NA</span></td><td class=vellipsis>&hellip;</td><td><span class=na>NA</span></td><td><span class=na>NA</span></td><td>5030.61</td><td>47.7756</td><td>13.1486</td></tr>\n",
       "  </tbody>\n",
       "  </table>\n",
       "  <div class='footer'>\n",
       "    <div class='frame_dimensions'>1 row &times; 145 columns</div>\n",
       "  </div>\n",
       "</div>\n"
      ],
      "text/plain": []
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "%%time\n",
    "dt_df.mean()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Using pandas"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%time\n",
    "#pd_df.mean()\n",
    "#Throws memory error"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### max of every column"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div class='datatable'>\n",
       "  <table class='frame'>\n",
       "  <thead>\n",
       "    <tr class='colnames'><td class='row_index'></td><th>id</th><th>member_id</th><th>loan_amnt</th><th>funded_amnt</th><th>funded_amnt_inv</th><th>term</th><th>int_rate</th><th>installment</th><th>grade</th><th>sub_grade</th><th class='vellipsis'>&hellip;</th><th>settlement_status</th><th>settlement_date</th><th>settlement_amount</th><th>settlement_percentage</th><th>settlement_term</th></tr>\n",
       "    <tr class='coltypes'><td class='row_index'></td><td class='bool' title='bool8'>&#x25AA;</td><td class='bool' title='bool8'>&#x25AA;</td><td class='int' title='int32'>&#x25AA;&#x25AA;&#x25AA;&#x25AA;</td><td class='int' title='int32'>&#x25AA;&#x25AA;&#x25AA;&#x25AA;</td><td class='real' title='float64'>&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;</td><td class='str' title='str32'>&#x25AA;&#x25AA;&#x25AA;&#x25AA;</td><td class='real' title='float64'>&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;</td><td class='real' title='float64'>&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;</td><td class='str' title='str32'>&#x25AA;&#x25AA;&#x25AA;&#x25AA;</td><td class='str' title='str32'>&#x25AA;&#x25AA;&#x25AA;&#x25AA;</td><th class='vellipsis'>&hellip;</th><td class='str' title='str32'>&#x25AA;&#x25AA;&#x25AA;&#x25AA;</td><td class='str' title='str32'>&#x25AA;&#x25AA;&#x25AA;&#x25AA;</td><td class='real' title='float64'>&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;</td><td class='real' title='float64'>&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;</td><td class='int' title='int32'>&#x25AA;&#x25AA;&#x25AA;&#x25AA;</td></tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr><td class='row_index'>0</td><td><span class=na>NA</span></td><td><span class=na>NA</span></td><td>40000</td><td>40000</td><td>40000</td><td><span class=na>NA</span></td><td>30.99</td><td>1719.83</td><td><span class=na>NA</span></td><td><span class=na>NA</span></td><td class=vellipsis>&hellip;</td><td><span class=na>NA</span></td><td><span class=na>NA</span></td><td>33601</td><td>521.35</td><td>181</td></tr>\n",
       "  </tbody>\n",
       "  </table>\n",
       "  <div class='footer'>\n",
       "    <div class='frame_dimensions'>1 row &times; 145 columns</div>\n",
       "  </div>\n",
       "</div>\n"
      ],
      "text/plain": []
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dt_df.max()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Number of Unique values in every column"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div class='datatable'>\n",
       "  <table class='frame'>\n",
       "  <thead>\n",
       "    <tr class='colnames'><td class='row_index'></td><th>id</th><th>member_id</th><th>loan_amnt</th><th>funded_amnt</th><th>funded_amnt_inv</th><th>term</th><th>int_rate</th><th>installment</th><th>grade</th><th>sub_grade</th><th class='vellipsis'>&hellip;</th><th>settlement_status</th><th>settlement_date</th><th>settlement_amount</th><th>settlement_percentage</th><th>settlement_term</th></tr>\n",
       "    <tr class='coltypes'><td class='row_index'></td><td class='int' title='int64'>&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;</td><td class='int' title='int64'>&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;</td><td class='int' title='int64'>&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;</td><td class='int' title='int64'>&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;</td><td class='int' title='int64'>&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;</td><td class='int' title='int64'>&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;</td><td class='int' title='int64'>&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;</td><td class='int' title='int64'>&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;</td><td class='int' title='int64'>&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;</td><td class='int' title='int64'>&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;</td><th class='vellipsis'>&hellip;</th><td class='int' title='int64'>&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;</td><td class='int' title='int64'>&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;</td><td class='int' title='int64'>&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;</td><td class='int' title='int64'>&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;</td><td class='int' title='int64'>&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;</td></tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr><td class='row_index'>0</td><td>0</td><td>0</td><td>1572</td><td>1572</td><td>10058</td><td>2</td><td>673</td><td>93296</td><td>7</td><td>35</td><td class=vellipsis>&hellip;</td><td>4</td><td>90</td><td>21519</td><td>2045</td><td>40</td></tr>\n",
       "  </tbody>\n",
       "  </table>\n",
       "  <div class='footer'>\n",
       "    <div class='frame_dimensions'>1 row &times; 145 columns</div>\n",
       "  </div>\n",
       "</div>\n"
      ],
      "text/plain": []
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "\n",
    "dt_df.nunique()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Data Manipulation\n",
    "Data Tables like dataframes are columnnar data structures.In datatable, the primary vehicle for all these operations is the square-bracket notation inspired by traditional matrix indexing but with more functionalities.\n",
    "\n",
    "![](https://i.imgur.com/LMjIWLm.png)\n",
    "\n",
    "The same DT[i, j] notation is used in mathematics when indexing matrices, in C/C++, in R, in pandas, in numpy, etc"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Selecting Subsets of Rows/Columns\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div class='datatable'>\n",
       "  <table class='frame'>\n",
       "  <thead>\n",
       "    <tr class='colnames'><td class='row_index'></td><th>funded_amnt</th></tr>\n",
       "    <tr class='coltypes'><td class='row_index'></td><td class='int' title='int32'>&#x25AA;&#x25AA;&#x25AA;&#x25AA;</td></tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr><td class='row_index'>0</td><td>2500</td></tr>\n",
       "    <tr><td class='row_index'>1</td><td>30000</td></tr>\n",
       "    <tr><td class='row_index'>2</td><td>5000</td></tr>\n",
       "    <tr><td class='row_index'>3</td><td>4000</td></tr>\n",
       "    <tr><td class='row_index'>4</td><td>30000</td></tr>\n",
       "    <tr><td class='row_index'>5</td><td>5550</td></tr>\n",
       "    <tr><td class='row_index'>6</td><td>2000</td></tr>\n",
       "    <tr><td class='row_index'>7</td><td>6000</td></tr>\n",
       "    <tr><td class='row_index'>8</td><td>5000</td></tr>\n",
       "    <tr><td class='row_index'>9</td><td>6000</td></tr>\n",
       "    <tr><td class='row_index'>&#x22EE;</td><td class='hellipsis'>&#x22EE;</td></tr>\n",
       "    <tr><td class='row_index'>2,260,663</td><td>12000</td></tr>\n",
       "    <tr><td class='row_index'>2,260,664</td><td>12000</td></tr>\n",
       "    <tr><td class='row_index'>2,260,665</td><td>10000</td></tr>\n",
       "    <tr><td class='row_index'>2,260,666</td><td>12000</td></tr>\n",
       "    <tr><td class='row_index'>2,260,667</td><td>16550</td></tr>\n",
       "  </tbody>\n",
       "  </table>\n",
       "  <div class='footer'>\n",
       "    <div class='frame_dimensions'>2,260,668 rows &times; 1 column</div>\n",
       "  </div>\n",
       "</div>\n"
      ],
      "text/plain": []
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dt_df[:,'funded_amnt'] # selecting the first column"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div class='datatable'>\n",
       "  <table class='frame'>\n",
       "  <thead>\n",
       "    <tr class='colnames'><td class='row_index'></td><th>id</th><th>member_id</th><th>loan_amnt</th></tr>\n",
       "    <tr class='coltypes'><td class='row_index'></td><td class='bool' title='bool8'>&#x25AA;</td><td class='bool' title='bool8'>&#x25AA;</td><td class='int' title='int32'>&#x25AA;&#x25AA;&#x25AA;&#x25AA;</td></tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr><td class='row_index'>0</td><td><span class=na>NA</span></td><td><span class=na>NA</span></td><td>2500</td></tr>\n",
       "    <tr><td class='row_index'>1</td><td><span class=na>NA</span></td><td><span class=na>NA</span></td><td>30000</td></tr>\n",
       "    <tr><td class='row_index'>2</td><td><span class=na>NA</span></td><td><span class=na>NA</span></td><td>5000</td></tr>\n",
       "    <tr><td class='row_index'>3</td><td><span class=na>NA</span></td><td><span class=na>NA</span></td><td>4000</td></tr>\n",
       "    <tr><td class='row_index'>4</td><td><span class=na>NA</span></td><td><span class=na>NA</span></td><td>30000</td></tr>\n",
       "  </tbody>\n",
       "  </table>\n",
       "  <div class='footer'>\n",
       "    <div class='frame_dimensions'>5 rows &times; 3 columns</div>\n",
       "  </div>\n",
       "</div>\n"
      ],
      "text/plain": []
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dt_df[:5,:3] #first 5 rows and first 3 columns"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Sorting the Frame\n",
    "### with datatable"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 534 ms, sys: 67.9 ms, total: 602 ms\n",
      "Wall time: 179 ms\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div class='datatable'>\n",
       "  <table class='frame'>\n",
       "  <thead>\n",
       "    <tr class='colnames'><td class='row_index'></td><th>id</th><th>member_id</th><th>loan_amnt</th><th>funded_amnt</th><th>funded_amnt_inv</th><th>term</th><th>int_rate</th><th>installment</th><th>grade</th><th>sub_grade</th><th class='vellipsis'>&hellip;</th><th>settlement_status</th><th>settlement_date</th><th>settlement_amount</th><th>settlement_percentage</th><th>settlement_term</th></tr>\n",
       "    <tr class='coltypes'><td class='row_index'></td><td class='bool' title='bool8'>&#x25AA;</td><td class='bool' title='bool8'>&#x25AA;</td><td class='int' title='int32'>&#x25AA;&#x25AA;&#x25AA;&#x25AA;</td><td class='int' title='int32'>&#x25AA;&#x25AA;&#x25AA;&#x25AA;</td><td class='real' title='float64'>&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;</td><td class='str' title='str32'>&#x25AA;&#x25AA;&#x25AA;&#x25AA;</td><td class='real' title='float64'>&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;</td><td class='real' title='float64'>&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;</td><td class='str' title='str32'>&#x25AA;&#x25AA;&#x25AA;&#x25AA;</td><td class='str' title='str32'>&#x25AA;&#x25AA;&#x25AA;&#x25AA;</td><th class='vellipsis'>&hellip;</th><td class='str' title='str32'>&#x25AA;&#x25AA;&#x25AA;&#x25AA;</td><td class='str' title='str32'>&#x25AA;&#x25AA;&#x25AA;&#x25AA;</td><td class='real' title='float64'>&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;</td><td class='real' title='float64'>&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;</td><td class='int' title='int32'>&#x25AA;&#x25AA;&#x25AA;&#x25AA;</td></tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr><td class='row_index'>0</td><td><span class=na>NA</span></td><td><span class=na>NA</span></td><td>1775</td><td>1775</td><td>0</td><td>36 months</td><td>12.72</td><td>59.57</td><td>C</td><td>C4</td><td class=vellipsis>&hellip;</td><td></td><td></td><td><span class=na>NA</span></td><td><span class=na>NA</span></td><td><span class=na>NA</span></td></tr>\n",
       "    <tr><td class='row_index'>1</td><td><span class=na>NA</span></td><td><span class=na>NA</span></td><td>7500</td><td>7500</td><td>0</td><td>36 months</td><td>10.01</td><td>242.04</td><td>B</td><td>B2</td><td class=vellipsis>&hellip;</td><td></td><td></td><td><span class=na>NA</span></td><td><span class=na>NA</span></td><td><span class=na>NA</span></td></tr>\n",
       "    <tr><td class='row_index'>2</td><td><span class=na>NA</span></td><td><span class=na>NA</span></td><td>4000</td><td>4000</td><td>0</td><td>36 months</td><td>7.68</td><td>124.76</td><td>A</td><td>A2</td><td class=vellipsis>&hellip;</td><td></td><td></td><td><span class=na>NA</span></td><td><span class=na>NA</span></td><td><span class=na>NA</span></td></tr>\n",
       "    <tr><td class='row_index'>3</td><td><span class=na>NA</span></td><td><span class=na>NA</span></td><td>3200</td><td>2100</td><td>0</td><td>36 months</td><td>7.68</td><td>65.5</td><td>A</td><td>A2</td><td class=vellipsis>&hellip;</td><td></td><td></td><td><span class=na>NA</span></td><td><span class=na>NA</span></td><td><span class=na>NA</span></td></tr>\n",
       "    <tr><td class='row_index'>4</td><td><span class=na>NA</span></td><td><span class=na>NA</span></td><td>7500</td><td>7500</td><td>0</td><td>36 months</td><td>11.59</td><td>247.65</td><td>C</td><td>C2</td><td class=vellipsis>&hellip;</td><td></td><td></td><td><span class=na>NA</span></td><td><span class=na>NA</span></td><td><span class=na>NA</span></td></tr>\n",
       "    <tr><td class='row_index'>5</td><td><span class=na>NA</span></td><td><span class=na>NA</span></td><td>7000</td><td>6100</td><td>0</td><td>36 months</td><td>11.28</td><td>200.52</td><td>C</td><td>C1</td><td class=vellipsis>&hellip;</td><td></td><td></td><td><span class=na>NA</span></td><td><span class=na>NA</span></td><td><span class=na>NA</span></td></tr>\n",
       "    <tr><td class='row_index'>6</td><td><span class=na>NA</span></td><td><span class=na>NA</span></td><td>7500</td><td>4750</td><td>0</td><td>36 months</td><td>11.28</td><td>156.14</td><td>C</td><td>C1</td><td class=vellipsis>&hellip;</td><td></td><td></td><td><span class=na>NA</span></td><td><span class=na>NA</span></td><td><span class=na>NA</span></td></tr>\n",
       "    <tr><td class='row_index'>7</td><td><span class=na>NA</span></td><td><span class=na>NA</span></td><td>5600</td><td>4300</td><td>0</td><td>36 months</td><td>8.32</td><td>135.39</td><td>A</td><td>A4</td><td class=vellipsis>&hellip;</td><td></td><td></td><td><span class=na>NA</span></td><td><span class=na>NA</span></td><td><span class=na>NA</span></td></tr>\n",
       "    <tr><td class='row_index'>8</td><td><span class=na>NA</span></td><td><span class=na>NA</span></td><td>7500</td><td>4800</td><td>0</td><td>36 months</td><td>11.59</td><td>158.5</td><td>C</td><td>C2</td><td class=vellipsis>&hellip;</td><td></td><td></td><td><span class=na>NA</span></td><td><span class=na>NA</span></td><td><span class=na>NA</span></td></tr>\n",
       "    <tr><td class='row_index'>9</td><td><span class=na>NA</span></td><td><span class=na>NA</span></td><td>7500</td><td>5550</td><td>0</td><td>36 months</td><td>8.32</td><td>174.74</td><td>A</td><td>A4</td><td class=vellipsis>&hellip;</td><td></td><td></td><td><span class=na>NA</span></td><td><span class=na>NA</span></td><td><span class=na>NA</span></td></tr>\n",
       "    <tr><td class='row_index'>&#x22EE;</td><td class='hellipsis'>&#x22EE;</td><td class='hellipsis'>&#x22EE;</td><td class='hellipsis'>&#x22EE;</td><td class='hellipsis'>&#x22EE;</td><td class='hellipsis'>&#x22EE;</td><td class='hellipsis'>&#x22EE;</td><td class='hellipsis'>&#x22EE;</td><td class='hellipsis'>&#x22EE;</td><td class='hellipsis'>&#x22EE;</td><td class='hellipsis'>&#x22EE;</td><td class='hellipsis'>&#x22F1;</td><td class='hellipsis'>&#x22EE;</td><td class='hellipsis'>&#x22EE;</td><td class='hellipsis'>&#x22EE;</td><td class='hellipsis'>&#x22EE;</td><td class='hellipsis'>&#x22EE;</td></tr>\n",
       "    <tr><td class='row_index'>2,260,663</td><td><span class=na>NA</span></td><td><span class=na>NA</span></td><td>40000</td><td>40000</td><td>40000</td><td>36 months</td><td>9.44</td><td>1280.2</td><td>B</td><td>B1</td><td class=vellipsis>&hellip;</td><td></td><td></td><td><span class=na>NA</span></td><td><span class=na>NA</span></td><td><span class=na>NA</span></td></tr>\n",
       "    <tr><td class='row_index'>2,260,664</td><td><span class=na>NA</span></td><td><span class=na>NA</span></td><td>40000</td><td>40000</td><td>40000</td><td>36 months</td><td>9.44</td><td>1280.2</td><td>B</td><td>B1</td><td class=vellipsis>&hellip;</td><td></td><td></td><td><span class=na>NA</span></td><td><span class=na>NA</span></td><td><span class=na>NA</span></td></tr>\n",
       "    <tr><td class='row_index'>2,260,665</td><td><span class=na>NA</span></td><td><span class=na>NA</span></td><td>40000</td><td>40000</td><td>40000</td><td>60 months</td><td>20</td><td>1059.76</td><td>D</td><td>D4</td><td class=vellipsis>&hellip;</td><td></td><td></td><td><span class=na>NA</span></td><td><span class=na>NA</span></td><td><span class=na>NA</span></td></tr>\n",
       "    <tr><td class='row_index'>2,260,666</td><td><span class=na>NA</span></td><td><span class=na>NA</span></td><td>40000</td><td>40000</td><td>40000</td><td>36 months</td><td>10.42</td><td>1298.59</td><td>B</td><td>B3</td><td class=vellipsis>&hellip;</td><td></td><td></td><td><span class=na>NA</span></td><td><span class=na>NA</span></td><td><span class=na>NA</span></td></tr>\n",
       "    <tr><td class='row_index'>2,260,667</td><td><span class=na>NA</span></td><td><span class=na>NA</span></td><td>40000</td><td>40000</td><td>40000</td><td>36 months</td><td>10.91</td><td>1307.85</td><td>B</td><td>B4</td><td class=vellipsis>&hellip;</td><td></td><td></td><td><span class=na>NA</span></td><td><span class=na>NA</span></td><td><span class=na>NA</span></td></tr>\n",
       "  </tbody>\n",
       "  </table>\n",
       "  <div class='footer'>\n",
       "    <div class='frame_dimensions'>2,260,668 rows &times; 145 columns</div>\n",
       "  </div>\n",
       "</div>\n"
      ],
      "text/plain": []
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "%%time\n",
    "dt_df.sort('funded_amnt_inv')\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### with pandas"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 8.76 s, sys: 2.87 s, total: 11.6 s\n",
      "Wall time: 12.4 s\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>member_id</th>\n",
       "      <th>loan_amnt</th>\n",
       "      <th>funded_amnt</th>\n",
       "      <th>funded_amnt_inv</th>\n",
       "      <th>term</th>\n",
       "      <th>int_rate</th>\n",
       "      <th>installment</th>\n",
       "      <th>grade</th>\n",
       "      <th>sub_grade</th>\n",
       "      <th>...</th>\n",
       "      <th>hardship_payoff_balance_amount</th>\n",
       "      <th>hardship_last_payment_amount</th>\n",
       "      <th>disbursement_method</th>\n",
       "      <th>debt_settlement_flag</th>\n",
       "      <th>debt_settlement_flag_date</th>\n",
       "      <th>settlement_status</th>\n",
       "      <th>settlement_date</th>\n",
       "      <th>settlement_amount</th>\n",
       "      <th>settlement_percentage</th>\n",
       "      <th>settlement_term</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2138210</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>5500</td>\n",
       "      <td>5500</td>\n",
       "      <td>0.0</td>\n",
       "      <td>36 months</td>\n",
       "      <td>9.76</td>\n",
       "      <td>176.86</td>\n",
       "      <td>B</td>\n",
       "      <td>B2</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Cash</td>\n",
       "      <td>N</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2138169</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>7500</td>\n",
       "      <td>7500</td>\n",
       "      <td>0.0</td>\n",
       "      <td>36 months</td>\n",
       "      <td>11.97</td>\n",
       "      <td>249.00</td>\n",
       "      <td>C</td>\n",
       "      <td>C4</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Cash</td>\n",
       "      <td>N</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2141012</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>7225</td>\n",
       "      <td>5300</td>\n",
       "      <td>0.0</td>\n",
       "      <td>36 months</td>\n",
       "      <td>14.18</td>\n",
       "      <td>181.61</td>\n",
       "      <td>E</td>\n",
       "      <td>E1</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Cash</td>\n",
       "      <td>N</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2141011</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>7500</td>\n",
       "      <td>7500</td>\n",
       "      <td>0.0</td>\n",
       "      <td>36 months</td>\n",
       "      <td>11.03</td>\n",
       "      <td>245.65</td>\n",
       "      <td>C</td>\n",
       "      <td>C1</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Cash</td>\n",
       "      <td>N</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2138262</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>7000</td>\n",
       "      <td>7000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>36 months</td>\n",
       "      <td>8.00</td>\n",
       "      <td>219.36</td>\n",
       "      <td>A</td>\n",
       "      <td>A3</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Cash</td>\n",
       "      <td>N</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2138261</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>3500</td>\n",
       "      <td>3500</td>\n",
       "      <td>0.0</td>\n",
       "      <td>36 months</td>\n",
       "      <td>8.32</td>\n",
       "      <td>110.20</td>\n",
       "      <td>A</td>\n",
       "      <td>A4</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Cash</td>\n",
       "      <td>N</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2141003</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>4000</td>\n",
       "      <td>4000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>36 months</td>\n",
       "      <td>18.54</td>\n",
       "      <td>145.70</td>\n",
       "      <td>G</td>\n",
       "      <td>G4</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Cash</td>\n",
       "      <td>N</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2140997</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2400</td>\n",
       "      <td>2400</td>\n",
       "      <td>0.0</td>\n",
       "      <td>36 months</td>\n",
       "      <td>13.17</td>\n",
       "      <td>81.07</td>\n",
       "      <td>D</td>\n",
       "      <td>D2</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Cash</td>\n",
       "      <td>N</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2140994</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>6000</td>\n",
       "      <td>6000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>36 months</td>\n",
       "      <td>14.12</td>\n",
       "      <td>205.42</td>\n",
       "      <td>D</td>\n",
       "      <td>D5</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Cash</td>\n",
       "      <td>N</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2138172</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>5000</td>\n",
       "      <td>5000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>36 months</td>\n",
       "      <td>10.08</td>\n",
       "      <td>161.53</td>\n",
       "      <td>B</td>\n",
       "      <td>B3</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Cash</td>\n",
       "      <td>N</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2140993</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>7500</td>\n",
       "      <td>7500</td>\n",
       "      <td>0.0</td>\n",
       "      <td>36 months</td>\n",
       "      <td>16.01</td>\n",
       "      <td>263.72</td>\n",
       "      <td>F</td>\n",
       "      <td>F1</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Cash</td>\n",
       "      <td>N</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2140992</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>7000</td>\n",
       "      <td>7000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>36 months</td>\n",
       "      <td>14.12</td>\n",
       "      <td>239.66</td>\n",
       "      <td>D</td>\n",
       "      <td>D5</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Cash</td>\n",
       "      <td>N</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2138257</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>6000</td>\n",
       "      <td>4725</td>\n",
       "      <td>0.0</td>\n",
       "      <td>36 months</td>\n",
       "      <td>10.39</td>\n",
       "      <td>153.33</td>\n",
       "      <td>B</td>\n",
       "      <td>B4</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Cash</td>\n",
       "      <td>N</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2140987</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>5000</td>\n",
       "      <td>5000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>36 months</td>\n",
       "      <td>9.70</td>\n",
       "      <td>160.64</td>\n",
       "      <td>B</td>\n",
       "      <td>B1</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Cash</td>\n",
       "      <td>N</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2140986</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>4500</td>\n",
       "      <td>4500</td>\n",
       "      <td>0.0</td>\n",
       "      <td>36 months</td>\n",
       "      <td>12.54</td>\n",
       "      <td>150.63</td>\n",
       "      <td>C</td>\n",
       "      <td>C5</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Cash</td>\n",
       "      <td>N</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2137960</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>7500</td>\n",
       "      <td>4750</td>\n",
       "      <td>0.0</td>\n",
       "      <td>36 months</td>\n",
       "      <td>11.28</td>\n",
       "      <td>156.14</td>\n",
       "      <td>C</td>\n",
       "      <td>C1</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Cash</td>\n",
       "      <td>N</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2140984</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>11000</td>\n",
       "      <td>11000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>36 months</td>\n",
       "      <td>11.91</td>\n",
       "      <td>364.89</td>\n",
       "      <td>C</td>\n",
       "      <td>C3</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Cash</td>\n",
       "      <td>N</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2138133</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1600</td>\n",
       "      <td>1600</td>\n",
       "      <td>0.0</td>\n",
       "      <td>36 months</td>\n",
       "      <td>11.66</td>\n",
       "      <td>52.89</td>\n",
       "      <td>C</td>\n",
       "      <td>C3</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Cash</td>\n",
       "      <td>N</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2138254</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>3500</td>\n",
       "      <td>3500</td>\n",
       "      <td>0.0</td>\n",
       "      <td>36 months</td>\n",
       "      <td>7.68</td>\n",
       "      <td>109.17</td>\n",
       "      <td>A</td>\n",
       "      <td>A2</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Cash</td>\n",
       "      <td>N</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2140978</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>15000</td>\n",
       "      <td>15000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>36 months</td>\n",
       "      <td>11.91</td>\n",
       "      <td>497.58</td>\n",
       "      <td>C</td>\n",
       "      <td>C3</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Cash</td>\n",
       "      <td>N</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2140977</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>4000</td>\n",
       "      <td>4000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>36 months</td>\n",
       "      <td>17.91</td>\n",
       "      <td>144.43</td>\n",
       "      <td>G</td>\n",
       "      <td>G2</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Cash</td>\n",
       "      <td>N</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2138249</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1500</td>\n",
       "      <td>1175</td>\n",
       "      <td>0.0</td>\n",
       "      <td>36 months</td>\n",
       "      <td>7.37</td>\n",
       "      <td>36.48</td>\n",
       "      <td>A</td>\n",
       "      <td>A1</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Cash</td>\n",
       "      <td>N</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2138111</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>3000</td>\n",
       "      <td>3000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>36 months</td>\n",
       "      <td>8.00</td>\n",
       "      <td>94.01</td>\n",
       "      <td>A</td>\n",
       "      <td>A3</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Cash</td>\n",
       "      <td>N</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2137974</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>5600</td>\n",
       "      <td>4300</td>\n",
       "      <td>0.0</td>\n",
       "      <td>36 months</td>\n",
       "      <td>8.32</td>\n",
       "      <td>135.39</td>\n",
       "      <td>A</td>\n",
       "      <td>A4</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Cash</td>\n",
       "      <td>N</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2137975</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>7500</td>\n",
       "      <td>4800</td>\n",
       "      <td>0.0</td>\n",
       "      <td>36 months</td>\n",
       "      <td>11.59</td>\n",
       "      <td>158.50</td>\n",
       "      <td>C</td>\n",
       "      <td>C2</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Cash</td>\n",
       "      <td>N</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2137977</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>7500</td>\n",
       "      <td>5550</td>\n",
       "      <td>0.0</td>\n",
       "      <td>36 months</td>\n",
       "      <td>8.32</td>\n",
       "      <td>174.74</td>\n",
       "      <td>A</td>\n",
       "      <td>A4</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Cash</td>\n",
       "      <td>N</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2138106</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>7000</td>\n",
       "      <td>7000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>36 months</td>\n",
       "      <td>8.00</td>\n",
       "      <td>219.36</td>\n",
       "      <td>A</td>\n",
       "      <td>A3</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Cash</td>\n",
       "      <td>N</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2137988</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>6000</td>\n",
       "      <td>5100</td>\n",
       "      <td>0.0</td>\n",
       "      <td>36 months</td>\n",
       "      <td>8.00</td>\n",
       "      <td>159.82</td>\n",
       "      <td>A</td>\n",
       "      <td>A3</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Cash</td>\n",
       "      <td>N</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2138162</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>7500</td>\n",
       "      <td>7500</td>\n",
       "      <td>0.0</td>\n",
       "      <td>36 months</td>\n",
       "      <td>10.39</td>\n",
       "      <td>243.38</td>\n",
       "      <td>B</td>\n",
       "      <td>B4</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Cash</td>\n",
       "      <td>N</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2138163</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>5000</td>\n",
       "      <td>5000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>36 months</td>\n",
       "      <td>12.29</td>\n",
       "      <td>166.77</td>\n",
       "      <td>C</td>\n",
       "      <td>C5</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Cash</td>\n",
       "      <td>N</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1605588</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>40000</td>\n",
       "      <td>40000</td>\n",
       "      <td>40000.0</td>\n",
       "      <td>36 months</td>\n",
       "      <td>22.91</td>\n",
       "      <td>1546.52</td>\n",
       "      <td>E</td>\n",
       "      <td>E1</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Cash</td>\n",
       "      <td>N</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>370115</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>40000</td>\n",
       "      <td>40000</td>\n",
       "      <td>40000.0</td>\n",
       "      <td>36 months</td>\n",
       "      <td>5.31</td>\n",
       "      <td>1204.42</td>\n",
       "      <td>A</td>\n",
       "      <td>A1</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Cash</td>\n",
       "      <td>N</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>370116</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>40000</td>\n",
       "      <td>40000</td>\n",
       "      <td>40000.0</td>\n",
       "      <td>60 months</td>\n",
       "      <td>18.45</td>\n",
       "      <td>1025.56</td>\n",
       "      <td>D</td>\n",
       "      <td>D2</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Cash</td>\n",
       "      <td>N</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>53289</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>40000</td>\n",
       "      <td>40000</td>\n",
       "      <td>40000.0</td>\n",
       "      <td>36 months</td>\n",
       "      <td>11.80</td>\n",
       "      <td>1324.76</td>\n",
       "      <td>B</td>\n",
       "      <td>B4</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Cash</td>\n",
       "      <td>N</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>182253</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>40000</td>\n",
       "      <td>40000</td>\n",
       "      <td>40000.0</td>\n",
       "      <td>36 months</td>\n",
       "      <td>11.55</td>\n",
       "      <td>1320.00</td>\n",
       "      <td>B</td>\n",
       "      <td>B4</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Cash</td>\n",
       "      <td>N</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>489926</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>40000</td>\n",
       "      <td>40000</td>\n",
       "      <td>40000.0</td>\n",
       "      <td>36 months</td>\n",
       "      <td>9.44</td>\n",
       "      <td>1280.20</td>\n",
       "      <td>B</td>\n",
       "      <td>B1</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Cash</td>\n",
       "      <td>N</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>489922</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>40000</td>\n",
       "      <td>40000</td>\n",
       "      <td>40000.0</td>\n",
       "      <td>60 months</td>\n",
       "      <td>12.62</td>\n",
       "      <td>902.37</td>\n",
       "      <td>C</td>\n",
       "      <td>C1</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Cash</td>\n",
       "      <td>N</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>826245</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>40000</td>\n",
       "      <td>40000</td>\n",
       "      <td>40000.0</td>\n",
       "      <td>60 months</td>\n",
       "      <td>29.49</td>\n",
       "      <td>1281.64</td>\n",
       "      <td>F</td>\n",
       "      <td>F2</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Cash</td>\n",
       "      <td>N</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>248418</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>40000</td>\n",
       "      <td>40000</td>\n",
       "      <td>40000.0</td>\n",
       "      <td>60 months</td>\n",
       "      <td>11.55</td>\n",
       "      <td>880.71</td>\n",
       "      <td>B</td>\n",
       "      <td>B4</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>DirectPay</td>\n",
       "      <td>N</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>489966</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>40000</td>\n",
       "      <td>40000</td>\n",
       "      <td>40000.0</td>\n",
       "      <td>36 months</td>\n",
       "      <td>7.35</td>\n",
       "      <td>1241.50</td>\n",
       "      <td>A</td>\n",
       "      <td>A4</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Cash</td>\n",
       "      <td>N</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2151510</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>40000</td>\n",
       "      <td>40000</td>\n",
       "      <td>40000.0</td>\n",
       "      <td>60 months</td>\n",
       "      <td>7.35</td>\n",
       "      <td>798.67</td>\n",
       "      <td>A</td>\n",
       "      <td>A4</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Cash</td>\n",
       "      <td>N</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>321183</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>40000</td>\n",
       "      <td>40000</td>\n",
       "      <td>40000.0</td>\n",
       "      <td>60 months</td>\n",
       "      <td>9.58</td>\n",
       "      <td>841.64</td>\n",
       "      <td>B</td>\n",
       "      <td>B1</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Cash</td>\n",
       "      <td>N</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>112364</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>40000</td>\n",
       "      <td>40000</td>\n",
       "      <td>40000.0</td>\n",
       "      <td>36 months</td>\n",
       "      <td>7.84</td>\n",
       "      <td>1250.51</td>\n",
       "      <td>A</td>\n",
       "      <td>A4</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Cash</td>\n",
       "      <td>N</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>490085</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>40000</td>\n",
       "      <td>40000</td>\n",
       "      <td>40000.0</td>\n",
       "      <td>36 months</td>\n",
       "      <td>6.08</td>\n",
       "      <td>1218.33</td>\n",
       "      <td>A</td>\n",
       "      <td>A2</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Cash</td>\n",
       "      <td>N</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>321199</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>40000</td>\n",
       "      <td>40000</td>\n",
       "      <td>40000.0</td>\n",
       "      <td>36 months</td>\n",
       "      <td>6.19</td>\n",
       "      <td>1220.33</td>\n",
       "      <td>A</td>\n",
       "      <td>A2</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Cash</td>\n",
       "      <td>N</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>182190</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>40000</td>\n",
       "      <td>40000</td>\n",
       "      <td>40000.0</td>\n",
       "      <td>60 months</td>\n",
       "      <td>12.73</td>\n",
       "      <td>904.61</td>\n",
       "      <td>B</td>\n",
       "      <td>B5</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Cash</td>\n",
       "      <td>N</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>140808</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>40000</td>\n",
       "      <td>40000</td>\n",
       "      <td>40000.0</td>\n",
       "      <td>60 months</td>\n",
       "      <td>11.55</td>\n",
       "      <td>880.71</td>\n",
       "      <td>B</td>\n",
       "      <td>B4</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Cash</td>\n",
       "      <td>N</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>112373</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>40000</td>\n",
       "      <td>40000</td>\n",
       "      <td>40000.0</td>\n",
       "      <td>36 months</td>\n",
       "      <td>6.11</td>\n",
       "      <td>1218.88</td>\n",
       "      <td>A</td>\n",
       "      <td>A1</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>DirectPay</td>\n",
       "      <td>N</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>490051</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>40000</td>\n",
       "      <td>40000</td>\n",
       "      <td>40000.0</td>\n",
       "      <td>36 months</td>\n",
       "      <td>6.72</td>\n",
       "      <td>1229.97</td>\n",
       "      <td>A</td>\n",
       "      <td>A3</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Cash</td>\n",
       "      <td>N</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2151534</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>40000</td>\n",
       "      <td>40000</td>\n",
       "      <td>40000.0</td>\n",
       "      <td>60 months</td>\n",
       "      <td>15.05</td>\n",
       "      <td>952.65</td>\n",
       "      <td>C</td>\n",
       "      <td>C4</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Cash</td>\n",
       "      <td>N</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>490050</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>40000</td>\n",
       "      <td>40000</td>\n",
       "      <td>40000.0</td>\n",
       "      <td>36 months</td>\n",
       "      <td>6.72</td>\n",
       "      <td>1229.97</td>\n",
       "      <td>A</td>\n",
       "      <td>A3</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Cash</td>\n",
       "      <td>N</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1517303</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>40000</td>\n",
       "      <td>40000</td>\n",
       "      <td>40000.0</td>\n",
       "      <td>36 months</td>\n",
       "      <td>9.44</td>\n",
       "      <td>1280.20</td>\n",
       "      <td>B</td>\n",
       "      <td>B1</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Cash</td>\n",
       "      <td>N</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>490004</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>40000</td>\n",
       "      <td>40000</td>\n",
       "      <td>40000.0</td>\n",
       "      <td>60 months</td>\n",
       "      <td>12.62</td>\n",
       "      <td>902.37</td>\n",
       "      <td>C</td>\n",
       "      <td>C1</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Cash</td>\n",
       "      <td>N</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>638339</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>40000</td>\n",
       "      <td>40000</td>\n",
       "      <td>40000.0</td>\n",
       "      <td>60 months</td>\n",
       "      <td>10.75</td>\n",
       "      <td>864.72</td>\n",
       "      <td>B</td>\n",
       "      <td>B4</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Cash</td>\n",
       "      <td>N</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1517293</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>40000</td>\n",
       "      <td>40000</td>\n",
       "      <td>40000.0</td>\n",
       "      <td>60 months</td>\n",
       "      <td>10.42</td>\n",
       "      <td>858.18</td>\n",
       "      <td>B</td>\n",
       "      <td>B3</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Cash</td>\n",
       "      <td>N</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>489999</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>40000</td>\n",
       "      <td>40000</td>\n",
       "      <td>40000.0</td>\n",
       "      <td>60 months</td>\n",
       "      <td>14.08</td>\n",
       "      <td>932.39</td>\n",
       "      <td>C</td>\n",
       "      <td>C3</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Cash</td>\n",
       "      <td>N</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>826237</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>40000</td>\n",
       "      <td>40000</td>\n",
       "      <td>40000.0</td>\n",
       "      <td>60 months</td>\n",
       "      <td>11.44</td>\n",
       "      <td>878.50</td>\n",
       "      <td>B</td>\n",
       "      <td>B4</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Cash</td>\n",
       "      <td>N</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>489975</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>40000</td>\n",
       "      <td>40000</td>\n",
       "      <td>40000.0</td>\n",
       "      <td>60 months</td>\n",
       "      <td>9.93</td>\n",
       "      <td>848.51</td>\n",
       "      <td>B</td>\n",
       "      <td>B2</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Cash</td>\n",
       "      <td>N</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>638322</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>40000</td>\n",
       "      <td>40000</td>\n",
       "      <td>40000.0</td>\n",
       "      <td>60 months</td>\n",
       "      <td>7.89</td>\n",
       "      <td>808.96</td>\n",
       "      <td>A</td>\n",
       "      <td>A5</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Cash</td>\n",
       "      <td>N</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2145538</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>40000</td>\n",
       "      <td>40000</td>\n",
       "      <td>40000.0</td>\n",
       "      <td>36 months</td>\n",
       "      <td>9.93</td>\n",
       "      <td>1289.38</td>\n",
       "      <td>B</td>\n",
       "      <td>B2</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Cash</td>\n",
       "      <td>N</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>2260668 rows × 145 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "         id  member_id  loan_amnt  funded_amnt  funded_amnt_inv        term  \\\n",
       "2138210 NaN        NaN       5500         5500              0.0   36 months   \n",
       "2138169 NaN        NaN       7500         7500              0.0   36 months   \n",
       "2141012 NaN        NaN       7225         5300              0.0   36 months   \n",
       "2141011 NaN        NaN       7500         7500              0.0   36 months   \n",
       "2138262 NaN        NaN       7000         7000              0.0   36 months   \n",
       "2138261 NaN        NaN       3500         3500              0.0   36 months   \n",
       "2141003 NaN        NaN       4000         4000              0.0   36 months   \n",
       "2140997 NaN        NaN       2400         2400              0.0   36 months   \n",
       "2140994 NaN        NaN       6000         6000              0.0   36 months   \n",
       "2138172 NaN        NaN       5000         5000              0.0   36 months   \n",
       "2140993 NaN        NaN       7500         7500              0.0   36 months   \n",
       "2140992 NaN        NaN       7000         7000              0.0   36 months   \n",
       "2138257 NaN        NaN       6000         4725              0.0   36 months   \n",
       "2140987 NaN        NaN       5000         5000              0.0   36 months   \n",
       "2140986 NaN        NaN       4500         4500              0.0   36 months   \n",
       "2137960 NaN        NaN       7500         4750              0.0   36 months   \n",
       "2140984 NaN        NaN      11000        11000              0.0   36 months   \n",
       "2138133 NaN        NaN       1600         1600              0.0   36 months   \n",
       "2138254 NaN        NaN       3500         3500              0.0   36 months   \n",
       "2140978 NaN        NaN      15000        15000              0.0   36 months   \n",
       "2140977 NaN        NaN       4000         4000              0.0   36 months   \n",
       "2138249 NaN        NaN       1500         1175              0.0   36 months   \n",
       "2138111 NaN        NaN       3000         3000              0.0   36 months   \n",
       "2137974 NaN        NaN       5600         4300              0.0   36 months   \n",
       "2137975 NaN        NaN       7500         4800              0.0   36 months   \n",
       "2137977 NaN        NaN       7500         5550              0.0   36 months   \n",
       "2138106 NaN        NaN       7000         7000              0.0   36 months   \n",
       "2137988 NaN        NaN       6000         5100              0.0   36 months   \n",
       "2138162 NaN        NaN       7500         7500              0.0   36 months   \n",
       "2138163 NaN        NaN       5000         5000              0.0   36 months   \n",
       "...      ..        ...        ...          ...              ...         ...   \n",
       "1605588 NaN        NaN      40000        40000          40000.0   36 months   \n",
       "370115  NaN        NaN      40000        40000          40000.0   36 months   \n",
       "370116  NaN        NaN      40000        40000          40000.0   60 months   \n",
       "53289   NaN        NaN      40000        40000          40000.0   36 months   \n",
       "182253  NaN        NaN      40000        40000          40000.0   36 months   \n",
       "489926  NaN        NaN      40000        40000          40000.0   36 months   \n",
       "489922  NaN        NaN      40000        40000          40000.0   60 months   \n",
       "826245  NaN        NaN      40000        40000          40000.0   60 months   \n",
       "248418  NaN        NaN      40000        40000          40000.0   60 months   \n",
       "489966  NaN        NaN      40000        40000          40000.0   36 months   \n",
       "2151510 NaN        NaN      40000        40000          40000.0   60 months   \n",
       "321183  NaN        NaN      40000        40000          40000.0   60 months   \n",
       "112364  NaN        NaN      40000        40000          40000.0   36 months   \n",
       "490085  NaN        NaN      40000        40000          40000.0   36 months   \n",
       "321199  NaN        NaN      40000        40000          40000.0   36 months   \n",
       "182190  NaN        NaN      40000        40000          40000.0   60 months   \n",
       "140808  NaN        NaN      40000        40000          40000.0   60 months   \n",
       "112373  NaN        NaN      40000        40000          40000.0   36 months   \n",
       "490051  NaN        NaN      40000        40000          40000.0   36 months   \n",
       "2151534 NaN        NaN      40000        40000          40000.0   60 months   \n",
       "490050  NaN        NaN      40000        40000          40000.0   36 months   \n",
       "1517303 NaN        NaN      40000        40000          40000.0   36 months   \n",
       "490004  NaN        NaN      40000        40000          40000.0   60 months   \n",
       "638339  NaN        NaN      40000        40000          40000.0   60 months   \n",
       "1517293 NaN        NaN      40000        40000          40000.0   60 months   \n",
       "489999  NaN        NaN      40000        40000          40000.0   60 months   \n",
       "826237  NaN        NaN      40000        40000          40000.0   60 months   \n",
       "489975  NaN        NaN      40000        40000          40000.0   60 months   \n",
       "638322  NaN        NaN      40000        40000          40000.0   60 months   \n",
       "2145538 NaN        NaN      40000        40000          40000.0   36 months   \n",
       "\n",
       "         int_rate  installment grade sub_grade  ...  \\\n",
       "2138210      9.76       176.86     B        B2  ...   \n",
       "2138169     11.97       249.00     C        C4  ...   \n",
       "2141012     14.18       181.61     E        E1  ...   \n",
       "2141011     11.03       245.65     C        C1  ...   \n",
       "2138262      8.00       219.36     A        A3  ...   \n",
       "2138261      8.32       110.20     A        A4  ...   \n",
       "2141003     18.54       145.70     G        G4  ...   \n",
       "2140997     13.17        81.07     D        D2  ...   \n",
       "2140994     14.12       205.42     D        D5  ...   \n",
       "2138172     10.08       161.53     B        B3  ...   \n",
       "2140993     16.01       263.72     F        F1  ...   \n",
       "2140992     14.12       239.66     D        D5  ...   \n",
       "2138257     10.39       153.33     B        B4  ...   \n",
       "2140987      9.70       160.64     B        B1  ...   \n",
       "2140986     12.54       150.63     C        C5  ...   \n",
       "2137960     11.28       156.14     C        C1  ...   \n",
       "2140984     11.91       364.89     C        C3  ...   \n",
       "2138133     11.66        52.89     C        C3  ...   \n",
       "2138254      7.68       109.17     A        A2  ...   \n",
       "2140978     11.91       497.58     C        C3  ...   \n",
       "2140977     17.91       144.43     G        G2  ...   \n",
       "2138249      7.37        36.48     A        A1  ...   \n",
       "2138111      8.00        94.01     A        A3  ...   \n",
       "2137974      8.32       135.39     A        A4  ...   \n",
       "2137975     11.59       158.50     C        C2  ...   \n",
       "2137977      8.32       174.74     A        A4  ...   \n",
       "2138106      8.00       219.36     A        A3  ...   \n",
       "2137988      8.00       159.82     A        A3  ...   \n",
       "2138162     10.39       243.38     B        B4  ...   \n",
       "2138163     12.29       166.77     C        C5  ...   \n",
       "...           ...          ...   ...       ...  ...   \n",
       "1605588     22.91      1546.52     E        E1  ...   \n",
       "370115       5.31      1204.42     A        A1  ...   \n",
       "370116      18.45      1025.56     D        D2  ...   \n",
       "53289       11.80      1324.76     B        B4  ...   \n",
       "182253      11.55      1320.00     B        B4  ...   \n",
       "489926       9.44      1280.20     B        B1  ...   \n",
       "489922      12.62       902.37     C        C1  ...   \n",
       "826245      29.49      1281.64     F        F2  ...   \n",
       "248418      11.55       880.71     B        B4  ...   \n",
       "489966       7.35      1241.50     A        A4  ...   \n",
       "2151510      7.35       798.67     A        A4  ...   \n",
       "321183       9.58       841.64     B        B1  ...   \n",
       "112364       7.84      1250.51     A        A4  ...   \n",
       "490085       6.08      1218.33     A        A2  ...   \n",
       "321199       6.19      1220.33     A        A2  ...   \n",
       "182190      12.73       904.61     B        B5  ...   \n",
       "140808      11.55       880.71     B        B4  ...   \n",
       "112373       6.11      1218.88     A        A1  ...   \n",
       "490051       6.72      1229.97     A        A3  ...   \n",
       "2151534     15.05       952.65     C        C4  ...   \n",
       "490050       6.72      1229.97     A        A3  ...   \n",
       "1517303      9.44      1280.20     B        B1  ...   \n",
       "490004      12.62       902.37     C        C1  ...   \n",
       "638339      10.75       864.72     B        B4  ...   \n",
       "1517293     10.42       858.18     B        B3  ...   \n",
       "489999      14.08       932.39     C        C3  ...   \n",
       "826237      11.44       878.50     B        B4  ...   \n",
       "489975       9.93       848.51     B        B2  ...   \n",
       "638322       7.89       808.96     A        A5  ...   \n",
       "2145538      9.93      1289.38     B        B2  ...   \n",
       "\n",
       "        hardship_payoff_balance_amount hardship_last_payment_amount  \\\n",
       "2138210                            NaN                          NaN   \n",
       "2138169                            NaN                          NaN   \n",
       "2141012                            NaN                          NaN   \n",
       "2141011                            NaN                          NaN   \n",
       "2138262                            NaN                          NaN   \n",
       "2138261                            NaN                          NaN   \n",
       "2141003                            NaN                          NaN   \n",
       "2140997                            NaN                          NaN   \n",
       "2140994                            NaN                          NaN   \n",
       "2138172                            NaN                          NaN   \n",
       "2140993                            NaN                          NaN   \n",
       "2140992                            NaN                          NaN   \n",
       "2138257                            NaN                          NaN   \n",
       "2140987                            NaN                          NaN   \n",
       "2140986                            NaN                          NaN   \n",
       "2137960                            NaN                          NaN   \n",
       "2140984                            NaN                          NaN   \n",
       "2138133                            NaN                          NaN   \n",
       "2138254                            NaN                          NaN   \n",
       "2140978                            NaN                          NaN   \n",
       "2140977                            NaN                          NaN   \n",
       "2138249                            NaN                          NaN   \n",
       "2138111                            NaN                          NaN   \n",
       "2137974                            NaN                          NaN   \n",
       "2137975                            NaN                          NaN   \n",
       "2137977                            NaN                          NaN   \n",
       "2138106                            NaN                          NaN   \n",
       "2137988                            NaN                          NaN   \n",
       "2138162                            NaN                          NaN   \n",
       "2138163                            NaN                          NaN   \n",
       "...                                ...                          ...   \n",
       "1605588                            NaN                          NaN   \n",
       "370115                             NaN                          NaN   \n",
       "370116                             NaN                          NaN   \n",
       "53289                              NaN                          NaN   \n",
       "182253                             NaN                          NaN   \n",
       "489926                             NaN                          NaN   \n",
       "489922                             NaN                          NaN   \n",
       "826245                             NaN                          NaN   \n",
       "248418                             NaN                          NaN   \n",
       "489966                             NaN                          NaN   \n",
       "2151510                            NaN                          NaN   \n",
       "321183                             NaN                          NaN   \n",
       "112364                             NaN                          NaN   \n",
       "490085                             NaN                          NaN   \n",
       "321199                             NaN                          NaN   \n",
       "182190                             NaN                          NaN   \n",
       "140808                             NaN                          NaN   \n",
       "112373                             NaN                          NaN   \n",
       "490051                             NaN                          NaN   \n",
       "2151534                            NaN                          NaN   \n",
       "490050                             NaN                          NaN   \n",
       "1517303                            NaN                          NaN   \n",
       "490004                             NaN                          NaN   \n",
       "638339                             NaN                          NaN   \n",
       "1517293                            NaN                          NaN   \n",
       "489999                             NaN                          NaN   \n",
       "826237                             NaN                          NaN   \n",
       "489975                             NaN                          NaN   \n",
       "638322                             NaN                          NaN   \n",
       "2145538                            NaN                          NaN   \n",
       "\n",
       "        disbursement_method  debt_settlement_flag debt_settlement_flag_date  \\\n",
       "2138210                Cash                     N                       NaN   \n",
       "2138169                Cash                     N                       NaN   \n",
       "2141012                Cash                     N                       NaN   \n",
       "2141011                Cash                     N                       NaN   \n",
       "2138262                Cash                     N                       NaN   \n",
       "2138261                Cash                     N                       NaN   \n",
       "2141003                Cash                     N                       NaN   \n",
       "2140997                Cash                     N                       NaN   \n",
       "2140994                Cash                     N                       NaN   \n",
       "2138172                Cash                     N                       NaN   \n",
       "2140993                Cash                     N                       NaN   \n",
       "2140992                Cash                     N                       NaN   \n",
       "2138257                Cash                     N                       NaN   \n",
       "2140987                Cash                     N                       NaN   \n",
       "2140986                Cash                     N                       NaN   \n",
       "2137960                Cash                     N                       NaN   \n",
       "2140984                Cash                     N                       NaN   \n",
       "2138133                Cash                     N                       NaN   \n",
       "2138254                Cash                     N                       NaN   \n",
       "2140978                Cash                     N                       NaN   \n",
       "2140977                Cash                     N                       NaN   \n",
       "2138249                Cash                     N                       NaN   \n",
       "2138111                Cash                     N                       NaN   \n",
       "2137974                Cash                     N                       NaN   \n",
       "2137975                Cash                     N                       NaN   \n",
       "2137977                Cash                     N                       NaN   \n",
       "2138106                Cash                     N                       NaN   \n",
       "2137988                Cash                     N                       NaN   \n",
       "2138162                Cash                     N                       NaN   \n",
       "2138163                Cash                     N                       NaN   \n",
       "...                     ...                   ...                       ...   \n",
       "1605588                Cash                     N                       NaN   \n",
       "370115                 Cash                     N                       NaN   \n",
       "370116                 Cash                     N                       NaN   \n",
       "53289                  Cash                     N                       NaN   \n",
       "182253                 Cash                     N                       NaN   \n",
       "489926                 Cash                     N                       NaN   \n",
       "489922                 Cash                     N                       NaN   \n",
       "826245                 Cash                     N                       NaN   \n",
       "248418            DirectPay                     N                       NaN   \n",
       "489966                 Cash                     N                       NaN   \n",
       "2151510                Cash                     N                       NaN   \n",
       "321183                 Cash                     N                       NaN   \n",
       "112364                 Cash                     N                       NaN   \n",
       "490085                 Cash                     N                       NaN   \n",
       "321199                 Cash                     N                       NaN   \n",
       "182190                 Cash                     N                       NaN   \n",
       "140808                 Cash                     N                       NaN   \n",
       "112373            DirectPay                     N                       NaN   \n",
       "490051                 Cash                     N                       NaN   \n",
       "2151534                Cash                     N                       NaN   \n",
       "490050                 Cash                     N                       NaN   \n",
       "1517303                Cash                     N                       NaN   \n",
       "490004                 Cash                     N                       NaN   \n",
       "638339                 Cash                     N                       NaN   \n",
       "1517293                Cash                     N                       NaN   \n",
       "489999                 Cash                     N                       NaN   \n",
       "826237                 Cash                     N                       NaN   \n",
       "489975                 Cash                     N                       NaN   \n",
       "638322                 Cash                     N                       NaN   \n",
       "2145538                Cash                     N                       NaN   \n",
       "\n",
       "        settlement_status settlement_date settlement_amount  \\\n",
       "2138210               NaN             NaN               NaN   \n",
       "2138169               NaN             NaN               NaN   \n",
       "2141012               NaN             NaN               NaN   \n",
       "2141011               NaN             NaN               NaN   \n",
       "2138262               NaN             NaN               NaN   \n",
       "2138261               NaN             NaN               NaN   \n",
       "2141003               NaN             NaN               NaN   \n",
       "2140997               NaN             NaN               NaN   \n",
       "2140994               NaN             NaN               NaN   \n",
       "2138172               NaN             NaN               NaN   \n",
       "2140993               NaN             NaN               NaN   \n",
       "2140992               NaN             NaN               NaN   \n",
       "2138257               NaN             NaN               NaN   \n",
       "2140987               NaN             NaN               NaN   \n",
       "2140986               NaN             NaN               NaN   \n",
       "2137960               NaN             NaN               NaN   \n",
       "2140984               NaN             NaN               NaN   \n",
       "2138133               NaN             NaN               NaN   \n",
       "2138254               NaN             NaN               NaN   \n",
       "2140978               NaN             NaN               NaN   \n",
       "2140977               NaN             NaN               NaN   \n",
       "2138249               NaN             NaN               NaN   \n",
       "2138111               NaN             NaN               NaN   \n",
       "2137974               NaN             NaN               NaN   \n",
       "2137975               NaN             NaN               NaN   \n",
       "2137977               NaN             NaN               NaN   \n",
       "2138106               NaN             NaN               NaN   \n",
       "2137988               NaN             NaN               NaN   \n",
       "2138162               NaN             NaN               NaN   \n",
       "2138163               NaN             NaN               NaN   \n",
       "...                   ...             ...               ...   \n",
       "1605588               NaN             NaN               NaN   \n",
       "370115                NaN             NaN               NaN   \n",
       "370116                NaN             NaN               NaN   \n",
       "53289                 NaN             NaN               NaN   \n",
       "182253                NaN             NaN               NaN   \n",
       "489926                NaN             NaN               NaN   \n",
       "489922                NaN             NaN               NaN   \n",
       "826245                NaN             NaN               NaN   \n",
       "248418                NaN             NaN               NaN   \n",
       "489966                NaN             NaN               NaN   \n",
       "2151510               NaN             NaN               NaN   \n",
       "321183                NaN             NaN               NaN   \n",
       "112364                NaN             NaN               NaN   \n",
       "490085                NaN             NaN               NaN   \n",
       "321199                NaN             NaN               NaN   \n",
       "182190                NaN             NaN               NaN   \n",
       "140808                NaN             NaN               NaN   \n",
       "112373                NaN             NaN               NaN   \n",
       "490051                NaN             NaN               NaN   \n",
       "2151534               NaN             NaN               NaN   \n",
       "490050                NaN             NaN               NaN   \n",
       "1517303               NaN             NaN               NaN   \n",
       "490004                NaN             NaN               NaN   \n",
       "638339                NaN             NaN               NaN   \n",
       "1517293               NaN             NaN               NaN   \n",
       "489999                NaN             NaN               NaN   \n",
       "826237                NaN             NaN               NaN   \n",
       "489975                NaN             NaN               NaN   \n",
       "638322                NaN             NaN               NaN   \n",
       "2145538               NaN             NaN               NaN   \n",
       "\n",
       "         settlement_percentage settlement_term  \n",
       "2138210                    NaN             NaN  \n",
       "2138169                    NaN             NaN  \n",
       "2141012                    NaN             NaN  \n",
       "2141011                    NaN             NaN  \n",
       "2138262                    NaN             NaN  \n",
       "2138261                    NaN             NaN  \n",
       "2141003                    NaN             NaN  \n",
       "2140997                    NaN             NaN  \n",
       "2140994                    NaN             NaN  \n",
       "2138172                    NaN             NaN  \n",
       "2140993                    NaN             NaN  \n",
       "2140992                    NaN             NaN  \n",
       "2138257                    NaN             NaN  \n",
       "2140987                    NaN             NaN  \n",
       "2140986                    NaN             NaN  \n",
       "2137960                    NaN             NaN  \n",
       "2140984                    NaN             NaN  \n",
       "2138133                    NaN             NaN  \n",
       "2138254                    NaN             NaN  \n",
       "2140978                    NaN             NaN  \n",
       "2140977                    NaN             NaN  \n",
       "2138249                    NaN             NaN  \n",
       "2138111                    NaN             NaN  \n",
       "2137974                    NaN             NaN  \n",
       "2137975                    NaN             NaN  \n",
       "2137977                    NaN             NaN  \n",
       "2138106                    NaN             NaN  \n",
       "2137988                    NaN             NaN  \n",
       "2138162                    NaN             NaN  \n",
       "2138163                    NaN             NaN  \n",
       "...                        ...             ...  \n",
       "1605588                    NaN             NaN  \n",
       "370115                     NaN             NaN  \n",
       "370116                     NaN             NaN  \n",
       "53289                      NaN             NaN  \n",
       "182253                     NaN             NaN  \n",
       "489926                     NaN             NaN  \n",
       "489922                     NaN             NaN  \n",
       "826245                     NaN             NaN  \n",
       "248418                     NaN             NaN  \n",
       "489966                     NaN             NaN  \n",
       "2151510                    NaN             NaN  \n",
       "321183                     NaN             NaN  \n",
       "112364                     NaN             NaN  \n",
       "490085                     NaN             NaN  \n",
       "321199                     NaN             NaN  \n",
       "182190                     NaN             NaN  \n",
       "140808                     NaN             NaN  \n",
       "112373                     NaN             NaN  \n",
       "490051                     NaN             NaN  \n",
       "2151534                    NaN             NaN  \n",
       "490050                     NaN             NaN  \n",
       "1517303                    NaN             NaN  \n",
       "490004                     NaN             NaN  \n",
       "638339                     NaN             NaN  \n",
       "1517293                    NaN             NaN  \n",
       "489999                     NaN             NaN  \n",
       "826237                     NaN             NaN  \n",
       "489975                     NaN             NaN  \n",
       "638322                     NaN             NaN  \n",
       "2145538                    NaN             NaN  \n",
       "\n",
       "[2260668 rows x 145 columns]"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "%%time\n",
    "pd_df.sort_values(by = 'funded_amnt_inv')\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Deleting Rows/Columns\n",
    "\n",
    "Here is how we can delete the column named 'member_id':"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "del dt_df[:, 'member_id']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Groupby\n",
    "\n",
    "Just like in pandas, datatable also has the groupby functionalities. Let's see how we can get the mean of `funded_amount` column grouped by the `grade` column."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 6.41 s, sys: 1.34 s, total: 7.76 s\n",
      "Wall time: 2.42 s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "for i in range(100):\n",
    "    dt_df[:, dt.sum(dt.f.funded_amnt), dt.by(dt.f.grade)]\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 12.9 s, sys: 859 ms, total: 13.7 s\n",
      "Wall time: 13.9 s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "for i in range(100):\n",
    "    pd_df.groupby(\"grade\")[\"funded_amnt\"].sum()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "What does **.f** stand for?\n",
    "\n",
    "f stands for `frame proxy`, and provides a simple way to refer to the Frame that we are currently operating upon. In case of our example, dt.f simply stands for dt_df."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Filtering Rows\n",
    "\n",
    "The syntax for filtering rows is pretty similar to that of GroupBy. Let us filter those rows of `loan_amnt`for which the values of `loan_amnt` are greater than `funded_amnt`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div class='datatable'>\n",
       "  <table class='frame'>\n",
       "  <thead>\n",
       "    <tr class='colnames'><td class='row_index'></td><th>loan_amnt</th></tr>\n",
       "    <tr class='coltypes'><td class='row_index'></td><td class='int' title='int32'>&#x25AA;&#x25AA;&#x25AA;&#x25AA;</td></tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr><td class='row_index'>0</td><td>18000</td></tr>\n",
       "    <tr><td class='row_index'>1</td><td>20000</td></tr>\n",
       "    <tr><td class='row_index'>2</td><td>20000</td></tr>\n",
       "    <tr><td class='row_index'>3</td><td>5000</td></tr>\n",
       "    <tr><td class='row_index'>4</td><td>2875</td></tr>\n",
       "    <tr><td class='row_index'>5</td><td>33425</td></tr>\n",
       "    <tr><td class='row_index'>6</td><td>32000</td></tr>\n",
       "    <tr><td class='row_index'>7</td><td>25000</td></tr>\n",
       "    <tr><td class='row_index'>8</td><td>27500</td></tr>\n",
       "    <tr><td class='row_index'>9</td><td>20000</td></tr>\n",
       "    <tr><td class='row_index'>&#x22EE;</td><td class='hellipsis'>&#x22EE;</td></tr>\n",
       "    <tr><td class='row_index'>2060</td><td>25000</td></tr>\n",
       "    <tr><td class='row_index'>2061</td><td>25000</td></tr>\n",
       "    <tr><td class='row_index'>2062</td><td>7200</td></tr>\n",
       "    <tr><td class='row_index'>2063</td><td>15600</td></tr>\n",
       "    <tr><td class='row_index'>2064</td><td>10500</td></tr>\n",
       "  </tbody>\n",
       "  </table>\n",
       "  <div class='footer'>\n",
       "    <div class='frame_dimensions'>2065 rows &times; 1 column</div>\n",
       "  </div>\n",
       "</div>\n"
      ],
      "text/plain": []
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dt_df[dt.f.loan_amnt>dt.f.funded_amnt,\"loan_amnt\"]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Saving the Frame\n",
    "It is also possible to write the Frame's content into a csv file so that it can be used in future."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "dt_df.to_csv('output.csv')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For more data manipulation functions, refer to the documentation page."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}