demo_notebook.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# NFStream: a Flexible Network Data Analysis Framework"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "6.5.3\n"
     ]
    }
   ],
   "source": [
    "import nfstream\n",
    "print(nfstream.__version__)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[**NFStream**][repo] is a multiplatform Python framework providing fast, flexible, and expressive data structures designed to make \n",
    "working with **online** or **offline** network data easy and intuitive. It aims to be Python's fundamental high-level \n",
    "building block for doing practical, **real-world** network flow data analysis. Additionally, it has the broader \n",
    "goal of becoming **a unifying network data analytics framework for researchers** providing data reproducibility \n",
    "across experiments.\n",
    "\n",
    "* **Performance:** NFStream is designed to be fast: [**AF_PACKET_V3/FANOUT**][packet] on Linux, multiprocessing, native\n",
    "[**CFFI based**][cffi] computation engine, and [**PyPy**][pypy] full support.\n",
    "* **Encrypted layer-7 visibility:** NFStream deep packet inspection is based on [**nDPI**][ndpi]. \n",
    "It allows NFStream to perform [**reliable**][reliable] encrypted applications identification and metadata \n",
    "fingerprinting (e.g. TLS, SSH, DHCP, HTTP).\n",
    "* **System visibility:** NFStream probes the monitored system's kernel to obtain information on open Internet sockets \n",
    "and collects guaranteed ground-truth (process name, PID, etc.) at the application level.\n",
    "* **Statistical features extraction:** NFStream provides state of the art of flow-based statistical feature extraction. \n",
    "It includes post-mortem statistical features (e.g., minimum, mean, standard deviation, and maximum of packet size and \n",
    "inter-arrival time) and early flow features (e.g. sequence of first n packets sizes, inter-arrival times, and directions).\n",
    "* **Flexibility:** NFStream is easily extensible using [**NFPlugins**][nfplugin]. It allows the creation of a new flow \n",
    "feature within a few lines of Python.\n",
    "* **Machine Learning oriented:** NFStream aims to make Machine Learning Approaches for network traffic management \n",
    "reproducible and deployable. By using NFStream as a common framework, researchers ensure that models are trained using \n",
    "the same feature computation logic, and thus, a fair comparison is possible. Moreover, trained models can be deployed \n",
    "and evaluated on live networks using [**NFPlugins**][nfplugin]. \n",
    "\n",
    "\n",
    "In this notebook, we demonstrate a subset of features provided by [**NFStream**][repo].\n",
    "\n",
    "[ndpi]: https://github.com/ntop/nDPI\n",
    "[nfplugin]: https://nfstream.github.io/docs/api#nfplugin\n",
    "[reliable]: http://people.ac.upc.edu/pbarlet/papers/ground-truth.pam2014.pdf\n",
    "[repo]: https://nfstream.org/\n",
    "[pypy]: https://www.pypy.org/\n",
    "[cffi]: https://cffi.readthedocs.io/en/latest/index.html"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "from nfstream import NFStreamer, NFPlugin\n",
    "import pandas as pd\n",
    "pd.set_option('display.max_columns', 500)\n",
    "pd.set_option('display.max_rows', 500)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Flow aggregation made simple"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the following, we are going to use the main object provided by nfstream, `NFStreamer` which have the following parameters:\n",
    "\n",
    "* `source` [default=None]: Packet capture source. Pcap file path or network interface name.\n",
    "* `decode_tunnels` [default=True]: Enable/Disable GTP/TZSP tunnels decoding.\n",
    "* `bpf_filter` [default=None]: Specify a [BPF filter][bpf] filter for filtering selected traffic.\n",
    "* `promiscuous_mode` [default=True]: Enable/Disable promiscuous capture mode.\n",
    "* `snapshot_length` [default=1500]: Control packet slicing size (truncation) in bytes.\n",
    "* `idle_timeout` [default=120]: Flows that are idle (no packets received) for more than this value in seconds are expired.\n",
    "* `active_timeout` [default=1800]: Flows that are active for more than this value in seconds are expired.\n",
    "* `accounting_mode` [default=0] : Specify the accounting mode that will be used to report bytes related features (0: Link layer, 1: IP layer, 2: Transport layer, 3: Payload).\n",
    "* `udps` [default=None]: Specify user defined NFPlugins used to extend NFStreamer.\n",
    "* `n_dissections` [default=20]: Number of per flow packets to dissect for L7 visibility feature. When set to 0, L7 visibility feature is disabled.\n",
    "* `statistical_analysis` [default=False]: Enable/Disable post-mortem flow statistical analysis.\n",
    "* `splt_analysis` [default=0]: Specify the sequence of first packets length for early statistical analysis. When set to 0, splt_analysis is disabled.\n",
    "* `max_nflows` [default=0]:\tSpecify the number of maximum flows to capture before returning. Unset when equal to 0.\n",
    "* `n_meters` [default=0]: Specify the number of parallel metering processes. When set to 0, NFStreamer will automatically scale metering according to available physical cores on the running host.\n",
    "* `performance_report` [default=0]: [**Performance report**](https://github.com/nfstream/nfstream/blob/master/assets/PERFORMANCE_REPORT.md) interval in seconds. Disabled whhen set to 0. Ignored for offline capture.\n",
    "* `system_visibility_mode` [default=0]\tEnable system process mapping by probing the host machine.\n",
    "* `system_visibility_poll_ms` [default=100]\tSet the polling interval in milliseconds for system process mapping feature (0 is the maximum achievable rate).\n",
    "\n",
    "`NFStreamer` returns a flow iterator. We can iterate over flows or convert it directly to pandas Dataframe using `to_pandas()` method.\n",
    "\n",
    "[bpf]: https://biot.com/capstats/bpf.html"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = NFStreamer(source=\"pcap/instagram.pcap\").to_pandas()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>expiration_id</th>\n",
       "      <th>src_ip</th>\n",
       "      <th>src_mac</th>\n",
       "      <th>src_oui</th>\n",
       "      <th>src_port</th>\n",
       "      <th>dst_ip</th>\n",
       "      <th>dst_mac</th>\n",
       "      <th>dst_oui</th>\n",
       "      <th>dst_port</th>\n",
       "      <th>protocol</th>\n",
       "      <th>ip_version</th>\n",
       "      <th>vlan_id</th>\n",
       "      <th>tunnel_id</th>\n",
       "      <th>bidirectional_first_seen_ms</th>\n",
       "      <th>bidirectional_last_seen_ms</th>\n",
       "      <th>bidirectional_duration_ms</th>\n",
       "      <th>bidirectional_packets</th>\n",
       "      <th>bidirectional_bytes</th>\n",
       "      <th>src2dst_first_seen_ms</th>\n",
       "      <th>src2dst_last_seen_ms</th>\n",
       "      <th>src2dst_duration_ms</th>\n",
       "      <th>src2dst_packets</th>\n",
       "      <th>src2dst_bytes</th>\n",
       "      <th>dst2src_first_seen_ms</th>\n",
       "      <th>dst2src_last_seen_ms</th>\n",
       "      <th>dst2src_duration_ms</th>\n",
       "      <th>dst2src_packets</th>\n",
       "      <th>dst2src_bytes</th>\n",
       "      <th>application_name</th>\n",
       "      <th>application_category_name</th>\n",
       "      <th>application_is_guessed</th>\n",
       "      <th>application_confidence</th>\n",
       "      <th>requested_server_name</th>\n",
       "      <th>client_fingerprint</th>\n",
       "      <th>server_fingerprint</th>\n",
       "      <th>user_agent</th>\n",
       "      <th>content_type</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>192.168.0.103</td>\n",
       "      <td>40:f3:08:c3:8e:e1</td>\n",
       "      <td>40:f3:08</td>\n",
       "      <td>33936</td>\n",
       "      <td>31.13.93.52</td>\n",
       "      <td>00:1b:2f:f0:7e:b4</td>\n",
       "      <td>00:1b:2f</td>\n",
       "      <td>443</td>\n",
       "      <td>6</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1436720898386</td>\n",
       "      <td>1436720908442</td>\n",
       "      <td>10056</td>\n",
       "      <td>68</td>\n",
       "      <td>45688</td>\n",
       "      <td>1436720898386</td>\n",
       "      <td>1436720908442</td>\n",
       "      <td>10056</td>\n",
       "      <td>34</td>\n",
       "      <td>5555</td>\n",
       "      <td>1436720898475</td>\n",
       "      <td>1436720908442</td>\n",
       "      <td>9967</td>\n",
       "      <td>34</td>\n",
       "      <td>40133</td>\n",
       "      <td>TLS</td>\n",
       "      <td>Web</td>\n",
       "      <td>0</td>\n",
       "      <td>6</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>192.168.0.106</td>\n",
       "      <td>00:16:44:1f:59:66</td>\n",
       "      <td>00:16:44</td>\n",
       "      <td>17500</td>\n",
       "      <td>255.255.255.255</td>\n",
       "      <td>ff:ff:ff:ff:ff:ff</td>\n",
       "      <td>ff:ff:ff</td>\n",
       "      <td>17500</td>\n",
       "      <td>17</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1436720906017</td>\n",
       "      <td>1436720906024</td>\n",
       "      <td>7</td>\n",
       "      <td>4</td>\n",
       "      <td>580</td>\n",
       "      <td>1436720906017</td>\n",
       "      <td>1436720906024</td>\n",
       "      <td>7</td>\n",
       "      <td>4</td>\n",
       "      <td>580</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>Dropbox</td>\n",
       "      <td>Cloud</td>\n",
       "      <td>0</td>\n",
       "      <td>6</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>192.168.0.106</td>\n",
       "      <td>00:16:44:1f:59:66</td>\n",
       "      <td>00:16:44</td>\n",
       "      <td>17500</td>\n",
       "      <td>192.168.0.255</td>\n",
       "      <td>ff:ff:ff:ff:ff:ff</td>\n",
       "      <td>ff:ff:ff</td>\n",
       "      <td>17500</td>\n",
       "      <td>17</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1436720906022</td>\n",
       "      <td>1436720906022</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>145</td>\n",
       "      <td>1436720906022</td>\n",
       "      <td>1436720906022</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>145</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>Dropbox</td>\n",
       "      <td>Cloud</td>\n",
       "      <td>0</td>\n",
       "      <td>6</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>192.168.0.1</td>\n",
       "      <td>00:1b:2f:f0:7e:b4</td>\n",
       "      <td>00:1b:2f</td>\n",
       "      <td>520</td>\n",
       "      <td>192.168.0.255</td>\n",
       "      <td>ff:ff:ff:ff:ff:ff</td>\n",
       "      <td>ff:ff:ff</td>\n",
       "      <td>520</td>\n",
       "      <td>17</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1436720906025</td>\n",
       "      <td>1436720906025</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>66</td>\n",
       "      <td>1436720906025</td>\n",
       "      <td>1436720906025</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>66</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>Unknown</td>\n",
       "      <td>Unspecified</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>192.168.0.103</td>\n",
       "      <td>00:00:00:00:00:00</td>\n",
       "      <td>00:00:00</td>\n",
       "      <td>0</td>\n",
       "      <td>192.168.0.103</td>\n",
       "      <td>00:00:00:00:00:00</td>\n",
       "      <td>00:00:00</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1436720908464</td>\n",
       "      <td>1436720911139</td>\n",
       "      <td>2675</td>\n",
       "      <td>5</td>\n",
       "      <td>510</td>\n",
       "      <td>1436720908464</td>\n",
       "      <td>1436720911139</td>\n",
       "      <td>2675</td>\n",
       "      <td>5</td>\n",
       "      <td>510</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>ICMP</td>\n",
       "      <td>Network</td>\n",
       "      <td>0</td>\n",
       "      <td>6</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   id  expiration_id         src_ip            src_mac   src_oui  src_port  \\\n",
       "0   0              0  192.168.0.103  40:f3:08:c3:8e:e1  40:f3:08     33936   \n",
       "1   1              0  192.168.0.106  00:16:44:1f:59:66  00:16:44     17500   \n",
       "2   2              0  192.168.0.106  00:16:44:1f:59:66  00:16:44     17500   \n",
       "3   3              0    192.168.0.1  00:1b:2f:f0:7e:b4  00:1b:2f       520   \n",
       "4   4              0  192.168.0.103  00:00:00:00:00:00  00:00:00         0   \n",
       "\n",
       "            dst_ip            dst_mac   dst_oui  dst_port  protocol  \\\n",
       "0      31.13.93.52  00:1b:2f:f0:7e:b4  00:1b:2f       443         6   \n",
       "1  255.255.255.255  ff:ff:ff:ff:ff:ff  ff:ff:ff     17500        17   \n",
       "2    192.168.0.255  ff:ff:ff:ff:ff:ff  ff:ff:ff     17500        17   \n",
       "3    192.168.0.255  ff:ff:ff:ff:ff:ff  ff:ff:ff       520        17   \n",
       "4    192.168.0.103  00:00:00:00:00:00  00:00:00         0         1   \n",
       "\n",
       "   ip_version  vlan_id  tunnel_id  bidirectional_first_seen_ms  \\\n",
       "0           4        0          0                1436720898386   \n",
       "1           4        0          0                1436720906017   \n",
       "2           4        0          0                1436720906022   \n",
       "3           4        0          0                1436720906025   \n",
       "4           4        0          0                1436720908464   \n",
       "\n",
       "   bidirectional_last_seen_ms  bidirectional_duration_ms  \\\n",
       "0               1436720908442                      10056   \n",
       "1               1436720906024                          7   \n",
       "2               1436720906022                          0   \n",
       "3               1436720906025                          0   \n",
       "4               1436720911139                       2675   \n",
       "\n",
       "   bidirectional_packets  bidirectional_bytes  src2dst_first_seen_ms  \\\n",
       "0                     68                45688          1436720898386   \n",
       "1                      4                  580          1436720906017   \n",
       "2                      1                  145          1436720906022   \n",
       "3                      1                   66          1436720906025   \n",
       "4                      5                  510          1436720908464   \n",
       "\n",
       "   src2dst_last_seen_ms  src2dst_duration_ms  src2dst_packets  src2dst_bytes  \\\n",
       "0         1436720908442                10056               34           5555   \n",
       "1         1436720906024                    7                4            580   \n",
       "2         1436720906022                    0                1            145   \n",
       "3         1436720906025                    0                1             66   \n",
       "4         1436720911139                 2675                5            510   \n",
       "\n",
       "   dst2src_first_seen_ms  dst2src_last_seen_ms  dst2src_duration_ms  \\\n",
       "0          1436720898475         1436720908442                 9967   \n",
       "1                      0                     0                    0   \n",
       "2                      0                     0                    0   \n",
       "3                      0                     0                    0   \n",
       "4                      0                     0                    0   \n",
       "\n",
       "   dst2src_packets  dst2src_bytes application_name application_category_name  \\\n",
       "0               34          40133              TLS                       Web   \n",
       "1                0              0          Dropbox                     Cloud   \n",
       "2                0              0          Dropbox                     Cloud   \n",
       "3                0              0          Unknown               Unspecified   \n",
       "4                0              0             ICMP                   Network   \n",
       "\n",
       "   application_is_guessed  application_confidence requested_server_name  \\\n",
       "0                       0                       6                   NaN   \n",
       "1                       0                       6                   NaN   \n",
       "2                       0                       6                   NaN   \n",
       "3                       0                       0                   NaN   \n",
       "4                       0                       6                   NaN   \n",
       "\n",
       "  client_fingerprint server_fingerprint user_agent  content_type  \n",
       "0                NaN                NaN        NaN           NaN  \n",
       "1                NaN                NaN        NaN           NaN  \n",
       "2                NaN                NaN        NaN           NaN  \n",
       "3                NaN                NaN        NaN           NaN  \n",
       "4                NaN                NaN        NaN           NaN  "
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(38, 38)"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can enable post-mortem statistical flow features extraction as follow:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = NFStreamer(source=\"pcap/instagram.pcap\", statistical_analysis=True).to_pandas()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>expiration_id</th>\n",
       "      <th>src_ip</th>\n",
       "      <th>src_mac</th>\n",
       "      <th>src_oui</th>\n",
       "      <th>src_port</th>\n",
       "      <th>dst_ip</th>\n",
       "      <th>dst_mac</th>\n",
       "      <th>dst_oui</th>\n",
       "      <th>dst_port</th>\n",
       "      <th>protocol</th>\n",
       "      <th>ip_version</th>\n",
       "      <th>vlan_id</th>\n",
       "      <th>tunnel_id</th>\n",
       "      <th>bidirectional_first_seen_ms</th>\n",
       "      <th>bidirectional_last_seen_ms</th>\n",
       "      <th>bidirectional_duration_ms</th>\n",
       "      <th>bidirectional_packets</th>\n",
       "      <th>bidirectional_bytes</th>\n",
       "      <th>src2dst_first_seen_ms</th>\n",
       "      <th>src2dst_last_seen_ms</th>\n",
       "      <th>src2dst_duration_ms</th>\n",
       "      <th>src2dst_packets</th>\n",
       "      <th>src2dst_bytes</th>\n",
       "      <th>dst2src_first_seen_ms</th>\n",
       "      <th>dst2src_last_seen_ms</th>\n",
       "      <th>dst2src_duration_ms</th>\n",
       "      <th>dst2src_packets</th>\n",
       "      <th>dst2src_bytes</th>\n",
       "      <th>bidirectional_min_ps</th>\n",
       "      <th>bidirectional_mean_ps</th>\n",
       "      <th>bidirectional_stddev_ps</th>\n",
       "      <th>bidirectional_max_ps</th>\n",
       "      <th>src2dst_min_ps</th>\n",
       "      <th>src2dst_mean_ps</th>\n",
       "      <th>src2dst_stddev_ps</th>\n",
       "      <th>src2dst_max_ps</th>\n",
       "      <th>dst2src_min_ps</th>\n",
       "      <th>dst2src_mean_ps</th>\n",
       "      <th>dst2src_stddev_ps</th>\n",
       "      <th>dst2src_max_ps</th>\n",
       "      <th>bidirectional_min_piat_ms</th>\n",
       "      <th>bidirectional_mean_piat_ms</th>\n",
       "      <th>bidirectional_stddev_piat_ms</th>\n",
       "      <th>bidirectional_max_piat_ms</th>\n",
       "      <th>src2dst_min_piat_ms</th>\n",
       "      <th>src2dst_mean_piat_ms</th>\n",
       "      <th>src2dst_stddev_piat_ms</th>\n",
       "      <th>src2dst_max_piat_ms</th>\n",
       "      <th>dst2src_min_piat_ms</th>\n",
       "      <th>dst2src_mean_piat_ms</th>\n",
       "      <th>dst2src_stddev_piat_ms</th>\n",
       "      <th>dst2src_max_piat_ms</th>\n",
       "      <th>bidirectional_syn_packets</th>\n",
       "      <th>bidirectional_cwr_packets</th>\n",
       "      <th>bidirectional_ece_packets</th>\n",
       "      <th>bidirectional_urg_packets</th>\n",
       "      <th>bidirectional_ack_packets</th>\n",
       "      <th>bidirectional_psh_packets</th>\n",
       "      <th>bidirectional_rst_packets</th>\n",
       "      <th>bidirectional_fin_packets</th>\n",
       "      <th>src2dst_syn_packets</th>\n",
       "      <th>src2dst_cwr_packets</th>\n",
       "      <th>src2dst_ece_packets</th>\n",
       "      <th>src2dst_urg_packets</th>\n",
       "      <th>src2dst_ack_packets</th>\n",
       "      <th>src2dst_psh_packets</th>\n",
       "      <th>src2dst_rst_packets</th>\n",
       "      <th>src2dst_fin_packets</th>\n",
       "      <th>dst2src_syn_packets</th>\n",
       "      <th>dst2src_cwr_packets</th>\n",
       "      <th>dst2src_ece_packets</th>\n",
       "      <th>dst2src_urg_packets</th>\n",
       "      <th>dst2src_ack_packets</th>\n",
       "      <th>dst2src_psh_packets</th>\n",
       "      <th>dst2src_rst_packets</th>\n",
       "      <th>dst2src_fin_packets</th>\n",
       "      <th>application_name</th>\n",
       "      <th>application_category_name</th>\n",
       "      <th>application_is_guessed</th>\n",
       "      <th>application_confidence</th>\n",
       "      <th>requested_server_name</th>\n",
       "      <th>client_fingerprint</th>\n",
       "      <th>server_fingerprint</th>\n",
       "      <th>user_agent</th>\n",
       "      <th>content_type</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>192.168.0.103</td>\n",
       "      <td>40:f3:08:c3:8e:e1</td>\n",
       "      <td>40:f3:08</td>\n",
       "      <td>33936</td>\n",
       "      <td>31.13.93.52</td>\n",
       "      <td>00:1b:2f:f0:7e:b4</td>\n",
       "      <td>00:1b:2f</td>\n",
       "      <td>443</td>\n",
       "      <td>6</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1436720898386</td>\n",
       "      <td>1436720908442</td>\n",
       "      <td>10056</td>\n",
       "      <td>68</td>\n",
       "      <td>45688</td>\n",
       "      <td>1436720898386</td>\n",
       "      <td>1436720908442</td>\n",
       "      <td>10056</td>\n",
       "      <td>34</td>\n",
       "      <td>5555</td>\n",
       "      <td>1436720898475</td>\n",
       "      <td>1436720908442</td>\n",
       "      <td>9967</td>\n",
       "      <td>34</td>\n",
       "      <td>40133</td>\n",
       "      <td>66</td>\n",
       "      <td>671.882353</td>\n",
       "      <td>661.76184</td>\n",
       "      <td>1464</td>\n",
       "      <td>66</td>\n",
       "      <td>163.382353</td>\n",
       "      <td>322.650107</td>\n",
       "      <td>1431</td>\n",
       "      <td>66</td>\n",
       "      <td>1180.382353</td>\n",
       "      <td>502.204535</td>\n",
       "      <td>1464</td>\n",
       "      <td>0</td>\n",
       "      <td>150.089552</td>\n",
       "      <td>951.791862</td>\n",
       "      <td>7669</td>\n",
       "      <td>0</td>\n",
       "      <td>304.727273</td>\n",
       "      <td>1349.724098</td>\n",
       "      <td>7669</td>\n",
       "      <td>0</td>\n",
       "      <td>302.030303</td>\n",
       "      <td>1358.385703</td>\n",
       "      <td>7709</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>68</td>\n",
       "      <td>10</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>34</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>34</td>\n",
       "      <td>7</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>TLS</td>\n",
       "      <td>Web</td>\n",
       "      <td>0</td>\n",
       "      <td>6</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>192.168.0.106</td>\n",
       "      <td>00:16:44:1f:59:66</td>\n",
       "      <td>00:16:44</td>\n",
       "      <td>17500</td>\n",
       "      <td>255.255.255.255</td>\n",
       "      <td>ff:ff:ff:ff:ff:ff</td>\n",
       "      <td>ff:ff:ff</td>\n",
       "      <td>17500</td>\n",
       "      <td>17</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1436720906017</td>\n",
       "      <td>1436720906024</td>\n",
       "      <td>7</td>\n",
       "      <td>4</td>\n",
       "      <td>580</td>\n",
       "      <td>1436720906017</td>\n",
       "      <td>1436720906024</td>\n",
       "      <td>7</td>\n",
       "      <td>4</td>\n",
       "      <td>580</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>145</td>\n",
       "      <td>145.000000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>145</td>\n",
       "      <td>145</td>\n",
       "      <td>145.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>145</td>\n",
       "      <td>0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>2.333333</td>\n",
       "      <td>1.527525</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>2.333333</td>\n",
       "      <td>1.527525</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>Dropbox</td>\n",
       "      <td>Cloud</td>\n",
       "      <td>0</td>\n",
       "      <td>6</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>192.168.0.106</td>\n",
       "      <td>00:16:44:1f:59:66</td>\n",
       "      <td>00:16:44</td>\n",
       "      <td>17500</td>\n",
       "      <td>192.168.0.255</td>\n",
       "      <td>ff:ff:ff:ff:ff:ff</td>\n",
       "      <td>ff:ff:ff</td>\n",
       "      <td>17500</td>\n",
       "      <td>17</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1436720906022</td>\n",
       "      <td>1436720906022</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>145</td>\n",
       "      <td>1436720906022</td>\n",
       "      <td>1436720906022</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>145</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>145</td>\n",
       "      <td>145.000000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>145</td>\n",
       "      <td>145</td>\n",
       "      <td>145.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>145</td>\n",
       "      <td>0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>Dropbox</td>\n",
       "      <td>Cloud</td>\n",
       "      <td>0</td>\n",
       "      <td>6</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>192.168.0.1</td>\n",
       "      <td>00:1b:2f:f0:7e:b4</td>\n",
       "      <td>00:1b:2f</td>\n",
       "      <td>520</td>\n",
       "      <td>192.168.0.255</td>\n",
       "      <td>ff:ff:ff:ff:ff:ff</td>\n",
       "      <td>ff:ff:ff</td>\n",
       "      <td>520</td>\n",
       "      <td>17</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1436720906025</td>\n",
       "      <td>1436720906025</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>66</td>\n",
       "      <td>1436720906025</td>\n",
       "      <td>1436720906025</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>66</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>66</td>\n",
       "      <td>66.000000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>66</td>\n",
       "      <td>66</td>\n",
       "      <td>66.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>66</td>\n",
       "      <td>0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>Unknown</td>\n",
       "      <td>Unspecified</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>192.168.0.103</td>\n",
       "      <td>00:00:00:00:00:00</td>\n",
       "      <td>00:00:00</td>\n",
       "      <td>0</td>\n",
       "      <td>192.168.0.103</td>\n",
       "      <td>00:00:00:00:00:00</td>\n",
       "      <td>00:00:00</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1436720908464</td>\n",
       "      <td>1436720911139</td>\n",
       "      <td>2675</td>\n",
       "      <td>5</td>\n",
       "      <td>510</td>\n",
       "      <td>1436720908464</td>\n",
       "      <td>1436720911139</td>\n",
       "      <td>2675</td>\n",
       "      <td>5</td>\n",
       "      <td>510</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>102</td>\n",
       "      <td>102.000000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>102</td>\n",
       "      <td>102</td>\n",
       "      <td>102.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>102</td>\n",
       "      <td>0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>668.750000</td>\n",
       "      <td>1173.672122</td>\n",
       "      <td>2420</td>\n",
       "      <td>0</td>\n",
       "      <td>668.750000</td>\n",
       "      <td>1173.672122</td>\n",
       "      <td>2420</td>\n",
       "      <td>0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>ICMP</td>\n",
       "      <td>Network</td>\n",
       "      <td>0</td>\n",
       "      <td>6</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   id  expiration_id         src_ip            src_mac   src_oui  src_port  \\\n",
       "0   0              0  192.168.0.103  40:f3:08:c3:8e:e1  40:f3:08     33936   \n",
       "1   1              0  192.168.0.106  00:16:44:1f:59:66  00:16:44     17500   \n",
       "2   2              0  192.168.0.106  00:16:44:1f:59:66  00:16:44     17500   \n",
       "3   3              0    192.168.0.1  00:1b:2f:f0:7e:b4  00:1b:2f       520   \n",
       "4   4              0  192.168.0.103  00:00:00:00:00:00  00:00:00         0   \n",
       "\n",
       "            dst_ip            dst_mac   dst_oui  dst_port  protocol  \\\n",
       "0      31.13.93.52  00:1b:2f:f0:7e:b4  00:1b:2f       443         6   \n",
       "1  255.255.255.255  ff:ff:ff:ff:ff:ff  ff:ff:ff     17500        17   \n",
       "2    192.168.0.255  ff:ff:ff:ff:ff:ff  ff:ff:ff     17500        17   \n",
       "3    192.168.0.255  ff:ff:ff:ff:ff:ff  ff:ff:ff       520        17   \n",
       "4    192.168.0.103  00:00:00:00:00:00  00:00:00         0         1   \n",
       "\n",
       "   ip_version  vlan_id  tunnel_id  bidirectional_first_seen_ms  \\\n",
       "0           4        0          0                1436720898386   \n",
       "1           4        0          0                1436720906017   \n",
       "2           4        0          0                1436720906022   \n",
       "3           4        0          0                1436720906025   \n",
       "4           4        0          0                1436720908464   \n",
       "\n",
       "   bidirectional_last_seen_ms  bidirectional_duration_ms  \\\n",
       "0               1436720908442                      10056   \n",
       "1               1436720906024                          7   \n",
       "2               1436720906022                          0   \n",
       "3               1436720906025                          0   \n",
       "4               1436720911139                       2675   \n",
       "\n",
       "   bidirectional_packets  bidirectional_bytes  src2dst_first_seen_ms  \\\n",
       "0                     68                45688          1436720898386   \n",
       "1                      4                  580          1436720906017   \n",
       "2                      1                  145          1436720906022   \n",
       "3                      1                   66          1436720906025   \n",
       "4                      5                  510          1436720908464   \n",
       "\n",
       "   src2dst_last_seen_ms  src2dst_duration_ms  src2dst_packets  src2dst_bytes  \\\n",
       "0         1436720908442                10056               34           5555   \n",
       "1         1436720906024                    7                4            580   \n",
       "2         1436720906022                    0                1            145   \n",
       "3         1436720906025                    0                1             66   \n",
       "4         1436720911139                 2675                5            510   \n",
       "\n",
       "   dst2src_first_seen_ms  dst2src_last_seen_ms  dst2src_duration_ms  \\\n",
       "0          1436720898475         1436720908442                 9967   \n",
       "1                      0                     0                    0   \n",
       "2                      0                     0                    0   \n",
       "3                      0                     0                    0   \n",
       "4                      0                     0                    0   \n",
       "\n",
       "   dst2src_packets  dst2src_bytes  bidirectional_min_ps  \\\n",
       "0               34          40133                    66   \n",
       "1                0              0                   145   \n",
       "2                0              0                   145   \n",
       "3                0              0                    66   \n",
       "4                0              0                   102   \n",
       "\n",
       "   bidirectional_mean_ps  bidirectional_stddev_ps  bidirectional_max_ps  \\\n",
       "0             671.882353                661.76184                  1464   \n",
       "1             145.000000                  0.00000                   145   \n",
       "2             145.000000                  0.00000                   145   \n",
       "3              66.000000                  0.00000                    66   \n",
       "4             102.000000                  0.00000                   102   \n",
       "\n",
       "   src2dst_min_ps  src2dst_mean_ps  src2dst_stddev_ps  src2dst_max_ps  \\\n",
       "0              66       163.382353         322.650107            1431   \n",
       "1             145       145.000000           0.000000             145   \n",
       "2             145       145.000000           0.000000             145   \n",
       "3              66        66.000000           0.000000              66   \n",
       "4             102       102.000000           0.000000             102   \n",
       "\n",
       "   dst2src_min_ps  dst2src_mean_ps  dst2src_stddev_ps  dst2src_max_ps  \\\n",
       "0              66      1180.382353         502.204535            1464   \n",
       "1               0         0.000000           0.000000               0   \n",
       "2               0         0.000000           0.000000               0   \n",
       "3               0         0.000000           0.000000               0   \n",
       "4               0         0.000000           0.000000               0   \n",
       "\n",
       "   bidirectional_min_piat_ms  bidirectional_mean_piat_ms  \\\n",
       "0                          0                  150.089552   \n",
       "1                          1                    2.333333   \n",
       "2                          0                    0.000000   \n",
       "3                          0                    0.000000   \n",
       "4                          0                  668.750000   \n",
       "\n",
       "   bidirectional_stddev_piat_ms  bidirectional_max_piat_ms  \\\n",
       "0                    951.791862                       7669   \n",
       "1                      1.527525                          4   \n",
       "2                      0.000000                          0   \n",
       "3                      0.000000                          0   \n",
       "4                   1173.672122                       2420   \n",
       "\n",
       "   src2dst_min_piat_ms  src2dst_mean_piat_ms  src2dst_stddev_piat_ms  \\\n",
       "0                    0            304.727273             1349.724098   \n",
       "1                    1              2.333333                1.527525   \n",
       "2                    0              0.000000                0.000000   \n",
       "3                    0              0.000000                0.000000   \n",
       "4                    0            668.750000             1173.672122   \n",
       "\n",
       "   src2dst_max_piat_ms  dst2src_min_piat_ms  dst2src_mean_piat_ms  \\\n",
       "0                 7669                    0            302.030303   \n",
       "1                    4                    0              0.000000   \n",
       "2                    0                    0              0.000000   \n",
       "3                    0                    0              0.000000   \n",
       "4                 2420                    0              0.000000   \n",
       "\n",
       "   dst2src_stddev_piat_ms  dst2src_max_piat_ms  bidirectional_syn_packets  \\\n",
       "0             1358.385703                 7709                          0   \n",
       "1                0.000000                    0                          0   \n",
       "2                0.000000                    0                          0   \n",
       "3                0.000000                    0                          0   \n",
       "4                0.000000                    0                          0   \n",
       "\n",
       "   bidirectional_cwr_packets  bidirectional_ece_packets  \\\n",
       "0                          0                          0   \n",
       "1                          0                          0   \n",
       "2                          0                          0   \n",
       "3                          0                          0   \n",
       "4                          0                          0   \n",
       "\n",
       "   bidirectional_urg_packets  bidirectional_ack_packets  \\\n",
       "0                          0                         68   \n",
       "1                          0                          0   \n",
       "2                          0                          0   \n",
       "3                          0                          0   \n",
       "4                          0                          0   \n",
       "\n",
       "   bidirectional_psh_packets  bidirectional_rst_packets  \\\n",
       "0                         10                          0   \n",
       "1                          0                          0   \n",
       "2                          0                          0   \n",
       "3                          0                          0   \n",
       "4                          0                          0   \n",
       "\n",
       "   bidirectional_fin_packets  src2dst_syn_packets  src2dst_cwr_packets  \\\n",
       "0                          0                    0                    0   \n",
       "1                          0                    0                    0   \n",
       "2                          0                    0                    0   \n",
       "3                          0                    0                    0   \n",
       "4                          0                    0                    0   \n",
       "\n",
       "   src2dst_ece_packets  src2dst_urg_packets  src2dst_ack_packets  \\\n",
       "0                    0                    0                   34   \n",
       "1                    0                    0                    0   \n",
       "2                    0                    0                    0   \n",
       "3                    0                    0                    0   \n",
       "4                    0                    0                    0   \n",
       "\n",
       "   src2dst_psh_packets  src2dst_rst_packets  src2dst_fin_packets  \\\n",
       "0                    3                    0                    0   \n",
       "1                    0                    0                    0   \n",
       "2                    0                    0                    0   \n",
       "3                    0                    0                    0   \n",
       "4                    0                    0                    0   \n",
       "\n",
       "   dst2src_syn_packets  dst2src_cwr_packets  dst2src_ece_packets  \\\n",
       "0                    0                    0                    0   \n",
       "1                    0                    0                    0   \n",
       "2                    0                    0                    0   \n",
       "3                    0                    0                    0   \n",
       "4                    0                    0                    0   \n",
       "\n",
       "   dst2src_urg_packets  dst2src_ack_packets  dst2src_psh_packets  \\\n",
       "0                    0                   34                    7   \n",
       "1                    0                    0                    0   \n",
       "2                    0                    0                    0   \n",
       "3                    0                    0                    0   \n",
       "4                    0                    0                    0   \n",
       "\n",
       "   dst2src_rst_packets  dst2src_fin_packets application_name  \\\n",
       "0                    0                    0              TLS   \n",
       "1                    0                    0          Dropbox   \n",
       "2                    0                    0          Dropbox   \n",
       "3                    0                    0          Unknown   \n",
       "4                    0                    0             ICMP   \n",
       "\n",
       "  application_category_name  application_is_guessed  application_confidence  \\\n",
       "0                       Web                       0                       6   \n",
       "1                     Cloud                       0                       6   \n",
       "2                     Cloud                       0                       6   \n",
       "3               Unspecified                       0                       0   \n",
       "4                   Network                       0                       6   \n",
       "\n",
       "  requested_server_name client_fingerprint server_fingerprint user_agent  \\\n",
       "0                   NaN                NaN                NaN        NaN   \n",
       "1                   NaN                NaN                NaN        NaN   \n",
       "2                   NaN                NaN                NaN        NaN   \n",
       "3                   NaN                NaN                NaN        NaN   \n",
       "4                   NaN                NaN                NaN        NaN   \n",
       "\n",
       "   content_type  \n",
       "0           NaN  \n",
       "1           NaN  \n",
       "2           NaN  \n",
       "3           NaN  \n",
       "4           NaN  "
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can enable early statistical flow features extraction as follow:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = NFStreamer(source=\"pcap/instagram.pcap\", splt_analysis=10).to_pandas()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>expiration_id</th>\n",
       "      <th>src_ip</th>\n",
       "      <th>src_mac</th>\n",
       "      <th>src_oui</th>\n",
       "      <th>src_port</th>\n",
       "      <th>dst_ip</th>\n",
       "      <th>dst_mac</th>\n",
       "      <th>dst_oui</th>\n",
       "      <th>dst_port</th>\n",
       "      <th>protocol</th>\n",
       "      <th>ip_version</th>\n",
       "      <th>vlan_id</th>\n",
       "      <th>tunnel_id</th>\n",
       "      <th>bidirectional_first_seen_ms</th>\n",
       "      <th>bidirectional_last_seen_ms</th>\n",
       "      <th>bidirectional_duration_ms</th>\n",
       "      <th>bidirectional_packets</th>\n",
       "      <th>bidirectional_bytes</th>\n",
       "      <th>src2dst_first_seen_ms</th>\n",
       "      <th>src2dst_last_seen_ms</th>\n",
       "      <th>src2dst_duration_ms</th>\n",
       "      <th>src2dst_packets</th>\n",
       "      <th>src2dst_bytes</th>\n",
       "      <th>dst2src_first_seen_ms</th>\n",
       "      <th>dst2src_last_seen_ms</th>\n",
       "      <th>dst2src_duration_ms</th>\n",
       "      <th>dst2src_packets</th>\n",
       "      <th>dst2src_bytes</th>\n",
       "      <th>splt_direction</th>\n",
       "      <th>splt_ps</th>\n",
       "      <th>splt_piat_ms</th>\n",
       "      <th>application_name</th>\n",
       "      <th>application_category_name</th>\n",
       "      <th>application_is_guessed</th>\n",
       "      <th>application_confidence</th>\n",
       "      <th>requested_server_name</th>\n",
       "      <th>client_fingerprint</th>\n",
       "      <th>server_fingerprint</th>\n",
       "      <th>user_agent</th>\n",
       "      <th>content_type</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>192.168.0.103</td>\n",
       "      <td>40:f3:08:c3:8e:e1</td>\n",
       "      <td>40:f3:08</td>\n",
       "      <td>33936</td>\n",
       "      <td>31.13.93.52</td>\n",
       "      <td>00:1b:2f:f0:7e:b4</td>\n",
       "      <td>00:1b:2f</td>\n",
       "      <td>443</td>\n",
       "      <td>6</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1436720898386</td>\n",
       "      <td>1436720908442</td>\n",
       "      <td>10056</td>\n",
       "      <td>68</td>\n",
       "      <td>45688</td>\n",
       "      <td>1436720898386</td>\n",
       "      <td>1436720908442</td>\n",
       "      <td>10056</td>\n",
       "      <td>34</td>\n",
       "      <td>5555</td>\n",
       "      <td>1436720898475</td>\n",
       "      <td>1436720908442</td>\n",
       "      <td>9967</td>\n",
       "      <td>34</td>\n",
       "      <td>40133</td>\n",
       "      <td>[0, 1, 1, 0, 0, 1, 1, 0, 1, 0]</td>\n",
       "      <td>[1431, 66, 679, 66, 1063, 66, 1464, 66, 209, 66]</td>\n",
       "      <td>[0, 89, 76, 0, 1523, 50, 340, 0, 2, 0]</td>\n",
       "      <td>TLS</td>\n",
       "      <td>Web</td>\n",
       "      <td>0</td>\n",
       "      <td>6</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>192.168.0.106</td>\n",
       "      <td>00:16:44:1f:59:66</td>\n",
       "      <td>00:16:44</td>\n",
       "      <td>17500</td>\n",
       "      <td>255.255.255.255</td>\n",
       "      <td>ff:ff:ff:ff:ff:ff</td>\n",
       "      <td>ff:ff:ff</td>\n",
       "      <td>17500</td>\n",
       "      <td>17</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1436720906017</td>\n",
       "      <td>1436720906024</td>\n",
       "      <td>7</td>\n",
       "      <td>4</td>\n",
       "      <td>580</td>\n",
       "      <td>1436720906017</td>\n",
       "      <td>1436720906024</td>\n",
       "      <td>7</td>\n",
       "      <td>4</td>\n",
       "      <td>580</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>[0, 0, 0, 0, -1, -1, -1, -1, -1, -1]</td>\n",
       "      <td>[145, 145, 145, 145, -1, -1, -1, -1, -1, -1]</td>\n",
       "      <td>[0, 2, 1, 4, -1, -1, -1, -1, -1, -1]</td>\n",
       "      <td>Dropbox</td>\n",
       "      <td>Cloud</td>\n",
       "      <td>0</td>\n",
       "      <td>6</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>192.168.0.106</td>\n",
       "      <td>00:16:44:1f:59:66</td>\n",
       "      <td>00:16:44</td>\n",
       "      <td>17500</td>\n",
       "      <td>192.168.0.255</td>\n",
       "      <td>ff:ff:ff:ff:ff:ff</td>\n",
       "      <td>ff:ff:ff</td>\n",
       "      <td>17500</td>\n",
       "      <td>17</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1436720906022</td>\n",
       "      <td>1436720906022</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>145</td>\n",
       "      <td>1436720906022</td>\n",
       "      <td>1436720906022</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>145</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>[0, -1, -1, -1, -1, -1, -1, -1, -1, -1]</td>\n",
       "      <td>[145, -1, -1, -1, -1, -1, -1, -1, -1, -1]</td>\n",
       "      <td>[0, -1, -1, -1, -1, -1, -1, -1, -1, -1]</td>\n",
       "      <td>Dropbox</td>\n",
       "      <td>Cloud</td>\n",
       "      <td>0</td>\n",
       "      <td>6</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>192.168.0.1</td>\n",
       "      <td>00:1b:2f:f0:7e:b4</td>\n",
       "      <td>00:1b:2f</td>\n",
       "      <td>520</td>\n",
       "      <td>192.168.0.255</td>\n",
       "      <td>ff:ff:ff:ff:ff:ff</td>\n",
       "      <td>ff:ff:ff</td>\n",
       "      <td>520</td>\n",
       "      <td>17</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1436720906025</td>\n",
       "      <td>1436720906025</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>66</td>\n",
       "      <td>1436720906025</td>\n",
       "      <td>1436720906025</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>66</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>[0, -1, -1, -1, -1, -1, -1, -1, -1, -1]</td>\n",
       "      <td>[66, -1, -1, -1, -1, -1, -1, -1, -1, -1]</td>\n",
       "      <td>[0, -1, -1, -1, -1, -1, -1, -1, -1, -1]</td>\n",
       "      <td>Unknown</td>\n",
       "      <td>Unspecified</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>192.168.0.103</td>\n",
       "      <td>40:f3:08:c3:8e:e1</td>\n",
       "      <td>40:f3:08</td>\n",
       "      <td>38816</td>\n",
       "      <td>46.33.70.160</td>\n",
       "      <td>00:1b:2f:f0:7e:b4</td>\n",
       "      <td>00:1b:2f</td>\n",
       "      <td>80</td>\n",
       "      <td>6</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1436720900684</td>\n",
       "      <td>1436720900750</td>\n",
       "      <td>66</td>\n",
       "      <td>52</td>\n",
       "      <td>58994</td>\n",
       "      <td>1436720900684</td>\n",
       "      <td>1436720900750</td>\n",
       "      <td>66</td>\n",
       "      <td>13</td>\n",
       "      <td>1118</td>\n",
       "      <td>1436720900716</td>\n",
       "      <td>1436720900744</td>\n",
       "      <td>28</td>\n",
       "      <td>39</td>\n",
       "      <td>57876</td>\n",
       "      <td>[0, 1, 0, 1, 1, 1, 1, 1, 1, 1]</td>\n",
       "      <td>[326, 1484, 66, 1484, 1484, 1484, 1484, 1484, ...</td>\n",
       "      <td>[0, 32, 1, 0, 1, 2, 2, 0, 0, 0]</td>\n",
       "      <td>HTTP.Instagram</td>\n",
       "      <td>SocialNetwork</td>\n",
       "      <td>0</td>\n",
       "      <td>6</td>\n",
       "      <td>photos-h.ak.instagram.com</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Instagram 7.1.1 Android (19/4.4.2; 480dpi; 108...</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   id  expiration_id         src_ip            src_mac   src_oui  src_port  \\\n",
       "0   0              0  192.168.0.103  40:f3:08:c3:8e:e1  40:f3:08     33936   \n",
       "1   1              0  192.168.0.106  00:16:44:1f:59:66  00:16:44     17500   \n",
       "2   2              0  192.168.0.106  00:16:44:1f:59:66  00:16:44     17500   \n",
       "3   3              0    192.168.0.1  00:1b:2f:f0:7e:b4  00:1b:2f       520   \n",
       "4   4              0  192.168.0.103  40:f3:08:c3:8e:e1  40:f3:08     38816   \n",
       "\n",
       "            dst_ip            dst_mac   dst_oui  dst_port  protocol  \\\n",
       "0      31.13.93.52  00:1b:2f:f0:7e:b4  00:1b:2f       443         6   \n",
       "1  255.255.255.255  ff:ff:ff:ff:ff:ff  ff:ff:ff     17500        17   \n",
       "2    192.168.0.255  ff:ff:ff:ff:ff:ff  ff:ff:ff     17500        17   \n",
       "3    192.168.0.255  ff:ff:ff:ff:ff:ff  ff:ff:ff       520        17   \n",
       "4     46.33.70.160  00:1b:2f:f0:7e:b4  00:1b:2f        80         6   \n",
       "\n",
       "   ip_version  vlan_id  tunnel_id  bidirectional_first_seen_ms  \\\n",
       "0           4        0          0                1436720898386   \n",
       "1           4        0          0                1436720906017   \n",
       "2           4        0          0                1436720906022   \n",
       "3           4        0          0                1436720906025   \n",
       "4           4        0          0                1436720900684   \n",
       "\n",
       "   bidirectional_last_seen_ms  bidirectional_duration_ms  \\\n",
       "0               1436720908442                      10056   \n",
       "1               1436720906024                          7   \n",
       "2               1436720906022                          0   \n",
       "3               1436720906025                          0   \n",
       "4               1436720900750                         66   \n",
       "\n",
       "   bidirectional_packets  bidirectional_bytes  src2dst_first_seen_ms  \\\n",
       "0                     68                45688          1436720898386   \n",
       "1                      4                  580          1436720906017   \n",
       "2                      1                  145          1436720906022   \n",
       "3                      1                   66          1436720906025   \n",
       "4                     52                58994          1436720900684   \n",
       "\n",
       "   src2dst_last_seen_ms  src2dst_duration_ms  src2dst_packets  src2dst_bytes  \\\n",
       "0         1436720908442                10056               34           5555   \n",
       "1         1436720906024                    7                4            580   \n",
       "2         1436720906022                    0                1            145   \n",
       "3         1436720906025                    0                1             66   \n",
       "4         1436720900750                   66               13           1118   \n",
       "\n",
       "   dst2src_first_seen_ms  dst2src_last_seen_ms  dst2src_duration_ms  \\\n",
       "0          1436720898475         1436720908442                 9967   \n",
       "1                      0                     0                    0   \n",
       "2                      0                     0                    0   \n",
       "3                      0                     0                    0   \n",
       "4          1436720900716         1436720900744                   28   \n",
       "\n",
       "   dst2src_packets  dst2src_bytes                           splt_direction  \\\n",
       "0               34          40133           [0, 1, 1, 0, 0, 1, 1, 0, 1, 0]   \n",
       "1                0              0     [0, 0, 0, 0, -1, -1, -1, -1, -1, -1]   \n",
       "2                0              0  [0, -1, -1, -1, -1, -1, -1, -1, -1, -1]   \n",
       "3                0              0  [0, -1, -1, -1, -1, -1, -1, -1, -1, -1]   \n",
       "4               39          57876           [0, 1, 0, 1, 1, 1, 1, 1, 1, 1]   \n",
       "\n",
       "                                             splt_ps  \\\n",
       "0   [1431, 66, 679, 66, 1063, 66, 1464, 66, 209, 66]   \n",
       "1       [145, 145, 145, 145, -1, -1, -1, -1, -1, -1]   \n",
       "2          [145, -1, -1, -1, -1, -1, -1, -1, -1, -1]   \n",
       "3           [66, -1, -1, -1, -1, -1, -1, -1, -1, -1]   \n",
       "4  [326, 1484, 66, 1484, 1484, 1484, 1484, 1484, ...   \n",
       "\n",
       "                              splt_piat_ms application_name  \\\n",
       "0   [0, 89, 76, 0, 1523, 50, 340, 0, 2, 0]              TLS   \n",
       "1     [0, 2, 1, 4, -1, -1, -1, -1, -1, -1]          Dropbox   \n",
       "2  [0, -1, -1, -1, -1, -1, -1, -1, -1, -1]          Dropbox   \n",
       "3  [0, -1, -1, -1, -1, -1, -1, -1, -1, -1]          Unknown   \n",
       "4          [0, 32, 1, 0, 1, 2, 2, 0, 0, 0]   HTTP.Instagram   \n",
       "\n",
       "  application_category_name  application_is_guessed  application_confidence  \\\n",
       "0                       Web                       0                       6   \n",
       "1                     Cloud                       0                       6   \n",
       "2                     Cloud                       0                       6   \n",
       "3               Unspecified                       0                       0   \n",
       "4             SocialNetwork                       0                       6   \n",
       "\n",
       "       requested_server_name client_fingerprint server_fingerprint  \\\n",
       "0                        NaN                NaN                NaN   \n",
       "1                        NaN                NaN                NaN   \n",
       "2                        NaN                NaN                NaN   \n",
       "3                        NaN                NaN                NaN   \n",
       "4  photos-h.ak.instagram.com                NaN                NaN   \n",
       "\n",
       "                                          user_agent  content_type  \n",
       "0                                                NaN           NaN  \n",
       "1                                                NaN           NaN  \n",
       "2                                                NaN           NaN  \n",
       "3                                                NaN           NaN  \n",
       "4  Instagram 7.1.1 Android (19/4.4.2; 480dpi; 108...           NaN  "
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can enable IP anonymization as follow:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = NFStreamer(source=\"pcap/instagram.pcap\", \n",
    "                statistical_analysis=True).to_pandas(columns_to_anonymize=[\"src_ip\", \"src_mac\", \"dst_ip\", \"dst_mac\"])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>expiration_id</th>\n",
       "      <th>src_ip</th>\n",
       "      <th>src_mac</th>\n",
       "      <th>src_oui</th>\n",
       "      <th>src_port</th>\n",
       "      <th>dst_ip</th>\n",
       "      <th>dst_mac</th>\n",
       "      <th>dst_oui</th>\n",
       "      <th>dst_port</th>\n",
       "      <th>protocol</th>\n",
       "      <th>ip_version</th>\n",
       "      <th>vlan_id</th>\n",
       "      <th>tunnel_id</th>\n",
       "      <th>bidirectional_first_seen_ms</th>\n",
       "      <th>bidirectional_last_seen_ms</th>\n",
       "      <th>bidirectional_duration_ms</th>\n",
       "      <th>bidirectional_packets</th>\n",
       "      <th>bidirectional_bytes</th>\n",
       "      <th>src2dst_first_seen_ms</th>\n",
       "      <th>src2dst_last_seen_ms</th>\n",
       "      <th>src2dst_duration_ms</th>\n",
       "      <th>src2dst_packets</th>\n",
       "      <th>src2dst_bytes</th>\n",
       "      <th>dst2src_first_seen_ms</th>\n",
       "      <th>dst2src_last_seen_ms</th>\n",
       "      <th>dst2src_duration_ms</th>\n",
       "      <th>dst2src_packets</th>\n",
       "      <th>dst2src_bytes</th>\n",
       "      <th>bidirectional_min_ps</th>\n",
       "      <th>bidirectional_mean_ps</th>\n",
       "      <th>bidirectional_stddev_ps</th>\n",
       "      <th>bidirectional_max_ps</th>\n",
       "      <th>src2dst_min_ps</th>\n",
       "      <th>src2dst_mean_ps</th>\n",
       "      <th>src2dst_stddev_ps</th>\n",
       "      <th>src2dst_max_ps</th>\n",
       "      <th>dst2src_min_ps</th>\n",
       "      <th>dst2src_mean_ps</th>\n",
       "      <th>dst2src_stddev_ps</th>\n",
       "      <th>dst2src_max_ps</th>\n",
       "      <th>bidirectional_min_piat_ms</th>\n",
       "      <th>bidirectional_mean_piat_ms</th>\n",
       "      <th>bidirectional_stddev_piat_ms</th>\n",
       "      <th>bidirectional_max_piat_ms</th>\n",
       "      <th>src2dst_min_piat_ms</th>\n",
       "      <th>src2dst_mean_piat_ms</th>\n",
       "      <th>src2dst_stddev_piat_ms</th>\n",
       "      <th>src2dst_max_piat_ms</th>\n",
       "      <th>dst2src_min_piat_ms</th>\n",
       "      <th>dst2src_mean_piat_ms</th>\n",
       "      <th>dst2src_stddev_piat_ms</th>\n",
       "      <th>dst2src_max_piat_ms</th>\n",
       "      <th>bidirectional_syn_packets</th>\n",
       "      <th>bidirectional_cwr_packets</th>\n",
       "      <th>bidirectional_ece_packets</th>\n",
       "      <th>bidirectional_urg_packets</th>\n",
       "      <th>bidirectional_ack_packets</th>\n",
       "      <th>bidirectional_psh_packets</th>\n",
       "      <th>bidirectional_rst_packets</th>\n",
       "      <th>bidirectional_fin_packets</th>\n",
       "      <th>src2dst_syn_packets</th>\n",
       "      <th>src2dst_cwr_packets</th>\n",
       "      <th>src2dst_ece_packets</th>\n",
       "      <th>src2dst_urg_packets</th>\n",
       "      <th>src2dst_ack_packets</th>\n",
       "      <th>src2dst_psh_packets</th>\n",
       "      <th>src2dst_rst_packets</th>\n",
       "      <th>src2dst_fin_packets</th>\n",
       "      <th>dst2src_syn_packets</th>\n",
       "      <th>dst2src_cwr_packets</th>\n",
       "      <th>dst2src_ece_packets</th>\n",
       "      <th>dst2src_urg_packets</th>\n",
       "      <th>dst2src_ack_packets</th>\n",
       "      <th>dst2src_psh_packets</th>\n",
       "      <th>dst2src_rst_packets</th>\n",
       "      <th>dst2src_fin_packets</th>\n",
       "      <th>application_name</th>\n",
       "      <th>application_category_name</th>\n",
       "      <th>application_is_guessed</th>\n",
       "      <th>application_confidence</th>\n",
       "      <th>requested_server_name</th>\n",
       "      <th>client_fingerprint</th>\n",
       "      <th>server_fingerprint</th>\n",
       "      <th>user_agent</th>\n",
       "      <th>content_type</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>5885370fbc1de250a4570351f2679e915e15245a5534bd...</td>\n",
       "      <td>b5d836f0b4088481bd22d1bcdbf78c8bb4ed6c5b5a3175...</td>\n",
       "      <td>40:f3:08</td>\n",
       "      <td>57936</td>\n",
       "      <td>3a44c94fd7c9aefa07df278f016460d75aa809c94a571c...</td>\n",
       "      <td>7f6b3b13330898c4dcf505e44b642c996b9674139831e0...</td>\n",
       "      <td>00:1b:2f</td>\n",
       "      <td>80</td>\n",
       "      <td>6</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1436720900687</td>\n",
       "      <td>1436720901200</td>\n",
       "      <td>513</td>\n",
       "      <td>58</td>\n",
       "      <td>50220</td>\n",
       "      <td>1436720900687</td>\n",
       "      <td>1436720901200</td>\n",
       "      <td>513</td>\n",
       "      <td>24</td>\n",
       "      <td>1837</td>\n",
       "      <td>1436720900744</td>\n",
       "      <td>1436720901200</td>\n",
       "      <td>456</td>\n",
       "      <td>34</td>\n",
       "      <td>48383</td>\n",
       "      <td>66</td>\n",
       "      <td>865.862069</td>\n",
       "      <td>696.739485</td>\n",
       "      <td>1484</td>\n",
       "      <td>66</td>\n",
       "      <td>76.541667</td>\n",
       "      <td>51.643409</td>\n",
       "      <td>319</td>\n",
       "      <td>186</td>\n",
       "      <td>1423.029412</td>\n",
       "      <td>252.360311</td>\n",
       "      <td>1484</td>\n",
       "      <td>0</td>\n",
       "      <td>9.000000</td>\n",
       "      <td>45.124035</td>\n",
       "      <td>321</td>\n",
       "      <td>0</td>\n",
       "      <td>22.304348</td>\n",
       "      <td>70.131976</td>\n",
       "      <td>322</td>\n",
       "      <td>0</td>\n",
       "      <td>13.818182</td>\n",
       "      <td>58.73109</td>\n",
       "      <td>323</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>58</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>24</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>34</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>HTTP.Instagram</td>\n",
       "      <td>SocialNetwork</td>\n",
       "      <td>0</td>\n",
       "      <td>6</td>\n",
       "      <td>photos-g.ak.instagram.com</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Instagram 7.1.1 Android (19/4.4.2; 480dpi; 108...</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>5885370fbc1de250a4570351f2679e915e15245a5534bd...</td>\n",
       "      <td>b5d836f0b4088481bd22d1bcdbf78c8bb4ed6c5b5a3175...</td>\n",
       "      <td>40:f3:08</td>\n",
       "      <td>38816</td>\n",
       "      <td>c26866c915aaa410921c4fc309477eb0ceba2caec77bcf...</td>\n",
       "      <td>7f6b3b13330898c4dcf505e44b642c996b9674139831e0...</td>\n",
       "      <td>00:1b:2f</td>\n",
       "      <td>80</td>\n",
       "      <td>6</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1436720900684</td>\n",
       "      <td>1436720900750</td>\n",
       "      <td>66</td>\n",
       "      <td>52</td>\n",
       "      <td>58994</td>\n",
       "      <td>1436720900684</td>\n",
       "      <td>1436720900750</td>\n",
       "      <td>66</td>\n",
       "      <td>13</td>\n",
       "      <td>1118</td>\n",
       "      <td>1436720900716</td>\n",
       "      <td>1436720900744</td>\n",
       "      <td>28</td>\n",
       "      <td>39</td>\n",
       "      <td>57876</td>\n",
       "      <td>66</td>\n",
       "      <td>1134.500000</td>\n",
       "      <td>612.257779</td>\n",
       "      <td>1484</td>\n",
       "      <td>66</td>\n",
       "      <td>86.000000</td>\n",
       "      <td>72.111026</td>\n",
       "      <td>326</td>\n",
       "      <td>1484</td>\n",
       "      <td>1484.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1484</td>\n",
       "      <td>0</td>\n",
       "      <td>1.294118</td>\n",
       "      <td>4.495750</td>\n",
       "      <td>32</td>\n",
       "      <td>0</td>\n",
       "      <td>5.500000</td>\n",
       "      <td>9.170110</td>\n",
       "      <td>33</td>\n",
       "      <td>0</td>\n",
       "      <td>0.736842</td>\n",
       "      <td>0.68514</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>52</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>13</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>39</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>HTTP.Instagram</td>\n",
       "      <td>SocialNetwork</td>\n",
       "      <td>0</td>\n",
       "      <td>6</td>\n",
       "      <td>photos-h.ak.instagram.com</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Instagram 7.1.1 Android (19/4.4.2; 480dpi; 108...</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>5885370fbc1de250a4570351f2679e915e15245a5534bd...</td>\n",
       "      <td>b5d836f0b4088481bd22d1bcdbf78c8bb4ed6c5b5a3175...</td>\n",
       "      <td>40:f3:08</td>\n",
       "      <td>37350</td>\n",
       "      <td>68df6e56301d6c238c302eaa732d2075ef802914771e7b...</td>\n",
       "      <td>7f6b3b13330898c4dcf505e44b642c996b9674139831e0...</td>\n",
       "      <td>00:1b:2f</td>\n",
       "      <td>80</td>\n",
       "      <td>6</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1436720901262</td>\n",
       "      <td>1436720901262</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>324</td>\n",
       "      <td>1436720901262</td>\n",
       "      <td>1436720901262</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>324</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>324</td>\n",
       "      <td>324.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>324</td>\n",
       "      <td>324</td>\n",
       "      <td>324.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>324</td>\n",
       "      <td>0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>HTTP.Instagram</td>\n",
       "      <td>SocialNetwork</td>\n",
       "      <td>0</td>\n",
       "      <td>6</td>\n",
       "      <td>photos-a.ak.instagram.com</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Instagram 7.1.1 Android (19/4.4.2; 480dpi; 108...</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>5885370fbc1de250a4570351f2679e915e15245a5534bd...</td>\n",
       "      <td>b5d836f0b4088481bd22d1bcdbf78c8bb4ed6c5b5a3175...</td>\n",
       "      <td>40:f3:08</td>\n",
       "      <td>33603</td>\n",
       "      <td>107d5f2f69c5e2f3da41be7ce2a59c0d818947212d6ef0...</td>\n",
       "      <td>7f6b3b13330898c4dcf505e44b642c996b9674139831e0...</td>\n",
       "      <td>00:1b:2f</td>\n",
       "      <td>53</td>\n",
       "      <td>17</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1436720908524</td>\n",
       "      <td>1436720908575</td>\n",
       "      <td>51</td>\n",
       "      <td>2</td>\n",
       "      <td>298</td>\n",
       "      <td>1436720908524</td>\n",
       "      <td>1436720908524</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>89</td>\n",
       "      <td>1436720908575</td>\n",
       "      <td>1436720908575</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>209</td>\n",
       "      <td>89</td>\n",
       "      <td>149.000000</td>\n",
       "      <td>84.852814</td>\n",
       "      <td>209</td>\n",
       "      <td>89</td>\n",
       "      <td>89.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>89</td>\n",
       "      <td>209</td>\n",
       "      <td>209.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>209</td>\n",
       "      <td>51</td>\n",
       "      <td>51.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>51</td>\n",
       "      <td>0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>DNS.Instagram</td>\n",
       "      <td>Network</td>\n",
       "      <td>0</td>\n",
       "      <td>6</td>\n",
       "      <td>igcdn-photos-a-a.akamaihd.net</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>d7feda5309e4f8477aac71903e83486f9f13566cc836f8...</td>\n",
       "      <td>7f6b3b13330898c4dcf505e44b642c996b9674139831e0...</td>\n",
       "      <td>00:1b:2f</td>\n",
       "      <td>80</td>\n",
       "      <td>5885370fbc1de250a4570351f2679e915e15245a5534bd...</td>\n",
       "      <td>b5d836f0b4088481bd22d1bcdbf78c8bb4ed6c5b5a3175...</td>\n",
       "      <td>40:f3:08</td>\n",
       "      <td>40855</td>\n",
       "      <td>6</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1436720952611</td>\n",
       "      <td>1436720952611</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>140</td>\n",
       "      <td>1436720952611</td>\n",
       "      <td>1436720952611</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>74</td>\n",
       "      <td>1436720952611</td>\n",
       "      <td>1436720952611</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>66</td>\n",
       "      <td>66</td>\n",
       "      <td>70.000000</td>\n",
       "      <td>5.656854</td>\n",
       "      <td>74</td>\n",
       "      <td>74</td>\n",
       "      <td>74.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>74</td>\n",
       "      <td>66</td>\n",
       "      <td>66.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>66</td>\n",
       "      <td>0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>HTTP</td>\n",
       "      <td>Web</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   id  expiration_id                                             src_ip  \\\n",
       "0   0              0  5885370fbc1de250a4570351f2679e915e15245a5534bd...   \n",
       "1   1              0  5885370fbc1de250a4570351f2679e915e15245a5534bd...   \n",
       "2   2              0  5885370fbc1de250a4570351f2679e915e15245a5534bd...   \n",
       "3   3              0  5885370fbc1de250a4570351f2679e915e15245a5534bd...   \n",
       "4   4              0  d7feda5309e4f8477aac71903e83486f9f13566cc836f8...   \n",
       "\n",
       "                                             src_mac   src_oui  src_port  \\\n",
       "0  b5d836f0b4088481bd22d1bcdbf78c8bb4ed6c5b5a3175...  40:f3:08     57936   \n",
       "1  b5d836f0b4088481bd22d1bcdbf78c8bb4ed6c5b5a3175...  40:f3:08     38816   \n",
       "2  b5d836f0b4088481bd22d1bcdbf78c8bb4ed6c5b5a3175...  40:f3:08     37350   \n",
       "3  b5d836f0b4088481bd22d1bcdbf78c8bb4ed6c5b5a3175...  40:f3:08     33603   \n",
       "4  7f6b3b13330898c4dcf505e44b642c996b9674139831e0...  00:1b:2f        80   \n",
       "\n",
       "                                              dst_ip  \\\n",
       "0  3a44c94fd7c9aefa07df278f016460d75aa809c94a571c...   \n",
       "1  c26866c915aaa410921c4fc309477eb0ceba2caec77bcf...   \n",
       "2  68df6e56301d6c238c302eaa732d2075ef802914771e7b...   \n",
       "3  107d5f2f69c5e2f3da41be7ce2a59c0d818947212d6ef0...   \n",
       "4  5885370fbc1de250a4570351f2679e915e15245a5534bd...   \n",
       "\n",
       "                                             dst_mac   dst_oui  dst_port  \\\n",
       "0  7f6b3b13330898c4dcf505e44b642c996b9674139831e0...  00:1b:2f        80   \n",
       "1  7f6b3b13330898c4dcf505e44b642c996b9674139831e0...  00:1b:2f        80   \n",
       "2  7f6b3b13330898c4dcf505e44b642c996b9674139831e0...  00:1b:2f        80   \n",
       "3  7f6b3b13330898c4dcf505e44b642c996b9674139831e0...  00:1b:2f        53   \n",
       "4  b5d836f0b4088481bd22d1bcdbf78c8bb4ed6c5b5a3175...  40:f3:08     40855   \n",
       "\n",
       "   protocol  ip_version  vlan_id  tunnel_id  bidirectional_first_seen_ms  \\\n",
       "0         6           4        0          0                1436720900687   \n",
       "1         6           4        0          0                1436720900684   \n",
       "2         6           4        0          0                1436720901262   \n",
       "3        17           4        0          0                1436720908524   \n",
       "4         6           4        0          0                1436720952611   \n",
       "\n",
       "   bidirectional_last_seen_ms  bidirectional_duration_ms  \\\n",
       "0               1436720901200                        513   \n",
       "1               1436720900750                         66   \n",
       "2               1436720901262                          0   \n",
       "3               1436720908575                         51   \n",
       "4               1436720952611                          0   \n",
       "\n",
       "   bidirectional_packets  bidirectional_bytes  src2dst_first_seen_ms  \\\n",
       "0                     58                50220          1436720900687   \n",
       "1                     52                58994          1436720900684   \n",
       "2                      1                  324          1436720901262   \n",
       "3                      2                  298          1436720908524   \n",
       "4                      2                  140          1436720952611   \n",
       "\n",
       "   src2dst_last_seen_ms  src2dst_duration_ms  src2dst_packets  src2dst_bytes  \\\n",
       "0         1436720901200                  513               24           1837   \n",
       "1         1436720900750                   66               13           1118   \n",
       "2         1436720901262                    0                1            324   \n",
       "3         1436720908524                    0                1             89   \n",
       "4         1436720952611                    0                1             74   \n",
       "\n",
       "   dst2src_first_seen_ms  dst2src_last_seen_ms  dst2src_duration_ms  \\\n",
       "0          1436720900744         1436720901200                  456   \n",
       "1          1436720900716         1436720900744                   28   \n",
       "2                      0                     0                    0   \n",
       "3          1436720908575         1436720908575                    0   \n",
       "4          1436720952611         1436720952611                    0   \n",
       "\n",
       "   dst2src_packets  dst2src_bytes  bidirectional_min_ps  \\\n",
       "0               34          48383                    66   \n",
       "1               39          57876                    66   \n",
       "2                0              0                   324   \n",
       "3                1            209                    89   \n",
       "4                1             66                    66   \n",
       "\n",
       "   bidirectional_mean_ps  bidirectional_stddev_ps  bidirectional_max_ps  \\\n",
       "0             865.862069               696.739485                  1484   \n",
       "1            1134.500000               612.257779                  1484   \n",
       "2             324.000000                 0.000000                   324   \n",
       "3             149.000000                84.852814                   209   \n",
       "4              70.000000                 5.656854                    74   \n",
       "\n",
       "   src2dst_min_ps  src2dst_mean_ps  src2dst_stddev_ps  src2dst_max_ps  \\\n",
       "0              66        76.541667          51.643409             319   \n",
       "1              66        86.000000          72.111026             326   \n",
       "2             324       324.000000           0.000000             324   \n",
       "3              89        89.000000           0.000000              89   \n",
       "4              74        74.000000           0.000000              74   \n",
       "\n",
       "   dst2src_min_ps  dst2src_mean_ps  dst2src_stddev_ps  dst2src_max_ps  \\\n",
       "0             186      1423.029412         252.360311            1484   \n",
       "1            1484      1484.000000           0.000000            1484   \n",
       "2               0         0.000000           0.000000               0   \n",
       "3             209       209.000000           0.000000             209   \n",
       "4              66        66.000000           0.000000              66   \n",
       "\n",
       "   bidirectional_min_piat_ms  bidirectional_mean_piat_ms  \\\n",
       "0                          0                    9.000000   \n",
       "1                          0                    1.294118   \n",
       "2                          0                    0.000000   \n",
       "3                         51                   51.000000   \n",
       "4                          0                    0.000000   \n",
       "\n",
       "   bidirectional_stddev_piat_ms  bidirectional_max_piat_ms  \\\n",
       "0                     45.124035                        321   \n",
       "1                      4.495750                         32   \n",
       "2                      0.000000                          0   \n",
       "3                      0.000000                         51   \n",
       "4                      0.000000                          0   \n",
       "\n",
       "   src2dst_min_piat_ms  src2dst_mean_piat_ms  src2dst_stddev_piat_ms  \\\n",
       "0                    0             22.304348               70.131976   \n",
       "1                    0              5.500000                9.170110   \n",
       "2                    0              0.000000                0.000000   \n",
       "3                    0              0.000000                0.000000   \n",
       "4                    0              0.000000                0.000000   \n",
       "\n",
       "   src2dst_max_piat_ms  dst2src_min_piat_ms  dst2src_mean_piat_ms  \\\n",
       "0                  322                    0             13.818182   \n",
       "1                   33                    0              0.736842   \n",
       "2                    0                    0              0.000000   \n",
       "3                    0                    0              0.000000   \n",
       "4                    0                    0              0.000000   \n",
       "\n",
       "   dst2src_stddev_piat_ms  dst2src_max_piat_ms  bidirectional_syn_packets  \\\n",
       "0                58.73109                  323                          0   \n",
       "1                 0.68514                    2                          0   \n",
       "2                 0.00000                    0                          0   \n",
       "3                 0.00000                    0                          0   \n",
       "4                 0.00000                    0                          1   \n",
       "\n",
       "   bidirectional_cwr_packets  bidirectional_ece_packets  \\\n",
       "0                          0                          0   \n",
       "1                          0                          0   \n",
       "2                          0                          0   \n",
       "3                          0                          0   \n",
       "4                          0                          0   \n",
       "\n",
       "   bidirectional_urg_packets  bidirectional_ack_packets  \\\n",
       "0                          0                         58   \n",
       "1                          0                         52   \n",
       "2                          0                          1   \n",
       "3                          0                          0   \n",
       "4                          0                          2   \n",
       "\n",
       "   bidirectional_psh_packets  bidirectional_rst_packets  \\\n",
       "0                          4                          0   \n",
       "1                          1                          0   \n",
       "2                          1                          0   \n",
       "3                          0                          0   \n",
       "4                          0                          0   \n",
       "\n",
       "   bidirectional_fin_packets  src2dst_syn_packets  src2dst_cwr_packets  \\\n",
       "0                          0                    0                    0   \n",
       "1                          0                    0                    0   \n",
       "2                          0                    0                    0   \n",
       "3                          0                    0                    0   \n",
       "4                          0                    1                    0   \n",
       "\n",
       "   src2dst_ece_packets  src2dst_urg_packets  src2dst_ack_packets  \\\n",
       "0                    0                    0                   24   \n",
       "1                    0                    0                   13   \n",
       "2                    0                    0                    1   \n",
       "3                    0                    0                    0   \n",
       "4                    0                    0                    1   \n",
       "\n",
       "   src2dst_psh_packets  src2dst_rst_packets  src2dst_fin_packets  \\\n",
       "0                    1                    0                    0   \n",
       "1                    1                    0                    0   \n",
       "2                    1                    0                    0   \n",
       "3                    0                    0                    0   \n",
       "4                    0                    0                    0   \n",
       "\n",
       "   dst2src_syn_packets  dst2src_cwr_packets  dst2src_ece_packets  \\\n",
       "0                    0                    0                    0   \n",
       "1                    0                    0                    0   \n",
       "2                    0                    0                    0   \n",
       "3                    0                    0                    0   \n",
       "4                    0                    0                    0   \n",
       "\n",
       "   dst2src_urg_packets  dst2src_ack_packets  dst2src_psh_packets  \\\n",
       "0                    0                   34                    3   \n",
       "1                    0                   39                    0   \n",
       "2                    0                    0                    0   \n",
       "3                    0                    0                    0   \n",
       "4                    0                    1                    0   \n",
       "\n",
       "   dst2src_rst_packets  dst2src_fin_packets application_name  \\\n",
       "0                    0                    0   HTTP.Instagram   \n",
       "1                    0                    0   HTTP.Instagram   \n",
       "2                    0                    0   HTTP.Instagram   \n",
       "3                    0                    0    DNS.Instagram   \n",
       "4                    0                    0             HTTP   \n",
       "\n",
       "  application_category_name  application_is_guessed  application_confidence  \\\n",
       "0             SocialNetwork                       0                       6   \n",
       "1             SocialNetwork                       0                       6   \n",
       "2             SocialNetwork                       0                       6   \n",
       "3                   Network                       0                       6   \n",
       "4                       Web                       1                       1   \n",
       "\n",
       "           requested_server_name client_fingerprint server_fingerprint  \\\n",
       "0      photos-g.ak.instagram.com                NaN                NaN   \n",
       "1      photos-h.ak.instagram.com                NaN                NaN   \n",
       "2      photos-a.ak.instagram.com                NaN                NaN   \n",
       "3  igcdn-photos-a-a.akamaihd.net                NaN                NaN   \n",
       "4                            NaN                NaN                NaN   \n",
       "\n",
       "                                          user_agent  content_type  \n",
       "0  Instagram 7.1.1 Android (19/4.4.2; 480dpi; 108...           NaN  \n",
       "1  Instagram 7.1.1 Android (19/4.4.2; 480dpi; 108...           NaN  \n",
       "2  Instagram 7.1.1 Android (19/4.4.2; 480dpi; 108...           NaN  \n",
       "3                                                NaN           NaN  \n",
       "4                                                NaN           NaN  "
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now that we have our Dataframe, we can start analyzing our data as any data. For example we can compute additional features:\n",
    "\n",
    "* Compute data ratio on both direction (src2dst and dst2src)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "df[\"src2dst_bytes_data_ratio\"] = df['src2dst_bytes'] / df['bidirectional_bytes']\n",
    "df[\"dst2src_bytes_data_ratio\"] = df['dst2src_bytes'] / df['bidirectional_bytes']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>expiration_id</th>\n",
       "      <th>src_ip</th>\n",
       "      <th>src_mac</th>\n",
       "      <th>src_oui</th>\n",
       "      <th>src_port</th>\n",
       "      <th>dst_ip</th>\n",
       "      <th>dst_mac</th>\n",
       "      <th>dst_oui</th>\n",
       "      <th>dst_port</th>\n",
       "      <th>protocol</th>\n",
       "      <th>ip_version</th>\n",
       "      <th>vlan_id</th>\n",
       "      <th>tunnel_id</th>\n",
       "      <th>bidirectional_first_seen_ms</th>\n",
       "      <th>bidirectional_last_seen_ms</th>\n",
       "      <th>bidirectional_duration_ms</th>\n",
       "      <th>bidirectional_packets</th>\n",
       "      <th>bidirectional_bytes</th>\n",
       "      <th>src2dst_first_seen_ms</th>\n",
       "      <th>src2dst_last_seen_ms</th>\n",
       "      <th>src2dst_duration_ms</th>\n",
       "      <th>src2dst_packets</th>\n",
       "      <th>src2dst_bytes</th>\n",
       "      <th>dst2src_first_seen_ms</th>\n",
       "      <th>dst2src_last_seen_ms</th>\n",
       "      <th>dst2src_duration_ms</th>\n",
       "      <th>dst2src_packets</th>\n",
       "      <th>dst2src_bytes</th>\n",
       "      <th>bidirectional_min_ps</th>\n",
       "      <th>bidirectional_mean_ps</th>\n",
       "      <th>bidirectional_stddev_ps</th>\n",
       "      <th>bidirectional_max_ps</th>\n",
       "      <th>src2dst_min_ps</th>\n",
       "      <th>src2dst_mean_ps</th>\n",
       "      <th>src2dst_stddev_ps</th>\n",
       "      <th>src2dst_max_ps</th>\n",
       "      <th>dst2src_min_ps</th>\n",
       "      <th>dst2src_mean_ps</th>\n",
       "      <th>dst2src_stddev_ps</th>\n",
       "      <th>dst2src_max_ps</th>\n",
       "      <th>bidirectional_min_piat_ms</th>\n",
       "      <th>bidirectional_mean_piat_ms</th>\n",
       "      <th>bidirectional_stddev_piat_ms</th>\n",
       "      <th>bidirectional_max_piat_ms</th>\n",
       "      <th>src2dst_min_piat_ms</th>\n",
       "      <th>src2dst_mean_piat_ms</th>\n",
       "      <th>src2dst_stddev_piat_ms</th>\n",
       "      <th>src2dst_max_piat_ms</th>\n",
       "      <th>dst2src_min_piat_ms</th>\n",
       "      <th>dst2src_mean_piat_ms</th>\n",
       "      <th>dst2src_stddev_piat_ms</th>\n",
       "      <th>dst2src_max_piat_ms</th>\n",
       "      <th>bidirectional_syn_packets</th>\n",
       "      <th>bidirectional_cwr_packets</th>\n",
       "      <th>bidirectional_ece_packets</th>\n",
       "      <th>bidirectional_urg_packets</th>\n",
       "      <th>bidirectional_ack_packets</th>\n",
       "      <th>bidirectional_psh_packets</th>\n",
       "      <th>bidirectional_rst_packets</th>\n",
       "      <th>bidirectional_fin_packets</th>\n",
       "      <th>src2dst_syn_packets</th>\n",
       "      <th>src2dst_cwr_packets</th>\n",
       "      <th>src2dst_ece_packets</th>\n",
       "      <th>src2dst_urg_packets</th>\n",
       "      <th>src2dst_ack_packets</th>\n",
       "      <th>src2dst_psh_packets</th>\n",
       "      <th>src2dst_rst_packets</th>\n",
       "      <th>src2dst_fin_packets</th>\n",
       "      <th>dst2src_syn_packets</th>\n",
       "      <th>dst2src_cwr_packets</th>\n",
       "      <th>dst2src_ece_packets</th>\n",
       "      <th>dst2src_urg_packets</th>\n",
       "      <th>dst2src_ack_packets</th>\n",
       "      <th>dst2src_psh_packets</th>\n",
       "      <th>dst2src_rst_packets</th>\n",
       "      <th>dst2src_fin_packets</th>\n",
       "      <th>application_name</th>\n",
       "      <th>application_category_name</th>\n",
       "      <th>application_is_guessed</th>\n",
       "      <th>application_confidence</th>\n",
       "      <th>requested_server_name</th>\n",
       "      <th>client_fingerprint</th>\n",
       "      <th>server_fingerprint</th>\n",
       "      <th>user_agent</th>\n",
       "      <th>content_type</th>\n",
       "      <th>src2dst_bytes_data_ratio</th>\n",
       "      <th>dst2src_bytes_data_ratio</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>5885370fbc1de250a4570351f2679e915e15245a5534bd...</td>\n",
       "      <td>b5d836f0b4088481bd22d1bcdbf78c8bb4ed6c5b5a3175...</td>\n",
       "      <td>40:f3:08</td>\n",
       "      <td>57936</td>\n",
       "      <td>3a44c94fd7c9aefa07df278f016460d75aa809c94a571c...</td>\n",
       "      <td>7f6b3b13330898c4dcf505e44b642c996b9674139831e0...</td>\n",
       "      <td>00:1b:2f</td>\n",
       "      <td>80</td>\n",
       "      <td>6</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1436720900687</td>\n",
       "      <td>1436720901200</td>\n",
       "      <td>513</td>\n",
       "      <td>58</td>\n",
       "      <td>50220</td>\n",
       "      <td>1436720900687</td>\n",
       "      <td>1436720901200</td>\n",
       "      <td>513</td>\n",
       "      <td>24</td>\n",
       "      <td>1837</td>\n",
       "      <td>1436720900744</td>\n",
       "      <td>1436720901200</td>\n",
       "      <td>456</td>\n",
       "      <td>34</td>\n",
       "      <td>48383</td>\n",
       "      <td>66</td>\n",
       "      <td>865.862069</td>\n",
       "      <td>696.739485</td>\n",
       "      <td>1484</td>\n",
       "      <td>66</td>\n",
       "      <td>76.541667</td>\n",
       "      <td>51.643409</td>\n",
       "      <td>319</td>\n",
       "      <td>186</td>\n",
       "      <td>1423.029412</td>\n",
       "      <td>252.360311</td>\n",
       "      <td>1484</td>\n",
       "      <td>0</td>\n",
       "      <td>9.000000</td>\n",
       "      <td>45.124035</td>\n",
       "      <td>321</td>\n",
       "      <td>0</td>\n",
       "      <td>22.304348</td>\n",
       "      <td>70.131976</td>\n",
       "      <td>322</td>\n",
       "      <td>0</td>\n",
       "      <td>13.818182</td>\n",
       "      <td>58.73109</td>\n",
       "      <td>323</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>58</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>24</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>34</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>HTTP.Instagram</td>\n",
       "      <td>SocialNetwork</td>\n",
       "      <td>0</td>\n",
       "      <td>6</td>\n",
       "      <td>photos-g.ak.instagram.com</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Instagram 7.1.1 Android (19/4.4.2; 480dpi; 108...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.036579</td>\n",
       "      <td>0.963421</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>5885370fbc1de250a4570351f2679e915e15245a5534bd...</td>\n",
       "      <td>b5d836f0b4088481bd22d1bcdbf78c8bb4ed6c5b5a3175...</td>\n",
       "      <td>40:f3:08</td>\n",
       "      <td>38816</td>\n",
       "      <td>c26866c915aaa410921c4fc309477eb0ceba2caec77bcf...</td>\n",
       "      <td>7f6b3b13330898c4dcf505e44b642c996b9674139831e0...</td>\n",
       "      <td>00:1b:2f</td>\n",
       "      <td>80</td>\n",
       "      <td>6</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1436720900684</td>\n",
       "      <td>1436720900750</td>\n",
       "      <td>66</td>\n",
       "      <td>52</td>\n",
       "      <td>58994</td>\n",
       "      <td>1436720900684</td>\n",
       "      <td>1436720900750</td>\n",
       "      <td>66</td>\n",
       "      <td>13</td>\n",
       "      <td>1118</td>\n",
       "      <td>1436720900716</td>\n",
       "      <td>1436720900744</td>\n",
       "      <td>28</td>\n",
       "      <td>39</td>\n",
       "      <td>57876</td>\n",
       "      <td>66</td>\n",
       "      <td>1134.500000</td>\n",
       "      <td>612.257779</td>\n",
       "      <td>1484</td>\n",
       "      <td>66</td>\n",
       "      <td>86.000000</td>\n",
       "      <td>72.111026</td>\n",
       "      <td>326</td>\n",
       "      <td>1484</td>\n",
       "      <td>1484.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1484</td>\n",
       "      <td>0</td>\n",
       "      <td>1.294118</td>\n",
       "      <td>4.495750</td>\n",
       "      <td>32</td>\n",
       "      <td>0</td>\n",
       "      <td>5.500000</td>\n",
       "      <td>9.170110</td>\n",
       "      <td>33</td>\n",
       "      <td>0</td>\n",
       "      <td>0.736842</td>\n",
       "      <td>0.68514</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>52</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>13</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>39</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>HTTP.Instagram</td>\n",
       "      <td>SocialNetwork</td>\n",
       "      <td>0</td>\n",
       "      <td>6</td>\n",
       "      <td>photos-h.ak.instagram.com</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Instagram 7.1.1 Android (19/4.4.2; 480dpi; 108...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.018951</td>\n",
       "      <td>0.981049</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>5885370fbc1de250a4570351f2679e915e15245a5534bd...</td>\n",
       "      <td>b5d836f0b4088481bd22d1bcdbf78c8bb4ed6c5b5a3175...</td>\n",
       "      <td>40:f3:08</td>\n",
       "      <td>37350</td>\n",
       "      <td>68df6e56301d6c238c302eaa732d2075ef802914771e7b...</td>\n",
       "      <td>7f6b3b13330898c4dcf505e44b642c996b9674139831e0...</td>\n",
       "      <td>00:1b:2f</td>\n",
       "      <td>80</td>\n",
       "      <td>6</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1436720901262</td>\n",
       "      <td>1436720901262</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>324</td>\n",
       "      <td>1436720901262</td>\n",
       "      <td>1436720901262</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>324</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>324</td>\n",
       "      <td>324.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>324</td>\n",
       "      <td>324</td>\n",
       "      <td>324.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>324</td>\n",
       "      <td>0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>HTTP.Instagram</td>\n",
       "      <td>SocialNetwork</td>\n",
       "      <td>0</td>\n",
       "      <td>6</td>\n",
       "      <td>photos-a.ak.instagram.com</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Instagram 7.1.1 Android (19/4.4.2; 480dpi; 108...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>5885370fbc1de250a4570351f2679e915e15245a5534bd...</td>\n",
       "      <td>b5d836f0b4088481bd22d1bcdbf78c8bb4ed6c5b5a3175...</td>\n",
       "      <td>40:f3:08</td>\n",
       "      <td>33603</td>\n",
       "      <td>107d5f2f69c5e2f3da41be7ce2a59c0d818947212d6ef0...</td>\n",
       "      <td>7f6b3b13330898c4dcf505e44b642c996b9674139831e0...</td>\n",
       "      <td>00:1b:2f</td>\n",
       "      <td>53</td>\n",
       "      <td>17</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1436720908524</td>\n",
       "      <td>1436720908575</td>\n",
       "      <td>51</td>\n",
       "      <td>2</td>\n",
       "      <td>298</td>\n",
       "      <td>1436720908524</td>\n",
       "      <td>1436720908524</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>89</td>\n",
       "      <td>1436720908575</td>\n",
       "      <td>1436720908575</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>209</td>\n",
       "      <td>89</td>\n",
       "      <td>149.000000</td>\n",
       "      <td>84.852814</td>\n",
       "      <td>209</td>\n",
       "      <td>89</td>\n",
       "      <td>89.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>89</td>\n",
       "      <td>209</td>\n",
       "      <td>209.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>209</td>\n",
       "      <td>51</td>\n",
       "      <td>51.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>51</td>\n",
       "      <td>0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>DNS.Instagram</td>\n",
       "      <td>Network</td>\n",
       "      <td>0</td>\n",
       "      <td>6</td>\n",
       "      <td>igcdn-photos-a-a.akamaihd.net</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.298658</td>\n",
       "      <td>0.701342</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>d7feda5309e4f8477aac71903e83486f9f13566cc836f8...</td>\n",
       "      <td>7f6b3b13330898c4dcf505e44b642c996b9674139831e0...</td>\n",
       "      <td>00:1b:2f</td>\n",
       "      <td>80</td>\n",
       "      <td>5885370fbc1de250a4570351f2679e915e15245a5534bd...</td>\n",
       "      <td>b5d836f0b4088481bd22d1bcdbf78c8bb4ed6c5b5a3175...</td>\n",
       "      <td>40:f3:08</td>\n",
       "      <td>40855</td>\n",
       "      <td>6</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1436720952611</td>\n",
       "      <td>1436720952611</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>140</td>\n",
       "      <td>1436720952611</td>\n",
       "      <td>1436720952611</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>74</td>\n",
       "      <td>1436720952611</td>\n",
       "      <td>1436720952611</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>66</td>\n",
       "      <td>66</td>\n",
       "      <td>70.000000</td>\n",
       "      <td>5.656854</td>\n",
       "      <td>74</td>\n",
       "      <td>74</td>\n",
       "      <td>74.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>74</td>\n",
       "      <td>66</td>\n",
       "      <td>66.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>66</td>\n",
       "      <td>0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>HTTP</td>\n",
       "      <td>Web</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.528571</td>\n",
       "      <td>0.471429</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   id  expiration_id                                             src_ip  \\\n",
       "0   0              0  5885370fbc1de250a4570351f2679e915e15245a5534bd...   \n",
       "1   1              0  5885370fbc1de250a4570351f2679e915e15245a5534bd...   \n",
       "2   2              0  5885370fbc1de250a4570351f2679e915e15245a5534bd...   \n",
       "3   3              0  5885370fbc1de250a4570351f2679e915e15245a5534bd...   \n",
       "4   4              0  d7feda5309e4f8477aac71903e83486f9f13566cc836f8...   \n",
       "\n",
       "                                             src_mac   src_oui  src_port  \\\n",
       "0  b5d836f0b4088481bd22d1bcdbf78c8bb4ed6c5b5a3175...  40:f3:08     57936   \n",
       "1  b5d836f0b4088481bd22d1bcdbf78c8bb4ed6c5b5a3175...  40:f3:08     38816   \n",
       "2  b5d836f0b4088481bd22d1bcdbf78c8bb4ed6c5b5a3175...  40:f3:08     37350   \n",
       "3  b5d836f0b4088481bd22d1bcdbf78c8bb4ed6c5b5a3175...  40:f3:08     33603   \n",
       "4  7f6b3b13330898c4dcf505e44b642c996b9674139831e0...  00:1b:2f        80   \n",
       "\n",
       "                                              dst_ip  \\\n",
       "0  3a44c94fd7c9aefa07df278f016460d75aa809c94a571c...   \n",
       "1  c26866c915aaa410921c4fc309477eb0ceba2caec77bcf...   \n",
       "2  68df6e56301d6c238c302eaa732d2075ef802914771e7b...   \n",
       "3  107d5f2f69c5e2f3da41be7ce2a59c0d818947212d6ef0...   \n",
       "4  5885370fbc1de250a4570351f2679e915e15245a5534bd...   \n",
       "\n",
       "                                             dst_mac   dst_oui  dst_port  \\\n",
       "0  7f6b3b13330898c4dcf505e44b642c996b9674139831e0...  00:1b:2f        80   \n",
       "1  7f6b3b13330898c4dcf505e44b642c996b9674139831e0...  00:1b:2f        80   \n",
       "2  7f6b3b13330898c4dcf505e44b642c996b9674139831e0...  00:1b:2f        80   \n",
       "3  7f6b3b13330898c4dcf505e44b642c996b9674139831e0...  00:1b:2f        53   \n",
       "4  b5d836f0b4088481bd22d1bcdbf78c8bb4ed6c5b5a3175...  40:f3:08     40855   \n",
       "\n",
       "   protocol  ip_version  vlan_id  tunnel_id  bidirectional_first_seen_ms  \\\n",
       "0         6           4        0          0                1436720900687   \n",
       "1         6           4        0          0                1436720900684   \n",
       "2         6           4        0          0                1436720901262   \n",
       "3        17           4        0          0                1436720908524   \n",
       "4         6           4        0          0                1436720952611   \n",
       "\n",
       "   bidirectional_last_seen_ms  bidirectional_duration_ms  \\\n",
       "0               1436720901200                        513   \n",
       "1               1436720900750                         66   \n",
       "2               1436720901262                          0   \n",
       "3               1436720908575                         51   \n",
       "4               1436720952611                          0   \n",
       "\n",
       "   bidirectional_packets  bidirectional_bytes  src2dst_first_seen_ms  \\\n",
       "0                     58                50220          1436720900687   \n",
       "1                     52                58994          1436720900684   \n",
       "2                      1                  324          1436720901262   \n",
       "3                      2                  298          1436720908524   \n",
       "4                      2                  140          1436720952611   \n",
       "\n",
       "   src2dst_last_seen_ms  src2dst_duration_ms  src2dst_packets  src2dst_bytes  \\\n",
       "0         1436720901200                  513               24           1837   \n",
       "1         1436720900750                   66               13           1118   \n",
       "2         1436720901262                    0                1            324   \n",
       "3         1436720908524                    0                1             89   \n",
       "4         1436720952611                    0                1             74   \n",
       "\n",
       "   dst2src_first_seen_ms  dst2src_last_seen_ms  dst2src_duration_ms  \\\n",
       "0          1436720900744         1436720901200                  456   \n",
       "1          1436720900716         1436720900744                   28   \n",
       "2                      0                     0                    0   \n",
       "3          1436720908575         1436720908575                    0   \n",
       "4          1436720952611         1436720952611                    0   \n",
       "\n",
       "   dst2src_packets  dst2src_bytes  bidirectional_min_ps  \\\n",
       "0               34          48383                    66   \n",
       "1               39          57876                    66   \n",
       "2                0              0                   324   \n",
       "3                1            209                    89   \n",
       "4                1             66                    66   \n",
       "\n",
       "   bidirectional_mean_ps  bidirectional_stddev_ps  bidirectional_max_ps  \\\n",
       "0             865.862069               696.739485                  1484   \n",
       "1            1134.500000               612.257779                  1484   \n",
       "2             324.000000                 0.000000                   324   \n",
       "3             149.000000                84.852814                   209   \n",
       "4              70.000000                 5.656854                    74   \n",
       "\n",
       "   src2dst_min_ps  src2dst_mean_ps  src2dst_stddev_ps  src2dst_max_ps  \\\n",
       "0              66        76.541667          51.643409             319   \n",
       "1              66        86.000000          72.111026             326   \n",
       "2             324       324.000000           0.000000             324   \n",
       "3              89        89.000000           0.000000              89   \n",
       "4              74        74.000000           0.000000              74   \n",
       "\n",
       "   dst2src_min_ps  dst2src_mean_ps  dst2src_stddev_ps  dst2src_max_ps  \\\n",
       "0             186      1423.029412         252.360311            1484   \n",
       "1            1484      1484.000000           0.000000            1484   \n",
       "2               0         0.000000           0.000000               0   \n",
       "3             209       209.000000           0.000000             209   \n",
       "4              66        66.000000           0.000000              66   \n",
       "\n",
       "   bidirectional_min_piat_ms  bidirectional_mean_piat_ms  \\\n",
       "0                          0                    9.000000   \n",
       "1                          0                    1.294118   \n",
       "2                          0                    0.000000   \n",
       "3                         51                   51.000000   \n",
       "4                          0                    0.000000   \n",
       "\n",
       "   bidirectional_stddev_piat_ms  bidirectional_max_piat_ms  \\\n",
       "0                     45.124035                        321   \n",
       "1                      4.495750                         32   \n",
       "2                      0.000000                          0   \n",
       "3                      0.000000                         51   \n",
       "4                      0.000000                          0   \n",
       "\n",
       "   src2dst_min_piat_ms  src2dst_mean_piat_ms  src2dst_stddev_piat_ms  \\\n",
       "0                    0             22.304348               70.131976   \n",
       "1                    0              5.500000                9.170110   \n",
       "2                    0              0.000000                0.000000   \n",
       "3                    0              0.000000                0.000000   \n",
       "4                    0              0.000000                0.000000   \n",
       "\n",
       "   src2dst_max_piat_ms  dst2src_min_piat_ms  dst2src_mean_piat_ms  \\\n",
       "0                  322                    0             13.818182   \n",
       "1                   33                    0              0.736842   \n",
       "2                    0                    0              0.000000   \n",
       "3                    0                    0              0.000000   \n",
       "4                    0                    0              0.000000   \n",
       "\n",
       "   dst2src_stddev_piat_ms  dst2src_max_piat_ms  bidirectional_syn_packets  \\\n",
       "0                58.73109                  323                          0   \n",
       "1                 0.68514                    2                          0   \n",
       "2                 0.00000                    0                          0   \n",
       "3                 0.00000                    0                          0   \n",
       "4                 0.00000                    0                          1   \n",
       "\n",
       "   bidirectional_cwr_packets  bidirectional_ece_packets  \\\n",
       "0                          0                          0   \n",
       "1                          0                          0   \n",
       "2                          0                          0   \n",
       "3                          0                          0   \n",
       "4                          0                          0   \n",
       "\n",
       "   bidirectional_urg_packets  bidirectional_ack_packets  \\\n",
       "0                          0                         58   \n",
       "1                          0                         52   \n",
       "2                          0                          1   \n",
       "3                          0                          0   \n",
       "4                          0                          2   \n",
       "\n",
       "   bidirectional_psh_packets  bidirectional_rst_packets  \\\n",
       "0                          4                          0   \n",
       "1                          1                          0   \n",
       "2                          1                          0   \n",
       "3                          0                          0   \n",
       "4                          0                          0   \n",
       "\n",
       "   bidirectional_fin_packets  src2dst_syn_packets  src2dst_cwr_packets  \\\n",
       "0                          0                    0                    0   \n",
       "1                          0                    0                    0   \n",
       "2                          0                    0                    0   \n",
       "3                          0                    0                    0   \n",
       "4                          0                    1                    0   \n",
       "\n",
       "   src2dst_ece_packets  src2dst_urg_packets  src2dst_ack_packets  \\\n",
       "0                    0                    0                   24   \n",
       "1                    0                    0                   13   \n",
       "2                    0                    0                    1   \n",
       "3                    0                    0                    0   \n",
       "4                    0                    0                    1   \n",
       "\n",
       "   src2dst_psh_packets  src2dst_rst_packets  src2dst_fin_packets  \\\n",
       "0                    1                    0                    0   \n",
       "1                    1                    0                    0   \n",
       "2                    1                    0                    0   \n",
       "3                    0                    0                    0   \n",
       "4                    0                    0                    0   \n",
       "\n",
       "   dst2src_syn_packets  dst2src_cwr_packets  dst2src_ece_packets  \\\n",
       "0                    0                    0                    0   \n",
       "1                    0                    0                    0   \n",
       "2                    0                    0                    0   \n",
       "3                    0                    0                    0   \n",
       "4                    0                    0                    0   \n",
       "\n",
       "   dst2src_urg_packets  dst2src_ack_packets  dst2src_psh_packets  \\\n",
       "0                    0                   34                    3   \n",
       "1                    0                   39                    0   \n",
       "2                    0                    0                    0   \n",
       "3                    0                    0                    0   \n",
       "4                    0                    1                    0   \n",
       "\n",
       "   dst2src_rst_packets  dst2src_fin_packets application_name  \\\n",
       "0                    0                    0   HTTP.Instagram   \n",
       "1                    0                    0   HTTP.Instagram   \n",
       "2                    0                    0   HTTP.Instagram   \n",
       "3                    0                    0    DNS.Instagram   \n",
       "4                    0                    0             HTTP   \n",
       "\n",
       "  application_category_name  application_is_guessed  application_confidence  \\\n",
       "0             SocialNetwork                       0                       6   \n",
       "1             SocialNetwork                       0                       6   \n",
       "2             SocialNetwork                       0                       6   \n",
       "3                   Network                       0                       6   \n",
       "4                       Web                       1                       1   \n",
       "\n",
       "           requested_server_name client_fingerprint server_fingerprint  \\\n",
       "0      photos-g.ak.instagram.com                NaN                NaN   \n",
       "1      photos-h.ak.instagram.com                NaN                NaN   \n",
       "2      photos-a.ak.instagram.com                NaN                NaN   \n",
       "3  igcdn-photos-a-a.akamaihd.net                NaN                NaN   \n",
       "4                            NaN                NaN                NaN   \n",
       "\n",
       "                                          user_agent  content_type  \\\n",
       "0  Instagram 7.1.1 Android (19/4.4.2; 480dpi; 108...           NaN   \n",
       "1  Instagram 7.1.1 Android (19/4.4.2; 480dpi; 108...           NaN   \n",
       "2  Instagram 7.1.1 Android (19/4.4.2; 480dpi; 108...           NaN   \n",
       "3                                                NaN           NaN   \n",
       "4                                                NaN           NaN   \n",
       "\n",
       "   src2dst_bytes_data_ratio  dst2src_bytes_data_ratio  \n",
       "0                  0.036579                  0.963421  \n",
       "1                  0.018951                  0.981049  \n",
       "2                  1.000000                  0.000000  \n",
       "3                  0.298658                  0.701342  \n",
       "4                  0.528571                  0.471429  "
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Filter data according to some criterias:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>expiration_id</th>\n",
       "      <th>src_ip</th>\n",
       "      <th>src_mac</th>\n",
       "      <th>src_oui</th>\n",
       "      <th>src_port</th>\n",
       "      <th>dst_ip</th>\n",
       "      <th>dst_mac</th>\n",
       "      <th>dst_oui</th>\n",
       "      <th>dst_port</th>\n",
       "      <th>protocol</th>\n",
       "      <th>ip_version</th>\n",
       "      <th>vlan_id</th>\n",
       "      <th>tunnel_id</th>\n",
       "      <th>bidirectional_first_seen_ms</th>\n",
       "      <th>bidirectional_last_seen_ms</th>\n",
       "      <th>bidirectional_duration_ms</th>\n",
       "      <th>bidirectional_packets</th>\n",
       "      <th>bidirectional_bytes</th>\n",
       "      <th>src2dst_first_seen_ms</th>\n",
       "      <th>src2dst_last_seen_ms</th>\n",
       "      <th>src2dst_duration_ms</th>\n",
       "      <th>src2dst_packets</th>\n",
       "      <th>src2dst_bytes</th>\n",
       "      <th>dst2src_first_seen_ms</th>\n",
       "      <th>dst2src_last_seen_ms</th>\n",
       "      <th>dst2src_duration_ms</th>\n",
       "      <th>dst2src_packets</th>\n",
       "      <th>dst2src_bytes</th>\n",
       "      <th>bidirectional_min_ps</th>\n",
       "      <th>bidirectional_mean_ps</th>\n",
       "      <th>bidirectional_stddev_ps</th>\n",
       "      <th>bidirectional_max_ps</th>\n",
       "      <th>src2dst_min_ps</th>\n",
       "      <th>src2dst_mean_ps</th>\n",
       "      <th>src2dst_stddev_ps</th>\n",
       "      <th>src2dst_max_ps</th>\n",
       "      <th>dst2src_min_ps</th>\n",
       "      <th>dst2src_mean_ps</th>\n",
       "      <th>dst2src_stddev_ps</th>\n",
       "      <th>dst2src_max_ps</th>\n",
       "      <th>bidirectional_min_piat_ms</th>\n",
       "      <th>bidirectional_mean_piat_ms</th>\n",
       "      <th>bidirectional_stddev_piat_ms</th>\n",
       "      <th>bidirectional_max_piat_ms</th>\n",
       "      <th>src2dst_min_piat_ms</th>\n",
       "      <th>src2dst_mean_piat_ms</th>\n",
       "      <th>src2dst_stddev_piat_ms</th>\n",
       "      <th>src2dst_max_piat_ms</th>\n",
       "      <th>dst2src_min_piat_ms</th>\n",
       "      <th>dst2src_mean_piat_ms</th>\n",
       "      <th>dst2src_stddev_piat_ms</th>\n",
       "      <th>dst2src_max_piat_ms</th>\n",
       "      <th>bidirectional_syn_packets</th>\n",
       "      <th>bidirectional_cwr_packets</th>\n",
       "      <th>bidirectional_ece_packets</th>\n",
       "      <th>bidirectional_urg_packets</th>\n",
       "      <th>bidirectional_ack_packets</th>\n",
       "      <th>bidirectional_psh_packets</th>\n",
       "      <th>bidirectional_rst_packets</th>\n",
       "      <th>bidirectional_fin_packets</th>\n",
       "      <th>src2dst_syn_packets</th>\n",
       "      <th>src2dst_cwr_packets</th>\n",
       "      <th>src2dst_ece_packets</th>\n",
       "      <th>src2dst_urg_packets</th>\n",
       "      <th>src2dst_ack_packets</th>\n",
       "      <th>src2dst_psh_packets</th>\n",
       "      <th>src2dst_rst_packets</th>\n",
       "      <th>src2dst_fin_packets</th>\n",
       "      <th>dst2src_syn_packets</th>\n",
       "      <th>dst2src_cwr_packets</th>\n",
       "      <th>dst2src_ece_packets</th>\n",
       "      <th>dst2src_urg_packets</th>\n",
       "      <th>dst2src_ack_packets</th>\n",
       "      <th>dst2src_psh_packets</th>\n",
       "      <th>dst2src_rst_packets</th>\n",
       "      <th>dst2src_fin_packets</th>\n",
       "      <th>application_name</th>\n",
       "      <th>application_category_name</th>\n",
       "      <th>application_is_guessed</th>\n",
       "      <th>application_confidence</th>\n",
       "      <th>requested_server_name</th>\n",
       "      <th>client_fingerprint</th>\n",
       "      <th>server_fingerprint</th>\n",
       "      <th>user_agent</th>\n",
       "      <th>content_type</th>\n",
       "      <th>src2dst_bytes_data_ratio</th>\n",
       "      <th>dst2src_bytes_data_ratio</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>5</td>\n",
       "      <td>0</td>\n",
       "      <td>5885370fbc1de250a4570351f2679e915e15245a5534bd...</td>\n",
       "      <td>b5d836f0b4088481bd22d1bcdbf78c8bb4ed6c5b5a3175...</td>\n",
       "      <td>40:f3:08</td>\n",
       "      <td>33763</td>\n",
       "      <td>6c662e3d71901ed2227ad7c8bd2e074e240ca921855da1...</td>\n",
       "      <td>7f6b3b13330898c4dcf505e44b642c996b9674139831e0...</td>\n",
       "      <td>00:1b:2f</td>\n",
       "      <td>443</td>\n",
       "      <td>6</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1436720908466</td>\n",
       "      <td>1436720910950</td>\n",
       "      <td>2484</td>\n",
       "      <td>11</td>\n",
       "      <td>5397</td>\n",
       "      <td>1436720908466</td>\n",
       "      <td>1436720908723</td>\n",
       "      <td>257</td>\n",
       "      <td>5</td>\n",
       "      <td>1279</td>\n",
       "      <td>1436720908518</td>\n",
       "      <td>1436720910950</td>\n",
       "      <td>2432</td>\n",
       "      <td>6</td>\n",
       "      <td>4118</td>\n",
       "      <td>66</td>\n",
       "      <td>490.636364</td>\n",
       "      <td>588.172640</td>\n",
       "      <td>1464</td>\n",
       "      <td>66</td>\n",
       "      <td>255.800000</td>\n",
       "      <td>424.405702</td>\n",
       "      <td>1015</td>\n",
       "      <td>66</td>\n",
       "      <td>686.333333</td>\n",
       "      <td>668.351006</td>\n",
       "      <td>1464</td>\n",
       "      <td>0</td>\n",
       "      <td>248.400000</td>\n",
       "      <td>698.093945</td>\n",
       "      <td>2227</td>\n",
       "      <td>0</td>\n",
       "      <td>64.250000</td>\n",
       "      <td>126.502635</td>\n",
       "      <td>254</td>\n",
       "      <td>0</td>\n",
       "      <td>486.4</td>\n",
       "      <td>976.910078</td>\n",
       "      <td>2227</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>11</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>5</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>6</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>TLS</td>\n",
       "      <td>Web</td>\n",
       "      <td>0</td>\n",
       "      <td>6</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.236984</td>\n",
       "      <td>0.763016</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>8</td>\n",
       "      <td>0</td>\n",
       "      <td>5885370fbc1de250a4570351f2679e915e15245a5534bd...</td>\n",
       "      <td>b5d836f0b4088481bd22d1bcdbf78c8bb4ed6c5b5a3175...</td>\n",
       "      <td>40:f3:08</td>\n",
       "      <td>41181</td>\n",
       "      <td>c9f555cb103cf7cb79a76242fe4b121682293ebb993677...</td>\n",
       "      <td>7f6b3b13330898c4dcf505e44b642c996b9674139831e0...</td>\n",
       "      <td>00:1b:2f</td>\n",
       "      <td>443</td>\n",
       "      <td>6</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1436720908576</td>\n",
       "      <td>1436720908733</td>\n",
       "      <td>157</td>\n",
       "      <td>14</td>\n",
       "      <td>5567</td>\n",
       "      <td>1436720908576</td>\n",
       "      <td>1436720908733</td>\n",
       "      <td>157</td>\n",
       "      <td>8</td>\n",
       "      <td>896</td>\n",
       "      <td>1436720908615</td>\n",
       "      <td>1436720908662</td>\n",
       "      <td>47</td>\n",
       "      <td>6</td>\n",
       "      <td>4671</td>\n",
       "      <td>66</td>\n",
       "      <td>397.642857</td>\n",
       "      <td>566.204041</td>\n",
       "      <td>1484</td>\n",
       "      <td>66</td>\n",
       "      <td>112.000000</td>\n",
       "      <td>86.328277</td>\n",
       "      <td>292</td>\n",
       "      <td>66</td>\n",
       "      <td>778.500000</td>\n",
       "      <td>720.057706</td>\n",
       "      <td>1484</td>\n",
       "      <td>0</td>\n",
       "      <td>12.076923</td>\n",
       "      <td>22.746654</td>\n",
       "      <td>71</td>\n",
       "      <td>0</td>\n",
       "      <td>22.428571</td>\n",
       "      <td>28.564880</td>\n",
       "      <td>71</td>\n",
       "      <td>0</td>\n",
       "      <td>9.4</td>\n",
       "      <td>17.728508</td>\n",
       "      <td>41</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>13</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>7</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>6</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>TLS.Instagram</td>\n",
       "      <td>SocialNetwork</td>\n",
       "      <td>0</td>\n",
       "      <td>6</td>\n",
       "      <td>igcdn-photos-a-a.akamaihd.net</td>\n",
       "      <td>54ae5fcb0159e2ddf6a50e149221c7c7</td>\n",
       "      <td>34d6f0ad0a79e4cfdf145e640cc93f78</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.160948</td>\n",
       "      <td>0.839052</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>9</td>\n",
       "      <td>0</td>\n",
       "      <td>5885370fbc1de250a4570351f2679e915e15245a5534bd...</td>\n",
       "      <td>b5d836f0b4088481bd22d1bcdbf78c8bb4ed6c5b5a3175...</td>\n",
       "      <td>40:f3:08</td>\n",
       "      <td>58690</td>\n",
       "      <td>7e269e3e2e4c87e46547c961422353c0034227df8cb6e5...</td>\n",
       "      <td>7f6b3b13330898c4dcf505e44b642c996b9674139831e0...</td>\n",
       "      <td>00:1b:2f</td>\n",
       "      <td>443</td>\n",
       "      <td>6</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1436720952561</td>\n",
       "      <td>1436720952561</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>169</td>\n",
       "      <td>1436720952561</td>\n",
       "      <td>1436720952561</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>169</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>66</td>\n",
       "      <td>84.500000</td>\n",
       "      <td>26.162951</td>\n",
       "      <td>103</td>\n",
       "      <td>66</td>\n",
       "      <td>84.500000</td>\n",
       "      <td>26.162951</td>\n",
       "      <td>103</td>\n",
       "      <td>0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>TLS</td>\n",
       "      <td>Web</td>\n",
       "      <td>0</td>\n",
       "      <td>6</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>11</td>\n",
       "      <td>0</td>\n",
       "      <td>5885370fbc1de250a4570351f2679e915e15245a5534bd...</td>\n",
       "      <td>b5d836f0b4088481bd22d1bcdbf78c8bb4ed6c5b5a3175...</td>\n",
       "      <td>40:f3:08</td>\n",
       "      <td>56382</td>\n",
       "      <td>8296dd65fdefef3bde6b1205337bd3270f4960d0747ae6...</td>\n",
       "      <td>7f6b3b13330898c4dcf505e44b642c996b9674139831e0...</td>\n",
       "      <td>00:1b:2f</td>\n",
       "      <td>443</td>\n",
       "      <td>6</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1436720898354</td>\n",
       "      <td>1436720899158</td>\n",
       "      <td>804</td>\n",
       "      <td>17</td>\n",
       "      <td>2647</td>\n",
       "      <td>1436720898354</td>\n",
       "      <td>1436720899158</td>\n",
       "      <td>804</td>\n",
       "      <td>9</td>\n",
       "      <td>1583</td>\n",
       "      <td>1436720898499</td>\n",
       "      <td>1436720899122</td>\n",
       "      <td>623</td>\n",
       "      <td>8</td>\n",
       "      <td>1064</td>\n",
       "      <td>66</td>\n",
       "      <td>155.705882</td>\n",
       "      <td>128.137994</td>\n",
       "      <td>530</td>\n",
       "      <td>66</td>\n",
       "      <td>175.888889</td>\n",
       "      <td>164.147376</td>\n",
       "      <td>530</td>\n",
       "      <td>66</td>\n",
       "      <td>133.000000</td>\n",
       "      <td>74.989523</td>\n",
       "      <td>231</td>\n",
       "      <td>0</td>\n",
       "      <td>50.250000</td>\n",
       "      <td>72.009722</td>\n",
       "      <td>181</td>\n",
       "      <td>0</td>\n",
       "      <td>100.500000</td>\n",
       "      <td>83.513900</td>\n",
       "      <td>183</td>\n",
       "      <td>0</td>\n",
       "      <td>89.0</td>\n",
       "      <td>84.261498</td>\n",
       "      <td>183</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>16</td>\n",
       "      <td>8</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>8</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>8</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>TLS.Instagram</td>\n",
       "      <td>SocialNetwork</td>\n",
       "      <td>0</td>\n",
       "      <td>6</td>\n",
       "      <td>telegraph-ash.instagram.com</td>\n",
       "      <td>54ae5fcb0159e2ddf6a50e149221c7c7</td>\n",
       "      <td>acb741bcdffb787c5a52654c78645bdf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.598036</td>\n",
       "      <td>0.401964</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>15</td>\n",
       "      <td>0</td>\n",
       "      <td>5885370fbc1de250a4570351f2679e915e15245a5534bd...</td>\n",
       "      <td>b5d836f0b4088481bd22d1bcdbf78c8bb4ed6c5b5a3175...</td>\n",
       "      <td>40:f3:08</td>\n",
       "      <td>41182</td>\n",
       "      <td>c9f555cb103cf7cb79a76242fe4b121682293ebb993677...</td>\n",
       "      <td>7f6b3b13330898c4dcf505e44b642c996b9674139831e0...</td>\n",
       "      <td>00:1b:2f</td>\n",
       "      <td>443</td>\n",
       "      <td>6</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1436720908577</td>\n",
       "      <td>1436720908737</td>\n",
       "      <td>160</td>\n",
       "      <td>14</td>\n",
       "      <td>5567</td>\n",
       "      <td>1436720908577</td>\n",
       "      <td>1436720908737</td>\n",
       "      <td>160</td>\n",
       "      <td>8</td>\n",
       "      <td>896</td>\n",
       "      <td>1436720908616</td>\n",
       "      <td>1436720908665</td>\n",
       "      <td>49</td>\n",
       "      <td>6</td>\n",
       "      <td>4671</td>\n",
       "      <td>66</td>\n",
       "      <td>397.642857</td>\n",
       "      <td>566.204041</td>\n",
       "      <td>1484</td>\n",
       "      <td>66</td>\n",
       "      <td>112.000000</td>\n",
       "      <td>86.328277</td>\n",
       "      <td>292</td>\n",
       "      <td>66</td>\n",
       "      <td>778.500000</td>\n",
       "      <td>720.057706</td>\n",
       "      <td>1484</td>\n",
       "      <td>0</td>\n",
       "      <td>12.307692</td>\n",
       "      <td>23.346608</td>\n",
       "      <td>71</td>\n",
       "      <td>1</td>\n",
       "      <td>22.857143</td>\n",
       "      <td>28.439577</td>\n",
       "      <td>71</td>\n",
       "      <td>0</td>\n",
       "      <td>9.8</td>\n",
       "      <td>20.801442</td>\n",
       "      <td>47</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>13</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>7</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>6</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>TLS.Instagram</td>\n",
       "      <td>SocialNetwork</td>\n",
       "      <td>0</td>\n",
       "      <td>6</td>\n",
       "      <td>igcdn-photos-a-a.akamaihd.net</td>\n",
       "      <td>54ae5fcb0159e2ddf6a50e149221c7c7</td>\n",
       "      <td>34d6f0ad0a79e4cfdf145e640cc93f78</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.160948</td>\n",
       "      <td>0.839052</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    id  expiration_id                                             src_ip  \\\n",
       "5    5              0  5885370fbc1de250a4570351f2679e915e15245a5534bd...   \n",
       "8    8              0  5885370fbc1de250a4570351f2679e915e15245a5534bd...   \n",
       "9    9              0  5885370fbc1de250a4570351f2679e915e15245a5534bd...   \n",
       "11  11              0  5885370fbc1de250a4570351f2679e915e15245a5534bd...   \n",
       "15  15              0  5885370fbc1de250a4570351f2679e915e15245a5534bd...   \n",
       "\n",
       "                                              src_mac   src_oui  src_port  \\\n",
       "5   b5d836f0b4088481bd22d1bcdbf78c8bb4ed6c5b5a3175...  40:f3:08     33763   \n",
       "8   b5d836f0b4088481bd22d1bcdbf78c8bb4ed6c5b5a3175...  40:f3:08     41181   \n",
       "9   b5d836f0b4088481bd22d1bcdbf78c8bb4ed6c5b5a3175...  40:f3:08     58690   \n",
       "11  b5d836f0b4088481bd22d1bcdbf78c8bb4ed6c5b5a3175...  40:f3:08     56382   \n",
       "15  b5d836f0b4088481bd22d1bcdbf78c8bb4ed6c5b5a3175...  40:f3:08     41182   \n",
       "\n",
       "                                               dst_ip  \\\n",
       "5   6c662e3d71901ed2227ad7c8bd2e074e240ca921855da1...   \n",
       "8   c9f555cb103cf7cb79a76242fe4b121682293ebb993677...   \n",
       "9   7e269e3e2e4c87e46547c961422353c0034227df8cb6e5...   \n",
       "11  8296dd65fdefef3bde6b1205337bd3270f4960d0747ae6...   \n",
       "15  c9f555cb103cf7cb79a76242fe4b121682293ebb993677...   \n",
       "\n",
       "                                              dst_mac   dst_oui  dst_port  \\\n",
       "5   7f6b3b13330898c4dcf505e44b642c996b9674139831e0...  00:1b:2f       443   \n",
       "8   7f6b3b13330898c4dcf505e44b642c996b9674139831e0...  00:1b:2f       443   \n",
       "9   7f6b3b13330898c4dcf505e44b642c996b9674139831e0...  00:1b:2f       443   \n",
       "11  7f6b3b13330898c4dcf505e44b642c996b9674139831e0...  00:1b:2f       443   \n",
       "15  7f6b3b13330898c4dcf505e44b642c996b9674139831e0...  00:1b:2f       443   \n",
       "\n",
       "    protocol  ip_version  vlan_id  tunnel_id  bidirectional_first_seen_ms  \\\n",
       "5          6           4        0          0                1436720908466   \n",
       "8          6           4        0          0                1436720908576   \n",
       "9          6           4        0          0                1436720952561   \n",
       "11         6           4        0          0                1436720898354   \n",
       "15         6           4        0          0                1436720908577   \n",
       "\n",
       "    bidirectional_last_seen_ms  bidirectional_duration_ms  \\\n",
       "5                1436720910950                       2484   \n",
       "8                1436720908733                        157   \n",
       "9                1436720952561                          0   \n",
       "11               1436720899158                        804   \n",
       "15               1436720908737                        160   \n",
       "\n",
       "    bidirectional_packets  bidirectional_bytes  src2dst_first_seen_ms  \\\n",
       "5                      11                 5397          1436720908466   \n",
       "8                      14                 5567          1436720908576   \n",
       "9                       2                  169          1436720952561   \n",
       "11                     17                 2647          1436720898354   \n",
       "15                     14                 5567          1436720908577   \n",
       "\n",
       "    src2dst_last_seen_ms  src2dst_duration_ms  src2dst_packets  src2dst_bytes  \\\n",
       "5          1436720908723                  257                5           1279   \n",
       "8          1436720908733                  157                8            896   \n",
       "9          1436720952561                    0                2            169   \n",
       "11         1436720899158                  804                9           1583   \n",
       "15         1436720908737                  160                8            896   \n",
       "\n",
       "    dst2src_first_seen_ms  dst2src_last_seen_ms  dst2src_duration_ms  \\\n",
       "5           1436720908518         1436720910950                 2432   \n",
       "8           1436720908615         1436720908662                   47   \n",
       "9                       0                     0                    0   \n",
       "11          1436720898499         1436720899122                  623   \n",
       "15          1436720908616         1436720908665                   49   \n",
       "\n",
       "    dst2src_packets  dst2src_bytes  bidirectional_min_ps  \\\n",
       "5                 6           4118                    66   \n",
       "8                 6           4671                    66   \n",
       "9                 0              0                    66   \n",
       "11                8           1064                    66   \n",
       "15                6           4671                    66   \n",
       "\n",
       "    bidirectional_mean_ps  bidirectional_stddev_ps  bidirectional_max_ps  \\\n",
       "5              490.636364               588.172640                  1464   \n",
       "8              397.642857               566.204041                  1484   \n",
       "9               84.500000                26.162951                   103   \n",
       "11             155.705882               128.137994                   530   \n",
       "15             397.642857               566.204041                  1484   \n",
       "\n",
       "    src2dst_min_ps  src2dst_mean_ps  src2dst_stddev_ps  src2dst_max_ps  \\\n",
       "5               66       255.800000         424.405702            1015   \n",
       "8               66       112.000000          86.328277             292   \n",
       "9               66        84.500000          26.162951             103   \n",
       "11              66       175.888889         164.147376             530   \n",
       "15              66       112.000000          86.328277             292   \n",
       "\n",
       "    dst2src_min_ps  dst2src_mean_ps  dst2src_stddev_ps  dst2src_max_ps  \\\n",
       "5               66       686.333333         668.351006            1464   \n",
       "8               66       778.500000         720.057706            1484   \n",
       "9                0         0.000000           0.000000               0   \n",
       "11              66       133.000000          74.989523             231   \n",
       "15              66       778.500000         720.057706            1484   \n",
       "\n",
       "    bidirectional_min_piat_ms  bidirectional_mean_piat_ms  \\\n",
       "5                           0                  248.400000   \n",
       "8                           0                   12.076923   \n",
       "9                           0                    0.000000   \n",
       "11                          0                   50.250000   \n",
       "15                          0                   12.307692   \n",
       "\n",
       "    bidirectional_stddev_piat_ms  bidirectional_max_piat_ms  \\\n",
       "5                     698.093945                       2227   \n",
       "8                      22.746654                         71   \n",
       "9                       0.000000                          0   \n",
       "11                     72.009722                        181   \n",
       "15                     23.346608                         71   \n",
       "\n",
       "    src2dst_min_piat_ms  src2dst_mean_piat_ms  src2dst_stddev_piat_ms  \\\n",
       "5                     0             64.250000              126.502635   \n",
       "8                     0             22.428571               28.564880   \n",
       "9                     0              0.000000                0.000000   \n",
       "11                    0            100.500000               83.513900   \n",
       "15                    1             22.857143               28.439577   \n",
       "\n",
       "    src2dst_max_piat_ms  dst2src_min_piat_ms  dst2src_mean_piat_ms  \\\n",
       "5                   254                    0                 486.4   \n",
       "8                    71                    0                   9.4   \n",
       "9                     0                    0                   0.0   \n",
       "11                  183                    0                  89.0   \n",
       "15                   71                    0                   9.8   \n",
       "\n",
       "    dst2src_stddev_piat_ms  dst2src_max_piat_ms  bidirectional_syn_packets  \\\n",
       "5               976.910078                 2227                          0   \n",
       "8                17.728508                   41                          2   \n",
       "9                 0.000000                    0                          0   \n",
       "11               84.261498                  183                          2   \n",
       "15               20.801442                   47                          2   \n",
       "\n",
       "    bidirectional_cwr_packets  bidirectional_ece_packets  \\\n",
       "5                           0                          0   \n",
       "8                           0                          0   \n",
       "9                           0                          0   \n",
       "11                          0                          0   \n",
       "15                          0                          0   \n",
       "\n",
       "    bidirectional_urg_packets  bidirectional_ack_packets  \\\n",
       "5                           0                         11   \n",
       "8                           0                         13   \n",
       "9                           0                          2   \n",
       "11                          0                         16   \n",
       "15                          0                         13   \n",
       "\n",
       "    bidirectional_psh_packets  bidirectional_rst_packets  \\\n",
       "5                           3                          0   \n",
       "8                           4                          0   \n",
       "9                           1                          0   \n",
       "11                          8                          0   \n",
       "15                          4                          0   \n",
       "\n",
       "    bidirectional_fin_packets  src2dst_syn_packets  src2dst_cwr_packets  \\\n",
       "5                           0                    0                    0   \n",
       "8                           0                    1                    0   \n",
       "9                           1                    0                    0   \n",
       "11                          0                    1                    0   \n",
       "15                          0                    1                    0   \n",
       "\n",
       "    src2dst_ece_packets  src2dst_urg_packets  src2dst_ack_packets  \\\n",
       "5                     0                    0                    5   \n",
       "8                     0                    0                    7   \n",
       "9                     0                    0                    2   \n",
       "11                    0                    0                    8   \n",
       "15                    0                    0                    7   \n",
       "\n",
       "    src2dst_psh_packets  src2dst_rst_packets  src2dst_fin_packets  \\\n",
       "5                     1                    0                    0   \n",
       "8                     2                    0                    0   \n",
       "9                     1                    0                    1   \n",
       "11                    4                    0                    0   \n",
       "15                    2                    0                    0   \n",
       "\n",
       "    dst2src_syn_packets  dst2src_cwr_packets  dst2src_ece_packets  \\\n",
       "5                     0                    0                    0   \n",
       "8                     1                    0                    0   \n",
       "9                     0                    0                    0   \n",
       "11                    1                    0                    0   \n",
       "15                    1                    0                    0   \n",
       "\n",
       "    dst2src_urg_packets  dst2src_ack_packets  dst2src_psh_packets  \\\n",
       "5                     0                    6                    2   \n",
       "8                     0                    6                    2   \n",
       "9                     0                    0                    0   \n",
       "11                    0                    8                    4   \n",
       "15                    0                    6                    2   \n",
       "\n",
       "    dst2src_rst_packets  dst2src_fin_packets application_name  \\\n",
       "5                     0                    0              TLS   \n",
       "8                     0                    0    TLS.Instagram   \n",
       "9                     0                    0              TLS   \n",
       "11                    0                    0    TLS.Instagram   \n",
       "15                    0                    0    TLS.Instagram   \n",
       "\n",
       "   application_category_name  application_is_guessed  application_confidence  \\\n",
       "5                        Web                       0                       6   \n",
       "8              SocialNetwork                       0                       6   \n",
       "9                        Web                       0                       6   \n",
       "11             SocialNetwork                       0                       6   \n",
       "15             SocialNetwork                       0                       6   \n",
       "\n",
       "            requested_server_name                client_fingerprint  \\\n",
       "5                             NaN                               NaN   \n",
       "8   igcdn-photos-a-a.akamaihd.net  54ae5fcb0159e2ddf6a50e149221c7c7   \n",
       "9                             NaN                               NaN   \n",
       "11    telegraph-ash.instagram.com  54ae5fcb0159e2ddf6a50e149221c7c7   \n",
       "15  igcdn-photos-a-a.akamaihd.net  54ae5fcb0159e2ddf6a50e149221c7c7   \n",
       "\n",
       "                  server_fingerprint user_agent  content_type  \\\n",
       "5                                NaN        NaN           NaN   \n",
       "8   34d6f0ad0a79e4cfdf145e640cc93f78        NaN           NaN   \n",
       "9                                NaN        NaN           NaN   \n",
       "11  acb741bcdffb787c5a52654c78645bdf        NaN           NaN   \n",
       "15  34d6f0ad0a79e4cfdf145e640cc93f78        NaN           NaN   \n",
       "\n",
       "    src2dst_bytes_data_ratio  dst2src_bytes_data_ratio  \n",
       "5                   0.236984                  0.763016  \n",
       "8                   0.160948                  0.839052  \n",
       "9                   1.000000                  0.000000  \n",
       "11                  0.598036                  0.401964  \n",
       "15                  0.160948                  0.839052  "
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df[df[\"dst_port\"] == 443].head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Extend nfstream"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In some use cases, we need to add features that are computed as packet level. Thus, nfstream handles such scenario using [**NFPlugin**][nfplugin].\n",
    "\n",
    "[nfplugin]: https://nfstream.github.io/docs/api#nfplugin\n",
    "\n",
    "* Let's suppose that we want bidirectional packets with exact IP size equal to 40 counter per flow."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "class Packet40Count(NFPlugin):\n",
    "    def on_init(self, pkt, flow): # flow creation with the first packet\n",
    "        if pkt.ip_size == 40:\n",
    "            flow.udps.packet_with_40_ip_size=1\n",
    "        else:\n",
    "            flow.udps.packet_with_40_ip_size=0\n",
    "        \n",
    "    def on_update(self, pkt, flow): # flow update with each packet belonging to the flow\n",
    "        if pkt.ip_size == 40:\n",
    "            flow.udps.packet_with_40_ip_size += 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = NFStreamer(source=\"pcap/google_ssl.pcap\", udps=[Packet40Count()]).to_pandas()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>expiration_id</th>\n",
       "      <th>src_ip</th>\n",
       "      <th>src_mac</th>\n",
       "      <th>src_oui</th>\n",
       "      <th>src_port</th>\n",
       "      <th>dst_ip</th>\n",
       "      <th>dst_mac</th>\n",
       "      <th>dst_oui</th>\n",
       "      <th>dst_port</th>\n",
       "      <th>protocol</th>\n",
       "      <th>ip_version</th>\n",
       "      <th>vlan_id</th>\n",
       "      <th>tunnel_id</th>\n",
       "      <th>bidirectional_first_seen_ms</th>\n",
       "      <th>bidirectional_last_seen_ms</th>\n",
       "      <th>bidirectional_duration_ms</th>\n",
       "      <th>bidirectional_packets</th>\n",
       "      <th>bidirectional_bytes</th>\n",
       "      <th>src2dst_first_seen_ms</th>\n",
       "      <th>src2dst_last_seen_ms</th>\n",
       "      <th>src2dst_duration_ms</th>\n",
       "      <th>src2dst_packets</th>\n",
       "      <th>src2dst_bytes</th>\n",
       "      <th>dst2src_first_seen_ms</th>\n",
       "      <th>dst2src_last_seen_ms</th>\n",
       "      <th>dst2src_duration_ms</th>\n",
       "      <th>dst2src_packets</th>\n",
       "      <th>dst2src_bytes</th>\n",
       "      <th>application_name</th>\n",
       "      <th>application_category_name</th>\n",
       "      <th>application_is_guessed</th>\n",
       "      <th>application_confidence</th>\n",
       "      <th>requested_server_name</th>\n",
       "      <th>client_fingerprint</th>\n",
       "      <th>server_fingerprint</th>\n",
       "      <th>user_agent</th>\n",
       "      <th>content_type</th>\n",
       "      <th>udps.packet_with_40_ip_size</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>172.31.3.224</td>\n",
       "      <td>80:c6:ca:00:9e:9f</td>\n",
       "      <td>80:c6:ca</td>\n",
       "      <td>42835</td>\n",
       "      <td>216.58.212.100</td>\n",
       "      <td>00:0e:8e:4d:b4:a8</td>\n",
       "      <td>00:0e:8e</td>\n",
       "      <td>443</td>\n",
       "      <td>6</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1434443394683</td>\n",
       "      <td>1434443401353</td>\n",
       "      <td>6670</td>\n",
       "      <td>28</td>\n",
       "      <td>9108</td>\n",
       "      <td>1434443394683</td>\n",
       "      <td>1434443401353</td>\n",
       "      <td>6670</td>\n",
       "      <td>16</td>\n",
       "      <td>1512</td>\n",
       "      <td>1434443394717</td>\n",
       "      <td>1434443401308</td>\n",
       "      <td>6591</td>\n",
       "      <td>12</td>\n",
       "      <td>7596</td>\n",
       "      <td>TLS</td>\n",
       "      <td>Web</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>14</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   id  expiration_id        src_ip            src_mac   src_oui  src_port  \\\n",
       "0   0              0  172.31.3.224  80:c6:ca:00:9e:9f  80:c6:ca     42835   \n",
       "\n",
       "           dst_ip            dst_mac   dst_oui  dst_port  protocol  \\\n",
       "0  216.58.212.100  00:0e:8e:4d:b4:a8  00:0e:8e       443         6   \n",
       "\n",
       "   ip_version  vlan_id  tunnel_id  bidirectional_first_seen_ms  \\\n",
       "0           4        0          0                1434443394683   \n",
       "\n",
       "   bidirectional_last_seen_ms  bidirectional_duration_ms  \\\n",
       "0               1434443401353                       6670   \n",
       "\n",
       "   bidirectional_packets  bidirectional_bytes  src2dst_first_seen_ms  \\\n",
       "0                     28                 9108          1434443394683   \n",
       "\n",
       "   src2dst_last_seen_ms  src2dst_duration_ms  src2dst_packets  src2dst_bytes  \\\n",
       "0         1434443401353                 6670               16           1512   \n",
       "\n",
       "   dst2src_first_seen_ms  dst2src_last_seen_ms  dst2src_duration_ms  \\\n",
       "0          1434443394717         1434443401308                 6591   \n",
       "\n",
       "   dst2src_packets  dst2src_bytes application_name application_category_name  \\\n",
       "0               12           7596              TLS                       Web   \n",
       "\n",
       "   application_is_guessed  application_confidence  requested_server_name  \\\n",
       "0                       1                       1                    NaN   \n",
       "\n",
       "   client_fingerprint  server_fingerprint  user_agent  content_type  \\\n",
       "0                 NaN                 NaN         NaN           NaN   \n",
       "\n",
       "   udps.packet_with_40_ip_size  \n",
       "0                           14  "
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Our Dataframe have a new column named `udps.packet_with_40_ip_size`."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}