demos/for_analysis.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Tutorial: Data Analysis in Graphistry\n",
    "\n",
    "1. Register\n",
    "2. Load table\n",
    "3. Plot: \n",
    "    - Simple: input is a list of edges\n",
    "    - Arbitrary: input is a table (_hypergraph_ transform)\n",
    "3. Advanced plotting\n",
    "4. Further reading\n",
    "    - [PyGraphistry](https://github.com/graphistry/pygraphistry)\n",
    "    - [PyGraphistry demos: database connectors, ...](demos_databases_apis)\n",
    "    - [graph-app-kit: Streamlit graph dashboarding](https://github.com/graphistry/graph-app-kit)\n",
    "    - [UI Guide](https://hub.graphistry.com/docs/ui/index/)\n",
    "    - [CSV upload notebook app](upload_csv_miniapp.ipynb)\n",
    "    \n",
    "## 1. Register\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 101,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "import graphistry\n",
    "\n",
    "# To specify Graphistry account & server, use:\n",
    "# graphistry.register(api=3, username='...', password='...', protocol='https', server='hub.graphistry.com')\n",
    "# For more options, see https://github.com/graphistry/pygraphistry#configure\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. Load table\n",
    "Graphistry works seamlessly with dataframes like [Pandas](https://pandas.pydata.org/) and GPU [RAPIDS cuDF](https://www.rapids.ai)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 94,
   "metadata": {
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>attackerIP</th>\n",
       "      <th>victimIP</th>\n",
       "      <th>victimPort</th>\n",
       "      <th>vulnName</th>\n",
       "      <th>count</th>\n",
       "      <th>time(max)</th>\n",
       "      <th>time(min)</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>64</th>\n",
       "      <td>178.77.190.33</td>\n",
       "      <td>172.31.14.66</td>\n",
       "      <td>445.0</td>\n",
       "      <td>MS08067 (NetAPI)</td>\n",
       "      <td>6</td>\n",
       "      <td>1.419968e+09</td>\n",
       "      <td>1.419967e+09</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>112.209.78.240</td>\n",
       "      <td>172.31.14.66</td>\n",
       "      <td>445.0</td>\n",
       "      <td>MS08067 (NetAPI)</td>\n",
       "      <td>10</td>\n",
       "      <td>1.414516e+09</td>\n",
       "      <td>1.414514e+09</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>182</th>\n",
       "      <td>79.140.174.193</td>\n",
       "      <td>172.31.14.66</td>\n",
       "      <td>445.0</td>\n",
       "      <td>MS08067 (NetAPI)</td>\n",
       "      <td>2</td>\n",
       "      <td>1.422062e+09</td>\n",
       "      <td>1.422062e+09</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         attackerIP      victimIP  victimPort          vulnName  count  \\\n",
       "64    178.77.190.33  172.31.14.66       445.0  MS08067 (NetAPI)      6   \n",
       "7    112.209.78.240  172.31.14.66       445.0  MS08067 (NetAPI)     10   \n",
       "182  79.140.174.193  172.31.14.66       445.0  MS08067 (NetAPI)      2   \n",
       "\n",
       "        time(max)     time(min)  \n",
       "64   1.419968e+09  1.419967e+09  \n",
       "7    1.414516e+09  1.414514e+09  \n",
       "182  1.422062e+09  1.422062e+09  "
      ]
     },
     "execution_count": 94,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "\n",
    "df = pd.read_csv('./data/honeypot.csv')\n",
    "\n",
    "df.sample(3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. Plot"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### A. Simple graphs\n",
    "* Build up a set of bindings. Simple graphs are:\n",
    "    - required: edge table, with src+dst ID columns, and optional additional property columns\n",
    "    - optional: node table, with matching node ID column\n",
    "* See [UI Guide](https://hub.graphistry.com/docs/ui/index/) for in-tool activity\n",
    "\n",
    "**Demo graph schema:**\n",
    "\n",
    "* **Input table**: Above alerts `df` with columns `| attackerIP | victimIP |`\n",
    "* **Edges**: Link `df`'s columns `attackerIP -> victimIP`\n",
    "* **Nodes**: Unspecified; Graphistry defaults to generating based on the edges\n",
    "* **Node colors**: Graphistry defaults to inferring the commmunity\n",
    "* **Node sizes**: Graphistry defaults to the number of edges (\"degree\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "outputs": [],
   "source": [
    "g = graphistry.edges(df, 'attackerIP', 'victimIP')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "\n",
       "            <iframe id=\"7f47a15d-7202-4dee-9947-87cf57c5d900\" src=\"/graph/graph.html?dataset=e7e6b3935fbb55559a1bfeceb47af599&type=vgraph&viztoken=afb946d460574779ae934c51e1716264&usertag=e9cf6564-pygraphistry-0.18.2&splashAfter=1625871462&info=true\"\n",
       "                    allowfullscreen=\"true\" webkitallowfullscreen=\"true\" mozallowfullscreen=\"true\"\n",
       "                    oallowfullscreen=\"true\" msallowfullscreen=\"true\"\n",
       "                    style=\"width:100%; height:500px; border: 1px solid #DDD; overflow: hidden\">\n",
       "            </iframe>\n",
       "        \n",
       "            <script>\n",
       "                try {\n",
       "                  $(\"#7f47a15d-7202-4dee-9947-87cf57c5d900\").bind('mousewheel', function(e) { e.preventDefault(); });\n",
       "                } catch (e) { console.error('exn catching scroll', e); }\n",
       "            </script>\n",
       "        "
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "g.plot()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## B. Hypergraphs -- Plot arbitrary tables\n",
    "\n",
    "The hypergraph transform is a convenient method to transform tables into graphs:\n",
    "\n",
    "* It extracts entities from the table and links them together\n",
    "* Entities get linked together when they are from the same row\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Approach 1: Treat each row as a node, and link it to each cell value in it\n",
    "\n",
    "**Demo graph schema:**\n",
    "* **Edges**: row -> attackerIP, row -> victimIP, row -> victimPort, row ->  volnName\n",
    "* **Nodes**: row, attackerIP, victimIP, victimPort, vulnName\n",
    "* **Node colors**: Automatic based on inferred commmunity\n",
    "* **node sizes**: Number of edges"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 93,
   "metadata": {
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "# links 880\n",
      "# events 220\n",
      "# attrib entities 221\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "            <iframe id=\"a40a84a3-3b7f-445a-96cc-a1bd8f3df67a\" src=\"/graph/graph.html?dataset=4434a493b4d643858a869587aa7539f4&type=arrow&viztoken=17902bd6-04ab-40c2-acc1-8bc65727ac4d&usertag=e9cf6564-pygraphistry-0.18.2&splashAfter=1625874550&info=true\"\n",
       "                    allowfullscreen=\"true\" webkitallowfullscreen=\"true\" mozallowfullscreen=\"true\"\n",
       "                    oallowfullscreen=\"true\" msallowfullscreen=\"true\"\n",
       "                    style=\"width:100%; height:500px; border: 1px solid #DDD; overflow: hidden\">\n",
       "            </iframe>\n",
       "        \n",
       "            <script>\n",
       "                try {\n",
       "                  $(\"#a40a84a3-3b7f-445a-96cc-a1bd8f3df67a\").bind('mousewheel', function(e) { e.preventDefault(); });\n",
       "                } catch (e) { console.error('exn catching scroll', e); }\n",
       "            </script>\n",
       "        "
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "execution_count": 93,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "hg1 = graphistry.hypergraph(\n",
    "    df,\n",
    "\n",
    "    # Optional: Subset of columns to turn into nodes; defaults to all\n",
    "    entity_types=['attackerIP', 'victimIP', 'victimPort', 'vulnName'],\n",
    "\n",
    "    # Optional: merge nodes when their IDs appear in multiple columns\n",
    "    # ... so replace nodes attackerIP::1.1.1.1 and victimIP::1.1.1.1\n",
    "    # ... with just one node ip::1.1.1.1\n",
    "    opts={\n",
    "        'CATEGORIES': {\n",
    "            'ip': ['attackerIP', 'victimIP']\n",
    "        }\n",
    "    })\n",
    "\n",
    "hg1_g = hg1['graph']\n",
    "hg1_g.plot()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Approach 2: Link values from column entries\n",
    "\n",
    "For more advanced hypergraph control, we can skip the row node, and control which edges are generated, by enabling `direct`.\n",
    "\n",
    "Demo graph schema:\n",
    "* Edges: \n",
    "  * attackerIP -> victimIP, attackerIP -> victimPort, attackerIP -> vulnName\n",
    "  * victimPort -> victimIP\n",
    "  * vulnName -> victimIP\n",
    "* Nodes: attackerIP, victimIP, victimPort, vulnName\n",
    "* Default colors: Automatic based on inferred commmunity\n",
    "* Default node size: Number of edges\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 102,
   "metadata": {
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "# links 1100\n",
      "# events 220\n",
      "# attrib entities 221\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "            <iframe id=\"fb220116-5a68-4a21-8d09-73bfd1abaa86\" src=\"/graph/graph.html?dataset=99ae2d0707c245a78354e0e188ea66e5&type=arrow&viztoken=5af79dc2-6116-489a-9153-bfe171f515d7&usertag=e9cf6564-pygraphistry-0.18.2&splashAfter=1625877067&info=true\"\n",
       "                    allowfullscreen=\"true\" webkitallowfullscreen=\"true\" mozallowfullscreen=\"true\"\n",
       "                    oallowfullscreen=\"true\" msallowfullscreen=\"true\"\n",
       "                    style=\"width:100%; height:500px; border: 1px solid #DDD; overflow: hidden\">\n",
       "            </iframe>\n",
       "        \n",
       "            <script>\n",
       "                try {\n",
       "                  $(\"#fb220116-5a68-4a21-8d09-73bfd1abaa86\").bind('mousewheel', function(e) { e.preventDefault(); });\n",
       "                } catch (e) { console.error('exn catching scroll', e); }\n",
       "            </script>\n",
       "        "
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "execution_count": 102,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "hg2 = graphistry.hypergraph(\n",
    "    df,\n",
    "    entity_types=['attackerIP', 'victimIP', 'victimPort', 'vulnName'],\n",
    "    direct=True,\n",
    "    opts={\n",
    "        # Optional: Without, creates edges that are all-to-all for each row \n",
    "        'EDGES': {\n",
    "            'attackerIP': ['victimIP', 'victimPort', 'vulnName'],\n",
    "            'victimPort': ['victimIP'],\n",
    "            'vulnName': ['victimIP']\n",
    "        },\n",
    "\n",
    "        # Optional: merge nodes when their IDs appear in multiple columns\n",
    "        # ... so replace nodes attackerIP::1.1.1.1 and victimIP::1.1.1.1\n",
    "        # ... with just one node ip::1.1.1.1\n",
    "        'CATEGORIES': {\n",
    "            'ip': ['attackerIP', 'victimIP']\n",
    "        }\n",
    "    })\n",
    "\n",
    "hg2_g = hg2['graph']\n",
    "hg2_g.plot()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. Advanced plotting\n",
    "\n",
    "\n",
    "You can then drive visual styles based on node and edge attributes\n",
    "\n",
    "This demo starts by computing a node table. By default, you do not need to explictly provide a table of nodes, but then you may lack data for node properties:\n",
    "\n",
    "* Regular inferred graph nodes will only have id and degree\n",
    "* Hypergraph edges and row nodes will have many properties, but hypergraph entity nodes will only have id, type/category, and degree\n",
    "\n",
    "**Demo schema:**\n",
    "\n",
    "* **Node table:** `| node_id | type | attacks |`\n",
    "* **Point size**: number of attacks\n",
    "* **Point icon & color**: attacker vs victim\n",
    "* **Edge color**: based on first attack"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>node_id</th>\n",
       "      <th>type</th>\n",
       "      <th>attacks</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>31</th>\n",
       "      <td>125.64.35.67</td>\n",
       "      <td>attacker</td>\n",
       "      <td>6.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>32</th>\n",
       "      <td>125.64.35.68</td>\n",
       "      <td>attacker</td>\n",
       "      <td>4.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>95</th>\n",
       "      <td>198.204.253.101</td>\n",
       "      <td>attacker</td>\n",
       "      <td>2.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>78</th>\n",
       "      <td>188.225.73.153</td>\n",
       "      <td>attacker</td>\n",
       "      <td>2.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>79</th>\n",
       "      <td>188.44.107.239</td>\n",
       "      <td>attacker</td>\n",
       "      <td>2.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            node_id      type  attacks\n",
       "31     125.64.35.67  attacker      6.0\n",
       "32     125.64.35.68  attacker      4.0\n",
       "95  198.204.253.101  attacker      2.0\n",
       "78   188.225.73.153  attacker      2.0\n",
       "79   188.44.107.239  attacker      2.0"
      ]
     },
     "execution_count": 62,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Cell:\n",
    "# Compute nodes_df by combining entities in attackerIP and victimIP\n",
    "# As part of this, compute attack counts for each node \n",
    "\n",
    "targets_df = (\n",
    "    df\n",
    "    [['victimIP']]\n",
    "    .drop_duplicates()\n",
    "    .rename(columns={'victimIP': 'node_id'})\n",
    "    .assign(type='victim')\n",
    ")\n",
    "\n",
    "attackers_df = (\n",
    "    df\n",
    "    .groupby(['attackerIP'])\n",
    "    .agg(attacks=pd.NamedAgg(column=\"attackerIP\", aggfunc=\"count\"))\n",
    "    .reset_index()\n",
    "    .rename(columns={'attackerIP': 'node_id'}).assign(type='attacker')\n",
    ")\n",
    "\n",
    "nodes_df = pd.concat([targets_df, attackers_df])\n",
    "\n",
    "nodes_df.sort_values(by='attacks', ascending=False)[:5]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 86,
   "metadata": {
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "\n",
       "            <iframe id=\"1656abf6-23d6-4c32-a6dd-a7ae726987af\" src=\"/graph/graph.html?dataset=4f7c1e5adf1c48c6a085e54cf0dfa78b&type=arrow&viztoken=bd539693-1281-4e9c-ab38-9bba11a8b4ff&usertag=e9cf6564-pygraphistry-0.18.2&splashAfter=1625873593&info=true&play=1000&pointSize=0.5\"\n",
       "                    allowfullscreen=\"true\" webkitallowfullscreen=\"true\" mozallowfullscreen=\"true\"\n",
       "                    oallowfullscreen=\"true\" msallowfullscreen=\"true\"\n",
       "                    style=\"width:100%; height:500px; border: 1px solid #DDD; overflow: hidden\">\n",
       "            </iframe>\n",
       "        \n",
       "            <script>\n",
       "                try {\n",
       "                  $(\"#1656abf6-23d6-4c32-a6dd-a7ae726987af\").bind('mousewheel', function(e) { e.preventDefault(); });\n",
       "                } catch (e) { console.error('exn catching scroll', e); }\n",
       "            </script>\n",
       "        "
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "execution_count": 86,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Cell:\n",
    "# Add\n",
    "\n",
    "\n",
    "# New encodings features requires api=3: `graphistry.register(api=3, username='...', password='...')\n",
    "\n",
    "g2 = (g\n",
    "      .nodes(nodes_df, 'node_id')\n",
    "\n",
    "      # 'red', '#f00', '#ff0000'\n",
    "      .encode_point_color('type', categorical_mapping={\n",
    "          'attacker': 'red',\n",
    "          'victim': 'white'\n",
    "      }, default_mapping='gray')\n",
    "\n",
    "      # Icons: https://fontawesome.com/v4.7/cheatsheet/\n",
    "      .encode_point_icon('type', categorical_mapping={\n",
    "          'attacker': 'bomb',\n",
    "          'victim': 'laptop'\n",
    "      })\n",
    "\n",
    "      # Gradient\n",
    "      .encode_edge_color('time(min)', palette=['blue', 'purple', 'red'], as_continuous=True)\n",
    "\n",
    "      .encode_point_size('attacks')\n",
    "\n",
    "      .addStyle(bg={'color': '#eee'}, page={'title': 'My Graph'})\n",
    "\n",
    "      # Options: https://hub.graphistry.com/docs/api/1/rest/url/\n",
    "      .settings(url_params={'play': 1000, 'pointSize': 0.5})\n",
    ")\n",
    "\n",
    "g2.plot(as_files=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Advanced bindings work with hypergraphs too\n",
    "\n",
    "Hypergraphs precompute a lot of values on nodes and edges, which we can use to drive clearer visualizations"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 104,
   "metadata": {
    "jupyter": {
     "outputs_hidden": false
    },
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>attackerIP</th>\n",
       "      <th>nodeTitle</th>\n",
       "      <th>type</th>\n",
       "      <th>category</th>\n",
       "      <th>nodeID</th>\n",
       "      <th>victimIP</th>\n",
       "      <th>victimPort</th>\n",
       "      <th>vulnName</th>\n",
       "      <th>EventID</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>159</th>\n",
       "      <td>77.52.11.94</td>\n",
       "      <td>77.52.11.94</td>\n",
       "      <td>attackerIP</td>\n",
       "      <td>ip</td>\n",
       "      <td>ip::77.52.11.94</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>162</th>\n",
       "      <td>78.187.242.78</td>\n",
       "      <td>78.187.242.78</td>\n",
       "      <td>attackerIP</td>\n",
       "      <td>ip</td>\n",
       "      <td>ip::78.187.242.78</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>170</th>\n",
       "      <td>81.47.128.144</td>\n",
       "      <td>81.47.128.144</td>\n",
       "      <td>attackerIP</td>\n",
       "      <td>ip</td>\n",
       "      <td>ip::81.47.128.144</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        attackerIP      nodeTitle        type category             nodeID  \\\n",
       "159    77.52.11.94    77.52.11.94  attackerIP       ip    ip::77.52.11.94   \n",
       "162  78.187.242.78  78.187.242.78  attackerIP       ip  ip::78.187.242.78   \n",
       "170  81.47.128.144  81.47.128.144  attackerIP       ip  ip::81.47.128.144   \n",
       "\n",
       "    victimIP  victimPort vulnName EventID  \n",
       "159      NaN         NaN      NaN     NaN  \n",
       "162      NaN         NaN      NaN     NaN  \n",
       "170      NaN         NaN      NaN     NaN  "
      ]
     },
     "execution_count": 104,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "hg2_g._nodes.sample(3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 103,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>edgeType</th>\n",
       "      <th>category</th>\n",
       "      <th>vulnName</th>\n",
       "      <th>dst</th>\n",
       "      <th>time(max)</th>\n",
       "      <th>time(min)</th>\n",
       "      <th>src</th>\n",
       "      <th>victimPort</th>\n",
       "      <th>victimIP</th>\n",
       "      <th>EventID</th>\n",
       "      <th>count</th>\n",
       "      <th>attackerIP</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>516</th>\n",
       "      <td>ip::vulnName</td>\n",
       "      <td>attackerIP::vulnName</td>\n",
       "      <td>MS08067 (NetAPI)</td>\n",
       "      <td>vulnName::MS08067 (NetAPI)</td>\n",
       "      <td>1.416885e+09</td>\n",
       "      <td>1.416881e+09</td>\n",
       "      <td>ip::186.149.87.94</td>\n",
       "      <td>445.0</td>\n",
       "      <td>172.31.14.66</td>\n",
       "      <td>EventID::76</td>\n",
       "      <td>3</td>\n",
       "      <td>186.149.87.94</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>932</th>\n",
       "      <td>vulnName::ip</td>\n",
       "      <td>vulnName::victimIP</td>\n",
       "      <td>MS08067 (NetAPI)</td>\n",
       "      <td>ip::172.31.14.66</td>\n",
       "      <td>1.423515e+09</td>\n",
       "      <td>1.423515e+09</td>\n",
       "      <td>vulnName::MS08067 (NetAPI)</td>\n",
       "      <td>445.0</td>\n",
       "      <td>172.31.14.66</td>\n",
       "      <td>EventID::52</td>\n",
       "      <td>1</td>\n",
       "      <td>176.119.227.9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>983</th>\n",
       "      <td>vulnName::ip</td>\n",
       "      <td>vulnName::victimIP</td>\n",
       "      <td>MS08067 (NetAPI)</td>\n",
       "      <td>ip::172.31.14.66</td>\n",
       "      <td>1.423932e+09</td>\n",
       "      <td>1.423932e+09</td>\n",
       "      <td>vulnName::MS08067 (NetAPI)</td>\n",
       "      <td>445.0</td>\n",
       "      <td>172.31.14.66</td>\n",
       "      <td>EventID::103</td>\n",
       "      <td>2</td>\n",
       "      <td>192.110.160.227</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         edgeType              category          vulnName  \\\n",
       "516  ip::vulnName  attackerIP::vulnName  MS08067 (NetAPI)   \n",
       "932  vulnName::ip    vulnName::victimIP  MS08067 (NetAPI)   \n",
       "983  vulnName::ip    vulnName::victimIP  MS08067 (NetAPI)   \n",
       "\n",
       "                            dst     time(max)     time(min)  \\\n",
       "516  vulnName::MS08067 (NetAPI)  1.416885e+09  1.416881e+09   \n",
       "932            ip::172.31.14.66  1.423515e+09  1.423515e+09   \n",
       "983            ip::172.31.14.66  1.423932e+09  1.423932e+09   \n",
       "\n",
       "                            src  victimPort      victimIP       EventID  \\\n",
       "516           ip::186.149.87.94       445.0  172.31.14.66   EventID::76   \n",
       "932  vulnName::MS08067 (NetAPI)       445.0  172.31.14.66   EventID::52   \n",
       "983  vulnName::MS08067 (NetAPI)       445.0  172.31.14.66  EventID::103   \n",
       "\n",
       "     count       attackerIP  \n",
       "516      3    186.149.87.94  \n",
       "932      1    176.119.227.9  \n",
       "983      2  192.110.160.227  "
      ]
     },
     "execution_count": 103,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "hg2_g._edges.sample(3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 113,
   "metadata": {
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "\n",
       "            <iframe id=\"e460129d-3b2a-42c8-a42f-9be3e91b9a45\" src=\"/graph/graph.html?dataset=ce34cffdb19142dfbf18b88985d54481&type=arrow&viztoken=63396a5b-bbf6-42ac-af74-2f2025b4d4d0&usertag=e9cf6564-pygraphistry-0.18.2&splashAfter=1625877441&info=true&pointsOfInterestMax=10\"\n",
       "                    allowfullscreen=\"true\" webkitallowfullscreen=\"true\" mozallowfullscreen=\"true\"\n",
       "                    oallowfullscreen=\"true\" msallowfullscreen=\"true\"\n",
       "                    style=\"width:100%; height:500px; border: 1px solid #DDD; overflow: hidden\">\n",
       "            </iframe>\n",
       "        \n",
       "            <script>\n",
       "                try {\n",
       "                  $(\"#e460129d-3b2a-42c8-a42f-9be3e91b9a45\").bind('mousewheel', function(e) { e.preventDefault(); });\n",
       "                } catch (e) { console.error('exn catching scroll', e); }\n",
       "            </script>\n",
       "        "
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "execution_count": 113,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "(hg2_g\n",
    "\n",
    " .encode_point_color('type', categorical_mapping={\n",
    "     'attackerIP': 'yellow',\n",
    "     'victimIP': 'blue'\n",
    " }, default_mapping='gray')\n",
    "\n",
    " .encode_point_icon('type', categorical_mapping={\n",
    "      'attackerIP': 'bomb',\n",
    "      'victimIP': 'laptop'\n",
    " }, default_mapping='')\n",
    "\n",
    " .encode_edge_color('time(min)', palette=['blue', 'purple', 'red'], as_continuous=True)\n",
    "\n",
    " .settings(url_params={'pointsOfInterestMax': 10})\n",
    "\n",
    ").plot()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Further reading:\n",
    "  - [PyGraphistry](https://github.com/graphistry/pygraphistry)\n",
    "  - [PyGraphistry demos: database connectors, ...](demos_databases_apis)\n",
    "  - [graph-app-kit: Streamlit graph dashboarding](https://github.com/graphistry/graph-app-kit)\n",
    "  - [UI Guide](https://hub.graphistry.com/docs/ui/index/)\n",
    "  - [CSV upload notebook app](upload_csv_miniapp.ipynb)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.11"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}