demos/gfql/benchmark_hops_cpu_gpu.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": [],
      "gpuType": "T4"
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "source": [
        "# GFQL CPU, GPU Benchmark\n",
        "\n",
        "This notebook examines GFQL progerty graph query performance on 1-8 hop queries using CPU + GPU modes on various real-world 100K - 100M edge graphs. The data comes from a variety of popular social networks. The single-threaded CPU mode benefits from GFQL's novel dataframe engine, and the GPU mode further adds single-GPU acceleration. Both the `chain()` and `hop()` methods are examined.\n",
        "\n",
        "The benchmark does not examine bigger-than-memory and distributed scenarios. The provided results here are from running on a free Google Colab T4 runtime, with a 2.2GHz Intel CPU (12 GB CPU RAM) and T4 Nvidia GPU (16 GB GPU RAM).\n",
        "\n",
        "## Data\n",
        "From [SNAP](https://snap.stanford.edu/data/)\n",
        "\n",
        "| Network | Nodes     | Edges        |\n",
        "|-------------|-----------|--------------|\n",
        "| **Facebook**| 4,039     | 88,234       |\n",
        "| **Twitter** | 81,306    | 2,420,766    |\n",
        "| **GPlus**   | 107,614   | 30,494,866   |\n",
        "| **Orkut**   | 3,072,441 | 117,185,082  |\n",
        "\n",
        "## Results\n",
        "\n",
        "Definitions:\n",
        "\n",
        "* GTEPS: Giga (billion) edges traversed per second\n",
        "\n",
        "* T edges / \\$: Estimated trillion edges traversed for 1\\$ USD based on observed GTEPS and a 3yr AWS reservation (as of 12/2023)\n",
        "\n",
        "Tasks:\n",
        "\n",
        "1. `chain()` - includes complex pre/post processing\n",
        "\n",
        "  **Task**: `g.chain([n({'id': some_id}), e_forward(hops=some_n)])`\n",
        "\n",
        "\n",
        "| **Dataset** | Max GPU Speedup      | CPU GTEPS   | GPU GTEPS   | T CPU edges / \\$ (t3.l) | T GPU edges / \\$ (g4dn.xl) |\n",
        "|-------------|--------------|-------------|-------------|----------------------------|--------------------------------|\n",
        "| **Facebook**| 1.1X  | 0.66 | 0.61 | 65.7                | 10.4                    |\n",
        "| **Twitter** | 17.4X   | 0.17 | 2.81 | 16.7                | 48.1                    |\n",
        "| **GPlus**   | 43.8X  | 0.09 | 2.87 | 8.5                | 49.2                    |\n",
        "| **Orkut**   | N/A            | N/A         | 12.15 | N/A                        | 208.3                    |\n",
        "| **AVG** | 20.7X | 0.30 | 4.61 | 30.3 | 79.0\n",
        "| **MAX** | 43.8X | 0.66 | 12.15 | 65.7 | 208.3\n",
        "\n",
        "\n",
        "2. `hop()` - core property search primitive similar to BFS\n",
        "\n",
        "  **Task**: `g.hop(nodes=[some_id], direction='forward', hops=some_n)`\n",
        "\n",
        "\n",
        "| **Dataset** | Max GPU Speedup | CPU GTEPS | GPU GTEPS | T CPU edges / \\$ (t3.l) | T GPU edges / \\$ (g4dn.xl) |\n",
        "|-------------|-------------|-----------|-----------|--------------------|--------------------------------|\n",
        "| **Facebook**| 3X          | 0.47      | 1.47     | 47.0        | 25.2                    |\n",
        "| **Twitter** | 42X         | 0.50      | 10.51      | 50.2        | 180.2                    |\n",
        "| **GPlus**   | 21X         | 0.26      | 4.11       | 26.2        | 70.4                    |\n",
        "| **Orkut**   | N/A         | N/A       | 41.50     | N/A                | 711.4                    |\n",
        "| **AVG** | 22X | 0.41 | 14.4 | 41.1 | 246.8\n",
        "| **MAX** | 42X | 0.50 | 41.50 | 50.2 | 711.4\n"
      ],
      "metadata": {
        "id": "GZxoiU8sQDk_"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Optional: GPU setup - Google Colab"
      ],
      "metadata": {
        "id": "SAj8lhREEOwS"
      }
    },
    {
      "cell_type": "markdown",
      "source": [],
      "metadata": {
        "id": "4hrEEAAm7DTO"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Report GPU used when GPU benchmarking\n",
        "! nvidia-smi"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "W2MF6ZsjDv3B",
        "outputId": "46088cbc-2db9-4529-f724-dc57ed85dfb7"
      },
      "execution_count": 1,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Tue Dec 26 00:50:30 2023       \n",
            "+---------------------------------------------------------------------------------------+\n",
            "| NVIDIA-SMI 535.104.05             Driver Version: 535.104.05   CUDA Version: 12.2     |\n",
            "|-----------------------------------------+----------------------+----------------------+\n",
            "| GPU  Name                 Persistence-M | Bus-Id        Disp.A | Volatile Uncorr. ECC |\n",
            "| Fan  Temp   Perf          Pwr:Usage/Cap |         Memory-Usage | GPU-Util  Compute M. |\n",
            "|                                         |                      |               MIG M. |\n",
            "|=========================================+======================+======================|\n",
            "|   0  Tesla T4                       Off | 00000000:00:04.0 Off |                    0 |\n",
            "| N/A   54C    P8              10W /  70W |      0MiB / 15360MiB |      0%      Default |\n",
            "|                                         |                      |                  N/A |\n",
            "+-----------------------------------------+----------------------+----------------------+\n",
            "                                                                                         \n",
            "+---------------------------------------------------------------------------------------+\n",
            "| Processes:                                                                            |\n",
            "|  GPU   GI   CI        PID   Type   Process name                            GPU Memory |\n",
            "|        ID   ID                                                             Usage      |\n",
            "|=======================================================================================|\n",
            "|  No running processes found                                                           |\n",
            "+---------------------------------------------------------------------------------------+\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# if in google colab\n",
        "!git clone https://github.com/rapidsai/rapidsai-csp-utils.git\n",
        "!python rapidsai-csp-utils/colab/pip-install.py"
      ],
      "metadata": {
        "id": "Aikh0x4ID_wK"
      },
      "execution_count": 8,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "import cudf\n",
        "cudf.__version__"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        },
        "id": "Lwekdei1dH3N",
        "outputId": "71f5b01d-7917-4283-8338-969167d6e1e8"
      },
      "execution_count": 3,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "'23.12.01'"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            }
          },
          "metadata": {},
          "execution_count": 3
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# 1. Install & configure"
      ],
      "metadata": {
        "id": "QQpsrtwBT7sa"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "#! pip install graphistry[igraph]\n",
        "\n",
        "!pip install -q igraph\n",
        "#!pip install -q git+https://github.com/graphistry/pygraphistry.git@dev/cugfql\n",
        "!pip install -q graphistry\n"
      ],
      "metadata": {
        "id": "cYjRbgkU9Sx8",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "2cf25531-9b8b-4715-ccc7-e79094d84ebd"
      },
      "execution_count": 2,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "  Preparing metadata (setup.py) ... \u001b[?25l\u001b[?25hdone\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Imports"
      ],
      "metadata": {
        "id": "Ff6Tt9DhkePl"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import pandas as pd\n",
        "\n",
        "import graphistry\n",
        "\n",
        "from graphistry import (\n",
        "\n",
        "    # graph operators\n",
        "    n, e_undirected, e_forward, e_reverse,\n",
        "\n",
        "    # attribute predicates\n",
        "    is_in, ge, startswith, contains, match as match_re\n",
        ")\n",
        "graphistry.__version__"
      ],
      "metadata": {
        "id": "S5_y0CbLkjft",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        },
        "outputId": "a68a9c4b-c9c5-4b8b-ea4f-7bf1e4ddf315"
      },
      "execution_count": 3,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "'0.32.0+12.g72e778c'"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            }
          },
          "metadata": {},
          "execution_count": 3
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "import cudf"
      ],
      "metadata": {
        "id": "I7Fg75jsG4co"
      },
      "execution_count": 6,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "#work around google colab shell encoding bugs\n",
        "\n",
        "import locale\n",
        "locale.getpreferredencoding = lambda: \"UTF-8\""
      ],
      "metadata": {
        "id": "uLZKph2-a5M4"
      },
      "execution_count": 7,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "# 2. Perf benchmarks"
      ],
      "metadata": {
        "id": "eU9SyauNUHtR"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Facebook: 88K edges"
      ],
      "metadata": {
        "id": "NA0Ym11fkB8j"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "df = pd.read_csv('https://raw.githubusercontent.com/graphistry/pygraphistry/master/demos/data/facebook_combined.txt', sep=' ', names=['s', 'd'])\n",
        "print(df.shape)\n",
        "df.head(5)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 224
        },
        "id": "vXuQogHekClJ",
        "outputId": "64db92c0-2704-438b-d0e4-25865acbb5e9"
      },
      "execution_count": 10,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "(88234, 2)\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "   s  d\n",
              "0  0  1\n",
              "1  0  2\n",
              "2  0  3\n",
              "3  0  4\n",
              "4  0  5"
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-1ff270ef-f23b-4ff0-aff0-61786d9d4eeb\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>s</th>\n",
              "      <th>d</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>0</td>\n",
              "      <td>2</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>0</td>\n",
              "      <td>3</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>0</td>\n",
              "      <td>4</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>0</td>\n",
              "      <td>5</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-1ff270ef-f23b-4ff0-aff0-61786d9d4eeb')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-1ff270ef-f23b-4ff0-aff0-61786d9d4eeb button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-1ff270ef-f23b-4ff0-aff0-61786d9d4eeb');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-54260500-6469-4e75-a1c8-1241a1fc89a1\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-54260500-6469-4e75-a1c8-1241a1fc89a1')\"\n",
              "            title=\"Suggest charts\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-54260500-6469-4e75-a1c8-1241a1fc89a1 button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "    </div>\n",
              "  </div>\n"
            ]
          },
          "metadata": {},
          "execution_count": 10
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "fg = graphistry.edges(df, 's', 'd').materialize_nodes()\n",
        "print(fg._nodes.shape, fg._edges.shape)\n",
        "fg._nodes.head(5)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 224
        },
        "id": "jEma7hvvkzkN",
        "outputId": "dbf21342-6b80-429c-bd3f-b1494c6854c7"
      },
      "execution_count": 11,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "(4039, 1) (88234, 2)\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "   id\n",
              "0   0\n",
              "1   1\n",
              "2   2\n",
              "3   3\n",
              "4   4"
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-6855e11a-489c-4631-b518-effafab7d91f\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>id</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>2</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>3</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>4</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-6855e11a-489c-4631-b518-effafab7d91f')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-6855e11a-489c-4631-b518-effafab7d91f button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-6855e11a-489c-4631-b518-effafab7d91f');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-e67d6c07-7215-434e-a899-7b42d2724764\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-e67d6c07-7215-434e-a899-7b42d2724764')\"\n",
              "            title=\"Suggest charts\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-e67d6c07-7215-434e-a899-7b42d2724764 button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "    </div>\n",
              "  </div>\n"
            ]
          },
          "metadata": {},
          "execution_count": 11
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "%%time\n",
        "for i in range(100):\n",
        "  fg2 = fg.chain([n({'id': 0}), e_forward(hops=2)])"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "5lEdCBw9lzd7",
        "outputId": "ed7451e0-401e-4edc-c8de-79c5afd0c95b"
      },
      "execution_count": 12,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "CPU times: user 13.6 s, sys: 2.08 s, total: 15.7 s\n",
            "Wall time: 18 s\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "%%time\n",
        "fg_gdf = fg.nodes(lambda g: cudf.DataFrame(g._nodes)).edges(lambda g: cudf.DataFrame(g._edges))\n",
        "for i in range(100):\n",
        "  fg2 = fg_gdf.chain([n({'id': 0}), e_forward(hops=2)])\n",
        "print(fg._nodes.shape, fg._edges.shape)\n",
        "print(fg2._nodes.shape, fg2._edges.shape)\n",
        "del fg_gdf\n",
        "del fg2"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "JFKIBa8mJCvJ",
        "outputId": "c22022f0-b33d-483a-db64-29992c5161e8"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "(4039, 1) (88234, 2)\n",
            "(1519, 1) (4060, 2)\n",
            "CPU times: user 11.8 s, sys: 28.1 ms, total: 11.8 s\n",
            "Wall time: 11.9 s\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "%%time\n",
        "for i in range(50):\n",
        "  fg2 = fg.chain([n({'id': 0}), e_forward(hops=5)])\n",
        "print(fg._nodes.shape, fg._edges.shape)\n",
        "print(fg2._nodes.shape, fg2._edges.shape)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "-KBGLexek5tS",
        "outputId": "2f462e6c-578a-4fa1-ec29-91bae753f4c5"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "(4039, 1) (88234, 2)\n",
            "(3829, 1) (86074, 2)\n",
            "CPU times: user 15.4 s, sys: 808 ms, total: 16.2 s\n",
            "Wall time: 16.2 s\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "%%time\n",
        "fg_gdf = fg.nodes(lambda g: cudf.DataFrame(g._nodes)).edges(lambda g: cudf.DataFrame(g._edges))\n",
        "for i in range(50):\n",
        "  fg2 = fg_gdf.chain([n({'id': 0}), e_forward(hops=5)])\n",
        "print(fg._nodes.shape, fg._edges.shape)\n",
        "print(fg2._nodes.shape, fg2._edges.shape)\n",
        "del fg_gdf\n",
        "del fg2"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "CVpcbhpdHFEF",
        "outputId": "aba04ee1-781e-4226-b593-b42415a55fc4"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "(4039, 1) (88234, 2)\n",
            "(3829, 1) (86074, 2)\n",
            "CPU times: user 9.82 s, sys: 133 ms, total: 9.95 s\n",
            "Wall time: 10.1 s\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "%%time\n",
        "for i in range(100):\n",
        "  fg2 = fg.chain([e_forward(source_node_match={'id': 0}, hops=5)])"
      ],
      "metadata": {
        "id": "1cFIyJF9pLjE",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "107329af-8e4b-428c-8b03-77ed00bdf5bf"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "CPU times: user 11.8 s, sys: 377 ms, total: 12.1 s\n",
            "Wall time: 13.1 s\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "%%time\n",
        "fg_gdf = fg.nodes(lambda g: cudf.DataFrame(g._nodes)).edges(lambda g: cudf.DataFrame(g._edges))\n",
        "for i in range(100):\n",
        "  fg2 = fg_gdf.chain([e_forward(source_node_match={'id': 0}, hops=5)])\n",
        "print(fg._nodes.shape, fg._edges.shape)\n",
        "print(fg2._nodes.shape, fg2._edges.shape)\n",
        "del fg_gdf\n",
        "del fg2\n",
        "\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "M5uRiD6uJVNW",
        "outputId": "5e938a19-2992-4280-80c2-784382d40113"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "(4039, 1) (88234, 2)\n",
            "(348, 1) (347, 2)\n",
            "CPU times: user 14.1 s, sys: 48.5 ms, total: 14.2 s\n",
            "Wall time: 14.2 s\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "%%time\n",
        "start_nodes = pd.DataFrame({fg._node: [0]})\n",
        "for i in range(100):\n",
        "  fg2 = fg.hop(\n",
        "      nodes=start_nodes,\n",
        "      direction='forward',\n",
        "      hops=2)\n",
        "print(fg2._nodes.shape, fg2._edges.shape)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Y9vgzfT69x41",
        "outputId": "6882c1ce-0df8-4087-dda4-0a105a8617e1"
      },
      "execution_count": 17,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "(1519, 1) (4060, 2)\n",
            "CPU times: user 4.5 s, sys: 1.35 s, total: 5.85 s\n",
            "Wall time: 6.09 s\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "%%time\n",
        "start_nodes = cudf.DataFrame({fg._node: [0]})\n",
        "fg_gdf = fg.nodes(cudf.from_pandas(fg._nodes)).edges(cudf.from_pandas(fg._edges))\n",
        "for i in range(100):\n",
        "  fg2 = fg_gdf.hop(\n",
        "      nodes=start_nodes,\n",
        "      direction='forward',\n",
        "      hops=2)\n",
        "print(fg2._nodes.shape, fg2._edges.shape)\n",
        "del start_nodes\n",
        "del fg_gdf\n",
        "del fg2"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "c7ybJqjc-T31",
        "outputId": "37ccc1fb-6460-4193-8aa7-22837ff06d0a"
      },
      "execution_count": 18,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "(1519, 1) (4060, 2)\n",
            "CPU times: user 2.58 s, sys: 6.75 ms, total: 2.59 s\n",
            "Wall time: 2.58 s\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "%%time\n",
        "start_nodes = pd.DataFrame({fg._node: [0]})\n",
        "for i in range(100):\n",
        "  fg2 = fg.hop(\n",
        "      nodes=start_nodes,\n",
        "      direction='forward',\n",
        "      hops=5)\n",
        "print(fg2._nodes.shape, fg2._edges.shape)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Dy7a4zDZ-7_G",
        "outputId": "077b5d9c-c9ae-411a-8228-3c026b07a910"
      },
      "execution_count": 19,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "(3829, 1) (86074, 2)\n",
            "CPU times: user 13.2 s, sys: 2 s, total: 15.2 s\n",
            "Wall time: 18.3 s\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "%%time\n",
        "start_nodes = cudf.DataFrame({fg._node: [0]})\n",
        "fg_gdf = fg.nodes(cudf.from_pandas(fg._nodes)).edges(cudf.from_pandas(fg._edges))\n",
        "for i in range(100):\n",
        "  fg2 = fg_gdf.hop(\n",
        "      nodes=start_nodes,\n",
        "      direction='forward',\n",
        "      hops=5)\n",
        "print(fg2._nodes.shape, fg2._edges.shape)\n",
        "del start_nodes\n",
        "del fg_gdf\n",
        "del fg2"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "N5aUtF1a--ML",
        "outputId": "0c2b67b8-fac6-45b3-dfbe-8002b5506e91"
      },
      "execution_count": 20,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "(3829, 1) (86074, 2)\n",
            "CPU times: user 5.72 s, sys: 159 ms, total: 5.88 s\n",
            "Wall time: 5.86 s\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Twitter\n",
        "\n",
        "- edges: 2420766\n",
        "- nodes: 81306"
      ],
      "metadata": {
        "id": "KrJKjXy2KLos"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "! wget 'https://snap.stanford.edu/data/twitter_combined.txt.gz'\n",
        "#! curl -L 'https://snap.stanford.edu/data/twitter_combined.txt.gz' -o twitter_combined.txt.gz"
      ],
      "metadata": {
        "id": "fO2qasGqpubr",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "d41a110e-9f7c-4710-9ce3-3f4906ab02ae"
      },
      "execution_count": 21,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "--2023-12-25 21:58:27--  https://snap.stanford.edu/data/twitter_combined.txt.gz\n",
            "Resolving snap.stanford.edu (snap.stanford.edu)... 171.64.75.80\n",
            "Connecting to snap.stanford.edu (snap.stanford.edu)|171.64.75.80|:443... connected.\n",
            "HTTP request sent, awaiting response... 200 OK\n",
            "Length: 10621918 (10M) [application/x-gzip]\n",
            "Saving to: ‘twitter_combined.txt.gz’\n",
            "\n",
            "twitter_combined.tx 100%[===================>]  10.13M  3.00MB/s    in 4.0s    \n",
            "\n",
            "2023-12-25 21:58:32 (2.52 MB/s) - ‘twitter_combined.txt.gz’ saved [10621918/10621918]\n",
            "\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "! gunzip twitter_combined.txt.gz"
      ],
      "metadata": {
        "id": "fn7zeA3SGlEo"
      },
      "execution_count": 22,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "! head -n 5 twitter_combined.txt"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "68TAZkhLGz9g",
        "outputId": "8ba7c23d-267f-4b59-d6c6-b3f66caec9cf"
      },
      "execution_count": 24,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "214328887 34428380\n",
            "17116707 28465635\n",
            "380580781 18996905\n",
            "221036078 153460275\n",
            "107830991 17868918\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "%%time\n",
        "te_df = pd.read_csv('twitter_combined.txt', sep=' ', names=['s', 'd'])\n",
        "te_df.shape"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "QU2wNeGXG2GC",
        "outputId": "349ac9c0-6f6c-4ce6-fec0-8bae75fca635"
      },
      "execution_count": 25,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "CPU times: user 474 ms, sys: 61.9 ms, total: 536 ms\n",
            "Wall time: 534 ms\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(2420766, 2)"
            ]
          },
          "metadata": {},
          "execution_count": 25
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "import graphistry"
      ],
      "metadata": {
        "id": "EK5gQH2iG5UU"
      },
      "execution_count": 26,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "%%time\n",
        "g = graphistry.edges(te_df, 's', 'd').materialize_nodes()\n",
        "g._nodes.shape"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ZtIW-eFGG_R4",
        "outputId": "0686e9b3-b684-4b93-da03-289244394338"
      },
      "execution_count": 27,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "CPU times: user 86.4 ms, sys: 106 ms, total: 193 ms\n",
            "Wall time: 191 ms\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(81306, 1)"
            ]
          },
          "metadata": {},
          "execution_count": 27
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "%%time\n",
        "for i in range(10):\n",
        "  g2 = g.chain([n({'id': 17116707}), e_forward(hops=1)])\n",
        "g2._nodes.shape, g2._edges.shape"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "yUaRfw4FHGMb",
        "outputId": "3945cc5a-c36c-451b-ac95-8af992a3546f"
      },
      "execution_count": 29,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "CPU times: user 11.8 s, sys: 8.4 s, total: 20.2 s\n",
            "Wall time: 23 s\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "((140, 1), (615, 2))"
            ]
          },
          "metadata": {},
          "execution_count": 29
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "%%time\n",
        "g_gdf = g.nodes(lambda g: cudf.DataFrame(g._nodes)).edges(lambda g: cudf.DataFrame(g._edges))\n",
        "for i in range(10):\n",
        "  out = g_gdf.chain([n({'id': 17116707}), e_forward(hops=1)])._nodes\n",
        "print(out.shape)\n",
        "del g_gdf\n",
        "del out"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "5hM4NBu2_eks",
        "outputId": "54505262-4871-44ee-e5e4-ad7ab32c13c2"
      },
      "execution_count": 30,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "(140, 1)\n",
            "CPU times: user 1.33 s, sys: 46.6 ms, total: 1.38 s\n",
            "Wall time: 1.63 s\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "%%time\n",
        "for i in range(10):\n",
        "  out = g.chain([n({'id': 17116707}), e_forward(hops=2)])\n",
        "print(out._nodes.shape, out._edges.shape)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "m2-MxD5lHX6u",
        "outputId": "e89b9d4b-6c04-45c7-9e7f-cbdbbe0a4730"
      },
      "execution_count": 31,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "(2345, 1) (68536, 2)\n",
            "CPU times: user 13.3 s, sys: 8.05 s, total: 21.4 s\n",
            "Wall time: 21.6 s\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "%%time\n",
        "g_gdf = g.nodes(lambda g: cudf.DataFrame(g._nodes)).edges(lambda g: cudf.DataFrame(g._edges))\n",
        "for i in range(10):\n",
        "  out = g_gdf.chain([n({'id': 17116707}), e_forward(hops=2)])._nodes\n",
        "print(out.shape)\n",
        "del g_gdf\n",
        "del out"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "7EQSRbIqLaGw",
        "outputId": "60c00a03-9e7b-46b5-fce3-f4f567a09430"
      },
      "execution_count": 36,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "(2345, 1)\n",
            "CPU times: user 1.67 s, sys: 55.8 ms, total: 1.72 s\n",
            "Wall time: 1.75 s\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "%%time\n",
        "for i in range(10):\n",
        "  out = g.chain([n({'id': 17116707}), e_forward(hops=8)])\n",
        "print(out._nodes.shape, out._edges.shape)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "hh6WnjI3ITpB",
        "outputId": "33138efe-a581-49ed-b2b4-247f8e9bdc09"
      },
      "execution_count": 37,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "(81304, 1) (2417796, 2)\n",
            "CPU times: user 1min 56s, sys: 17.1 s, total: 2min 13s\n",
            "Wall time: 2min 22s\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "%%time\n",
        "g_gdf = g.nodes(lambda g: cudf.DataFrame(g._nodes)).edges(lambda g: cudf.DataFrame(g._edges))\n",
        "for i in range(10):\n",
        "  out = g_gdf.chain([n({'id': 17116707}), e_forward(hops=8)])._nodes\n",
        "print(out.shape)\n",
        "del g_gdf\n",
        "del out"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "7jFFVUenM87j",
        "outputId": "2cceb720-9de3-488e-8b74-b820fd06e6c1"
      },
      "execution_count": 38,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "(81304, 1)\n",
            "CPU times: user 5.3 s, sys: 1.48 s, total: 6.78 s\n",
            "Wall time: 7.89 s\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "%%time\n",
        "start_nodes = pd.DataFrame({g._node: [17116707]})\n",
        "for i in range(10):\n",
        "  g2 = g.hop(\n",
        "      nodes=start_nodes,\n",
        "      direction='forward',\n",
        "      hops=1)\n",
        "print(g2._nodes.shape, g2._edges.shape)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "_5LD0bZB_lU4",
        "outputId": "bc31bd03-e79f-46d2-ea8f-3b01d9ef39a2"
      },
      "execution_count": 39,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "(0, 1) (0, 2)\n",
            "CPU times: user 2.58 s, sys: 1.59 s, total: 4.17 s\n",
            "Wall time: 6.02 s\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "%%time\n",
        "start_nodes = cudf.DataFrame({g._node: [17116707]})\n",
        "g_gdf = g.nodes(cudf.from_pandas(g._nodes)).edges(cudf.from_pandas(g._edges))\n",
        "for i in range(10):\n",
        "  g2 = g_gdf.hop(\n",
        "      nodes=start_nodes,\n",
        "      direction='forward',\n",
        "      hops=5)\n",
        "print(g2._nodes.shape, g2._edges.shape)\n",
        "del start_nodes\n",
        "del g_gdf\n",
        "del g2"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "M_rHjqtvACQw",
        "outputId": "8d3e308e-b1e2-452b-f402-573be0dd5b58"
      },
      "execution_count": 44,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "(61827, 1) (1473599, 2)\n",
            "CPU times: user 822 ms, sys: 179 ms, total: 1 s\n",
            "Wall time: 997 ms\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "%%time\n",
        "start_nodes = pd.DataFrame({g._node: [17116707]})\n",
        "for i in range(10):\n",
        "  g2 = g.hop(\n",
        "      nodes=start_nodes,\n",
        "      direction='forward',\n",
        "      hops=2)\n",
        "print(g2._nodes.shape, g2._edges.shape)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "0zEIucaCAbj_",
        "outputId": "83e64b0f-2b3a-4e4b-d189-3e6a8ef78f53"
      },
      "execution_count": 40,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "(2345, 1) (68536, 2)\n",
            "CPU times: user 8.93 s, sys: 5.92 s, total: 14.9 s\n",
            "Wall time: 15.8 s\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "%%time\n",
        "start_nodes = cudf.DataFrame({g._node: [17116707]})\n",
        "g_gdf = g.nodes(cudf.from_pandas(g._nodes)).edges(cudf.from_pandas(g._edges))\n",
        "for i in range(10):\n",
        "  g2 = g_gdf.hop(\n",
        "      nodes=start_nodes,\n",
        "      direction='forward',\n",
        "      hops=2)\n",
        "print(g2._nodes.shape, g2._edges.shape)\n",
        "del start_nodes\n",
        "del g_gdf\n",
        "del g2"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "LKJh5gRtAdIj",
        "outputId": "e3c7883d-74c0-4d55-b238-88457296c6bc"
      },
      "execution_count": 41,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "(2345, 1) (68536, 2)\n",
            "CPU times: user 374 ms, sys: 6.92 ms, total: 381 ms\n",
            "Wall time: 379 ms\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "%%time\n",
        "start_nodes = pd.DataFrame({g._node: [17116707]})\n",
        "for i in range(10):\n",
        "  g2 = g.hop(\n",
        "      nodes=start_nodes,\n",
        "      direction='forward',\n",
        "      hops=8)\n",
        "print(g2._nodes.shape, g2._edges.shape)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "JZwxdofNAfmb",
        "outputId": "2731be4c-75d9-47f4-8602-4f2d6cb2ddac"
      },
      "execution_count": 42,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "(81304, 1) (2417796, 2)\n",
            "CPU times: user 38.8 s, sys: 8.7 s, total: 47.5 s\n",
            "Wall time: 48.2 s\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "%%time\n",
        "start_nodes = cudf.DataFrame({g._node: [17116707]})\n",
        "g_gdf = g.nodes(cudf.from_pandas(g._nodes)).edges(cudf.from_pandas(g._edges))\n",
        "for i in range(10):\n",
        "  g2 = g_gdf.hop(\n",
        "      nodes=start_nodes,\n",
        "      direction='forward',\n",
        "      hops=8)\n",
        "print(g2._nodes.shape, g2._edges.shape)\n",
        "del start_nodes\n",
        "del g_gdf\n",
        "del g2"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "9o_og8bSAhe3",
        "outputId": "dd3e4f8f-f426-4705-98c4-60f1912ba28a"
      },
      "execution_count": 43,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "(81304, 1) (2417796, 2)\n",
            "CPU times: user 1.8 s, sys: 506 ms, total: 2.3 s\n",
            "Wall time: 2.3 s\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### GPlus\n",
        "\n",
        "- edges: 30494866\n",
        "- nodes: 107614"
      ],
      "metadata": {
        "id": "9dZzAAVONCD2"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "! wget https://snap.stanford.edu/data/gplus_combined.txt.gz"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "-nhWGNekKpcZ",
        "outputId": "e2175290-337c-4faa-e5d8-4bc401583326"
      },
      "execution_count": 4,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "--2023-12-26 18:36:29--  https://snap.stanford.edu/data/gplus_combined.txt.gz\n",
            "Resolving snap.stanford.edu (snap.stanford.edu)... 171.64.75.80\n",
            "Connecting to snap.stanford.edu (snap.stanford.edu)|171.64.75.80|:443... connected.\n",
            "HTTP request sent, awaiting response... 200 OK\n",
            "Length: 398930514 (380M) [application/x-gzip]\n",
            "Saving to: ‘gplus_combined.txt.gz’\n",
            "\n",
            "gplus_combined.txt. 100%[===================>] 380.45M  34.7MB/s    in 9.6s    \n",
            "\n",
            "2023-12-26 18:36:39 (39.7 MB/s) - ‘gplus_combined.txt.gz’ saved [398930514/398930514]\n",
            "\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "! gunzip gplus_combined.txt.gz"
      ],
      "metadata": {
        "id": "g5wgA_c2KqwJ"
      },
      "execution_count": 5,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "%%time\n",
        "ge_df = pd.read_csv('gplus_combined.txt', sep=' ', names=['s', 'd'])\n",
        "print(ge_df.shape)\n",
        "ge_df.head(5)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 258
        },
        "id": "52hgDbr0Kti6",
        "outputId": "217203fc-7095-4784-c4c4-d46ee9c78808"
      },
      "execution_count": 6,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "(30494866, 2)\n",
            "CPU times: user 16 s, sys: 1.45 s, total: 17.5 s\n",
            "Wall time: 22.5 s\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "                       s                      d\n",
              "0  116374117927631468606  101765416973555767821\n",
              "1  112188647432305746617  107727150903234299458\n",
              "2  116719211656774388392  100432456209427807893\n",
              "3  117421021456205115327  101096322838605097368\n",
              "4  116407635616074189669  113556266482860931616"
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-0d42e599-73d6-4059-91f3-83f525b5891f\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>s</th>\n",
              "      <th>d</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>116374117927631468606</td>\n",
              "      <td>101765416973555767821</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>112188647432305746617</td>\n",
              "      <td>107727150903234299458</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>116719211656774388392</td>\n",
              "      <td>100432456209427807893</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>117421021456205115327</td>\n",
              "      <td>101096322838605097368</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>116407635616074189669</td>\n",
              "      <td>113556266482860931616</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-0d42e599-73d6-4059-91f3-83f525b5891f')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-0d42e599-73d6-4059-91f3-83f525b5891f button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-0d42e599-73d6-4059-91f3-83f525b5891f');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-623952d7-77ea-4e29-b39b-c80edd4a7219\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-623952d7-77ea-4e29-b39b-c80edd4a7219')\"\n",
              "            title=\"Suggest charts\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-623952d7-77ea-4e29-b39b-c80edd4a7219 button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "    </div>\n",
              "  </div>\n"
            ]
          },
          "metadata": {},
          "execution_count": 6
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "%%time\n",
        "gg = graphistry.edges(ge_df, 's', 'd').materialize_nodes()\n",
        "gg = graphistry.edges(ge_df, 's', 'd').nodes(gg._nodes, 'id')\n",
        "print(gg._edges.shape, gg._nodes.shape)\n",
        "gg._nodes.head(5)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 258
        },
        "id": "w5YkN-nLK6UV",
        "outputId": "dc98380d-54c2-4b36-c56e-5e8401c4ffa4"
      },
      "execution_count": 7,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "(30494866, 2) (107614, 1)\n",
            "CPU times: user 4.49 s, sys: 1.25 s, total: 5.74 s\n",
            "Wall time: 5.97 s\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "                      id\n",
              "0  116374117927631468606\n",
              "1  112188647432305746617\n",
              "2  116719211656774388392\n",
              "3  117421021456205115327\n",
              "4  116407635616074189669"
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-c1d2fbc6-366f-4484-8c81-15eaec2f8e5e\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>id</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>116374117927631468606</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>112188647432305746617</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>116719211656774388392</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>117421021456205115327</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>116407635616074189669</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-c1d2fbc6-366f-4484-8c81-15eaec2f8e5e')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-c1d2fbc6-366f-4484-8c81-15eaec2f8e5e button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-c1d2fbc6-366f-4484-8c81-15eaec2f8e5e');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-df8cfeb9-5932-49da-82f5-2c8dc224b580\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-df8cfeb9-5932-49da-82f5-2c8dc224b580')\"\n",
              "            title=\"Suggest charts\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-df8cfeb9-5932-49da-82f5-2c8dc224b580 button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "    </div>\n",
              "  </div>\n"
            ]
          },
          "metadata": {},
          "execution_count": 7
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "%%time\n",
        "gg.chain([ n({'id': '116374117927631468606'})])._nodes"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 115
        },
        "id": "NKtz54uELX-8",
        "outputId": "5d8f3eef-893d-47cc-e7a9-c5cbfec8270c"
      },
      "execution_count": 49,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "CPU times: user 534 ms, sys: 598 ms, total: 1.13 s\n",
            "Wall time: 1.65 s\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "                      id\n",
              "0  116374117927631468606"
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-9c79ab83-1d00-433a-ac53-258bafc7faac\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>id</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>116374117927631468606</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-9c79ab83-1d00-433a-ac53-258bafc7faac')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-9c79ab83-1d00-433a-ac53-258bafc7faac button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-9c79ab83-1d00-433a-ac53-258bafc7faac');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "    </div>\n",
              "  </div>\n"
            ]
          },
          "metadata": {},
          "execution_count": 49
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "%%time\n",
        "out = gg.chain([ n({'id': '116374117927631468606'}), e_forward(hops=1)])._nodes\n",
        "out.shape"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "iNWdi00VLmZG",
        "outputId": "ecfb56a6-c564-4bf6-f43f-2c95a103f4be"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "CPU times: user 27.5 s, sys: 11.1 s, total: 38.5 s\n",
            "Wall time: 39.5 s\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(1473, 1)"
            ]
          },
          "metadata": {},
          "execution_count": 75
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "%%time\n",
        "gg_gdf = gg.nodes(lambda g: cudf.DataFrame(g._nodes)).edges(lambda g: cudf.DataFrame(g._edges))\n",
        "out = gg_gdf.chain([ n({'id': '116374117927631468606'}), e_forward(hops=1)])\n",
        "print(out._nodes.shape, out._edges.shape)\n",
        "del gg_gdf\n",
        "del out"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Q6p3h6uCOABh",
        "outputId": "817fc80f-ef5d-4070-eb48-a12344be709c"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "(1473, 1) (13375, 2)\n",
            "CPU times: user 4.57 s, sys: 2.11 s, total: 6.68 s\n",
            "Wall time: 7.63 s\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "%%time\n",
        "out = gg.chain([ n({'id': '116374117927631468606'}), e_forward(hops=2)])._nodes\n",
        "out.shape"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "6UdCcMdqLw-P",
        "outputId": "70742c79-b22b-4db2-c548-cb1e25d572eb"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "CPU times: user 45.8 s, sys: 17 s, total: 1min 2s\n",
            "Wall time: 1min 5s\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(44073, 1)"
            ]
          },
          "metadata": {},
          "execution_count": 77
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "%%time\n",
        "gg_gdf = gg.nodes(lambda g: cudf.DataFrame(g._nodes)).edges(lambda g: cudf.DataFrame(g._edges))\n",
        "out = gg_gdf.chain([ n({'id': '116374117927631468606'}), e_forward(hops=2)])\n",
        "print(out._nodes.shape, out._edges.shape)\n",
        "del gg_gdf\n",
        "del out"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "QElqatDyNYCS",
        "outputId": "0e15bd3e-d2d9-4965-df7d-c8856d036680"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "(44073, 1) (2069325, 2)\n",
            "CPU times: user 4.97 s, sys: 2.36 s, total: 7.34 s\n",
            "Wall time: 10.6 s\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "%%time\n",
        "out = gg.chain([ n({'id': '116374117927631468606'}), e_forward(hops=3)])._nodes\n",
        "out.shape"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "3HJOItZ4MQMG",
        "outputId": "f5be7bb4-7f09-4f80-c549-e703e99f5067"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "CPU times: user 3min 45s, sys: 1min 5s, total: 4min 50s\n",
            "Wall time: 4min 52s\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(102414, 1)"
            ]
          },
          "metadata": {},
          "execution_count": 79
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "%%time\n",
        "gg_gdf = gg.nodes(lambda g: cudf.DataFrame(g._nodes)).edges(lambda g: cudf.DataFrame(g._edges))\n",
        "out = gg_gdf.chain([ n({'id': '116374117927631468606'}), e_forward(hops=3)])\n",
        "print(out._nodes.shape, out._edges.shape)\n",
        "del gg_gdf\n",
        "del out"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "G32t_xthOUle",
        "outputId": "7721741f-9c86-41aa-eb0b-2c8f0db2ed54"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "(102414, 1) (24851333, 2)\n",
            "CPU times: user 6.95 s, sys: 2.63 s, total: 9.57 s\n",
            "Wall time: 9.84 s\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "%%time\n",
        "out = gg.chain([ n({'id': '116374117927631468606'}), e_forward(hops=4)])\n",
        "print(out._nodes.shape, out._edges.shape)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "bXy2yyJsMsEG",
        "outputId": "911f2680-067c-44f2-9ba2-7f27d3c9bc6b"
      },
      "execution_count": 8,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "(105479, 1) (30450354, 2)\n",
            "CPU times: user 4min 36s, sys: 1min 25s, total: 6min 2s\n",
            "Wall time: 6min 4s\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "%%time\n",
        "gg_gdf = gg.nodes(lambda g: cudf.DataFrame(g._nodes)).edges(lambda g: cudf.DataFrame(g._edges))\n",
        "out = gg_gdf.chain([ n({'id': '116374117927631468606'}), e_forward(hops=4)])\n",
        "print(out._nodes.shape, out._edges.shape)\n",
        "del gg_gdf\n",
        "del out"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Vt8hhjWDP_W_",
        "outputId": "824ae644-e1cf-4239-bda9-84aecde52ad8"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "(105479, 1) (30450354, 2)\n",
            "CPU times: user 7.44 s, sys: 2.45 s, total: 9.88 s\n",
            "Wall time: 9.9 s\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "%%time\n",
        "out = gg.chain([ n({'id': '116374117927631468606'}), e_forward(hops=5)])\n",
        "print(out._nodes.shape, out._edges.shape)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "_z4KpNZaOH8t",
        "outputId": "2417f78b-e1b7-452d-8e26-7df259620c88"
      },
      "execution_count": 9,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "(105604, 1) (30468335, 2)\n",
            "CPU times: user 5min 36s, sys: 1min 39s, total: 7min 16s\n",
            "Wall time: 7min 15s\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "%%time\n",
        "gg_gdf = gg.nodes(lambda g: cudf.DataFrame(g._nodes)).edges(lambda g: cudf.DataFrame(g._edges))\n",
        "out = gg_gdf.chain([ n({'id': '116374117927631468606'}), e_forward(hops=5)])\n",
        "print(out._nodes.shape, out._edges.shape)\n",
        "del gg_gdf\n",
        "del out"
      ],
      "metadata": {
        "id": "spUBH9EHSz2O",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "22340ce3-e8d4-4a72-b485-9839c667b965"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "(105604, 1) (30468335, 2)\n",
            "CPU times: user 8.82 s, sys: 2.71 s, total: 11.5 s\n",
            "Wall time: 11.9 s\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "%%time\n",
        "start_nodes = pd.DataFrame({gg._node: ['116374117927631468606']})\n",
        "for i in range(1):\n",
        "  g2 = gg.hop(\n",
        "      nodes=start_nodes,\n",
        "      direction='forward',\n",
        "      hops=1)\n",
        "print(g2._nodes.shape, g2._edges.shape)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "vCsdmc62A7OM",
        "outputId": "adc05d29-c628-49ed-cd6d-8921c6dcd206"
      },
      "execution_count": 50,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "(1473, 1) (13375, 2)\n",
            "CPU times: user 19.9 s, sys: 9.36 s, total: 29.2 s\n",
            "Wall time: 41.8 s\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "%%time\n",
        "start_nodes = cudf.DataFrame({gg._node: ['116374117927631468606']})\n",
        "gg_gdf = gg.nodes(cudf.from_pandas(gg._nodes)).edges(cudf.from_pandas(gg._edges))\n",
        "for i in range(1):\n",
        "  g2 = gg_gdf.hop(\n",
        "      nodes=start_nodes,\n",
        "      direction='forward',\n",
        "      hops=1)\n",
        "print(g2._nodes.shape, g2._edges.shape)\n",
        "del start_nodes\n",
        "del gg_gdf\n",
        "del g2"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "J3kV8NBYBQdW",
        "outputId": "76073248-43e1-4c3c-c004-67324cc1d312"
      },
      "execution_count": 52,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "(1473, 1) (13375, 2)\n",
            "CPU times: user 3.71 s, sys: 2.09 s, total: 5.8 s\n",
            "Wall time: 6.05 s\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "%%time\n",
        "start_nodes = pd.DataFrame({gg._node: ['116374117927631468606']})\n",
        "for i in range(1):\n",
        "  g2 = gg.hop(\n",
        "      nodes=start_nodes,\n",
        "      direction='forward',\n",
        "      hops=2)\n",
        "print(g2._nodes.shape, g2._edges.shape)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ONv1RQeWBeeK",
        "outputId": "58d57fa4-be72-45bc-abfa-5de9d1102f55"
      },
      "execution_count": 53,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "(44073, 1) (2069325, 2)\n",
            "CPU times: user 27.8 s, sys: 13.2 s, total: 41 s\n",
            "Wall time: 43.9 s\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "%%time\n",
        "start_nodes = cudf.DataFrame({gg._node: ['116374117927631468606']})\n",
        "gg_gdf = gg.nodes(cudf.from_pandas(gg._nodes)).edges(cudf.from_pandas(gg._edges))\n",
        "for i in range(1):\n",
        "  g2 = gg_gdf.hop(\n",
        "      nodes=start_nodes,\n",
        "      direction='forward',\n",
        "      hops=2)\n",
        "print(g2._nodes.shape, g2._edges.shape)\n",
        "del start_nodes\n",
        "del gg_gdf\n",
        "del g2"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ke5SZZ01BgqR",
        "outputId": "4173fd28-a11b-4300-d28b-6fdb87e8e9f3"
      },
      "execution_count": 54,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "(44073, 1) (2069325, 2)\n",
            "CPU times: user 4.26 s, sys: 2.37 s, total: 6.63 s\n",
            "Wall time: 7.91 s\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "%%time\n",
        "start_nodes = pd.DataFrame({gg._node: ['116374117927631468606']})\n",
        "for i in range(1):\n",
        "  g2 = gg.hop(\n",
        "      nodes=start_nodes,\n",
        "      direction='forward',\n",
        "      hops=3)\n",
        "print(g2._nodes.shape, g2._edges.shape)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "U795pIBUBiZV",
        "outputId": "d499433c-cc0c-4bbf-c69f-36b5d55402d9"
      },
      "execution_count": 55,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "(102414, 1) (24851333, 2)\n",
            "CPU times: user 1min 3s, sys: 22.7 s, total: 1min 26s\n",
            "Wall time: 1min 35s\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "%%time\n",
        "start_nodes = cudf.DataFrame({gg._node: ['116374117927631468606']})\n",
        "gg_gdf = gg.nodes(cudf.from_pandas(gg._nodes)).edges(cudf.from_pandas(gg._edges))\n",
        "for i in range(1):\n",
        "  g2 = gg_gdf.hop(\n",
        "      nodes=start_nodes,\n",
        "      direction='forward',\n",
        "      hops=3)\n",
        "print(g2._nodes.shape, g2._edges.shape)\n",
        "del start_nodes\n",
        "del gg_gdf\n",
        "del g2"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "kIZYwSe1Bj2e",
        "outputId": "b7e1ed9f-47d1-412e-9593-ecc436ac1486"
      },
      "execution_count": 56,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "(102414, 1) (24851333, 2)\n",
            "CPU times: user 3.96 s, sys: 2.11 s, total: 6.07 s\n",
            "Wall time: 6.05 s\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "%%time\n",
        "start_nodes = pd.DataFrame({gg._node: ['116374117927631468606']})\n",
        "for i in range(1):\n",
        "  g2 = gg.hop(\n",
        "      nodes=start_nodes,\n",
        "      direction='forward',\n",
        "      hops=4)\n",
        "print(g2._nodes.shape, g2._edges.shape)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "YTI5sD6YBpYL",
        "outputId": "b37bf2df-07dc-404c-8a83-a83f28e38bf6"
      },
      "execution_count": 57,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "(105479, 1) (30450354, 2)\n",
            "CPU times: user 1min 34s, sys: 30.6 s, total: 2min 5s\n",
            "Wall time: 2min 5s\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "%%time\n",
        "start_nodes = cudf.DataFrame({gg._node: ['116374117927631468606']})\n",
        "gg_gdf = gg.nodes(cudf.from_pandas(gg._nodes)).edges(cudf.from_pandas(gg._edges))\n",
        "for i in range(1):\n",
        "  g2 = gg_gdf.hop(\n",
        "      nodes=start_nodes,\n",
        "      direction='forward',\n",
        "      hops=4)\n",
        "print(g2._nodes.shape, g2._edges.shape)\n",
        "del start_nodes\n",
        "del gg_gdf\n",
        "del g2"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "d5WBazICBrSz",
        "outputId": "ef95e893-3a0f-4d47-ede4-bd8a6faebf98"
      },
      "execution_count": 58,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "(105479, 1) (30450354, 2)\n",
            "CPU times: user 5.25 s, sys: 2.41 s, total: 7.67 s\n",
            "Wall time: 7.69 s\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "%%time\n",
        "start_nodes = pd.DataFrame({gg._node: ['116374117927631468606']})\n",
        "for i in range(1):\n",
        "  g2 = gg.hop(\n",
        "      nodes=start_nodes,\n",
        "      direction='forward',\n",
        "      hops=5)\n",
        "print(g2._nodes.shape, g2._edges.shape)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ozQlRPaFBtPD",
        "outputId": "4f1655c4-38fd-47f9-942d-836585e0d866"
      },
      "execution_count": 59,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "(105604, 1) (30468335, 2)\n",
            "CPU times: user 2min 16s, sys: 39.1 s, total: 2min 55s\n",
            "Wall time: 2min 58s\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "%%time\n",
        "start_nodes = cudf.DataFrame({gg._node: ['116374117927631468606']})\n",
        "gg_gdf = gg.nodes(cudf.from_pandas(gg._nodes)).edges(cudf.from_pandas(gg._edges))\n",
        "for i in range(1):\n",
        "  g2 = gg_gdf.hop(\n",
        "      nodes=start_nodes,\n",
        "      direction='forward',\n",
        "      hops=5)\n",
        "print(g2._nodes.shape, g2._edges.shape)\n",
        "del start_nodes\n",
        "del gg_gdf\n",
        "del g2"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "-ACkMG20B6HM",
        "outputId": "f26c03a9-9f25-4f93-c7d3-0e8676694040"
      },
      "execution_count": 60,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "(105604, 1) (30468335, 2)\n",
            "CPU times: user 5.79 s, sys: 2.51 s, total: 8.3 s\n",
            "Wall time: 8.29 s\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Orkut\n",
        "- 117M edges\n",
        "- 3M nodes"
      ],
      "metadata": {
        "id": "R03M_swxarKC"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "! wget https://snap.stanford.edu/data/bigdata/communities/com-orkut.ungraph.txt.gz"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "QoabYR2maxPo",
        "outputId": "2bb6275d-46bb-42da-ec05-d0e5a58b1f77"
      },
      "execution_count": 8,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "--2023-12-26 00:55:52--  https://snap.stanford.edu/data/bigdata/communities/com-orkut.ungraph.txt.gz\n",
            "Resolving snap.stanford.edu (snap.stanford.edu)... 171.64.75.80\n",
            "Connecting to snap.stanford.edu (snap.stanford.edu)|171.64.75.80|:443... connected.\n",
            "HTTP request sent, awaiting response... 200 OK\n",
            "Length: 447251958 (427M) [application/x-gzip]\n",
            "Saving to: ‘com-orkut.ungraph.txt.gz’\n",
            "\n",
            "com-orkut.ungraph.t 100%[===================>] 426.53M  45.1MB/s    in 9.7s    \n",
            "\n",
            "2023-12-26 00:56:02 (44.0 MB/s) - ‘com-orkut.ungraph.txt.gz’ saved [447251958/447251958]\n",
            "\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "! gunzip com-orkut.ungraph.txt.gz"
      ],
      "metadata": {
        "id": "BvvfFPKWbAVJ"
      },
      "execution_count": 9,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "! head -n 7 com-orkut.ungraph.txt"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "YsWwRoPqbPIb",
        "outputId": "2eb4f862-b4e1-42bf-ff5d-eec10b27cedc"
      },
      "execution_count": 10,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "# Undirected graph: ../../data/output/orkut.txt\n",
            "# Orkut\n",
            "# Nodes: 3072441 Edges: 117185083\n",
            "# FromNodeId\tToNodeId\n",
            "1\t2\n",
            "1\t3\n",
            "1\t4\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "import pandas as pd\n",
        "\n",
        "import graphistry\n",
        "\n",
        "from graphistry import (\n",
        "\n",
        "    # graph operators\n",
        "    n, e_undirected, e_forward, e_reverse,\n",
        "\n",
        "    # attribute predicates\n",
        "    is_in, ge, startswith, contains, match as match_re\n",
        ")\n",
        "\n",
        "import cudf\n",
        "\n",
        "#work around google colab shell encoding bugs\n",
        "import locale\n",
        "locale.getpreferredencoding = lambda: \"UTF-8\"\n",
        "\n",
        "cudf.__version__, graphistry.__version__"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "cbMC8r2ldjbW",
        "outputId": "82688d53-7d56-4563-d65e-7c5cd32ac14e"
      },
      "execution_count": 11,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "('23.12.01', '0.32.0+12.g72e778c')"
            ]
          },
          "metadata": {},
          "execution_count": 11
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "! nvidia-smi"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "TopFxAvnh_Cv",
        "outputId": "cc9d9dc9-e594-4190-fe84-3f1b6dce8a1a"
      },
      "execution_count": 12,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Tue Dec 26 00:56:27 2023       \n",
            "+---------------------------------------------------------------------------------------+\n",
            "| NVIDIA-SMI 535.104.05             Driver Version: 535.104.05   CUDA Version: 12.2     |\n",
            "|-----------------------------------------+----------------------+----------------------+\n",
            "| GPU  Name                 Persistence-M | Bus-Id        Disp.A | Volatile Uncorr. ECC |\n",
            "| Fan  Temp   Perf          Pwr:Usage/Cap |         Memory-Usage | GPU-Util  Compute M. |\n",
            "|                                         |                      |               MIG M. |\n",
            "|=========================================+======================+======================|\n",
            "|   0  Tesla T4                       Off | 00000000:00:04.0 Off |                    0 |\n",
            "| N/A   47C    P0              27W /  70W |    103MiB / 15360MiB |      0%      Default |\n",
            "|                                         |                      |                  N/A |\n",
            "+-----------------------------------------+----------------------+----------------------+\n",
            "                                                                                         \n",
            "+---------------------------------------------------------------------------------------+\n",
            "| Processes:                                                                            |\n",
            "|  GPU   GI   CI        PID   Type   Process name                            GPU Memory |\n",
            "|        ID   ID                                                             Usage      |\n",
            "|=======================================================================================|\n",
            "+---------------------------------------------------------------------------------------+\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "%%time\n",
        "co_df = cudf.read_csv('com-orkut.ungraph.txt', sep='\\t', names=['s', 'd'], skiprows=5).to_pandas()\n",
        "print(co_df.shape)\n",
        "print(co_df.head(5))\n",
        "print(co_df.dtypes)\n",
        "#del co_df"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Oczs87ITbJgw",
        "outputId": "ac203ddd-e684-4eb9-a586-f6a49fd1625d"
      },
      "execution_count": 13,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "(117185082, 2)\n",
            "   s  d\n",
            "0  1  3\n",
            "1  1  4\n",
            "2  1  5\n",
            "3  1  6\n",
            "4  1  7\n",
            "s    int64\n",
            "d    int64\n",
            "dtype: object\n",
            "CPU times: user 2.56 s, sys: 4.2 s, total: 6.76 s\n",
            "Wall time: 6.76 s\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "%%time\n",
        "co_g = graphistry.edges(cudf.DataFrame(co_df), 's', 'd').materialize_nodes(engine='cudf')\n",
        "co_g = co_g.nodes(lambda g: g._nodes.to_pandas()).edges(lambda g: g._edges.to_pandas())\n",
        "print(co_g._nodes.shape, co_g._edges.shape)\n",
        "co_g._nodes.head(5)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 258
        },
        "id": "gGSDjTtveFAT",
        "outputId": "e7b38f4f-dc07-4f35-9bab-9c80a80bbf0b"
      },
      "execution_count": 14,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "(3072441, 1) (117185082, 2)\n",
            "CPU times: user 1.96 s, sys: 2.95 s, total: 4.91 s\n",
            "Wall time: 4.92 s\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "   id\n",
              "0   1\n",
              "1   2\n",
              "2   3\n",
              "3   4\n",
              "4   5"
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-a4040d48-f85c-42f4-ab06-9114b9f80264\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>id</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>2</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>3</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>4</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>5</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-a4040d48-f85c-42f4-ab06-9114b9f80264')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-a4040d48-f85c-42f4-ab06-9114b9f80264 button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-a4040d48-f85c-42f4-ab06-9114b9f80264');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-39f20af4-a46e-49ab-8703-96d31e61ff88\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-39f20af4-a46e-49ab-8703-96d31e61ff88')\"\n",
              "            title=\"Suggest charts\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-39f20af4-a46e-49ab-8703-96d31e61ff88 button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "    </div>\n",
              "  </div>\n"
            ]
          },
          "metadata": {},
          "execution_count": 14
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "! nvidia-smi"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "V5qL8K7-dqIZ",
        "outputId": "e08319fc-74d3-4f33-df0f-f98950dc8c99"
      },
      "execution_count": 15,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Tue Dec 26 00:56:39 2023       \n",
            "+---------------------------------------------------------------------------------------+\n",
            "| NVIDIA-SMI 535.104.05             Driver Version: 535.104.05   CUDA Version: 12.2     |\n",
            "|-----------------------------------------+----------------------+----------------------+\n",
            "| GPU  Name                 Persistence-M | Bus-Id        Disp.A | Volatile Uncorr. ECC |\n",
            "| Fan  Temp   Perf          Pwr:Usage/Cap |         Memory-Usage | GPU-Util  Compute M. |\n",
            "|                                         |                      |               MIG M. |\n",
            "|=========================================+======================+======================|\n",
            "|   0  Tesla T4                       Off | 00000000:00:04.0 Off |                    0 |\n",
            "| N/A   49C    P0              27W /  70W |   2819MiB / 15360MiB |      0%      Default |\n",
            "|                                         |                      |                  N/A |\n",
            "+-----------------------------------------+----------------------+----------------------+\n",
            "                                                                                         \n",
            "+---------------------------------------------------------------------------------------+\n",
            "| Processes:                                                                            |\n",
            "|  GPU   GI   CI        PID   Type   Process name                            GPU Memory |\n",
            "|        ID   ID                                                             Usage      |\n",
            "|=======================================================================================|\n",
            "+---------------------------------------------------------------------------------------+\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "%%time\n",
        "# crashes\n",
        "if False:\n",
        "  out = co_g.chain([ n({'id': 1}), e_forward(hops=1)])._nodes\n",
        "  print(out.shape)\n",
        "  del out"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "hCbxZ8UmhRLp",
        "outputId": "519aed6c-733d-41f4-d462-e57f5e32b131"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "CPU times: user 4 µs, sys: 1 µs, total: 5 µs\n",
            "Wall time: 47.7 µs\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "%%time\n",
        "co_gdf = co_g.nodes(lambda g: cudf.DataFrame(g._nodes)).edges(lambda g: cudf.DataFrame(g._edges))\n",
        "! nvidia-smi\n",
        "for i in range(10):\n",
        "  out = co_gdf.chain([ n({'id': 1}), e_forward(hops=1)])\n",
        "! nvidia-smi\n",
        "print(out._nodes.shape, out._edges.shape)\n",
        "del co_gdf\n",
        "del out"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Q682scC_eC-S",
        "outputId": "7ff5f829-0de7-4a6c-a77d-e2857896a8a5"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Mon Dec 25 06:23:46 2023       \n",
            "+---------------------------------------------------------------------------------------+\n",
            "| NVIDIA-SMI 535.104.05             Driver Version: 535.104.05   CUDA Version: 12.2     |\n",
            "|-----------------------------------------+----------------------+----------------------+\n",
            "| GPU  Name                 Persistence-M | Bus-Id        Disp.A | Volatile Uncorr. ECC |\n",
            "| Fan  Temp   Perf          Pwr:Usage/Cap |         Memory-Usage | GPU-Util  Compute M. |\n",
            "|                                         |                      |               MIG M. |\n",
            "|=========================================+======================+======================|\n",
            "|   0  Tesla T4                       Off | 00000000:00:04.0 Off |                    0 |\n",
            "| N/A   63C    P0              30W /  70W |   1925MiB / 15360MiB |     35%      Default |\n",
            "|                                         |                      |                  N/A |\n",
            "+-----------------------------------------+----------------------+----------------------+\n",
            "                                                                                         \n",
            "+---------------------------------------------------------------------------------------+\n",
            "| Processes:                                                                            |\n",
            "|  GPU   GI   CI        PID   Type   Process name                            GPU Memory |\n",
            "|        ID   ID                                                             Usage      |\n",
            "|=======================================================================================|\n",
            "+---------------------------------------------------------------------------------------+\n",
            "Mon Dec 25 06:23:49 2023       \n",
            "+---------------------------------------------------------------------------------------+\n",
            "| NVIDIA-SMI 535.104.05             Driver Version: 535.104.05   CUDA Version: 12.2     |\n",
            "|-----------------------------------------+----------------------+----------------------+\n",
            "| GPU  Name                 Persistence-M | Bus-Id        Disp.A | Volatile Uncorr. ECC |\n",
            "| Fan  Temp   Perf          Pwr:Usage/Cap |         Memory-Usage | GPU-Util  Compute M. |\n",
            "|                                         |                      |               MIG M. |\n",
            "|=========================================+======================+======================|\n",
            "|   0  Tesla T4                       Off | 00000000:00:04.0 Off |                    0 |\n",
            "| N/A   66C    P0              72W /  70W |   2845MiB / 15360MiB |     84%      Default |\n",
            "|                                         |                      |                  N/A |\n",
            "+-----------------------------------------+----------------------+----------------------+\n",
            "                                                                                         \n",
            "+---------------------------------------------------------------------------------------+\n",
            "| Processes:                                                                            |\n",
            "|  GPU   GI   CI        PID   Type   Process name                            GPU Memory |\n",
            "|        ID   ID                                                             Usage      |\n",
            "|=======================================================================================|\n",
            "+---------------------------------------------------------------------------------------+\n",
            "(12, 1) (11, 2)\n",
            "CPU times: user 4.42 s, sys: 131 ms, total: 4.55 s\n",
            "Wall time: 4.42 s\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "%%time\n",
        "co_gdf = co_g.nodes(lambda g: cudf.DataFrame(g._nodes)).edges(lambda g: cudf.DataFrame(g._edges))\n",
        "! nvidia-smi\n",
        "for i in range(10):\n",
        "  out = co_gdf.chain([ n({'id': 1}), e_forward(hops=2)])\n",
        "! nvidia-smi\n",
        "print(out._nodes.shape, out._edges.shape)\n",
        "del co_gdf\n",
        "del out"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "i0AXhfqVbVsm",
        "outputId": "8271f469-a73f-48e3-e1a9-3077026ab8ec"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Mon Dec 25 06:24:52 2023       \n",
            "+---------------------------------------------------------------------------------------+\n",
            "| NVIDIA-SMI 535.104.05             Driver Version: 535.104.05   CUDA Version: 12.2     |\n",
            "|-----------------------------------------+----------------------+----------------------+\n",
            "| GPU  Name                 Persistence-M | Bus-Id        Disp.A | Volatile Uncorr. ECC |\n",
            "| Fan  Temp   Perf          Pwr:Usage/Cap |         Memory-Usage | GPU-Util  Compute M. |\n",
            "|                                         |                      |               MIG M. |\n",
            "|=========================================+======================+======================|\n",
            "|   0  Tesla T4                       Off | 00000000:00:04.0 Off |                    0 |\n",
            "| N/A   61C    P0              29W /  70W |   1925MiB / 15360MiB |     22%      Default |\n",
            "|                                         |                      |                  N/A |\n",
            "+-----------------------------------------+----------------------+----------------------+\n",
            "                                                                                         \n",
            "+---------------------------------------------------------------------------------------+\n",
            "| Processes:                                                                            |\n",
            "|  GPU   GI   CI        PID   Type   Process name                            GPU Memory |\n",
            "|        ID   ID                                                             Usage      |\n",
            "|=======================================================================================|\n",
            "+---------------------------------------------------------------------------------------+\n",
            "Mon Dec 25 06:24:58 2023       \n",
            "+---------------------------------------------------------------------------------------+\n",
            "| NVIDIA-SMI 535.104.05             Driver Version: 535.104.05   CUDA Version: 12.2     |\n",
            "|-----------------------------------------+----------------------+----------------------+\n",
            "| GPU  Name                 Persistence-M | Bus-Id        Disp.A | Volatile Uncorr. ECC |\n",
            "| Fan  Temp   Perf          Pwr:Usage/Cap |         Memory-Usage | GPU-Util  Compute M. |\n",
            "|                                         |                      |               MIG M. |\n",
            "|=========================================+======================+======================|\n",
            "|   0  Tesla T4                       Off | 00000000:00:04.0 Off |                    0 |\n",
            "| N/A   64C    P0              71W /  70W |   2845MiB / 15360MiB |     57%      Default |\n",
            "|                                         |                      |                  N/A |\n",
            "+-----------------------------------------+----------------------+----------------------+\n",
            "                                                                                         \n",
            "+---------------------------------------------------------------------------------------+\n",
            "| Processes:                                                                            |\n",
            "|  GPU   GI   CI        PID   Type   Process name                            GPU Memory |\n",
            "|        ID   ID                                                             Usage      |\n",
            "|=======================================================================================|\n",
            "+---------------------------------------------------------------------------------------+\n",
            "(391, 1) (461, 2)\n",
            "CPU times: user 5.34 s, sys: 132 ms, total: 5.47 s\n",
            "Wall time: 6.13 s\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "%%time\n",
        "co_gdf = co_g.nodes(lambda g: cudf.DataFrame(g._nodes)).edges(lambda g: cudf.DataFrame(g._edges))\n",
        "! nvidia-smi\n",
        "for i in range(10):\n",
        "  out = co_gdf.chain([ n({'id': 1}), e_forward(hops=3)])\n",
        "! nvidia-smi\n",
        "print(out._nodes.shape, out._edges.shape)\n",
        "del co_gdf\n",
        "del out"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Hid0-iPKhpOd",
        "outputId": "ecaeb534-d4d7-48fa-d4e1-c80b22626afe"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Mon Dec 25 06:25:25 2023       \n",
            "+---------------------------------------------------------------------------------------+\n",
            "| NVIDIA-SMI 535.104.05             Driver Version: 535.104.05   CUDA Version: 12.2     |\n",
            "|-----------------------------------------+----------------------+----------------------+\n",
            "| GPU  Name                 Persistence-M | Bus-Id        Disp.A | Volatile Uncorr. ECC |\n",
            "| Fan  Temp   Perf          Pwr:Usage/Cap |         Memory-Usage | GPU-Util  Compute M. |\n",
            "|                                         |                      |               MIG M. |\n",
            "|=========================================+======================+======================|\n",
            "|   0  Tesla T4                       Off | 00000000:00:04.0 Off |                    0 |\n",
            "| N/A   61C    P0              29W /  70W |   1925MiB / 15360MiB |     31%      Default |\n",
            "|                                         |                      |                  N/A |\n",
            "+-----------------------------------------+----------------------+----------------------+\n",
            "                                                                                         \n",
            "+---------------------------------------------------------------------------------------+\n",
            "| Processes:                                                                            |\n",
            "|  GPU   GI   CI        PID   Type   Process name                            GPU Memory |\n",
            "|        ID   ID                                                             Usage      |\n",
            "|=======================================================================================|\n",
            "+---------------------------------------------------------------------------------------+\n",
            "Mon Dec 25 06:25:31 2023       \n",
            "+---------------------------------------------------------------------------------------+\n",
            "| NVIDIA-SMI 535.104.05             Driver Version: 535.104.05   CUDA Version: 12.2     |\n",
            "|-----------------------------------------+----------------------+----------------------+\n",
            "| GPU  Name                 Persistence-M | Bus-Id        Disp.A | Volatile Uncorr. ECC |\n",
            "| Fan  Temp   Perf          Pwr:Usage/Cap |         Memory-Usage | GPU-Util  Compute M. |\n",
            "|                                         |                      |               MIG M. |\n",
            "|=========================================+======================+======================|\n",
            "|   0  Tesla T4                       Off | 00000000:00:04.0 Off |                    0 |\n",
            "| N/A   65C    P0              71W /  70W |   2849MiB / 15360MiB |     58%      Default |\n",
            "|                                         |                      |                  N/A |\n",
            "+-----------------------------------------+----------------------+----------------------+\n",
            "                                                                                         \n",
            "+---------------------------------------------------------------------------------------+\n",
            "| Processes:                                                                            |\n",
            "|  GPU   GI   CI        PID   Type   Process name                            GPU Memory |\n",
            "|        ID   ID                                                             Usage      |\n",
            "|=======================================================================================|\n",
            "+---------------------------------------------------------------------------------------+\n",
            "(21767, 1) (28480, 2)\n",
            "CPU times: user 6.25 s, sys: 100 ms, total: 6.35 s\n",
            "Wall time: 6.37 s\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "%%time\n",
        "co_gdf = co_g.nodes(lambda g: cudf.DataFrame(g._nodes)).edges(lambda g: cudf.DataFrame(g._edges))\n",
        "! nvidia-smi\n",
        "for i in range(10):\n",
        "  out = co_gdf.chain([ n({'id': 1}), e_forward(hops=4)])\n",
        "! nvidia-smi\n",
        "print(out._nodes.shape, out._edges.shape)\n",
        "del co_gdf\n",
        "del out"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "buutj-ZjhrEe",
        "outputId": "ae11addd-6bea-44e9-81c0-b431e1db8089"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Mon Dec 25 06:26:04 2023       \n",
            "+---------------------------------------------------------------------------------------+\n",
            "| NVIDIA-SMI 535.104.05             Driver Version: 535.104.05   CUDA Version: 12.2     |\n",
            "|-----------------------------------------+----------------------+----------------------+\n",
            "| GPU  Name                 Persistence-M | Bus-Id        Disp.A | Volatile Uncorr. ECC |\n",
            "| Fan  Temp   Perf          Pwr:Usage/Cap |         Memory-Usage | GPU-Util  Compute M. |\n",
            "|                                         |                      |               MIG M. |\n",
            "|=========================================+======================+======================|\n",
            "|   0  Tesla T4                       Off | 00000000:00:04.0 Off |                    0 |\n",
            "| N/A   61C    P0              29W /  70W |   1927MiB / 15360MiB |     36%      Default |\n",
            "|                                         |                      |                  N/A |\n",
            "+-----------------------------------------+----------------------+----------------------+\n",
            "                                                                                         \n",
            "+---------------------------------------------------------------------------------------+\n",
            "| Processes:                                                                            |\n",
            "|  GPU   GI   CI        PID   Type   Process name                            GPU Memory |\n",
            "|        ID   ID                                                             Usage      |\n",
            "|=======================================================================================|\n",
            "+---------------------------------------------------------------------------------------+\n",
            "Mon Dec 25 06:26:13 2023       \n",
            "+---------------------------------------------------------------------------------------+\n",
            "| NVIDIA-SMI 535.104.05             Driver Version: 535.104.05   CUDA Version: 12.2     |\n",
            "|-----------------------------------------+----------------------+----------------------+\n",
            "| GPU  Name                 Persistence-M | Bus-Id        Disp.A | Volatile Uncorr. ECC |\n",
            "| Fan  Temp   Perf          Pwr:Usage/Cap |         Memory-Usage | GPU-Util  Compute M. |\n",
            "|                                         |                      |               MIG M. |\n",
            "|=========================================+======================+======================|\n",
            "|   0  Tesla T4                       Off | 00000000:00:04.0 Off |                    0 |\n",
            "| N/A   65C    P0              71W /  70W |   2931MiB / 15360MiB |     90%      Default |\n",
            "|                                         |                      |                  N/A |\n",
            "+-----------------------------------------+----------------------+----------------------+\n",
            "                                                                                         \n",
            "+---------------------------------------------------------------------------------------+\n",
            "| Processes:                                                                            |\n",
            "|  GPU   GI   CI        PID   Type   Process name                            GPU Memory |\n",
            "|        ID   ID                                                             Usage      |\n",
            "|=======================================================================================|\n",
            "+---------------------------------------------------------------------------------------+\n",
            "(718640, 1) (2210961, 2)\n",
            "CPU times: user 9.01 s, sys: 1.03 s, total: 10 s\n",
            "Wall time: 9.84 s\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "%%time\n",
        "co_gdf = co_g.nodes(lambda g: cudf.DataFrame(g._nodes)).edges(lambda g: cudf.DataFrame(g._edges))\n",
        "! nvidia-smi\n",
        "for i in range(10):\n",
        "  out = co_gdf.chain([ n({'id': 1}), e_forward(hops=5)])\n",
        "! nvidia-smi\n",
        "print(out._nodes.shape, out._edges.shape)\n",
        "del co_gdf\n",
        "del out"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "bK4C9Ly0hso-",
        "outputId": "8a9a32ab-03e2-42b4-8b71-2bcf797b31b1"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Mon Dec 25 06:27:18 2023       \n",
            "+---------------------------------------------------------------------------------------+\n",
            "| NVIDIA-SMI 535.104.05             Driver Version: 535.104.05   CUDA Version: 12.2     |\n",
            "|-----------------------------------------+----------------------+----------------------+\n",
            "| GPU  Name                 Persistence-M | Bus-Id        Disp.A | Volatile Uncorr. ECC |\n",
            "| Fan  Temp   Perf          Pwr:Usage/Cap |         Memory-Usage | GPU-Util  Compute M. |\n",
            "|                                         |                      |               MIG M. |\n",
            "|=========================================+======================+======================|\n",
            "|   0  Tesla T4                       Off | 00000000:00:04.0 Off |                    0 |\n",
            "| N/A   60C    P0              29W /  70W |   1927MiB / 15360MiB |     28%      Default |\n",
            "|                                         |                      |                  N/A |\n",
            "+-----------------------------------------+----------------------+----------------------+\n",
            "                                                                                         \n",
            "+---------------------------------------------------------------------------------------+\n",
            "| Processes:                                                                            |\n",
            "|  GPU   GI   CI        PID   Type   Process name                            GPU Memory |\n",
            "|        ID   ID                                                             Usage      |\n",
            "|=======================================================================================|\n",
            "+---------------------------------------------------------------------------------------+\n",
            "Mon Dec 25 06:27:57 2023       \n",
            "+---------------------------------------------------------------------------------------+\n",
            "| NVIDIA-SMI 535.104.05             Driver Version: 535.104.05   CUDA Version: 12.2     |\n",
            "|-----------------------------------------+----------------------+----------------------+\n",
            "| GPU  Name                 Persistence-M | Bus-Id        Disp.A | Volatile Uncorr. ECC |\n",
            "| Fan  Temp   Perf          Pwr:Usage/Cap |         Memory-Usage | GPU-Util  Compute M. |\n",
            "|                                         |                      |               MIG M. |\n",
            "|=========================================+======================+======================|\n",
            "|   0  Tesla T4                       Off | 00000000:00:04.0 Off |                    0 |\n",
            "| N/A   72C    P0              43W /  70W |   4351MiB / 15360MiB |    100%      Default |\n",
            "|                                         |                      |                  N/A |\n",
            "+-----------------------------------------+----------------------+----------------------+\n",
            "                                                                                         \n",
            "+---------------------------------------------------------------------------------------+\n",
            "| Processes:                                                                            |\n",
            "|  GPU   GI   CI        PID   Type   Process name                            GPU Memory |\n",
            "|        ID   ID                                                             Usage      |\n",
            "|=======================================================================================|\n",
            "+---------------------------------------------------------------------------------------+\n",
            "(3041556, 1) (47622917, 2)\n",
            "CPU times: user 34.9 s, sys: 4.76 s, total: 39.6 s\n",
            "Wall time: 39.2 s\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "%%time\n",
        "co_gdf = co_g.nodes(lambda g: cudf.DataFrame(g._nodes)).edges(lambda g: cudf.DataFrame(g._edges))\n",
        "out = co_gdf.chain([ n({'id': 1}), e_forward(hops=6)])._nodes\n",
        "print(out.shape)\n",
        "del co_gdf\n",
        "del out"
      ],
      "metadata": {
        "id": "qrga-la0hwhh"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "!lscpu\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "eiXFImxF-rzw",
        "outputId": "b807cc3d-ed1a-4bef-c6e0-bfc2df7356ff"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Architecture:            x86_64\n",
            "  CPU op-mode(s):        32-bit, 64-bit\n",
            "  Address sizes:         46 bits physical, 48 bits virtual\n",
            "  Byte Order:            Little Endian\n",
            "CPU(s):                  2\n",
            "  On-line CPU(s) list:   0,1\n",
            "Vendor ID:               GenuineIntel\n",
            "  Model name:            Intel(R) Xeon(R) CPU @ 2.20GHz\n",
            "    CPU family:          6\n",
            "    Model:               79\n",
            "    Thread(s) per core:  2\n",
            "    Core(s) per socket:  1\n",
            "    Socket(s):           1\n",
            "    Stepping:            0\n",
            "    BogoMIPS:            4399.99\n",
            "    Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clf\n",
            "                         lush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc rep_\n",
            "                         good nopl xtopology nonstop_tsc cpuid tsc_known_freq pni pclmulqdq ssse3 fm\n",
            "                         a cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt aes xsave avx f16c rdrand hyp\n",
            "                         ervisor lahf_lm abm 3dnowprefetch invpcid_single ssbd ibrs ibpb stibp fsgsb\n",
            "                         ase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm rdseed adx smap xsa\n",
            "                         veopt arat md_clear arch_capabilities\n",
            "Virtualization features: \n",
            "  Hypervisor vendor:     KVM\n",
            "  Virtualization type:   full\n",
            "Caches (sum of all):     \n",
            "  L1d:                   32 KiB (1 instance)\n",
            "  L1i:                   32 KiB (1 instance)\n",
            "  L2:                    256 KiB (1 instance)\n",
            "  L3:                    55 MiB (1 instance)\n",
            "NUMA:                    \n",
            "  NUMA node(s):          1\n",
            "  NUMA node0 CPU(s):     0,1\n",
            "Vulnerabilities:         \n",
            "  Gather data sampling:  Not affected\n",
            "  Itlb multihit:         Not affected\n",
            "  L1tf:                  Mitigation; PTE Inversion\n",
            "  Mds:                   Vulnerable; SMT Host state unknown\n",
            "  Meltdown:              Vulnerable\n",
            "  Mmio stale data:       Vulnerable\n",
            "  Retbleed:              Vulnerable\n",
            "  Spec rstack overflow:  Not affected\n",
            "  Spec store bypass:     Vulnerable\n",
            "  Spectre v1:            Vulnerable: __user pointer sanitization and usercopy barriers only; no swap\n",
            "                         gs barriers\n",
            "  Spectre v2:            Vulnerable, IBPB: disabled, STIBP: disabled, PBRSB-eIBRS: Not affected\n",
            "  Srbds:                 Not affected\n",
            "  Tsx async abort:       Vulnerable\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "!free -h\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "wJohLi58-sN5",
        "outputId": "c3e144f6-c19a-4c68-e867-f5e7fa2e9df4"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "               total        used        free      shared  buff/cache   available\n",
            "Mem:            12Gi       717Mi       8.0Gi       1.0Mi       3.9Gi        11Gi\n",
            "Swap:             0B          0B          0B\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "%%time\n",
        "start_nodes = pd.DataFrame({'id': [1]})\n",
        "! nvidia-smi\n",
        "for i in range(1):\n",
        "  g2 = co_g.hop(\n",
        "      nodes=start_nodes,\n",
        "      direction='forward',\n",
        "      hops=1)\n",
        "! nvidia-smi\n",
        "print(g2._nodes.shape, g2._edges.shape)\n",
        "#del start_nodes\n",
        "#del co_gdf\n",
        "#del g2"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "zak4Inhco5il",
        "outputId": "30bcf2bc-853e-4e5e-8c57-ba0cd9429554"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Tue Dec 26 01:01:43 2023       \n",
            "+---------------------------------------------------------------------------------------+\n",
            "| NVIDIA-SMI 535.104.05             Driver Version: 535.104.05   CUDA Version: 12.2     |\n",
            "|-----------------------------------------+----------------------+----------------------+\n",
            "| GPU  Name                 Persistence-M | Bus-Id        Disp.A | Volatile Uncorr. ECC |\n",
            "| Fan  Temp   Perf          Pwr:Usage/Cap |         Memory-Usage | GPU-Util  Compute M. |\n",
            "|                                         |                      |               MIG M. |\n",
            "|=========================================+======================+======================|\n",
            "|   0  Tesla T4                       Off | 00000000:00:04.0 Off |                    0 |\n",
            "| N/A   64C    P0              30W /  70W |   2821MiB / 15360MiB |      0%      Default |\n",
            "|                                         |                      |                  N/A |\n",
            "+-----------------------------------------+----------------------+----------------------+\n",
            "                                                                                         \n",
            "+---------------------------------------------------------------------------------------+\n",
            "| Processes:                                                                            |\n",
            "|  GPU   GI   CI        PID   Type   Process name                            GPU Memory |\n",
            "|        ID   ID                                                             Usage      |\n",
            "|=======================================================================================|\n",
            "+---------------------------------------------------------------------------------------+\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "%%time\n",
        "start_nodes = cudf.DataFrame({'id': [1]})\n",
        "co_gdf = co_g.nodes(lambda g: cudf.DataFrame(g._nodes)).edges(lambda g: cudf.DataFrame(g._edges))\n",
        "! nvidia-smi\n",
        "for i in range(10):\n",
        "  g2 = co_gdf.hop(\n",
        "      nodes=start_nodes,\n",
        "      direction='forward',\n",
        "      hops=1)\n",
        "! nvidia-smi\n",
        "print(g2._nodes.shape, g2._edges.shape)\n",
        "del start_nodes\n",
        "del co_gdf\n",
        "del g2"
      ],
      "metadata": {
        "id": "-SmFlCBS_Bgx",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "d2326cf7-3ea6-4f99-9548-f2e98ece59a4"
      },
      "execution_count": 16,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Tue Dec 26 00:56:45 2023       \n",
            "+---------------------------------------------------------------------------------------+\n",
            "| NVIDIA-SMI 535.104.05             Driver Version: 535.104.05   CUDA Version: 12.2     |\n",
            "|-----------------------------------------+----------------------+----------------------+\n",
            "| GPU  Name                 Persistence-M | Bus-Id        Disp.A | Volatile Uncorr. ECC |\n",
            "| Fan  Temp   Perf          Pwr:Usage/Cap |         Memory-Usage | GPU-Util  Compute M. |\n",
            "|                                         |                      |               MIG M. |\n",
            "|=========================================+======================+======================|\n",
            "|   0  Tesla T4                       Off | 00000000:00:04.0 Off |                    0 |\n",
            "| N/A   49C    P0              28W /  70W |   1923MiB / 15360MiB |     37%      Default |\n",
            "|                                         |                      |                  N/A |\n",
            "+-----------------------------------------+----------------------+----------------------+\n",
            "                                                                                         \n",
            "+---------------------------------------------------------------------------------------+\n",
            "| Processes:                                                                            |\n",
            "|  GPU   GI   CI        PID   Type   Process name                            GPU Memory |\n",
            "|        ID   ID                                                             Usage      |\n",
            "|=======================================================================================|\n",
            "+---------------------------------------------------------------------------------------+\n",
            "Tue Dec 26 00:56:47 2023       \n",
            "+---------------------------------------------------------------------------------------+\n",
            "| NVIDIA-SMI 535.104.05             Driver Version: 535.104.05   CUDA Version: 12.2     |\n",
            "|-----------------------------------------+----------------------+----------------------+\n",
            "| GPU  Name                 Persistence-M | Bus-Id        Disp.A | Volatile Uncorr. ECC |\n",
            "| Fan  Temp   Perf          Pwr:Usage/Cap |         Memory-Usage | GPU-Util  Compute M. |\n",
            "|                                         |                      |               MIG M. |\n",
            "|=========================================+======================+======================|\n",
            "|   0  Tesla T4                       Off | 00000000:00:04.0 Off |                    0 |\n",
            "| N/A   52C    P0              70W /  70W |   2819MiB / 15360MiB |     79%      Default |\n",
            "|                                         |                      |                  N/A |\n",
            "+-----------------------------------------+----------------------+----------------------+\n",
            "                                                                                         \n",
            "+---------------------------------------------------------------------------------------+\n",
            "| Processes:                                                                            |\n",
            "|  GPU   GI   CI        PID   Type   Process name                            GPU Memory |\n",
            "|        ID   ID                                                             Usage      |\n",
            "|=======================================================================================|\n",
            "+---------------------------------------------------------------------------------------+\n",
            "(12, 1) (11, 2)\n",
            "CPU times: user 1.6 s, sys: 37.3 ms, total: 1.64 s\n",
            "Wall time: 1.84 s\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "%%time\n",
        "start_nodes = cudf.DataFrame({'id': [1]})\n",
        "co_gdf = co_g.nodes(lambda g: cudf.DataFrame(g._nodes)).edges(lambda g: cudf.DataFrame(g._edges))\n",
        "! nvidia-smi\n",
        "for i in range(10):\n",
        "  g2 = co_gdf.hop(\n",
        "      nodes=start_nodes,\n",
        "      direction='forward',\n",
        "      hops=2)\n",
        "! nvidia-smi\n",
        "print(g2._nodes.shape, g2._edges.shape)\n",
        "del start_nodes\n",
        "del co_gdf\n",
        "del g2"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "fjjt3YnYnabv",
        "outputId": "05762f50-bfe1-4d23-9153-31431418c8e5"
      },
      "execution_count": 17,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Tue Dec 26 00:56:47 2023       \n",
            "+---------------------------------------------------------------------------------------+\n",
            "| NVIDIA-SMI 535.104.05             Driver Version: 535.104.05   CUDA Version: 12.2     |\n",
            "|-----------------------------------------+----------------------+----------------------+\n",
            "| GPU  Name                 Persistence-M | Bus-Id        Disp.A | Volatile Uncorr. ECC |\n",
            "| Fan  Temp   Perf          Pwr:Usage/Cap |         Memory-Usage | GPU-Util  Compute M. |\n",
            "|                                         |                      |               MIG M. |\n",
            "|=========================================+======================+======================|\n",
            "|   0  Tesla T4                       Off | 00000000:00:04.0 Off |                    0 |\n",
            "| N/A   51C    P0              35W /  70W |   1923MiB / 15360MiB |     59%      Default |\n",
            "|                                         |                      |                  N/A |\n",
            "+-----------------------------------------+----------------------+----------------------+\n",
            "                                                                                         \n",
            "+---------------------------------------------------------------------------------------+\n",
            "| Processes:                                                                            |\n",
            "|  GPU   GI   CI        PID   Type   Process name                            GPU Memory |\n",
            "|        ID   ID                                                             Usage      |\n",
            "|=======================================================================================|\n",
            "+---------------------------------------------------------------------------------------+\n",
            "Tue Dec 26 00:56:49 2023       \n",
            "+---------------------------------------------------------------------------------------+\n",
            "| NVIDIA-SMI 535.104.05             Driver Version: 535.104.05   CUDA Version: 12.2     |\n",
            "|-----------------------------------------+----------------------+----------------------+\n",
            "| GPU  Name                 Persistence-M | Bus-Id        Disp.A | Volatile Uncorr. ECC |\n",
            "| Fan  Temp   Perf          Pwr:Usage/Cap |         Memory-Usage | GPU-Util  Compute M. |\n",
            "|                                         |                      |               MIG M. |\n",
            "|=========================================+======================+======================|\n",
            "|   0  Tesla T4                       Off | 00000000:00:04.0 Off |                    0 |\n",
            "| N/A   53C    P0              59W /  70W |   2821MiB / 15360MiB |     86%      Default |\n",
            "|                                         |                      |                  N/A |\n",
            "+-----------------------------------------+----------------------+----------------------+\n",
            "                                                                                         \n",
            "+---------------------------------------------------------------------------------------+\n",
            "| Processes:                                                                            |\n",
            "|  GPU   GI   CI        PID   Type   Process name                            GPU Memory |\n",
            "|        ID   ID                                                             Usage      |\n",
            "|=======================================================================================|\n",
            "+---------------------------------------------------------------------------------------+\n",
            "(391, 1) (461, 2)\n",
            "CPU times: user 2.32 s, sys: 58.5 ms, total: 2.38 s\n",
            "Wall time: 2.51 s\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "%%time\n",
        "start_nodes = cudf.DataFrame({'id': [1]})\n",
        "co_gdf = co_g.nodes(lambda g: cudf.DataFrame(g._nodes)).edges(lambda g: cudf.DataFrame(g._edges))\n",
        "! nvidia-smi\n",
        "for i in range(10):\n",
        "  g2 = co_gdf.hop(\n",
        "      nodes=start_nodes,\n",
        "      direction='forward',\n",
        "      hops=3)\n",
        "! nvidia-smi\n",
        "print(g2._nodes.shape, g2._edges.shape)\n",
        "del start_nodes\n",
        "del co_gdf\n",
        "del g2"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "oIouuORgnbcY",
        "outputId": "f07abe4c-5137-4ee3-935a-afbb2c5eaa1e"
      },
      "execution_count": 18,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Tue Dec 26 00:56:50 2023       \n",
            "+---------------------------------------------------------------------------------------+\n",
            "| NVIDIA-SMI 535.104.05             Driver Version: 535.104.05   CUDA Version: 12.2     |\n",
            "|-----------------------------------------+----------------------+----------------------+\n",
            "| GPU  Name                 Persistence-M | Bus-Id        Disp.A | Volatile Uncorr. ECC |\n",
            "| Fan  Temp   Perf          Pwr:Usage/Cap |         Memory-Usage | GPU-Util  Compute M. |\n",
            "|                                         |                      |               MIG M. |\n",
            "|=========================================+======================+======================|\n",
            "|   0  Tesla T4                       Off | 00000000:00:04.0 Off |                    0 |\n",
            "| N/A   52C    P0              36W /  70W |   1925MiB / 15360MiB |     55%      Default |\n",
            "|                                         |                      |                  N/A |\n",
            "+-----------------------------------------+----------------------+----------------------+\n",
            "                                                                                         \n",
            "+---------------------------------------------------------------------------------------+\n",
            "| Processes:                                                                            |\n",
            "|  GPU   GI   CI        PID   Type   Process name                            GPU Memory |\n",
            "|        ID   ID                                                             Usage      |\n",
            "|=======================================================================================|\n",
            "+---------------------------------------------------------------------------------------+\n",
            "Tue Dec 26 00:56:53 2023       \n",
            "+---------------------------------------------------------------------------------------+\n",
            "| NVIDIA-SMI 535.104.05             Driver Version: 535.104.05   CUDA Version: 12.2     |\n",
            "|-----------------------------------------+----------------------+----------------------+\n",
            "| GPU  Name                 Persistence-M | Bus-Id        Disp.A | Volatile Uncorr. ECC |\n",
            "| Fan  Temp   Perf          Pwr:Usage/Cap |         Memory-Usage | GPU-Util  Compute M. |\n",
            "|                                         |                      |               MIG M. |\n",
            "|=========================================+======================+======================|\n",
            "|   0  Tesla T4                       Off | 00000000:00:04.0 Off |                    0 |\n",
            "| N/A   54C    P0              75W /  70W |   2825MiB / 15360MiB |     74%      Default |\n",
            "|                                         |                      |                  N/A |\n",
            "+-----------------------------------------+----------------------+----------------------+\n",
            "                                                                                         \n",
            "+---------------------------------------------------------------------------------------+\n",
            "| Processes:                                                                            |\n",
            "|  GPU   GI   CI        PID   Type   Process name                            GPU Memory |\n",
            "|        ID   ID                                                             Usage      |\n",
            "|=======================================================================================|\n",
            "+---------------------------------------------------------------------------------------+\n",
            "(21767, 1) (28480, 2)\n",
            "CPU times: user 3.04 s, sys: 63.6 ms, total: 3.1 s\n",
            "Wall time: 3.25 s\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "%%time\n",
        "start_nodes = cudf.DataFrame({'id': [1]})\n",
        "co_gdf = co_g.nodes(lambda g: cudf.DataFrame(g._nodes)).edges(lambda g: cudf.DataFrame(g._edges))\n",
        "! nvidia-smi\n",
        "for i in range(10):\n",
        "  g2 = co_gdf.hop(\n",
        "      nodes=start_nodes,\n",
        "      direction='forward',\n",
        "      hops=4)\n",
        "! nvidia-smi\n",
        "print(g2._nodes.shape, g2._edges.shape)\n",
        "del start_nodes\n",
        "del co_gdf\n",
        "del g2"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "oNLZGjwInc85",
        "outputId": "534097cf-4022-48cc-9419-a00c135f69e1"
      },
      "execution_count": 19,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Tue Dec 26 00:56:53 2023       \n",
            "+---------------------------------------------------------------------------------------+\n",
            "| NVIDIA-SMI 535.104.05             Driver Version: 535.104.05   CUDA Version: 12.2     |\n",
            "|-----------------------------------------+----------------------+----------------------+\n",
            "| GPU  Name                 Persistence-M | Bus-Id        Disp.A | Volatile Uncorr. ECC |\n",
            "| Fan  Temp   Perf          Pwr:Usage/Cap |         Memory-Usage | GPU-Util  Compute M. |\n",
            "|                                         |                      |               MIG M. |\n",
            "|=========================================+======================+======================|\n",
            "|   0  Tesla T4                       Off | 00000000:00:04.0 Off |                    0 |\n",
            "| N/A   54C    P0              36W /  70W |   1927MiB / 15360MiB |     54%      Default |\n",
            "|                                         |                      |                  N/A |\n",
            "+-----------------------------------------+----------------------+----------------------+\n",
            "                                                                                         \n",
            "+---------------------------------------------------------------------------------------+\n",
            "| Processes:                                                                            |\n",
            "|  GPU   GI   CI        PID   Type   Process name                            GPU Memory |\n",
            "|        ID   ID                                                             Usage      |\n",
            "|=======================================================================================|\n",
            "+---------------------------------------------------------------------------------------+\n",
            "Tue Dec 26 00:56:58 2023       \n",
            "+---------------------------------------------------------------------------------------+\n",
            "| NVIDIA-SMI 535.104.05             Driver Version: 535.104.05   CUDA Version: 12.2     |\n",
            "|-----------------------------------------+----------------------+----------------------+\n",
            "| GPU  Name                 Persistence-M | Bus-Id        Disp.A | Volatile Uncorr. ECC |\n",
            "| Fan  Temp   Perf          Pwr:Usage/Cap |         Memory-Usage | GPU-Util  Compute M. |\n",
            "|                                         |                      |               MIG M. |\n",
            "|=========================================+======================+======================|\n",
            "|   0  Tesla T4                       Off | 00000000:00:04.0 Off |                    0 |\n",
            "| N/A   56C    P0              38W /  70W |   2907MiB / 15360MiB |     89%      Default |\n",
            "|                                         |                      |                  N/A |\n",
            "+-----------------------------------------+----------------------+----------------------+\n",
            "                                                                                         \n",
            "+---------------------------------------------------------------------------------------+\n",
            "| Processes:                                                                            |\n",
            "|  GPU   GI   CI        PID   Type   Process name                            GPU Memory |\n",
            "|        ID   ID                                                             Usage      |\n",
            "|=======================================================================================|\n",
            "+---------------------------------------------------------------------------------------+\n",
            "(718640, 1) (2210961, 2)\n",
            "CPU times: user 4.58 s, sys: 309 ms, total: 4.89 s\n",
            "Wall time: 5.02 s\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "%%time\n",
        "start_nodes = cudf.DataFrame({'id': [1]})\n",
        "co_gdf = co_g.nodes(lambda g: cudf.DataFrame(g._nodes)).edges(lambda g: cudf.DataFrame(g._edges))\n",
        "! nvidia-smi\n",
        "for i in range(10):\n",
        "  g2 = co_gdf.hop(\n",
        "      nodes=start_nodes,\n",
        "      direction='forward',\n",
        "      hops=5)\n",
        "! nvidia-smi\n",
        "print(g2._nodes.shape, g2._edges.shape)\n",
        "del start_nodes\n",
        "del co_gdf\n",
        "del g2"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ePqaeujMneX8",
        "outputId": "ffd88fff-016e-4ac0-ecb9-fa06baca60f8"
      },
      "execution_count": 20,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Tue Dec 26 00:56:58 2023       \n",
            "+---------------------------------------------------------------------------------------+\n",
            "| NVIDIA-SMI 535.104.05             Driver Version: 535.104.05   CUDA Version: 12.2     |\n",
            "|-----------------------------------------+----------------------+----------------------+\n",
            "| GPU  Name                 Persistence-M | Bus-Id        Disp.A | Volatile Uncorr. ECC |\n",
            "| Fan  Temp   Perf          Pwr:Usage/Cap |         Memory-Usage | GPU-Util  Compute M. |\n",
            "|                                         |                      |               MIG M. |\n",
            "|=========================================+======================+======================|\n",
            "|   0  Tesla T4                       Off | 00000000:00:04.0 Off |                    0 |\n",
            "| N/A   55C    P0              37W /  70W |   1925MiB / 15360MiB |     59%      Default |\n",
            "|                                         |                      |                  N/A |\n",
            "+-----------------------------------------+----------------------+----------------------+\n",
            "                                                                                         \n",
            "+---------------------------------------------------------------------------------------+\n",
            "| Processes:                                                                            |\n",
            "|  GPU   GI   CI        PID   Type   Process name                            GPU Memory |\n",
            "|        ID   ID                                                             Usage      |\n",
            "|=======================================================================================|\n",
            "+---------------------------------------------------------------------------------------+\n",
            "Tue Dec 26 00:57:10 2023       \n",
            "+---------------------------------------------------------------------------------------+\n",
            "| NVIDIA-SMI 535.104.05             Driver Version: 535.104.05   CUDA Version: 12.2     |\n",
            "|-----------------------------------------+----------------------+----------------------+\n",
            "| GPU  Name                 Persistence-M | Bus-Id        Disp.A | Volatile Uncorr. ECC |\n",
            "| Fan  Temp   Perf          Pwr:Usage/Cap |         Memory-Usage | GPU-Util  Compute M. |\n",
            "|                                         |                      |               MIG M. |\n",
            "|=========================================+======================+======================|\n",
            "|   0  Tesla T4                       Off | 00000000:00:04.0 Off |                    0 |\n",
            "| N/A   60C    P0              48W /  70W |   4325MiB / 15360MiB |     99%      Default |\n",
            "|                                         |                      |                  N/A |\n",
            "+-----------------------------------------+----------------------+----------------------+\n",
            "                                                                                         \n",
            "+---------------------------------------------------------------------------------------+\n",
            "| Processes:                                                                            |\n",
            "|  GPU   GI   CI        PID   Type   Process name                            GPU Memory |\n",
            "|        ID   ID                                                             Usage      |\n",
            "|=======================================================================================|\n",
            "+---------------------------------------------------------------------------------------+\n",
            "(3041556, 1) (47622917, 2)\n",
            "CPU times: user 10.8 s, sys: 1.29 s, total: 12.1 s\n",
            "Wall time: 12 s\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "%%time\n",
        "start_nodes = cudf.DataFrame({'id': [1]})\n",
        "co_gdf = co_g.nodes(lambda g: cudf.DataFrame(g._nodes)).edges(lambda g: cudf.DataFrame(g._edges))\n",
        "! nvidia-smi\n",
        "for i in range(10):\n",
        "  g2 = co_gdf.hop(\n",
        "      nodes=start_nodes,\n",
        "      direction='forward',\n",
        "      hops=6)\n",
        "! nvidia-smi\n",
        "print(g2._nodes.shape, g2._edges.shape)\n",
        "del start_nodes\n",
        "del co_gdf\n",
        "del g2"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "PTBkoIVHnfzK",
        "outputId": "5615ecd7-47ea-46ab-fd36-13bce4b3c787"
      },
      "execution_count": 21,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Tue Dec 26 00:57:10 2023       \n",
            "+---------------------------------------------------------------------------------------+\n",
            "| NVIDIA-SMI 535.104.05             Driver Version: 535.104.05   CUDA Version: 12.2     |\n",
            "|-----------------------------------------+----------------------+----------------------+\n",
            "| GPU  Name                 Persistence-M | Bus-Id        Disp.A | Volatile Uncorr. ECC |\n",
            "| Fan  Temp   Perf          Pwr:Usage/Cap |         Memory-Usage | GPU-Util  Compute M. |\n",
            "|                                         |                      |               MIG M. |\n",
            "|=========================================+======================+======================|\n",
            "|   0  Tesla T4                       Off | 00000000:00:04.0 Off |                    0 |\n",
            "| N/A   59C    P0              38W /  70W |   1925MiB / 15360MiB |     44%      Default |\n",
            "|                                         |                      |                  N/A |\n",
            "+-----------------------------------------+----------------------+----------------------+\n",
            "                                                                                         \n",
            "+---------------------------------------------------------------------------------------+\n",
            "| Processes:                                                                            |\n",
            "|  GPU   GI   CI        PID   Type   Process name                            GPU Memory |\n",
            "|        ID   ID                                                             Usage      |\n",
            "|=======================================================================================|\n",
            "+---------------------------------------------------------------------------------------+\n",
            "Tue Dec 26 00:57:38 2023       \n",
            "+---------------------------------------------------------------------------------------+\n",
            "| NVIDIA-SMI 535.104.05             Driver Version: 535.104.05   CUDA Version: 12.2     |\n",
            "|-----------------------------------------+----------------------+----------------------+\n",
            "| GPU  Name                 Persistence-M | Bus-Id        Disp.A | Volatile Uncorr. ECC |\n",
            "| Fan  Temp   Perf          Pwr:Usage/Cap |         Memory-Usage | GPU-Util  Compute M. |\n",
            "|                                         |                      |               MIG M. |\n",
            "|=========================================+======================+======================|\n",
            "|   0  Tesla T4                       Off | 00000000:00:04.0 Off |                    0 |\n",
            "| N/A   68C    P0              55W /  70W |   6445MiB / 15360MiB |     95%      Default |\n",
            "|                                         |                      |                  N/A |\n",
            "+-----------------------------------------+----------------------+----------------------+\n",
            "                                                                                         \n",
            "+---------------------------------------------------------------------------------------+\n",
            "| Processes:                                                                            |\n",
            "|  GPU   GI   CI        PID   Type   Process name                            GPU Memory |\n",
            "|        ID   ID                                                             Usage      |\n",
            "|=======================================================================================|\n",
            "+---------------------------------------------------------------------------------------+\n",
            "(3071927, 1) (117032738, 2)\n",
            "CPU times: user 23.5 s, sys: 2.68 s, total: 26.2 s\n",
            "Wall time: 28.2 s\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "Ygc2nrkznlCu"
      },
      "execution_count": null,
      "outputs": []
    }
  ]
}