examples/Advanced Usage.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Advanced Usage\n",
    "\n",
    "GraphLog provides an array of datasets, thus making it a perfect candidate to test multi-task, continual and meta-learning in graphs. Each dataset is derived by its own set of **rules**.\n",
    "\n",
    "## Similarity\n",
    "\n",
    "Two datasets can have highly overlapping rules to highly non-overlapping rules. This provides GraphLog an unique way to define the notion of task **similarity**. Two datasets are highly similar if the underlying rules are similar."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "from graphlog import GraphLog\n",
    "gl = GraphLog()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First, lets get the available datasets in GraphLog"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "datasets = gl.get_dataset_names_by_split()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'rule_3'"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "datasets[\"train\"][0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To calculate dataset similarity, we compute the overlap between the actual rules used in the datasets. GraphLog provides an easy API to do so."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.95"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "gl.compute_similarity(\"rule_0\",\"rule_1\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We see that the datasets `rule_0` and `rule_1` are 95\\% similar. To get top 10 similar datasets as of `rule_0`, we can call the following method:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[('rule_0', 1.0),\n",
       " ('rule_1', 0.95),\n",
       " ('rule_2', 0.9),\n",
       " ('rule_3', 0.85),\n",
       " ('rule_4', 0.8),\n",
       " ('rule_5', 0.75),\n",
       " ('rule_6', 0.7),\n",
       " ('rule_7', 0.65),\n",
       " ('rule_8', 0.6),\n",
       " ('rule_9', 0.55)]"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "gl.get_most_similar_datasets(\"rule_0\",10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## MultiTask training\n",
    "\n",
    "By providing an easy way to extract datasets and also by grouping them in terms of similarity, we can easily train and in a multi-task scenario. Below we provide a dummy snippet to do so."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data_ids = gl.get_most_similar_datasets(\"rule_0\",10)\n",
    "for epoch in range(100):\n",
    "    dataset = gl.get_dataset_by_name(random.choice(data_ids))\n",
    "    train_loader = gl.get_dataloader_by_mode(dataset, \"train\")\n",
    "    for batch_id, batch in enumerate(train_loader):\n",
    "        graphs = batch.graphs\n",
    "        queries = batch.queries\n",
    "        labels = batch.targets\n",
    "        logits = your_model(graphs, queries)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Difficulty\n",
    "\n",
    "GraphLog also provides an additional option of categorizing each dataset on their relative _difficulty_. We compute difficulty by the scores of supervised learning methods as a proxy. For more details how we label each dataset as per their difficulty, please check out our paper!\n",
    "\n",
    "We provide additional meta-data to categorize the datasets with respect to their difficulty. To access it, simply call the following API. This will load the datasets directly in memory."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "easy_datasets = gl.get_easy_datasets()\n",
    "moderate_datasets = gl.get_moderate_datasets()\n",
    "hard_datasets = gl.get_hard_datasets()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Continual Learning\n",
    "\n",
    "Using any of the above categorizations, GraphLog also provides an option of evaluating models in a continual learning scenario. Here, we provide a simple example to evaluate continual learning on a rolling window of similar datasets, based on overlapping rules. `get_sorted_dataset_ids(mode=\"train\")` api will return the datasets in the order they were originally created in the paper, which follows a rolling similarity."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "dataset_names = gl.get_sorted_dataset_ids(mode=\"train\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for data_id in dataset_names:\n",
    "    dataset = gl.get_dataset_by_name(data_id)\n",
    "    for epoch in range(100):\n",
    "        train_loader = gl.get_dataloader_by_mode(dataset, \"train\")\n",
    "        for batch_id, batch in enumerate(train_loader):\n",
    "            graphs = batch.graphs\n",
    "            queries = batch.queries\n",
    "            labels = batch.targets\n",
    "            logits = your_model(graphs, queries)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python (graphlog)",
   "language": "python",
   "name": "graphlog"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}