viz-1b-flights-splom.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[Home](https://walterra.github.io/jupyter2kibana/)\n",
    "\n",
    "*To make sure the Vega charts render correctly, view the notebook not from the Github repo but the published website here: \n",
    "https://walterra.github.io/jupyter2kibana/viz-1b-flights-splom.html*\n",
    "\n",
    "# viz-1b-flights-splom.ipynb\n",
    "\n",
    "This notebook builds on the learnings from [the first one](ttps://walterra.github.io/jupyter2kibana/viz-1a-flights-histogram.html) and uses the same data to create and deploy a scatterplot matrix. The difference here is that we moved the code to create the SavedObject in Kibana to a helper function into [kibana_vega_util.py](https://github.com/walterra/jupyter2kibana/blob/master/kibana_vega_util.py) so we can reuse it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import datetime\n",
    "import altair as alt\n",
    "import eland as ed\n",
    "import json\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import urllib3\n",
    "import warnings\n",
    "\n",
    "alt.data_transformers.disable_max_rows()\n",
    "#Only in development environments with self signed certificates enable this setting to disable warnings\n",
    "urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)\n",
    "\n",
    "# For rendering the notebook to HTML hide all warnings\n",
    "warnings.filterwarnings('ignore')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The following cell will only output a visualization populated with data if the ES instance referenced has security disabled (not recommended, do only for demo purposes). The cell still needs to be run to create the Vega specification used in the cell after that to save the visualization in Kibana."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "\n",
       "<div id=\"altair-viz-1844c50e5147426293617d8cfbf34291\"></div>\n",
       "<script type=\"text/javascript\">\n",
       "  var VEGA_DEBUG = (typeof VEGA_DEBUG == \"undefined\") ? {} : VEGA_DEBUG;\n",
       "  (function(spec, embedOpt){\n",
       "    let outputDiv = document.currentScript.previousElementSibling;\n",
       "    if (outputDiv.id !== \"altair-viz-1844c50e5147426293617d8cfbf34291\") {\n",
       "      outputDiv = document.getElementById(\"altair-viz-1844c50e5147426293617d8cfbf34291\");\n",
       "    }\n",
       "    const paths = {\n",
       "      \"vega\": \"https://cdn.jsdelivr.net/npm//vega@5?noext\",\n",
       "      \"vega-lib\": \"https://cdn.jsdelivr.net/npm//vega-lib?noext\",\n",
       "      \"vega-lite\": \"https://cdn.jsdelivr.net/npm//vega-lite@4.17.0?noext\",\n",
       "      \"vega-embed\": \"https://cdn.jsdelivr.net/npm//vega-embed@6?noext\",\n",
       "    };\n",
       "\n",
       "    function maybeLoadScript(lib, version) {\n",
       "      var key = `${lib.replace(\"-\", \"\")}_version`;\n",
       "      return (VEGA_DEBUG[key] == version) ?\n",
       "        Promise.resolve(paths[lib]) :\n",
       "        new Promise(function(resolve, reject) {\n",
       "          var s = document.createElement('script');\n",
       "          document.getElementsByTagName(\"head\")[0].appendChild(s);\n",
       "          s.async = true;\n",
       "          s.onload = () => {\n",
       "            VEGA_DEBUG[key] = version;\n",
       "            return resolve(paths[lib]);\n",
       "          };\n",
       "          s.onerror = () => reject(`Error loading script: ${paths[lib]}`);\n",
       "          s.src = paths[lib];\n",
       "        });\n",
       "    }\n",
       "\n",
       "    function showError(err) {\n",
       "      outputDiv.innerHTML = `<div class=\"error\" style=\"color:red;\">${err}</div>`;\n",
       "      throw err;\n",
       "    }\n",
       "\n",
       "    function displayChart(vegaEmbed) {\n",
       "      vegaEmbed(outputDiv, spec, embedOpt)\n",
       "        .catch(err => showError(`Javascript Error: ${err.message}<br>This usually means there's a typo in your chart specification. See the javascript console for the full traceback.`));\n",
       "    }\n",
       "\n",
       "    if(typeof define === \"function\" && define.amd) {\n",
       "      requirejs.config({paths});\n",
       "      require([\"vega-embed\"], displayChart, err => showError(`Error loading script: ${err.message}`));\n",
       "    } else {\n",
       "      maybeLoadScript(\"vega\", \"5\")\n",
       "        .then(() => maybeLoadScript(\"vega-lite\", \"4.17.0\"))\n",
       "        .then(() => maybeLoadScript(\"vega-embed\", \"6\"))\n",
       "        .catch(showError)\n",
       "        .then(() => displayChart(vegaEmbed));\n",
       "    }\n",
       "  })({\"config\": {\"view\": {\"continuousWidth\": 400, \"continuousHeight\": 300}}, \"repeat\": {\"column\": [\"FlightTimeMin\", \"FlightDelayMin\", \"DistanceKilometers\", \"AvgTicketPrice\"], \"row\": [\"AvgTicketPrice\", \"DistanceKilometers\", \"FlightDelayMin\", \"FlightTimeMin\"]}, \"spec\": {\"data\": {\"url\": \"https://walterra.github.io/jupyter2kibana/data/kibana_sample_data_flights.json\", \"format\": {\"property\": \"hits.hits\", \"type\": \"json\"}}, \"mark\": {\"type\": \"circle\", \"opacity\": 0.5, \"size\": 6}, \"encoding\": {\"x\": {\"field\": {\"repeat\": \"column\"}, \"type\": \"quantitative\"}, \"y\": {\"field\": {\"repeat\": \"row\"}, \"type\": \"quantitative\"}}, \"height\": 100, \"selection\": {\"selector001\": {\"type\": \"interval\", \"bind\": \"scales\", \"encodings\": [\"x\", \"y\"]}}, \"transform\": [{\"calculate\": \"datum._source.Carrier\", \"as\": \"Carrier\"}, {\"calculate\": \"datum._source.AvgTicketPrice\", \"as\": \"AvgTicketPrice\"}, {\"calculate\": \"datum._source.DistanceKilometers\", \"as\": \"DistanceKilometers\"}, {\"calculate\": \"datum._source.DistanceMiles\", \"as\": \"DistanceMiles\"}, {\"calculate\": \"datum._source.FlightDelayMin\", \"as\": \"FlightDelayMin\"}, {\"calculate\": \"datum._source.FlightTimeMin\", \"as\": \"FlightTimeMin\"}, {\"calculate\": \"datum._source.dayOfWeek\", \"as\": \"dayOfWeek\"}], \"width\": 100}, \"$schema\": \"https://vega.github.io/schema/vega-lite/v4.17.0.json\"}, {\"mode\": \"vega-lite\"});\n",
       "</script>"
      ],
      "text/plain": [
       "alt.RepeatChart(...)"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Note: To create the Vega spec using Altair we reference ES via URL first. This will only work\n",
    "# for non-secured ES instances. If your ES instance runs using SSL and/or authentication the chart\n",
    "# in this cell will render empty. You can still save the visualization in Kibana correctly in the\n",
    "# next cell because there the URL gets replaced with an Elasticsearch query\n",
    "# to be used via the Kibana Vega plugin.\n",
    "\n",
    "# WARNING:\n",
    "# Do the following approach using a proxy only for demo purposes in a development environment.\n",
    "# It will expose a secured ES instance unsecured!\n",
    "# To make this work for demo purposes run the nodejs based proxy in a separate terminal like this:\n",
    "# NODE_TLS_REJECT_UNAUTHORIZED='0' node proxy\n",
    "\n",
    "# URL as ES endpoint\n",
    "# url = 'http://localhost:9220/kibana_sample_data_flights/_search?size=10000'\n",
    "\n",
    "# URL static fallback\n",
    "url = 'https://walterra.github.io/jupyter2kibana/data/kibana_sample_data_flights.json'\n",
    "\n",
    "url_data = alt.Data(url=url, format=alt.DataFormat(property='hits.hits',type='json'))\n",
    "\n",
    "fields = [\n",
    "    'Carrier',\n",
    "    'AvgTicketPrice',\n",
    "    'DistanceKilometers',\n",
    "    'DistanceMiles',\n",
    "    'FlightDelayMin',\n",
    "    'FlightTimeMin',\n",
    "    'dayOfWeek'\n",
    "]\n",
    "\n",
    "rename_dict = dict((a, 'datum._source.'+a) for a in fields)\n",
    "\n",
    "chart = alt.Chart(url_data).mark_circle(\n",
    "    opacity=.5,\n",
    "    size=6\n",
    ").transform_calculate(**rename_dict).encode(\n",
    "    alt.X(alt.repeat(\"column\"), type='quantitative'),\n",
    "    alt.Y(alt.repeat(\"row\"), type='quantitative')\n",
    ").properties(\n",
    "    width=100,\n",
    "    height=100\n",
    ").repeat(\n",
    "    row=['AvgTicketPrice', 'DistanceKilometers', 'FlightDelayMin', 'FlightTimeMin'],\n",
    "    column=['FlightTimeMin', 'FlightDelayMin', 'DistanceKilometers', 'AvgTicketPrice']\n",
    ").interactive()\n",
    "\n",
    "chart"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<Response [409]>"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from kibana_vega_util import saveVegaVis\n",
    "\n",
    "saveVegaVis(\n",
    "    'kibana_sample_data_flights',\n",
    "    'def-vega-splom-1',\n",
    "    chart,\n",
    "    resultSize=1000,\n",
    "    # Only in development environments with self signed certificates fall back to use `verify=False`\n",
    "    verify=False\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "env-jupyter2kibana",
   "language": "python",
   "name": "env-jupyter2kibana"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}