demo-river.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "\n",
    "\n",
    "![River-logo](river_logo.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`River` is a Python library for streaming (online) machine learning.\n",
    "\n",
    "**Announcement:** In November 2020, `river` was released. This is the result of the merge of the two most popular open-source packages `creme` and `scikit-multiflow`.\n",
    "\n",
    "### Key features\n",
    "\n",
    "- **Incremental learning** - All the tools in `river` can be updated with a single sample at a time.\n",
    "\n",
    "- **Adaptive learning** - Adaptive methods are robust against concept drift in dynamic environments.\n",
    "\n",
    "- **General-purpose** - `River` caters for different machine learning problems, including regression, classification, unsupervised learning, and ad-hoc tasks.\n",
    "\n",
    "- **Efficient** - By design, streaming techniques efficiently handle resources such as memory and processing time, given the unbounded nature of data streams.\n",
    "\n",
    "- **Easy to use** - `River` is intended for users with any experience level. As a machine learning package, it caters for practitioners as well as researchers.\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# PART I. Using `river`\n",
    "\n",
    "## From batch to stream learning\n",
    "---\n",
    "\n",
    "Traditional machine learning follows a sequence of steps. A minimal pipeline includes:\n",
    "\n",
    "1. Data preprocessing (cleanup, filtering, feature engineering, etc.)\n",
    "2. Training (create a model from the data)\n",
    "3. Testing (use the model to obtain predictions)\n",
    "\n",
    "This approach assumes that **all** the data is available for training and data is processed in **batches**.\n",
    "\n",
    "Let's see an example based on the [`Iris` dataset](https://scikit-learn.org/stable/auto_examples/datasets/plot_iris_dataset.html) available in `scikit-learn`:\n",
    "\n",
    "> This data set consists of 3 different types of irises’ (Setosa, Versicolour, and Virginica) petal and sepal length, stored in a 150x4 numpy.ndarray"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn import datasets as skdatasets\n",
    "from sklearn.naive_bayes import GaussianNB as GaussianNBSKL   # Renamed to avoid name conflicts\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.metrics import accuracy_score, confusion_matrix\n",
    "\n",
    "iris = skdatasets.load_iris()\n",
    "x = iris.data\n",
    "y = iris.target"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `load_iris()` returns the data set and related information. Data features and the corresponding targets (classification labels) are accessible via the `data` and `target` properties respectively.\n",
    "\n",
    "We can assume that the data has been already preprocessed. We need to split the dataset into $train$ and $test$ sets where $train \\notin test$. We can use the `train_test_split` method as follows:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_train, X_test, y_train, y_test = train_test_split(\n",
    "    iris.data, iris.target, test_size=0.3, stratify=iris.target, random_state=42\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Shape (n_rows, n_cols) of X: (150, 4), X_train: (105, 4), X_test: (45, 4)\n"
     ]
    }
   ],
   "source": [
    "print(f'Shape (n_rows, n_cols) of X: {iris.data.shape}, '\n",
    "      f'X_train: {X_train.shape}, X_test: {X_test.shape}')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we proceed with the model training. In this example, we will train a Gaussian Naive Bayes model (`GaussianNB`)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "GaussianNB(priors=None, var_smoothing=1e-09)"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model = GaussianNBSKL()\n",
    "model.fit(X=X_train, y=y_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And we use the trained model to get predictions:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "y_pred = model.predict(X=X_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`y_pred` contains the predicted class labels for each sample in `X_test`. Since we know the true labels for these samples (`y_test`) we can evaluate the performance of the model. In this case, we will use the `accuracy_score` method:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model accuracy: 0.9111\n"
     ]
    }
   ],
   "source": [
    "print(f'Model accuracy: {accuracy_score(y_test, y_pred):.4f}')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The trained model has an accuracy of ~91%, which is decent considering the simplicity of the dataset.\n",
    "\n",
    "As mentioned, batch methods expect **all the data** available for training. However, in multiple real-world applications, this requirement is difficult to meet, especially in dynamic environments that constantly generate data.\n",
    "\n",
    "A common way to address this situation is to collect data over time and generate new models. However, this presents additional challenges:\n",
    "\n",
    "- Should we train new models on new data only or extend the old data?\n",
    "- How much new data is enough to trigger the creation of a new model?\n",
    "- Is the new model better than the old one?\n",
    "- Storing all data might be unfeasible or impractical in some cases (e.g. financial markets)\n",
    "- ..."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Stream learning\n",
    "\n",
    "A data stream is an infinite sequence of elements. Stream learning methods are specially designed to learn from data streams. Streaming models adhere to a set of requirements:\n",
    "\n",
    "- **Process the data one sample at a time**. A sample is seen only once and it is gone after processing.\n",
    "- **Be efficient**. Resources are limited (memory, time) but streams are infinite.\n",
    "- **Predict at any time**. Models continually learn and provide predictions at any point in the stream.\n",
    "\n",
    "A streaming model **updates** its internal state as it processes the data stream."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can simulate a data stream from the previous example by \"looping\" over the data *one sample at a time* as follows:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(array([4.6, 3.1, 1.5, 0.2]), 0)"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "for xi, yi in zip(X_train, y_train):\n",
    "    # Here is where learning takes place\n",
    "    pass\n",
    "\n",
    "xi, yi    # last data sample"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Notice that the data is encoded in `numpy.ndarrays`, an efficient data structure when dealing with large 2D data.\n",
    "\n",
    "In `river`, data is represented as `dict`. Dictionaries are fast to process 1D data and elements are easily accessible 'by key'.\n",
    "\n",
    "Before using this data in `river` we must transform it. This is easily done in Python:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "({'sepal length (cm)': 4.6,\n",
       "  'sepal width (cm)': 3.1,\n",
       "  'petal length (cm)': 1.5,\n",
       "  'petal width (cm)': 0.2},\n",
       " 0)"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "for xi, yi in zip(X_train, y_train):\n",
    "    x = dict(zip(iris.feature_names, xi))\n",
    "\n",
    "x, yi"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In our previous example, we trained a Gaussian Naive Bayes model using the implementation from `scikit-learn`. The Naive Bayes method is an **incremental method**.\n",
    "\n",
    "A small number of [incremental methods](https://scikit-learn.org/stable/modules/computing.html#scaling-with-instances-using-out-of-core-learning) are available in `scikit-learn`. These methods provide a `partial_fit` method for incremental training using *mini-batches*.\n",
    "\n",
    "In the following example, we will use the implementation of `GaussianNB` from `river`.\n",
    "\n",
    "In `river`, training and predicting are perfomed via the `learn_one` and `predict_one` methods."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [],
   "source": [
    "from river.naive_bayes import GaussianNB\n",
    "\n",
    "model = GaussianNB()\n",
    "for xi, yi in zip(X_train, y_train):\n",
    "    x = dict(zip(iris.feature_names, xi))\n",
    "    model.learn_one(x=x, y=yi)              # Train the model\n",
    "\n",
    "y_pred = []\n",
    "for xi in X_test:\n",
    "    x = dict(zip(iris.feature_names, xi))\n",
    "    y_pred.append(model.predict_one(x=x))   # Predict class-label"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The model's predictions are in `y_pred`. Again, we measure the accuracy of the model by comparing predictions to the ground truth:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model accuracy: 0.9111\n"
     ]
    }
   ],
   "source": [
    "print(f'Model accuracy: {accuracy_score(y_test, y_pred):.4f}')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Hurray! We got the same performance from the model trained one sample at a time...\n",
    "\n",
    "But wait, this is NOT different from the \"batch\" setting. We are still doing \"training\" and then \"testing\" using two \"for\" loops.\n",
    "\n",
    "In the streaming setting, training and predicting happen inside a single loop. Remember, a data sample is available only once in the stream.\n",
    "\n",
    "To simplify our code, we can use the `iter_sklearn` method to convert a `scikit-learn` dataset object into an iterable that returns the data as `dict`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model accuracy: 0.9195\n"
     ]
    }
   ],
   "source": [
    "from river.stream import iter_sklearn_dataset\n",
    "\n",
    "y_pred = []\n",
    "y_true = []\n",
    "\n",
    "model = GaussianNB()\n",
    "\n",
    "for x, y in iter_sklearn_dataset(iris, shuffle=True, seed=42):\n",
    "   y_p = model.predict_one(x)   # Predict class-label\n",
    "   model.learn_one(x, y)        # Train the model\n",
    "   if y_p is None:              # Process next sample if the model is empty\n",
    "       continue\n",
    "   y_pred.append(y_p)\n",
    "   y_true.append(y)\n",
    "\n",
    "print(f'Model accuracy: {accuracy_score(y_true, y_pred):.4f}')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This time we have successfully implemented a stream learning task. This is known as **prequential evaluation** or **interleaved test-then-train** evaluation. \n",
    "\n",
    "Notice the *order* of the training and prediction operations. When a new data sample ($x_i$, $y_i$) arrives:\n",
    "\n",
    "1. Predict the label ($y_i'$) for the new data sample ($x_i$), not \"seen\" by the model yet\n",
    "2. Train the model using the new sample ($x_i$, $y_i$)\n",
    "\n",
    "This **order matters**!\n",
    "\n",
    "A benefit from this approach is that we can use *all* labeled samples in the stream to measure a model's performance. Notice that the model provides predictions at any point in the stream, depending on its current internal state. Batch methods can only predict after they finished training."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the previous code snippet, we used two auxiliary variables to store the true labels and the predicted labels. This is because `accuracy_score` requires access to all the data. However, storing all the true and predicted labels from a data stream is *impractical*.\n",
    "\n",
    "In `river`, all methods can be updated one single sample at a time, including performance metrics! Let's see how to update a metric."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Accuracy: 91.95%"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from river.metrics import Accuracy\n",
    "\n",
    "metric = Accuracy()\n",
    "for yt, yp in zip(y_true, y_pred):\n",
    "    metric.update(y_true=yt, y_pred=yp)\n",
    "metric"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `update` method takes true and predicted pairs and updates the metric. We can compare the confusion matrices:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Confusion matrix from all data:\n",
      "[[48  1  0]\n",
      " [ 2 46  2]\n",
      " [ 0  7 43]]\n",
      "Confusion matrix from updatable metric:\n",
      "           0     1     2\n",
      "     0    48     1     0\n",
      "     1     2    46     2\n",
      "     2     0     7    43\n"
     ]
    }
   ],
   "source": [
    "print('Confusion matrix from all data:')\n",
    "print(confusion_matrix(y_true, y_pred))\n",
    "print('Confusion matrix from updatable metric:')\n",
    "print(metric.cm)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we can rewrite our example."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "149 samples analyzed.\n",
      "Accuracy: 91.95%\n"
     ]
    }
   ],
   "source": [
    "model = GaussianNB()\n",
    "metric = Accuracy()\n",
    "\n",
    "for x, y in iter_sklearn_dataset(iris, shuffle=True, seed=42):\n",
    "   y_p = model.predict_one(x)   # Predict class\n",
    "   if y_p is not None:\n",
    "       metric.update(y_true=y, y_pred=y_p)\n",
    "   model.learn_one(x, y)        # Train the model\n",
    "\n",
    "print(f'{len(y_pred)} samples analyzed.')   \n",
    "print(metric)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### The `progressive_val_score` method\n",
    "\n",
    "For convenience, the `progressive_val_score` method implements the prequential evaluation. It requires three components: \n",
    "\n",
    "- a data stream\n",
    "- a model\n",
    "- a metric"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[10] Accuracy: 44.44%\n",
      "[20] Accuracy: 73.68%\n",
      "[30] Accuracy: 82.76%\n",
      "[40] Accuracy: 84.62%\n",
      "[50] Accuracy: 85.71%\n",
      "[60] Accuracy: 88.14%\n",
      "[70] Accuracy: 88.41%\n",
      "[80] Accuracy: 89.87%\n",
      "[90] Accuracy: 89.89%\n",
      "[100] Accuracy: 90.91%\n",
      "[110] Accuracy: 91.74%\n",
      "[120] Accuracy: 92.44%\n",
      "[130] Accuracy: 92.25%\n",
      "[140] Accuracy: 92.81%\n",
      "[150] Accuracy: 91.95%\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "Accuracy: 91.95%"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from river.evaluate import progressive_val_score\n",
    "\n",
    "# Setup stream and estimators\n",
    "stream = iter_sklearn_dataset(iris, shuffle=True, seed=42)\n",
    "nb = GaussianNB()\n",
    "metric = Accuracy()\n",
    "\n",
    "# Setup evaluator\n",
    "progressive_val_score(dataset=stream, model=nb, metric=metric, print_every=10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Decision trees for data streams\n",
    "---\n",
    "\n",
    "### `NEWeather` dataset\n",
    "\n",
    "**Description:** The National Oceanic and Atmospheric Administration (NOAA),\n",
    "has compiled a database of weather measurements from over 7,000 weather \n",
    "stations worldwide. Records date back to the mid-1900s. Daily measurements\n",
    "include a variety of features (temperature, pressure, wind speed, etc.) as\n",
    "well as a series of indicators for precipitation and other weather-related\n",
    "events. The `NEweather` dataset contains data from this database, specifically\n",
    "from the Offutt Air Force Base in Bellevue, Nebraska ranging for over 50 years\n",
    "(1949-1999).\n",
    "\n",
    "**Features:** 8 Daily weather measurements\n",
    " \n",
    "|       Attribute      | Description |\n",
    "|:--------------------:|:-----------------------------|\n",
    "| `temp`                   | Temperature\n",
    "| `dew_pnt`                | Dew Point\n",
    "| `sea_lvl_press`          | Sea Level Pressure\n",
    "| `visibility`             | Visibility\n",
    "| `avg_wind_spd`           | Average Wind Speed\n",
    "| `max_sustained_wind_spd` | Maximum Sustained Wind Speed\n",
    "| `max_temp`               | Maximum Temperature\n",
    "| `min_temp`               | Minimum Temperature\n",
    "\n",
    "\n",
    "**Class:** `rain` | 0: no rain, 1: rain\n",
    " \n",
    "**Samples:** 18,159\n",
    "\n",
    "---\n",
    "\n",
    "In this example, we load the data from a csv file with `pandas.read_csv`. \n",
    "\n",
    "Next, we use the `iter_pandas` utility method to iterate over the `DataFrame`. The model in this example is a `HoeffdingTreeClassifier`, a type of decision tree designed for data streams."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[1,000] Accuracy: 71.57%\n",
      "[2,000] Accuracy: 71.54%\n",
      "[3,000] Accuracy: 71.86%\n",
      "[4,000] Accuracy: 72.97%\n",
      "[5,000] Accuracy: 73.13%\n",
      "[6,000] Accuracy: 73.21%\n",
      "[7,000] Accuracy: 73.88%\n",
      "[8,000] Accuracy: 73.57%\n",
      "[9,000] Accuracy: 73.70%\n",
      "[10,000] Accuracy: 73.77%\n",
      "[11,000] Accuracy: 74.15%\n",
      "[12,000] Accuracy: 74.36%\n",
      "[13,000] Accuracy: 74.53%\n",
      "[14,000] Accuracy: 74.38%\n",
      "[15,000] Accuracy: 74.57%\n",
      "[16,000] Accuracy: 74.57%\n",
      "[17,000] Accuracy: 74.46%\n",
      "[18,000] Accuracy: 74.17%\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "Accuracy: 74.15%"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "\n",
    "from river.stream import iter_pandas\n",
    "from river.tree import HoeffdingTreeClassifier\n",
    "\n",
    "data = pd.read_csv(\"NEweather.csv\")\n",
    "features = data.columns[:-2]\n",
    "stream = iter_pandas(X=data[features], y=data['rain'])\n",
    "\n",
    "model = HoeffdingTreeClassifier()\n",
    "metric = Accuracy()\n",
    "\n",
    "progressive_val_score(dataset=stream,\n",
    "                      model=model,\n",
    "                      metric=metric,\n",
    "                      print_every=1000)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Tree-based models are popular due to their interpretability. They use a tree data structure to model the data. When a sample arrives, it traverses the tree until it reaches a leaf node. Internal nodes define the path for a data sample based on the values of its features. Leaf nodes are models that provide predictions for unlabeled-samples and can update their internal state using the labels from labeled samples."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/svg+xml": [
       "<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?>\n",
       "<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
       " \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
       "<!-- Generated by graphviz version 2.44.1 (20200629.0846)\n",
       " -->\n",
       "<!-- Pages: 1 -->\n",
       "<svg width=\"410pt\" height=\"368pt\"\n",
       " viewBox=\"0.00 0.00 410.00 368.00\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n",
       "<g id=\"graph0\" class=\"graph\" transform=\"scale(1 1) rotate(0) translate(4 364)\">\n",
       "<polygon fill=\"white\" stroke=\"transparent\" points=\"-4,4 -4,-364 406,-364 406,4 -4,4\"/>\n",
       "<!-- 0 -->\n",
       "<g id=\"node1\" class=\"node\">\n",
       "<title>0</title>\n",
       "<polygon fill=\"#f4caac\" stroke=\"black\" stroke-width=\"1.2\" points=\"200,-360 144,-360 144,-324 200,-324 200,-360\"/>\n",
       "<text text-anchor=\"middle\" x=\"172\" y=\"-339.2\" font-family=\"trebuchet\" font-size=\"11.00\">dew_pnt</text>\n",
       "</g>\n",
       "<!-- 1 -->\n",
       "<g id=\"node2\" class=\"node\">\n",
       "<title>1</title>\n",
       "<polygon fill=\"#eb9f67\" stroke=\"black\" stroke-width=\"1.2\" points=\"157,-288 101,-288 101,-252 157,-252 157,-288\"/>\n",
       "<text text-anchor=\"middle\" x=\"129\" y=\"-267.2\" font-family=\"trebuchet\" font-size=\"11.00\">dew_pnt</text>\n",
       "</g>\n",
       "<!-- 0&#45;&gt;1 -->\n",
       "<g id=\"edge1\" class=\"edge\">\n",
       "<title>0&#45;&gt;1</title>\n",
       "<path fill=\"none\" stroke=\"black\" stroke-width=\"0.6\" d=\"M150.5,-323.83C150.5,-323.83 150.5,-298.41 150.5,-298.41\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" stroke-width=\"0.6\" points=\"154,-298.41 150.5,-288.41 147,-298.41 154,-298.41\"/>\n",
       "<text text-anchor=\"middle\" x=\"139.5\" y=\"-313.52\" font-family=\"Times,serif\" font-size=\"7.00\">&lt;= 31.0</text>\n",
       "</g>\n",
       "<!-- 6 -->\n",
       "<g id=\"node7\" class=\"node\">\n",
       "<title>6</title>\n",
       "<polygon fill=\"#f9e2d2\" stroke=\"black\" stroke-width=\"1.2\" points=\"243,-288 189,-288 189,-252 243,-252 243,-288\"/>\n",
       "<text text-anchor=\"middle\" x=\"216\" y=\"-267.2\" font-family=\"trebuchet\" font-size=\"11.00\">visibility</text>\n",
       "</g>\n",
       "<!-- 0&#45;&gt;6 -->\n",
       "<g id=\"edge6\" class=\"edge\">\n",
       "<title>0&#45;&gt;6</title>\n",
       "<path fill=\"none\" stroke=\"black\" stroke-width=\"0.6\" d=\"M194.5,-323.83C194.5,-323.83 194.5,-298.41 194.5,-298.41\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" stroke-width=\"0.6\" points=\"198,-298.41 194.5,-288.41 191,-298.41 198,-298.41\"/>\n",
       "<text text-anchor=\"middle\" x=\"185\" y=\"-313.52\" font-family=\"Times,serif\" font-size=\"7.00\">&gt; 31.0</text>\n",
       "</g>\n",
       "<!-- 2 -->\n",
       "<g id=\"node3\" class=\"node\">\n",
       "<title>2</title>\n",
       "<polygon fill=\"#e78a47\" stroke=\"black\" stroke-width=\"1.2\" points=\"84,-216 0,-216 0,-168 84,-168 84,-216\"/>\n",
       "<text text-anchor=\"middle\" x=\"42\" y=\"-207.2\" font-family=\"trebuchet\" font-size=\"11.00\">0</text>\n",
       "<text text-anchor=\"middle\" x=\"42\" y=\"-195.2\" font-family=\"trebuchet\" font-size=\"11.00\">P(0) = 0.96</text>\n",
       "<text text-anchor=\"middle\" x=\"42\" y=\"-183.2\" font-family=\"trebuchet\" font-size=\"11.00\">P(1) = 0.036</text>\n",
       "<text text-anchor=\"middle\" x=\"42\" y=\"-171.2\" font-family=\"trebuchet\" font-size=\"11.00\">samples: 1921</text>\n",
       "</g>\n",
       "<!-- 1&#45;&gt;2 -->\n",
       "<g id=\"edge2\" class=\"edge\">\n",
       "<title>1&#45;&gt;2</title>\n",
       "<path fill=\"none\" stroke=\"black\" stroke-width=\"0.6\" d=\"M100.99,-270C75.56,-270 42,-270 42,-270 42,-270 42,-226.25 42,-226.25\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" stroke-width=\"0.6\" points=\"45.5,-226.25 42,-216.25 38.5,-226.25 45.5,-226.25\"/>\n",
       "<text text-anchor=\"middle\" x=\"38.62\" y=\"-272.4\" font-family=\"Times,serif\" font-size=\"7.00\">&lt;= 16.0</text>\n",
       "</g>\n",
       "<!-- 3 -->\n",
       "<g id=\"node4\" class=\"node\">\n",
       "<title>3</title>\n",
       "<polygon fill=\"#efb489\" stroke=\"black\" stroke-width=\"1.2\" points=\"156,-210 102,-210 102,-174 156,-174 156,-210\"/>\n",
       "<text text-anchor=\"middle\" x=\"129\" y=\"-189.2\" font-family=\"trebuchet\" font-size=\"11.00\">visibility</text>\n",
       "</g>\n",
       "<!-- 1&#45;&gt;3 -->\n",
       "<g id=\"edge3\" class=\"edge\">\n",
       "<title>1&#45;&gt;3</title>\n",
       "<path fill=\"none\" stroke=\"black\" stroke-width=\"0.6\" d=\"M129,-251.91C129,-251.91 129,-220.26 129,-220.26\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" stroke-width=\"0.6\" points=\"132.5,-220.26 129,-210.26 125.5,-220.26 132.5,-220.26\"/>\n",
       "<text text-anchor=\"middle\" x=\"119.5\" y=\"-238.49\" font-family=\"Times,serif\" font-size=\"7.00\">&gt; 16.0</text>\n",
       "</g>\n",
       "<!-- 4 -->\n",
       "<g id=\"node5\" class=\"node\">\n",
       "<title>4</title>\n",
       "<polygon fill=\"#f7d8c1\" stroke=\"black\" stroke-width=\"1.2\" points=\"98,-132 20,-132 20,-84 98,-84 98,-132\"/>\n",
       "<text text-anchor=\"middle\" x=\"59\" y=\"-123.2\" font-family=\"trebuchet\" font-size=\"11.00\">0</text>\n",
       "<text text-anchor=\"middle\" x=\"59\" y=\"-111.2\" font-family=\"trebuchet\" font-size=\"11.00\">P(0) = 0.66</text>\n",
       "<text text-anchor=\"middle\" x=\"59\" y=\"-99.2\" font-family=\"trebuchet\" font-size=\"11.00\">P(1) = 0.34</text>\n",
       "<text text-anchor=\"middle\" x=\"59\" y=\"-87.2\" font-family=\"trebuchet\" font-size=\"11.00\">samples: 577</text>\n",
       "</g>\n",
       "<!-- 3&#45;&gt;4 -->\n",
       "<g id=\"edge4\" class=\"edge\">\n",
       "<title>3&#45;&gt;4</title>\n",
       "<path fill=\"none\" stroke=\"black\" stroke-width=\"0.6\" d=\"M101.79,-192C95.7,-192 91,-192 91,-192 91,-192 91,-142.3 91,-142.3\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" stroke-width=\"0.6\" points=\"94.5,-142.3 91,-132.3 87.5,-142.3 94.5,-142.3\"/>\n",
       "<text text-anchor=\"middle\" x=\"100.5\" y=\"-166.94\" font-family=\"Times,serif\" font-size=\"7.00\">&lt;= 9.1</text>\n",
       "</g>\n",
       "<!-- 5 -->\n",
       "<g id=\"node6\" class=\"node\">\n",
       "<title>5</title>\n",
       "<polygon fill=\"#eca16c\" stroke=\"black\" stroke-width=\"1.2\" points=\"200,-132 116,-132 116,-84 200,-84 200,-132\"/>\n",
       "<text text-anchor=\"middle\" x=\"158\" y=\"-123.2\" font-family=\"trebuchet\" font-size=\"11.00\">0</text>\n",
       "<text text-anchor=\"middle\" x=\"158\" y=\"-111.2\" font-family=\"trebuchet\" font-size=\"11.00\">P(0) = 0.87</text>\n",
       "<text text-anchor=\"middle\" x=\"158\" y=\"-99.2\" font-family=\"trebuchet\" font-size=\"11.00\">P(1) = 0.13</text>\n",
       "<text text-anchor=\"middle\" x=\"158\" y=\"-87.2\" font-family=\"trebuchet\" font-size=\"11.00\">samples: 1537</text>\n",
       "</g>\n",
       "<!-- 3&#45;&gt;5 -->\n",
       "<g id=\"edge5\" class=\"edge\">\n",
       "<title>3&#45;&gt;5</title>\n",
       "<path fill=\"none\" stroke=\"black\" stroke-width=\"0.6\" d=\"M136,-173.77C136,-173.77 136,-142.25 136,-142.25\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" stroke-width=\"0.6\" points=\"139.5,-142.25 136,-132.25 132.5,-142.25 139.5,-142.25\"/>\n",
       "<text text-anchor=\"middle\" x=\"128.5\" y=\"-160.41\" font-family=\"Times,serif\" font-size=\"7.00\">&gt; 9.1</text>\n",
       "</g>\n",
       "<!-- 7 -->\n",
       "<g id=\"node8\" class=\"node\">\n",
       "<title>7</title>\n",
       "<polygon fill=\"#d1e8f9\" stroke=\"black\" stroke-width=\"1.2\" points=\"258,-216 174,-216 174,-168 258,-168 258,-216\"/>\n",
       "<text text-anchor=\"middle\" x=\"216\" y=\"-207.2\" font-family=\"trebuchet\" font-size=\"11.00\">1</text>\n",
       "<text text-anchor=\"middle\" x=\"216\" y=\"-195.2\" font-family=\"trebuchet\" font-size=\"11.00\">P(1) = 0.62</text>\n",
       "<text text-anchor=\"middle\" x=\"216\" y=\"-183.2\" font-family=\"trebuchet\" font-size=\"11.00\">P(0) = 0.38</text>\n",
       "<text text-anchor=\"middle\" x=\"216\" y=\"-171.2\" font-family=\"trebuchet\" font-size=\"11.00\">samples: 2496</text>\n",
       "</g>\n",
       "<!-- 6&#45;&gt;7 -->\n",
       "<g id=\"edge7\" class=\"edge\">\n",
       "<title>6&#45;&gt;7</title>\n",
       "<path fill=\"none\" stroke=\"black\" stroke-width=\"0.6\" d=\"M216,-251.91C216,-251.91 216,-226.2 216,-226.2\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" stroke-width=\"0.6\" points=\"219.5,-226.2 216,-216.2 212.5,-226.2 219.5,-226.2\"/>\n",
       "<text text-anchor=\"middle\" x=\"206.5\" y=\"-241.45\" font-family=\"Times,serif\" font-size=\"7.00\">&lt;= 9.7</text>\n",
       "</g>\n",
       "<!-- 8 -->\n",
       "<g id=\"node9\" class=\"node\">\n",
       "<title>8</title>\n",
       "<polygon fill=\"#f6d6be\" stroke=\"black\" stroke-width=\"1.2\" points=\"360,-210 276,-210 276,-174 360,-174 360,-210\"/>\n",
       "<text text-anchor=\"middle\" x=\"318\" y=\"-189.2\" font-family=\"trebuchet\" font-size=\"11.00\">avg_wind_spd</text>\n",
       "</g>\n",
       "<!-- 6&#45;&gt;8 -->\n",
       "<g id=\"edge8\" class=\"edge\">\n",
       "<title>6&#45;&gt;8</title>\n",
       "<path fill=\"none\" stroke=\"black\" stroke-width=\"0.6\" d=\"M243.07,-270C273.28,-270 318,-270 318,-270 318,-270 318,-220.09 318,-220.09\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" stroke-width=\"0.6\" points=\"321.5,-220.09 318,-210.09 314.5,-220.09 321.5,-220.09\"/>\n",
       "<text text-anchor=\"middle\" x=\"297.99\" y=\"-272.4\" font-family=\"Times,serif\" font-size=\"7.00\">&gt; 9.7</text>\n",
       "</g>\n",
       "<!-- 9 -->\n",
       "<g id=\"node10\" class=\"node\">\n",
       "<title>9</title>\n",
       "<polygon fill=\"#f4c9aa\" stroke=\"black\" stroke-width=\"1.2\" points=\"300,-126 246,-126 246,-90 300,-90 300,-126\"/>\n",
       "<text text-anchor=\"middle\" x=\"273\" y=\"-105.2\" font-family=\"trebuchet\" font-size=\"11.00\">visibility</text>\n",
       "</g>\n",
       "<!-- 8&#45;&gt;9 -->\n",
       "<g id=\"edge9\" class=\"edge\">\n",
       "<title>8&#45;&gt;9</title>\n",
       "<path fill=\"none\" stroke=\"black\" stroke-width=\"0.6\" d=\"M288,-173.77C288,-173.77 288,-136.19 288,-136.19\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" stroke-width=\"0.6\" points=\"291.5,-136.19 288,-126.19 284.5,-136.19 291.5,-136.19\"/>\n",
       "<text text-anchor=\"middle\" x=\"278.5\" y=\"-157.38\" font-family=\"Times,serif\" font-size=\"7.00\">&lt;= 8.2</text>\n",
       "</g>\n",
       "<!-- 12 -->\n",
       "<g id=\"node13\" class=\"node\">\n",
       "<title>12</title>\n",
       "<polygon fill=\"#fdf7f2\" stroke=\"black\" stroke-width=\"1.2\" points=\"402,-132 318,-132 318,-84 402,-84 402,-132\"/>\n",
       "<text text-anchor=\"middle\" x=\"360\" y=\"-123.2\" font-family=\"trebuchet\" font-size=\"11.00\">0</text>\n",
       "<text text-anchor=\"middle\" x=\"360\" y=\"-111.2\" font-family=\"trebuchet\" font-size=\"11.00\">P(0) = 0.53</text>\n",
       "<text text-anchor=\"middle\" x=\"360\" y=\"-99.2\" font-family=\"trebuchet\" font-size=\"11.00\">P(1) = 0.47</text>\n",
       "<text text-anchor=\"middle\" x=\"360\" y=\"-87.2\" font-family=\"trebuchet\" font-size=\"11.00\">samples: 1526</text>\n",
       "</g>\n",
       "<!-- 8&#45;&gt;12 -->\n",
       "<g id=\"edge12\" class=\"edge\">\n",
       "<title>8&#45;&gt;12</title>\n",
       "<path fill=\"none\" stroke=\"black\" stroke-width=\"0.6\" d=\"M339,-173.77C339,-173.77 339,-142.25 339,-142.25\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" stroke-width=\"0.6\" points=\"342.5,-142.25 339,-132.25 335.5,-142.25 342.5,-142.25\"/>\n",
       "<text text-anchor=\"middle\" x=\"331.5\" y=\"-160.41\" font-family=\"Times,serif\" font-size=\"7.00\">&gt; 8.2</text>\n",
       "</g>\n",
       "<!-- 10 -->\n",
       "<g id=\"node11\" class=\"node\">\n",
       "<title>10</title>\n",
       "<polygon fill=\"#fdf4ee\" stroke=\"black\" stroke-width=\"1.2\" points=\"264,-48 180,-48 180,0 264,0 264,-48\"/>\n",
       "<text text-anchor=\"middle\" x=\"222\" y=\"-39.2\" font-family=\"trebuchet\" font-size=\"11.00\">0</text>\n",
       "<text text-anchor=\"middle\" x=\"222\" y=\"-27.2\" font-family=\"trebuchet\" font-size=\"11.00\">P(1) = 0.46</text>\n",
       "<text text-anchor=\"middle\" x=\"222\" y=\"-15.2\" font-family=\"trebuchet\" font-size=\"11.00\">P(0) = 0.54</text>\n",
       "<text text-anchor=\"middle\" x=\"222\" y=\"-3.2\" font-family=\"trebuchet\" font-size=\"11.00\">samples: 1502</text>\n",
       "</g>\n",
       "<!-- 9&#45;&gt;10 -->\n",
       "<g id=\"edge10\" class=\"edge\">\n",
       "<title>9&#45;&gt;10</title>\n",
       "<path fill=\"none\" stroke=\"black\" stroke-width=\"0.6\" d=\"M255,-89.77C255,-89.77 255,-58.25 255,-58.25\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" stroke-width=\"0.6\" points=\"258.5,-58.25 255,-48.25 251.5,-58.25 258.5,-58.25\"/>\n",
       "<text text-anchor=\"middle\" x=\"244\" y=\"-76.41\" font-family=\"Times,serif\" font-size=\"7.00\">&lt;= 15.0</text>\n",
       "</g>\n",
       "<!-- 11 -->\n",
       "<g id=\"node12\" class=\"node\">\n",
       "<title>11</title>\n",
       "<polygon fill=\"#efb286\" stroke=\"black\" stroke-width=\"1.2\" points=\"366,-48 282,-48 282,0 366,0 366,-48\"/>\n",
       "<text text-anchor=\"middle\" x=\"324\" y=\"-39.2\" font-family=\"trebuchet\" font-size=\"11.00\">0</text>\n",
       "<text text-anchor=\"middle\" x=\"324\" y=\"-27.2\" font-family=\"trebuchet\" font-size=\"11.00\">P(1) = 0.19</text>\n",
       "<text text-anchor=\"middle\" x=\"324\" y=\"-15.2\" font-family=\"trebuchet\" font-size=\"11.00\">P(0) = 0.81</text>\n",
       "<text text-anchor=\"middle\" x=\"324\" y=\"-3.2\" font-family=\"trebuchet\" font-size=\"11.00\">samples: 1346</text>\n",
       "</g>\n",
       "<!-- 9&#45;&gt;11 -->\n",
       "<g id=\"edge11\" class=\"edge\">\n",
       "<title>9&#45;&gt;11</title>\n",
       "<path fill=\"none\" stroke=\"black\" stroke-width=\"0.6\" d=\"M291,-89.77C291,-89.77 291,-58.25 291,-58.25\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" stroke-width=\"0.6\" points=\"294.5,-58.25 291,-48.25 287.5,-58.25 294.5,-58.25\"/>\n",
       "<text text-anchor=\"middle\" x=\"281.5\" y=\"-76.41\" font-family=\"Times,serif\" font-size=\"7.00\">&gt; 15.0</text>\n",
       "</g>\n",
       "</g>\n",
       "</svg>\n"
      ],
      "text/plain": [
       "<graphviz.dot.Digraph at 0x7fcf537194f0>"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.draw()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Pipelines\n",
    "---\n",
    "\n",
    "Pipelines are powerful tools that \"chain\" a sequence of operations. \n",
    "\n",
    "For example, logistic regression is sensitive to the scale of the features. Rescaling the data so that each feature has mean $\\mu=0$ and variance $\\sigma^2=1$ is generally considered good practice.\n",
    "\n",
    "We can apply the rescaling and fit the logistic regression sequentially in an elegant manner using a Pipeline.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[1,000] Accuracy: 74.80%\n",
      "[2,000] Accuracy: 76.30%\n",
      "[3,000] Accuracy: 77.63%\n",
      "[4,000] Accuracy: 76.80%\n",
      "[5,000] Accuracy: 76.60%\n",
      "[6,000] Accuracy: 76.48%\n",
      "[7,000] Accuracy: 76.97%\n",
      "[8,000] Accuracy: 76.40%\n",
      "[9,000] Accuracy: 76.27%\n",
      "[10,000] Accuracy: 76.28%\n",
      "[11,000] Accuracy: 76.50%\n",
      "[12,000] Accuracy: 76.73%\n",
      "[13,000] Accuracy: 76.50%\n",
      "[14,000] Accuracy: 76.34%\n",
      "[15,000] Accuracy: 76.43%\n",
      "[16,000] Accuracy: 76.49%\n",
      "[17,000] Accuracy: 76.29%\n",
      "[18,000] Accuracy: 76.20%\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "Accuracy: 76.23%"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from river import linear_model\n",
    "from river import optim\n",
    "from river import preprocessing\n",
    "from river import compose\n",
    "\n",
    "data = pd.read_csv(\"NEweather.csv\")\n",
    "features = data.columns[:-2]\n",
    "stream = iter_pandas(X=data[features], y=data['rain'])\n",
    "\n",
    "scaler = preprocessing.StandardScaler()\n",
    "optimizer = optim.SGD(lr=0.01)\n",
    "model = compose.Pipeline(\n",
    "    ('scale', preprocessing.StandardScaler()),\n",
    "    ('log_reg', linear_model.LogisticRegression())\n",
    ")\n",
    "\n",
    "metric = Accuracy()\n",
    "\n",
    "progressive_val_score(dataset=stream,\n",
    "                      model=model,\n",
    "                      metric=metric,\n",
    "                      print_every=1000)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `draw` method is useful to visualize the above pipeline."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/svg+xml": [
       "<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?>\n",
       "<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
       " \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
       "<!-- Generated by graphviz version 2.44.1 (20200629.0846)\n",
       " -->\n",
       "<!-- Pages: 1 -->\n",
       "<svg width=\"165pt\" height=\"260pt\"\n",
       " viewBox=\"0.00 0.00 165.29 260.00\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n",
       "<g id=\"graph0\" class=\"graph\" transform=\"scale(1 1) rotate(0) translate(4 256)\">\n",
       "<polygon fill=\"white\" stroke=\"transparent\" points=\"-4,4 -4,-256 161.29,-256 161.29,4 -4,4\"/>\n",
       "<!-- x -->\n",
       "<g id=\"node1\" class=\"node\">\n",
       "<title>x</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"78.64\" cy=\"-234\" rx=\"27\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"78.64\" y=\"-230.3\" font-family=\"Times,serif\" font-size=\"14.00\">x</text>\n",
       "</g>\n",
       "<!-- StandardScaler -->\n",
       "<g id=\"node2\" class=\"node\">\n",
       "<title>StandardScaler</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"78.64\" cy=\"-162\" rx=\"63.89\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"78.64\" y=\"-158.3\" font-family=\"Times,serif\" font-size=\"14.00\">StandardScaler</text>\n",
       "</g>\n",
       "<!-- x&#45;&gt;StandardScaler -->\n",
       "<g id=\"edge1\" class=\"edge\">\n",
       "<title>x&#45;&gt;StandardScaler</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M78.64,-215.7C78.64,-207.98 78.64,-198.71 78.64,-190.11\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"82.14,-190.1 78.64,-180.1 75.14,-190.1 82.14,-190.1\"/>\n",
       "</g>\n",
       "<!-- LogisticRegression -->\n",
       "<g id=\"node3\" class=\"node\">\n",
       "<title>LogisticRegression</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"78.64\" cy=\"-90\" rx=\"78.79\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"78.64\" y=\"-86.3\" font-family=\"Times,serif\" font-size=\"14.00\">LogisticRegression</text>\n",
       "</g>\n",
       "<!-- StandardScaler&#45;&gt;LogisticRegression -->\n",
       "<g id=\"edge2\" class=\"edge\">\n",
       "<title>StandardScaler&#45;&gt;LogisticRegression</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M78.64,-143.7C78.64,-135.98 78.64,-126.71 78.64,-118.11\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"82.14,-118.1 78.64,-108.1 75.14,-118.1 82.14,-118.1\"/>\n",
       "</g>\n",
       "<!-- y -->\n",
       "<g id=\"node4\" class=\"node\">\n",
       "<title>y</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"78.64\" cy=\"-18\" rx=\"27\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"78.64\" y=\"-14.3\" font-family=\"Times,serif\" font-size=\"14.00\">y</text>\n",
       "</g>\n",
       "<!-- LogisticRegression&#45;&gt;y -->\n",
       "<g id=\"edge3\" class=\"edge\">\n",
       "<title>LogisticRegression&#45;&gt;y</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M78.64,-71.7C78.64,-63.98 78.64,-54.71 78.64,-46.11\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"82.14,-46.1 78.64,-36.1 75.14,-46.1 82.14,-46.1\"/>\n",
       "</g>\n",
       "</g>\n",
       "</svg>\n"
      ],
      "text/plain": [
       "<graphviz.dot.Digraph at 0x7fcf553e83d0>"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.draw()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Regression\n",
    "---\n",
    "\n",
    "### Bike Sharing Dataset\n",
    "\n",
    "Contains the **hourly and daily count of rental bikes** between the years 2011 and 2012 in the Capital bike-share system with the corresponding weather and seasonal information. Attributes include weather, temperature, date, time, etc.\n",
    "\n",
    "|      Attribute      | Description |\n",
    "|:-------------------:|:-----------------------------------------------------------------|\n",
    "| `instant`             | record index\n",
    "| `dteday`              | date\n",
    "| `season`              | 1:winter<br>2:spring<br>3:summer<br>4:fall)\n",
    "| `yr`                  | year (0: 2011, 1:2012)\n",
    "| `mnth`                | month ( 1 to 12)\n",
    "| `hr`                  | hour (0 to 23)\n",
    "| `holiday`             | weather day is holiday or not\n",
    "| `weekday`             | day of the week\n",
    "| `workingday`          | if day is neither weekend nor holiday is 1, otherwise is 0.\n",
    "| `weathersit`          | 1: Clear, Few clouds, Partly cloudy, Partly cloudy<br>2: Mist + Cloudy, Mist + Broken clouds, Mist + Few clouds, Mist<br>3: Light Snow, Light Rain + Thunderstorm + Scattered clouds, Light Rain + Scattered clouds<br>4: Heavy Rain + Ice Pallets + Thunderstorm + Mist, Snow + Fog\n",
    "| `temp`                | Normalized temperature in Celsius.\n",
    "| `atemp`               | Normalized feeling temperature in Celsius.\n",
    "| `hum`                 | Normalized humidity.\n",
    "| `windspeed`           | Normalized wind speed.\n",
    "| `casual`              | count of casual users\n",
    "| `registered`          | count of registered users\n",
    "\n",
    "**Target:** `cnt` | number of bikes rented including both casual and registered\n",
    " \n",
    "**Samples:** 17,389"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "from river import metrics\n",
    "from river import neighbors\n",
    "\n",
    "data = pd.read_csv(\"bike.csv\")\n",
    "features = data.columns[:-2]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For this example we will use the k-nearest neighbor algorithm. The target value for a query sample is estimated by aggregating the target values of the k-neighbors on a sliding window."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[1,000] R2: 0.569228\n",
      "[2,000] R2: 0.647591\n",
      "[3,000] R2: 0.717623\n",
      "[4,000] R2: 0.748093\n",
      "[5,000] R2: 0.760055\n",
      "[6,000] R2: 0.746103\n",
      "[7,000] R2: 0.737243\n",
      "[8,000] R2: 0.730013\n",
      "[9,000] R2: 0.723937\n",
      "[10,000] R2: 0.719661\n",
      "[11,000] R2: 0.731367\n",
      "[12,000] R2: 0.737862\n",
      "[13,000] R2: 0.74638\n",
      "[14,000] R2: 0.748483\n",
      "[15,000] R2: 0.751023\n",
      "[16,000] R2: 0.750326\n",
      "[17,000] R2: 0.746482\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "R2: 0.74682"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "stream = iter_pandas(X=data[features], y=data['cnt'])\n",
    "\n",
    "model = neighbors.KNNRegressor(n_neighbors=3, window_size=100)\n",
    "\n",
    "metric = metrics.R2()\n",
    "\n",
    "progressive_val_score(dataset=stream,\n",
    "                      model=model,\n",
    "                      metric=metric,\n",
    "                      print_every=1000)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The implementation of the k-nearest neighbor algorithm assumes that all the features are numeric.\n",
    "\n",
    "In the `bike` data, there are some categorical features. We can easily filter them in a pipeline:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[1,000] R2: 0.723243\n",
      "[2,000] R2: 0.744373\n",
      "[3,000] R2: 0.77689\n",
      "[4,000] R2: 0.823322\n",
      "[5,000] R2: 0.832047\n",
      "[6,000] R2: 0.819849\n",
      "[7,000] R2: 0.81477\n",
      "[8,000] R2: 0.80626\n",
      "[9,000] R2: 0.800177\n",
      "[10,000] R2: 0.79539\n",
      "[11,000] R2: 0.803151\n",
      "[12,000] R2: 0.801147\n",
      "[13,000] R2: 0.806475\n",
      "[14,000] R2: 0.814549\n",
      "[15,000] R2: 0.8178\n",
      "[16,000] R2: 0.815114\n",
      "[17,000] R2: 0.813029\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "R2: 0.812576"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "stream = iter_pandas(X=data[features], y=data['cnt'])\n",
    "\n",
    "model = (\n",
    "    compose.Discard('season', 'mnth', 'weekday', 'weathersit') |\n",
    "    neighbors.KNNRegressor(n_neighbors=3, window_size=100)\n",
    ")\n",
    "\n",
    "metric = metrics.R2()\n",
    "\n",
    "progressive_val_score(dataset=stream,\n",
    "                      model=model,\n",
    "                      metric=metric,\n",
    "                      print_every=1000)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/svg+xml": [
       "<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?>\n",
       "<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
       " \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
       "<!-- Generated by graphviz version 2.44.1 (20200629.0846)\n",
       " -->\n",
       "<!-- Pages: 1 -->\n",
       "<svg width=\"323pt\" height=\"260pt\"\n",
       " viewBox=\"0.00 0.00 322.57 260.00\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n",
       "<g id=\"graph0\" class=\"graph\" transform=\"scale(1 1) rotate(0) translate(4 256)\">\n",
       "<polygon fill=\"white\" stroke=\"transparent\" points=\"-4,4 -4,-256 318.57,-256 318.57,4 -4,4\"/>\n",
       "<!-- x -->\n",
       "<g id=\"node1\" class=\"node\">\n",
       "<title>x</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"157.29\" cy=\"-234\" rx=\"27\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"157.29\" y=\"-230.3\" font-family=\"Times,serif\" font-size=\"14.00\">x</text>\n",
       "</g>\n",
       "<!-- ~[&#39;mnth&#39;, &#39;season&#39;, &#39;weathersit&#39;, &#39;weekday&#39;] -->\n",
       "<g id=\"node2\" class=\"node\">\n",
       "<title>~[&#39;mnth&#39;, &#39;season&#39;, &#39;weathersit&#39;, &#39;weekday&#39;]</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"157.29\" cy=\"-162\" rx=\"157.07\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"157.29\" y=\"-158.3\" font-family=\"Times,serif\" font-size=\"14.00\">~[&#39;mnth&#39;, &#39;season&#39;, &#39;weathersit&#39;, &#39;weekday&#39;]</text>\n",
       "</g>\n",
       "<!-- x&#45;&gt;~[&#39;mnth&#39;, &#39;season&#39;, &#39;weathersit&#39;, &#39;weekday&#39;] -->\n",
       "<g id=\"edge1\" class=\"edge\">\n",
       "<title>x&#45;&gt;~[&#39;mnth&#39;, &#39;season&#39;, &#39;weathersit&#39;, &#39;weekday&#39;]</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M157.29,-215.7C157.29,-207.98 157.29,-198.71 157.29,-190.11\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"160.79,-190.1 157.29,-180.1 153.79,-190.1 160.79,-190.1\"/>\n",
       "</g>\n",
       "<!-- KNNRegressor -->\n",
       "<g id=\"node3\" class=\"node\">\n",
       "<title>KNNRegressor</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"157.29\" cy=\"-90\" rx=\"64.99\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"157.29\" y=\"-86.3\" font-family=\"Times,serif\" font-size=\"14.00\">KNNRegressor</text>\n",
       "</g>\n",
       "<!-- ~[&#39;mnth&#39;, &#39;season&#39;, &#39;weathersit&#39;, &#39;weekday&#39;]&#45;&gt;KNNRegressor -->\n",
       "<g id=\"edge2\" class=\"edge\">\n",
       "<title>~[&#39;mnth&#39;, &#39;season&#39;, &#39;weathersit&#39;, &#39;weekday&#39;]&#45;&gt;KNNRegressor</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M157.29,-143.7C157.29,-135.98 157.29,-126.71 157.29,-118.11\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"160.79,-118.1 157.29,-108.1 153.79,-118.1 160.79,-118.1\"/>\n",
       "</g>\n",
       "<!-- y -->\n",
       "<g id=\"node4\" class=\"node\">\n",
       "<title>y</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"157.29\" cy=\"-18\" rx=\"27\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"157.29\" y=\"-14.3\" font-family=\"Times,serif\" font-size=\"14.00\">y</text>\n",
       "</g>\n",
       "<!-- KNNRegressor&#45;&gt;y -->\n",
       "<g id=\"edge3\" class=\"edge\">\n",
       "<title>KNNRegressor&#45;&gt;y</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M157.29,-71.7C157.29,-63.98 157.29,-54.71 157.29,-46.11\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"160.79,-46.1 157.29,-36.1 153.79,-46.1 160.79,-46.1\"/>\n",
       "</g>\n",
       "</g>\n",
       "</svg>\n"
      ],
      "text/plain": [
       "<graphviz.dot.Digraph at 0x7fcf561825b0>"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.draw()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Concept Drift\n",
    "---\n",
    "\n",
    "Dynamic environments are challenging for machine learning methods because data changes over time.\n",
    "\n",
    "For this example, we will generate a synthetic data stream by concatenating data from 3 different distributions:\n",
    "- $dist_a$: $\\mu=0.8$, $\\sigma=0.05$\n",
    "- $dist_b$: $\\mu=0.4$, $\\sigma=0.02$\n",
    "- $dist_c$: $\\mu=0.6$, $\\sigma=0.1$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "\n",
       "    <div class=\"bk-root\">\n",
       "        <a href=\"https://bokeh.org\" target=\"_blank\" class=\"bk-logo bk-logo-small bk-logo-notebook\"></a>\n",
       "        <span id=\"1001\">Loading BokehJS ...</span>\n",
       "    </div>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/javascript": [
       "\n",
       "(function(root) {\n",
       "  function now() {\n",
       "    return new Date();\n",
       "  }\n",
       "\n",
       "  var force = true;\n",
       "\n",
       "  if (typeof root._bokeh_onload_callbacks === \"undefined\" || force === true) {\n",
       "    root._bokeh_onload_callbacks = [];\n",
       "    root._bokeh_is_loading = undefined;\n",
       "  }\n",
       "\n",
       "  var JS_MIME_TYPE = 'application/javascript';\n",
       "  var HTML_MIME_TYPE = 'text/html';\n",
       "  var EXEC_MIME_TYPE = 'application/vnd.bokehjs_exec.v0+json';\n",
       "  var CLASS_NAME = 'output_bokeh rendered_html';\n",
       "\n",
       "  /**\n",
       "   * Render data to the DOM node\n",
       "   */\n",
       "  function render(props, node) {\n",
       "    var script = document.createElement(\"script\");\n",
       "    node.appendChild(script);\n",
       "  }\n",
       "\n",
       "  /**\n",
       "   * Handle when an output is cleared or removed\n",
       "   */\n",
       "  function handleClearOutput(event, handle) {\n",
       "    var cell = handle.cell;\n",
       "\n",
       "    var id = cell.output_area._bokeh_element_id;\n",
       "    var server_id = cell.output_area._bokeh_server_id;\n",
       "    // Clean up Bokeh references\n",
       "    if (id != null && id in Bokeh.index) {\n",
       "      Bokeh.index[id].model.document.clear();\n",
       "      delete Bokeh.index[id];\n",
       "    }\n",
       "\n",
       "    if (server_id !== undefined) {\n",
       "      // Clean up Bokeh references\n",
       "      var cmd = \"from bokeh.io.state import curstate; print(curstate().uuid_to_server['\" + server_id + \"'].get_sessions()[0].document.roots[0]._id)\";\n",
       "      cell.notebook.kernel.execute(cmd, {\n",
       "        iopub: {\n",
       "          output: function(msg) {\n",
       "            var id = msg.content.text.trim();\n",
       "            if (id in Bokeh.index) {\n",
       "              Bokeh.index[id].model.document.clear();\n",
       "              delete Bokeh.index[id];\n",
       "            }\n",
       "          }\n",
       "        }\n",
       "      });\n",
       "      // Destroy server and session\n",
       "      var cmd = \"import bokeh.io.notebook as ion; ion.destroy_server('\" + server_id + \"')\";\n",
       "      cell.notebook.kernel.execute(cmd);\n",
       "    }\n",
       "  }\n",
       "\n",
       "  /**\n",
       "   * Handle when a new output is added\n",
       "   */\n",
       "  function handleAddOutput(event, handle) {\n",
       "    var output_area = handle.output_area;\n",
       "    var output = handle.output;\n",
       "\n",
       "    // limit handleAddOutput to display_data with EXEC_MIME_TYPE content only\n",
       "    if ((output.output_type != \"display_data\") || (!output.data.hasOwnProperty(EXEC_MIME_TYPE))) {\n",
       "      return\n",
       "    }\n",
       "\n",
       "    var toinsert = output_area.element.find(\".\" + CLASS_NAME.split(' ')[0]);\n",
       "\n",
       "    if (output.metadata[EXEC_MIME_TYPE][\"id\"] !== undefined) {\n",
       "      toinsert[toinsert.length - 1].firstChild.textContent = output.data[JS_MIME_TYPE];\n",
       "      // store reference to embed id on output_area\n",
       "      output_area._bokeh_element_id = output.metadata[EXEC_MIME_TYPE][\"id\"];\n",
       "    }\n",
       "    if (output.metadata[EXEC_MIME_TYPE][\"server_id\"] !== undefined) {\n",
       "      var bk_div = document.createElement(\"div\");\n",
       "      bk_div.innerHTML = output.data[HTML_MIME_TYPE];\n",
       "      var script_attrs = bk_div.children[0].attributes;\n",
       "      for (var i = 0; i < script_attrs.length; i++) {\n",
       "        toinsert[toinsert.length - 1].firstChild.setAttribute(script_attrs[i].name, script_attrs[i].value);\n",
       "        toinsert[toinsert.length - 1].firstChild.textContent = bk_div.children[0].textContent\n",
       "      }\n",
       "      // store reference to server id on output_area\n",
       "      output_area._bokeh_server_id = output.metadata[EXEC_MIME_TYPE][\"server_id\"];\n",
       "    }\n",
       "  }\n",
       "\n",
       "  function register_renderer(events, OutputArea) {\n",
       "\n",
       "    function append_mime(data, metadata, element) {\n",
       "      // create a DOM node to render to\n",
       "      var toinsert = this.create_output_subarea(\n",
       "        metadata,\n",
       "        CLASS_NAME,\n",
       "        EXEC_MIME_TYPE\n",
       "      );\n",
       "      this.keyboard_manager.register_events(toinsert);\n",
       "      // Render to node\n",
       "      var props = {data: data, metadata: metadata[EXEC_MIME_TYPE]};\n",
       "      render(props, toinsert[toinsert.length - 1]);\n",
       "      element.append(toinsert);\n",
       "      return toinsert\n",
       "    }\n",
       "\n",
       "    /* Handle when an output is cleared or removed */\n",
       "    events.on('clear_output.CodeCell', handleClearOutput);\n",
       "    events.on('delete.Cell', handleClearOutput);\n",
       "\n",
       "    /* Handle when a new output is added */\n",
       "    events.on('output_added.OutputArea', handleAddOutput);\n",
       "\n",
       "    /**\n",
       "     * Register the mime type and append_mime function with output_area\n",
       "     */\n",
       "    OutputArea.prototype.register_mime_type(EXEC_MIME_TYPE, append_mime, {\n",
       "      /* Is output safe? */\n",
       "      safe: true,\n",
       "      /* Index of renderer in `output_area.display_order` */\n",
       "      index: 0\n",
       "    });\n",
       "  }\n",
       "\n",
       "  // register the mime type if in Jupyter Notebook environment and previously unregistered\n",
       "  if (root.Jupyter !== undefined) {\n",
       "    var events = require('base/js/events');\n",
       "    var OutputArea = require('notebook/js/outputarea').OutputArea;\n",
       "\n",
       "    if (OutputArea.prototype.mime_types().indexOf(EXEC_MIME_TYPE) == -1) {\n",
       "      register_renderer(events, OutputArea);\n",
       "    }\n",
       "  }\n",
       "\n",
       "  \n",
       "  if (typeof (root._bokeh_timeout) === \"undefined\" || force === true) {\n",
       "    root._bokeh_timeout = Date.now() + 5000;\n",
       "    root._bokeh_failed_load = false;\n",
       "  }\n",
       "\n",
       "  var NB_LOAD_WARNING = {'data': {'text/html':\n",
       "     \"<div style='background-color: #fdd'>\\n\"+\n",
       "     \"<p>\\n\"+\n",
       "     \"BokehJS does not appear to have successfully loaded. If loading BokehJS from CDN, this \\n\"+\n",
       "     \"may be due to a slow or bad network connection. Possible fixes:\\n\"+\n",
       "     \"</p>\\n\"+\n",
       "     \"<ul>\\n\"+\n",
       "     \"<li>re-rerun `output_notebook()` to attempt to load from CDN again, or</li>\\n\"+\n",
       "     \"<li>use INLINE resources instead, as so:</li>\\n\"+\n",
       "     \"</ul>\\n\"+\n",
       "     \"<code>\\n\"+\n",
       "     \"from bokeh.resources import INLINE\\n\"+\n",
       "     \"output_notebook(resources=INLINE)\\n\"+\n",
       "     \"</code>\\n\"+\n",
       "     \"</div>\"}};\n",
       "\n",
       "  function display_loaded() {\n",
       "    var el = document.getElementById(\"1001\");\n",
       "    if (el != null) {\n",
       "      el.textContent = \"BokehJS is loading...\";\n",
       "    }\n",
       "    if (root.Bokeh !== undefined) {\n",
       "      if (el != null) {\n",
       "        el.textContent = \"BokehJS \" + root.Bokeh.version + \" successfully loaded.\";\n",
       "      }\n",
       "    } else if (Date.now() < root._bokeh_timeout) {\n",
       "      setTimeout(display_loaded, 100)\n",
       "    }\n",
       "  }\n",
       "\n",
       "\n",
       "  function run_callbacks() {\n",
       "    try {\n",
       "      root._bokeh_onload_callbacks.forEach(function(callback) {\n",
       "        if (callback != null)\n",
       "          callback();\n",
       "      });\n",
       "    } finally {\n",
       "      delete root._bokeh_onload_callbacks\n",
       "    }\n",
       "    console.debug(\"Bokeh: all callbacks have finished\");\n",
       "  }\n",
       "\n",
       "  function load_libs(css_urls, js_urls, callback) {\n",
       "    if (css_urls == null) css_urls = [];\n",
       "    if (js_urls == null) js_urls = [];\n",
       "\n",
       "    root._bokeh_onload_callbacks.push(callback);\n",
       "    if (root._bokeh_is_loading > 0) {\n",
       "      console.debug(\"Bokeh: BokehJS is being loaded, scheduling callback at\", now());\n",
       "      return null;\n",
       "    }\n",
       "    if (js_urls == null || js_urls.length === 0) {\n",
       "      run_callbacks();\n",
       "      return null;\n",
       "    }\n",
       "    console.debug(\"Bokeh: BokehJS not loaded, scheduling load and callback at\", now());\n",
       "    root._bokeh_is_loading = css_urls.length + js_urls.length;\n",
       "\n",
       "    function on_load() {\n",
       "      root._bokeh_is_loading--;\n",
       "      if (root._bokeh_is_loading === 0) {\n",
       "        console.debug(\"Bokeh: all BokehJS libraries/stylesheets loaded\");\n",
       "        run_callbacks()\n",
       "      }\n",
       "    }\n",
       "\n",
       "    function on_error() {\n",
       "      console.error(\"failed to load \" + url);\n",
       "    }\n",
       "\n",
       "    for (var i = 0; i < css_urls.length; i++) {\n",
       "      var url = css_urls[i];\n",
       "      const element = document.createElement(\"link\");\n",
       "      element.onload = on_load;\n",
       "      element.onerror = on_error;\n",
       "      element.rel = \"stylesheet\";\n",
       "      element.type = \"text/css\";\n",
       "      element.href = url;\n",
       "      console.debug(\"Bokeh: injecting link tag for BokehJS stylesheet: \", url);\n",
       "      document.body.appendChild(element);\n",
       "    }\n",
       "\n",
       "    const hashes = {\"https://cdn.bokeh.org/bokeh/release/bokeh-2.2.1.min.js\": \"qkRvDQVAIfzsJo40iRBbxt6sttt0hv4lh74DG7OK4MCHv4C5oohXYoHUM5W11uqS\", \"https://cdn.bokeh.org/bokeh/release/bokeh-widgets-2.2.1.min.js\": \"Sb7Mr06a9TNlet/GEBeKaf5xH3eb6AlCzwjtU82wNPyDrnfoiVl26qnvlKjmcAd+\", \"https://cdn.bokeh.org/bokeh/release/bokeh-tables-2.2.1.min.js\": \"HaJ15vgfmcfRtB4c4YBOI4f1MUujukqInOWVqZJZZGK7Q+ivud0OKGSTn/Vm2iso\"};\n",
       "\n",
       "    for (var i = 0; i < js_urls.length; i++) {\n",
       "      var url = js_urls[i];\n",
       "      var element = document.createElement('script');\n",
       "      element.onload = on_load;\n",
       "      element.onerror = on_error;\n",
       "      element.async = false;\n",
       "      element.src = url;\n",
       "      if (url in hashes) {\n",
       "        element.crossOrigin = \"anonymous\";\n",
       "        element.integrity = \"sha384-\" + hashes[url];\n",
       "      }\n",
       "      console.debug(\"Bokeh: injecting script tag for BokehJS library: \", url);\n",
       "      document.head.appendChild(element);\n",
       "    }\n",
       "  };\n",
       "\n",
       "  function inject_raw_css(css) {\n",
       "    const element = document.createElement(\"style\");\n",
       "    element.appendChild(document.createTextNode(css));\n",
       "    document.body.appendChild(element);\n",
       "  }\n",
       "\n",
       "  \n",
       "  var js_urls = [\"https://cdn.bokeh.org/bokeh/release/bokeh-2.2.1.min.js\", \"https://cdn.bokeh.org/bokeh/release/bokeh-widgets-2.2.1.min.js\", \"https://cdn.bokeh.org/bokeh/release/bokeh-tables-2.2.1.min.js\"];\n",
       "  var css_urls = [];\n",
       "  \n",
       "\n",
       "  var inline_js = [\n",
       "    function(Bokeh) {\n",
       "      Bokeh.set_log_level(\"info\");\n",
       "    },\n",
       "    function(Bokeh) {\n",
       "    \n",
       "    \n",
       "    }\n",
       "  ];\n",
       "\n",
       "  function run_inline_js() {\n",
       "    \n",
       "    if (root.Bokeh !== undefined || force === true) {\n",
       "      \n",
       "    for (var i = 0; i < inline_js.length; i++) {\n",
       "      inline_js[i].call(root, root.Bokeh);\n",
       "    }\n",
       "    if (force === true) {\n",
       "        display_loaded();\n",
       "      }} else if (Date.now() < root._bokeh_timeout) {\n",
       "      setTimeout(run_inline_js, 100);\n",
       "    } else if (!root._bokeh_failed_load) {\n",
       "      console.log(\"Bokeh: BokehJS failed to load within specified timeout.\");\n",
       "      root._bokeh_failed_load = true;\n",
       "    } else if (force !== true) {\n",
       "      var cell = $(document.getElementById(\"1001\")).parents('.cell').data().cell;\n",
       "      cell.output_area.append_execute_result(NB_LOAD_WARNING)\n",
       "    }\n",
       "\n",
       "  }\n",
       "\n",
       "  if (root._bokeh_is_loading === 0) {\n",
       "    console.debug(\"Bokeh: BokehJS loaded, going straight to plotting\");\n",
       "    run_inline_js();\n",
       "  } else {\n",
       "    load_libs(css_urls, js_urls, function() {\n",
       "      console.debug(\"Bokeh: BokehJS plotting callback run at\", now());\n",
       "      run_inline_js();\n",
       "    });\n",
       "  }\n",
       "}(window));"
      ],
      "application/vnd.bokehjs_load.v0+json": "\n(function(root) {\n  function now() {\n    return new Date();\n  }\n\n  var force = true;\n\n  if (typeof root._bokeh_onload_callbacks === \"undefined\" || force === true) {\n    root._bokeh_onload_callbacks = [];\n    root._bokeh_is_loading = undefined;\n  }\n\n  \n\n  \n  if (typeof (root._bokeh_timeout) === \"undefined\" || force === true) {\n    root._bokeh_timeout = Date.now() + 5000;\n    root._bokeh_failed_load = false;\n  }\n\n  var NB_LOAD_WARNING = {'data': {'text/html':\n     \"<div style='background-color: #fdd'>\\n\"+\n     \"<p>\\n\"+\n     \"BokehJS does not appear to have successfully loaded. If loading BokehJS from CDN, this \\n\"+\n     \"may be due to a slow or bad network connection. Possible fixes:\\n\"+\n     \"</p>\\n\"+\n     \"<ul>\\n\"+\n     \"<li>re-rerun `output_notebook()` to attempt to load from CDN again, or</li>\\n\"+\n     \"<li>use INLINE resources instead, as so:</li>\\n\"+\n     \"</ul>\\n\"+\n     \"<code>\\n\"+\n     \"from bokeh.resources import INLINE\\n\"+\n     \"output_notebook(resources=INLINE)\\n\"+\n     \"</code>\\n\"+\n     \"</div>\"}};\n\n  function display_loaded() {\n    var el = document.getElementById(\"1001\");\n    if (el != null) {\n      el.textContent = \"BokehJS is loading...\";\n    }\n    if (root.Bokeh !== undefined) {\n      if (el != null) {\n        el.textContent = \"BokehJS \" + root.Bokeh.version + \" successfully loaded.\";\n      }\n    } else if (Date.now() < root._bokeh_timeout) {\n      setTimeout(display_loaded, 100)\n    }\n  }\n\n\n  function run_callbacks() {\n    try {\n      root._bokeh_onload_callbacks.forEach(function(callback) {\n        if (callback != null)\n          callback();\n      });\n    } finally {\n      delete root._bokeh_onload_callbacks\n    }\n    console.debug(\"Bokeh: all callbacks have finished\");\n  }\n\n  function load_libs(css_urls, js_urls, callback) {\n    if (css_urls == null) css_urls = [];\n    if (js_urls == null) js_urls = [];\n\n    root._bokeh_onload_callbacks.push(callback);\n    if (root._bokeh_is_loading > 0) {\n      console.debug(\"Bokeh: BokehJS is being loaded, scheduling callback at\", now());\n      return null;\n    }\n    if (js_urls == null || js_urls.length === 0) {\n      run_callbacks();\n      return null;\n    }\n    console.debug(\"Bokeh: BokehJS not loaded, scheduling load and callback at\", now());\n    root._bokeh_is_loading = css_urls.length + js_urls.length;\n\n    function on_load() {\n      root._bokeh_is_loading--;\n      if (root._bokeh_is_loading === 0) {\n        console.debug(\"Bokeh: all BokehJS libraries/stylesheets loaded\");\n        run_callbacks()\n      }\n    }\n\n    function on_error() {\n      console.error(\"failed to load \" + url);\n    }\n\n    for (var i = 0; i < css_urls.length; i++) {\n      var url = css_urls[i];\n      const element = document.createElement(\"link\");\n      element.onload = on_load;\n      element.onerror = on_error;\n      element.rel = \"stylesheet\";\n      element.type = \"text/css\";\n      element.href = url;\n      console.debug(\"Bokeh: injecting link tag for BokehJS stylesheet: \", url);\n      document.body.appendChild(element);\n    }\n\n    const hashes = {\"https://cdn.bokeh.org/bokeh/release/bokeh-2.2.1.min.js\": \"qkRvDQVAIfzsJo40iRBbxt6sttt0hv4lh74DG7OK4MCHv4C5oohXYoHUM5W11uqS\", \"https://cdn.bokeh.org/bokeh/release/bokeh-widgets-2.2.1.min.js\": \"Sb7Mr06a9TNlet/GEBeKaf5xH3eb6AlCzwjtU82wNPyDrnfoiVl26qnvlKjmcAd+\", \"https://cdn.bokeh.org/bokeh/release/bokeh-tables-2.2.1.min.js\": \"HaJ15vgfmcfRtB4c4YBOI4f1MUujukqInOWVqZJZZGK7Q+ivud0OKGSTn/Vm2iso\"};\n\n    for (var i = 0; i < js_urls.length; i++) {\n      var url = js_urls[i];\n      var element = document.createElement('script');\n      element.onload = on_load;\n      element.onerror = on_error;\n      element.async = false;\n      element.src = url;\n      if (url in hashes) {\n        element.crossOrigin = \"anonymous\";\n        element.integrity = \"sha384-\" + hashes[url];\n      }\n      console.debug(\"Bokeh: injecting script tag for BokehJS library: \", url);\n      document.head.appendChild(element);\n    }\n  };\n\n  function inject_raw_css(css) {\n    const element = document.createElement(\"style\");\n    element.appendChild(document.createTextNode(css));\n    document.body.appendChild(element);\n  }\n\n  \n  var js_urls = [\"https://cdn.bokeh.org/bokeh/release/bokeh-2.2.1.min.js\", \"https://cdn.bokeh.org/bokeh/release/bokeh-widgets-2.2.1.min.js\", \"https://cdn.bokeh.org/bokeh/release/bokeh-tables-2.2.1.min.js\"];\n  var css_urls = [];\n  \n\n  var inline_js = [\n    function(Bokeh) {\n      Bokeh.set_log_level(\"info\");\n    },\n    function(Bokeh) {\n    \n    \n    }\n  ];\n\n  function run_inline_js() {\n    \n    if (root.Bokeh !== undefined || force === true) {\n      \n    for (var i = 0; i < inline_js.length; i++) {\n      inline_js[i].call(root, root.Bokeh);\n    }\n    if (force === true) {\n        display_loaded();\n      }} else if (Date.now() < root._bokeh_timeout) {\n      setTimeout(run_inline_js, 100);\n    } else if (!root._bokeh_failed_load) {\n      console.log(\"Bokeh: BokehJS failed to load within specified timeout.\");\n      root._bokeh_failed_load = true;\n    } else if (force !== true) {\n      var cell = $(document.getElementById(\"1001\")).parents('.cell').data().cell;\n      cell.output_area.append_execute_result(NB_LOAD_WARNING)\n    }\n\n  }\n\n  if (root._bokeh_is_loading === 0) {\n    console.debug(\"Bokeh: BokehJS loaded, going straight to plotting\");\n    run_inline_js();\n  } else {\n    load_libs(css_urls, js_urls, function() {\n      console.debug(\"Bokeh: BokehJS plotting callback run at\", now());\n      run_inline_js();\n    });\n  }\n}(window));"
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "\n",
       "\n",
       "\n",
       "\n",
       "\n",
       "\n",
       "  <div class=\"bk-root\" id=\"7b14f533-52e8-4e31-9616-d4e7a6a69779\" data-root-id=\"1187\"></div>\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/javascript": [
       "(function(root) {\n",
       "  function embed_document(root) {\n",
       "    \n",
       "  var docs_json = {\"533397ec-c880-462e-82c8-bb08e6e28c2f\":{\"roots\":{\"references\":[{\"attributes\":{\"children\":[{\"id\":\"1186\"},{\"id\":\"1184\"}]},\"id\":\"1187\",\"type\":\"Column\"},{\"attributes\":{\"label\":{\"value\":\"dist_b\"},\"renderers\":[{\"id\":\"1127\"}]},\"id\":\"1139\",\"type\":\"LegendItem\"},{\"attributes\":{\"bottom\":{\"value\":0},\"fill_alpha\":{\"value\":0.1},\"fill_color\":{\"value\":\"#ccebc5\"},\"left\":{\"field\":\"left\"},\"line_alpha\":{\"value\":0.1},\"line_color\":{\"value\":\"#ccebc5\"},\"right\":{\"field\":\"right\"},\"top\":{\"field\":\"top\"}},\"id\":\"1142\",\"type\":\"Quad\"},{\"attributes\":{\"source\":{\"id\":\"1109\"}},\"id\":\"1113\",\"type\":\"CDSView\"},{\"attributes\":{\"bottom\":{\"value\":0},\"fill_color\":{\"value\":\"#b3cde3\"},\"left\":{\"field\":\"left\"},\"line_color\":{\"value\":\"#b3cde3\"},\"right\":{\"field\":\"right\"},\"top\":{\"field\":\"top\"}},\"id\":\"1125\",\"type\":\"Quad\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.1},\"fill_color\":{\"value\":\"#ccebc5\"},\"line_alpha\":{\"value\":0.1},\"line_color\":{\"value\":\"#ccebc5\"},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"1064\",\"type\":\"Circle\"},{\"attributes\":{},\"id\":\"1137\",\"type\":\"UnionRenderers\"},{\"attributes\":{\"data_source\":{\"id\":\"1062\"},\"glyph\":{\"id\":\"1063\"},\"hover_glyph\":null,\"muted_glyph\":null,\"nonselection_glyph\":{\"id\":\"1064\"},\"selection_glyph\":null,\"view\":{\"id\":\"1066\"}},\"id\":\"1065\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"data_source\":{\"id\":\"1124\"},\"glyph\":{\"id\":\"1125\"},\"hover_glyph\":null,\"muted_glyph\":null,\"nonselection_glyph\":{\"id\":\"1126\"},\"selection_glyph\":null,\"view\":{\"id\":\"1128\"}},\"id\":\"1127\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"data_source\":{\"id\":\"1109\"},\"glyph\":{\"id\":\"1110\"},\"hover_glyph\":null,\"muted_glyph\":null,\"nonselection_glyph\":{\"id\":\"1111\"},\"selection_glyph\":null,\"view\":{\"id\":\"1113\"}},\"id\":\"1112\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"items\":[{\"id\":\"1123\"},{\"id\":\"1139\"},{\"id\":\"1157\"}]},\"id\":\"1122\",\"type\":\"Legend\"},{\"attributes\":{\"fill_color\":{\"value\":\"#ccebc5\"},\"line_color\":{\"value\":\"#ccebc5\"},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"1063\",\"type\":\"Circle\"},{\"attributes\":{\"data\":{\"x\":[2000,2001,2002,2003,2004,2005,2006,2007,2008,2009,2010,2011,2012,2013,2014,2015,2016,2017,2018,2019,2020,2021,2022,2023,2024,2025,2026,2027,2028,2029,2030,2031,2032,2033,2034,2035,2036,2037,2038,2039,2040,2041,2042,2043,2044,2045,2046,2047,2048,2049,2050,2051,2052,2053,2054,2055,2056,2057,2058,2059,2060,2061,2062,2063,2064,2065,2066,2067,2068,2069,2070,2071,2072,2073,2074,2075,2076,2077,2078,2079,2080,2081,2082,2083,2084,2085,2086,2087,2088,2089,2090,2091,2092,2093,2094,2095,2096,2097,2098,2099,2100,2101,2102,2103,2104,2105,2106,2107,2108,2109,2110,2111,2112,2113,2114,2115,2116,2117,2118,2119,2120,2121,2122,2123,2124,2125,2126,2127,2128,2129,2130,2131,2132,2133,2134,2135,2136,2137,2138,2139,2140,2141,2142,2143,2144,2145,2146,2147,2148,2149,2150,2151,2152,2153,2154,2155,2156,2157,2158,2159,2160,2161,2162,2163,2164,2165,2166,2167,2168,2169,2170,2171,2172,2173,2174,2175,2176,2177,2178,2179,2180,2181,2182,2183,2184,2185,2186,2187,2188,2189,2190,2191,2192,2193,2194,2195,2196,2197,2198,2199,2200,2201,2202,2203,2204,2205,2206,2207,2208,2209,2210,2211,2212,2213,2214,2215,2216,2217,2218,2219,2220,2221,2222,2223,2224,2225,2226,2227,2228,2229,2230,2231,2232,2233,2234,2235,2236,2237,2238,2239,2240,2241,2242,2243,2244,2245,2246,2247,2248,2249,2250,2251,2252,2253,2254,2255,2256,2257,2258,2259,2260,2261,2262,2263,2264,2265,2266,2267,2268,2269,2270,2271,2272,2273,2274,2275,2276,2277,2278,2279,2280,2281,2282,2283,2284,2285,2286,2287,2288,2289,2290,2291,2292,2293,2294,2295,2296,2297,2298,2299,2300,2301,2302,2303,2304,2305,2306,2307,2308,2309,2310,2311,2312,2313,2314,2315,2316,2317,2318,2319,2320,2321,2322,2323,2324,2325,2326,2327,2328,2329,2330,2331,2332,2333,2334,2335,2336,2337,2338,2339,2340,2341,2342,2343,2344,2345,2346,2347,2348,2349,2350,2351,2352,2353,2354,2355,2356,2357,2358,2359,2360,2361,2362,2363,2364,2365,2366,2367,2368,2369,2370,2371,2372,2373,2374,2375,2376,2377,2378,2379,2380,2381,2382,2383,2384,2385,2386,2387,2388,2389,2390,2391,2392,2393,2394,2395,2396,2397,2398,2399,2400,2401,2402,2403,2404,2405,2406,2407,2408,2409,2410,2411,2412,2413,2414,2415,2416,2417,2418,2419,2420,2421,2422,2423,2424,2425,2426,2427,2428,2429,2430,2431,2432,2433,2434,2435,2436,2437,2438,2439,2440,2441,2442,2443,2444,2445,2446,2447,2448,2449,2450,2451,2452,2453,2454,2455,2456,2457,2458,2459,2460,2461,2462,2463,2464,2465,2466,2467,2468,2469,2470,2471,2472,2473,2474,2475,2476,2477,2478,2479,2480,2481,2482,2483,2484,2485,2486,2487,2488,2489,2490,2491,2492,2493,2494,2495,2496,2497,2498,2499,2500,2501,2502,2503,2504,2505,2506,2507,2508,2509,2510,2511,2512,2513,2514,2515,2516,2517,2518,2519,2520,2521,2522,2523,2524,2525,2526,2527,2528,2529,2530,2531,2532,2533,2534,2535,2536,2537,2538,2539,2540,2541,2542,2543,2544,2545,2546,2547,2548,2549,2550,2551,2552,2553,2554,2555,2556,2557,2558,2559,2560,2561,2562,2563,2564,2565,2566,2567,2568,2569,2570,2571,2572,2573,2574,2575,2576,2577,2578,2579,2580,2581,2582,2583,2584,2585,2586,2587,2588,2589,2590,2591,2592,2593,2594,2595,2596,2597,2598,2599,2600,2601,2602,2603,2604,2605,2606,2607,2608,2609,2610,2611,2612,2613,2614,2615,2616,2617,2618,2619,2620,2621,2622,2623,2624,2625,2626,2627,2628,2629,2630,2631,2632,2633,2634,2635,2636,2637,2638,2639,2640,2641,2642,2643,2644,2645,2646,2647,2648,2649,2650,2651,2652,2653,2654,2655,2656,2657,2658,2659,2660,2661,2662,2663,2664,2665,2666,2667,2668,2669,2670,2671,2672,2673,2674,2675,2676,2677,2678,2679,2680,2681,2682,2683,2684,2685,2686,2687,2688,2689,2690,2691,2692,2693,2694,2695,2696,2697,2698,2699,2700,2701,2702,2703,2704,2705,2706,2707,2708,2709,2710,2711,2712,2713,2714,2715,2716,2717,2718,2719,2720,2721,2722,2723,2724,2725,2726,2727,2728,2729,2730,2731,2732,2733,2734,2735,2736,2737,2738,2739,2740,2741,2742,2743,2744,2745,2746,2747,2748,2749,2750,2751,2752,2753,2754,2755,2756,2757,2758,2759,2760,2761,2762,2763,2764,2765,2766,2767,2768,2769,2770,2771,2772,2773,2774,2775,2776,2777,2778,2779,2780,2781,2782,2783,2784,2785,2786,2787,2788,2789,2790,2791,2792,2793,2794,2795,2796,2797,2798,2799,2800,2801,2802,2803,2804,2805,2806,2807,2808,2809,2810,2811,2812,2813,2814,2815,2816,2817,2818,2819,2820,2821,2822,2823,2824,2825,2826,2827,2828,2829,2830,2831,2832,2833,2834,2835,2836,2837,2838,2839,2840,2841,2842,2843,2844,2845,2846,2847,2848,2849,2850,2851,2852,2853,2854,2855,2856,2857,2858,2859,2860,2861,2862,2863,2864,2865,2866,2867,2868,2869,2870,2871,2872,2873,2874,2875,2876,2877,2878,2879,2880,2881,2882,2883,2884,2885,2886,2887,2888,2889,2890,2891,2892,2893,2894,2895,2896,2897,2898,2899,2900,2901,2902,2903,2904,2905,2906,2907,2908,2909,2910,2911,2912,2913,2914,2915,2916,2917,2918,2919,2920,2921,2922,2923,2924,2925,2926,2927,2928,2929,2930,2931,2932,2933,2934,2935,2936,2937,2938,2939,2940,2941,2942,2943,2944,2945,2946,2947,2948,2949,2950,2951,2952,2953,2954,2955,2956,2957,2958,2959,2960,2961,2962,2963,2964,2965,2966,2967,2968,2969,2970,2971,2972,2973,2974,2975,2976,2977,2978,2979,2980,2981,2982,2983,2984,2985,2986,2987,2988,2989,2990,2991,2992,2993,2994,2995,2996,2997,2998,2999],\"y\":{\"__ndarray__\":\"wz+TDRgK4T9iKyVwz7ziP6bwobIMquA/neB4/Oo24j8QVMHX5kfaP8uM6yHu4eM/CmyqATA04z9waVhL15XgP9o6hglBT+U/Omav6EEz5j9X6E3skhzcP6LeczhgwuA/tp+4Wx694D8QDbE7OzHgP2DYr6602uU/z1qc6WqU4j8ccspqglriP6AzYZtFk9k/nB7wBLw75T+/Y3jB/3feP312yftLZOM/YzkqXG0W5D+trFP92Y3nP0/aIlYHBt4/KQBsNTsT0z/W7Hk+z8njPz5ZNcMt9ug/dMsKFSUq5z8jSln3897jP//5dp5zoOE/V76nE2XI5T8qzJbDqxXgP7kONapuueQ/LOu7+UfR5D/g7TFVuZfmP/7XY9vpB+w/3awEy6p05D/V8ZvKQZLhP0rbxOA/HuM/OE2CL/IT2z+0L/lLF/rgP55TnnPq4+E/JU6V5d2F4T9JERxVArDjP3YWST15keA/yejfXozJ5j8KxOmfXjPjP4nGAsiUK+M/8YumsZYm4j+pBJkLVbLjP8Ty+9ge1+U/DRM60Nhs4D8+FTsrEhjhPw55MTxjOuI/GEpgOFPJ3T8S0cRfEJTgP88Mu3EUreE/wqPj5n//5T9lOTdNSgrkPytPbfLJ0eM/4mGyb0Hs5T/vx3c/qsLiPwBiNZ0Rc+Q/W1eJB6Te4j+Eb9q3lQzkP8lPANrNVeE/PskBWKTK1j8/zZLRMsXiP3MuIxy3wOc/vZL4pfQp5j8F3/+5DUrmP9gp/5nWJ+c/3htfC9R74z8oJJRN19TjP79ZjzU4OeE/pd/z6kUw4j/GehPFoyvlP1/FxRBsDec//K5HMvfA4j84phSrZ8LhPwN5xuuhM+M/GYPCZLUf5T94awPMySjdP4AY6k1WtNc/rgInQTNw4T9+0bh/YpbeP6MHEXrukuE/L/++ayO64j+ETYce5r/hP+orMLgP2ec/ySMti94+5D9KNYw9WSnkP3Q2437pMOU/wf18K4kc3z/Oi3+8cIbmP1kACkci9eI/qOM+LXVY5T9AUFrkRIrfP6qPQL24dNw/N6fcFzfR5T/LksH/jWfkP89UIkc9UOA/TdYKosVq4D+RSxz7J83mPzDV3eSqyd4/CW3ar+B06D9qrq5XaVHgP1RCu7ArPuU/flkf6f4l4j/8vdLFVCHlP0YyJTl2deE/NuCaDdet4j9TiLOSuFTjP1xTuYNq/N8/6U9lUxWS5T/s2aaDxo7hP1RjslLud+I/3wakBaEE4D/mN8DERQDWPw+dSca1luI/g+NeU53r6j/qCCml8rXlP3zIbwFsI+M/rNJG/9Zb4j/j46KjmkXjP+meCGVm8+Q/3uzJm8HX3j/y16VlDcTmP3v3H4k9feU/yC7BqYl/5T/lry06ZprkP6ibfRxFQ+M//4O9bWha5T/YVZKeAxjlP9dgQ3srEeI/X4mBME5d4T/4u2kDpIbjP8INWEUoKOg/IeA+OT943j83p9awAQLdP6h4gg0suuM/+PfjH+Nc4z/JnfGUM8HjP4TeyOQL++M/ej1mS1p84j9w94A1svfnP440nJp6J9w/Wyffx36M4D97nHkbnd7iP6gDRg8z4ts/aFfA2yOj4j//dYO/RoToP52CAGpPROM/3+bDrzDw4z/e4Az9vlXeP3YNE9hGOuE/K2Hp0Nb/4T8kKNOD7S7iP7segmIfTec/BOAJfg/85D9KfQAlY0nfP+CNuP8i/eM/Iz2m31fL5D/U2JzINdnmPypjL5v6Qeg/nFV+CUXn3z9SogsO8prgP7Gk/gP8Wd4/mHeurX1z4j944cejPbHkPwHhb0EGXOY/lL6qHfv04j85hgxQLi3iP5PEXiiFr+M/cj0TCg2H4D9UGB478+LpP0ZvCX5oHNw/iuyBG4rK4z84PNgu8KzpP7g1EzvFOOM/r0EyIIWX4j8Bwi92YQ7iP1SNh1Jsn+I/prd2ZNaX5z/m2xZGfD7YP66vK45jG+g/fA5ie2NJ3T92HUgnwljiP+M+3B6Q0+E/Xy5FxFcV5T8mMr52CCzcP6gqRtXPreQ///uPsoOt6T8SGzDI+azdP0COhXebzuM/E9qleecU4T8W1Ho4FpDkPzIirMfiQuM/OrICpbEl4T9zRA1v3cLkP/IGumxS+eg//FDsxc+W4j+QZnth04DlP7m17WeAH94/SfU19K8j4D+Jvx/6MbbkPxqAxuP88uc/Ovp8uYRW5D/XFiosvjLiP7d+0ZieMuM/pGFMYrtl3j8YaKhAayLlP74lydoBBuY/5kJF/NjA4T/Lnos3JLLhP/ai9ucBDeQ/UgDnbnDN4T/5vnwvBv3iP3m9Y8ZK6+k/ChTgJ9Zo4j/eHsfSpQ3iPwudOde7IOE/ubhxkdeU5T+IG3y8vZ7iP483q807H+E/+ZdIvZxt5z9eziuHI77nP9Cxd0BVR+E/8Kk0zUJ02j+iMI0ijmzmP/thTutZAuE/D+nYZ+G65T8Q5rr3lMrZP5j5521qDuY/smW/EgSm3j+dIxuL14nlP9PTsXrkPuM/nHGXmrsl4D/90ot/weXhP/V2ypxuZeU/Lapc4PKJ4z/rWy588xHlP1i9HNVMhuk/ODdE6otj3D+evjNCBWDoP8Ng7zmwiOM/blAHhupS4D+iLYBs0OLdP8FWiMdFmOI/lTnQPDcm5j+PGvE1tsriP8TswVsjCeg/FHYpM8sc3T9kJcH2SynjP2BVQ4R6Yt4/Iuc+VRZd5D/acmX3vAnmP1i/aA6D2uE/JPq/8Ke71T9YSoCmLdbjPyRQqOn5mOQ/wNCU9GV+5D+BiW+CjSfnP3C9Pe2Vi98/rv1J225g5T90cscmGgTnP/6RvbJcBNs//CvuCw855D9dqaEGLZbhP7+qA6hq8OI/JqJHPgRQ5D9Y29f2D6ThP/3x7fGoCeE/YQIney1P4z9VmqVmDnHfP1Xi9FRic98/f9aZGr5f5T/R+yBCNwzfPzE6NA8LVeU/7bUgoCeu5D+8auqX1lrbP2yJ6pcjCOE/YsqfF2oF5z+y79jfbQ/gP+U5N5HCtuE/exaEZLit5D8LxeoY9LTlPzenFawjZeI/FXniOrhJ4T9IZa/HZ8DnPxa3CeS0w+g/wOiknBRV5j/I1Bm2F3njP+88sJcRneA/p9GbYuaK4D+OaJ9ZPt/kP3qMpKrzieQ/QXCvumOv5z/fl6N4DEjlPyLA6tPAx9w/juvjmvSQ5j/cOke3lgHgP9YngZam+OE/asV9/CkA5D+imboBDZbpP8ZAV2TYJu0/Y7NoPzok5T/Mi58VIJ3iP2lnsigR6eQ/xMgACV4K5j+NTiRTgiziP8b2/E7W8eg/PPoHiazv4z/UdtF988rkP1WZQKO5VOU/DkQDUpCM5D+ULlzGdeLlP8iSFNOLOeE/vM4E1tZp4T926h5F6FvfP1zP2Uk9m+Q/mpVNBk6x5T8aYdAgNarkP4zOLrfxjug/eQ06oZ0u4z8N+nN9CFflP34wZt3Bad8/pnUrghb24T/QNTkK/GzlPyRXqNHI6uU/uPfe3GFC4j+UuaK/i/jiP/kc3QGEr9w/ZWvvv7gO4j9ahk0VmgzmP83AW29oCuU/MstxaVjN5D8A3K5/+VvjP9I0pHP8OOM/rLv20UMW4T/dHFQkrm/lP/LjDqAPjOQ/6dz2B0PG5D+QmKCyRYThPwm28+pcntg/gnDlzPy15j9CgAe176rhP0AXEgJqcOA/OwnKRExr5T80X9kwEPLhP/I5hPh/l+Y/KsjiuaYs5T/e0ssTLmPlP3MZtwxrqN0/6AR5RgYU5z/n4EtoNwnkP2pJvyuwBOI/6Vjs8Kmo4z/eeIRYNwjbP4aH9ef3geQ/XbZSsuDP3z9EmWZcL77dPwzyda0qqNw/iRt33bbC5j9gdv4ZtS/hP6FkozahG+g/8CEwfUV84T+0qqOREHnbP7nohxsWQd8/121ZrpMn5z+80YD7fLPiP7IIHAULcuE/IPdIdEm24z9QHriqQs7kP3IMVreSxuY/Ill1gc/V5z9cRRo/egziP2CVSLDN6d0/LQJjRX3g4T9ll376awjkP3hu1U2wHeA/4me9Ghoj4D9y24y+1EzkPw/S9g9YC+M/5tZ7OhFO4z8ovv0y18XgP108C6t0duI/fPYXdhE+4D9a8XtWcwzmP9TmpbhGg+Y/UagiFpuV2j8yoeCkvjngPwRwMol80tw/UYnzTLMe4T8fa0792O7iP3YwKiBNH90//NBWLwNA4D8ehmgohPnfP0taN0/+3OM/4VjywgFs4z8W/Vnd8+PgP6MVfTkNxOM/8zukE1xz4T++WIT+qFTiP/HQMIoXjug/EgHwQU595z9fXZ+FKhXeP+m1MLPq2uU/CzYFZOXL5T/JGpfeFQ3fPwSYO1Hx0eU/DPtc5hIf6D+LRJ/7ODffP1BrkumUQ+A/CWAuR9505j94cbbkmxHkP0COSCj49uQ/FMuF7zVK5D+sA70SQnPkP/1tJaob6d0/UkZupCmN5j+AQsr/U/HmP9kcZ3lId+I/XPE6bJUP4z/WhSVe1ZncP4C1Lz162OQ/x7Q+V9QH5T+JBmbMQADjP2uIkjBFzOY/trpuIIYh4j8xFwmluQHlPyjjtS3I398/DGK69Ocf4z+lPkF2XqTiPzSgKLLF6+M/lFdFyXsE4j+MQ/dlf8fiP7dXB9nl1+U/xDLDGWfN4T8S4b4fLx7cP7G+04l1zOg/RK9PHuOe5z/PJxEkKCHePw+jCmk1aOU/3XXSvimX4T8FCVx5DAvkP4f72Hg6JOQ/yhU/Uixz4j/qpm6gI7HgP1A4vz6C/eA/BuBG1KdE4D/QzXFwxYngP2YoN+Qq/OI/GSZjc9no4D8hsqfi72HlP5ypLVON9ec/iAHuNwVY4T/4NHT2UffjP/PBScOKzOQ/8VoqQN215D/agZgsbHHjP2hX7f2ak+U/xyO0IkK+5D+kJLcnnXreP+aMGHA2++U/oh9NkxEK5j/IZYOA6sHgP4hKO1dHUuM/WRoorfti5T+1LRTpuofiP/6JgQEvouY/g19nMa/D6j/LTWL7Ua/gP0BHR0+yj90/CLrZkxYs5D//HTjTTYLlP9PjIQMTduI/NRb4DLbZ5z8iTBlvOtXdP7tFgsTWauU/mLgP+uBC4T+kZqjhtrPoP/XUrvR3k+k/13YNqSe/4D8r9x+3AHHhPzmSe9hf8+U/pMyyBBD24T+1WQDH/w3jP2Qqpon/R+M/Oi7HHjUd2j8oCt023SfjP6BnSs0s/uA/V9JC2Tuf4T96vjrG0tHnP5Zh8jW9W94/GKo5sGDN5T+s+OpurE7iPwYUhT0FTuI/G+dDM6y65T9JbxAgxUnkP8O8OGulBuU/+FNePVZM5j8QFNHLyiPiP/xpzNmnPeE/F3bUR3103z8+scwQBo/gP3CnkSAZo+w/p6Rdbdcu5z84gfe8zMDdP/JJiVSv790/jNNm1vG95D/DkUnyhPPkP1WSuXvY9OQ/hm9CGFdi4j8W3OaIbcziP2a8HYXMP+Q/fsDcRJB54z/VT2BbTDLYP9O/ebnyduI/JctQVLp54D8hUKCyu8LjP5ZEh2O5wOw/XH7jsD1g5D/DNFhCWzLiP2JEgfxWJuY/UF3SL56+5D/J52tmV4vkPw++V/5UJOU/wiwfpevH6T+XcUVpjSnfPz/5swszt+Q/5Bz9/k484D/wkk6lRvrkP2xzrKycQuA/DZ4DCUvd4T9m5g9cdkHiP/XrZ8yWUuY/UeqxDpkj5j/CQ5qpam3eP/3F07czYOM/xgvThOnt4D/A4ZEud1jePwOcBW1rgeI/M0Caq3024j+W24SIKvfqP9h/1HThleQ/n7mYvU+V3T8zDruBiWThP731VM3qd+Y/8yPQZA3i6T+wyvLI7D/cPzlmkvZUXto/9sMzCeUv6T9oxRI3X3LkP4yOHQTla+A/mW+0Cyvp5D/nEOmr/YfVP3udVBj3NeM/C1pnSMA/5D/EjHmyXCjmPyPie5076d8/XCCmGWV54z9T6uaWFz3gPw2QLg5pBOQ/ZySJd7tV4D9srTpA4uThP4sSSJNtA+A/7Jf/XZxI5T9YN5dycfLlP6dnQ7P/cuI/pzfsSYZS4z+SP58fByHdP7r+qnO0PeI/y6FYhxIK4z8U++RYJ5brP2XYJqCNSN8/o10mbKfi5z/4phWj02PmP34gcesYGeQ/jGuFpdzT6D9B1KRfB2TkP0zcbcz/ceQ/GFrb/a4H4z+IQGExjB3nPxEf9HZtteY/aruaKaUy4T8+GH8F8vTdP35TNRPrjOI/BmXU886j4D8OtpEhKf/iP+XuMMnoKOc/95WJF5S84T9cCmx8HhDjPzgqDnO7YuM/349ZDTzq5T/yWTi1GAXYP5PUJO0wQuE/hnUOmkjg4z8wtZT/TQrnP6eko8c7oOE/BrbbJdtj2j9EBR4c3i7lPzCirwy2KuE/0DeNCUDJ3j+CeVJ7vDPhP2Gsfm0JnOI/XQI+DctD4T8NMEryzEjYPzq67J0TEuY/w0rP35NH5z/LkcRD7UjlP5e+tXFqHt8/TUfXXCOO4j8EKcGU2y3jP1TUM3y7HeU/FHD4W7RW5T/QIYLLw9ngP8B7HepeduM/oY782c2p5D8h5/pEz9vnPw/4De5UdOU/WH+JYZi55T8SxQTr3nfjP81g9uultuc/HAvMr+CC5D8JCQn6y3HgP/oAxjbXsOc/3nHXH+xu5T+Dm24iQsjhP+KLhzxmquE/RieKmScm5D+exYsXpK7kP+fYDmPS1OM/t6xXXDky5D8iFM4ZzqPoP2uGIxYCoeY/g/+SYV7P4z/4ejJWUjbmPydFc3y2yt8/LX0FjS555D/Q9DQ+Yv3oP2sKVKmTgOI/94Ure9F84D8VTpE/Ah3hP6U65i0lbd8/gmwRs1Kw4D964x4FrgPiPwpE6NyEZ90/P4Aemv4/4z8d5B1wWBbmP5h8Xu6QS+A/dmrEIcwP6D/aGaQQpNXkP4bfFp1zf+Y/w0iDS00Y4T+/i/8gOu/lP7w2nCBgad8/ps33oX0M5j+v2oGF+L/jP7Bh52Ns+OQ/s4TJSE7g3j/6g/Jy9VXgP+1EBQlzHOU/pmpoNVAq4D+T+aWZqK7kPwYFND57pt0/0UiCOQbq5T88EzCnCIPeP7XuUuSk9+Q/oDMxhLcz5T9g0wmPZ/jgP6iAlgT/D+U/vXG+5o0I5D/6/lLfx3nhPw6oyuf78d8/ikBmhdXW2T82bi0holHkP8ouNYygYtw/rkjuU06B4z+D9csaYlviPyehUsONX+U/WR7sqsM74j9kR9n9UCXiP7Gecpy/iuU/1Csq9q1F5D8sHejoMTbkP3VcF7+fs+Q/FIcnxYSS3D//4hRKKZ/lP3qMMzfvKOU/vY+1lqrk3z8OuozQP2viP96pQ93/EuM/ANkRLgXF4j9vVqy1skTkP3AlKJLGx+c/GmsqIGKp5j/gLmDHdQDeP52ZgLvMMOU/QE2Gt9xz5z8o9YbTEnDkP7jMyGDusOY/wSwj5qaj6T/v0vam0nnmP3nbggBv/+M/9pVrI1aL5j/DXYJh5KnjP4iWEAjIRuM/wAgUS1IT4j8nZ8e+8jPoPzlYxUS+lOA/X5r8AIQa6D8etf9NsMzkPw4Q9NnGcd0/0otS2GBg5D9eYR9xt/XYP0FcJb2dM+U/hXKPsmEI5j9ybfwm8U3hPxzNngdpmOM/6p2Qtwt06T/0ocNt5JThPw+tAF58n98/F2xqifrw5z9KnK+Q83rpP31QEj06NuM/6GltX8pv5j+1CwUXI37nPxA/fvj00uA/v4SOMKOl4T/QiHFLiCPnP7VOdE6Ql+g/1WBG308A5T8OtMPBdGLgP1TjUh4Aj+k/d0MviDqA4T8C45O2pyDiP/vPvaWkS+Q/oUNyQ14s6D/p9FXxxu7lP35WJ80Oh+Q/jbPE2Lmu5D+YGi/j2FbjPx7hN4pt/OQ/K9Q5Afg11j+4i34bOEDiPxeu7RPk+OM/dI89GpQI3z+aYmUSrW/kP9pzHHqyi+I/pGYdI0nR6D9goqVY0TvoP9cYIbGcteE/JgQ5PaK34j9NTIBFn/biP31KtlczweE/ncVN5/bS4z8rwJzsA8bgPyHCM/X/Kd8/GgN4NVMx5T/9tWYf6TblP1NUqPVNoOA/+iXe16wQ5j/x7Q3Azy3hP85XbMyXAuQ/aj8FzznT5T+pj18iqhfjPw7CBUi5p+Q/3mrPDSSM4T+gV9ErQZLiP0ryKFmPiuI/h91rX5rP4D9yP5X4YqLiPy63tc4kfdw/Ahgvy9ig4T9kWJt61EniP/fUdmlWYeI/MonMg11t4j9sHvhOmgDjP6jgtF/1u+Q/vGoduZr/5T/TJTWO6R7hP+EjSmITs94/fGScSx273z9hcolXFaThP2KuUPPnEuI/YKArUmu84D+Y+Uer1xreP2y3vLqLveE/HT6zHtPH4z/yJGbspRrlP5sp6RyPauE/BAp1R9/g4T+w+ZN6DDzgPyvEl+1xGeM/SVC90Sp84D/m9GDQtAjlPwb587Nv+No/+kMtZJcM4j89U2H83SnkP1i9pBfMyeM/PG8fS1XR6z8vTwWVwVHkPxqo+mBb+d8//M515f7k4j/1YJqwMgjbP9HaGzUs8eI/NW6xrsKI4D/rV2h0FiHmP2EWXEAEceE/4mPUnIvT4j912GcPjSrhP9BoT8Kqw+g/YKGEtQks4j9pjVJEbhDpP+99GjMnzuU/kxCO6xu+5D/F+U4lRGHkP+/kqXrMdeQ/gQNC6xkQ2j+K6/scvU7iP1oCR6yN5+U/UcKmtyQL4z88kXsPI2jdP1q8rjBFCOM/ojN3BzSn6D/DzHJYpzDnP9BdSwRoAOM/ZEFPk6A+5j9VP18b9QbiP2uS2dvGfts/M01ioeA94D8W5Ay7hDPoP2WuoK++UuI/E3xN4dFr4j88gZySkj3iP9MQ8ZURS+k/dMjGfy9n6D+rJfIj00zqPwD9ewDpsOI/DZwWJmAl5D84s6loiLHcP0tybswa7ec/V/LbJxzt3j9EYqlUyOTjP25kplyUYN8/yxAr4MxQ4T/utbSKUIXgP9RRpIQ6QeE/JNJ3+Yx54T9BnWfmDnLhPzn76/zd3eU/0qpUWzFU3z/9zOcW3ufjP+umSb/zGOc/rsMsH8qO4T+v/DyyFOfgPyfFn65aduI/Al0NgqH55j+pHUmFNtLjPy8t63wHgOE//xQafZe/5D++nQuQZ1jfPxyMYKVeYeU/sFr+XO6B5D+z4rwVCzfiP2+x9QI7hOA/Nvqou9Rc4D90zk+GUunkP7AohRf7Iec/kRMQFB4p4T8OXPXbtarkPziCodIDC9k/De7G+UdU4T8DOnCrwRnjP04eocL8SeA/KxsMlAM04D+NF0CtJxDhP9CH2nSQIuQ/pboadb2Z4j8qGLMtSBPYP2YvZUnY09g//t4MWQ5C4T+9vK8II6rkP+a3RVjj0NQ/jFWoL9KZ4T9HiYgGGYThPwl2HbyGpOc/CqEURsr34T8VnMDP8WzkP1ubKfjqpuM/fvSUDK7D2D/njBnug6jlPwRYCB+n4+M/uSYD3JLT5D8CsQTtksPvP9n4fRPKD9k/4S6HTRK46D/QvwIWuUfiP2SKS3CUHuQ/fcMil+oN4z9Yf3pkqtfhPxQZ0JJkYOE/VOe8MCJB5D9hPSmhqbDcP1toaw8TmuU/7F066EDe4T8fsxD35CrfP9sC2qmMwuE/2pLbfw455z+YkI9Lp3zhP72ySl/YWOQ/MWP2jmbc4D8N6K3uFqDlPwrUsJfEZOU/dO85kpQd6T9rhKXFYKPiP/Mxr0b2VuU/pZQjdImD4z/Og7w1x1jnP0u6Px9V5uA/TH5gI4KV5T9R/bQ3tJPiP0d+rvad/uI/1U0zF11+4z9FBl7Y1QHkPwM69jP80+I/Ilxy0Rfj4z943EeL/j/oPwk6WPp6WuY//wMTITb75T/MAx+jBb7hP4ySweG7WuA/zb1kP8hB5j/P+j6H2wDmP5+cE6Cc6ec/fyRASGtE4T9A1ppbJ3fiPwAe5OZE6Ns/FLMiC7Dx4T+otRYvl2PmP6EiOwcMteE/q29gjOE/5T+uDJPNI5fiPw2abeKqPOQ/xGHFvQUO3z+Tj7HqAHnjPzzKEwu5QdM/waAcPBia4j+6Vu5aCPfbP6Dqy5QmD+c/Pr2lVg2w5T+46sQyf7HkP9G3O1M1Eeg/GsFn+9gp4D9w5pDSo8roP/nvufDZLuY/Atbr9Wlx4j8WQZwBgNXmPwmoAkXZUd8/GuE2DMKP4D+1nDmvpEDhPwORvojcgeE/NKNZAVaj3z9hSl5PJh3nP9ZaZn8lX+I/PaqV2fhT5D/uE2DZ91/hP3djhweGv9o/Saw6VW4Q5D90vN+HXCnaP+KL6Jj5+uI/wvbWeY2k3T9qoQIvLY/pP9IbyNDJHeY/PgGjYNWJ4z+OcdqybT7nP2Y/IuDlfeA/xwArFmvw5D82gi184tbjP/SbYjl8C+Q/oE+ZEYNF5z/BdIbtQYvlP89T0fC4H+Q/epD4TAjP2z93aTDmuiDgP6CxfUWtzuI/8JNfzrB/4z8qiISZrCrfPw6i03TY6uo/qLgjFW4N6D9eTmGQdCDlPxbTvWw2buM/NiThwFuF4j8slgVuYyfgP4cr3QqscuM/nWVU01kG5D/gAN9D5nPeP+xQ/RL1ROQ/ZHJ8BAO04j8=\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[1000]}},\"selected\":{\"id\":\"1078\"},\"selection_policy\":{\"id\":\"1077\"}},\"id\":\"1062\",\"type\":\"ColumnDataSource\"},{\"attributes\":{\"label\":{\"value\":\"dist_c\"},\"renderers\":[{\"id\":\"1065\"}]},\"id\":\"1079\",\"type\":\"LegendItem\"},{\"attributes\":{},\"id\":\"1138\",\"type\":\"Selection\"},{\"attributes\":{\"text\":\"distributions\"},\"id\":\"1081\",\"type\":\"Title\"},{\"attributes\":{\"source\":{\"id\":\"1062\"}},\"id\":\"1066\",\"type\":\"CDSView\"},{\"attributes\":{\"formatter\":{\"id\":\"1115\"},\"ticker\":{\"id\":\"1096\"}},\"id\":\"1095\",\"type\":\"LinearAxis\"},{\"attributes\":{},\"id\":\"1115\",\"type\":\"BasicTickFormatter\"},{\"attributes\":{},\"id\":\"1077\",\"type\":\"UnionRenderers\"},{\"attributes\":{\"bottom\":{\"value\":0},\"fill_color\":{\"value\":\"#ccebc5\"},\"left\":{\"field\":\"left\"},\"line_color\":{\"value\":\"#ccebc5\"},\"right\":{\"field\":\"right\"},\"top\":{\"field\":\"top\"}},\"id\":\"1141\",\"type\":\"Quad\"},{\"attributes\":{},\"id\":\"1117\",\"type\":\"BasicTickFormatter\"},{\"attributes\":{},\"id\":\"1078\",\"type\":\"Selection\"},{\"attributes\":{\"background_fill_color\":\"#fafafa\",\"below\":[{\"id\":\"1013\"}],\"center\":[{\"id\":\"1016\"},{\"id\":\"1020\"},{\"id\":\"1044\"}],\"left\":[{\"id\":\"1017\"}],\"plot_height\":400,\"renderers\":[{\"id\":\"1034\"},{\"id\":\"1049\"},{\"id\":\"1065\"}],\"title\":{\"id\":\"1003\"},\"toolbar\":{\"id\":\"1026\"},\"toolbar_location\":null,\"x_range\":{\"id\":\"1005\"},\"x_scale\":{\"id\":\"1009\"},\"y_range\":{\"id\":\"1007\"},\"y_scale\":{\"id\":\"1011\"}},\"id\":\"1002\",\"subtype\":\"Figure\",\"type\":\"Plot\"},{\"attributes\":{\"data\":{\"left\":{\"__ndarray__\":\"KQBsNTsT0z+Zlqxh1PbTPwkt7Y1t2tQ/ecMtuga+1T/pWW7mn6HWP1jwrhI5hdc/yIbvPtJo2D84HTBra0zZP6izcJcEMNo/GEqxw50T2z+I4PHvNvfbP/h2MhzQ2tw/aA1zSGm+3T/Yo7N0AqLeP0g69KCbhd8/XGiaZpo04D+Us7r8ZqbgP8z+2pIzGOE/BEr7KACK4T88lRu/zPvhP3TgO1WZbeI/rCtc62Xf4j/kdnyBMlHjPxzCnBf/wuM/VA29rcs05D+MWN1DmKbkP8Sj/dlkGOU/++4dcDGK5T8zOj4G/vvlP2uFXpzKbeY/o9B+Mpff5j/bG5/IY1HnPxNnv14ww+c/S7Lf9Pw06D+D/f+KyaboP7tIICGWGOk/8pNAt2KK6T8q32BNL/zpP2IqgeP7beo/mnWhecjf6j/SwMEPlVHrPwoM4qVhw+s/QlcCPC417D96oiLS+qbsP7LtQmjHGO0/6jhj/pOK7T8ihIOUYPztP1rPoyotbu4/khrEwPnf7j/KZeRWxlHvPw==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[50]},\"right\":{\"__ndarray__\":\"mZasYdT20z8JLe2NbdrUP3nDLboGvtU/6Vlu5p+h1j9Y8K4SOYXXP8iG7z7SaNg/OB0wa2tM2T+os3CXBDDaPxhKscOdE9s/iODx7zb32z/4djIc0NrcP2gNc0hpvt0/2KOzdAKi3j9IOvSgm4XfP1xommaaNOA/lLO6/Gam4D/M/tqSMxjhPwRK+ygAiuE/PJUbv8z74T904DtVmW3iP6wrXOtl3+I/5HZ8gTJR4z8cwpwX/8LjP1QNva3LNOQ/jFjdQ5im5D/Eo/3ZZBjlP/vuHXAxiuU/Mzo+Bv775T9rhV6cym3mP6PQfjKX3+Y/2xufyGNR5z8TZ79eMMPnP0uy3/T8NOg/g/3/ismm6D+7SCAhlhjpP/KTQLdiiuk/Kt9gTS/86T9iKoHj+23qP5p1oXnI3+o/0sDBD5VR6z8KDOKlYcPrP0JXAjwuNew/eqIi0vqm7D+y7UJoxxjtP+o4Y/6Tiu0/IoSDlGD87T9az6MqLW7uP5IaxMD53+4/ymXkVsZR7z8CsQTtksPvPw==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[50]},\"top\":{\"__ndarray__\":\"sJixhbRtwj+wmLGFtG2yP7CYsYW0bcI/sJixhbRtwj/FmLGFtG2yPwhlisiOpNs/CGWKyI6k2z8IZYrIjqTbP9z+HachCec/sJixhbRt4j/Gy2cWa7v0P7CYsYW0bfI/qbGboSAZ/z+wmLGFtG0CQCV/Vj3ZRgFAfUsvgLN9CkCwmLGFtG0CQG8+yUWOrA9Akn7lEGrLDEBvPslFjqwPQP0rEse0aRBAbz7JRY6sD0Dc/h2nIQkHQP0rEse0aRBAkn7lEGrLDEAlf1Y92UYRQO+xm6EgGQ9AootLS4+cB0DGy2cWa7sEQB6YQFlF8v0/kn7lEGrL/D8lf1Y92UbxP5J+5RBqy/w/8jHUN9hW6T+aZfv0/R/wP0ZlisiOpNs/8jHUN9hW6T+wmLGFtG2yP7CYsYW0bbI/CGWKyI6kyz+wmLGFtG2yP7CYsYW0bcI/sJixhbRtsj+wmLGFtG2yP7CYsYW0bbI/AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACwmLGFtG2yPw==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[50]}},\"selected\":{\"id\":\"1156\"},\"selection_policy\":{\"id\":\"1155\"}},\"id\":\"1140\",\"type\":\"ColumnDataSource\"},{\"attributes\":{\"source\":{\"id\":\"1140\"}},\"id\":\"1144\",\"type\":\"CDSView\"},{\"attributes\":{\"data_source\":{\"id\":\"1140\"},\"glyph\":{\"id\":\"1141\"},\"hover_glyph\":null,\"muted_glyph\":null,\"nonselection_glyph\":{\"id\":\"1142\"},\"selection_glyph\":null,\"view\":{\"id\":\"1144\"}},\"id\":\"1143\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"children\":[[{\"id\":\"1002\"},0,0],[{\"id\":\"1080\"},0,1]]},\"id\":\"1184\",\"type\":\"GridBox\"},{\"attributes\":{\"bottom\":{\"value\":0},\"fill_color\":{\"value\":\"#fbb4ae\"},\"left\":{\"field\":\"left\"},\"line_color\":{\"value\":\"#fbb4ae\"},\"right\":{\"field\":\"right\"},\"top\":{\"field\":\"top\"}},\"id\":\"1110\",\"type\":\"Quad\"},{\"attributes\":{},\"id\":\"1155\",\"type\":\"UnionRenderers\"},{\"attributes\":{\"background_fill_color\":\"#fafafa\",\"below\":[{\"id\":\"1091\"}],\"center\":[{\"id\":\"1094\"},{\"id\":\"1098\"},{\"id\":\"1122\"}],\"left\":[{\"id\":\"1095\"}],\"plot_height\":400,\"plot_width\":300,\"renderers\":[{\"id\":\"1112\"},{\"id\":\"1127\"},{\"id\":\"1143\"}],\"title\":{\"id\":\"1081\"},\"toolbar\":{\"id\":\"1104\"},\"toolbar_location\":null,\"x_range\":{\"id\":\"1083\"},\"x_scale\":{\"id\":\"1087\"},\"y_range\":{\"id\":\"1085\"},\"y_scale\":{\"id\":\"1089\"}},\"id\":\"1080\",\"subtype\":\"Figure\",\"type\":\"Plot\"},{\"attributes\":{\"data\":{\"left\":{\"__ndarray__\":\"Jn/0Ffpp5D8TuZNHF6TkPwDzMnk03uQ/7SzSqlEY5T/ZZnHcblLlP8agEA6MjOU/s9qvP6nG5T+gFE9xxgDmP41O7qLjOuY/eoiN1AB15j9mwiwGHq/mP1P8yzc76eY/QDZraVgj5z8tcAqbdV3nPxqqqcySl+c/B+RI/q/R5z/0HegvzQvoP+BXh2HqReg/zZEmkweA6D+6y8XEJLroP6cFZfZB9Og/lD8EKF8u6T+AeaNZfGjpP26zQouZouk/Wu3hvLbc6T9HJ4Hu0xbqPzRhICDxUOo/IZu/UQ6L6j8O1V6DK8XqP/oO/rRI/+o/50id5mU56z/UgjwYg3PrP8G820mgres/rvZ6e73n6z+bMBqt2iHsP4hqud73W+w/dKRYEBWW7D9h3vdBMtDsP04Yl3NPCu0/O1I2pWxE7T8ojNXWiX7tPxTGdAinuO0/AgAUOsTy7T/uObNr4SzuP9tzUp3+Zu4/yK3xzhuh7j+155AAOdvuP6IhMDJWFe8/jlvPY3NP7z98lW6VkInvPw==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[50]},\"right\":{\"__ndarray__\":\"E7mTRxek5D8A8zJ5NN7kP+0s0qpRGOU/2WZx3G5S5T/GoBAOjIzlP7Parz+pxuU/oBRPccYA5j+NTu6i4zrmP3qIjdQAdeY/ZsIsBh6v5j9T/Ms3O+nmP0A2a2lYI+c/LXAKm3Vd5z8aqqnMkpfnPwfkSP6v0ec/9B3oL80L6D/gV4dh6kXoP82RJpMHgOg/usvFxCS66D+nBWX2QfToP5Q/BChfLuk/gHmjWXxo6T9us0KLmaLpP1rt4by23Ok/RyeB7tMW6j80YSAg8VDqPyGbv1EOi+o/DtVegyvF6j/6Dv60SP/qP+dIneZlOes/1II8GINz6z/BvNtJoK3rP672enu95+s/mzAardoh7D+Iarne91vsP3SkWBAVluw/Yd73QTLQ7D9OGJdzTwrtPztSNqVsRO0/KIzV1ol+7T8UxnQIp7jtPwIAFDrE8u0/7jmza+Es7j/bc1Kd/mbuP8it8c4boe4/teeQADnb7j+iITAyVhXvP45bz2NzT+8/fJVulZCJ7z9ozw3HrcPvPw==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[50]},\"top\":{\"__ndarray__\":\"E4/QkR0Lwj8AAAAAAAAAAAAAAAAAAAAAY4/QkR0Lwj8Tj9CRHQvSPxOP0JEdC9I/E4/QkR0Lwj+e1rharBDbPxOP0JEdC/I/Y4/QkR0L8j/YskT25I32P4Do8gwQUv0/cfEP5sFyDkDYskT25I0GQIDo8gwQUg1AfRx8Vyi8E0DeE19+dpsSQCVSKm5TgBpAgOjyDBBSHUBeyPoumDIhQNiyRPbkjRZAj1tHRwWhG0DYhbO4a+ogQHJ2ntIaAx9ABZjtas8rI0D4bIH5aOIdQJ7WuFqsEBtA6XWe0hoDH0AoxX6oSM8YQIDo8gwQUg1AIoazuGvqEEDKu2HPlq4HQGL6LL9zk/8/qQElzBJaEEDYskT25I32P2OP0JEdCwJAE4/QkR0L8j9i+iy/c5PvP9iyRPbkjeY/E4/QkR0L0j8U17harBDrP8SO0JEdC8I/Y4/QkR0Lwj8AAAAAAAAAABOP0JEdC8I/AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABjj9CRHQvCPw==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[50]}},\"selected\":{\"id\":\"1121\"},\"selection_policy\":{\"id\":\"1120\"}},\"id\":\"1109\",\"type\":\"ColumnDataSource\"},{\"attributes\":{},\"id\":\"1083\",\"type\":\"DataRange1d\"},{\"attributes\":{},\"id\":\"1085\",\"type\":\"DataRange1d\"},{\"attributes\":{},\"id\":\"1092\",\"type\":\"BasicTicker\"},{\"attributes\":{\"toolbars\":[{\"id\":\"1026\"},{\"id\":\"1104\"}],\"tools\":[{\"id\":\"1021\"},{\"id\":\"1022\"},{\"id\":\"1023\"},{\"id\":\"1024\"},{\"id\":\"1099\"},{\"id\":\"1100\"},{\"id\":\"1101\"},{\"id\":\"1102\"}]},\"id\":\"1185\",\"type\":\"ProxyToolbar\"},{\"attributes\":{},\"id\":\"1005\",\"type\":\"DataRange1d\"},{\"attributes\":{},\"id\":\"1087\",\"type\":\"LinearScale\"},{\"attributes\":{\"formatter\":{\"id\":\"1117\"},\"ticker\":{\"id\":\"1092\"}},\"id\":\"1091\",\"type\":\"LinearAxis\"},{\"attributes\":{\"toolbar\":{\"id\":\"1185\"},\"toolbar_location\":\"above\"},\"id\":\"1186\",\"type\":\"ToolbarBox\"},{\"attributes\":{},\"id\":\"1089\",\"type\":\"LinearScale\"},{\"attributes\":{\"axis\":{\"id\":\"1091\"},\"ticker\":null},\"id\":\"1094\",\"type\":\"Grid\"},{\"attributes\":{\"axis\":{\"id\":\"1095\"},\"dimension\":1,\"ticker\":null},\"id\":\"1098\",\"type\":\"Grid\"},{\"attributes\":{},\"id\":\"1096\",\"type\":\"BasicTicker\"},{\"attributes\":{\"bottom\":{\"value\":0},\"fill_alpha\":{\"value\":0.1},\"fill_color\":{\"value\":\"#fbb4ae\"},\"left\":{\"field\":\"left\"},\"line_alpha\":{\"value\":0.1},\"line_color\":{\"value\":\"#fbb4ae\"},\"right\":{\"field\":\"right\"},\"top\":{\"field\":\"top\"}},\"id\":\"1111\",\"type\":\"Quad\"},{\"attributes\":{\"active_drag\":\"auto\",\"active_inspect\":\"auto\",\"active_multi\":null,\"active_scroll\":\"auto\",\"active_tap\":\"auto\",\"tools\":[{\"id\":\"1099\"},{\"id\":\"1100\"},{\"id\":\"1101\"},{\"id\":\"1102\"}]},\"id\":\"1104\",\"type\":\"Toolbar\"},{\"attributes\":{\"overlay\":{\"id\":\"1103\"}},\"id\":\"1100\",\"type\":\"BoxZoomTool\"},{\"attributes\":{},\"id\":\"1099\",\"type\":\"PanTool\"},{\"attributes\":{},\"id\":\"1101\",\"type\":\"ResetTool\"},{\"attributes\":{},\"id\":\"1120\",\"type\":\"UnionRenderers\"},{\"attributes\":{},\"id\":\"1102\",\"type\":\"SaveTool\"},{\"attributes\":{},\"id\":\"1121\",\"type\":\"Selection\"},{\"attributes\":{},\"id\":\"1007\",\"type\":\"DataRange1d\"},{\"attributes\":{},\"id\":\"1009\",\"type\":\"LinearScale\"},{\"attributes\":{},\"id\":\"1156\",\"type\":\"Selection\"},{\"attributes\":{\"label\":{\"value\":\"dist_c\"},\"renderers\":[{\"id\":\"1143\"}]},\"id\":\"1157\",\"type\":\"LegendItem\"},{\"attributes\":{\"bottom_units\":\"screen\",\"fill_alpha\":0.5,\"fill_color\":\"lightgrey\",\"left_units\":\"screen\",\"level\":\"overlay\",\"line_alpha\":1.0,\"line_color\":\"black\",\"line_dash\":[4,4],\"line_width\":2,\"right_units\":\"screen\",\"top_units\":\"screen\"},\"id\":\"1103\",\"type\":\"BoxAnnotation\"},{\"attributes\":{\"bottom\":{\"value\":0},\"fill_alpha\":{\"value\":0.1},\"fill_color\":{\"value\":\"#b3cde3\"},\"left\":{\"field\":\"left\"},\"line_alpha\":{\"value\":0.1},\"line_color\":{\"value\":\"#b3cde3\"},\"right\":{\"field\":\"right\"},\"top\":{\"field\":\"top\"}},\"id\":\"1126\",\"type\":\"Quad\"},{\"attributes\":{\"label\":{\"value\":\"dist_a\"},\"renderers\":[{\"id\":\"1112\"}]},\"id\":\"1123\",\"type\":\"LegendItem\"},{\"attributes\":{\"source\":{\"id\":\"1124\"}},\"id\":\"1128\",\"type\":\"CDSView\"},{\"attributes\":{\"data\":{\"left\":{\"__ndarray__\":\"kqF77BfW1T8w/Qo5Sv7VP81YmoV8JtY/a7Qp0q5O1j8JELke4XbWP6ZrSGsTn9Y/RMfXt0XH1j/iImcEeO/WP39+9lCqF9c/HdqFndw/1z+6NRXqDmjXP1iRpDZBkNc/9uwzg3O41z+TSMPPpeDXPzGkUhzYCNg/z//haAox2D9sW3G1PFnYPwq3AAJvgdg/qBKQTqGp2D9Fbh+b09HYP+PJrucF+tg/gCU+NDgi2T8egc2AakrZP7zcXM2cctk/WTjsGc+a2T/3k3tmAcPZP5XvCrMz69k/Mkua/2UT2j/QpilMmDvaP24CuZjKY9o/C15I5fyL2j+pudcxL7TaP0YVZ35h3No/5HD2ypME2z+CzIUXxizbPyAoFWT4VNs/vYOksCp92z9b3zP9XKXbP/g6w0mPzds/lpZSlsH12z808uHi8x3cP9FNcS8mRtw/b6kAfFhu3D8MBZDIipbcP6pgHxW9vtw/SLyuYe/m3D/mFz6uIQ/dP4NzzfpTN90/Ic9cR4Zf3T++KuyTuIfdPw==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[50]},\"right\":{\"__ndarray__\":\"MP0KOUr+1T/NWJqFfCbWP2u0KdKuTtY/CRC5HuF21j+ma0hrE5/WP0TH17dFx9Y/4iJnBHjv1j9/fvZQqhfXPx3ahZ3cP9c/ujUV6g5o1z9YkaQ2QZDXP/bsM4NzuNc/k0jDz6Xg1z8xpFIc2AjYP8//4WgKMdg/bFtxtTxZ2D8KtwACb4HYP6gSkE6hqdg/RW4fm9PR2D/jya7nBfrYP4AlPjQ4Itk/HoHNgGpK2T+83FzNnHLZP1k47BnPmtk/95N7ZgHD2T+V7wqzM+vZPzJLmv9lE9o/0KYpTJg72j9uArmYymPaPwteSOX8i9o/qbnXMS+02j9GFWd+YdzaP+Rw9sqTBNs/gsyFF8Ys2z8gKBVk+FTbP72DpLAqfds/W98z/Vyl2z/4OsNJj83bP5aWUpbB9ds/NPLh4vMd3D/RTXEvJkbcP2+pAHxYbtw/DAWQyIqW3D+qYB8Vvb7cP0i8rmHv5tw/5hc+riEP3T+Dc836UzfdPyHPXEeGX90/virsk7iH3T9chnvg6q/dPw==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[50]},\"top\":{\"__ndarray__\":\"o5uUXc1NAEASYIdiFRbaP21fh2IVFto/kYflCZCQ8z8SYIdiFRbqPwAAAAAAAAAAkYflCZCQA0ALnJRdzU0AQJGH5QmQkPM/DBK9sy7vEUCbEb2zLu8RQHbpXgy0dBhAE9avuHa3G0BtX4diFRYaQKOblF3NTSBAEepeDLR0KECMwvm0QGEkQGLVr7h2tytAjPQyXtaGMUBnmpsNxuYqQIu5HrMltjBAGC9HCYdXMkB6rkphA6Q3QI8bmLVJmjVAjML5tEBhNEAYL0cJh1cyQIwvRwmHVzJAkYflCZCQM0CI/Q1g8TE1QBRM2A7YWC1Af3M2tlLTJkAUTNgO2FgtQG1fh2IVFipAZ5qbDcbmKkBZS9gO2FgdQBHqXgy0dBhAduleDLR0GEAMEr2zLu8hQFlL2A7YWA1AbV+HYhUWCkALnJRdzU0QQJGH5QmQkANAEmCHYhUW+j9tX4diFRbqP5GH5QmQkPM/kYflCZCQ8z8AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABtX4diFRbqPw==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[50]}},\"selected\":{\"id\":\"1138\"},\"selection_policy\":{\"id\":\"1137\"}},\"id\":\"1124\",\"type\":\"ColumnDataSource\"},{\"attributes\":{},\"id\":\"1039\",\"type\":\"BasicTickFormatter\"},{\"attributes\":{},\"id\":\"1037\",\"type\":\"BasicTickFormatter\"},{\"attributes\":{\"fill_color\":{\"value\":\"#fbb4ae\"},\"line_color\":{\"value\":\"#fbb4ae\"},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"1032\",\"type\":\"Circle\"},{\"attributes\":{\"text\":\"drift stream\"},\"id\":\"1003\",\"type\":\"Title\"},{\"attributes\":{},\"id\":\"1023\",\"type\":\"ResetTool\"},{\"attributes\":{},\"id\":\"1042\",\"type\":\"UnionRenderers\"},{\"attributes\":{\"axis_label\":\"samples\",\"formatter\":{\"id\":\"1039\"},\"ticker\":{\"id\":\"1014\"}},\"id\":\"1013\",\"type\":\"LinearAxis\"},{\"attributes\":{},\"id\":\"1043\",\"type\":\"Selection\"},{\"attributes\":{\"axis\":{\"id\":\"1017\"},\"dimension\":1,\"ticker\":null},\"id\":\"1020\",\"type\":\"Grid\"},{\"attributes\":{},\"id\":\"1014\",\"type\":\"BasicTicker\"},{\"attributes\":{},\"id\":\"1018\",\"type\":\"BasicTicker\"},{\"attributes\":{\"axis_label\":\"value\",\"formatter\":{\"id\":\"1037\"},\"ticker\":{\"id\":\"1018\"}},\"id\":\"1017\",\"type\":\"LinearAxis\"},{\"attributes\":{\"data\":{\"x\":[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85,86,87,88,89,90,91,92,93,94,95,96,97,98,99,100,101,102,103,104,105,106,107,108,109,110,111,112,113,114,115,116,117,118,119,120,121,122,123,124,125,126,127,128,129,130,131,132,133,134,135,136,137,138,139,140,141,142,143,144,145,146,147,148,149,150,151,152,153,154,155,156,157,158,159,160,161,162,163,164,165,166,167,168,169,170,171,172,173,174,175,176,177,178,179,180,181,182,183,184,185,186,187,188,189,190,191,192,193,194,195,196,197,198,199,200,201,202,203,204,205,206,207,208,209,210,211,212,213,214,215,216,217,218,219,220,221,222,223,224,225,226,227,228,229,230,231,232,233,234,235,236,237,238,239,240,241,242,243,244,245,246,247,248,249,250,251,252,253,254,255,256,257,258,259,260,261,262,263,264,265,266,267,268,269,270,271,272,273,274,275,276,277,278,279,280,281,282,283,284,285,286,287,288,289,290,291,292,293,294,295,296,297,298,299,300,301,302,303,304,305,306,307,308,309,310,311,312,313,314,315,316,317,318,319,320,321,322,323,324,325,326,327,328,329,330,331,332,333,334,335,336,337,338,339,340,341,342,343,344,345,346,347,348,349,350,351,352,353,354,355,356,357,358,359,360,361,362,363,364,365,366,367,368,369,370,371,372,373,374,375,376,377,378,379,380,381,382,383,384,385,386,387,388,389,390,391,392,393,394,395,396,397,398,399,400,401,402,403,404,405,406,407,408,409,410,411,412,413,414,415,416,417,418,419,420,421,422,423,424,425,426,427,428,429,430,431,432,433,434,435,436,437,438,439,440,441,442,443,444,445,446,447,448,449,450,451,452,453,454,455,456,457,458,459,460,461,462,463,464,465,466,467,468,469,470,471,472,473,474,475,476,477,478,479,480,481,482,483,484,485,486,487,488,489,490,491,492,493,494,495,496,497,498,499,500,501,502,503,504,505,506,507,508,509,510,511,512,513,514,515,516,517,518,519,520,521,522,523,524,525,526,527,528,529,530,531,532,533,534,535,536,537,538,539,540,541,542,543,544,545,546,547,548,549,550,551,552,553,554,555,556,557,558,559,560,561,562,563,564,565,566,567,568,569,570,571,572,573,574,575,576,577,578,579,580,581,582,583,584,585,586,587,588,589,590,591,592,593,594,595,596,597,598,599,600,601,602,603,604,605,606,607,608,609,610,611,612,613,614,615,616,617,618,619,620,621,622,623,624,625,626,627,628,629,630,631,632,633,634,635,636,637,638,639,640,641,642,643,644,645,646,647,648,649,650,651,652,653,654,655,656,657,658,659,660,661,662,663,664,665,666,667,668,669,670,671,672,673,674,675,676,677,678,679,680,681,682,683,684,685,686,687,688,689,690,691,692,693,694,695,696,697,698,699,700,701,702,703,704,705,706,707,708,709,710,711,712,713,714,715,716,717,718,719,720,721,722,723,724,725,726,727,728,729,730,731,732,733,734,735,736,737,738,739,740,741,742,743,744,745,746,747,748,749,750,751,752,753,754,755,756,757,758,759,760,761,762,763,764,765,766,767,768,769,770,771,772,773,774,775,776,777,778,779,780,781,782,783,784,785,786,787,788,789,790,791,792,793,794,795,796,797,798,799,800,801,802,803,804,805,806,807,808,809,810,811,812,813,814,815,816,817,818,819,820,821,822,823,824,825,826,827,828,829,830,831,832,833,834,835,836,837,838,839,840,841,842,843,844,845,846,847,848,849,850,851,852,853,854,855,856,857,858,859,860,861,862,863,864,865,866,867,868,869,870,871,872,873,874,875,876,877,878,879,880,881,882,883,884,885,886,887,888,889,890,891,892,893,894,895,896,897,898,899,900,901,902,903,904,905,906,907,908,909,910,911,912,913,914,915,916,917,918,919,920,921,922,923,924,925,926,927,928,929,930,931,932,933,934,935,936,937,938,939,940,941,942,943,944,945,946,947,948,949,950,951,952,953,954,955,956,957,958,959,960,961,962,963,964,965,966,967,968,969,970,971,972,973,974,975,976,977,978,979,980,981,982,983,984,985,986,987,988,989,990,991,992,993,994,995,996,997,998,999],\"y\":{\"__ndarray__\":\"atSd2g1l6j8j04aJ92DpP+w3aKrkouo/nioA224J7D+G587WsDnpP0P9hI+yOek/zj7TEHIg7D8APr/28NPqPwpcg6RN2eg/rejeJNV36j+EkqG7yNvoP84GuUvW2ug/hcneLrX86T8HYRug64nmP5hPG9sS1+Y/YiOrekmz6D/bdzyPvvrnP1vDM9JQGuo/zj/sX6wl6D/TysrSHlfnP1SBKQPu8es/sbcuPR896T8jtQpyQrXpP3fisewFUuc/S0Ac75666D8K6DGtCMfpP37nBC0nwuc/Jr98ZHwz6j8DvYkRlKPoP/BNIEsfIuk/HoC3FiSj6D8Jf4ILS5DsP0x2O1ASlOk/LQUMklzo5z9QwnSvg+rqPzFaad2Kpec/hmQJiSbv6T/tx0RL63bmP7TD1DKTeec/7nMp/zvq6T/kB+puE8jqPy3a3t3K3+k/fnyo/zpq6T/OLtGWRB7pPzOHIVX/O+c/GF7EdsBy6D8IIr8f7NzoP6TGSOaYSus/1zTTllgm6j/ffctwdcfmPzUiaURYHuo/bIzYy9776D+JYwQvVYToP0Cy84EklOo/4rJLvOU/6z9oofFlDRfrP+8+v0PbQeg/jrgWVfIa6T/qYaAWSSHqP5prtOsuKes/C5cYilTV6D8H1WTYjU3pP8jBRvhx1Oc/mxqKPqKv5z+wipAbaebqP83Hpqwdxes/YCGeyhp86T9xprSmpTTrP7WduuK5Leo/eL8M5FuR6D9xHwStoC3qPwRwTmyUD+w/7Huv9+yK6T8fL+xgehrsPyAgcGqNaOU/pHf0UkDq6j9KrIslQb3pP0ItJ2ggH+k/3nAwai+/6T/qyc7kfWvmPwsFQ1WfP+k/DJ0ykd8r6j9TMMgF8vbrP5jWXAdRxeg/4xO05nBO6D8SOoWPFMzoP1TAFniMEOs/vdEAp0Eg6j/bwWk3nMDoP5qEWJfVa+o/RgA561zB6T841pJjWybrP8diXP8Jeug/iY/Tu2MT6T9G4oIU/vjoPzKBWu8kQuc/Aa7CD+QS6j8H8LM6hwTqPwYU6Mixm+k/bCUwW4M56T8zfIs43VXnP3Fpsr1N7eg/gAvZYDkN6T9UyiO7/FDoPyiIPpGJV+k/ThozZxk/6j+dZtmGLp7sPxlaFGkb4ek/S/ZBtxcD6j8R8UNhG3vpP4J0/Nqrh+Y/fuMsa72O6T9Hr/wxRbLpP78BkkGLiu0/US5FF89K6T/QOm8hHRXqPwyHoM5hi+k/Bctn0+i65z+9JO9Bs23rP15GQH6Xzeo/ZNO9T5vd6j/nOP1pHSXoPzktpz4v2Os/Memr3GZb5z+RE8IG+onqP0mCE4XPGu0/e4CWVuAD6D9eF6z6pLHoP8gVkM1qwuk/+5TrWWDL6D/R8hPBch7nPzrQ8PKutek/PGdr+3rm5z+7iZtdlVvqPwYbPvsAIeg/lGR6AHQU7D+PfSGKx1joP+iNmgCvFek/4+8eENHm6j+ywzEfcKHnP0EKuoDE9uk/SfS6cwGx6z+IgRfELAfnP3g1AN055ek/svE8SQwE6j81eoKr1dnqP56fyerxnuc/zTlLsL185z85MtEjY2/qP2/rN7M+E+o/rMnZrTMA6j8q4OhTgSfqP4Ao+tYPg+g/ln1PKrv46T9bruZ5pBHqP2BcvGwAdeg/ZOzEPNKV7D+GIjiVrlvqP1xy2V+ksec/gQtVPIam6j98Yb3RXgroP942DWf92+o/N2fbJil06z8CPV3ScknoPybDp+gyJOs/8Co80axC6j8mov3aUOrqP7MMhsKGouw/erSEyhY16T90b4Ci3mToPx9NckBBLeg/mbsDsXFL6D8P9ljmBHrpPwlJMfpVJeo/ylywu+4K6j85dN0MauzqPyc+evLsnuk/m57hsffs6z/LINRRMi3pP3a+qQLI8+0/b4WUk9+Z6j9g4KUdgjroP47f/mH24uc/SRvhfzhf6j8G2CHUET7pP+jsa/oNvuo/ItlaKXBb6j9hslzvxHvpP2Oz0NfAPug/gQ8MWx4t5z/zgx8dteLoP8/sr4Vh+Oo/BILI9Erx6T8dUeNrWJvnP2wdru+I4Ok/Q9NADm036j+EGjVuki/oP7vGfdiQ2Ok/guj8OXGx6T/QbUp6cMXnP1ow7FImLOo/kveaHkx/6j8nhwjBN1XrPyOECcE8Ses/MHoZf05l5z+7T2CEdxnoP6DqR/aObOo/FLMz9gts6j/NTadDkGzqP2jPDcetw+8/MHAOz2+D6j/y26RJumrrP/DYcO9bIOs/QZ1b7Gik6j8OVkE5dxjpP+azekp50Oo/iXW2AA1d6D+k8odemTjpP90+EovL0ug/qkjLuSK76T/EB2Ikr03tP/hfQafEnOY/FFRQMrGy6j+GZpIVCAXnP/5jCvVL2Og/YRibWKJX6z8zSDnZ7bPpP5Ysgt4n4Oc/YdXckZx06D8dYMOW9q/qP3jlnRtxbug/Z7We7ULy6T8Gz1wnRKzpP9bA9Fm0jug/UpGGbsIH7T/gRMjTQJ3qP46COgIaXOY/ZDaAwPjl6T9rnXRCiIroP1CNsbbB9uo/ANkEx/tU6D+clbOcmmrpP12WplpxaOo/oRWinDb86j+sgENm9a3nP38lbJqWEOk/8Xp7+Q/X6D9guI0Q/43oP2vyAOW6bOw/tRagAns/6j9/7axVJJXnP1tZoWaOEes/Qq0+ztX+7D/T6uZtf0DrP4C5ohxEK+c/wxYv+kHT6D8y6c3chqDrP8ypBxS9d+g/FFmocGNP6j+A9CLe49bqP53ZN+XtHeg/rH396TeB6T8mf/QV+mnkP0XePMUC9uc/VTXp6CUy6T+zI9sMgprnP+gC4FQ8Nuw/g2dOaNBP5z9Q9lqXW+XoP7qGP74mz+k/c2vbDvLn6z8yMXiKeE3nP7Q80igIdus/pXc9ncqd6T+raV71kgfoP6aGSKngVuo/7oCdhSLr6T8D9HxMwKPoP9soDeEwtuk/FQp+isb76D9rNqLBGMjpP37+jQjPqOo/vZlThDsj7D9AF9A8l57nP8JHQlxKA+0/JwK9WwZ65j8R4nrHbVvpP2EKXCGTiuo/3WGdu7EM6j9dBKXSiproP9Vue2ZaROk/r5aHtKrP6D8lN985MqjoP5a1O9aY9eo/EtETY9Ur6j/2bkjDyH3oPx/B4HsTCus/uS9NSngX6j/rHPRejObqP6QWIfh+m+o/Ot44LAtG6D+ln01cJrToP67SkAOxy+o/HC6nj5uT6j9jlbXoCZHpP91gkESoyek/idp9ee6k6z9BUgzYSqfoPzZdGeuweeo/gBbqKchG6T8Lf0sScEDpPz5rIbSoW+s/m8cAx7Dr6j9wKpxE0ObqP7pbfflSsOs/sj4uAzSi6T/uMUiN7bDqP6RuxcWDGuk/PnHd52Ae6j8e55UcS2TpP4+UNV1Uwek/WPpZVmCN6j9c9mDxdEroP1cE2U6k8uw/tDMlCIn95z9JpDSyRKjnPwiFpFP2c+s/jqc5c93d6j9xcF5OPZnqP6g07mb4muo/8GzLbpWU6T8s1/aoFSroP3vttkimuOk/+4VRDDyE6D8+JKFQAinrP5PmEoRdXek/1In08XlH6D+JuRra9RXpP2gq7Jm8Quo/VGjcy7Ky6D/OGeuK0UjoP6uyXw5q/ek/VQrsNPD96T8C9lrB9MnoPzMhXaep2Og/qsiszKX46T/y3ovzdkjnP8WYl1MaWec/wsBiQ1Nz6D8N9T8LLELpP0K3A5/yGOo/1WV86ef16z/4AM665fjqP+ayYNQWWOk/Q/Enm8+R6T/Aj+LG9v7nPzvJY1sEkuk/gWhDjF0j6T/6iv0XyR3qP5OGCybERug/ke5vB1Nu6j9VC4HsaA3sP3humEUNbek/VbyHICQ+6j9h409xSLTqPzM5kZVC9eg/+qVaZmP16T+F20gCwp7pP/rJZ66bwek/r8M3pvlc6D8KlJSto6PpP7mUcIGUZeo/JT81Cf3r6z/DRMNvhCLrPzhVFSWLC+0/BvhNYEtf6D/2vmMM5/7qPyZoF2ey5Ok/6yp+FIsa7T+T+rdhhU7oPzXBCWKmQeg/AyvKuRak6D/S9d/9pjPmPzZ6szBAwug/ERVixahi6D/3shCIM9fpP+BHuE+VJeo/AG0TXxSa7D9LWz3D5B7rP+4tA95Mreg/SeZm/psp6D9Y1U8QF2PqP0oBTh3VfOc/POh++MOH7D9WV0PcsnzrP/yblvds2eg/odm/bObb5j/qmGloJcTrP1Z7Cjqvauk/mgI3DJyU6z9KBgm+hQznP7f/1pIYpOg/760Xcb+b6T9AoE/f16zpP4nml9BA4eg/gzIpJriY6j/Qnet7TeTnP1S9ZgdIX+k/Oxbggt/K6T//hOZrUGzqPwSVB9QTveo/TvZSVPLM5z//+eYROiXnP0Z1obnvpOs/uiDvP7ch6j9B/R8ZBWfoPyoxY6zzFOw/lJ/r9vrI6T9GHFbfo3zrP+nZCG1Btek/at59rq7l7D/hM4xtlmjsP0GmINGfM+k/hb+5Mo4n6z/18lQs8qHqP2VmOQUxyus/6XavCl4O6D8uG0BPm7LqP/Ovk3MhS+s//mf/ZTjJ5j8TOMLz77TnPxK2pZtUVuY/BGP0PkAr6T/aBt1bgb/qP/mqoif3AOw/Fog3APO36T9tLUNRrjTsP/x8LHlPZOc/5jhqAeXf5j/AZGkA2YLpP9ny/cfpNuo/8X6jTjWM6T+k7U+VxkrmPyJmwa8Ydek/01IUD0qD5z87mzPb5avqP9YCVjbCL+o/7HuyD6AY6D/Wd5K/HsfoP3MCIAO/5+c/COlkOO1/6T/n7AiI0yDrP3OthrvYBeg/jkw5tQ5o6j/pnQoPaMDoP2J5ktvWVOg/doFAp8Jt6T8ccByTkPHnP6y6OUPTtug/AV7E//Ku5z8cAkz2Wb7sP/aIXUALqOk/ssoEEP566D//XRcFP/HpP9rYeSaXa+k/vyYUQhc/6T+AKmulKZXqP8ZkWwrgz+o/i2/ahU7A6D+mYmauvq3oPzE901bwKOk/4sjYq7vq5T9HqjpN+iznP37eU8F4yes/0opt92A77D+R7VBHmDPpP6eRuvrBheo/6gVRixYZ6j9AjvqmtYbuP16nyYktZOs/R2nmdzRl6T8CyTXsNRLoP7tXjH6ZB+c/jcL8TvDs6T8vHuF5zGPoP0upSn0LU+c/r+rlhMOQ6D99RnISmd7nP0fFAzmnTOw/clKeOrgC6z/epqObVZbpPz3nk/3I9+s/dFlLQUq56T9M+d1n0TjoP1T/OLx4Cew/F5TuaVZ26j//bCh1vvDnPxmMnSSjS+k/ekoWX/Iy6D9NWPeJNGPnP1yz1Fr2FOs/CANCcrKn7D84Uigpv1znP/H7+TMxgOo/zmAgyBiP6D/+4iXNEtLoP2oSSJj0pug/DeP5mbU36D/v3yF2ea3pP2RAQio+Reg/t1O8DWEI6j+1Pp7ABYXpPzvTzhS6N+k/4JYNp9wl6D9BTRa+Wq3oPziSahwCz+o/+sL3ksZm6j+HSs+AMQnoP8XO3lhJwuk/HAccQF7N6j/Ts8/Ez+3mP1qNswspeOo/C9eCdjCK6D9LzPo0UYPqP5pxoBP4YOg/8dCg/lG25j9O5/Nn9f7mP+PxDqxLrek/JhZrevsD6j8WhAkhMSfoP90SY98qn+o/FQ0LmArx5j99A4OhiH7pP1PzBFmRqec/PgVRoZuO6D9Ar/61A63pP96zMbgsOeg/VA9QBxb86D9y14IMxzXrP2LpZBpOreg/svPqRObv6j+44iFA38rnPw59NZibcuo/MqqaBhHo6z/q+8DjNqXlP7KqeBMxU+g/THqP//eF6j++WIW/bkbpPyKuxhCfMeo//9uyKTWi6D+SLoIyEb3pP9IDoqjVWek/tH6fbOx36z/R1huPzwHqP0/ZDc7hI+o//3F6K+Xw6D/Ac7Bh4NHoP8gdKZZs6Og/WcOr5yo76j9oAX4tKu3oPyxRL7JKEOo/naOTJa/r7D9Ow0ylaf7qPywuDI4PFOk/rYJLAp6F6z+kT7fLc/LoP9WTs8DIVuY/zoC+FrD85z/gHVzZUpvmPwy0XL2eCek/hZ8z6CSh6T9aoAnLREjsP8Asd2uCH+o/Dl6QPts/6T9LryMUU+3qPzTZGkLrD+Y/eg+8kxv66T8JZJWsWNXqPz6ySZj4O+c/iPxp5xRu6z9smlmFPyTqP5LLi4F/7+g/DUl6lsmc6j/CP98ArTvtP6H0tagX5Ok/jio6akX/6T9gqVgect3oPzPvg/WAPeg/cmj0nrTt6j8uvWG08jroP9b3Jt3ptuk/VoYllvPV6D/a1mRGyl3qP47LiZtEIuo/Zs9njJNC6z8G/iOBssjoPwjZdykPK+k/+uwpybII6D+ynzYTnuPoPxgHl2wkNOo/nbIMnKnP6j9ncauO4R/oPwk4xWPK/eo/pahHiN7E6z9A/kZk8ULqP9eHk+dVmuw/ofby66dc6D9rvlAYypvnPzQofUMJweY/ZopLN2H+6z9Md+/PoKXqPyqm8x/Vguk/Y+svcEYM6j8MWwmFmcznP2UhZEhhg+0/W3RKbIfO6T+Z48F5aMbpP/NGEL/fwuo//ZzYEp9e6j+0Zp2KTfXpP152gljSVeg/w5jxo7Za6j9Rz/0repzsPx3K9R2vwOs/LyV2Uysm7D+qPUfANMjoP589ewNCBOg/5Kas4hNm6T/A9tLHbLDpP+ZtheXHWes/7sci0l3k5j9OElmTGgzsP7mMXkXhWOk/KhIi4L/q6D8sl6bDCvvnPwg7EU3F8+Y/dQi1SsXq6j9Ip8uLobfpP+qWXWU7iec/6MmGvyKH5z+Ue7YFEBDpPzOABTE7Rew/nYJSeUUv6T/x1DGj6THnPyANdJLxNOk/xu8EduQp6T+9rAWK9EjlP8vx315cg+k/hPoBXAI76T8GoOYgxLbqPyTZ2LLujuw/UY9XgQpn6z+aFcaTdivpP5s+6fJd1Oc/7uDS7qW37T/JdVUa27HpP47BLjFOn+k/6Vu55reP6T/HiOhKvOrpPya4TFB4Xuk/lvxbx6Cu6D9FjqxIm7noP1CJsisvjOk/qtT5YQO76D8zslNNnnXoPyQwRZ4xxek/FYYJTSkx6T/+mfixogHsP6IHqkTDW+U/FGONY65Y6z+gSkka/5frP7GJWeBWSOY/H7CKMzwN6T/oo8YzdQHpP+KEPU0VWec/mNLgmwFb6D+o/j9ItdLnP5LNJnlUZ+w/o6A2l9oY6z/Vf8bQbaLrPzs0j2Yyweo/gwLl8CPL5z+Q++mpwcLoPzMm8z0MYuo/cXEINgSl5z+GGnnnpL3qP/rR0KcpN+k/hiQpyhIA6T9kHu0nz7zqP61uD/yRT+o/uibejb8F6T+cwwokdnTrP1F83uTL3uc/+9MmIeOV6j9NgUPGiIzqP8cVpE3PGuk/tBRBIC8f6j8c80LVJJnnPxmVBtsUFOs/GTqrNN1N6T+ZzlwefsPoP72sljBGR+s/ufWfzxl56D/lspC6sVjnP8zxZTMBHOc/mu1cWdKR6j8HgePZIo3nP2AiPhtdaOw/eX6aetdE5j/eCHvxd1DsP5hummII8Ok/xHnFfvxx6T/4sz6xZrroP0sSFg4WPeo/L9O/UC+K6T9/APsvg13rP3sVsjxjyOk/rp3J4SnX6T/SyJkYqgTpP/glJ2tGguk/jERr9KwX6j8UOF5yHd3mP0iUvhxicec/y2MzfQrK6j9YpMcjlt/pP7tehYw9Tuk/Rgu5iSah6T9kJfUu+CfqP+wxArKDvOg/C4U2b85a6D8OwaJ20enpPwiOVMfbCOg/DXkQAdJA6j8oWfvEOODmP9NK4WMkP+s/5GqrCS1b6j/QsHJDeALqP5KX1jccLOs/NKrLQMdD7D8iErICFjnrP9c1OvKTp+Y/NNnxOnyN5z+gLp6frJnoP+xl5XFJpOk/5gcfFaJt6j+jK3XYVXDoPwpBwoMZ5uk//O9o9TFk6D/q9pFPH5/oP/uRTX5uWec/Sszbk3Ef6D8KmEAy82/nP+8Sb9/hCeg/Y5s28ytJ6z9ELDzpuRToP+8VmN3Sz+0/V+cqu6lj6j9+0oMST+XpP573YkMEOug/JzTIaXK46j/Kv1yZ0a3oPzGCn0GTy+k/WZOk6zWy7T80KU39QHLpP9h/iqRXcOs/aNMqNZR56D8FunXKRIvpP5oQNoHrbuw/WWnLVsuY6D8mPx2b+n/sP23w88R+u+o/H4QNrzaz6D/sxJRbopzqP2gpUVP1J+s/OeGsGUuY6j82uLqabxbnP1/Nqb3Db+g/ZdnRYzc06T9qqHOwHHvpP4xuWsnTl+o/R4+F5mLi6T+nTmqYpHbnPwJU8ztUNeo/vm/lJ7KT6j+V9gPi437qPy8/SqxJVOs/D4T7rCzv6j880f4erlXqP6v5HzHcfOk/lZUKOUXx5j+3GttVkknqP2U8gDur7uk/UhCAtNYI6j8yOPzOpI7nPx22D5vM3uc/mgE+rvhI6z9tEwbxZYnpP3J3pSC+sOo/4pE4/zKl6T/gLuPByaXpP9lAu8nrGes/lw1oYzrG6D+QFQ+Y+MDpPxTSzoVA3Og//E5OXqHn6D/b/JqN9hrpP2nOoAOW9Ok/cLgkK4HV6D8LWGss9ZvrP75TsDkrK+g/WT4ZsA5N6T/hruJ0fOXoPzy8azpI6us/oMi03Bvq6T9MHoxXPkDrP8Id/E0dOec/4d+W2fsG6j+J8m0m/gXrP5pGnrNNu+k/+0uiUAVO6z9EWoD5t8XoPzh64GPe2us/eQw2ijpH7T9YSyg4+wTpP7kTWkYf4+g/sPkVAujs6z/PvIW+lyDsP7tinsBvw+g/eGGd0X3t6D/mSJpXLibpPxYHAL/pcuc/HBwv9lEh6D8cjSPPTf7nP3VJpjAcX+g/0Gk+oGSL6T8q+nPLiPnpP/8bSFyvFOw/eephl6wA6D8w4slIxyzrPyFZTT7zQek/WCBB9FaF6T/s5GiNAa7qP+xs0am7zec/YEAPKzw26j9mmAphx93pP4Wpj9tOY+o/YmFHIQsQ6j/pDN56SoftP2V2QLdhlOg/kfyfiBrA6D+gtXaUXJroP0LZOJoTtug/ZOwut4aU6D+fahIFn4DrP67pJ0Rw3+s/bw1wg9Kv6D/+ZdHKqkToP8CblRqxWuo/JCUh0Wi36D8jLLZP2ZzqP7Oo9p637Ok/msVUT8As5z+XwwlIdRPsP4hMzwUxeew/EJ5xDJqe6D8wvxglzProP9JaQMKwDuo/mHix75ci6j8bKND4VqfqP088zdL60Ow/tkWhVh9R6T8XJx0RnlLoP2x+AX+hZOc/iQ6+BzZu6D8vbTH8B4zpPxDxcKGmeOw//FsXH5bF6D+hPq93Q/XpP5sj14jfkuk/umSgql2A6z857WaroaTtPxeLfPknwOg/BTibJyDR6D+So7fNSUXrP5ti5RrnsOo/B4Po5QKO7D/cTHjoxojqP0ks+hdvBuk/0UZ7P4iL6j8S5riYuV/rP9TocWSr6eo/posvI2Fp6j/PNUWMgk7rPygdGiGLeOs/DZuAebvP6z+FdgzDT6PqP6BZyv8lVek/UMUhpbHV6T+2LA88yYfrP+Dgqa/7Sug/29NgzJsw6j9YSkEJffjoP+mxkLZfpek/Yad4/kCl6z+rgSXK3+fpP/KrMtOerOk/kilFWZps5z+Ukzr1Q8vqP7yJpoX9oeo/vNJoTKsP7T8W7oO2iBvpP3r1aS1d8+k/vrXiX7//6T/trqWQuR/sP0+CoSORcuk/d4CmIOML6j8NVkcrmJLqP81YG/wI5uk/NKyCpL3i6D8+xCxpGenpPxpabwxcUes/uuVZqyP15z9Tgxx7ENDpP8sjlJzUeug/ZxVHUheD6z8r0HbgsynnP4+EgWWqtOg/aOfEIRs06j8HQ0Sx1hrsPxz7dy+rfuk/wsHJtTC26D9Uniw3H5zsP0xLb1Z+SOc/i06jFfgU5j+Q/o111E3qP/6JImb1y+g/8rbnk0335z/IsjEovrvqPwlsLfV1/ek/m2Enq42y6D9Zpxf0L43nP3L9rGH1/uo/UtiRIuyj6j9YOcdB+nDpP86SAob7jew/GBdhFEvj5z/ZPhyxvijnPz1vxscxfug/oY5Kj+2G6T+oRDSXRf3pP5kGeyrKNuk/nqewSM0p6j+lHf8t+JjnP4rJlUr36Os/mFyybPN36T98NDSPPmPrP/NSEPX6Jeo/iF93pa9U6j8vfJEH+oLqP9IPvz77UOo/owpQ99ug6j8z5yZYBbrrPw6LlFYY6uk/USJMCAK86j8HAWYh2HTpP/IEvtV45+s/2gX/uYyE6D9r1TvkQ3vsP9MX0rsmiek/tCfn9I1P5z/Fn01SEs7pPwFqYCikgug/YWYKd+3x6j8gTA0ESY7oP4m4Q+DX4ug/C2jT5KST5j8gSEvYVeDoP5DiXW7HuOU/8InuWdUQ5z+23BjbENHqP07B/bd22+o/E4YPD95H6j+CZWjNhg3oP37BorMOhuk/P3qt2B+Y6T/ehT5CIr/nP0jsd1ZkAew/0jlBtPcA6z9kTbnTFz/pP81U2MicpOk/uOQBH/Tu6T94bI4uTlXmPzDSQixbNOk/wyNXXkKC6D99xTAhVv/nP6baERl2Juk/SaXbse557D8dOivYFqDqP4yNVySlr+g/RuKZQSGE6j8=\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[1000]}},\"selected\":{\"id\":\"1043\"},\"selection_policy\":{\"id\":\"1042\"}},\"id\":\"1031\",\"type\":\"ColumnDataSource\"},{\"attributes\":{},\"id\":\"1011\",\"type\":\"LinearScale\"},{\"attributes\":{\"axis\":{\"id\":\"1013\"},\"ticker\":null},\"id\":\"1016\",\"type\":\"Grid\"},{\"attributes\":{\"items\":[{\"id\":\"1045\"},{\"id\":\"1061\"},{\"id\":\"1079\"}]},\"id\":\"1044\",\"type\":\"Legend\"},{\"attributes\":{},\"id\":\"1021\",\"type\":\"PanTool\"},{\"attributes\":{\"overlay\":{\"id\":\"1025\"}},\"id\":\"1022\",\"type\":\"BoxZoomTool\"},{\"attributes\":{\"active_drag\":\"auto\",\"active_inspect\":\"auto\",\"active_multi\":null,\"active_scroll\":\"auto\",\"active_tap\":\"auto\",\"tools\":[{\"id\":\"1021\"},{\"id\":\"1022\"},{\"id\":\"1023\"},{\"id\":\"1024\"}]},\"id\":\"1026\",\"type\":\"Toolbar\"},{\"attributes\":{},\"id\":\"1024\",\"type\":\"SaveTool\"},{\"attributes\":{\"label\":{\"value\":\"dist_a\"},\"renderers\":[{\"id\":\"1034\"}]},\"id\":\"1045\",\"type\":\"LegendItem\"},{\"attributes\":{\"data_source\":{\"id\":\"1046\"},\"glyph\":{\"id\":\"1047\"},\"hover_glyph\":null,\"muted_glyph\":null,\"nonselection_glyph\":{\"id\":\"1048\"},\"selection_glyph\":null,\"view\":{\"id\":\"1050\"}},\"id\":\"1049\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.1},\"fill_color\":{\"value\":\"#b3cde3\"},\"line_alpha\":{\"value\":0.1},\"line_color\":{\"value\":\"#b3cde3\"},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"1048\",\"type\":\"Circle\"},{\"attributes\":{\"fill_color\":{\"value\":\"#b3cde3\"},\"line_color\":{\"value\":\"#b3cde3\"},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"1047\",\"type\":\"Circle\"},{\"attributes\":{\"data\":{\"x\":[1000,1001,1002,1003,1004,1005,1006,1007,1008,1009,1010,1011,1012,1013,1014,1015,1016,1017,1018,1019,1020,1021,1022,1023,1024,1025,1026,1027,1028,1029,1030,1031,1032,1033,1034,1035,1036,1037,1038,1039,1040,1041,1042,1043,1044,1045,1046,1047,1048,1049,1050,1051,1052,1053,1054,1055,1056,1057,1058,1059,1060,1061,1062,1063,1064,1065,1066,1067,1068,1069,1070,1071,1072,1073,1074,1075,1076,1077,1078,1079,1080,1081,1082,1083,1084,1085,1086,1087,1088,1089,1090,1091,1092,1093,1094,1095,1096,1097,1098,1099,1100,1101,1102,1103,1104,1105,1106,1107,1108,1109,1110,1111,1112,1113,1114,1115,1116,1117,1118,1119,1120,1121,1122,1123,1124,1125,1126,1127,1128,1129,1130,1131,1132,1133,1134,1135,1136,1137,1138,1139,1140,1141,1142,1143,1144,1145,1146,1147,1148,1149,1150,1151,1152,1153,1154,1155,1156,1157,1158,1159,1160,1161,1162,1163,1164,1165,1166,1167,1168,1169,1170,1171,1172,1173,1174,1175,1176,1177,1178,1179,1180,1181,1182,1183,1184,1185,1186,1187,1188,1189,1190,1191,1192,1193,1194,1195,1196,1197,1198,1199,1200,1201,1202,1203,1204,1205,1206,1207,1208,1209,1210,1211,1212,1213,1214,1215,1216,1217,1218,1219,1220,1221,1222,1223,1224,1225,1226,1227,1228,1229,1230,1231,1232,1233,1234,1235,1236,1237,1238,1239,1240,1241,1242,1243,1244,1245,1246,1247,1248,1249,1250,1251,1252,1253,1254,1255,1256,1257,1258,1259,1260,1261,1262,1263,1264,1265,1266,1267,1268,1269,1270,1271,1272,1273,1274,1275,1276,1277,1278,1279,1280,1281,1282,1283,1284,1285,1286,1287,1288,1289,1290,1291,1292,1293,1294,1295,1296,1297,1298,1299,1300,1301,1302,1303,1304,1305,1306,1307,1308,1309,1310,1311,1312,1313,1314,1315,1316,1317,1318,1319,1320,1321,1322,1323,1324,1325,1326,1327,1328,1329,1330,1331,1332,1333,1334,1335,1336,1337,1338,1339,1340,1341,1342,1343,1344,1345,1346,1347,1348,1349,1350,1351,1352,1353,1354,1355,1356,1357,1358,1359,1360,1361,1362,1363,1364,1365,1366,1367,1368,1369,1370,1371,1372,1373,1374,1375,1376,1377,1378,1379,1380,1381,1382,1383,1384,1385,1386,1387,1388,1389,1390,1391,1392,1393,1394,1395,1396,1397,1398,1399,1400,1401,1402,1403,1404,1405,1406,1407,1408,1409,1410,1411,1412,1413,1414,1415,1416,1417,1418,1419,1420,1421,1422,1423,1424,1425,1426,1427,1428,1429,1430,1431,1432,1433,1434,1435,1436,1437,1438,1439,1440,1441,1442,1443,1444,1445,1446,1447,1448,1449,1450,1451,1452,1453,1454,1455,1456,1457,1458,1459,1460,1461,1462,1463,1464,1465,1466,1467,1468,1469,1470,1471,1472,1473,1474,1475,1476,1477,1478,1479,1480,1481,1482,1483,1484,1485,1486,1487,1488,1489,1490,1491,1492,1493,1494,1495,1496,1497,1498,1499,1500,1501,1502,1503,1504,1505,1506,1507,1508,1509,1510,1511,1512,1513,1514,1515,1516,1517,1518,1519,1520,1521,1522,1523,1524,1525,1526,1527,1528,1529,1530,1531,1532,1533,1534,1535,1536,1537,1538,1539,1540,1541,1542,1543,1544,1545,1546,1547,1548,1549,1550,1551,1552,1553,1554,1555,1556,1557,1558,1559,1560,1561,1562,1563,1564,1565,1566,1567,1568,1569,1570,1571,1572,1573,1574,1575,1576,1577,1578,1579,1580,1581,1582,1583,1584,1585,1586,1587,1588,1589,1590,1591,1592,1593,1594,1595,1596,1597,1598,1599,1600,1601,1602,1603,1604,1605,1606,1607,1608,1609,1610,1611,1612,1613,1614,1615,1616,1617,1618,1619,1620,1621,1622,1623,1624,1625,1626,1627,1628,1629,1630,1631,1632,1633,1634,1635,1636,1637,1638,1639,1640,1641,1642,1643,1644,1645,1646,1647,1648,1649,1650,1651,1652,1653,1654,1655,1656,1657,1658,1659,1660,1661,1662,1663,1664,1665,1666,1667,1668,1669,1670,1671,1672,1673,1674,1675,1676,1677,1678,1679,1680,1681,1682,1683,1684,1685,1686,1687,1688,1689,1690,1691,1692,1693,1694,1695,1696,1697,1698,1699,1700,1701,1702,1703,1704,1705,1706,1707,1708,1709,1710,1711,1712,1713,1714,1715,1716,1717,1718,1719,1720,1721,1722,1723,1724,1725,1726,1727,1728,1729,1730,1731,1732,1733,1734,1735,1736,1737,1738,1739,1740,1741,1742,1743,1744,1745,1746,1747,1748,1749,1750,1751,1752,1753,1754,1755,1756,1757,1758,1759,1760,1761,1762,1763,1764,1765,1766,1767,1768,1769,1770,1771,1772,1773,1774,1775,1776,1777,1778,1779,1780,1781,1782,1783,1784,1785,1786,1787,1788,1789,1790,1791,1792,1793,1794,1795,1796,1797,1798,1799,1800,1801,1802,1803,1804,1805,1806,1807,1808,1809,1810,1811,1812,1813,1814,1815,1816,1817,1818,1819,1820,1821,1822,1823,1824,1825,1826,1827,1828,1829,1830,1831,1832,1833,1834,1835,1836,1837,1838,1839,1840,1841,1842,1843,1844,1845,1846,1847,1848,1849,1850,1851,1852,1853,1854,1855,1856,1857,1858,1859,1860,1861,1862,1863,1864,1865,1866,1867,1868,1869,1870,1871,1872,1873,1874,1875,1876,1877,1878,1879,1880,1881,1882,1883,1884,1885,1886,1887,1888,1889,1890,1891,1892,1893,1894,1895,1896,1897,1898,1899,1900,1901,1902,1903,1904,1905,1906,1907,1908,1909,1910,1911,1912,1913,1914,1915,1916,1917,1918,1919,1920,1921,1922,1923,1924,1925,1926,1927,1928,1929,1930,1931,1932,1933,1934,1935,1936,1937,1938,1939,1940,1941,1942,1943,1944,1945,1946,1947,1948,1949,1950,1951,1952,1953,1954,1955,1956,1957,1958,1959,1960,1961,1962,1963,1964,1965,1966,1967,1968,1969,1970,1971,1972,1973,1974,1975,1976,1977,1978,1979,1980,1981,1982,1983,1984,1985,1986,1987,1988,1989,1990,1991,1992,1993,1994,1995,1996,1997,1998,1999],\"y\":{\"__ndarray__\":\"ejDHCiRk2z9mlL5+lcjaPz8UC8Ijrdk/4/sVnJzF2D+3HBbRZH7aP3TB6IuJGto/0Tue2e++2j8Xhi2su2naPy7jiXaE8do/7xH00Dbq2D9+Pb2fSEnbP76dzHRZ2tk/QzN+GZ9B3D91C+dVxLfYPwF+aMxw0ts/Sf9Pj3Pa2T81BH6yJMTYP1JOcFAK+9g/AyuK66Aw2T9EevA3lyTaPxYlNjjsRNo/B6zTJ5zd2D9QsraWnpHZP55EP0OUV9w/W6ewbK7P2z+iAA8VkyjaPzFPFJANptk/dri2ju7A2T/r7r03o2LaP1ZogopzStg/E4iLSkNF2T9G2XWf1nbXP/yB4NxqHNo/51yTVKxt2j+AMFX7RPvYP1Gd9i1dnds/E5ps7PAH2D/1h4DuwLnXP8SCs/sl49k/OFVtkrbw2j9DD2uxY8HbP/91F5k7A9k/IK8c6A/72j87roNG+4zZP0bRj5IIYdk/m+5CXii72j/U/75oWm/aP5fnzD0Mldc/zp4qyG592z/C/cEN1F3bP1qJ4JidzNg/DT/CAUwb2j8UDRHbezvaP2t/GIAE79k/ojVoqEbl2D8g7pi/hb3YP7SjGvg5kdk/TSGGPeEZ2z98yQcWukvaP77aazcoINk/iBY5YXiW2j/Lm52GMPTVP8tYKw0HEts/vcGt1Idf1z9tU6bZ1SLZPwWKs+G0Ktg/Tn6d2Fvx1z/5YQrP+hXbP43Dx/pXANk/IcPMcyQL2j9CTdedOYrZP+fgW6/qNdo/3wom48Wy2T9zZGFpMPXXPwRZo4QO4No/STe3S8732D8xytsLipvXP88NhrJQDdk/AhuzdF6F2z8Mz3teM7DaP71yBopaJ9k/XqiCvScn2T/iB3jlNDDZPw0H9dgbQtw/2or7qsAW2j8yLKcbhCbaP4mqTQo069o/VEi+r9jn2T/Mh+CRt0TZP+2xspRCWdk/dydDQCOC2T/4UygsZ43ZP59PEZcHiNo/ueYAI5+q2T9Y93Tnq4naP3euPZomf9k/5h90/12z2T8kPKVg1ArXP6iKk7zcxdo/GE32KCML2j9TvoPBoODaP7Lv8haO5NU//uXkKutF3D94Oyn522vZP08fYba6BNs/8ULc99dE2D+GqwLNZGLaP92morJqQNg/MfIqETTN2D8yH/EvygzcP6BXOQAeW9k/kjX/MNng2T8n09crtLbaP4QeF2kGPNo/QdD9puPK2T/+FHO/MBHaP3VWSbYmrdw/+wNbL7iG2T/Blr4Cf9vZP46F8+Dg8do/ehM81tsD2z/bB0/rkB7bP+NDrCzmato/ttYAZg8j2D8dTqjE17DbPxY+vC33Idg/e6O6gMT82T+OXQ9bcKLYP/1FGEmVhNk/KB+aMFQF2j/w4kD/5gLaPzMCzuDaI9o/op06g2Gq2z9CJpDQNi7aP6oYyEGYSdk/pDbkF4PV2j/78didXR/bPxUkgmRWB9g/RMVpJltd2j8j2pU7XH/aP6DeBSoYONk/f9V1QGRc2z8u02cGbmjZPyZvZ7e/wtk/vWV5SONg2T9xbRl3tJ7ZPxswX2FfMtg/qZdfYLnB1z/2ahlhFqTbP/fVI1URhNg/6wUClb1U2D/KXwgl+tfWPxcojZo5yNg/MWKjyAzo1z8/X6nQp7PbP07ldzJ/5No/9LF2XRu42D9JkzOcrXvcPwcV5Q5O29o/tG3BxCgv2T8Bb5NAmGbWP8BAu8pLiNw/V2EZ0EPS1z9vkJCMb37XPzKsagat6No/LOyT3Q653D/QbgXSMl/bP+whvrlhUto/l0HVNX1c2j+YnixJP7HaPwE29yRJkto/rwZzpb312T//tUSfvrvZP8fgaegWhdk/EaZXd4qi2D/rYzjeoD3ZP6Ty2Bbabtc/q/AxRmB52T9FdTqRqFXYP2rCeNH5L9g/JWkUN4zU2T+7OxQ3u2HbPyb/jJSDxto/FAnydfqW1z/g7jCTUVXYP8zyBjXfzdo/D+h6lahX2D9Vkm8B/k/ZP4Rme1DXTdo/bobG+kpc2D8kDV0jIbzZP/2JPG535Nc/DSPIOYvU2D/mT1jaYgLaP12jqJmbj9c/yD6STO8p2j/sRxRDKpPZP8snB9GjTto/yrwD4fji2T9d+kT+mVjbP7hRUTGiwtk/URvSc+QM2T+FwnqorMHZP2r37r2gS9o/peKeR5yp2T/l8oet5qbZPzPf5ESSs9g/+QvLcGHA2D9L+72K/s3XP9mzEbzm1ts/ljgEygIC2D8g/Md/jLbYP1K9Czsxrtg/x8BBTNm+2j9IJX1t8zjZP1NzlMp1Mts/KdSSHOm82D9f1ZxTBfXZP5MpOpzfh9g/GBpYwIVY3D/iV7+fchTYP8KzsD4f/9k/LZp7qEZp2j/bp42VMSHaP7qAbIviXNk/dy9Ncw9v2T+A9MvF9KfZP5wdpy1uadk/RwgFpHHV2j8ULpi18W3cP0uSrczr4tg/kI2aMr7Y1z9pKwz3q3zZPyjEyk3r5tw/eHKrfUCS2D+s8feutLLbP62hPlhZv9s/Wc+hQDPk2D/EdS9cC1TaP69nzmgxr9s/hXVbD14d2T/OAKgK5FbZP/fjFqv+2tg/0su9ohVN2D+SVH1A2MTYP+aOggyKCNg/Bu6BusSk2T9OeV6QS53YP4G40/s05tk/t5AEncOb1z+BNHvYBQbaP0B8wBO7qto/kwGx7koM1z8rkUW9KxTaP6wFd/DhK9s/zCuGjjkN2D+/uRskq73bP8DiTnbnIto/7vdH65Sy2D8308ZZU4fZP4awv3SNUNo/EcLzZIGy2T9xvnC6I0raP/O2vObpa9g/3S+anRjR2T9fFvOhWsrXP+LGHkgnddk/LHjOVGhx2D9hU1vglKjYP2VkaZykLts/eh39UzP/2j805SPPM2HaP8X4l8OrM9g/G8XhVesx2T+OYFecGyfbPxiZJa0JyNk/Bd82EJmR3D+vuuJ9exraP0UG+diH2Nk/NPYTCE802T8qj7HSW8XZPyWsy4CjZ9k/RlekD6KB2j+Ix2mbF9PaPx/pdQYMmNg/7wWbr2Hl1z9xCniH6T/XPzhZRwAPQNo/EqkngAww2D8mMigMJNjWP6qIl+/tGNo/9eKUjoHK3D8VdMRVnJfZP/sP301brNo/uK6x8Wm02T8Y2XciMnnZPxLnyFLDxto/nq86m3s62T+XGdIXOPHZPxBQYpEDA9o/qKpaHK6+2D99aNYkrN7aP+Qzb+lEYNk/ZxzhGfWh2D+wtcJOZ0naPxfIm6Qwc9g/YKhYm9Wi2T9YYaOknJbZP9iCTx9t/do/+UeHLCY12j/228AvZpHZP2d4J8+Qpdo/vG5jqSRh2z9gPLceYlDaPycUawr+nNk/0pAFzbzr1z/kU9pjlTzYP978+36VNdk/Yd+5AuDR2D+oInFPVVzZP/hIeb0prNk/bGvicitH2j8KgN67f4LZP/sx2VAEOdo/KX0dG7qu2T/spx10RxLXP3Usnm/MZdg/a+7mqGJq2T+ViM4MNQ3YP0fw+UQvXto/ZoxqSTKP2z9LwRLB9ijbP+xU2baoU9k/ohSgxRSC2z+W7e66UMrZP+iTfcckK9k/Zj20p5nQ2D/64YqafDbZPw+B5fdmGtk/5eobJnHR2T9XA5aFN87ZP1B8Yh6Zmtk/W+BYocYo2j8tFKg/wB/bP98zY/m/0No/672JVAez1z+0XNgqu1TWP3zz6QjCy9o/vUMQgbPZ1z8UFlnp8k/ZPyLHs2ctGtg/IuFchiBL1z9C8SXJBkvaP2nLhyZckto/lO8DZ7Dc2D+JPBA4kUjWP6NKZUub5tg/uFiMgPwZ2j+DAH6B/bTXPyHt5PGu1dk/iqwXUZWU2T8XBNQXbFfaP7s5dbrIwNk/HPc1qb5a2D/XU+9KsSHbPwUn8SynZdk/VjAUIaeQ2T/0cMloyWfYP7e9vWNYCNk/Xsz/96p32D/gRz7V7WDZPxj/Tx1+yts/+YnjMA7Y1z/BHm5C3ojXP/Yw0U+se9s/DZ3cQQJV2T9trfG1W77YPyp0tyBb7to/YBbq4ibT2D/h+5tl8u/bP3Yj1SS+d9o/OrLkn7j52D+tgre6gVzcP0vNrIwe09g/odBdEsWM2j+M69IUrPvZP7vZ7ZInRNs/Mx42p0aZ2z9CxndNFqTZPy6ACFS4otg/S6wn3FIw2j93xtq2hrvYPwv1gsNYLdw/FD2CA1fG2T844AU15CHZP9Ay+LQd1tk/L3/QdCzg1z/xVQC1OFvYP6A++53zIts/a9tfUFnC2D/1EnBXjELYPz0CtUtzSdo/FF9vrCEe2z8YH2LGL4XaPxRqxyL839o/dvjSBp2h2D/9tAZ6s8fXPxMVR5iOhds/KVLTQN0v2T/cfJYzaEfZP5EinJnSTNs/st4Yjt1P2j9SbPU8/C7aP1iB7ygHX9w/76fUYLvG2D/+T6F4osnaP6VmIjVIrNk/Fk7ewJzx2T/OPh/Tco7bP+fNbPMBQNo/RBvgJf1J2j8G1koJCvnaP7v+aCUDItk/lbE4l5uG2D/abgGnPEPYP1S8YadDFdc/yQZ8wWA73D8Lv1fGGTDYP9Z/NoAZUdk/b9g/0eQ+2T8vFiu+VP7ZPyyV2ZjmpNo/ldIOWY+z2j8ZmfOEidrYPzrKH2rWYtk/kErUHzL22T9MyFTgG0jZP54KA4VLqNs/EPdmkHs62j9bYeOdZ4raPw1GcxzQcto/1JTNuB8a2z/d9vjJ6tTZPxJbk3On8Nc/IMUB25Ec2j+z0dDSKcTYP7d/zARi7Ng/4kAFYb1Z2j+DMhNQXC/bP4NpO/2RoNk/epELZMz+2T9GmjC9YcfbP2j1wnB96Nk/xb8MmR7u3D+NjsT851LaP7tdRB2iWNc/bw+6eXSQ2j9bnZZ7fxbaP1TzF+c5QNs/ZjcKJTB22j8v+fsRO2zZP19FLAVsCNg/wUpUdRtV2T8bR+zF5oLYP7tQccNf29g/CDGXInda2j9CpsRXy7zbPwSQKgztGto/fsJkqLsR2D9m1zKTSSvaP7c/GlO2Ids/VAJxTcnR2D8qNdxsr23ZPzcULrxpntk/UVKsjmeY2D959OhNB27aP73W4rv3cdk/FCA7iBIj2j/tVDRcyXbYP4c8IvBACtk/hrMiV0+G2j+m358Wbh/ZP/B/6dN9z9s/k8jcr6UW2T/dVliFOePZP9mbN/0wy9o/A5tqddTI1z+RSEpAnljXPzBNuEWspdc/qGrD1FI32z9KJChixOTYPwtn8eje39w/8UGvFLXg2D8jiKjiEtbZPytjqijrkts/nD9W3vQq3D/sqs8NHT3cP1dYz7SOJds/yRjjMCrp2j/Mr/FcwlvaPwfebkKnmNo/gMN4yvzk2D+10V4afo3YP8/LrYd+mNk/Bu2oetVh2T+MQFoTFgXZP/YtKdDKfdo/Xhe+aKLS2j/m3Qa1kbbZPzglfM7Bfds/6h+Qxn0j2D/REGFEJFrZP9bWei+2rtg/eOT2KPk11z+hN2TXgX7ZP8f/w620cdk/vVhb94aJ2z/uH2DrTWjaP3zGBZX+Sdg/L+NHKCb52z86SIxUtSnbP2DtmH5XWNo/JqScvGJP2T9y/eAANl/YP9128qGiH9k/kwwtdFz+2j9uNlooJAPcPwe6BEBKk9s/Diw08Gn52D9CykxIuSrYP4T2tv7Dx9k/jFVmJx5W1z+X8GrffgPaP1r5Mcw7adk/Nb3tn+MA2T/wDP81DI/XPxya53/lQdo/r+0sagvr2D/QoNvcPRrYP9v5Pchq7NU//GWDe5WQ2T/MA9fWVN7bP7lTlR/2uds/2/M7ltED2T+P0xNkRNTYP/dhGDw1M9o/sKaM9XJS2D9Ys0y9ffzZP7gRTxKhlNo/bm30N6Qr2z8+Ti8RyHjZP2+jHjXcVtk/R/sdFed52D9CXjXZpYrYP2okbixjT9k/g4rJc/oR2j+wGg+i9sTaP2hP+BBqktg/UrKIZLmC2z/FC5oawkDZPx+mci6Zktk/SG/58MCk2D/MImyWOX/WPwyB7bNIu9o/AyXNiwyL2j8HRlAiaj3ZPxtPTDCNr9k/ew7vt6lC2j8+s9TAlZnXP89b3XE97Ng/23q3Wd2d2j+rvlYtmP7XP8HQMAbL+dk/PYksRRPd1z++vjOVcDLaP4SDmMbrjdk/Y7T1hVqI1z+wXscEQxfbPzrSm5fjqNg/uvEBsxiQ2D/gPlCQUtvZP8qS2kj8Eds/AAYlSrxM2D/5Ikquz63ZPxEU01UdJto/suktg7d82j/Z2SWXatPZP+amxRJVIdk/R2rwTGqK2D/S9Gbf07XZP24/og9IOtg/D4uq9VTc1T8LDgLkpijaPwo7gCbNwdo/o/budVCT1j9I2NM/u07YP09pHAZ8ZNo/gdJe1Mw73D/rdxLnaaDZP87qekkMq9g/XjtvIqtd2T9upOGXHlzbP546ezLuxdg/fenwg7iT2D+QPGYjavvYP3w9oJk2Ydg/ObKT7svB2T+5BdSB+a3bP8AFKXR3A9o/Wxxop+hG2T8OKFGy+jnZP5agmKZfmdc/2XqoN/q62j+Yp4UkGIDZP0q899h1Xtk/XIZ74Oqv3T/3f3TPfvvZP35pu8tAo9g/xYHwIeQN2T+mQWk17BHbPzjyJGm3vtk/8+0zGU7C1z8vXqgc0MbaP1kznpypvtg/Y8oFNXH/2z+zpjOZgvvaPwlsCYUFB9k/Zxh/Cl092z+OxJe71a/ZPwrk03AJsdo/3n+W72842j9Hj9QORITYPxUrL7q4xtg/VAW9+Rjr2j+m4RWc5ivZP/BB2xxVFdk/s95jHKBg2D96RkSqZyTaPykH+rpyPdw/s1HUcso72D8mwHhGiaHZP/HqGT9baNs/JPe+y4B/2T9c3/NL1S3aPyE+TpJ5Pdg/Qst5mvIl2j8pYRzNRlzZP4YWEqCa3No/VV/lwq4e2z9b7V3xJercP62QNsGIV9o/hHE5YFsE2j+XzrC8S9nZPzcuNt3fJdk/KhrKroMI2j+2DZ+IzTjZP1LGhCHN0Nk/8T85sVlJ2z+/t5mwxk/YP3mrTosdD9s/UeVLODFJ2z86GEpL6XLZP2wInIJP4tY/dHXRy23S2D9IIWdjmULbP6CWs0kbktk/BT+RDSZS2D9sGbUCMvTYPwAENusNrdo/sEOT78BM2j+DK9eFTkvZP2jjViNmIdk/AdpoizoZ2T+qbLg5WGvYP3WXYiLtqts/dMI6Zvsv2T9EEs5BcCjbP7KnDNcajNs/j9jk/Lng2j/PgvapKgzZPwukRe/kHdo/r8Lo7KuR2T+oBeqXeXHYP/i/ItTiA9o/ZKXhkkAX2D9TBEJexx7bP3BfHa1aAdk/QvVQGITb2T/pdlGCbfbZP3Y3oBPDRNk/TO4MC9lZ2j9z8zrL+/3YP8eellAbt9o/QbEPpIzg1z8/uWcWA8PZP1UVzcDyFNw/InD6uc9R2D9ezi47hLvYP28go1D/Qdo/In8AA3LU2T8AmaWVfgzaP+9gOpflOdo/fQNZ5ZVp2j/WdT/4OQXbP1R2Pq7jH9o/fuLqcItK2T/LpNor/XXaP7yxez8nCNw/Ma+VeSNu2T+AISMoRFrYP3Fv8UFeBNs/ZvDJSydy2T9X0F/eqNHWP7Thi3tIr9o/6e9XBC/q2D8WoSoe73vZPybe6SBiBto/mp8S3wXY2T9SDl27EoLaP9SzAFvmCtk/Lni2CLxB2j9M1CM/jUTZP5jk3YOxi9o/87yxXT5j2j9cwG9QE2fYP2IO92F0/do/Z8VBuvnp2D/ZOa9mYqLaP2mymOTzEdo/rbh6n+/z2z/mcivqX1DZP4wNhsgiJ9k/tDK9kDyT2T+B71YJQTbZP69AI5e5n9o/OjuVo/eH1z9dbfJPVEDYPzxad8yzO9g/Nku+Lf/Q2j/Ggw5AIsrbPzIhtsNfd9k/zEGzzkdi2T+sEVkAjrDZP+N9KARTFts/YB2vlLlp2D+vsIZwtefZPxU6PBsn2do/6+X1bMw92j8a6tDbuNfZP78vv26f4do/VvAjCs4j1j8Ooj3nuXfaPxxhQsFFw9g/cS98+7xB1z81PNVhHEHaP0qaPG64W9s/4lQ5lY9s2T856K+C1tHaP9vxjE3pqds/+tR5nnhI2z8thbPL+7LbP5g6HsTHjNo/uxapblGy2T99Mr73qozXP+NO52H4SNk/x2/a8kiF2D/uOHd5+WDcP/jyiTr3X9k/H+OtxPjB2T/kH9+LT07aP6yHAT3jp9k/s7yEyQjF2z8OVF/8kc3YP7cG4XRe2dk/E1Eup06m2D+niAUWDunXP0jHaLAi0dg/aIxwunaN2T91Koce7QzZP8kpSB61ttg/g1cHJMfM1z+qaAQzXn7ZP3mbnoGIrNc/O8kit6eS2j/2T9AlnbTZPzzhfFH9u9c/73I7Q0c02T+zVcglIqPYP/kTBeYvAto/Xua8u9ZQ2z+bS3G7JDPXP0oz169Kv9k/8QcAsCBl2T+62PavlXXaP5JZVNR139k/C7Yp1zGj2D+U1YBqDTHZP2TFfRzClNg/TeXzni/62j+bNcVZlaDZP9KfsQiVCNw/AeeKALmF2T+iiB4ieLHYPzONigiWqdc/mNzmR79K1z+vCUOlgpLXP976v84h8dk/eIJJjExA2j+ZhdS1eZPXPzPCCdvivto/f1eBgE/72D8zX4d9s8nZP8Yi9nfkqds/3M1B73m/2j+lqtidm0HZPyx2SfqSddg/nqe4PX7Y1j/YS6ql8q3YP+bEPrZqVNk/BpOt+h5W2D+D4LH3lm7ZP4w/OGfIstk/M7woUOtP2T/aGXdum8TYP34hGmHd0Nk/z4rIRGoq2j/fxrZLTDTYP5BIvS/vZ9s/TpCUbUt52T+hoivSxp/ZPw0/iO6qgdo/UiXiLwXm2T+f1fR+7NHaP9rfGFKv99k/VxcgoerQ2D+k1LC2DhDaP9rkUeTSItg/aqWHPyy92T/hE/wQto7ZPzAQH35nVdk/AnCPF3tv2T+DYRGp9DDXP+IK7zfL5dg/mIWr/gW42T8xenpT+83ZP1V6/vXZSNg/CJeD3lg42z/wGl+WxX3YP86mdJBF19o/YpeqQpUl2j87zcof2MXYP8hOuHhV39s/a0/SFXgS2D99nS3XycbaP8/GQ4d44do/4hdF5tm92D+fEzkO4mHbP/8qli2qR9k/6uMU+jL42T+gEIX65u7ZP03OVveWbdk/48euIEmj2j+dYR9CnJ3aP1QIZRukXNc/GbQHlwFF2z+i8wGq1XjXP1MMf9rx69o/NK8seswK2z/z7OezHDTYP35J8f/7Etk/5CqmXkgv2D/XUZ63LFPZP0eFt8+pNNk/QVcKRBSZ2j+IRqlD9kbbP1DpNgvwYts/lqonh2Ph2D+ZwxLMClXZP3+nspH4cdc/dKeyUYiR2D9U8qU4w9XaP3k7joL+qts/ecdq6SAF2D/RsoMYdtfYP2WESYbykNk/f3esOGf12T+NY2UiT5DYP7v5L2uOJNo/tZYaE1X+2D80NXJB3ZTZP81afz6bTNo/60TLVrSb2T96cfXlmgrZPzDt+9yuddk/kguALaV82T/QyzzTWiDZP3H5eDbMRNk/dQEt2HGl2z+s4gXvZlHaPwdXFurGONk/2It4X/p92j9sxqTKNizZP5biXG0CGts/xso7b7kS2j+igppxcHbZP6wAX9FOLNo/k58a9dqW1z8KRXR+YSjYP2J1UelfEtg/a50viXDI2T8+wOl9T9HbP+CVYJ/AdNw/1FirNK1q2j/cJM16tz3aP6idRO1tS9c/hB190sbn2D+UnliCdZfYP5rU8iApztg/oTyXrIBi2T+C5KPF5/7YP/PI0+UFEdc/UkUN3KyO2j9zhIlkFTrYP4WY4hT/59k/2WTHQzxB3D/7XNUHVmzYPys4zbJ5XNY/Vtmz0+E72T+UkLvJXALbPyxO87BPG9w/JnYdgbMG2D8b4TLAWzzaP6fSoIcgAdk/KBgEAuJ22T+Y/JIyGfzcP3A8kE8brdc/YM5W077s2T+kifyL2jLaP9xaYfVj/do/42/MBbW52T+2PJDAiv7ZP40wqLxDGdk/fNWnmsnx2T8MPZqLJCnZP8GTS9RGZdo/GngScHwg2T+WReUSKxXaP0QPdtsCkNk/OYWaipUK2z/jMsphwojZP8KCRducVNc/Vnp6Pxs32z+hWItEz3DYP6qiYbRfw9g/8oO26GnW2D8TGefS+VvbP5iLFEPI3dY/0JRGBsed3T+WZuMQpvPaP7DixjzA4tk/e7zpwZyH2T+Q7KmcK/fZP/oWiYRcRNo/03jNNAZt2j/jrKcUqU/aP9XrFCz0ttk//uxKp+9Y2T8F25HgB2jZP62cJoK7Wdk/FEAJIR0N2z/lDye/GFzaP5Khe+wX1tU/ruiFi4Zw2j+Mz4lBadnZPyFVHTJ4k9k/VbdZT0oa2T+uYHIC8wnbP2k/b+EV0No/a38Q7Fec2D+Xoeee+x7aP8xsHCA1W9g/yjcb14bV1z93x66/PMzYP2WdZmgwtNo/fC0YhuvR2j/qyxhTukHaP8cNDgszh9o/vwrpxb1C2j8g1zw5ZsfYPyBwnuMhJ9o/2XNF0eCf2j8755AkxJDaP6pf1+EuH9s/ipTlbrKB2j9yXWM2wwzaP7MU+k5E+No/Rfbk1eiQ2T9nRauXoHjYP9zJNo0qZNk/mPgbo4Kl2D8=\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[1000]}},\"selected\":{\"id\":\"1060\"},\"selection_policy\":{\"id\":\"1059\"}},\"id\":\"1046\",\"type\":\"ColumnDataSource\"},{\"attributes\":{\"label\":{\"value\":\"dist_b\"},\"renderers\":[{\"id\":\"1049\"}]},\"id\":\"1061\",\"type\":\"LegendItem\"},{\"attributes\":{\"data_source\":{\"id\":\"1031\"},\"glyph\":{\"id\":\"1032\"},\"hover_glyph\":null,\"muted_glyph\":null,\"nonselection_glyph\":{\"id\":\"1033\"},\"selection_glyph\":null,\"view\":{\"id\":\"1035\"}},\"id\":\"1034\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"source\":{\"id\":\"1031\"}},\"id\":\"1035\",\"type\":\"CDSView\"},{\"attributes\":{\"source\":{\"id\":\"1046\"}},\"id\":\"1050\",\"type\":\"CDSView\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.1},\"fill_color\":{\"value\":\"#fbb4ae\"},\"line_alpha\":{\"value\":0.1},\"line_color\":{\"value\":\"#fbb4ae\"},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"1033\",\"type\":\"Circle\"},{\"attributes\":{\"bottom_units\":\"screen\",\"fill_alpha\":0.5,\"fill_color\":\"lightgrey\",\"left_units\":\"screen\",\"level\":\"overlay\",\"line_alpha\":1.0,\"line_color\":\"black\",\"line_dash\":[4,4],\"line_width\":2,\"right_units\":\"screen\",\"top_units\":\"screen\"},\"id\":\"1025\",\"type\":\"BoxAnnotation\"},{\"attributes\":{},\"id\":\"1059\",\"type\":\"UnionRenderers\"},{\"attributes\":{},\"id\":\"1060\",\"type\":\"Selection\"}],\"root_ids\":[\"1187\"]},\"title\":\"Bokeh Application\",\"version\":\"2.2.1\"}};\n",
       "  var render_items = [{\"docid\":\"533397ec-c880-462e-82c8-bb08e6e28c2f\",\"root_ids\":[\"1187\"],\"roots\":{\"1187\":\"7b14f533-52e8-4e31-9616-d4e7a6a69779\"}}];\n",
       "  root.Bokeh.embed.embed_items_notebook(docs_json, render_items);\n",
       "\n",
       "  }\n",
       "  if (root.Bokeh !== undefined) {\n",
       "    embed_document(root);\n",
       "  } else {\n",
       "    var attempts = 0;\n",
       "    var timer = setInterval(function(root) {\n",
       "      if (root.Bokeh !== undefined) {\n",
       "        clearInterval(timer);\n",
       "        embed_document(root);\n",
       "      } else {\n",
       "        attempts++;\n",
       "        if (attempts > 100) {\n",
       "          clearInterval(timer);\n",
       "          console.log(\"Bokeh: ERROR: Unable to run BokehJS code because BokehJS library is missing\");\n",
       "        }\n",
       "      }\n",
       "    }, 10, root)\n",
       "  }\n",
       "})(window);"
      ],
      "application/vnd.bokehjs_exec.v0+json": ""
     },
     "metadata": {
      "application/vnd.bokehjs_exec.v0+json": {
       "id": "1187"
      }
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import numpy as np\n",
    "from utils import plot_data\n",
    "\n",
    "# Generate data for 3 distributions\n",
    "random_state = np.random.RandomState(seed=42)\n",
    "dist_a = random_state.normal(0.8, 0.05, 1000)\n",
    "dist_b = random_state.normal(0.4, 0.02, 1000)\n",
    "dist_c = random_state.normal(0.6, 0.1, 1000)\n",
    "\n",
    "# Concatenate data to simulate a data stream with 2 drifts\n",
    "stream = np.concatenate((dist_a, dist_b, dist_c))\n",
    "\n",
    "\n",
    "plot_data(dist_a, dist_b, dist_c)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "As observed above, the synthetic data stream has 3 **abrupt drifts**. An analogy could be trending topics on Twitter which can change quickly.\n",
    "\n",
    "The goal is to detect that drift has occurred, after samples **1000** and **2000** in the synthetic data stream.\n",
    "\n",
    "In this example, we will use the ADaptive WINdowing (`ADWIN`) drift detection method.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Change detected at index 1055\n",
      "Change detected at index 2079\n"
     ]
    }
   ],
   "source": [
    "from river import drift\n",
    "\n",
    "drift_detector = drift.ADWIN()\n",
    "drifts = []\n",
    "\n",
    "for i, val in enumerate(stream):\n",
    "    drift_detector.update(val)           # Data is processed one sample at a time\n",
    "    if drift_detector.change_detected:\n",
    "        print(f'Change detected at index {i}')\n",
    "        drifts.append(i)\n",
    "        drift_detector.reset()           # As a best practice, we reset the detector"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "\n",
       "\n",
       "\n",
       "\n",
       "\n",
       "\n",
       "  <div class=\"bk-root\" id=\"2e5ae3f6-1df1-478c-9e44-72a2ae453797\" data-root-id=\"1519\"></div>\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/javascript": [
       "(function(root) {\n",
       "  function embed_document(root) {\n",
       "    \n",
       "  var docs_json = {\"be42b67d-9497-4a6b-a75d-6e97633e57dd\":{\"roots\":{\"references\":[{\"attributes\":{\"children\":[{\"id\":\"1518\"},{\"id\":\"1516\"}]},\"id\":\"1519\",\"type\":\"Column\"},{\"attributes\":{\"data\":{\"x\":[1000,1001,1002,1003,1004,1005,1006,1007,1008,1009,1010,1011,1012,1013,1014,1015,1016,1017,1018,1019,1020,1021,1022,1023,1024,1025,1026,1027,1028,1029,1030,1031,1032,1033,1034,1035,1036,1037,1038,1039,1040,1041,1042,1043,1044,1045,1046,1047,1048,1049,1050,1051,1052,1053,1054,1055,1056,1057,1058,1059,1060,1061,1062,1063,1064,1065,1066,1067,1068,1069,1070,1071,1072,1073,1074,1075,1076,1077,1078,1079,1080,1081,1082,1083,1084,1085,1086,1087,1088,1089,1090,1091,1092,1093,1094,1095,1096,1097,1098,1099,1100,1101,1102,1103,1104,1105,1106,1107,1108,1109,1110,1111,1112,1113,1114,1115,1116,1117,1118,1119,1120,1121,1122,1123,1124,1125,1126,1127,1128,1129,1130,1131,1132,1133,1134,1135,1136,1137,1138,1139,1140,1141,1142,1143,1144,1145,1146,1147,1148,1149,1150,1151,1152,1153,1154,1155,1156,1157,1158,1159,1160,1161,1162,1163,1164,1165,1166,1167,1168,1169,1170,1171,1172,1173,1174,1175,1176,1177,1178,1179,1180,1181,1182,1183,1184,1185,1186,1187,1188,1189,1190,1191,1192,1193,1194,1195,1196,1197,1198,1199,1200,1201,1202,1203,1204,1205,1206,1207,1208,1209,1210,1211,1212,1213,1214,1215,1216,1217,1218,1219,1220,1221,1222,1223,1224,1225,1226,1227,1228,1229,1230,1231,1232,1233,1234,1235,1236,1237,1238,1239,1240,1241,1242,1243,1244,1245,1246,1247,1248,1249,1250,1251,1252,1253,1254,1255,1256,1257,1258,1259,1260,1261,1262,1263,1264,1265,1266,1267,1268,1269,1270,1271,1272,1273,1274,1275,1276,1277,1278,1279,1280,1281,1282,1283,1284,1285,1286,1287,1288,1289,1290,1291,1292,1293,1294,1295,1296,1297,1298,1299,1300,1301,1302,1303,1304,1305,1306,1307,1308,1309,1310,1311,1312,1313,1314,1315,1316,1317,1318,1319,1320,1321,1322,1323,1324,1325,1326,1327,1328,1329,1330,1331,1332,1333,1334,1335,1336,1337,1338,1339,1340,1341,1342,1343,1344,1345,1346,1347,1348,1349,1350,1351,1352,1353,1354,1355,1356,1357,1358,1359,1360,1361,1362,1363,1364,1365,1366,1367,1368,1369,1370,1371,1372,1373,1374,1375,1376,1377,1378,1379,1380,1381,1382,1383,1384,1385,1386,1387,1388,1389,1390,1391,1392,1393,1394,1395,1396,1397,1398,1399,1400,1401,1402,1403,1404,1405,1406,1407,1408,1409,1410,1411,1412,1413,1414,1415,1416,1417,1418,1419,1420,1421,1422,1423,1424,1425,1426,1427,1428,1429,1430,1431,1432,1433,1434,1435,1436,1437,1438,1439,1440,1441,1442,1443,1444,1445,1446,1447,1448,1449,1450,1451,1452,1453,1454,1455,1456,1457,1458,1459,1460,1461,1462,1463,1464,1465,1466,1467,1468,1469,1470,1471,1472,1473,1474,1475,1476,1477,1478,1479,1480,1481,1482,1483,1484,1485,1486,1487,1488,1489,1490,1491,1492,1493,1494,1495,1496,1497,1498,1499,1500,1501,1502,1503,1504,1505,1506,1507,1508,1509,1510,1511,1512,1513,1514,1515,1516,1517,1518,1519,1520,1521,1522,1523,1524,1525,1526,1527,1528,1529,1530,1531,1532,1533,1534,1535,1536,1537,1538,1539,1540,1541,1542,1543,1544,1545,1546,1547,1548,1549,1550,1551,1552,1553,1554,1555,1556,1557,1558,1559,1560,1561,1562,1563,1564,1565,1566,1567,1568,1569,1570,1571,1572,1573,1574,1575,1576,1577,1578,1579,1580,1581,1582,1583,1584,1585,1586,1587,1588,1589,1590,1591,1592,1593,1594,1595,1596,1597,1598,1599,1600,1601,1602,1603,1604,1605,1606,1607,1608,1609,1610,1611,1612,1613,1614,1615,1616,1617,1618,1619,1620,1621,1622,1623,1624,1625,1626,1627,1628,1629,1630,1631,1632,1633,1634,1635,1636,1637,1638,1639,1640,1641,1642,1643,1644,1645,1646,1647,1648,1649,1650,1651,1652,1653,1654,1655,1656,1657,1658,1659,1660,1661,1662,1663,1664,1665,1666,1667,1668,1669,1670,1671,1672,1673,1674,1675,1676,1677,1678,1679,1680,1681,1682,1683,1684,1685,1686,1687,1688,1689,1690,1691,1692,1693,1694,1695,1696,1697,1698,1699,1700,1701,1702,1703,1704,1705,1706,1707,1708,1709,1710,1711,1712,1713,1714,1715,1716,1717,1718,1719,1720,1721,1722,1723,1724,1725,1726,1727,1728,1729,1730,1731,1732,1733,1734,1735,1736,1737,1738,1739,1740,1741,1742,1743,1744,1745,1746,1747,1748,1749,1750,1751,1752,1753,1754,1755,1756,1757,1758,1759,1760,1761,1762,1763,1764,1765,1766,1767,1768,1769,1770,1771,1772,1773,1774,1775,1776,1777,1778,1779,1780,1781,1782,1783,1784,1785,1786,1787,1788,1789,1790,1791,1792,1793,1794,1795,1796,1797,1798,1799,1800,1801,1802,1803,1804,1805,1806,1807,1808,1809,1810,1811,1812,1813,1814,1815,1816,1817,1818,1819,1820,1821,1822,1823,1824,1825,1826,1827,1828,1829,1830,1831,1832,1833,1834,1835,1836,1837,1838,1839,1840,1841,1842,1843,1844,1845,1846,1847,1848,1849,1850,1851,1852,1853,1854,1855,1856,1857,1858,1859,1860,1861,1862,1863,1864,1865,1866,1867,1868,1869,1870,1871,1872,1873,1874,1875,1876,1877,1878,1879,1880,1881,1882,1883,1884,1885,1886,1887,1888,1889,1890,1891,1892,1893,1894,1895,1896,1897,1898,1899,1900,1901,1902,1903,1904,1905,1906,1907,1908,1909,1910,1911,1912,1913,1914,1915,1916,1917,1918,1919,1920,1921,1922,1923,1924,1925,1926,1927,1928,1929,1930,1931,1932,1933,1934,1935,1936,1937,1938,1939,1940,1941,1942,1943,1944,1945,1946,1947,1948,1949,1950,1951,1952,1953,1954,1955,1956,1957,1958,1959,1960,1961,1962,1963,1964,1965,1966,1967,1968,1969,1970,1971,1972,1973,1974,1975,1976,1977,1978,1979,1980,1981,1982,1983,1984,1985,1986,1987,1988,1989,1990,1991,1992,1993,1994,1995,1996,1997,1998,1999],\"y\":{\"__ndarray__\":\"ejDHCiRk2z9mlL5+lcjaPz8UC8Ijrdk/4/sVnJzF2D+3HBbRZH7aP3TB6IuJGto/0Tue2e++2j8Xhi2su2naPy7jiXaE8do/7xH00Dbq2D9+Pb2fSEnbP76dzHRZ2tk/QzN+GZ9B3D91C+dVxLfYPwF+aMxw0ts/Sf9Pj3Pa2T81BH6yJMTYP1JOcFAK+9g/AyuK66Aw2T9EevA3lyTaPxYlNjjsRNo/B6zTJ5zd2D9QsraWnpHZP55EP0OUV9w/W6ewbK7P2z+iAA8VkyjaPzFPFJANptk/dri2ju7A2T/r7r03o2LaP1ZogopzStg/E4iLSkNF2T9G2XWf1nbXP/yB4NxqHNo/51yTVKxt2j+AMFX7RPvYP1Gd9i1dnds/E5ps7PAH2D/1h4DuwLnXP8SCs/sl49k/OFVtkrbw2j9DD2uxY8HbP/91F5k7A9k/IK8c6A/72j87roNG+4zZP0bRj5IIYdk/m+5CXii72j/U/75oWm/aP5fnzD0Mldc/zp4qyG592z/C/cEN1F3bP1qJ4JidzNg/DT/CAUwb2j8UDRHbezvaP2t/GIAE79k/ojVoqEbl2D8g7pi/hb3YP7SjGvg5kdk/TSGGPeEZ2z98yQcWukvaP77aazcoINk/iBY5YXiW2j/Lm52GMPTVP8tYKw0HEts/vcGt1Idf1z9tU6bZ1SLZPwWKs+G0Ktg/Tn6d2Fvx1z/5YQrP+hXbP43Dx/pXANk/IcPMcyQL2j9CTdedOYrZP+fgW6/qNdo/3wom48Wy2T9zZGFpMPXXPwRZo4QO4No/STe3S8732D8xytsLipvXP88NhrJQDdk/AhuzdF6F2z8Mz3teM7DaP71yBopaJ9k/XqiCvScn2T/iB3jlNDDZPw0H9dgbQtw/2or7qsAW2j8yLKcbhCbaP4mqTQo069o/VEi+r9jn2T/Mh+CRt0TZP+2xspRCWdk/dydDQCOC2T/4UygsZ43ZP59PEZcHiNo/ueYAI5+q2T9Y93Tnq4naP3euPZomf9k/5h90/12z2T8kPKVg1ArXP6iKk7zcxdo/GE32KCML2j9TvoPBoODaP7Lv8haO5NU//uXkKutF3D94Oyn522vZP08fYba6BNs/8ULc99dE2D+GqwLNZGLaP92morJqQNg/MfIqETTN2D8yH/EvygzcP6BXOQAeW9k/kjX/MNng2T8n09crtLbaP4QeF2kGPNo/QdD9puPK2T/+FHO/MBHaP3VWSbYmrdw/+wNbL7iG2T/Blr4Cf9vZP46F8+Dg8do/ehM81tsD2z/bB0/rkB7bP+NDrCzmato/ttYAZg8j2D8dTqjE17DbPxY+vC33Idg/e6O6gMT82T+OXQ9bcKLYP/1FGEmVhNk/KB+aMFQF2j/w4kD/5gLaPzMCzuDaI9o/op06g2Gq2z9CJpDQNi7aP6oYyEGYSdk/pDbkF4PV2j/78didXR/bPxUkgmRWB9g/RMVpJltd2j8j2pU7XH/aP6DeBSoYONk/f9V1QGRc2z8u02cGbmjZPyZvZ7e/wtk/vWV5SONg2T9xbRl3tJ7ZPxswX2FfMtg/qZdfYLnB1z/2ahlhFqTbP/fVI1URhNg/6wUClb1U2D/KXwgl+tfWPxcojZo5yNg/MWKjyAzo1z8/X6nQp7PbP07ldzJ/5No/9LF2XRu42D9JkzOcrXvcPwcV5Q5O29o/tG3BxCgv2T8Bb5NAmGbWP8BAu8pLiNw/V2EZ0EPS1z9vkJCMb37XPzKsagat6No/LOyT3Q653D/QbgXSMl/bP+whvrlhUto/l0HVNX1c2j+YnixJP7HaPwE29yRJkto/rwZzpb312T//tUSfvrvZP8fgaegWhdk/EaZXd4qi2D/rYzjeoD3ZP6Ty2Bbabtc/q/AxRmB52T9FdTqRqFXYP2rCeNH5L9g/JWkUN4zU2T+7OxQ3u2HbPyb/jJSDxto/FAnydfqW1z/g7jCTUVXYP8zyBjXfzdo/D+h6lahX2D9Vkm8B/k/ZP4Rme1DXTdo/bobG+kpc2D8kDV0jIbzZP/2JPG535Nc/DSPIOYvU2D/mT1jaYgLaP12jqJmbj9c/yD6STO8p2j/sRxRDKpPZP8snB9GjTto/yrwD4fji2T9d+kT+mVjbP7hRUTGiwtk/URvSc+QM2T+FwnqorMHZP2r37r2gS9o/peKeR5yp2T/l8oet5qbZPzPf5ESSs9g/+QvLcGHA2D9L+72K/s3XP9mzEbzm1ts/ljgEygIC2D8g/Md/jLbYP1K9Czsxrtg/x8BBTNm+2j9IJX1t8zjZP1NzlMp1Mts/KdSSHOm82D9f1ZxTBfXZP5MpOpzfh9g/GBpYwIVY3D/iV7+fchTYP8KzsD4f/9k/LZp7qEZp2j/bp42VMSHaP7qAbIviXNk/dy9Ncw9v2T+A9MvF9KfZP5wdpy1uadk/RwgFpHHV2j8ULpi18W3cP0uSrczr4tg/kI2aMr7Y1z9pKwz3q3zZPyjEyk3r5tw/eHKrfUCS2D+s8feutLLbP62hPlhZv9s/Wc+hQDPk2D/EdS9cC1TaP69nzmgxr9s/hXVbD14d2T/OAKgK5FbZP/fjFqv+2tg/0su9ohVN2D+SVH1A2MTYP+aOggyKCNg/Bu6BusSk2T9OeV6QS53YP4G40/s05tk/t5AEncOb1z+BNHvYBQbaP0B8wBO7qto/kwGx7koM1z8rkUW9KxTaP6wFd/DhK9s/zCuGjjkN2D+/uRskq73bP8DiTnbnIto/7vdH65Sy2D8308ZZU4fZP4awv3SNUNo/EcLzZIGy2T9xvnC6I0raP/O2vObpa9g/3S+anRjR2T9fFvOhWsrXP+LGHkgnddk/LHjOVGhx2D9hU1vglKjYP2VkaZykLts/eh39UzP/2j805SPPM2HaP8X4l8OrM9g/G8XhVesx2T+OYFecGyfbPxiZJa0JyNk/Bd82EJmR3D+vuuJ9exraP0UG+diH2Nk/NPYTCE802T8qj7HSW8XZPyWsy4CjZ9k/RlekD6KB2j+Ix2mbF9PaPx/pdQYMmNg/7wWbr2Hl1z9xCniH6T/XPzhZRwAPQNo/EqkngAww2D8mMigMJNjWP6qIl+/tGNo/9eKUjoHK3D8VdMRVnJfZP/sP301brNo/uK6x8Wm02T8Y2XciMnnZPxLnyFLDxto/nq86m3s62T+XGdIXOPHZPxBQYpEDA9o/qKpaHK6+2D99aNYkrN7aP+Qzb+lEYNk/ZxzhGfWh2D+wtcJOZ0naPxfIm6Qwc9g/YKhYm9Wi2T9YYaOknJbZP9iCTx9t/do/+UeHLCY12j/228AvZpHZP2d4J8+Qpdo/vG5jqSRh2z9gPLceYlDaPycUawr+nNk/0pAFzbzr1z/kU9pjlTzYP978+36VNdk/Yd+5AuDR2D+oInFPVVzZP/hIeb0prNk/bGvicitH2j8KgN67f4LZP/sx2VAEOdo/KX0dG7qu2T/spx10RxLXP3Usnm/MZdg/a+7mqGJq2T+ViM4MNQ3YP0fw+UQvXto/ZoxqSTKP2z9LwRLB9ijbP+xU2baoU9k/ohSgxRSC2z+W7e66UMrZP+iTfcckK9k/Zj20p5nQ2D/64YqafDbZPw+B5fdmGtk/5eobJnHR2T9XA5aFN87ZP1B8Yh6Zmtk/W+BYocYo2j8tFKg/wB/bP98zY/m/0No/672JVAez1z+0XNgqu1TWP3zz6QjCy9o/vUMQgbPZ1z8UFlnp8k/ZPyLHs2ctGtg/IuFchiBL1z9C8SXJBkvaP2nLhyZckto/lO8DZ7Dc2D+JPBA4kUjWP6NKZUub5tg/uFiMgPwZ2j+DAH6B/bTXPyHt5PGu1dk/iqwXUZWU2T8XBNQXbFfaP7s5dbrIwNk/HPc1qb5a2D/XU+9KsSHbPwUn8SynZdk/VjAUIaeQ2T/0cMloyWfYP7e9vWNYCNk/Xsz/96p32D/gRz7V7WDZPxj/Tx1+yts/+YnjMA7Y1z/BHm5C3ojXP/Yw0U+se9s/DZ3cQQJV2T9trfG1W77YPyp0tyBb7to/YBbq4ibT2D/h+5tl8u/bP3Yj1SS+d9o/OrLkn7j52D+tgre6gVzcP0vNrIwe09g/odBdEsWM2j+M69IUrPvZP7vZ7ZInRNs/Mx42p0aZ2z9CxndNFqTZPy6ACFS4otg/S6wn3FIw2j93xtq2hrvYPwv1gsNYLdw/FD2CA1fG2T844AU15CHZP9Ay+LQd1tk/L3/QdCzg1z/xVQC1OFvYP6A++53zIts/a9tfUFnC2D/1EnBXjELYPz0CtUtzSdo/FF9vrCEe2z8YH2LGL4XaPxRqxyL839o/dvjSBp2h2D/9tAZ6s8fXPxMVR5iOhds/KVLTQN0v2T/cfJYzaEfZP5EinJnSTNs/st4Yjt1P2j9SbPU8/C7aP1iB7ygHX9w/76fUYLvG2D/+T6F4osnaP6VmIjVIrNk/Fk7ewJzx2T/OPh/Tco7bP+fNbPMBQNo/RBvgJf1J2j8G1koJCvnaP7v+aCUDItk/lbE4l5uG2D/abgGnPEPYP1S8YadDFdc/yQZ8wWA73D8Lv1fGGTDYP9Z/NoAZUdk/b9g/0eQ+2T8vFiu+VP7ZPyyV2ZjmpNo/ldIOWY+z2j8ZmfOEidrYPzrKH2rWYtk/kErUHzL22T9MyFTgG0jZP54KA4VLqNs/EPdmkHs62j9bYeOdZ4raPw1GcxzQcto/1JTNuB8a2z/d9vjJ6tTZPxJbk3On8Nc/IMUB25Ec2j+z0dDSKcTYP7d/zARi7Ng/4kAFYb1Z2j+DMhNQXC/bP4NpO/2RoNk/epELZMz+2T9GmjC9YcfbP2j1wnB96Nk/xb8MmR7u3D+NjsT851LaP7tdRB2iWNc/bw+6eXSQ2j9bnZZ7fxbaP1TzF+c5QNs/ZjcKJTB22j8v+fsRO2zZP19FLAVsCNg/wUpUdRtV2T8bR+zF5oLYP7tQccNf29g/CDGXInda2j9CpsRXy7zbPwSQKgztGto/fsJkqLsR2D9m1zKTSSvaP7c/GlO2Ids/VAJxTcnR2D8qNdxsr23ZPzcULrxpntk/UVKsjmeY2D959OhNB27aP73W4rv3cdk/FCA7iBIj2j/tVDRcyXbYP4c8IvBACtk/hrMiV0+G2j+m358Wbh/ZP/B/6dN9z9s/k8jcr6UW2T/dVliFOePZP9mbN/0wy9o/A5tqddTI1z+RSEpAnljXPzBNuEWspdc/qGrD1FI32z9KJChixOTYPwtn8eje39w/8UGvFLXg2D8jiKjiEtbZPytjqijrkts/nD9W3vQq3D/sqs8NHT3cP1dYz7SOJds/yRjjMCrp2j/Mr/FcwlvaPwfebkKnmNo/gMN4yvzk2D+10V4afo3YP8/LrYd+mNk/Bu2oetVh2T+MQFoTFgXZP/YtKdDKfdo/Xhe+aKLS2j/m3Qa1kbbZPzglfM7Bfds/6h+Qxn0j2D/REGFEJFrZP9bWei+2rtg/eOT2KPk11z+hN2TXgX7ZP8f/w620cdk/vVhb94aJ2z/uH2DrTWjaP3zGBZX+Sdg/L+NHKCb52z86SIxUtSnbP2DtmH5XWNo/JqScvGJP2T9y/eAANl/YP9128qGiH9k/kwwtdFz+2j9uNlooJAPcPwe6BEBKk9s/Diw08Gn52D9CykxIuSrYP4T2tv7Dx9k/jFVmJx5W1z+X8GrffgPaP1r5Mcw7adk/Nb3tn+MA2T/wDP81DI/XPxya53/lQdo/r+0sagvr2D/QoNvcPRrYP9v5Pchq7NU//GWDe5WQ2T/MA9fWVN7bP7lTlR/2uds/2/M7ltED2T+P0xNkRNTYP/dhGDw1M9o/sKaM9XJS2D9Ys0y9ffzZP7gRTxKhlNo/bm30N6Qr2z8+Ti8RyHjZP2+jHjXcVtk/R/sdFed52D9CXjXZpYrYP2okbixjT9k/g4rJc/oR2j+wGg+i9sTaP2hP+BBqktg/UrKIZLmC2z/FC5oawkDZPx+mci6Zktk/SG/58MCk2D/MImyWOX/WPwyB7bNIu9o/AyXNiwyL2j8HRlAiaj3ZPxtPTDCNr9k/ew7vt6lC2j8+s9TAlZnXP89b3XE97Ng/23q3Wd2d2j+rvlYtmP7XP8HQMAbL+dk/PYksRRPd1z++vjOVcDLaP4SDmMbrjdk/Y7T1hVqI1z+wXscEQxfbPzrSm5fjqNg/uvEBsxiQ2D/gPlCQUtvZP8qS2kj8Eds/AAYlSrxM2D/5Ikquz63ZPxEU01UdJto/suktg7d82j/Z2SWXatPZP+amxRJVIdk/R2rwTGqK2D/S9Gbf07XZP24/og9IOtg/D4uq9VTc1T8LDgLkpijaPwo7gCbNwdo/o/budVCT1j9I2NM/u07YP09pHAZ8ZNo/gdJe1Mw73D/rdxLnaaDZP87qekkMq9g/XjtvIqtd2T9upOGXHlzbP546ezLuxdg/fenwg7iT2D+QPGYjavvYP3w9oJk2Ydg/ObKT7svB2T+5BdSB+a3bP8AFKXR3A9o/Wxxop+hG2T8OKFGy+jnZP5agmKZfmdc/2XqoN/q62j+Yp4UkGIDZP0q899h1Xtk/XIZ74Oqv3T/3f3TPfvvZP35pu8tAo9g/xYHwIeQN2T+mQWk17BHbPzjyJGm3vtk/8+0zGU7C1z8vXqgc0MbaP1kznpypvtg/Y8oFNXH/2z+zpjOZgvvaPwlsCYUFB9k/Zxh/Cl092z+OxJe71a/ZPwrk03AJsdo/3n+W72842j9Hj9QORITYPxUrL7q4xtg/VAW9+Rjr2j+m4RWc5ivZP/BB2xxVFdk/s95jHKBg2D96RkSqZyTaPykH+rpyPdw/s1HUcso72D8mwHhGiaHZP/HqGT9baNs/JPe+y4B/2T9c3/NL1S3aPyE+TpJ5Pdg/Qst5mvIl2j8pYRzNRlzZP4YWEqCa3No/VV/lwq4e2z9b7V3xJercP62QNsGIV9o/hHE5YFsE2j+XzrC8S9nZPzcuNt3fJdk/KhrKroMI2j+2DZ+IzTjZP1LGhCHN0Nk/8T85sVlJ2z+/t5mwxk/YP3mrTosdD9s/UeVLODFJ2z86GEpL6XLZP2wInIJP4tY/dHXRy23S2D9IIWdjmULbP6CWs0kbktk/BT+RDSZS2D9sGbUCMvTYPwAENusNrdo/sEOT78BM2j+DK9eFTkvZP2jjViNmIdk/AdpoizoZ2T+qbLg5WGvYP3WXYiLtqts/dMI6Zvsv2T9EEs5BcCjbP7KnDNcajNs/j9jk/Lng2j/PgvapKgzZPwukRe/kHdo/r8Lo7KuR2T+oBeqXeXHYP/i/ItTiA9o/ZKXhkkAX2D9TBEJexx7bP3BfHa1aAdk/QvVQGITb2T/pdlGCbfbZP3Y3oBPDRNk/TO4MC9lZ2j9z8zrL+/3YP8eellAbt9o/QbEPpIzg1z8/uWcWA8PZP1UVzcDyFNw/InD6uc9R2D9ezi47hLvYP28go1D/Qdo/In8AA3LU2T8AmaWVfgzaP+9gOpflOdo/fQNZ5ZVp2j/WdT/4OQXbP1R2Pq7jH9o/fuLqcItK2T/LpNor/XXaP7yxez8nCNw/Ma+VeSNu2T+AISMoRFrYP3Fv8UFeBNs/ZvDJSydy2T9X0F/eqNHWP7Thi3tIr9o/6e9XBC/q2D8WoSoe73vZPybe6SBiBto/mp8S3wXY2T9SDl27EoLaP9SzAFvmCtk/Lni2CLxB2j9M1CM/jUTZP5jk3YOxi9o/87yxXT5j2j9cwG9QE2fYP2IO92F0/do/Z8VBuvnp2D/ZOa9mYqLaP2mymOTzEdo/rbh6n+/z2z/mcivqX1DZP4wNhsgiJ9k/tDK9kDyT2T+B71YJQTbZP69AI5e5n9o/OjuVo/eH1z9dbfJPVEDYPzxad8yzO9g/Nku+Lf/Q2j/Ggw5AIsrbPzIhtsNfd9k/zEGzzkdi2T+sEVkAjrDZP+N9KARTFts/YB2vlLlp2D+vsIZwtefZPxU6PBsn2do/6+X1bMw92j8a6tDbuNfZP78vv26f4do/VvAjCs4j1j8Ooj3nuXfaPxxhQsFFw9g/cS98+7xB1z81PNVhHEHaP0qaPG64W9s/4lQ5lY9s2T856K+C1tHaP9vxjE3pqds/+tR5nnhI2z8thbPL+7LbP5g6HsTHjNo/uxapblGy2T99Mr73qozXP+NO52H4SNk/x2/a8kiF2D/uOHd5+WDcP/jyiTr3X9k/H+OtxPjB2T/kH9+LT07aP6yHAT3jp9k/s7yEyQjF2z8OVF/8kc3YP7cG4XRe2dk/E1Eup06m2D+niAUWDunXP0jHaLAi0dg/aIxwunaN2T91Koce7QzZP8kpSB61ttg/g1cHJMfM1z+qaAQzXn7ZP3mbnoGIrNc/O8kit6eS2j/2T9AlnbTZPzzhfFH9u9c/73I7Q0c02T+zVcglIqPYP/kTBeYvAto/Xua8u9ZQ2z+bS3G7JDPXP0oz169Kv9k/8QcAsCBl2T+62PavlXXaP5JZVNR139k/C7Yp1zGj2D+U1YBqDTHZP2TFfRzClNg/TeXzni/62j+bNcVZlaDZP9KfsQiVCNw/AeeKALmF2T+iiB4ieLHYPzONigiWqdc/mNzmR79K1z+vCUOlgpLXP976v84h8dk/eIJJjExA2j+ZhdS1eZPXPzPCCdvivto/f1eBgE/72D8zX4d9s8nZP8Yi9nfkqds/3M1B73m/2j+lqtidm0HZPyx2SfqSddg/nqe4PX7Y1j/YS6ql8q3YP+bEPrZqVNk/BpOt+h5W2D+D4LH3lm7ZP4w/OGfIstk/M7woUOtP2T/aGXdum8TYP34hGmHd0Nk/z4rIRGoq2j/fxrZLTDTYP5BIvS/vZ9s/TpCUbUt52T+hoivSxp/ZPw0/iO6qgdo/UiXiLwXm2T+f1fR+7NHaP9rfGFKv99k/VxcgoerQ2D+k1LC2DhDaP9rkUeTSItg/aqWHPyy92T/hE/wQto7ZPzAQH35nVdk/AnCPF3tv2T+DYRGp9DDXP+IK7zfL5dg/mIWr/gW42T8xenpT+83ZP1V6/vXZSNg/CJeD3lg42z/wGl+WxX3YP86mdJBF19o/YpeqQpUl2j87zcof2MXYP8hOuHhV39s/a0/SFXgS2D99nS3XycbaP8/GQ4d44do/4hdF5tm92D+fEzkO4mHbP/8qli2qR9k/6uMU+jL42T+gEIX65u7ZP03OVveWbdk/48euIEmj2j+dYR9CnJ3aP1QIZRukXNc/GbQHlwFF2z+i8wGq1XjXP1MMf9rx69o/NK8seswK2z/z7OezHDTYP35J8f/7Etk/5CqmXkgv2D/XUZ63LFPZP0eFt8+pNNk/QVcKRBSZ2j+IRqlD9kbbP1DpNgvwYts/lqonh2Ph2D+ZwxLMClXZP3+nspH4cdc/dKeyUYiR2D9U8qU4w9XaP3k7joL+qts/ecdq6SAF2D/RsoMYdtfYP2WESYbykNk/f3esOGf12T+NY2UiT5DYP7v5L2uOJNo/tZYaE1X+2D80NXJB3ZTZP81afz6bTNo/60TLVrSb2T96cfXlmgrZPzDt+9yuddk/kguALaV82T/QyzzTWiDZP3H5eDbMRNk/dQEt2HGl2z+s4gXvZlHaPwdXFurGONk/2It4X/p92j9sxqTKNizZP5biXG0CGts/xso7b7kS2j+igppxcHbZP6wAX9FOLNo/k58a9dqW1z8KRXR+YSjYP2J1UelfEtg/a50viXDI2T8+wOl9T9HbP+CVYJ/AdNw/1FirNK1q2j/cJM16tz3aP6idRO1tS9c/hB190sbn2D+UnliCdZfYP5rU8iApztg/oTyXrIBi2T+C5KPF5/7YP/PI0+UFEdc/UkUN3KyO2j9zhIlkFTrYP4WY4hT/59k/2WTHQzxB3D/7XNUHVmzYPys4zbJ5XNY/Vtmz0+E72T+UkLvJXALbPyxO87BPG9w/JnYdgbMG2D8b4TLAWzzaP6fSoIcgAdk/KBgEAuJ22T+Y/JIyGfzcP3A8kE8brdc/YM5W077s2T+kifyL2jLaP9xaYfVj/do/42/MBbW52T+2PJDAiv7ZP40wqLxDGdk/fNWnmsnx2T8MPZqLJCnZP8GTS9RGZdo/GngScHwg2T+WReUSKxXaP0QPdtsCkNk/OYWaipUK2z/jMsphwojZP8KCRducVNc/Vnp6Pxs32z+hWItEz3DYP6qiYbRfw9g/8oO26GnW2D8TGefS+VvbP5iLFEPI3dY/0JRGBsed3T+WZuMQpvPaP7DixjzA4tk/e7zpwZyH2T+Q7KmcK/fZP/oWiYRcRNo/03jNNAZt2j/jrKcUqU/aP9XrFCz0ttk//uxKp+9Y2T8F25HgB2jZP62cJoK7Wdk/FEAJIR0N2z/lDye/GFzaP5Khe+wX1tU/ruiFi4Zw2j+Mz4lBadnZPyFVHTJ4k9k/VbdZT0oa2T+uYHIC8wnbP2k/b+EV0No/a38Q7Fec2D+Xoeee+x7aP8xsHCA1W9g/yjcb14bV1z93x66/PMzYP2WdZmgwtNo/fC0YhuvR2j/qyxhTukHaP8cNDgszh9o/vwrpxb1C2j8g1zw5ZsfYPyBwnuMhJ9o/2XNF0eCf2j8755AkxJDaP6pf1+EuH9s/ipTlbrKB2j9yXWM2wwzaP7MU+k5E+No/Rfbk1eiQ2T9nRauXoHjYP9zJNo0qZNk/mPgbo4Kl2D8=\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[1000]}},\"selected\":{\"id\":\"1390\"},\"selection_policy\":{\"id\":\"1389\"}},\"id\":\"1376\",\"type\":\"ColumnDataSource\"},{\"attributes\":{},\"id\":\"1467\",\"type\":\"UnionRenderers\"},{\"attributes\":{\"axis_label\":\"samples\",\"formatter\":{\"id\":\"1369\"},\"ticker\":{\"id\":\"1344\"}},\"id\":\"1343\",\"type\":\"LinearAxis\"},{\"attributes\":{\"fill_color\":{\"value\":\"#b3cde3\"},\"line_color\":{\"value\":\"#b3cde3\"},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"1377\",\"type\":\"Circle\"},{\"attributes\":{},\"id\":\"1468\",\"type\":\"Selection\"},{\"attributes\":{},\"id\":\"1351\",\"type\":\"PanTool\"},{\"attributes\":{},\"id\":\"1348\",\"type\":\"BasicTicker\"},{\"attributes\":{\"data\":{\"x\":[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85,86,87,88,89,90,91,92,93,94,95,96,97,98,99,100,101,102,103,104,105,106,107,108,109,110,111,112,113,114,115,116,117,118,119,120,121,122,123,124,125,126,127,128,129,130,131,132,133,134,135,136,137,138,139,140,141,142,143,144,145,146,147,148,149,150,151,152,153,154,155,156,157,158,159,160,161,162,163,164,165,166,167,168,169,170,171,172,173,174,175,176,177,178,179,180,181,182,183,184,185,186,187,188,189,190,191,192,193,194,195,196,197,198,199,200,201,202,203,204,205,206,207,208,209,210,211,212,213,214,215,216,217,218,219,220,221,222,223,224,225,226,227,228,229,230,231,232,233,234,235,236,237,238,239,240,241,242,243,244,245,246,247,248,249,250,251,252,253,254,255,256,257,258,259,260,261,262,263,264,265,266,267,268,269,270,271,272,273,274,275,276,277,278,279,280,281,282,283,284,285,286,287,288,289,290,291,292,293,294,295,296,297,298,299,300,301,302,303,304,305,306,307,308,309,310,311,312,313,314,315,316,317,318,319,320,321,322,323,324,325,326,327,328,329,330,331,332,333,334,335,336,337,338,339,340,341,342,343,344,345,346,347,348,349,350,351,352,353,354,355,356,357,358,359,360,361,362,363,364,365,366,367,368,369,370,371,372,373,374,375,376,377,378,379,380,381,382,383,384,385,386,387,388,389,390,391,392,393,394,395,396,397,398,399,400,401,402,403,404,405,406,407,408,409,410,411,412,413,414,415,416,417,418,419,420,421,422,423,424,425,426,427,428,429,430,431,432,433,434,435,436,437,438,439,440,441,442,443,444,445,446,447,448,449,450,451,452,453,454,455,456,457,458,459,460,461,462,463,464,465,466,467,468,469,470,471,472,473,474,475,476,477,478,479,480,481,482,483,484,485,486,487,488,489,490,491,492,493,494,495,496,497,498,499,500,501,502,503,504,505,506,507,508,509,510,511,512,513,514,515,516,517,518,519,520,521,522,523,524,525,526,527,528,529,530,531,532,533,534,535,536,537,538,539,540,541,542,543,544,545,546,547,548,549,550,551,552,553,554,555,556,557,558,559,560,561,562,563,564,565,566,567,568,569,570,571,572,573,574,575,576,577,578,579,580,581,582,583,584,585,586,587,588,589,590,591,592,593,594,595,596,597,598,599,600,601,602,603,604,605,606,607,608,609,610,611,612,613,614,615,616,617,618,619,620,621,622,623,624,625,626,627,628,629,630,631,632,633,634,635,636,637,638,639,640,641,642,643,644,645,646,647,648,649,650,651,652,653,654,655,656,657,658,659,660,661,662,663,664,665,666,667,668,669,670,671,672,673,674,675,676,677,678,679,680,681,682,683,684,685,686,687,688,689,690,691,692,693,694,695,696,697,698,699,700,701,702,703,704,705,706,707,708,709,710,711,712,713,714,715,716,717,718,719,720,721,722,723,724,725,726,727,728,729,730,731,732,733,734,735,736,737,738,739,740,741,742,743,744,745,746,747,748,749,750,751,752,753,754,755,756,757,758,759,760,761,762,763,764,765,766,767,768,769,770,771,772,773,774,775,776,777,778,779,780,781,782,783,784,785,786,787,788,789,790,791,792,793,794,795,796,797,798,799,800,801,802,803,804,805,806,807,808,809,810,811,812,813,814,815,816,817,818,819,820,821,822,823,824,825,826,827,828,829,830,831,832,833,834,835,836,837,838,839,840,841,842,843,844,845,846,847,848,849,850,851,852,853,854,855,856,857,858,859,860,861,862,863,864,865,866,867,868,869,870,871,872,873,874,875,876,877,878,879,880,881,882,883,884,885,886,887,888,889,890,891,892,893,894,895,896,897,898,899,900,901,902,903,904,905,906,907,908,909,910,911,912,913,914,915,916,917,918,919,920,921,922,923,924,925,926,927,928,929,930,931,932,933,934,935,936,937,938,939,940,941,942,943,944,945,946,947,948,949,950,951,952,953,954,955,956,957,958,959,960,961,962,963,964,965,966,967,968,969,970,971,972,973,974,975,976,977,978,979,980,981,982,983,984,985,986,987,988,989,990,991,992,993,994,995,996,997,998,999],\"y\":{\"__ndarray__\":\"atSd2g1l6j8j04aJ92DpP+w3aKrkouo/nioA224J7D+G587WsDnpP0P9hI+yOek/zj7TEHIg7D8APr/28NPqPwpcg6RN2eg/rejeJNV36j+EkqG7yNvoP84GuUvW2ug/hcneLrX86T8HYRug64nmP5hPG9sS1+Y/YiOrekmz6D/bdzyPvvrnP1vDM9JQGuo/zj/sX6wl6D/TysrSHlfnP1SBKQPu8es/sbcuPR896T8jtQpyQrXpP3fisewFUuc/S0Ac75666D8K6DGtCMfpP37nBC0nwuc/Jr98ZHwz6j8DvYkRlKPoP/BNIEsfIuk/HoC3FiSj6D8Jf4ILS5DsP0x2O1ASlOk/LQUMklzo5z9QwnSvg+rqPzFaad2Kpec/hmQJiSbv6T/tx0RL63bmP7TD1DKTeec/7nMp/zvq6T/kB+puE8jqPy3a3t3K3+k/fnyo/zpq6T/OLtGWRB7pPzOHIVX/O+c/GF7EdsBy6D8IIr8f7NzoP6TGSOaYSus/1zTTllgm6j/ffctwdcfmPzUiaURYHuo/bIzYy9776D+JYwQvVYToP0Cy84EklOo/4rJLvOU/6z9oofFlDRfrP+8+v0PbQeg/jrgWVfIa6T/qYaAWSSHqP5prtOsuKes/C5cYilTV6D8H1WTYjU3pP8jBRvhx1Oc/mxqKPqKv5z+wipAbaebqP83Hpqwdxes/YCGeyhp86T9xprSmpTTrP7WduuK5Leo/eL8M5FuR6D9xHwStoC3qPwRwTmyUD+w/7Huv9+yK6T8fL+xgehrsPyAgcGqNaOU/pHf0UkDq6j9KrIslQb3pP0ItJ2ggH+k/3nAwai+/6T/qyc7kfWvmPwsFQ1WfP+k/DJ0ykd8r6j9TMMgF8vbrP5jWXAdRxeg/4xO05nBO6D8SOoWPFMzoP1TAFniMEOs/vdEAp0Eg6j/bwWk3nMDoP5qEWJfVa+o/RgA561zB6T841pJjWybrP8diXP8Jeug/iY/Tu2MT6T9G4oIU/vjoPzKBWu8kQuc/Aa7CD+QS6j8H8LM6hwTqPwYU6Mixm+k/bCUwW4M56T8zfIs43VXnP3Fpsr1N7eg/gAvZYDkN6T9UyiO7/FDoPyiIPpGJV+k/ThozZxk/6j+dZtmGLp7sPxlaFGkb4ek/S/ZBtxcD6j8R8UNhG3vpP4J0/Nqrh+Y/fuMsa72O6T9Hr/wxRbLpP78BkkGLiu0/US5FF89K6T/QOm8hHRXqPwyHoM5hi+k/Bctn0+i65z+9JO9Bs23rP15GQH6Xzeo/ZNO9T5vd6j/nOP1pHSXoPzktpz4v2Os/Memr3GZb5z+RE8IG+onqP0mCE4XPGu0/e4CWVuAD6D9eF6z6pLHoP8gVkM1qwuk/+5TrWWDL6D/R8hPBch7nPzrQ8PKutek/PGdr+3rm5z+7iZtdlVvqPwYbPvsAIeg/lGR6AHQU7D+PfSGKx1joP+iNmgCvFek/4+8eENHm6j+ywzEfcKHnP0EKuoDE9uk/SfS6cwGx6z+IgRfELAfnP3g1AN055ek/svE8SQwE6j81eoKr1dnqP56fyerxnuc/zTlLsL185z85MtEjY2/qP2/rN7M+E+o/rMnZrTMA6j8q4OhTgSfqP4Ao+tYPg+g/ln1PKrv46T9bruZ5pBHqP2BcvGwAdeg/ZOzEPNKV7D+GIjiVrlvqP1xy2V+ksec/gQtVPIam6j98Yb3RXgroP942DWf92+o/N2fbJil06z8CPV3ScknoPybDp+gyJOs/8Co80axC6j8mov3aUOrqP7MMhsKGouw/erSEyhY16T90b4Ci3mToPx9NckBBLeg/mbsDsXFL6D8P9ljmBHrpPwlJMfpVJeo/ylywu+4K6j85dN0MauzqPyc+evLsnuk/m57hsffs6z/LINRRMi3pP3a+qQLI8+0/b4WUk9+Z6j9g4KUdgjroP47f/mH24uc/SRvhfzhf6j8G2CHUET7pP+jsa/oNvuo/ItlaKXBb6j9hslzvxHvpP2Oz0NfAPug/gQ8MWx4t5z/zgx8dteLoP8/sr4Vh+Oo/BILI9Erx6T8dUeNrWJvnP2wdru+I4Ok/Q9NADm036j+EGjVuki/oP7vGfdiQ2Ok/guj8OXGx6T/QbUp6cMXnP1ow7FImLOo/kveaHkx/6j8nhwjBN1XrPyOECcE8Ses/MHoZf05l5z+7T2CEdxnoP6DqR/aObOo/FLMz9gts6j/NTadDkGzqP2jPDcetw+8/MHAOz2+D6j/y26RJumrrP/DYcO9bIOs/QZ1b7Gik6j8OVkE5dxjpP+azekp50Oo/iXW2AA1d6D+k8odemTjpP90+EovL0ug/qkjLuSK76T/EB2Ikr03tP/hfQafEnOY/FFRQMrGy6j+GZpIVCAXnP/5jCvVL2Og/YRibWKJX6z8zSDnZ7bPpP5Ysgt4n4Oc/YdXckZx06D8dYMOW9q/qP3jlnRtxbug/Z7We7ULy6T8Gz1wnRKzpP9bA9Fm0jug/UpGGbsIH7T/gRMjTQJ3qP46COgIaXOY/ZDaAwPjl6T9rnXRCiIroP1CNsbbB9uo/ANkEx/tU6D+clbOcmmrpP12WplpxaOo/oRWinDb86j+sgENm9a3nP38lbJqWEOk/8Xp7+Q/X6D9guI0Q/43oP2vyAOW6bOw/tRagAns/6j9/7axVJJXnP1tZoWaOEes/Qq0+ztX+7D/T6uZtf0DrP4C5ohxEK+c/wxYv+kHT6D8y6c3chqDrP8ypBxS9d+g/FFmocGNP6j+A9CLe49bqP53ZN+XtHeg/rH396TeB6T8mf/QV+mnkP0XePMUC9uc/VTXp6CUy6T+zI9sMgprnP+gC4FQ8Nuw/g2dOaNBP5z9Q9lqXW+XoP7qGP74mz+k/c2vbDvLn6z8yMXiKeE3nP7Q80igIdus/pXc9ncqd6T+raV71kgfoP6aGSKngVuo/7oCdhSLr6T8D9HxMwKPoP9soDeEwtuk/FQp+isb76D9rNqLBGMjpP37+jQjPqOo/vZlThDsj7D9AF9A8l57nP8JHQlxKA+0/JwK9WwZ65j8R4nrHbVvpP2EKXCGTiuo/3WGdu7EM6j9dBKXSiproP9Vue2ZaROk/r5aHtKrP6D8lN985MqjoP5a1O9aY9eo/EtETY9Ur6j/2bkjDyH3oPx/B4HsTCus/uS9NSngX6j/rHPRejObqP6QWIfh+m+o/Ot44LAtG6D+ln01cJrToP67SkAOxy+o/HC6nj5uT6j9jlbXoCZHpP91gkESoyek/idp9ee6k6z9BUgzYSqfoPzZdGeuweeo/gBbqKchG6T8Lf0sScEDpPz5rIbSoW+s/m8cAx7Dr6j9wKpxE0ObqP7pbfflSsOs/sj4uAzSi6T/uMUiN7bDqP6RuxcWDGuk/PnHd52Ae6j8e55UcS2TpP4+UNV1Uwek/WPpZVmCN6j9c9mDxdEroP1cE2U6k8uw/tDMlCIn95z9JpDSyRKjnPwiFpFP2c+s/jqc5c93d6j9xcF5OPZnqP6g07mb4muo/8GzLbpWU6T8s1/aoFSroP3vttkimuOk/+4VRDDyE6D8+JKFQAinrP5PmEoRdXek/1In08XlH6D+JuRra9RXpP2gq7Jm8Quo/VGjcy7Ky6D/OGeuK0UjoP6uyXw5q/ek/VQrsNPD96T8C9lrB9MnoPzMhXaep2Og/qsiszKX46T/y3ovzdkjnP8WYl1MaWec/wsBiQ1Nz6D8N9T8LLELpP0K3A5/yGOo/1WV86ef16z/4AM665fjqP+ayYNQWWOk/Q/Enm8+R6T/Aj+LG9v7nPzvJY1sEkuk/gWhDjF0j6T/6iv0XyR3qP5OGCybERug/ke5vB1Nu6j9VC4HsaA3sP3humEUNbek/VbyHICQ+6j9h409xSLTqPzM5kZVC9eg/+qVaZmP16T+F20gCwp7pP/rJZ66bwek/r8M3pvlc6D8KlJSto6PpP7mUcIGUZeo/JT81Cf3r6z/DRMNvhCLrPzhVFSWLC+0/BvhNYEtf6D/2vmMM5/7qPyZoF2ey5Ok/6yp+FIsa7T+T+rdhhU7oPzXBCWKmQeg/AyvKuRak6D/S9d/9pjPmPzZ6szBAwug/ERVixahi6D/3shCIM9fpP+BHuE+VJeo/AG0TXxSa7D9LWz3D5B7rP+4tA95Mreg/SeZm/psp6D9Y1U8QF2PqP0oBTh3VfOc/POh++MOH7D9WV0PcsnzrP/yblvds2eg/odm/bObb5j/qmGloJcTrP1Z7Cjqvauk/mgI3DJyU6z9KBgm+hQznP7f/1pIYpOg/760Xcb+b6T9AoE/f16zpP4nml9BA4eg/gzIpJriY6j/Qnet7TeTnP1S9ZgdIX+k/Oxbggt/K6T//hOZrUGzqPwSVB9QTveo/TvZSVPLM5z//+eYROiXnP0Z1obnvpOs/uiDvP7ch6j9B/R8ZBWfoPyoxY6zzFOw/lJ/r9vrI6T9GHFbfo3zrP+nZCG1Btek/at59rq7l7D/hM4xtlmjsP0GmINGfM+k/hb+5Mo4n6z/18lQs8qHqP2VmOQUxyus/6XavCl4O6D8uG0BPm7LqP/Ovk3MhS+s//mf/ZTjJ5j8TOMLz77TnPxK2pZtUVuY/BGP0PkAr6T/aBt1bgb/qP/mqoif3AOw/Fog3APO36T9tLUNRrjTsP/x8LHlPZOc/5jhqAeXf5j/AZGkA2YLpP9ny/cfpNuo/8X6jTjWM6T+k7U+VxkrmPyJmwa8Ydek/01IUD0qD5z87mzPb5avqP9YCVjbCL+o/7HuyD6AY6D/Wd5K/HsfoP3MCIAO/5+c/COlkOO1/6T/n7AiI0yDrP3OthrvYBeg/jkw5tQ5o6j/pnQoPaMDoP2J5ktvWVOg/doFAp8Jt6T8ccByTkPHnP6y6OUPTtug/AV7E//Ku5z8cAkz2Wb7sP/aIXUALqOk/ssoEEP566D//XRcFP/HpP9rYeSaXa+k/vyYUQhc/6T+AKmulKZXqP8ZkWwrgz+o/i2/ahU7A6D+mYmauvq3oPzE901bwKOk/4sjYq7vq5T9HqjpN+iznP37eU8F4yes/0opt92A77D+R7VBHmDPpP6eRuvrBheo/6gVRixYZ6j9AjvqmtYbuP16nyYktZOs/R2nmdzRl6T8CyTXsNRLoP7tXjH6ZB+c/jcL8TvDs6T8vHuF5zGPoP0upSn0LU+c/r+rlhMOQ6D99RnISmd7nP0fFAzmnTOw/clKeOrgC6z/epqObVZbpPz3nk/3I9+s/dFlLQUq56T9M+d1n0TjoP1T/OLx4Cew/F5TuaVZ26j//bCh1vvDnPxmMnSSjS+k/ekoWX/Iy6D9NWPeJNGPnP1yz1Fr2FOs/CANCcrKn7D84Uigpv1znP/H7+TMxgOo/zmAgyBiP6D/+4iXNEtLoP2oSSJj0pug/DeP5mbU36D/v3yF2ea3pP2RAQio+Reg/t1O8DWEI6j+1Pp7ABYXpPzvTzhS6N+k/4JYNp9wl6D9BTRa+Wq3oPziSahwCz+o/+sL3ksZm6j+HSs+AMQnoP8XO3lhJwuk/HAccQF7N6j/Ts8/Ez+3mP1qNswspeOo/C9eCdjCK6D9LzPo0UYPqP5pxoBP4YOg/8dCg/lG25j9O5/Nn9f7mP+PxDqxLrek/JhZrevsD6j8WhAkhMSfoP90SY98qn+o/FQ0LmArx5j99A4OhiH7pP1PzBFmRqec/PgVRoZuO6D9Ar/61A63pP96zMbgsOeg/VA9QBxb86D9y14IMxzXrP2LpZBpOreg/svPqRObv6j+44iFA38rnPw59NZibcuo/MqqaBhHo6z/q+8DjNqXlP7KqeBMxU+g/THqP//eF6j++WIW/bkbpPyKuxhCfMeo//9uyKTWi6D+SLoIyEb3pP9IDoqjVWek/tH6fbOx36z/R1huPzwHqP0/ZDc7hI+o//3F6K+Xw6D/Ac7Bh4NHoP8gdKZZs6Og/WcOr5yo76j9oAX4tKu3oPyxRL7JKEOo/naOTJa/r7D9Ow0ylaf7qPywuDI4PFOk/rYJLAp6F6z+kT7fLc/LoP9WTs8DIVuY/zoC+FrD85z/gHVzZUpvmPwy0XL2eCek/hZ8z6CSh6T9aoAnLREjsP8Asd2uCH+o/Dl6QPts/6T9LryMUU+3qPzTZGkLrD+Y/eg+8kxv66T8JZJWsWNXqPz6ySZj4O+c/iPxp5xRu6z9smlmFPyTqP5LLi4F/7+g/DUl6lsmc6j/CP98ArTvtP6H0tagX5Ok/jio6akX/6T9gqVgect3oPzPvg/WAPeg/cmj0nrTt6j8uvWG08jroP9b3Jt3ptuk/VoYllvPV6D/a1mRGyl3qP47LiZtEIuo/Zs9njJNC6z8G/iOBssjoPwjZdykPK+k/+uwpybII6D+ynzYTnuPoPxgHl2wkNOo/nbIMnKnP6j9ncauO4R/oPwk4xWPK/eo/pahHiN7E6z9A/kZk8ULqP9eHk+dVmuw/ofby66dc6D9rvlAYypvnPzQofUMJweY/ZopLN2H+6z9Md+/PoKXqPyqm8x/Vguk/Y+svcEYM6j8MWwmFmcznP2UhZEhhg+0/W3RKbIfO6T+Z48F5aMbpP/NGEL/fwuo//ZzYEp9e6j+0Zp2KTfXpP152gljSVeg/w5jxo7Za6j9Rz/0repzsPx3K9R2vwOs/LyV2Uysm7D+qPUfANMjoP589ewNCBOg/5Kas4hNm6T/A9tLHbLDpP+ZtheXHWes/7sci0l3k5j9OElmTGgzsP7mMXkXhWOk/KhIi4L/q6D8sl6bDCvvnPwg7EU3F8+Y/dQi1SsXq6j9Ip8uLobfpP+qWXWU7iec/6MmGvyKH5z+Ue7YFEBDpPzOABTE7Rew/nYJSeUUv6T/x1DGj6THnPyANdJLxNOk/xu8EduQp6T+9rAWK9EjlP8vx315cg+k/hPoBXAI76T8GoOYgxLbqPyTZ2LLujuw/UY9XgQpn6z+aFcaTdivpP5s+6fJd1Oc/7uDS7qW37T/JdVUa27HpP47BLjFOn+k/6Vu55reP6T/HiOhKvOrpPya4TFB4Xuk/lvxbx6Cu6D9FjqxIm7noP1CJsisvjOk/qtT5YQO76D8zslNNnnXoPyQwRZ4xxek/FYYJTSkx6T/+mfixogHsP6IHqkTDW+U/FGONY65Y6z+gSkka/5frP7GJWeBWSOY/H7CKMzwN6T/oo8YzdQHpP+KEPU0VWec/mNLgmwFb6D+o/j9ItdLnP5LNJnlUZ+w/o6A2l9oY6z/Vf8bQbaLrPzs0j2Yyweo/gwLl8CPL5z+Q++mpwcLoPzMm8z0MYuo/cXEINgSl5z+GGnnnpL3qP/rR0KcpN+k/hiQpyhIA6T9kHu0nz7zqP61uD/yRT+o/uibejb8F6T+cwwokdnTrP1F83uTL3uc/+9MmIeOV6j9NgUPGiIzqP8cVpE3PGuk/tBRBIC8f6j8c80LVJJnnPxmVBtsUFOs/GTqrNN1N6T+ZzlwefsPoP72sljBGR+s/ufWfzxl56D/lspC6sVjnP8zxZTMBHOc/mu1cWdKR6j8HgePZIo3nP2AiPhtdaOw/eX6aetdE5j/eCHvxd1DsP5hummII8Ok/xHnFfvxx6T/4sz6xZrroP0sSFg4WPeo/L9O/UC+K6T9/APsvg13rP3sVsjxjyOk/rp3J4SnX6T/SyJkYqgTpP/glJ2tGguk/jERr9KwX6j8UOF5yHd3mP0iUvhxicec/y2MzfQrK6j9YpMcjlt/pP7tehYw9Tuk/Rgu5iSah6T9kJfUu+CfqP+wxArKDvOg/C4U2b85a6D8OwaJ20enpPwiOVMfbCOg/DXkQAdJA6j8oWfvEOODmP9NK4WMkP+s/5GqrCS1b6j/QsHJDeALqP5KX1jccLOs/NKrLQMdD7D8iErICFjnrP9c1OvKTp+Y/NNnxOnyN5z+gLp6frJnoP+xl5XFJpOk/5gcfFaJt6j+jK3XYVXDoPwpBwoMZ5uk//O9o9TFk6D/q9pFPH5/oP/uRTX5uWec/Sszbk3Ef6D8KmEAy82/nP+8Sb9/hCeg/Y5s28ytJ6z9ELDzpuRToP+8VmN3Sz+0/V+cqu6lj6j9+0oMST+XpP573YkMEOug/JzTIaXK46j/Kv1yZ0a3oPzGCn0GTy+k/WZOk6zWy7T80KU39QHLpP9h/iqRXcOs/aNMqNZR56D8FunXKRIvpP5oQNoHrbuw/WWnLVsuY6D8mPx2b+n/sP23w88R+u+o/H4QNrzaz6D/sxJRbopzqP2gpUVP1J+s/OeGsGUuY6j82uLqabxbnP1/Nqb3Db+g/ZdnRYzc06T9qqHOwHHvpP4xuWsnTl+o/R4+F5mLi6T+nTmqYpHbnPwJU8ztUNeo/vm/lJ7KT6j+V9gPi437qPy8/SqxJVOs/D4T7rCzv6j880f4erlXqP6v5HzHcfOk/lZUKOUXx5j+3GttVkknqP2U8gDur7uk/UhCAtNYI6j8yOPzOpI7nPx22D5vM3uc/mgE+rvhI6z9tEwbxZYnpP3J3pSC+sOo/4pE4/zKl6T/gLuPByaXpP9lAu8nrGes/lw1oYzrG6D+QFQ+Y+MDpPxTSzoVA3Og//E5OXqHn6D/b/JqN9hrpP2nOoAOW9Ok/cLgkK4HV6D8LWGss9ZvrP75TsDkrK+g/WT4ZsA5N6T/hruJ0fOXoPzy8azpI6us/oMi03Bvq6T9MHoxXPkDrP8Id/E0dOec/4d+W2fsG6j+J8m0m/gXrP5pGnrNNu+k/+0uiUAVO6z9EWoD5t8XoPzh64GPe2us/eQw2ijpH7T9YSyg4+wTpP7kTWkYf4+g/sPkVAujs6z/PvIW+lyDsP7tinsBvw+g/eGGd0X3t6D/mSJpXLibpPxYHAL/pcuc/HBwv9lEh6D8cjSPPTf7nP3VJpjAcX+g/0Gk+oGSL6T8q+nPLiPnpP/8bSFyvFOw/eephl6wA6D8w4slIxyzrPyFZTT7zQek/WCBB9FaF6T/s5GiNAa7qP+xs0am7zec/YEAPKzw26j9mmAphx93pP4Wpj9tOY+o/YmFHIQsQ6j/pDN56SoftP2V2QLdhlOg/kfyfiBrA6D+gtXaUXJroP0LZOJoTtug/ZOwut4aU6D+fahIFn4DrP67pJ0Rw3+s/bw1wg9Kv6D/+ZdHKqkToP8CblRqxWuo/JCUh0Wi36D8jLLZP2ZzqP7Oo9p637Ok/msVUT8As5z+XwwlIdRPsP4hMzwUxeew/EJ5xDJqe6D8wvxglzProP9JaQMKwDuo/mHix75ci6j8bKND4VqfqP088zdL60Ow/tkWhVh9R6T8XJx0RnlLoP2x+AX+hZOc/iQ6+BzZu6D8vbTH8B4zpPxDxcKGmeOw//FsXH5bF6D+hPq93Q/XpP5sj14jfkuk/umSgql2A6z857WaroaTtPxeLfPknwOg/BTibJyDR6D+So7fNSUXrP5ti5RrnsOo/B4Po5QKO7D/cTHjoxojqP0ks+hdvBuk/0UZ7P4iL6j8S5riYuV/rP9TocWSr6eo/posvI2Fp6j/PNUWMgk7rPygdGiGLeOs/DZuAebvP6z+FdgzDT6PqP6BZyv8lVek/UMUhpbHV6T+2LA88yYfrP+Dgqa/7Sug/29NgzJsw6j9YSkEJffjoP+mxkLZfpek/Yad4/kCl6z+rgSXK3+fpP/KrMtOerOk/kilFWZps5z+Ukzr1Q8vqP7yJpoX9oeo/vNJoTKsP7T8W7oO2iBvpP3r1aS1d8+k/vrXiX7//6T/trqWQuR/sP0+CoSORcuk/d4CmIOML6j8NVkcrmJLqP81YG/wI5uk/NKyCpL3i6D8+xCxpGenpPxpabwxcUes/uuVZqyP15z9Tgxx7ENDpP8sjlJzUeug/ZxVHUheD6z8r0HbgsynnP4+EgWWqtOg/aOfEIRs06j8HQ0Sx1hrsPxz7dy+rfuk/wsHJtTC26D9Uniw3H5zsP0xLb1Z+SOc/i06jFfgU5j+Q/o111E3qP/6JImb1y+g/8rbnk0335z/IsjEovrvqPwlsLfV1/ek/m2Enq42y6D9Zpxf0L43nP3L9rGH1/uo/UtiRIuyj6j9YOcdB+nDpP86SAob7jew/GBdhFEvj5z/ZPhyxvijnPz1vxscxfug/oY5Kj+2G6T+oRDSXRf3pP5kGeyrKNuk/nqewSM0p6j+lHf8t+JjnP4rJlUr36Os/mFyybPN36T98NDSPPmPrP/NSEPX6Jeo/iF93pa9U6j8vfJEH+oLqP9IPvz77UOo/owpQ99ug6j8z5yZYBbrrPw6LlFYY6uk/USJMCAK86j8HAWYh2HTpP/IEvtV45+s/2gX/uYyE6D9r1TvkQ3vsP9MX0rsmiek/tCfn9I1P5z/Fn01SEs7pPwFqYCikgug/YWYKd+3x6j8gTA0ESY7oP4m4Q+DX4ug/C2jT5KST5j8gSEvYVeDoP5DiXW7HuOU/8InuWdUQ5z+23BjbENHqP07B/bd22+o/E4YPD95H6j+CZWjNhg3oP37BorMOhuk/P3qt2B+Y6T/ehT5CIr/nP0jsd1ZkAew/0jlBtPcA6z9kTbnTFz/pP81U2MicpOk/uOQBH/Tu6T94bI4uTlXmPzDSQixbNOk/wyNXXkKC6D99xTAhVv/nP6baERl2Juk/SaXbse557D8dOivYFqDqP4yNVySlr+g/RuKZQSGE6j8=\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[1000]}},\"selected\":{\"id\":\"1373\"},\"selection_policy\":{\"id\":\"1372\"}},\"id\":\"1361\",\"type\":\"ColumnDataSource\"},{\"attributes\":{\"label\":{\"value\":\"dist_c\"},\"renderers\":[{\"id\":\"1473\"}]},\"id\":\"1487\",\"type\":\"LegendItem\"},{\"attributes\":{\"bottom\":{\"value\":0},\"fill_color\":{\"value\":\"#fbb4ae\"},\"left\":{\"field\":\"left\"},\"line_color\":{\"value\":\"#fbb4ae\"},\"right\":{\"field\":\"right\"},\"top\":{\"field\":\"top\"}},\"id\":\"1440\",\"type\":\"Quad\"},{\"attributes\":{},\"id\":\"1354\",\"type\":\"SaveTool\"},{\"attributes\":{\"bottom\":{\"value\":0},\"fill_alpha\":{\"value\":0.1},\"fill_color\":{\"value\":\"#ccebc5\"},\"left\":{\"field\":\"left\"},\"line_alpha\":{\"value\":0.1},\"line_color\":{\"value\":\"#ccebc5\"},\"right\":{\"field\":\"right\"},\"top\":{\"field\":\"top\"}},\"id\":\"1472\",\"type\":\"Quad\"},{\"attributes\":{},\"id\":\"1353\",\"type\":\"ResetTool\"},{\"attributes\":{\"data\":{\"left\":{\"__ndarray__\":\"KQBsNTsT0z+Zlqxh1PbTPwkt7Y1t2tQ/ecMtuga+1T/pWW7mn6HWP1jwrhI5hdc/yIbvPtJo2D84HTBra0zZP6izcJcEMNo/GEqxw50T2z+I4PHvNvfbP/h2MhzQ2tw/aA1zSGm+3T/Yo7N0AqLeP0g69KCbhd8/XGiaZpo04D+Us7r8ZqbgP8z+2pIzGOE/BEr7KACK4T88lRu/zPvhP3TgO1WZbeI/rCtc62Xf4j/kdnyBMlHjPxzCnBf/wuM/VA29rcs05D+MWN1DmKbkP8Sj/dlkGOU/++4dcDGK5T8zOj4G/vvlP2uFXpzKbeY/o9B+Mpff5j/bG5/IY1HnPxNnv14ww+c/S7Lf9Pw06D+D/f+KyaboP7tIICGWGOk/8pNAt2KK6T8q32BNL/zpP2IqgeP7beo/mnWhecjf6j/SwMEPlVHrPwoM4qVhw+s/QlcCPC417D96oiLS+qbsP7LtQmjHGO0/6jhj/pOK7T8ihIOUYPztP1rPoyotbu4/khrEwPnf7j/KZeRWxlHvPw==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[50]},\"right\":{\"__ndarray__\":\"mZasYdT20z8JLe2NbdrUP3nDLboGvtU/6Vlu5p+h1j9Y8K4SOYXXP8iG7z7SaNg/OB0wa2tM2T+os3CXBDDaPxhKscOdE9s/iODx7zb32z/4djIc0NrcP2gNc0hpvt0/2KOzdAKi3j9IOvSgm4XfP1xommaaNOA/lLO6/Gam4D/M/tqSMxjhPwRK+ygAiuE/PJUbv8z74T904DtVmW3iP6wrXOtl3+I/5HZ8gTJR4z8cwpwX/8LjP1QNva3LNOQ/jFjdQ5im5D/Eo/3ZZBjlP/vuHXAxiuU/Mzo+Bv775T9rhV6cym3mP6PQfjKX3+Y/2xufyGNR5z8TZ79eMMPnP0uy3/T8NOg/g/3/ismm6D+7SCAhlhjpP/KTQLdiiuk/Kt9gTS/86T9iKoHj+23qP5p1oXnI3+o/0sDBD5VR6z8KDOKlYcPrP0JXAjwuNew/eqIi0vqm7D+y7UJoxxjtP+o4Y/6Tiu0/IoSDlGD87T9az6MqLW7uP5IaxMD53+4/ymXkVsZR7z8CsQTtksPvPw==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[50]},\"top\":{\"__ndarray__\":\"sJixhbRtwj+wmLGFtG2yP7CYsYW0bcI/sJixhbRtwj/FmLGFtG2yPwhlisiOpNs/CGWKyI6k2z8IZYrIjqTbP9z+HachCec/sJixhbRt4j/Gy2cWa7v0P7CYsYW0bfI/qbGboSAZ/z+wmLGFtG0CQCV/Vj3ZRgFAfUsvgLN9CkCwmLGFtG0CQG8+yUWOrA9Akn7lEGrLDEBvPslFjqwPQP0rEse0aRBAbz7JRY6sD0Dc/h2nIQkHQP0rEse0aRBAkn7lEGrLDEAlf1Y92UYRQO+xm6EgGQ9AootLS4+cB0DGy2cWa7sEQB6YQFlF8v0/kn7lEGrL/D8lf1Y92UbxP5J+5RBqy/w/8jHUN9hW6T+aZfv0/R/wP0ZlisiOpNs/8jHUN9hW6T+wmLGFtG2yP7CYsYW0bbI/CGWKyI6kyz+wmLGFtG2yP7CYsYW0bcI/sJixhbRtsj+wmLGFtG2yP7CYsYW0bbI/AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACwmLGFtG2yPw==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[50]}},\"selected\":{\"id\":\"1486\"},\"selection_policy\":{\"id\":\"1485\"}},\"id\":\"1470\",\"type\":\"ColumnDataSource\"},{\"attributes\":{\"source\":{\"id\":\"1470\"}},\"id\":\"1474\",\"type\":\"CDSView\"},{\"attributes\":{\"data_source\":{\"id\":\"1470\"},\"glyph\":{\"id\":\"1471\"},\"hover_glyph\":null,\"muted_glyph\":null,\"nonselection_glyph\":{\"id\":\"1472\"},\"selection_glyph\":null,\"view\":{\"id\":\"1474\"}},\"id\":\"1473\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"source\":{\"id\":\"1361\"}},\"id\":\"1365\",\"type\":\"CDSView\"},{\"attributes\":{},\"id\":\"1485\",\"type\":\"UnionRenderers\"},{\"attributes\":{\"label\":{\"value\":\"dist_a\"},\"renderers\":[{\"id\":\"1364\"}]},\"id\":\"1375\",\"type\":\"LegendItem\"},{\"attributes\":{},\"id\":\"1486\",\"type\":\"Selection\"},{\"attributes\":{},\"id\":\"1367\",\"type\":\"BasicTickFormatter\"},{\"attributes\":{\"items\":[{\"id\":\"1375\"},{\"id\":\"1391\"},{\"id\":\"1409\"}]},\"id\":\"1374\",\"type\":\"Legend\"},{\"attributes\":{\"toolbar\":{\"id\":\"1517\"},\"toolbar_location\":\"above\"},\"id\":\"1518\",\"type\":\"ToolbarBox\"},{\"attributes\":{\"dimension\":\"height\",\"line_color\":\"red\",\"line_width\":2,\"location\":1055},\"id\":\"1488\",\"type\":\"Span\"},{\"attributes\":{},\"id\":\"1339\",\"type\":\"LinearScale\"},{\"attributes\":{\"dimension\":\"height\",\"line_color\":\"red\",\"line_width\":2,\"location\":2079},\"id\":\"1489\",\"type\":\"Span\"},{\"attributes\":{},\"id\":\"1341\",\"type\":\"LinearScale\"},{\"attributes\":{\"data\":{\"x\":[2000,2001,2002,2003,2004,2005,2006,2007,2008,2009,2010,2011,2012,2013,2014,2015,2016,2017,2018,2019,2020,2021,2022,2023,2024,2025,2026,2027,2028,2029,2030,2031,2032,2033,2034,2035,2036,2037,2038,2039,2040,2041,2042,2043,2044,2045,2046,2047,2048,2049,2050,2051,2052,2053,2054,2055,2056,2057,2058,2059,2060,2061,2062,2063,2064,2065,2066,2067,2068,2069,2070,2071,2072,2073,2074,2075,2076,2077,2078,2079,2080,2081,2082,2083,2084,2085,2086,2087,2088,2089,2090,2091,2092,2093,2094,2095,2096,2097,2098,2099,2100,2101,2102,2103,2104,2105,2106,2107,2108,2109,2110,2111,2112,2113,2114,2115,2116,2117,2118,2119,2120,2121,2122,2123,2124,2125,2126,2127,2128,2129,2130,2131,2132,2133,2134,2135,2136,2137,2138,2139,2140,2141,2142,2143,2144,2145,2146,2147,2148,2149,2150,2151,2152,2153,2154,2155,2156,2157,2158,2159,2160,2161,2162,2163,2164,2165,2166,2167,2168,2169,2170,2171,2172,2173,2174,2175,2176,2177,2178,2179,2180,2181,2182,2183,2184,2185,2186,2187,2188,2189,2190,2191,2192,2193,2194,2195,2196,2197,2198,2199,2200,2201,2202,2203,2204,2205,2206,2207,2208,2209,2210,2211,2212,2213,2214,2215,2216,2217,2218,2219,2220,2221,2222,2223,2224,2225,2226,2227,2228,2229,2230,2231,2232,2233,2234,2235,2236,2237,2238,2239,2240,2241,2242,2243,2244,2245,2246,2247,2248,2249,2250,2251,2252,2253,2254,2255,2256,2257,2258,2259,2260,2261,2262,2263,2264,2265,2266,2267,2268,2269,2270,2271,2272,2273,2274,2275,2276,2277,2278,2279,2280,2281,2282,2283,2284,2285,2286,2287,2288,2289,2290,2291,2292,2293,2294,2295,2296,2297,2298,2299,2300,2301,2302,2303,2304,2305,2306,2307,2308,2309,2310,2311,2312,2313,2314,2315,2316,2317,2318,2319,2320,2321,2322,2323,2324,2325,2326,2327,2328,2329,2330,2331,2332,2333,2334,2335,2336,2337,2338,2339,2340,2341,2342,2343,2344,2345,2346,2347,2348,2349,2350,2351,2352,2353,2354,2355,2356,2357,2358,2359,2360,2361,2362,2363,2364,2365,2366,2367,2368,2369,2370,2371,2372,2373,2374,2375,2376,2377,2378,2379,2380,2381,2382,2383,2384,2385,2386,2387,2388,2389,2390,2391,2392,2393,2394,2395,2396,2397,2398,2399,2400,2401,2402,2403,2404,2405,2406,2407,2408,2409,2410,2411,2412,2413,2414,2415,2416,2417,2418,2419,2420,2421,2422,2423,2424,2425,2426,2427,2428,2429,2430,2431,2432,2433,2434,2435,2436,2437,2438,2439,2440,2441,2442,2443,2444,2445,2446,2447,2448,2449,2450,2451,2452,2453,2454,2455,2456,2457,2458,2459,2460,2461,2462,2463,2464,2465,2466,2467,2468,2469,2470,2471,2472,2473,2474,2475,2476,2477,2478,2479,2480,2481,2482,2483,2484,2485,2486,2487,2488,2489,2490,2491,2492,2493,2494,2495,2496,2497,2498,2499,2500,2501,2502,2503,2504,2505,2506,2507,2508,2509,2510,2511,2512,2513,2514,2515,2516,2517,2518,2519,2520,2521,2522,2523,2524,2525,2526,2527,2528,2529,2530,2531,2532,2533,2534,2535,2536,2537,2538,2539,2540,2541,2542,2543,2544,2545,2546,2547,2548,2549,2550,2551,2552,2553,2554,2555,2556,2557,2558,2559,2560,2561,2562,2563,2564,2565,2566,2567,2568,2569,2570,2571,2572,2573,2574,2575,2576,2577,2578,2579,2580,2581,2582,2583,2584,2585,2586,2587,2588,2589,2590,2591,2592,2593,2594,2595,2596,2597,2598,2599,2600,2601,2602,2603,2604,2605,2606,2607,2608,2609,2610,2611,2612,2613,2614,2615,2616,2617,2618,2619,2620,2621,2622,2623,2624,2625,2626,2627,2628,2629,2630,2631,2632,2633,2634,2635,2636,2637,2638,2639,2640,2641,2642,2643,2644,2645,2646,2647,2648,2649,2650,2651,2652,2653,2654,2655,2656,2657,2658,2659,2660,2661,2662,2663,2664,2665,2666,2667,2668,2669,2670,2671,2672,2673,2674,2675,2676,2677,2678,2679,2680,2681,2682,2683,2684,2685,2686,2687,2688,2689,2690,2691,2692,2693,2694,2695,2696,2697,2698,2699,2700,2701,2702,2703,2704,2705,2706,2707,2708,2709,2710,2711,2712,2713,2714,2715,2716,2717,2718,2719,2720,2721,2722,2723,2724,2725,2726,2727,2728,2729,2730,2731,2732,2733,2734,2735,2736,2737,2738,2739,2740,2741,2742,2743,2744,2745,2746,2747,2748,2749,2750,2751,2752,2753,2754,2755,2756,2757,2758,2759,2760,2761,2762,2763,2764,2765,2766,2767,2768,2769,2770,2771,2772,2773,2774,2775,2776,2777,2778,2779,2780,2781,2782,2783,2784,2785,2786,2787,2788,2789,2790,2791,2792,2793,2794,2795,2796,2797,2798,2799,2800,2801,2802,2803,2804,2805,2806,2807,2808,2809,2810,2811,2812,2813,2814,2815,2816,2817,2818,2819,2820,2821,2822,2823,2824,2825,2826,2827,2828,2829,2830,2831,2832,2833,2834,2835,2836,2837,2838,2839,2840,2841,2842,2843,2844,2845,2846,2847,2848,2849,2850,2851,2852,2853,2854,2855,2856,2857,2858,2859,2860,2861,2862,2863,2864,2865,2866,2867,2868,2869,2870,2871,2872,2873,2874,2875,2876,2877,2878,2879,2880,2881,2882,2883,2884,2885,2886,2887,2888,2889,2890,2891,2892,2893,2894,2895,2896,2897,2898,2899,2900,2901,2902,2903,2904,2905,2906,2907,2908,2909,2910,2911,2912,2913,2914,2915,2916,2917,2918,2919,2920,2921,2922,2923,2924,2925,2926,2927,2928,2929,2930,2931,2932,2933,2934,2935,2936,2937,2938,2939,2940,2941,2942,2943,2944,2945,2946,2947,2948,2949,2950,2951,2952,2953,2954,2955,2956,2957,2958,2959,2960,2961,2962,2963,2964,2965,2966,2967,2968,2969,2970,2971,2972,2973,2974,2975,2976,2977,2978,2979,2980,2981,2982,2983,2984,2985,2986,2987,2988,2989,2990,2991,2992,2993,2994,2995,2996,2997,2998,2999],\"y\":{\"__ndarray__\":\"wz+TDRgK4T9iKyVwz7ziP6bwobIMquA/neB4/Oo24j8QVMHX5kfaP8uM6yHu4eM/CmyqATA04z9waVhL15XgP9o6hglBT+U/Omav6EEz5j9X6E3skhzcP6LeczhgwuA/tp+4Wx694D8QDbE7OzHgP2DYr6602uU/z1qc6WqU4j8ccspqglriP6AzYZtFk9k/nB7wBLw75T+/Y3jB/3feP312yftLZOM/YzkqXG0W5D+trFP92Y3nP0/aIlYHBt4/KQBsNTsT0z/W7Hk+z8njPz5ZNcMt9ug/dMsKFSUq5z8jSln3897jP//5dp5zoOE/V76nE2XI5T8qzJbDqxXgP7kONapuueQ/LOu7+UfR5D/g7TFVuZfmP/7XY9vpB+w/3awEy6p05D/V8ZvKQZLhP0rbxOA/HuM/OE2CL/IT2z+0L/lLF/rgP55TnnPq4+E/JU6V5d2F4T9JERxVArDjP3YWST15keA/yejfXozJ5j8KxOmfXjPjP4nGAsiUK+M/8YumsZYm4j+pBJkLVbLjP8Ty+9ge1+U/DRM60Nhs4D8+FTsrEhjhPw55MTxjOuI/GEpgOFPJ3T8S0cRfEJTgP88Mu3EUreE/wqPj5n//5T9lOTdNSgrkPytPbfLJ0eM/4mGyb0Hs5T/vx3c/qsLiPwBiNZ0Rc+Q/W1eJB6Te4j+Eb9q3lQzkP8lPANrNVeE/PskBWKTK1j8/zZLRMsXiP3MuIxy3wOc/vZL4pfQp5j8F3/+5DUrmP9gp/5nWJ+c/3htfC9R74z8oJJRN19TjP79ZjzU4OeE/pd/z6kUw4j/GehPFoyvlP1/FxRBsDec//K5HMvfA4j84phSrZ8LhPwN5xuuhM+M/GYPCZLUf5T94awPMySjdP4AY6k1WtNc/rgInQTNw4T9+0bh/YpbeP6MHEXrukuE/L/++ayO64j+ETYce5r/hP+orMLgP2ec/ySMti94+5D9KNYw9WSnkP3Q2437pMOU/wf18K4kc3z/Oi3+8cIbmP1kACkci9eI/qOM+LXVY5T9AUFrkRIrfP6qPQL24dNw/N6fcFzfR5T/LksH/jWfkP89UIkc9UOA/TdYKosVq4D+RSxz7J83mPzDV3eSqyd4/CW3ar+B06D9qrq5XaVHgP1RCu7ArPuU/flkf6f4l4j/8vdLFVCHlP0YyJTl2deE/NuCaDdet4j9TiLOSuFTjP1xTuYNq/N8/6U9lUxWS5T/s2aaDxo7hP1RjslLud+I/3wakBaEE4D/mN8DERQDWPw+dSca1luI/g+NeU53r6j/qCCml8rXlP3zIbwFsI+M/rNJG/9Zb4j/j46KjmkXjP+meCGVm8+Q/3uzJm8HX3j/y16VlDcTmP3v3H4k9feU/yC7BqYl/5T/lry06ZprkP6ibfRxFQ+M//4O9bWha5T/YVZKeAxjlP9dgQ3srEeI/X4mBME5d4T/4u2kDpIbjP8INWEUoKOg/IeA+OT943j83p9awAQLdP6h4gg0suuM/+PfjH+Nc4z/JnfGUM8HjP4TeyOQL++M/ej1mS1p84j9w94A1svfnP440nJp6J9w/Wyffx36M4D97nHkbnd7iP6gDRg8z4ts/aFfA2yOj4j//dYO/RoToP52CAGpPROM/3+bDrzDw4z/e4Az9vlXeP3YNE9hGOuE/K2Hp0Nb/4T8kKNOD7S7iP7segmIfTec/BOAJfg/85D9KfQAlY0nfP+CNuP8i/eM/Iz2m31fL5D/U2JzINdnmPypjL5v6Qeg/nFV+CUXn3z9SogsO8prgP7Gk/gP8Wd4/mHeurX1z4j944cejPbHkPwHhb0EGXOY/lL6qHfv04j85hgxQLi3iP5PEXiiFr+M/cj0TCg2H4D9UGB478+LpP0ZvCX5oHNw/iuyBG4rK4z84PNgu8KzpP7g1EzvFOOM/r0EyIIWX4j8Bwi92YQ7iP1SNh1Jsn+I/prd2ZNaX5z/m2xZGfD7YP66vK45jG+g/fA5ie2NJ3T92HUgnwljiP+M+3B6Q0+E/Xy5FxFcV5T8mMr52CCzcP6gqRtXPreQ///uPsoOt6T8SGzDI+azdP0COhXebzuM/E9qleecU4T8W1Ho4FpDkPzIirMfiQuM/OrICpbEl4T9zRA1v3cLkP/IGumxS+eg//FDsxc+W4j+QZnth04DlP7m17WeAH94/SfU19K8j4D+Jvx/6MbbkPxqAxuP88uc/Ovp8uYRW5D/XFiosvjLiP7d+0ZieMuM/pGFMYrtl3j8YaKhAayLlP74lydoBBuY/5kJF/NjA4T/Lnos3JLLhP/ai9ucBDeQ/UgDnbnDN4T/5vnwvBv3iP3m9Y8ZK6+k/ChTgJ9Zo4j/eHsfSpQ3iPwudOde7IOE/ubhxkdeU5T+IG3y8vZ7iP483q807H+E/+ZdIvZxt5z9eziuHI77nP9Cxd0BVR+E/8Kk0zUJ02j+iMI0ijmzmP/thTutZAuE/D+nYZ+G65T8Q5rr3lMrZP5j5521qDuY/smW/EgSm3j+dIxuL14nlP9PTsXrkPuM/nHGXmrsl4D/90ot/weXhP/V2ypxuZeU/Lapc4PKJ4z/rWy588xHlP1i9HNVMhuk/ODdE6otj3D+evjNCBWDoP8Ng7zmwiOM/blAHhupS4D+iLYBs0OLdP8FWiMdFmOI/lTnQPDcm5j+PGvE1tsriP8TswVsjCeg/FHYpM8sc3T9kJcH2SynjP2BVQ4R6Yt4/Iuc+VRZd5D/acmX3vAnmP1i/aA6D2uE/JPq/8Ke71T9YSoCmLdbjPyRQqOn5mOQ/wNCU9GV+5D+BiW+CjSfnP3C9Pe2Vi98/rv1J225g5T90cscmGgTnP/6RvbJcBNs//CvuCw855D9dqaEGLZbhP7+qA6hq8OI/JqJHPgRQ5D9Y29f2D6ThP/3x7fGoCeE/YQIney1P4z9VmqVmDnHfP1Xi9FRic98/f9aZGr5f5T/R+yBCNwzfPzE6NA8LVeU/7bUgoCeu5D+8auqX1lrbP2yJ6pcjCOE/YsqfF2oF5z+y79jfbQ/gP+U5N5HCtuE/exaEZLit5D8LxeoY9LTlPzenFawjZeI/FXniOrhJ4T9IZa/HZ8DnPxa3CeS0w+g/wOiknBRV5j/I1Bm2F3njP+88sJcRneA/p9GbYuaK4D+OaJ9ZPt/kP3qMpKrzieQ/QXCvumOv5z/fl6N4DEjlPyLA6tPAx9w/juvjmvSQ5j/cOke3lgHgP9YngZam+OE/asV9/CkA5D+imboBDZbpP8ZAV2TYJu0/Y7NoPzok5T/Mi58VIJ3iP2lnsigR6eQ/xMgACV4K5j+NTiRTgiziP8b2/E7W8eg/PPoHiazv4z/UdtF988rkP1WZQKO5VOU/DkQDUpCM5D+ULlzGdeLlP8iSFNOLOeE/vM4E1tZp4T926h5F6FvfP1zP2Uk9m+Q/mpVNBk6x5T8aYdAgNarkP4zOLrfxjug/eQ06oZ0u4z8N+nN9CFflP34wZt3Bad8/pnUrghb24T/QNTkK/GzlPyRXqNHI6uU/uPfe3GFC4j+UuaK/i/jiP/kc3QGEr9w/ZWvvv7gO4j9ahk0VmgzmP83AW29oCuU/MstxaVjN5D8A3K5/+VvjP9I0pHP8OOM/rLv20UMW4T/dHFQkrm/lP/LjDqAPjOQ/6dz2B0PG5D+QmKCyRYThPwm28+pcntg/gnDlzPy15j9CgAe176rhP0AXEgJqcOA/OwnKRExr5T80X9kwEPLhP/I5hPh/l+Y/KsjiuaYs5T/e0ssTLmPlP3MZtwxrqN0/6AR5RgYU5z/n4EtoNwnkP2pJvyuwBOI/6Vjs8Kmo4z/eeIRYNwjbP4aH9ef3geQ/XbZSsuDP3z9EmWZcL77dPwzyda0qqNw/iRt33bbC5j9gdv4ZtS/hP6FkozahG+g/8CEwfUV84T+0qqOREHnbP7nohxsWQd8/121ZrpMn5z+80YD7fLPiP7IIHAULcuE/IPdIdEm24z9QHriqQs7kP3IMVreSxuY/Ill1gc/V5z9cRRo/egziP2CVSLDN6d0/LQJjRX3g4T9ll376awjkP3hu1U2wHeA/4me9Ghoj4D9y24y+1EzkPw/S9g9YC+M/5tZ7OhFO4z8ovv0y18XgP108C6t0duI/fPYXdhE+4D9a8XtWcwzmP9TmpbhGg+Y/UagiFpuV2j8yoeCkvjngPwRwMol80tw/UYnzTLMe4T8fa0792O7iP3YwKiBNH90//NBWLwNA4D8ehmgohPnfP0taN0/+3OM/4VjywgFs4z8W/Vnd8+PgP6MVfTkNxOM/8zukE1xz4T++WIT+qFTiP/HQMIoXjug/EgHwQU595z9fXZ+FKhXeP+m1MLPq2uU/CzYFZOXL5T/JGpfeFQ3fPwSYO1Hx0eU/DPtc5hIf6D+LRJ/7ODffP1BrkumUQ+A/CWAuR9505j94cbbkmxHkP0COSCj49uQ/FMuF7zVK5D+sA70SQnPkP/1tJaob6d0/UkZupCmN5j+AQsr/U/HmP9kcZ3lId+I/XPE6bJUP4z/WhSVe1ZncP4C1Lz162OQ/x7Q+V9QH5T+JBmbMQADjP2uIkjBFzOY/trpuIIYh4j8xFwmluQHlPyjjtS3I398/DGK69Ocf4z+lPkF2XqTiPzSgKLLF6+M/lFdFyXsE4j+MQ/dlf8fiP7dXB9nl1+U/xDLDGWfN4T8S4b4fLx7cP7G+04l1zOg/RK9PHuOe5z/PJxEkKCHePw+jCmk1aOU/3XXSvimX4T8FCVx5DAvkP4f72Hg6JOQ/yhU/Uixz4j/qpm6gI7HgP1A4vz6C/eA/BuBG1KdE4D/QzXFwxYngP2YoN+Qq/OI/GSZjc9no4D8hsqfi72HlP5ypLVON9ec/iAHuNwVY4T/4NHT2UffjP/PBScOKzOQ/8VoqQN215D/agZgsbHHjP2hX7f2ak+U/xyO0IkK+5D+kJLcnnXreP+aMGHA2++U/oh9NkxEK5j/IZYOA6sHgP4hKO1dHUuM/WRoorfti5T+1LRTpuofiP/6JgQEvouY/g19nMa/D6j/LTWL7Ua/gP0BHR0+yj90/CLrZkxYs5D//HTjTTYLlP9PjIQMTduI/NRb4DLbZ5z8iTBlvOtXdP7tFgsTWauU/mLgP+uBC4T+kZqjhtrPoP/XUrvR3k+k/13YNqSe/4D8r9x+3AHHhPzmSe9hf8+U/pMyyBBD24T+1WQDH/w3jP2Qqpon/R+M/Oi7HHjUd2j8oCt023SfjP6BnSs0s/uA/V9JC2Tuf4T96vjrG0tHnP5Zh8jW9W94/GKo5sGDN5T+s+OpurE7iPwYUhT0FTuI/G+dDM6y65T9JbxAgxUnkP8O8OGulBuU/+FNePVZM5j8QFNHLyiPiP/xpzNmnPeE/F3bUR3103z8+scwQBo/gP3CnkSAZo+w/p6Rdbdcu5z84gfe8zMDdP/JJiVSv790/jNNm1vG95D/DkUnyhPPkP1WSuXvY9OQ/hm9CGFdi4j8W3OaIbcziP2a8HYXMP+Q/fsDcRJB54z/VT2BbTDLYP9O/ebnyduI/JctQVLp54D8hUKCyu8LjP5ZEh2O5wOw/XH7jsD1g5D/DNFhCWzLiP2JEgfxWJuY/UF3SL56+5D/J52tmV4vkPw++V/5UJOU/wiwfpevH6T+XcUVpjSnfPz/5swszt+Q/5Bz9/k484D/wkk6lRvrkP2xzrKycQuA/DZ4DCUvd4T9m5g9cdkHiP/XrZ8yWUuY/UeqxDpkj5j/CQ5qpam3eP/3F07czYOM/xgvThOnt4D/A4ZEud1jePwOcBW1rgeI/M0Caq3024j+W24SIKvfqP9h/1HThleQ/n7mYvU+V3T8zDruBiWThP731VM3qd+Y/8yPQZA3i6T+wyvLI7D/cPzlmkvZUXto/9sMzCeUv6T9oxRI3X3LkP4yOHQTla+A/mW+0Cyvp5D/nEOmr/YfVP3udVBj3NeM/C1pnSMA/5D/EjHmyXCjmPyPie5076d8/XCCmGWV54z9T6uaWFz3gPw2QLg5pBOQ/ZySJd7tV4D9srTpA4uThP4sSSJNtA+A/7Jf/XZxI5T9YN5dycfLlP6dnQ7P/cuI/pzfsSYZS4z+SP58fByHdP7r+qnO0PeI/y6FYhxIK4z8U++RYJ5brP2XYJqCNSN8/o10mbKfi5z/4phWj02PmP34gcesYGeQ/jGuFpdzT6D9B1KRfB2TkP0zcbcz/ceQ/GFrb/a4H4z+IQGExjB3nPxEf9HZtteY/aruaKaUy4T8+GH8F8vTdP35TNRPrjOI/BmXU886j4D8OtpEhKf/iP+XuMMnoKOc/95WJF5S84T9cCmx8HhDjPzgqDnO7YuM/349ZDTzq5T/yWTi1GAXYP5PUJO0wQuE/hnUOmkjg4z8wtZT/TQrnP6eko8c7oOE/BrbbJdtj2j9EBR4c3i7lPzCirwy2KuE/0DeNCUDJ3j+CeVJ7vDPhP2Gsfm0JnOI/XQI+DctD4T8NMEryzEjYPzq67J0TEuY/w0rP35NH5z/LkcRD7UjlP5e+tXFqHt8/TUfXXCOO4j8EKcGU2y3jP1TUM3y7HeU/FHD4W7RW5T/QIYLLw9ngP8B7HepeduM/oY782c2p5D8h5/pEz9vnPw/4De5UdOU/WH+JYZi55T8SxQTr3nfjP81g9uultuc/HAvMr+CC5D8JCQn6y3HgP/oAxjbXsOc/3nHXH+xu5T+Dm24iQsjhP+KLhzxmquE/RieKmScm5D+exYsXpK7kP+fYDmPS1OM/t6xXXDky5D8iFM4ZzqPoP2uGIxYCoeY/g/+SYV7P4z/4ejJWUjbmPydFc3y2yt8/LX0FjS555D/Q9DQ+Yv3oP2sKVKmTgOI/94Ure9F84D8VTpE/Ah3hP6U65i0lbd8/gmwRs1Kw4D964x4FrgPiPwpE6NyEZ90/P4Aemv4/4z8d5B1wWBbmP5h8Xu6QS+A/dmrEIcwP6D/aGaQQpNXkP4bfFp1zf+Y/w0iDS00Y4T+/i/8gOu/lP7w2nCBgad8/ps33oX0M5j+v2oGF+L/jP7Bh52Ns+OQ/s4TJSE7g3j/6g/Jy9VXgP+1EBQlzHOU/pmpoNVAq4D+T+aWZqK7kPwYFND57pt0/0UiCOQbq5T88EzCnCIPeP7XuUuSk9+Q/oDMxhLcz5T9g0wmPZ/jgP6iAlgT/D+U/vXG+5o0I5D/6/lLfx3nhPw6oyuf78d8/ikBmhdXW2T82bi0holHkP8ouNYygYtw/rkjuU06B4z+D9csaYlviPyehUsONX+U/WR7sqsM74j9kR9n9UCXiP7Gecpy/iuU/1Csq9q1F5D8sHejoMTbkP3VcF7+fs+Q/FIcnxYSS3D//4hRKKZ/lP3qMMzfvKOU/vY+1lqrk3z8OuozQP2viP96pQ93/EuM/ANkRLgXF4j9vVqy1skTkP3AlKJLGx+c/GmsqIGKp5j/gLmDHdQDeP52ZgLvMMOU/QE2Gt9xz5z8o9YbTEnDkP7jMyGDusOY/wSwj5qaj6T/v0vam0nnmP3nbggBv/+M/9pVrI1aL5j/DXYJh5KnjP4iWEAjIRuM/wAgUS1IT4j8nZ8e+8jPoPzlYxUS+lOA/X5r8AIQa6D8etf9NsMzkPw4Q9NnGcd0/0otS2GBg5D9eYR9xt/XYP0FcJb2dM+U/hXKPsmEI5j9ybfwm8U3hPxzNngdpmOM/6p2Qtwt06T/0ocNt5JThPw+tAF58n98/F2xqifrw5z9KnK+Q83rpP31QEj06NuM/6GltX8pv5j+1CwUXI37nPxA/fvj00uA/v4SOMKOl4T/QiHFLiCPnP7VOdE6Ql+g/1WBG308A5T8OtMPBdGLgP1TjUh4Aj+k/d0MviDqA4T8C45O2pyDiP/vPvaWkS+Q/oUNyQ14s6D/p9FXxxu7lP35WJ80Oh+Q/jbPE2Lmu5D+YGi/j2FbjPx7hN4pt/OQ/K9Q5Afg11j+4i34bOEDiPxeu7RPk+OM/dI89GpQI3z+aYmUSrW/kP9pzHHqyi+I/pGYdI0nR6D9goqVY0TvoP9cYIbGcteE/JgQ5PaK34j9NTIBFn/biP31KtlczweE/ncVN5/bS4z8rwJzsA8bgPyHCM/X/Kd8/GgN4NVMx5T/9tWYf6TblP1NUqPVNoOA/+iXe16wQ5j/x7Q3Azy3hP85XbMyXAuQ/aj8FzznT5T+pj18iqhfjPw7CBUi5p+Q/3mrPDSSM4T+gV9ErQZLiP0ryKFmPiuI/h91rX5rP4D9yP5X4YqLiPy63tc4kfdw/Ahgvy9ig4T9kWJt61EniP/fUdmlWYeI/MonMg11t4j9sHvhOmgDjP6jgtF/1u+Q/vGoduZr/5T/TJTWO6R7hP+EjSmITs94/fGScSx273z9hcolXFaThP2KuUPPnEuI/YKArUmu84D+Y+Uer1xreP2y3vLqLveE/HT6zHtPH4z/yJGbspRrlP5sp6RyPauE/BAp1R9/g4T+w+ZN6DDzgPyvEl+1xGeM/SVC90Sp84D/m9GDQtAjlPwb587Nv+No/+kMtZJcM4j89U2H83SnkP1i9pBfMyeM/PG8fS1XR6z8vTwWVwVHkPxqo+mBb+d8//M515f7k4j/1YJqwMgjbP9HaGzUs8eI/NW6xrsKI4D/rV2h0FiHmP2EWXEAEceE/4mPUnIvT4j912GcPjSrhP9BoT8Kqw+g/YKGEtQks4j9pjVJEbhDpP+99GjMnzuU/kxCO6xu+5D/F+U4lRGHkP+/kqXrMdeQ/gQNC6xkQ2j+K6/scvU7iP1oCR6yN5+U/UcKmtyQL4z88kXsPI2jdP1q8rjBFCOM/ojN3BzSn6D/DzHJYpzDnP9BdSwRoAOM/ZEFPk6A+5j9VP18b9QbiP2uS2dvGfts/M01ioeA94D8W5Ay7hDPoP2WuoK++UuI/E3xN4dFr4j88gZySkj3iP9MQ8ZURS+k/dMjGfy9n6D+rJfIj00zqPwD9ewDpsOI/DZwWJmAl5D84s6loiLHcP0tybswa7ec/V/LbJxzt3j9EYqlUyOTjP25kplyUYN8/yxAr4MxQ4T/utbSKUIXgP9RRpIQ6QeE/JNJ3+Yx54T9BnWfmDnLhPzn76/zd3eU/0qpUWzFU3z/9zOcW3ufjP+umSb/zGOc/rsMsH8qO4T+v/DyyFOfgPyfFn65aduI/Al0NgqH55j+pHUmFNtLjPy8t63wHgOE//xQafZe/5D++nQuQZ1jfPxyMYKVeYeU/sFr+XO6B5D+z4rwVCzfiP2+x9QI7hOA/Nvqou9Rc4D90zk+GUunkP7AohRf7Iec/kRMQFB4p4T8OXPXbtarkPziCodIDC9k/De7G+UdU4T8DOnCrwRnjP04eocL8SeA/KxsMlAM04D+NF0CtJxDhP9CH2nSQIuQ/pboadb2Z4j8qGLMtSBPYP2YvZUnY09g//t4MWQ5C4T+9vK8II6rkP+a3RVjj0NQ/jFWoL9KZ4T9HiYgGGYThPwl2HbyGpOc/CqEURsr34T8VnMDP8WzkP1ubKfjqpuM/fvSUDK7D2D/njBnug6jlPwRYCB+n4+M/uSYD3JLT5D8CsQTtksPvP9n4fRPKD9k/4S6HTRK46D/QvwIWuUfiP2SKS3CUHuQ/fcMil+oN4z9Yf3pkqtfhPxQZ0JJkYOE/VOe8MCJB5D9hPSmhqbDcP1toaw8TmuU/7F066EDe4T8fsxD35CrfP9sC2qmMwuE/2pLbfw455z+YkI9Lp3zhP72ySl/YWOQ/MWP2jmbc4D8N6K3uFqDlPwrUsJfEZOU/dO85kpQd6T9rhKXFYKPiP/Mxr0b2VuU/pZQjdImD4z/Og7w1x1jnP0u6Px9V5uA/TH5gI4KV5T9R/bQ3tJPiP0d+rvad/uI/1U0zF11+4z9FBl7Y1QHkPwM69jP80+I/Ilxy0Rfj4z943EeL/j/oPwk6WPp6WuY//wMTITb75T/MAx+jBb7hP4ySweG7WuA/zb1kP8hB5j/P+j6H2wDmP5+cE6Cc6ec/fyRASGtE4T9A1ppbJ3fiPwAe5OZE6Ns/FLMiC7Dx4T+otRYvl2PmP6EiOwcMteE/q29gjOE/5T+uDJPNI5fiPw2abeKqPOQ/xGHFvQUO3z+Tj7HqAHnjPzzKEwu5QdM/waAcPBia4j+6Vu5aCPfbP6Dqy5QmD+c/Pr2lVg2w5T+46sQyf7HkP9G3O1M1Eeg/GsFn+9gp4D9w5pDSo8roP/nvufDZLuY/Atbr9Wlx4j8WQZwBgNXmPwmoAkXZUd8/GuE2DMKP4D+1nDmvpEDhPwORvojcgeE/NKNZAVaj3z9hSl5PJh3nP9ZaZn8lX+I/PaqV2fhT5D/uE2DZ91/hP3djhweGv9o/Saw6VW4Q5D90vN+HXCnaP+KL6Jj5+uI/wvbWeY2k3T9qoQIvLY/pP9IbyNDJHeY/PgGjYNWJ4z+OcdqybT7nP2Y/IuDlfeA/xwArFmvw5D82gi184tbjP/SbYjl8C+Q/oE+ZEYNF5z/BdIbtQYvlP89T0fC4H+Q/epD4TAjP2z93aTDmuiDgP6CxfUWtzuI/8JNfzrB/4z8qiISZrCrfPw6i03TY6uo/qLgjFW4N6D9eTmGQdCDlPxbTvWw2buM/NiThwFuF4j8slgVuYyfgP4cr3QqscuM/nWVU01kG5D/gAN9D5nPeP+xQ/RL1ROQ/ZHJ8BAO04j8=\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[1000]}},\"selected\":{\"id\":\"1408\"},\"selection_policy\":{\"id\":\"1407\"}},\"id\":\"1392\",\"type\":\"ColumnDataSource\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.1},\"fill_color\":{\"value\":\"#fbb4ae\"},\"line_alpha\":{\"value\":0.1},\"line_color\":{\"value\":\"#fbb4ae\"},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"1363\",\"type\":\"Circle\"},{\"attributes\":{},\"id\":\"1369\",\"type\":\"BasicTickFormatter\"},{\"attributes\":{\"fill_color\":{\"value\":\"#fbb4ae\"},\"line_color\":{\"value\":\"#fbb4ae\"},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"1362\",\"type\":\"Circle\"},{\"attributes\":{\"active_drag\":\"auto\",\"active_inspect\":\"auto\",\"active_multi\":null,\"active_scroll\":\"auto\",\"active_tap\":\"auto\",\"tools\":[{\"id\":\"1351\"},{\"id\":\"1352\"},{\"id\":\"1353\"},{\"id\":\"1354\"}]},\"id\":\"1356\",\"type\":\"Toolbar\"},{\"attributes\":{\"axis\":{\"id\":\"1343\"},\"ticker\":null},\"id\":\"1346\",\"type\":\"Grid\"},{\"attributes\":{\"fill_color\":{\"value\":\"#ccebc5\"},\"line_color\":{\"value\":\"#ccebc5\"},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"1393\",\"type\":\"Circle\"},{\"attributes\":{},\"id\":\"1344\",\"type\":\"BasicTicker\"},{\"attributes\":{\"data_source\":{\"id\":\"1361\"},\"glyph\":{\"id\":\"1362\"},\"hover_glyph\":null,\"muted_glyph\":null,\"nonselection_glyph\":{\"id\":\"1363\"},\"selection_glyph\":null,\"view\":{\"id\":\"1365\"}},\"id\":\"1364\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"overlay\":{\"id\":\"1355\"}},\"id\":\"1352\",\"type\":\"BoxZoomTool\"},{\"attributes\":{\"axis_label\":\"value\",\"formatter\":{\"id\":\"1367\"},\"ticker\":{\"id\":\"1348\"}},\"id\":\"1347\",\"type\":\"LinearAxis\"},{\"attributes\":{},\"id\":\"1335\",\"type\":\"DataRange1d\"},{\"attributes\":{},\"id\":\"1337\",\"type\":\"DataRange1d\"},{\"attributes\":{\"axis\":{\"id\":\"1347\"},\"dimension\":1,\"ticker\":null},\"id\":\"1350\",\"type\":\"Grid\"},{\"attributes\":{\"bottom_units\":\"screen\",\"fill_alpha\":0.5,\"fill_color\":\"lightgrey\",\"left_units\":\"screen\",\"level\":\"overlay\",\"line_alpha\":1.0,\"line_color\":\"black\",\"line_dash\":[4,4],\"line_width\":2,\"right_units\":\"screen\",\"top_units\":\"screen\"},\"id\":\"1433\",\"type\":\"BoxAnnotation\"},{\"attributes\":{},\"id\":\"1373\",\"type\":\"Selection\"},{\"attributes\":{\"background_fill_color\":\"#fafafa\",\"below\":[{\"id\":\"1343\"}],\"center\":[{\"id\":\"1346\"},{\"id\":\"1350\"},{\"id\":\"1374\"},{\"id\":\"1488\"},{\"id\":\"1489\"}],\"left\":[{\"id\":\"1347\"}],\"plot_height\":400,\"renderers\":[{\"id\":\"1364\"},{\"id\":\"1379\"},{\"id\":\"1395\"}],\"title\":{\"id\":\"1333\"},\"toolbar\":{\"id\":\"1356\"},\"toolbar_location\":null,\"x_range\":{\"id\":\"1335\"},\"x_scale\":{\"id\":\"1339\"},\"y_range\":{\"id\":\"1337\"},\"y_scale\":{\"id\":\"1341\"}},\"id\":\"1332\",\"subtype\":\"Figure\",\"type\":\"Plot\"},{\"attributes\":{},\"id\":\"1372\",\"type\":\"UnionRenderers\"},{\"attributes\":{\"text\":\"drift stream\"},\"id\":\"1333\",\"type\":\"Title\"},{\"attributes\":{\"data_source\":{\"id\":\"1392\"},\"glyph\":{\"id\":\"1393\"},\"hover_glyph\":null,\"muted_glyph\":null,\"nonselection_glyph\":{\"id\":\"1394\"},\"selection_glyph\":null,\"view\":{\"id\":\"1396\"}},\"id\":\"1395\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"items\":[{\"id\":\"1453\"},{\"id\":\"1469\"},{\"id\":\"1487\"}]},\"id\":\"1452\",\"type\":\"Legend\"},{\"attributes\":{},\"id\":\"1413\",\"type\":\"DataRange1d\"},{\"attributes\":{\"source\":{\"id\":\"1454\"}},\"id\":\"1458\",\"type\":\"CDSView\"},{\"attributes\":{\"bottom_units\":\"screen\",\"fill_alpha\":0.5,\"fill_color\":\"lightgrey\",\"left_units\":\"screen\",\"level\":\"overlay\",\"line_alpha\":1.0,\"line_color\":\"black\",\"line_dash\":[4,4],\"line_width\":2,\"right_units\":\"screen\",\"top_units\":\"screen\"},\"id\":\"1355\",\"type\":\"BoxAnnotation\"},{\"attributes\":{},\"id\":\"1432\",\"type\":\"SaveTool\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.1},\"fill_color\":{\"value\":\"#ccebc5\"},\"line_alpha\":{\"value\":0.1},\"line_color\":{\"value\":\"#ccebc5\"},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"1394\",\"type\":\"Circle\"},{\"attributes\":{\"label\":{\"value\":\"dist_c\"},\"renderers\":[{\"id\":\"1395\"}]},\"id\":\"1409\",\"type\":\"LegendItem\"},{\"attributes\":{\"text\":\"distributions\"},\"id\":\"1411\",\"type\":\"Title\"},{\"attributes\":{\"source\":{\"id\":\"1392\"}},\"id\":\"1396\",\"type\":\"CDSView\"},{\"attributes\":{},\"id\":\"1445\",\"type\":\"BasicTickFormatter\"},{\"attributes\":{\"formatter\":{\"id\":\"1445\"},\"ticker\":{\"id\":\"1426\"}},\"id\":\"1425\",\"type\":\"LinearAxis\"},{\"attributes\":{},\"id\":\"1447\",\"type\":\"BasicTickFormatter\"},{\"attributes\":{},\"id\":\"1407\",\"type\":\"UnionRenderers\"},{\"attributes\":{\"data\":{\"left\":{\"__ndarray__\":\"Jn/0Ffpp5D8TuZNHF6TkPwDzMnk03uQ/7SzSqlEY5T/ZZnHcblLlP8agEA6MjOU/s9qvP6nG5T+gFE9xxgDmP41O7qLjOuY/eoiN1AB15j9mwiwGHq/mP1P8yzc76eY/QDZraVgj5z8tcAqbdV3nPxqqqcySl+c/B+RI/q/R5z/0HegvzQvoP+BXh2HqReg/zZEmkweA6D+6y8XEJLroP6cFZfZB9Og/lD8EKF8u6T+AeaNZfGjpP26zQouZouk/Wu3hvLbc6T9HJ4Hu0xbqPzRhICDxUOo/IZu/UQ6L6j8O1V6DK8XqP/oO/rRI/+o/50id5mU56z/UgjwYg3PrP8G820mgres/rvZ6e73n6z+bMBqt2iHsP4hqud73W+w/dKRYEBWW7D9h3vdBMtDsP04Yl3NPCu0/O1I2pWxE7T8ojNXWiX7tPxTGdAinuO0/AgAUOsTy7T/uObNr4SzuP9tzUp3+Zu4/yK3xzhuh7j+155AAOdvuP6IhMDJWFe8/jlvPY3NP7z98lW6VkInvPw==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[50]},\"right\":{\"__ndarray__\":\"E7mTRxek5D8A8zJ5NN7kP+0s0qpRGOU/2WZx3G5S5T/GoBAOjIzlP7Parz+pxuU/oBRPccYA5j+NTu6i4zrmP3qIjdQAdeY/ZsIsBh6v5j9T/Ms3O+nmP0A2a2lYI+c/LXAKm3Vd5z8aqqnMkpfnPwfkSP6v0ec/9B3oL80L6D/gV4dh6kXoP82RJpMHgOg/usvFxCS66D+nBWX2QfToP5Q/BChfLuk/gHmjWXxo6T9us0KLmaLpP1rt4by23Ok/RyeB7tMW6j80YSAg8VDqPyGbv1EOi+o/DtVegyvF6j/6Dv60SP/qP+dIneZlOes/1II8GINz6z/BvNtJoK3rP672enu95+s/mzAardoh7D+Iarne91vsP3SkWBAVluw/Yd73QTLQ7D9OGJdzTwrtPztSNqVsRO0/KIzV1ol+7T8UxnQIp7jtPwIAFDrE8u0/7jmza+Es7j/bc1Kd/mbuP8it8c4boe4/teeQADnb7j+iITAyVhXvP45bz2NzT+8/fJVulZCJ7z9ozw3HrcPvPw==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[50]},\"top\":{\"__ndarray__\":\"E4/QkR0Lwj8AAAAAAAAAAAAAAAAAAAAAY4/QkR0Lwj8Tj9CRHQvSPxOP0JEdC9I/E4/QkR0Lwj+e1rharBDbPxOP0JEdC/I/Y4/QkR0L8j/YskT25I32P4Do8gwQUv0/cfEP5sFyDkDYskT25I0GQIDo8gwQUg1AfRx8Vyi8E0DeE19+dpsSQCVSKm5TgBpAgOjyDBBSHUBeyPoumDIhQNiyRPbkjRZAj1tHRwWhG0DYhbO4a+ogQHJ2ntIaAx9ABZjtas8rI0D4bIH5aOIdQJ7WuFqsEBtA6XWe0hoDH0AoxX6oSM8YQIDo8gwQUg1AIoazuGvqEEDKu2HPlq4HQGL6LL9zk/8/qQElzBJaEEDYskT25I32P2OP0JEdCwJAE4/QkR0L8j9i+iy/c5PvP9iyRPbkjeY/E4/QkR0L0j8U17harBDrP8SO0JEdC8I/Y4/QkR0Lwj8AAAAAAAAAABOP0JEdC8I/AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABjj9CRHQvCPw==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[50]}},\"selected\":{\"id\":\"1451\"},\"selection_policy\":{\"id\":\"1450\"}},\"id\":\"1439\",\"type\":\"ColumnDataSource\"},{\"attributes\":{},\"id\":\"1408\",\"type\":\"Selection\"},{\"attributes\":{\"bottom\":{\"value\":0},\"fill_alpha\":{\"value\":0.1},\"fill_color\":{\"value\":\"#fbb4ae\"},\"left\":{\"field\":\"left\"},\"line_alpha\":{\"value\":0.1},\"line_color\":{\"value\":\"#fbb4ae\"},\"right\":{\"field\":\"right\"},\"top\":{\"field\":\"top\"}},\"id\":\"1441\",\"type\":\"Quad\"},{\"attributes\":{},\"id\":\"1389\",\"type\":\"UnionRenderers\"},{\"attributes\":{\"toolbars\":[{\"id\":\"1356\"},{\"id\":\"1434\"}],\"tools\":[{\"id\":\"1351\"},{\"id\":\"1352\"},{\"id\":\"1353\"},{\"id\":\"1354\"},{\"id\":\"1429\"},{\"id\":\"1430\"},{\"id\":\"1431\"},{\"id\":\"1432\"}]},\"id\":\"1517\",\"type\":\"ProxyToolbar\"},{\"attributes\":{\"label\":{\"value\":\"dist_b\"},\"renderers\":[{\"id\":\"1379\"}]},\"id\":\"1391\",\"type\":\"LegendItem\"},{\"attributes\":{\"source\":{\"id\":\"1439\"}},\"id\":\"1443\",\"type\":\"CDSView\"},{\"attributes\":{\"background_fill_color\":\"#fafafa\",\"below\":[{\"id\":\"1421\"}],\"center\":[{\"id\":\"1424\"},{\"id\":\"1428\"},{\"id\":\"1452\"}],\"left\":[{\"id\":\"1425\"}],\"plot_height\":400,\"plot_width\":300,\"renderers\":[{\"id\":\"1442\"},{\"id\":\"1457\"},{\"id\":\"1473\"}],\"title\":{\"id\":\"1411\"},\"toolbar\":{\"id\":\"1434\"},\"toolbar_location\":null,\"x_range\":{\"id\":\"1413\"},\"x_scale\":{\"id\":\"1417\"},\"y_range\":{\"id\":\"1415\"},\"y_scale\":{\"id\":\"1419\"}},\"id\":\"1410\",\"subtype\":\"Figure\",\"type\":\"Plot\"},{\"attributes\":{},\"id\":\"1390\",\"type\":\"Selection\"},{\"attributes\":{},\"id\":\"1422\",\"type\":\"BasicTicker\"},{\"attributes\":{},\"id\":\"1415\",\"type\":\"DataRange1d\"},{\"attributes\":{\"formatter\":{\"id\":\"1447\"},\"ticker\":{\"id\":\"1422\"}},\"id\":\"1421\",\"type\":\"LinearAxis\"},{\"attributes\":{},\"id\":\"1417\",\"type\":\"LinearScale\"},{\"attributes\":{},\"id\":\"1419\",\"type\":\"LinearScale\"},{\"attributes\":{\"axis\":{\"id\":\"1421\"},\"ticker\":null},\"id\":\"1424\",\"type\":\"Grid\"},{\"attributes\":{\"axis\":{\"id\":\"1425\"},\"dimension\":1,\"ticker\":null},\"id\":\"1428\",\"type\":\"Grid\"},{\"attributes\":{},\"id\":\"1450\",\"type\":\"UnionRenderers\"},{\"attributes\":{\"source\":{\"id\":\"1376\"}},\"id\":\"1380\",\"type\":\"CDSView\"},{\"attributes\":{},\"id\":\"1426\",\"type\":\"BasicTicker\"},{\"attributes\":{\"data_source\":{\"id\":\"1439\"},\"glyph\":{\"id\":\"1440\"},\"hover_glyph\":null,\"muted_glyph\":null,\"nonselection_glyph\":{\"id\":\"1441\"},\"selection_glyph\":null,\"view\":{\"id\":\"1443\"}},\"id\":\"1442\",\"type\":\"GlyphRenderer\"},{\"attributes\":{},\"id\":\"1451\",\"type\":\"Selection\"},{\"attributes\":{\"active_drag\":\"auto\",\"active_inspect\":\"auto\",\"active_multi\":null,\"active_scroll\":\"auto\",\"active_tap\":\"auto\",\"tools\":[{\"id\":\"1429\"},{\"id\":\"1430\"},{\"id\":\"1431\"},{\"id\":\"1432\"}]},\"id\":\"1434\",\"type\":\"Toolbar\"},{\"attributes\":{\"overlay\":{\"id\":\"1433\"}},\"id\":\"1430\",\"type\":\"BoxZoomTool\"},{\"attributes\":{\"children\":[[{\"id\":\"1332\"},0,0],[{\"id\":\"1410\"},0,1]]},\"id\":\"1516\",\"type\":\"GridBox\"},{\"attributes\":{},\"id\":\"1429\",\"type\":\"PanTool\"},{\"attributes\":{\"label\":{\"value\":\"dist_a\"},\"renderers\":[{\"id\":\"1442\"}]},\"id\":\"1453\",\"type\":\"LegendItem\"},{\"attributes\":{},\"id\":\"1431\",\"type\":\"ResetTool\"},{\"attributes\":{\"data_source\":{\"id\":\"1376\"},\"glyph\":{\"id\":\"1377\"},\"hover_glyph\":null,\"muted_glyph\":null,\"nonselection_glyph\":{\"id\":\"1378\"},\"selection_glyph\":null,\"view\":{\"id\":\"1380\"}},\"id\":\"1379\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"data\":{\"left\":{\"__ndarray__\":\"kqF77BfW1T8w/Qo5Sv7VP81YmoV8JtY/a7Qp0q5O1j8JELke4XbWP6ZrSGsTn9Y/RMfXt0XH1j/iImcEeO/WP39+9lCqF9c/HdqFndw/1z+6NRXqDmjXP1iRpDZBkNc/9uwzg3O41z+TSMPPpeDXPzGkUhzYCNg/z//haAox2D9sW3G1PFnYPwq3AAJvgdg/qBKQTqGp2D9Fbh+b09HYP+PJrucF+tg/gCU+NDgi2T8egc2AakrZP7zcXM2cctk/WTjsGc+a2T/3k3tmAcPZP5XvCrMz69k/Mkua/2UT2j/QpilMmDvaP24CuZjKY9o/C15I5fyL2j+pudcxL7TaP0YVZ35h3No/5HD2ypME2z+CzIUXxizbPyAoFWT4VNs/vYOksCp92z9b3zP9XKXbP/g6w0mPzds/lpZSlsH12z808uHi8x3cP9FNcS8mRtw/b6kAfFhu3D8MBZDIipbcP6pgHxW9vtw/SLyuYe/m3D/mFz6uIQ/dP4NzzfpTN90/Ic9cR4Zf3T++KuyTuIfdPw==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[50]},\"right\":{\"__ndarray__\":\"MP0KOUr+1T/NWJqFfCbWP2u0KdKuTtY/CRC5HuF21j+ma0hrE5/WP0TH17dFx9Y/4iJnBHjv1j9/fvZQqhfXPx3ahZ3cP9c/ujUV6g5o1z9YkaQ2QZDXP/bsM4NzuNc/k0jDz6Xg1z8xpFIc2AjYP8//4WgKMdg/bFtxtTxZ2D8KtwACb4HYP6gSkE6hqdg/RW4fm9PR2D/jya7nBfrYP4AlPjQ4Itk/HoHNgGpK2T+83FzNnHLZP1k47BnPmtk/95N7ZgHD2T+V7wqzM+vZPzJLmv9lE9o/0KYpTJg72j9uArmYymPaPwteSOX8i9o/qbnXMS+02j9GFWd+YdzaP+Rw9sqTBNs/gsyFF8Ys2z8gKBVk+FTbP72DpLAqfds/W98z/Vyl2z/4OsNJj83bP5aWUpbB9ds/NPLh4vMd3D/RTXEvJkbcP2+pAHxYbtw/DAWQyIqW3D+qYB8Vvb7cP0i8rmHv5tw/5hc+riEP3T+Dc836UzfdPyHPXEeGX90/virsk7iH3T9chnvg6q/dPw==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[50]},\"top\":{\"__ndarray__\":\"o5uUXc1NAEASYIdiFRbaP21fh2IVFto/kYflCZCQ8z8SYIdiFRbqPwAAAAAAAAAAkYflCZCQA0ALnJRdzU0AQJGH5QmQkPM/DBK9sy7vEUCbEb2zLu8RQHbpXgy0dBhAE9avuHa3G0BtX4diFRYaQKOblF3NTSBAEepeDLR0KECMwvm0QGEkQGLVr7h2tytAjPQyXtaGMUBnmpsNxuYqQIu5HrMltjBAGC9HCYdXMkB6rkphA6Q3QI8bmLVJmjVAjML5tEBhNEAYL0cJh1cyQIwvRwmHVzJAkYflCZCQM0CI/Q1g8TE1QBRM2A7YWC1Af3M2tlLTJkAUTNgO2FgtQG1fh2IVFipAZ5qbDcbmKkBZS9gO2FgdQBHqXgy0dBhAduleDLR0GEAMEr2zLu8hQFlL2A7YWA1AbV+HYhUWCkALnJRdzU0QQJGH5QmQkANAEmCHYhUW+j9tX4diFRbqP5GH5QmQkPM/kYflCZCQ8z8AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABtX4diFRbqPw==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[50]}},\"selected\":{\"id\":\"1468\"},\"selection_policy\":{\"id\":\"1467\"}},\"id\":\"1454\",\"type\":\"ColumnDataSource\"},{\"attributes\":{\"label\":{\"value\":\"dist_b\"},\"renderers\":[{\"id\":\"1457\"}]},\"id\":\"1469\",\"type\":\"LegendItem\"},{\"attributes\":{\"bottom\":{\"value\":0},\"fill_color\":{\"value\":\"#b3cde3\"},\"left\":{\"field\":\"left\"},\"line_color\":{\"value\":\"#b3cde3\"},\"right\":{\"field\":\"right\"},\"top\":{\"field\":\"top\"}},\"id\":\"1455\",\"type\":\"Quad\"},{\"attributes\":{\"bottom\":{\"value\":0},\"fill_color\":{\"value\":\"#ccebc5\"},\"left\":{\"field\":\"left\"},\"line_color\":{\"value\":\"#ccebc5\"},\"right\":{\"field\":\"right\"},\"top\":{\"field\":\"top\"}},\"id\":\"1471\",\"type\":\"Quad\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.1},\"fill_color\":{\"value\":\"#b3cde3\"},\"line_alpha\":{\"value\":0.1},\"line_color\":{\"value\":\"#b3cde3\"},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"1378\",\"type\":\"Circle\"},{\"attributes\":{\"bottom\":{\"value\":0},\"fill_alpha\":{\"value\":0.1},\"fill_color\":{\"value\":\"#b3cde3\"},\"left\":{\"field\":\"left\"},\"line_alpha\":{\"value\":0.1},\"line_color\":{\"value\":\"#b3cde3\"},\"right\":{\"field\":\"right\"},\"top\":{\"field\":\"top\"}},\"id\":\"1456\",\"type\":\"Quad\"},{\"attributes\":{\"data_source\":{\"id\":\"1454\"},\"glyph\":{\"id\":\"1455\"},\"hover_glyph\":null,\"muted_glyph\":null,\"nonselection_glyph\":{\"id\":\"1456\"},\"selection_glyph\":null,\"view\":{\"id\":\"1458\"}},\"id\":\"1457\",\"type\":\"GlyphRenderer\"}],\"root_ids\":[\"1519\"]},\"title\":\"Bokeh Application\",\"version\":\"2.2.1\"}};\n",
       "  var render_items = [{\"docid\":\"be42b67d-9497-4a6b-a75d-6e97633e57dd\",\"root_ids\":[\"1519\"],\"roots\":{\"1519\":\"2e5ae3f6-1df1-478c-9e44-72a2ae453797\"}}];\n",
       "  root.Bokeh.embed.embed_items_notebook(docs_json, render_items);\n",
       "\n",
       "  }\n",
       "  if (root.Bokeh !== undefined) {\n",
       "    embed_document(root);\n",
       "  } else {\n",
       "    var attempts = 0;\n",
       "    var timer = setInterval(function(root) {\n",
       "      if (root.Bokeh !== undefined) {\n",
       "        clearInterval(timer);\n",
       "        embed_document(root);\n",
       "      } else {\n",
       "        attempts++;\n",
       "        if (attempts > 100) {\n",
       "          clearInterval(timer);\n",
       "          console.log(\"Bokeh: ERROR: Unable to run BokehJS code because BokehJS library is missing\");\n",
       "        }\n",
       "      }\n",
       "    }, 10, root)\n",
       "  }\n",
       "})(window);"
      ],
      "application/vnd.bokehjs_exec.v0+json": ""
     },
     "metadata": {
      "application/vnd.bokehjs_exec.v0+json": {
       "id": "1519"
      }
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_data(dist_a, dist_b, dist_c, drifts)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Impact on predictive performance\n",
    "\n",
    "Concept drift can negatively impact learning methods if not properly handled. Multiple real-world applications suffer **model degradation** as the models can not adapt to changes in the data.\n",
    "\n",
    "In this example, we will use two popular streaming models:\n",
    "\n",
    "1. The `Hoeffding Tree`\n",
    "2. The `Hoeffding Adaptive Tree`\n",
    "\n",
    "The `Hoeffding Adaptive Tree` uses `ADWIN` to detect changes. If change is detected in a given branch, an alternate branch is created and eventually replaces the original branch if it shows better performance on new data.\n",
    "\n",
    "We will load the data from a csv file. The data was generated using the `AGRAWAL` data generator with 3 **gradual drifts** at the 5k, 10k, and 15k marks.\n",
    "\n",
    "---\n",
    "### `AGRAWAL` dataset\n",
    "\n",
    "Obtained using a synthetic data generator, contains 9 features, 6 numeric and 3 categorical.\n",
    "\n",
    "There are 10 functions for generating binary class labels from the features. These functions determine whether a **loan** should be approved.\n",
    "\n",
    "| Feature    | Description            | Values                                                                |\n",
    "|------------|------------------------|-----------------------------------------------------------------------|\n",
    "| `salary`     | salary                 | uniformly distributed from 20k to 150k                                |\n",
    "| `commission` | commission             | if (salary <   75k) then 0 else uniformly distributed from 10k to 75k |\n",
    "| `age`        | age                    | uniformly distributed from 20 to 80                                   |\n",
    "| `elevel`     | education level        | uniformly chosen from 0 to 4                                          |\n",
    "| `car`        | car maker              | uniformly chosen from 1 to 20                                         |\n",
    "| `zipcode`    | zip code of the town   | uniformly chosen from 0 to 8                                          |\n",
    "| `hvalue`     | value of the house     | uniformly distributed from 50k x zipcode to 100k x zipcode            |\n",
    "| `hyears`     | years house owned      | uniformly distributed from 1 to 30                                    |\n",
    "| `loan`       | total loan amount      | uniformly distributed from 0 to 500k                                  |\n",
    "\n",
    "**Class:** `y` | 0: no loan, 1: loan\n",
    " \n",
    "**Samples:** 20,000\n",
    "\n",
    "`elevel`, `car`, and `zipcode` are categorical features.\n",
    "\n",
    "---### Impact on predictive performance\n",
    "\n",
    "Concept drift can negatively impact learning methods if not properly handled. Multiple real-world applications suffer **model degradation** as the models can not adapt to changes in the data.\n",
    "\n",
    "In this example, we will use two popular streaming models:\n",
    "\n",
    "1. The `Hoeffding Tree`\n",
    "2. The `Hoeffding Adaptive Tree`\n",
    "\n",
    "The `Hoeffding Adaptive Tree` uses `ADWIN` to detect changes. If change is detected in a given branch, an alternate branch is created and eventually replaces the original branch if it shows better performance on new data.\n",
    "\n",
    "We will load the data from a csv file. The data was generated using the `AGRAWAL` data generator with 3 **gradual drifts** at the 5k, 10k, and 15k marks.\n",
    "\n",
    "---\n",
    "### `AGRAWAL` dataset\n",
    "\n",
    "Obtained using a synthetic data generator, contains 9 features, 6 numeric and 3 categorical.\n",
    "\n",
    "There are 10 functions for generating binary class labels from the features. These functions determine whether a **loan** should be approved.\n",
    "\n",
    "| Feature    | Description            | Values                                                                |\n",
    "|------------|------------------------|-----------------------------------------------------------------------|\n",
    "| `salary`     | salary                 | uniformly distributed from 20k to 150k                                |\n",
    "| `commission` | commission             | if (salary <   75k) then 0 else uniformly distributed from 10k to 75k |\n",
    "| `age`        | age                    | uniformly distributed from 20 to 80                                   |\n",
    "| `elevel`     | education level        | uniformly chosen from 0 to 4                                          |\n",
    "| `car`        | car maker              | uniformly chosen from 1 to 20                                         |\n",
    "| `zipcode`    | zip code of the town   | uniformly chosen from 0 to 8                                          |\n",
    "| `hvalue`     | value of the house     | uniformly distributed from 50k x zipcode to 100k x zipcode            |\n",
    "| `hyears`     | years house owned      | uniformly distributed from 1 to 30                                    |\n",
    "| `loan`       | total loan amount      | uniformly distributed from 0 to 500k                                  |\n",
    "\n",
    "**Class:** `y` | 0: no loan, 1: loan\n",
    " \n",
    "**Samples:** 20,000\n",
    "\n",
    "`elevel`, `car`, and `zipcode` are categorical features.\n",
    "\n",
    "---"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[1,000] Accuracy: 82.48%\n",
      "[2,000] Accuracy: 82.89%\n",
      "[3,000] Accuracy: 84.96%\n",
      "[4,000] Accuracy: 86.50%\n",
      "[5,000] Accuracy: 87.16%\n",
      "[6,000] Accuracy: 80.85%\n",
      "[7,000] Accuracy: 76.93%\n",
      "[8,000] Accuracy: 74.75%\n",
      "[9,000] Accuracy: 74.19%\n",
      "[10,000] Accuracy: 74.48%\n",
      "[11,000] Accuracy: 73.63%\n",
      "[12,000] Accuracy: 73.56%\n",
      "[13,000] Accuracy: 73.91%\n",
      "[14,000] Accuracy: 74.64%\n",
      "[15,000] Accuracy: 75.61%\n",
      "[16,000] Accuracy: 74.25%\n",
      "[17,000] Accuracy: 73.26%\n",
      "[18,000] Accuracy: 72.83%\n",
      "[19,000] Accuracy: 72.50%\n",
      "[20,000] Accuracy: 72.36%\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "Accuracy: 72.36%"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from river.tree import HoeffdingTreeClassifier\n",
    "from river.evaluate import progressive_val_score\n",
    "from river.stream import iter_pandas\n",
    "from river.metrics import Accuracy\n",
    "\n",
    "import pandas as pd\n",
    "\n",
    "data = pd.read_csv(\"agr_a_20k.csv\")\n",
    "features = data.columns[:-2]\n",
    "stream = iter_pandas(X=data[features], y=data['class'])\n",
    "model = HoeffdingTreeClassifier(nominal_attributes=['elevel', 'car', 'zipcode'])\n",
    "metric = Accuracy()\n",
    "\n",
    "progressive_val_score(dataset=stream,\n",
    "                      model=model,\n",
    "                      metric=metric,\n",
    "                      print_every=1000)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/svg+xml": [
       "<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?>\n",
       "<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
       " \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
       "<!-- Generated by graphviz version 2.44.1 (20200629.0846)\n",
       " -->\n",
       "<!-- Pages: 1 -->\n",
       "<svg width=\"818pt\" height=\"440pt\"\n",
       " viewBox=\"0.00 0.00 818.00 440.00\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n",
       "<g id=\"graph0\" class=\"graph\" transform=\"scale(1 1) rotate(0) translate(4 436)\">\n",
       "<polygon fill=\"white\" stroke=\"transparent\" points=\"-4,4 -4,-436 814,-436 814,4 -4,4\"/>\n",
       "<!-- 0 -->\n",
       "<g id=\"node1\" class=\"node\">\n",
       "<title>0</title>\n",
       "<polygon fill=\"#f6d5bd\" stroke=\"black\" stroke-width=\"1.2\" points=\"449,-432 395,-432 395,-396 449,-396 449,-432\"/>\n",
       "<text text-anchor=\"middle\" x=\"422\" y=\"-411.2\" font-family=\"trebuchet\" font-size=\"11.00\">age</text>\n",
       "</g>\n",
       "<!-- 1 -->\n",
       "<g id=\"node2\" class=\"node\">\n",
       "<title>1</title>\n",
       "<polygon fill=\"#fbede3\" stroke=\"black\" stroke-width=\"1.2\" points=\"355,-360 301,-360 301,-324 355,-324 355,-360\"/>\n",
       "<text text-anchor=\"middle\" x=\"328\" y=\"-339.2\" font-family=\"trebuchet\" font-size=\"11.00\">age</text>\n",
       "</g>\n",
       "<!-- 0&#45;&gt;1 -->\n",
       "<g id=\"edge1\" class=\"edge\">\n",
       "<title>0&#45;&gt;1</title>\n",
       "<path fill=\"none\" stroke=\"black\" stroke-width=\"0.6\" d=\"M404.5,-395.91C404.5,-372.87 404.5,-336 404.5,-336 404.5,-336 365.19,-336 365.19,-336\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" stroke-width=\"0.6\" points=\"365.19,-332.5 355.19,-336 365.19,-339.5 365.19,-332.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"393.5\" y=\"-348.7\" font-family=\"Times,serif\" font-size=\"7.00\">&lt;= 64.0</text>\n",
       "</g>\n",
       "<!-- 14 -->\n",
       "<g id=\"node15\" class=\"node\">\n",
       "<title>14</title>\n",
       "<polygon fill=\"#f0b58b\" stroke=\"black\" stroke-width=\"1.2\" points=\"558.5,-360 487.5,-360 487.5,-324 558.5,-324 558.5,-360\"/>\n",
       "<text text-anchor=\"middle\" x=\"523\" y=\"-339.2\" font-family=\"trebuchet\" font-size=\"11.00\">commission</text>\n",
       "</g>\n",
       "<!-- 0&#45;&gt;14 -->\n",
       "<g id=\"edge14\" class=\"edge\">\n",
       "<title>0&#45;&gt;14</title>\n",
       "<path fill=\"none\" stroke=\"black\" stroke-width=\"0.6\" d=\"M440.5,-395.97C440.5,-376.42 440.5,-348 440.5,-348 440.5,-348 477.05,-348 477.05,-348\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" stroke-width=\"0.6\" points=\"477.05,-351.5 487.05,-348 477.05,-344.5 477.05,-351.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"431\" y=\"-356.11\" font-family=\"Times,serif\" font-size=\"7.00\">&gt; 64.0</text>\n",
       "</g>\n",
       "<!-- 2 -->\n",
       "<g id=\"node3\" class=\"node\">\n",
       "<title>2</title>\n",
       "<polygon fill=\"#f2c29e\" stroke=\"black\" stroke-width=\"1.2\" points=\"228,-288 174,-288 174,-252 228,-252 228,-288\"/>\n",
       "<text text-anchor=\"middle\" x=\"201\" y=\"-267.2\" font-family=\"trebuchet\" font-size=\"11.00\">salary</text>\n",
       "</g>\n",
       "<!-- 1&#45;&gt;2 -->\n",
       "<g id=\"edge2\" class=\"edge\">\n",
       "<title>1&#45;&gt;2</title>\n",
       "<path fill=\"none\" stroke=\"black\" stroke-width=\"0.6\" d=\"M300.76,-342C263.44,-342 201,-342 201,-342 201,-342 201,-298.17 201,-298.17\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" stroke-width=\"0.6\" points=\"204.5,-298.17 201,-288.17 197.5,-298.17 204.5,-298.17\"/>\n",
       "<text text-anchor=\"middle\" x=\"217.96\" y=\"-344.4\" font-family=\"Times,serif\" font-size=\"7.00\">&lt;= 36.0</text>\n",
       "</g>\n",
       "<!-- 7 -->\n",
       "<g id=\"node8\" class=\"node\">\n",
       "<title>7</title>\n",
       "<polygon fill=\"#bcdef6\" stroke=\"black\" stroke-width=\"1.2\" points=\"363.5,-288 292.5,-288 292.5,-252 363.5,-252 363.5,-288\"/>\n",
       "<text text-anchor=\"middle\" x=\"328\" y=\"-267.2\" font-family=\"trebuchet\" font-size=\"11.00\">commission</text>\n",
       "</g>\n",
       "<!-- 1&#45;&gt;7 -->\n",
       "<g id=\"edge7\" class=\"edge\">\n",
       "<title>1&#45;&gt;7</title>\n",
       "<path fill=\"none\" stroke=\"black\" stroke-width=\"0.6\" d=\"M328,-323.83C328,-323.83 328,-298.41 328,-298.41\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" stroke-width=\"0.6\" points=\"331.5,-298.41 328,-288.41 324.5,-298.41 331.5,-298.41\"/>\n",
       "<text text-anchor=\"middle\" x=\"318.5\" y=\"-313.52\" font-family=\"Times,serif\" font-size=\"7.00\">&gt; 36.0</text>\n",
       "</g>\n",
       "<!-- 3 -->\n",
       "<g id=\"node4\" class=\"node\">\n",
       "<title>3</title>\n",
       "<polygon fill=\"#f0b68d\" stroke=\"black\" stroke-width=\"1.2\" points=\"144,-210 90,-210 90,-174 144,-174 144,-210\"/>\n",
       "<text text-anchor=\"middle\" x=\"117\" y=\"-189.2\" font-family=\"trebuchet\" font-size=\"11.00\">hyears</text>\n",
       "</g>\n",
       "<!-- 2&#45;&gt;3 -->\n",
       "<g id=\"edge3\" class=\"edge\">\n",
       "<title>2&#45;&gt;3</title>\n",
       "<path fill=\"none\" stroke=\"black\" stroke-width=\"0.6\" d=\"M173.96,-270C149.4,-270 117,-270 117,-270 117,-270 117,-220.09 117,-220.09\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" stroke-width=\"0.6\" points=\"120.5,-220.09 117,-210.09 113.5,-220.09 120.5,-220.09\"/>\n",
       "<text text-anchor=\"middle\" x=\"102.03\" y=\"-272.4\" font-family=\"Times,serif\" font-size=\"7.00\">&lt;= 140000.0</text>\n",
       "</g>\n",
       "<!-- 6 -->\n",
       "<g id=\"node7\" class=\"node\">\n",
       "<title>6</title>\n",
       "<polygon fill=\"#dcedfa\" stroke=\"black\" stroke-width=\"1.2\" points=\"240,-216 162,-216 162,-168 240,-168 240,-216\"/>\n",
       "<text text-anchor=\"middle\" x=\"201\" y=\"-207.2\" font-family=\"trebuchet\" font-size=\"11.00\">1</text>\n",
       "<text text-anchor=\"middle\" x=\"201\" y=\"-195.2\" font-family=\"trebuchet\" font-size=\"11.00\">P(0) = 0.41</text>\n",
       "<text text-anchor=\"middle\" x=\"201\" y=\"-183.2\" font-family=\"trebuchet\" font-size=\"11.00\">P(1) = 0.59</text>\n",
       "<text text-anchor=\"middle\" x=\"201\" y=\"-171.2\" font-family=\"trebuchet\" font-size=\"11.00\">samples: 416</text>\n",
       "</g>\n",
       "<!-- 2&#45;&gt;6 -->\n",
       "<g id=\"edge6\" class=\"edge\">\n",
       "<title>2&#45;&gt;6</title>\n",
       "<path fill=\"none\" stroke=\"black\" stroke-width=\"0.6\" d=\"M201,-251.91C201,-251.91 201,-226.2 201,-226.2\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" stroke-width=\"0.6\" points=\"204.5,-226.2 201,-216.2 197.5,-226.2 204.5,-226.2\"/>\n",
       "<text text-anchor=\"middle\" x=\"184\" y=\"-241.45\" font-family=\"Times,serif\" font-size=\"7.00\">&gt; 140000.0</text>\n",
       "</g>\n",
       "<!-- 4 -->\n",
       "<g id=\"node5\" class=\"node\">\n",
       "<title>4</title>\n",
       "<polygon fill=\"#fcefe6\" stroke=\"black\" stroke-width=\"1.2\" points=\"78,-132 0,-132 0,-84 78,-84 78,-132\"/>\n",
       "<text text-anchor=\"middle\" x=\"39\" y=\"-123.2\" font-family=\"trebuchet\" font-size=\"11.00\">0</text>\n",
       "<text text-anchor=\"middle\" x=\"39\" y=\"-111.2\" font-family=\"trebuchet\" font-size=\"11.00\">P(0) = 0.56</text>\n",
       "<text text-anchor=\"middle\" x=\"39\" y=\"-99.2\" font-family=\"trebuchet\" font-size=\"11.00\">P(1) = 0.44</text>\n",
       "<text text-anchor=\"middle\" x=\"39\" y=\"-87.2\" font-family=\"trebuchet\" font-size=\"11.00\">samples: 171</text>\n",
       "</g>\n",
       "<!-- 3&#45;&gt;4 -->\n",
       "<g id=\"edge4\" class=\"edge\">\n",
       "<title>3&#45;&gt;4</title>\n",
       "<path fill=\"none\" stroke=\"black\" stroke-width=\"0.6\" d=\"M89.72,-192C67.23,-192 39,-192 39,-192 39,-192 39,-142.3 39,-142.3\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" stroke-width=\"0.6\" points=\"42.5,-142.3 39,-132.3 35.5,-142.3 42.5,-142.3\"/>\n",
       "<text text-anchor=\"middle\" x=\"30.01\" y=\"-194.4\" font-family=\"Times,serif\" font-size=\"7.00\">&lt;= 3.6</text>\n",
       "</g>\n",
       "<!-- 5 -->\n",
       "<g id=\"node6\" class=\"node\">\n",
       "<title>5</title>\n",
       "<polygon fill=\"#fbede3\" stroke=\"black\" stroke-width=\"1.2\" points=\"180,-132 96,-132 96,-84 180,-84 180,-132\"/>\n",
       "<text text-anchor=\"middle\" x=\"138\" y=\"-123.2\" font-family=\"trebuchet\" font-size=\"11.00\">0</text>\n",
       "<text text-anchor=\"middle\" x=\"138\" y=\"-111.2\" font-family=\"trebuchet\" font-size=\"11.00\">P(0) = 0.57</text>\n",
       "<text text-anchor=\"middle\" x=\"138\" y=\"-99.2\" font-family=\"trebuchet\" font-size=\"11.00\">P(1) = 0.43</text>\n",
       "<text text-anchor=\"middle\" x=\"138\" y=\"-87.2\" font-family=\"trebuchet\" font-size=\"11.00\">samples: 1440</text>\n",
       "</g>\n",
       "<!-- 3&#45;&gt;5 -->\n",
       "<g id=\"edge5\" class=\"edge\">\n",
       "<title>3&#45;&gt;5</title>\n",
       "<path fill=\"none\" stroke=\"black\" stroke-width=\"0.6\" d=\"M120,-173.77C120,-173.77 120,-142.25 120,-142.25\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" stroke-width=\"0.6\" points=\"123.5,-142.25 120,-132.25 116.5,-142.25 123.5,-142.25\"/>\n",
       "<text text-anchor=\"middle\" x=\"112.5\" y=\"-160.41\" font-family=\"Times,serif\" font-size=\"7.00\">&gt; 3.6</text>\n",
       "</g>\n",
       "<!-- 8 -->\n",
       "<g id=\"node9\" class=\"node\">\n",
       "<title>8</title>\n",
       "<polygon fill=\"#cee7f9\" stroke=\"black\" stroke-width=\"1.2\" points=\"312,-210 258,-210 258,-174 312,-174 312,-210\"/>\n",
       "<text text-anchor=\"middle\" x=\"285\" y=\"-189.2\" font-family=\"trebuchet\" font-size=\"11.00\">salary</text>\n",
       "</g>\n",
       "<!-- 7&#45;&gt;8 -->\n",
       "<g id=\"edge8\" class=\"edge\">\n",
       "<title>7&#45;&gt;8</title>\n",
       "<path fill=\"none\" stroke=\"black\" stroke-width=\"0.6\" d=\"M302.25,-251.91C302.25,-251.91 302.25,-220.26 302.25,-220.26\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" stroke-width=\"0.6\" points=\"305.75,-220.26 302.25,-210.26 298.75,-220.26 305.75,-220.26\"/>\n",
       "<text text-anchor=\"middle\" x=\"285.25\" y=\"-238.49\" font-family=\"Times,serif\" font-size=\"7.00\">&lt;= 34000.0</text>\n",
       "</g>\n",
       "<!-- 13 -->\n",
       "<g id=\"node14\" class=\"node\">\n",
       "<title>13</title>\n",
       "<polygon fill=\"#a6d3f3\" stroke=\"black\" stroke-width=\"1.2\" points=\"414,-216 330,-216 330,-168 414,-168 414,-216\"/>\n",
       "<text text-anchor=\"middle\" x=\"372\" y=\"-207.2\" font-family=\"trebuchet\" font-size=\"11.00\">1</text>\n",
       "<text text-anchor=\"middle\" x=\"372\" y=\"-195.2\" font-family=\"trebuchet\" font-size=\"11.00\">P(1) = 0.72</text>\n",
       "<text text-anchor=\"middle\" x=\"372\" y=\"-183.2\" font-family=\"trebuchet\" font-size=\"11.00\">P(0) = 0.28</text>\n",
       "<text text-anchor=\"middle\" x=\"372\" y=\"-171.2\" font-family=\"trebuchet\" font-size=\"11.00\">samples: 2134</text>\n",
       "</g>\n",
       "<!-- 7&#45;&gt;13 -->\n",
       "<g id=\"edge13\" class=\"edge\">\n",
       "<title>7&#45;&gt;13</title>\n",
       "<path fill=\"none\" stroke=\"black\" stroke-width=\"0.6\" d=\"M346.75,-251.91C346.75,-251.91 346.75,-226.2 346.75,-226.2\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" stroke-width=\"0.6\" points=\"350.25,-226.2 346.75,-216.2 343.25,-226.2 350.25,-226.2\"/>\n",
       "<text text-anchor=\"middle\" x=\"331.75\" y=\"-241.45\" font-family=\"Times,serif\" font-size=\"7.00\">&gt; 34000.0</text>\n",
       "</g>\n",
       "<!-- 9 -->\n",
       "<g id=\"node10\" class=\"node\">\n",
       "<title>9</title>\n",
       "<polygon fill=\"#cfe7f9\" stroke=\"black\" stroke-width=\"1.2\" points=\"292,-126 238,-126 238,-90 292,-90 292,-126\"/>\n",
       "<text text-anchor=\"middle\" x=\"265\" y=\"-105.2\" font-family=\"trebuchet\" font-size=\"11.00\">salary</text>\n",
       "</g>\n",
       "<!-- 8&#45;&gt;9 -->\n",
       "<g id=\"edge9\" class=\"edge\">\n",
       "<title>8&#45;&gt;9</title>\n",
       "<path fill=\"none\" stroke=\"black\" stroke-width=\"0.6\" d=\"M275,-173.77C275,-173.77 275,-136.19 275,-136.19\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" stroke-width=\"0.6\" points=\"278.5,-136.19 275,-126.19 271.5,-136.19 278.5,-136.19\"/>\n",
       "<text text-anchor=\"middle\" x=\"256.5\" y=\"-157.38\" font-family=\"Times,serif\" font-size=\"7.00\">&lt;= 140000.0</text>\n",
       "</g>\n",
       "<!-- 12 -->\n",
       "<g id=\"node13\" class=\"node\">\n",
       "<title>12</title>\n",
       "<polygon fill=\"#62b1ea\" stroke=\"black\" stroke-width=\"1.2\" points=\"388,-132 310,-132 310,-84 388,-84 388,-132\"/>\n",
       "<text text-anchor=\"middle\" x=\"349\" y=\"-123.2\" font-family=\"trebuchet\" font-size=\"11.00\">1</text>\n",
       "<text text-anchor=\"middle\" x=\"349\" y=\"-111.2\" font-family=\"trebuchet\" font-size=\"11.00\">P(1) = 0.9</text>\n",
       "<text text-anchor=\"middle\" x=\"349\" y=\"-99.2\" font-family=\"trebuchet\" font-size=\"11.00\">P(0) = 0.1</text>\n",
       "<text text-anchor=\"middle\" x=\"349\" y=\"-87.2\" font-family=\"trebuchet\" font-size=\"11.00\">samples: 533</text>\n",
       "</g>\n",
       "<!-- 8&#45;&gt;12 -->\n",
       "<g id=\"edge12\" class=\"edge\">\n",
       "<title>8&#45;&gt;12</title>\n",
       "<path fill=\"none\" stroke=\"black\" stroke-width=\"0.6\" d=\"M312.29,-192C317.31,-192 321,-192 321,-192 321,-192 321,-142.3 321,-142.3\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" stroke-width=\"0.6\" points=\"324.5,-142.3 321,-132.3 317.5,-142.3 324.5,-142.3\"/>\n",
       "<text text-anchor=\"middle\" x=\"304\" y=\"-165.91\" font-family=\"Times,serif\" font-size=\"7.00\">&gt; 140000.0</text>\n",
       "</g>\n",
       "<!-- 10 -->\n",
       "<g id=\"node11\" class=\"node\">\n",
       "<title>10</title>\n",
       "<polygon fill=\"#d5eaf9\" stroke=\"black\" stroke-width=\"1.2\" points=\"257,-48 173,-48 173,0 257,0 257,-48\"/>\n",
       "<text text-anchor=\"middle\" x=\"215\" y=\"-39.2\" font-family=\"trebuchet\" font-size=\"11.00\">1</text>\n",
       "<text text-anchor=\"middle\" x=\"215\" y=\"-27.2\" font-family=\"trebuchet\" font-size=\"11.00\">P(1) = 0.61</text>\n",
       "<text text-anchor=\"middle\" x=\"215\" y=\"-15.2\" font-family=\"trebuchet\" font-size=\"11.00\">P(0) = 0.39</text>\n",
       "<text text-anchor=\"middle\" x=\"215\" y=\"-3.2\" font-family=\"trebuchet\" font-size=\"11.00\">samples: 2399</text>\n",
       "</g>\n",
       "<!-- 9&#45;&gt;10 -->\n",
       "<g id=\"edge10\" class=\"edge\">\n",
       "<title>9&#45;&gt;10</title>\n",
       "<path fill=\"none\" stroke=\"black\" stroke-width=\"0.6\" d=\"M247.5,-89.77C247.5,-89.77 247.5,-58.25 247.5,-58.25\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" stroke-width=\"0.6\" points=\"251,-58.25 247.5,-48.25 244,-58.25 251,-58.25\"/>\n",
       "<text text-anchor=\"middle\" x=\"229\" y=\"-76.41\" font-family=\"Times,serif\" font-size=\"7.00\">&lt;= 130000.0</text>\n",
       "</g>\n",
       "<!-- 11 -->\n",
       "<g id=\"node12\" class=\"node\">\n",
       "<title>11</title>\n",
       "<polygon fill=\"#78bced\" stroke=\"black\" stroke-width=\"1.2\" points=\"353,-48 275,-48 275,0 353,0 353,-48\"/>\n",
       "<text text-anchor=\"middle\" x=\"314\" y=\"-39.2\" font-family=\"trebuchet\" font-size=\"11.00\">1</text>\n",
       "<text text-anchor=\"middle\" x=\"314\" y=\"-27.2\" font-family=\"trebuchet\" font-size=\"11.00\">P(1) = 0.84</text>\n",
       "<text text-anchor=\"middle\" x=\"314\" y=\"-15.2\" font-family=\"trebuchet\" font-size=\"11.00\">P(0) = 0.16</text>\n",
       "<text text-anchor=\"middle\" x=\"314\" y=\"-3.2\" font-family=\"trebuchet\" font-size=\"11.00\">samples: 338</text>\n",
       "</g>\n",
       "<!-- 9&#45;&gt;11 -->\n",
       "<g id=\"edge11\" class=\"edge\">\n",
       "<title>9&#45;&gt;11</title>\n",
       "<path fill=\"none\" stroke=\"black\" stroke-width=\"0.6\" d=\"M283.5,-89.77C283.5,-89.77 283.5,-58.25 283.5,-58.25\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" stroke-width=\"0.6\" points=\"287,-58.25 283.5,-48.25 280,-58.25 287,-58.25\"/>\n",
       "<text text-anchor=\"middle\" x=\"266.5\" y=\"-76.41\" font-family=\"Times,serif\" font-size=\"7.00\">&gt; 130000.0</text>\n",
       "</g>\n",
       "<!-- 15 -->\n",
       "<g id=\"node16\" class=\"node\">\n",
       "<title>15</title>\n",
       "<polygon fill=\"#fdf5ef\" stroke=\"black\" stroke-width=\"1.2\" points=\"558.5,-288 487.5,-288 487.5,-252 558.5,-252 558.5,-288\"/>\n",
       "<text text-anchor=\"middle\" x=\"523\" y=\"-267.2\" font-family=\"trebuchet\" font-size=\"11.00\">commission</text>\n",
       "</g>\n",
       "<!-- 14&#45;&gt;15 -->\n",
       "<g id=\"edge15\" class=\"edge\">\n",
       "<title>14&#45;&gt;15</title>\n",
       "<path fill=\"none\" stroke=\"black\" stroke-width=\"0.6\" d=\"M523,-323.83C523,-323.83 523,-298.41 523,-298.41\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" stroke-width=\"0.6\" points=\"526.5,-298.41 523,-288.41 519.5,-298.41 526.5,-298.41\"/>\n",
       "<text text-anchor=\"middle\" x=\"506\" y=\"-313.52\" font-family=\"Times,serif\" font-size=\"7.00\">&lt;= 20000.0</text>\n",
       "</g>\n",
       "<!-- 18 -->\n",
       "<g id=\"node19\" class=\"node\">\n",
       "<title>18</title>\n",
       "<polygon fill=\"#e78c4a\" stroke=\"black\" stroke-width=\"1.2\" points=\"696,-288 642,-288 642,-252 696,-252 696,-288\"/>\n",
       "<text text-anchor=\"middle\" x=\"669\" y=\"-267.2\" font-family=\"trebuchet\" font-size=\"11.00\">salary</text>\n",
       "</g>\n",
       "<!-- 14&#45;&gt;18 -->\n",
       "<g id=\"edge18\" class=\"edge\">\n",
       "<title>14&#45;&gt;18</title>\n",
       "<path fill=\"none\" stroke=\"black\" stroke-width=\"0.6\" d=\"M558.72,-342C601.94,-342 669,-342 669,-342 669,-342 669,-298.17 669,-298.17\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" stroke-width=\"0.6\" points=\"672.5,-298.17 669,-288.17 665.5,-298.17 672.5,-298.17\"/>\n",
       "<text text-anchor=\"middle\" x=\"620.77\" y=\"-344.4\" font-family=\"Times,serif\" font-size=\"7.00\">&gt; 20000.0</text>\n",
       "</g>\n",
       "<!-- 16 -->\n",
       "<g id=\"node17\" class=\"node\">\n",
       "<title>16</title>\n",
       "<polygon fill=\"#f5fafe\" stroke=\"black\" stroke-width=\"1.2\" points=\"516,-216 432,-216 432,-168 516,-168 516,-216\"/>\n",
       "<text text-anchor=\"middle\" x=\"474\" y=\"-207.2\" font-family=\"trebuchet\" font-size=\"11.00\">1</text>\n",
       "<text text-anchor=\"middle\" x=\"474\" y=\"-195.2\" font-family=\"trebuchet\" font-size=\"11.00\">P(1) = 0.52</text>\n",
       "<text text-anchor=\"middle\" x=\"474\" y=\"-183.2\" font-family=\"trebuchet\" font-size=\"11.00\">P(0) = 0.48</text>\n",
       "<text text-anchor=\"middle\" x=\"474\" y=\"-171.2\" font-family=\"trebuchet\" font-size=\"11.00\">samples: 2004</text>\n",
       "</g>\n",
       "<!-- 15&#45;&gt;16 -->\n",
       "<g id=\"edge16\" class=\"edge\">\n",
       "<title>15&#45;&gt;16</title>\n",
       "<path fill=\"none\" stroke=\"black\" stroke-width=\"0.6\" d=\"M501.75,-251.91C501.75,-251.91 501.75,-226.2 501.75,-226.2\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" stroke-width=\"0.6\" points=\"505.25,-226.2 501.75,-216.2 498.25,-226.2 505.25,-226.2\"/>\n",
       "<text text-anchor=\"middle\" x=\"486.75\" y=\"-241.45\" font-family=\"Times,serif\" font-size=\"7.00\">&lt;= 9300.0</text>\n",
       "</g>\n",
       "<!-- 17 -->\n",
       "<g id=\"node18\" class=\"node\">\n",
       "<title>17</title>\n",
       "<polygon fill=\"#eca16b\" stroke=\"black\" stroke-width=\"1.2\" points=\"612,-216 534,-216 534,-168 612,-168 612,-216\"/>\n",
       "<text text-anchor=\"middle\" x=\"573\" y=\"-207.2\" font-family=\"trebuchet\" font-size=\"11.00\">0</text>\n",
       "<text text-anchor=\"middle\" x=\"573\" y=\"-195.2\" font-family=\"trebuchet\" font-size=\"11.00\">P(1) = 0.13</text>\n",
       "<text text-anchor=\"middle\" x=\"573\" y=\"-183.2\" font-family=\"trebuchet\" font-size=\"11.00\">P(0) = 0.87</text>\n",
       "<text text-anchor=\"middle\" x=\"573\" y=\"-171.2\" font-family=\"trebuchet\" font-size=\"11.00\">samples: 224</text>\n",
       "</g>\n",
       "<!-- 15&#45;&gt;17 -->\n",
       "<g id=\"edge17\" class=\"edge\">\n",
       "<title>15&#45;&gt;17</title>\n",
       "<path fill=\"none\" stroke=\"black\" stroke-width=\"0.6\" d=\"M546.25,-251.91C546.25,-251.91 546.25,-226.2 546.25,-226.2\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" stroke-width=\"0.6\" points=\"549.75,-226.2 546.25,-216.2 542.75,-226.2 549.75,-226.2\"/>\n",
       "<text text-anchor=\"middle\" x=\"533.25\" y=\"-241.45\" font-family=\"Times,serif\" font-size=\"7.00\">&gt; 9300.0</text>\n",
       "</g>\n",
       "<!-- 19 -->\n",
       "<g id=\"node20\" class=\"node\">\n",
       "<title>19</title>\n",
       "<polygon fill=\"#fcf0e8\" stroke=\"black\" stroke-width=\"1.2\" points=\"708,-216 630,-216 630,-168 708,-168 708,-216\"/>\n",
       "<text text-anchor=\"middle\" x=\"669\" y=\"-207.2\" font-family=\"trebuchet\" font-size=\"11.00\">0</text>\n",
       "<text text-anchor=\"middle\" x=\"669\" y=\"-195.2\" font-family=\"trebuchet\" font-size=\"11.00\">P(0) = 0.56</text>\n",
       "<text text-anchor=\"middle\" x=\"669\" y=\"-183.2\" font-family=\"trebuchet\" font-size=\"11.00\">P(1) = 0.44</text>\n",
       "<text text-anchor=\"middle\" x=\"669\" y=\"-171.2\" font-family=\"trebuchet\" font-size=\"11.00\">samples: 222</text>\n",
       "</g>\n",
       "<!-- 18&#45;&gt;19 -->\n",
       "<g id=\"edge19\" class=\"edge\">\n",
       "<title>18&#45;&gt;19</title>\n",
       "<path fill=\"none\" stroke=\"black\" stroke-width=\"0.6\" d=\"M669,-251.91C669,-251.91 669,-226.2 669,-226.2\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" stroke-width=\"0.6\" points=\"672.5,-226.2 669,-216.2 665.5,-226.2 672.5,-226.2\"/>\n",
       "<text text-anchor=\"middle\" x=\"652\" y=\"-241.45\" font-family=\"Times,serif\" font-size=\"7.00\">&lt;= 31000.0</text>\n",
       "</g>\n",
       "<!-- 20 -->\n",
       "<g id=\"node21\" class=\"node\">\n",
       "<title>20</title>\n",
       "<polygon fill=\"#eb9d65\" stroke=\"black\" stroke-width=\"1.2\" points=\"810,-216 726,-216 726,-168 810,-168 810,-216\"/>\n",
       "<text text-anchor=\"middle\" x=\"768\" y=\"-207.2\" font-family=\"trebuchet\" font-size=\"11.00\">0</text>\n",
       "<text text-anchor=\"middle\" x=\"768\" y=\"-195.2\" font-family=\"trebuchet\" font-size=\"11.00\">P(0) = 0.89</text>\n",
       "<text text-anchor=\"middle\" x=\"768\" y=\"-183.2\" font-family=\"trebuchet\" font-size=\"11.00\">P(1) = 0.11</text>\n",
       "<text text-anchor=\"middle\" x=\"768\" y=\"-171.2\" font-family=\"trebuchet\" font-size=\"11.00\">samples: 1029</text>\n",
       "</g>\n",
       "<!-- 18&#45;&gt;20 -->\n",
       "<g id=\"edge20\" class=\"edge\">\n",
       "<title>18&#45;&gt;20</title>\n",
       "<path fill=\"none\" stroke=\"black\" stroke-width=\"0.6\" d=\"M696.05,-270C725.36,-270 768,-270 768,-270 768,-270 768,-226.25 768,-226.25\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" stroke-width=\"0.6\" points=\"771.5,-226.25 768,-216.25 764.5,-226.25 771.5,-226.25\"/>\n",
       "<text text-anchor=\"middle\" x=\"738.9\" y=\"-272.4\" font-family=\"Times,serif\" font-size=\"7.00\">&gt; 31000.0</text>\n",
       "</g>\n",
       "</g>\n",
       "</svg>\n"
      ],
      "text/plain": [
       "<graphviz.dot.Digraph at 0x7fcf563382e0>"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.draw()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[1,000] Accuracy: 83.78%\n",
      "[2,000] Accuracy: 87.54%\n",
      "[3,000] Accuracy: 88.83%\n",
      "[4,000] Accuracy: 90.15%\n",
      "[5,000] Accuracy: 90.62%\n",
      "[6,000] Accuracy: 83.83%\n",
      "[7,000] Accuracy: 80.34%\n",
      "[8,000] Accuracy: 78.50%\n",
      "[9,000] Accuracy: 76.99%\n",
      "[10,000] Accuracy: 75.40%\n",
      "[11,000] Accuracy: 73.44%\n",
      "[12,000] Accuracy: 74.61%\n",
      "[13,000] Accuracy: 75.77%\n",
      "[14,000] Accuracy: 76.93%\n",
      "[15,000] Accuracy: 77.99%\n",
      "[16,000] Accuracy: 76.38%\n",
      "[17,000] Accuracy: 76.36%\n",
      "[18,000] Accuracy: 76.64%\n",
      "[19,000] Accuracy: 76.82%\n",
      "[20,000] Accuracy: 76.94%\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "Accuracy: 76.94%"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from river.tree import HoeffdingAdaptiveTreeClassifier\n",
    "from river.stream import iter_pandas\n",
    "from river.metrics import Accuracy\n",
    "\n",
    "import pandas as pd\n",
    "\n",
    "data = pd.read_csv(\"agr_a_20k.csv\")\n",
    "features = data.columns[:-2]\n",
    "stream = iter_pandas(X=data[features], y=data['class'])\n",
    "model = HoeffdingAdaptiveTreeClassifier(nominal_attributes=['elevel', 'car', 'zipcode'], seed=42)\n",
    "metric = Accuracy()\n",
    "\n",
    "progressive_val_score(dataset=stream, model=model, metric=metric, print_every=1000)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/svg+xml": [
       "<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?>\n",
       "<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
       " \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
       "<!-- Generated by graphviz version 2.44.1 (20200629.0846)\n",
       " -->\n",
       "<!-- Pages: 1 -->\n",
       "<svg width=\"470pt\" height=\"296pt\"\n",
       " viewBox=\"0.00 0.00 470.00 296.00\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n",
       "<g id=\"graph0\" class=\"graph\" transform=\"scale(1 1) rotate(0) translate(4 292)\">\n",
       "<polygon fill=\"white\" stroke=\"transparent\" points=\"-4,4 -4,-292 466,-292 466,4 -4,4\"/>\n",
       "<!-- 0 -->\n",
       "<g id=\"node1\" class=\"node\">\n",
       "<title>0</title>\n",
       "<polygon fill=\"#f9e1d0\" stroke=\"black\" stroke-width=\"1.2\" points=\"344,-288 290,-288 290,-252 344,-252 344,-288\"/>\n",
       "<text text-anchor=\"middle\" x=\"317\" y=\"-267.2\" font-family=\"trebuchet\" font-size=\"11.00\">salary</text>\n",
       "</g>\n",
       "<!-- 1 -->\n",
       "<g id=\"node2\" class=\"node\">\n",
       "<title>1</title>\n",
       "<polygon fill=\"#eaf5fc\" stroke=\"black\" stroke-width=\"1.2\" points=\"301,-210 247,-210 247,-174 301,-174 301,-210\"/>\n",
       "<text text-anchor=\"middle\" x=\"274\" y=\"-189.2\" font-family=\"trebuchet\" font-size=\"11.00\">salary</text>\n",
       "</g>\n",
       "<!-- 0&#45;&gt;1 -->\n",
       "<g id=\"edge1\" class=\"edge\">\n",
       "<title>0&#45;&gt;1</title>\n",
       "<path fill=\"none\" stroke=\"black\" stroke-width=\"0.6\" d=\"M295.5,-251.91C295.5,-251.91 295.5,-220.26 295.5,-220.26\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" stroke-width=\"0.6\" points=\"299,-220.26 295.5,-210.26 292,-220.26 299,-220.26\"/>\n",
       "<text text-anchor=\"middle\" x=\"277\" y=\"-238.49\" font-family=\"Times,serif\" font-size=\"7.00\">&lt;= 100000.0</text>\n",
       "</g>\n",
       "<!-- 9 -->\n",
       "<g id=\"node10\" class=\"node\">\n",
       "<title>9</title>\n",
       "<polygon fill=\"#e99355\" stroke=\"black\" stroke-width=\"1.2\" points=\"403,-216 319,-216 319,-168 403,-168 403,-216\"/>\n",
       "<text text-anchor=\"middle\" x=\"361\" y=\"-207.2\" font-family=\"trebuchet\" font-size=\"11.00\">1</text>\n",
       "<text text-anchor=\"middle\" x=\"361\" y=\"-195.2\" font-family=\"trebuchet\" font-size=\"11.00\">P(1) = 0.93</text>\n",
       "<text text-anchor=\"middle\" x=\"361\" y=\"-183.2\" font-family=\"trebuchet\" font-size=\"11.00\">P(0) = 0.071</text>\n",
       "<text text-anchor=\"middle\" x=\"361\" y=\"-171.2\" font-family=\"trebuchet\" font-size=\"11.00\">samples: 2229</text>\n",
       "</g>\n",
       "<!-- 0&#45;&gt;9 -->\n",
       "<g id=\"edge9\" class=\"edge\">\n",
       "<title>0&#45;&gt;9</title>\n",
       "<path fill=\"none\" stroke=\"black\" stroke-width=\"0.6\" d=\"M331.5,-251.91C331.5,-251.91 331.5,-226.2 331.5,-226.2\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" stroke-width=\"0.6\" points=\"335,-226.2 331.5,-216.2 328,-226.2 335,-226.2\"/>\n",
       "<text text-anchor=\"middle\" x=\"314.5\" y=\"-241.45\" font-family=\"Times,serif\" font-size=\"7.00\">&gt; 100000.0</text>\n",
       "</g>\n",
       "<!-- 2 -->\n",
       "<g id=\"node3\" class=\"node\">\n",
       "<title>2</title>\n",
       "<polygon fill=\"#f0b68c\" stroke=\"black\" stroke-width=\"1.2\" points=\"258,-126 204,-126 204,-90 258,-90 258,-126\"/>\n",
       "<text text-anchor=\"middle\" x=\"231\" y=\"-105.2\" font-family=\"trebuchet\" font-size=\"11.00\">elevel</text>\n",
       "</g>\n",
       "<!-- 1&#45;&gt;2 -->\n",
       "<g id=\"edge2\" class=\"edge\">\n",
       "<title>1&#45;&gt;2</title>\n",
       "<path fill=\"none\" stroke=\"black\" stroke-width=\"0.6\" d=\"M252.5,-173.77C252.5,-173.77 252.5,-136.19 252.5,-136.19\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" stroke-width=\"0.6\" points=\"256,-136.19 252.5,-126.19 249,-136.19 256,-136.19\"/>\n",
       "<text text-anchor=\"middle\" x=\"235.5\" y=\"-157.38\" font-family=\"Times,serif\" font-size=\"7.00\">&lt;= 42000.0</text>\n",
       "</g>\n",
       "<!-- 8 -->\n",
       "<g id=\"node9\" class=\"node\">\n",
       "<title>8</title>\n",
       "<polygon fill=\"#bbddf6\" stroke=\"black\" stroke-width=\"1.2\" points=\"360,-132 276,-132 276,-84 360,-84 360,-132\"/>\n",
       "<text text-anchor=\"middle\" x=\"318\" y=\"-123.2\" font-family=\"trebuchet\" font-size=\"11.00\">0</text>\n",
       "<text text-anchor=\"middle\" x=\"318\" y=\"-111.2\" font-family=\"trebuchet\" font-size=\"11.00\">P(0) = 0.67</text>\n",
       "<text text-anchor=\"middle\" x=\"318\" y=\"-99.2\" font-family=\"trebuchet\" font-size=\"11.00\">P(1) = 0.33</text>\n",
       "<text text-anchor=\"middle\" x=\"318\" y=\"-87.2\" font-family=\"trebuchet\" font-size=\"11.00\">samples: 2551</text>\n",
       "</g>\n",
       "<!-- 1&#45;&gt;8 -->\n",
       "<g id=\"edge8\" class=\"edge\">\n",
       "<title>1&#45;&gt;8</title>\n",
       "<path fill=\"none\" stroke=\"black\" stroke-width=\"0.6\" d=\"M288.5,-173.77C288.5,-173.77 288.5,-142.25 288.5,-142.25\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" stroke-width=\"0.6\" points=\"292,-142.25 288.5,-132.25 285,-142.25 292,-142.25\"/>\n",
       "<text text-anchor=\"middle\" x=\"273.5\" y=\"-160.41\" font-family=\"Times,serif\" font-size=\"7.00\">&gt; 42000.0</text>\n",
       "</g>\n",
       "<!-- 3 -->\n",
       "<g id=\"node4\" class=\"node\">\n",
       "<title>3</title>\n",
       "<polygon fill=\"#f0f8fd\" stroke=\"black\" stroke-width=\"1.2\" points=\"78,-48 0,-48 0,0 78,0 78,-48\"/>\n",
       "<text text-anchor=\"middle\" x=\"39\" y=\"-39.2\" font-family=\"trebuchet\" font-size=\"11.00\">0</text>\n",
       "<text text-anchor=\"middle\" x=\"39\" y=\"-27.2\" font-family=\"trebuchet\" font-size=\"11.00\">P(0) = 0.54</text>\n",
       "<text text-anchor=\"middle\" x=\"39\" y=\"-15.2\" font-family=\"trebuchet\" font-size=\"11.00\">P(1) = 0.46</text>\n",
       "<text text-anchor=\"middle\" x=\"39\" y=\"-3.2\" font-family=\"trebuchet\" font-size=\"11.00\">samples: 145</text>\n",
       "</g>\n",
       "<!-- 2&#45;&gt;3 -->\n",
       "<g id=\"edge3\" class=\"edge\">\n",
       "<title>2&#45;&gt;3</title>\n",
       "<path fill=\"none\" stroke=\"black\" stroke-width=\"0.6\" d=\"M203.9,-114C150.97,-114 39,-114 39,-114 39,-114 39,-58.12 39,-58.12\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" stroke-width=\"0.6\" points=\"42.5,-58.12 39,-48.12 35.5,-58.12 42.5,-58.12\"/>\n",
       "<text text-anchor=\"middle\" x=\"85.51\" y=\"-116.4\" font-family=\"Times,serif\" font-size=\"7.00\"> = 0.0</text>\n",
       "</g>\n",
       "<!-- 4 -->\n",
       "<g id=\"node5\" class=\"node\">\n",
       "<title>4</title>\n",
       "<polygon fill=\"#fdf7f2\" stroke=\"black\" stroke-width=\"1.2\" points=\"174,-48 96,-48 96,0 174,0 174,-48\"/>\n",
       "<text text-anchor=\"middle\" x=\"135\" y=\"-39.2\" font-family=\"trebuchet\" font-size=\"11.00\">1</text>\n",
       "<text text-anchor=\"middle\" x=\"135\" y=\"-27.2\" font-family=\"trebuchet\" font-size=\"11.00\">P(0) = 0.47</text>\n",
       "<text text-anchor=\"middle\" x=\"135\" y=\"-15.2\" font-family=\"trebuchet\" font-size=\"11.00\">P(1) = 0.53</text>\n",
       "<text text-anchor=\"middle\" x=\"135\" y=\"-3.2\" font-family=\"trebuchet\" font-size=\"11.00\">samples: 109</text>\n",
       "</g>\n",
       "<!-- 2&#45;&gt;4 -->\n",
       "<g id=\"edge4\" class=\"edge\">\n",
       "<title>2&#45;&gt;4</title>\n",
       "<path fill=\"none\" stroke=\"black\" stroke-width=\"0.6\" d=\"M203.75,-102C175.37,-102 135,-102 135,-102 135,-102 135,-58.25 135,-58.25\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" stroke-width=\"0.6\" points=\"138.5,-58.25 135,-48.25 131.5,-58.25 138.5,-58.25\"/>\n",
       "<text text-anchor=\"middle\" x=\"139.5\" y=\"-104.4\" font-family=\"Times,serif\" font-size=\"7.00\"> = 1.0</text>\n",
       "</g>\n",
       "<!-- 5 -->\n",
       "<g id=\"node6\" class=\"node\">\n",
       "<title>5</title>\n",
       "<polygon fill=\"#e58139\" stroke=\"black\" stroke-width=\"1.2\" points=\"270,-42 192,-42 192,-6 270,-6 270,-42\"/>\n",
       "<text text-anchor=\"middle\" x=\"231\" y=\"-33.2\" font-family=\"trebuchet\" font-size=\"11.00\">1</text>\n",
       "<text text-anchor=\"middle\" x=\"231\" y=\"-21.2\" font-family=\"trebuchet\" font-size=\"11.00\">P(1) = 1.0</text>\n",
       "<text text-anchor=\"middle\" x=\"231\" y=\"-9.2\" font-family=\"trebuchet\" font-size=\"11.00\">samples: 120</text>\n",
       "</g>\n",
       "<!-- 2&#45;&gt;5 -->\n",
       "<g id=\"edge5\" class=\"edge\">\n",
       "<title>2&#45;&gt;5</title>\n",
       "<path fill=\"none\" stroke=\"black\" stroke-width=\"0.6\" d=\"M217.5,-89.77C217.5,-89.77 217.5,-52.19 217.5,-52.19\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" stroke-width=\"0.6\" points=\"221,-52.19 217.5,-42.19 214,-52.19 221,-52.19\"/>\n",
       "<text text-anchor=\"middle\" x=\"209.5\" y=\"-73.38\" font-family=\"Times,serif\" font-size=\"7.00\"> = 2.0</text>\n",
       "</g>\n",
       "<!-- 6 -->\n",
       "<g id=\"node7\" class=\"node\">\n",
       "<title>6</title>\n",
       "<polygon fill=\"#e58139\" stroke=\"black\" stroke-width=\"1.2\" points=\"366,-42 288,-42 288,-6 366,-6 366,-42\"/>\n",
       "<text text-anchor=\"middle\" x=\"327\" y=\"-33.2\" font-family=\"trebuchet\" font-size=\"11.00\">1</text>\n",
       "<text text-anchor=\"middle\" x=\"327\" y=\"-21.2\" font-family=\"trebuchet\" font-size=\"11.00\">P(1) = 1.0</text>\n",
       "<text text-anchor=\"middle\" x=\"327\" y=\"-9.2\" font-family=\"trebuchet\" font-size=\"11.00\">samples: 104</text>\n",
       "</g>\n",
       "<!-- 2&#45;&gt;6 -->\n",
       "<g id=\"edge6\" class=\"edge\">\n",
       "<title>2&#45;&gt;6</title>\n",
       "<path fill=\"none\" stroke=\"black\" stroke-width=\"0.6\" d=\"M231,-89.85C231,-76.26 231,-60 231,-60 231,-60 324,-60 324,-60 324,-60 324,-52.42 324,-52.42\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" stroke-width=\"0.6\" points=\"327.5,-52.42 324,-42.42 320.5,-52.42 327.5,-52.42\"/>\n",
       "<text text-anchor=\"middle\" x=\"258.36\" y=\"-62.4\" font-family=\"Times,serif\" font-size=\"7.00\"> = 3.0</text>\n",
       "</g>\n",
       "<!-- 7 -->\n",
       "<g id=\"node8\" class=\"node\">\n",
       "<title>7</title>\n",
       "<polygon fill=\"#e58139\" stroke=\"black\" stroke-width=\"1.2\" points=\"462,-42 384,-42 384,-6 462,-6 462,-42\"/>\n",
       "<text text-anchor=\"middle\" x=\"423\" y=\"-33.2\" font-family=\"trebuchet\" font-size=\"11.00\">1</text>\n",
       "<text text-anchor=\"middle\" x=\"423\" y=\"-21.2\" font-family=\"trebuchet\" font-size=\"11.00\">P(1) = 1.0</text>\n",
       "<text text-anchor=\"middle\" x=\"423\" y=\"-9.2\" font-family=\"trebuchet\" font-size=\"11.00\">samples: 122</text>\n",
       "</g>\n",
       "<!-- 2&#45;&gt;7 -->\n",
       "<g id=\"edge7\" class=\"edge\">\n",
       "<title>2&#45;&gt;7</title>\n",
       "<path fill=\"none\" stroke=\"black\" stroke-width=\"0.6\" d=\"M244.5,-89.58C244.5,-80.72 244.5,-72 244.5,-72 244.5,-72 393.5,-72 393.5,-72 393.5,-72 393.5,-52.15 393.5,-52.15\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" stroke-width=\"0.6\" points=\"397,-52.15 393.5,-42.15 390,-52.15 397,-52.15\"/>\n",
       "<text text-anchor=\"middle\" x=\"312.14\" y=\"-74.4\" font-family=\"Times,serif\" font-size=\"7.00\"> = 4.0</text>\n",
       "</g>\n",
       "</g>\n",
       "</svg>\n"
      ],
      "text/plain": [
       "<graphviz.dot.Digraph at 0x7fcf56338940>"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.draw()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "# PART II. Extending functionality\n",
    "\n",
    "## Implement a simple estimator\n",
    "\n",
    "\n",
    "A core design element in `river` is that it should be easy to create new methods or extend existing methods.\n",
    "\n",
    "In the following example, we show how to implement the `MajorityClassClassifier`.\n",
    "\n",
    "The Majority Class is one of the simplest classifiers: it predicts the class of a new sample to be the most frequent at that point in the stream.\n",
    "\n",
    "It is used mostly as a baseline, but also as a default classifier at the leaves of decision trees.\n",
    "\n",
    "`River` provides a set of `mixins` for different learning tasks. In this case, we want to create a classifier so we extend the `base.Classifier` mixin as follows: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [],
   "source": [
    "from river import base\n",
    "from collections import Counter\n",
    "\n",
    "class MajorityClassClassifier(base.Classifier):\n",
    "    def __init__(self):\n",
    "        # Initialization\n",
    "        self._counts = Counter()\n",
    "    \n",
    "    def learn_one(self, x:dict, y):\n",
    "        # Learn one sample\n",
    "        self._counts.update([y])\n",
    "\n",
    "    def predict_one(self, x:dict):\n",
    "        # Predict class for one sample\n",
    "        mc = self._counts.most_common()\n",
    "        if mc:\n",
    "            return mc[0][0]\n",
    "        return 0   # Counter is empty\n",
    "\n",
    "    def predict_proba_one(self, x:dict):\n",
    "        # Predict class probability for one sample\n",
    "        total = sum(self._counts)\n",
    "        y_proba = {}\n",
    "        if total > 0:    # Protect division by zero\n",
    "            for x, cnt in self._counts.items():\n",
    "                y_proba[x] = cnt / total\n",
    "        return y_proba\n",
    "    \n",
    "    @property\n",
    "    def _multiclass(self):\n",
    "        return True"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we can test our implementation. This time we will use data from the `SEA` generator which corresponds to a binary classification problem.\n",
    "\n",
    "Synthetic data generators are useful since they do not store the data but generate it on demand. Although data generators are infinite, for this example, we limit the number of samples generated using the `take()` method."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[1,000] Accuracy: 69.70%\n",
      "[2,000] Accuracy: 69.30%\n",
      "[3,000] Accuracy: 69.23%\n",
      "[4,000] Accuracy: 69.17%\n",
      "[5,000] Accuracy: 69.12%\n",
      "[6,000] Accuracy: 68.80%\n",
      "[7,000] Accuracy: 68.61%\n",
      "[8,000] Accuracy: 68.45%\n",
      "[9,000] Accuracy: 68.47%\n",
      "[10,000] Accuracy: 68.34%\n",
      "[11,000] Accuracy: 68.37%\n",
      "[12,000] Accuracy: 68.30%\n",
      "[13,000] Accuracy: 68.32%\n",
      "[14,000] Accuracy: 68.32%\n",
      "[15,000] Accuracy: 68.27%\n",
      "[16,000] Accuracy: 68.34%\n",
      "[17,000] Accuracy: 68.34%\n",
      "[18,000] Accuracy: 68.21%\n",
      "[19,000] Accuracy: 68.18%\n",
      "[20,000] Accuracy: 68.20%\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "Accuracy: 68.20%"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from river import synth\n",
    "from river import neighbors\n",
    "\n",
    "stream = synth.SEA(seed=42).take(20000)\n",
    "\n",
    "model = MajorityClassClassifier()\n",
    "\n",
    "metric = Accuracy()\n",
    "\n",
    "progressive_val_score(dataset=stream, model=model, metric=metric, print_every=1000)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Which is consistent with the class distribution as the data is slightly unbalanced in favor of the `True` class."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.6821"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model._counts[True] / sum(model._counts.values())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As mentioned, the performance of simple methods such as the majority class classifier is as a baseline.\n",
    "\n",
    "For example, we can compare it against the Gaussian Naive Bayes classifier:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[1,000] Accuracy: 92.99%\n",
      "[2,000] Accuracy: 93.70%\n",
      "[3,000] Accuracy: 94.23%\n",
      "[4,000] Accuracy: 94.37%\n",
      "[5,000] Accuracy: 94.40%\n",
      "[6,000] Accuracy: 94.05%\n",
      "[7,000] Accuracy: 94.00%\n",
      "[8,000] Accuracy: 93.96%\n",
      "[9,000] Accuracy: 94.02%\n",
      "[10,000] Accuracy: 94.03%\n",
      "[11,000] Accuracy: 94.16%\n",
      "[12,000] Accuracy: 94.27%\n",
      "[13,000] Accuracy: 94.25%\n",
      "[14,000] Accuracy: 94.23%\n",
      "[15,000] Accuracy: 94.14%\n",
      "[16,000] Accuracy: 94.16%\n",
      "[17,000] Accuracy: 94.10%\n",
      "[18,000] Accuracy: 94.08%\n",
      "[19,000] Accuracy: 94.08%\n",
      "[20,000] Accuracy: 94.11%\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "Accuracy: 94.11%"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "stream = synth.SEA(seed=42).take(20000)\n",
    "\n",
    "model = GaussianNB()\n",
    "\n",
    "metric = Accuracy()\n",
    "\n",
    "progressive_val_score(dataset=stream, model=model, metric=metric, print_every=1000)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "This concludes this introduction to stream learning with `river`. \n",
    "\n",
    "There is a lot more to discover and learn!\n",
    "\n",
    "\n",
    "## Additional resources:\n",
    "- River webpage: [https://riverml.xyz](https://riverml.xyz/)\n",
    "  - Documentation\n",
    "  - Examples\n",
    "- GitHub repository: [https://github.com/online-ml/river/](https://github.com/online-ml/river/)\n",
    "  - Follow development\n",
    "  - Interact with the development team and the project's community ([Discussions](https://github.com/online-ml/river/discussions))\n",
    "  - Report issues and bugs\n",
    "- River: machine learning for streaming data in Python ([arXiv](https://arxiv.org/abs/2012.04740))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 3.8",
   "language": "python",
   "name": "py38"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}