From bddb4883f2d7466c3c02d291fdb1d2b22b2aeab4 Mon Sep 17 00:00:00 2001 From: Jean-Luc Stevens Date: Mon, 8 Jul 2019 02:17:01 -0500 Subject: [PATCH] Continued polish for SciPy tutorial (#197) * Further tweaks to most notebooks * Added freeform exercise * Flakes fixes * Updated the inline exercise style in Custom Interactivity * Fixed error in environment.yml channel spec * Removed remaining WIP banners * Clarified nature of hvplot output and mentioned projecting with geoviews * Improvements to the Building a Dashboard exercise * Completely reworked the Plotting exercise notebook * Renamed tutorial sections consistently * Fixed headers * Cleaned up exercises --- doc/tutorial/index.rst | 10 +- examples/environment.yml | 2 +- examples/tutorial/01_Overview.ipynb | 8 +- ...o_Panel.ipynb => 02_Building_Panels.ipynb} | 26 +- ...vity.ipynb => 03_Interlinked_Panels.ipynb} | 150 ++++++-- ...ng_.plot.ipynb => 04_Basic_Plotting.ipynb} | 41 +-- ...s_Plots.ipynb => 05_Composing_Plots.ipynb} | 51 ++- ...ivity.ipynb => 06_Interlinked_Plots.ipynb} | 276 +++++++++++---- examples/tutorial/07_Large_Data.ipynb | 24 +- .../tutorial/08_Advanced_Dashboards.ipynb | 25 +- .../exercises/Advanced_Dashboarding.ipynb | 59 +++- .../exercises/Building_a_Dashboard.ipynb | 26 +- examples/tutorial/exercises/Plotting.ipynb | 334 ++++++++++++++++-- examples/tutorial/index.ipynb | 32 +- 14 files changed, 815 insertions(+), 249 deletions(-) rename examples/tutorial/{02_Introduction_to_Panel.ipynb => 02_Building_Panels.ipynb} (83%) rename examples/tutorial/{03_Panel_Interactivity.ipynb => 03_Interlinked_Panels.ipynb} (50%) rename examples/tutorial/{04_Using_.plot.ipynb => 04_Basic_Plotting.ipynb} (81%) rename examples/tutorial/{05_Composing_Holoviews_Plots.ipynb => 05_Composing_Plots.ipynb} (83%) rename examples/tutorial/{06_Custom_Interactivity.ipynb => 06_Interlinked_Plots.ipynb} (72%) diff --git a/doc/tutorial/index.rst b/doc/tutorial/index.rst index d4154890..f232ff62 100644 --- a/doc/tutorial/index.rst +++ b/doc/tutorial/index.rst @@ -11,10 +11,10 @@ HoloViz Tutorial Setup Overview - Introduction to Panel - Adding Interactivity to Panel - Using .plot - Composing Holoviews Plots - Custom Interactivity + Building Panels + Interlinked Panels + Basic Plotting + Composing Plots + Interlinked Plots Large Data Advanced Dashboards diff --git a/examples/environment.yml b/examples/environment.yml index 93fe1b7e..5a0d78f7 100644 --- a/examples/environment.yml +++ b/examples/environment.yml @@ -1,6 +1,6 @@ name: holoviz-tutorial channels: - - holoviz + - pyviz - defaults dependencies: - bokeh ==1.2.0 diff --git a/examples/tutorial/01_Overview.ipynb b/examples/tutorial/01_Overview.ipynb index 5c61d564..206b7a8a 100644 --- a/examples/tutorial/01_Overview.ipynb +++ b/examples/tutorial/01_Overview.ipynb @@ -124,7 +124,7 @@ "\n", "

\n", "

\n", - "These tools are the most fully supported and are often all that many people will need.\n", + "These tools are the most fully supported and are often entirely sufficient on their own.\n", "

\n", "\n", "
" @@ -136,7 +136,7 @@ "source": [ "## HoloViz libraries\n", "\n", - "\n", + "\n", "\n", "To address the above issues, we have developed a set of open-source\n", "Python packages to streamline the process of working with small and\n", @@ -184,7 +184,9 @@ "* make simple but powerful plots out of Pandas dataframes and Xarray multidimensional arrays\n", "* handle columnar data, big data, geo data, array data\n", "* provide custom interactive links between views of datasets\n", - "* handle the whole process from getting the data in, cleaning it, exploring it visually, creating plots for communication, building dashboards for sharing your analyses, and deploying dashboards." + "* handle the whole process from getting the data in, cleaning it, exploring it visually, creating plots for communication, building dashboards for sharing your analyses, and deploying dashboards.\n", + "\n", + "The tutorial is organized around the most general to the most specific, in terms of tool support. We first look at [Panel](https://panel.pyviz.org) package, which works with nearly any plotting library, then [hvPlot](https://hvplot.pyviz.org), which works with nearly any data library and shares an API with many other plotting libraries, and then dive deeper into HoloViz-specific approaches that let you work with large data, provide deep interactivity, and other advanced features." ] }, { diff --git a/examples/tutorial/02_Introduction_to_Panel.ipynb b/examples/tutorial/02_Building_Panels.ipynb similarity index 83% rename from examples/tutorial/02_Introduction_to_Panel.ipynb rename to examples/tutorial/02_Building_Panels.ipynb index 5db03615..c94b769d 100644 --- a/examples/tutorial/02_Introduction_to_Panel.ipynb +++ b/examples/tutorial/02_Building_Panels.ipynb @@ -6,25 +6,25 @@ "source": [ "\n", "\n", - "

Tutorial 2: Introduction to Panel

" + "

Tutorial 2: Building Panels

" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "\n", + "\n", "\n", - "Panel allows you to build interactive apps and dashboards in the notebook and then deploy them as a separate server. One of the main aims behind Panel is to allow you to lay out and add interactive controls to many of objects you are used to working with in the PyData ecosystem. This ability to a) deploy dashboards straight from your notebook and b) display all the plots and objects you are used to working with allows it to plug easily into your existing workflows and build interactive components and dashboards you can share with colleagues or outside of your organization.\n", + "Panel is designed to make it simple to add interactive controls to your existing plots and data displays, simple to build apps for your own use in a notebook, simple to deploy apps as standalone dashboards to share with colleagues, and seamlessly shift back and forth between each of these tasks as your needs evolve. If there is one thing you should take away from this tutorial, it's Panel!\n", "\n", - "In this section of the tutorial we will get you familiarized with the three main types of components, to get you started on displaying objects with Panel, laying them out, and then connecting them to interactive widgets to build simple apps. Finally we will take you through an exercise to build a simple dashboard from a number of existing components.\n", + "In this section of the tutorial we will get you familiarized with the three main types of components in Panel, to get you started on displaying objects, laying them out, and then connecting them to interactive widgets to build simple apps. Finally we will take you through an exercise to build a simple dashboard from a number of existing components. For now, we won't show code for any particular plotting library, but if you have a favorite one already, you should be able to use it with Panel in the exercises.\n", "\n", "## Component types\n", "\n", - "Before we start building interactive apps with Panel we will learn about the three main types of components in Panel:\n", + "Before we start building interactive apps, we will learn about the three main types of components in Panel:\n", "\n", - "* **Pane**: A Pane provides a view of an external object (e.g. text, images, plots etc.) by wrapping it\n", - "* **Panel**: A Panel lays out multiple components in a row, column or grid.\n", + "* **Pane**: A Pane provides a view of an external object (text, image, plot, etc.) by wrapping it\n", + "* **Panel**: A Panel lays out multiple components in a row, column, or grid.\n", "* **Widget**: A Widget provides input controls to add interactive features to your Panel.\n", "\n", "If you ever want to discover how a particular component works, see the [reference gallery](https://panel.pyviz.org/reference/index.html).\n", @@ -83,7 +83,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Panel transformed the `str` object and wrapped it in a so-called `Markdown` `Pane`. The ``pn.panel`` function attempts to find the most appropriate representation for different objects whether it is a string, an image, or even a plot. So if we provide the location of a PNG file instead, the ``panel`` function will automatically infer that it should be rendered as an image:" + "Panel transformed the `str` object and wrapped it in a so-called `Markdown` `Pane`. The ``pn.panel`` function attempts to find the most appropriate representation for different objects whether it is a string, an image, or even a plot. So if we provide the location of a PNG file instead as a path or a URL, the ``panel`` function will automatically infer that it should be rendered as an image:" ] }, { @@ -151,7 +151,7 @@ "source": [ "## Laying out content\n", "\n", - "In addition to `Pane` objects Panel provides `Panel` objects which allow laying out components. The principal layouts are by ``Row`` or ``Column``. These components act just like a regular ``list`` in Python:" + "In addition to `Pane` objects, Panel provides `Panel` objects that allow laying out components. The principal layouts are by ``Row`` or ``Column``. These components act just like a regular ``list`` in Python:" ] }, { @@ -169,7 +169,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Panels may be nested arbitrarily to construct complex layouts. Internally, Panel will call the ``pn.panel`` function on any objects which are not already a known component type, making it easy to lay out objects without explicitly wrapping them in a panel component, but wrapping it explicitly allows passing arguments such as `width=500`:" + "Panels may be nested arbitrarily to construct complex layouts. Internally, Panel will call the ``pn.panel`` function on any objects which are not already a known component type, making it easy to lay out objects without explicitly wrapping them in a panel component, though wrapping it explicitly can help ensure that it is the type you expect:" ] }, { @@ -199,7 +199,7 @@ "source": [ "#### Exercise\n", "\n", - "Use ``Row`` and ``Column`` panels to lay out the objects (text, images, or plots) you rendered in the previous exercise." + "Use ``Row`` and ``Column`` panels to lay out some of the objects (text, images, or plots) you rendered in the previous exercise." ] }, { @@ -277,7 +277,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "As we discussed above ``Row`` and ``Column`` Panels behave much like lists. Just like lists we can use all the standard methods such as ``append``, ``pop``, ``remove`` or even setting to modify them. E.g. we could dynamically add a caption for the logo:" + "As we discussed above ``Row`` and ``Column`` Panels behave much like lists. Just like lists, we can use ``append``, ``pop``, ``remove``, and all the other standard methods (including setting with =) to modify them. E.g. we could dynamically add a caption for the logo:" ] }, { @@ -352,7 +352,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Once you are happy with the app or dashboard you have built you can annotate it with ``.servable()``. In the notebook, this annotation has no effect, but it will later indicate to the ``panel serve`` command that you want this particular component to be served. If you then go to the commandline and run ``panel serve 02_Introduction_to_Panel.ipynb``, the code cells in the notebook will be executed and the resulting dashboard will be served." + "Once you are happy with the app or dashboard you have built you can annotate it with ``.servable()``. In the notebook, this annotation has no effect, but it will later indicate to the ``panel serve`` command that you want this particular component to be served. If you then go to the command line and run ``panel serve 02_Introduction_to_Panel.ipynb``, the code cells in the notebook will be executed and the resulting dashboard will be available as a web server." ] }, { diff --git a/examples/tutorial/03_Panel_Interactivity.ipynb b/examples/tutorial/03_Interlinked_Panels.ipynb similarity index 50% rename from examples/tutorial/03_Panel_Interactivity.ipynb rename to examples/tutorial/03_Interlinked_Panels.ipynb index cff21e93..acbfe9c8 100644 --- a/examples/tutorial/03_Panel_Interactivity.ipynb +++ b/examples/tutorial/03_Interlinked_Panels.ipynb @@ -6,7 +6,7 @@ "source": [ "\n", "\n", - "

Tutorial 3: Adding Interactivity to Panel

" + "

Tutorial 3: Interlinked Panels

" ] }, { @@ -24,16 +24,86 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "In the previous section we learned the very basics of working with Panel. Specifically we looked at the different types of components, how to update them and how to serve a Panel application or dashboard. However to start building actual apps with Panel we need to be able to add interactivity by linking different components together. In this section we will learn how to link widgets to outputs to start building some simple interactive applications." + "In the previous section we learned the very basics of working with Panel. Specifically we looked at the different types of components, how to update them and how to serve a Panel application or dashboard. However to start building actual apps with Panel we need to be able to add interactivity by linking different components together. In this section we will learn how to link widgets to outputs to start building some simple interactive applications.\n", + "\n", + "In this example we will use the earthquake data we started playing with in the previous exercise, so will start by loading it:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "import dask.dataframe as dd\n", + "\n", + "df = dd.read_parquet('../data/earthquakes.parq', columns=['time', 'place', 'mag']).reset_index(drop=True).persist()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## pn.interact\n", + "\n", + "The simplest way to get interactivity is to use Panel's `interact` function, modeled on the similar function in `ipywidgets`. For instance, if you have a function that returns a row of a dataframe given an index, you can very easily make a panel with a widget to control the row displayed." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "def select_row(row=0):\n", + " return df.loc[row].compute()\n", + "\n", + "pn.interact(select_row, row=(0, len(df)-1))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "This approach can be used for any function that returns a displayable object, calling the function whenever one of the parameters of that function has changed. \n", + "\n", + "In the spirit of shortcuts and not dead ends, let's see what's in the object returned by `interact`:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "app = pn.interact(select_row, row=(0, len(df)-1))\n", + "\n", + "print(app)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Here, `interact` has constructed a Column panel consisting of one Column of widgets (with one widget), and one Row of output (with one HTML pane). This object, once created, is a full compositional Panel object, and can be reconfigured and expanded if you wish without breaking the connections between widgets and values:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "pn.Column(\"## Choose a row\", pn.Row(app[0], app[1]))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "## Widgets\n", + "## Widgets and links\n", "\n", - "One of the most common patterns when building an application is linking a widget to some output and updating it in response. As we briefly mentioned in the last section, a widget is an input control which allows a user to change a ``value`` using some graphical UI. A simple example is a ColorPicker:" + "`pn.interact` constructs widgets automatically that can then be reconfigured, but if you want more control, you'll want to instantiate widgets explicitly. A widget is an input control that allows a user to change a ``value`` using some graphical UI. A simple example is a ColorPicker:" ] }, { @@ -51,7 +121,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "When we change the color the ``value`` updates:" + "Here the widget `value` is a [Parameter](https://param.pyviz.org) that is set to a string specifying a color. Parameters are an extended type of Python attribute that declare their type, range, etc. so that other code can interact with them in a consistent way. When we change the color using the widget the ``value`` parameter updates, and vice versa if you change the value parameter manually:" ] }, { @@ -63,11 +133,20 @@ "color_picker.value" ] }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# color_picker.value='#559977'" + ] + }, { "cell_type": "markdown", "metadata": {}, "source": [ - "As a very simple example we can link the slider ``value`` to the ``background`` parameter on a ``HTML`` pane:" + "Now that we have a widget, we can link its ``value`` parameter to a parameter on some other object. Here, let's link it to the ``background`` parameter on a ``HTML`` pane:" ] }, { @@ -77,7 +156,6 @@ "outputs": [], "source": [ "html = pn.pane.HTML('', width=200, height=200, background=color_picker.value)\n", - "\n", "color_picker.link(html, value='background')\n", "\n", "html" @@ -87,7 +165,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Whenever the color is changed the background of the HTML pane will update. Linking may also be done directly in Javascript, which allows building apps which do not require a live server or notebook kernel. For more detail see the [Links user guide](http://panel.pyviz.org/user_guide/Links.html)." + "Whenever the color is changed the background of the HTML pane will update, because the `link` method set up the HTML pane to \"watch\" for changes in that value and update as needed. Linking may also be done directly in Javascript, which allows building apps which do not require a live server or notebook kernel. For more detail see the [Links user guide](http://panel.pyviz.org/user_guide/Links.html)." ] }, { @@ -96,7 +174,7 @@ "source": [ "#### Exercise\n", "\n", - "Use tab-completion on the ``pn.widget`` namespace to discover some other widget, construct a string pane and then link the widget value to the pane ``object`` parameter. Finally display both the widget and the pane in a panel." + "Use tab-completion on the ``pn.widgets`` namespace to discover some other widget, construct a string pane initialized to the `value` parameter of that widget, and then link the widget's value to the string pane's ``object`` parameter. Finally display both the widget and the pane in a panel." ] }, { @@ -110,22 +188,25 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "## Callbacks\n", - "\n", - "The ability to link parameters is somewhat limited, the more general approach is to write callbacks in response to changes in some parameter, e.g. the ``value`` of a widget. All parameters can be watched using the ``.param.watch`` API, which will call the provided callback with an event object containing the old and new value of the widget.\n", + "
Solution
\n", "\n", - "In this example we will use the earthquake data we started playing with in the previous exercise, so will start by loading it:" + "```python\n", + "w = pn.widgets.FloatSlider()\n", + "s = pn.pane.Str(w.value)\n", + "w.link(s, value=\"object\")\n", + "pn.Row(w,s)\n", + "```\n", + " \n", + "
" ] }, { - "cell_type": "code", - "execution_count": null, + "cell_type": "markdown", "metadata": {}, - "outputs": [], "source": [ - "import dask.dataframe as dd\n", + "## Callbacks\n", "\n", - "df = dd.read_parquet('../data/earthquakes.parq', columns=['time', 'place', 'mag']).reset_index(drop=True).persist()" + "The ability to link parameters is somewhat limited, in that it can only pass values through unchanged. Panel also supports the more general approach of writing callbacks in response to changes in some parameter, e.g. the ``value`` of a widget. All parameters can be watched using the ``.param.watch`` API, which will call the provided callback with an event object containing the old and new value of the widget." ] }, { @@ -206,7 +287,14 @@ "metadata": {}, "outputs": [], "source": [ - "pn.Row(row_slider, row_pane)" + "pn.Column(row_slider, row_pane)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "As you can see, we've fairly laboriously built up what `pn.interact` gave us immediately above, but doing it in this way should help you see how everything fits together." ] }, { @@ -215,9 +303,9 @@ "source": [ "## Declaring interactive components\n", "\n", - "There are a variety of ways of linking different components in Panel, and while writing callbacks like above is the most flexible it is also quite verbose and can quickly become quite messy. A different approach is to express dependencies between the parameters of one object and declare those as inputs to a function.\n", + "In between the fully automated `pn.interact` and the fully manual but quite messy and verbose callbacks above, Panel offers a very concise, powerful approach of declaring dependencies between the parameters of one object and the arguments to a function. In practice, this middle ground provides enough control for nearly any app, without the complexity of explicit chains of callbacks.\n", "\n", - "As a very simple example we will declare a ``TextInput`` widget and then write a function, which uses Markdown syntax to convert the text into a title. Using the ``pn.depends`` decorator we can then declare that this function depends on the ``value`` of the widget. Finally we lay out the widget and the function:" + "As a very simple example we will declare a ``TextInput`` widget and then write a function that uses Markdown syntax to convert the text into a title. Using the ``pn.depends`` decorator we can then declare that this function depends on the ``value`` of the widget. Finally we lay out the widget and the function:" ] }, { @@ -232,14 +320,24 @@ "def title_text(value):\n", " return '## ' + value\n", "\n", - "pn.Row(text_input, title_text)" + "app2 = pn.Row(text_input, title_text)\n", + "app2" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "When entering some text into the widget (and pressing enter) we will see that it appears as a title. In this way we can easily declare dynamic components which depend on the value of a widget (or any other component which has parameters)." + "After entering some text into the widget (and pressing enter), the title text should update (if you have a live Python server running). In this way we can easily declare dynamic components that depend on the value of a widget (or any other component that has Parameters). Regardless of whether you use interact, callbacks, or dependency declarations, all of the apps will be some composition of panes into a panel, with links or callbacks connecting things up:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "print(app2)" ] }, { @@ -248,7 +346,7 @@ "source": [ "#### Exercise\n", "\n", - "Declare two ``Spinner`` widgets with an initial value of 1, then declare a function which depends on the values of both widgets and adds them together. Finally lay out the two widgets and the function in a Panel:" + "Declare two ``Spinner`` widgets with an initial value of 1, then declare a function that depends on the values of both widgets and adds them together. Finally lay out the two widgets and the function in a Panel:" ] }, { @@ -282,7 +380,9 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Now that we have learned not only how to display different types of data, lay it out but also how to link parameters and build interactive components we can start building actual apps and dashboards. Before we move on to plotting and visualization let us quickly recap what we have learned by adding interactivity to [the dashboard we previously build](./exercises/Building_a_Dashboard.ipynb)." + "# Moving onwards\n", + "\n", + "Now that we have learned to link parameters between displayed objects and build interactive components, we can start building actual apps and dashboards. Before we move on to plotting and visualization let us quickly use what we have learned by adding interactivity to [the dashboard we built in the previous exercise](./exercises/Building_a_Dashboard.ipynb)." ] } ], diff --git a/examples/tutorial/04_Using_.plot.ipynb b/examples/tutorial/04_Basic_Plotting.ipynb similarity index 81% rename from examples/tutorial/04_Using_.plot.ipynb rename to examples/tutorial/04_Basic_Plotting.ipynb index 1a09ea74..a77e144c 100644 --- a/examples/tutorial/04_Using_.plot.ipynb +++ b/examples/tutorial/04_Basic_Plotting.ipynb @@ -6,15 +6,7 @@ "source": [ "\n", "\n", - "

Tutorial 4. Using .plot

" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "
WORK IN PROGRESS: We are in the progress of updating these materials in anticipation of a tutorial at the 2019 SciPy conference. Work will be complete by the morning of July 8th 2019. Check out this tag to access the materials as they were before these changes started. For the latest version of the tutorial, visit holoviz.org.\n", - "
" + "

Tutorial 4. Basic Plotting

" ] }, { @@ -23,7 +15,7 @@ "source": [ "Previous sections have focused on putting various simple types of data together in notebooks and deployed servers, but most people will want to include plots as well. In this section, we'll focus on one of the simplest (but still powerful) ways to get a plot.\n", "\n", - "If you have tried to visualize a `pandas.DataFrame` before, then you have likely encountered the [Pandas .plot() API](https://pandas.pydata.org/pandas-docs/stable/user_guide/visualization.html). This basic plotting interface uses [Matplotlib](http://matplotlib.org) to render static PNGs in a Jupyter notebook or for exporting from Python, with a command that can be as simple as `df.plot()` for a DataFrame with two columns. \n", + "If you have tried to visualize a `pandas.DataFrame` before, then you have likely encountered the [Pandas .plot() API](https://pandas.pydata.org/pandas-docs/stable/user_guide/visualization.html). This basic plotting interface uses [Matplotlib](http://matplotlib.org) to render static PNGs in a Jupyter notebook or for exporting from Python, with a command that can be as simple as `df.plot()` for a DataFrame with one or two columns. \n", "\n", "The Pandas .plot() API has emerged as a de-facto standard for high-level plotting APIs in Python, and is now supported by many different libraries that use other underlying plotting engines to provide additional power and flexibility. Thus learning this API allows you to access capabilities provided by a wide variety of underlying tools, with relatively little additional effort. The libraries currently supporting this API include:\n", "\n", @@ -34,9 +26,9 @@ "- [Cufflinks](https://github.com/santosjorge/cufflinks) -- Plotly-based interactive plots for Pandas data.\n", "- [PdVega](https://altair-viz.github.io/pdvega) -- Vega-lite-based, JSON-encoded interactive plots for Pandas data.\n", "\n", - "In this notebook we'll explore what is possible with the default `.plot` API and demonstrate the additional capabilities of `.hvplot`, using a tabular dataset of earthquakes and other seismological events queried \n", + "In this notebook we'll explore what is possible with the default `.plot` API and demonstrate the additional capabilities of `.hvplot`, using the same tabular dataset of earthquakes and other seismological events queried \n", "from the [USGS Earthquake Catalog](https://earthquake.usgs.gov/earthquakes/search) using its \n", - "[API](https://github.com/pyviz/holoviz/wiki/Creating-the-USGS-Earthquake-dataset). Of course, this particular dataset, is just an example; the same approach can be used with just about any tabular dataset." + "[API](https://github.com/pyviz/holoviz/wiki/Creating-the-USGS-Earthquake-dataset) as in previous sections. Of course, this particular dataset is just an example; the same approach can be used with just about any tabular dataset." ] }, { @@ -45,7 +37,7 @@ "source": [ "### Read in the data\n", "\n", - "Here we'll read in the data using Dask, which works well with a relatively large dataset like this (2.1 million rows) and in most cases can be `persist`ed into main memory for higher performance:" + "Here we'll read in the data using Dask, which works well with a relatively large dataset like this (2.1 million rows). We'll use `.persist()` to bring the whole dataset into main memory (which should be feasible on any recent machine) for higher performance:" ] }, { @@ -73,7 +65,9 @@ "source": [ "### Using Pandas `.plot`\n", "\n", - "The first thing that we'd like to do with this data is visualize the locations of every earthquake. So we would like to make a `scatter` plot where `x='longitude'` and `y='latitude'`. If you are familiar with the `pandas.plot` API you might expect to execute: `df.plot.scatter(x='longitude', y='latitude')`. Feel free to try this out in a new cell, but it will throw an error: `AttributeError: 'DataFrame' object has no attribute 'plot'`. Since we have a Dask dataframe rather than a Pandas dataframe, we need to first convert it to Pandas. In order to make the data more manageable for now, we'll use just a fraction (1%) of it and call that `small_df`. " + "The first thing that we'd like to do with this data is visualize the locations of every earthquake. So we would like to make a scatter or points plot where `x='longitude'` and `y='latitude'`. \n", + "\n", + "If you are familiar with the `pandas.plot` API, you might expect to execute `df.plot.scatter(x='longitude', y='latitude')`. Feel free to try this out in a new cell, but it will throw an error: `AttributeError: 'DataFrame' object has no attribute 'plot'`. Since we have a Dask dataframe rather than a Pandas dataframe, we need to first convert it to Pandas to use `.plot`. In order to make the data more manageable for now, we'll briefly use just a fraction (1%) of it and call that `small_df`. " ] }, { @@ -116,7 +110,8 @@ "metadata": {}, "source": [ "### Using `.hvplot`\n", - "That is a good place to start and you can start to see the structure of the edges of the plates (which in some cases correspond with the edges of the continents and in others are between two continents). You can make a very similar plot with the same arguments using hvplot. " + "\n", + "As you can see above, the Pandas API gives you a usable plot very easily, where you can start to see the structure of the edges of the plates (which in some cases correspond with the edges of the continents and in others are between two continents). You can make a very similar plot with the same arguments using hvplot. " ] }, { @@ -141,9 +136,9 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Here if you hover over any of the datapoints, you can see some information about it, which makes the hvPlot output much more useful than the original static PNG. You can also pan and zoom to focus on any particular region of the data of interest.\n", + "Here unlike in the Pandas `.plot()` static PNG, if you hover over any of the datapoints, you can see the actual location values, and you can also pan and zoom to focus on any particular region of the data of interest.\n", "\n", - "You might have noticed that many of the dots in the scatter that we've just created lie on top of one another. This is called [\"overplotting\"](http://datashader.org/user_guide/1_Plotting_Pitfalls.html#1.-Overplotting) and can be avoided in a variety of ways, such as by making the dots slightly transparent, or binning the data. These approaches have the downside of introducing bias because you need to choose the alpha or the edges of the bins, and in order to do that, you have to make assumptions about the data. For an initial exploration of a new dataset, it's much safer if you can just *see* the data, before you impose any assumptions about its form or structure. \n" + "You might have noticed that many of the dots in the scatter that we've just created lie on top of one another. This is called [\"overplotting\"](http://datashader.org/user_guide/1_Plotting_Pitfalls.html#1.-Overplotting) and can be avoided in a variety of ways, such as by making the dots slightly transparent, or binning the data. These approaches have the downside of introducing bias because you need to choose the alpha or the edges of the bins, and in order to do that, you have to make assumptions about the data. For an initial exploration of a new dataset, it's much safer if you can just ***see*** the data, before you impose any assumptions about its form or structure. \n" ] }, { @@ -182,7 +177,7 @@ "source": [ "### Datashader\n", "\n", - "Instead of these traditional approaches, we can use [Datashader](datashader.org), which aggregates data into each pixel without any arbitrary parameter settings. In `hvplot` we can activate this capability by setting `datashade=True`." + "To avoid some of the problems of traditional scatter/point plots we can use [Datashader](datashader.org), which aggregates data into each pixel without any arbitrary parameter settings. In `hvplot` we can activate this capability by setting `datashade=True`." ] }, { @@ -198,7 +193,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "You can already see a lot more detail, but remember that we are still only plotting 1% of the data (21k earthquakes). With datashader, we can easily plot all of the full, original dataset:" + "We can already see a lot more detail, but remember that we are still only plotting 1% of the data (21k earthquakes). With datashader, we can easily plot all of the full, original dataset:" ] }, { @@ -223,7 +218,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Here you can see all of the rich detail in this set of locations, and if you have a live Python process running, you can zoom in and see additional detail at each zoom level, without tuning any parameters or making any assumptions about the form or structure of the data. We'll come back to Datashader later, but for now the important thing to know about it is that it lets us work with arbitrarily large datasets in a web browser conveniently.\n", + "Here you can see all of the rich detail in this set of millions of earthquake event locations. If you have a live Python process running, you can zoom in and see additional detail at each zoom level, without tuning any parameters or making any assumptions about the form or structure of the data. We'll come back to Datashader later, but for now the important thing to know about it is that it lets us work with arbitrarily large datasets in a web browser conveniently.\n", "\n", "Note that the `.hvplot()` API works here even though `df` is a Dask and not Pandas object, because unlike the other `.plot` libraries, `hvplot` doesn't just target Pandas objects. Instead hvplot can be used with: \n", " - Pandas : DataFrame, Series (columnar/tabular data)\n", @@ -241,7 +236,7 @@ "source": [ "#### Exercise\n", "\n", - "If you are brave and don't mind refreshing your browser tab if it dies, create a scatter without setting datashade=True." + "If you are brave and don't mind refreshing your browser tab if it dies, create a scatter for the full dataset without setting datashade=True." ] }, { @@ -259,7 +254,7 @@ "\n", "As a final note, we should really use `hvplot.points` instead of `hvplot.scatter` in this instance. The former does not exist in the standard pandas `.plot` API which is why we use `hvplot.scatter` up until now.\n", "\n", - "The reason scatter is inappropriate is that it implies that the y-axis (latitude) is a *dependent variable* with respect to the x-axis (latitude). In reality, this is not the case as earthquakes can happen at anywhere on the Earth's two-dimensional surface. For this reason, it is best to use `hvplot.points` to present earthquake locations as will be explained further in the next notebook." + "The reason scatter is inappropriate is that it implies that the y-axis (latitude) is a *dependent variable* with respect to the x-axis (latitude). In reality, this is not the case, as earthquakes can happen at anywhere on the Earth's two-dimensional surface. For this reason, it is best to use `hvplot.points` for earthquake locations, as will be explained further in the next notebook." ] }, { @@ -277,7 +272,7 @@ "source": [ "### Statistical Plots\n", "\n", - "Let's dive into some of the other capabilities of `.hvplot()`, starting with the frequency of different magnitude earthquakes.\n", + "Let's dive into some of the other capabilities of `.plot()` and `.hvplot()`, starting with the frequency of different magnitude earthquakes.\n", "\n", "| Magnitude | Earthquake Effect | Estimated Number Each Year |\n", "|---------------|-------------------|----------------------------|\n", diff --git a/examples/tutorial/05_Composing_Holoviews_Plots.ipynb b/examples/tutorial/05_Composing_Plots.ipynb similarity index 83% rename from examples/tutorial/05_Composing_Holoviews_Plots.ipynb rename to examples/tutorial/05_Composing_Plots.ipynb index a610231d..ee890e74 100644 --- a/examples/tutorial/05_Composing_Holoviews_Plots.ipynb +++ b/examples/tutorial/05_Composing_Plots.ipynb @@ -6,23 +6,25 @@ "source": [ "\n", "\n", - "

Tutorial 5. Composing Holoviews Plots

" + "

Tutorial 5. Composing Plots

" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "So far we have generated plots using hvplot, but we haven't discussed what exactly these plots are and all the things that you can do with them. The short answer is that they are [holoviews](https://holoviews.org) objects rendered using the [bokeh](https://bokeh.org) backend. In the [last notebook](04_Using_.plot.ipynb) we saw holoviews supporting grouping and linking between subplots and bokeh supporting interactivity. \n", + "So far we have generated plots using [hvPlot](http://hvplot.pyviz.org), but we haven't discussed what exactly these plots are and how they differ from the output of other libraries offering the `.plot` API. It turns out that as in the previous `pn.interact` example, the `.hvplot()` output is actually a rich, compositional object that can be used in many different ways, not just as an immediate plot. Specifically, hvPlot generates [HoloViews](https://holoviews.org) objects rendered using the [bokeh](https://bokeh.org) backend. In the previous notebook we saw that these objects are rendered as interactive Bokeh plots that support hovering, panning, and zooming. \n", "\n", - "In this notebook, we'll deconstruct hvplot output to show what individual elements are and discuss how they can be combined and used to compose layered visualizations." + "In this notebook, we'll examine the output of hvPlot calls to take a look at individual HoloViews objects. Then we will see how these \"elements\" offer us powerful ways of combining and composing layered visualizations." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "### Read in the data" + "### Read in the data\n", + "\n", + "We'll read in the data as before, and also reindex by time so that we can do more easily do resampling. " ] }, { @@ -44,13 +46,6 @@ "#df.head()" ] }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Next we'll clean the data as we did in the last notebook. We'll also reindex by time so that we can do more easily do resampling. " - ] - }, { "cell_type": "code", "execution_count": null, @@ -86,7 +81,24 @@ "outputs": [], "source": [ "monthly_count = cleaned_reindexed_df.id.resample('1M').count().rename('count')\n", - "monthly_count_plot = monthly_count.hvplot(title='Monthly count')\n", + "monthly_count_plot = monthly_count.hvplot(title='Monthly count')" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The first thing to note is that with `hvplot`, it is common to grab a handle on the returned output. Unlike the matplotlib based `.plot` API of pandas, where an axis object is returned (with plotting display occuring as a side-effect if matplotlib inline is loaded), grabbing the output of `hvplot` has no side-effects at all (as would be true for typical Python objects as well).\n", + "\n", + "When working with the HoloViews object returned by `hvplot`, plotting only occurs when we look at the object itself:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ "monthly_count_plot" ] }, @@ -94,7 +106,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Let's take a closer look at this plot object." + "Now we have a handle on this object, we can look at its textual representation by printing it:" ] }, { @@ -249,7 +261,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "This return value is actually a HoloViews element which has a printed representation:" + "As always, this return value is actually a HoloViews element which has a printed representation:" ] }, { @@ -265,9 +277,9 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "As mentioned earlier, the notion of a 'scatter' plot implies that there is an *independent variable* and at least one *dependent variable*. This is reflected in the printed representation where the independent variables are in the square brackets and the dependent ones are in parentheses - we can now see that this scatter object implies that latitude is dependent on longitude which is incorrect. We will learn more about HoloViews objects in the next notebook.\n", + "As mentioned earlier, the notion of a 'scatter' plot implies that there is an *independent variable* and at least one *dependent variable*. This is reflected in the printed representation where the independent variables are in the square brackets and the dependent ones are in parentheses - we can now see that this scatter object implies that latitude is dependent on longitude, which is incorrect. We will learn more about HoloViews objects in the next notebook, and we'll fix the dimensions below.\n", "\n", - "We can adjust the options to create a better plot. First we'll use [colorcet](https://colorcet.pyviz.org) to get a colormap that doesn't have white at one end, to avoid ambiguity with the page background. We can choose one from the website and use the plotting module to make sure it looks good. " + "But first, let's adjust the options to create a better plot. First we'll use [colorcet](https://colorcet.pyviz.org) to get a colormap that doesn't have white at one end, to avoid ambiguity with the page background. We can choose one from the website and use the HoloViews/Bokeh-based colorcet plotting module to make sure it looks good. " ] }, { @@ -457,6 +469,9 @@ "cell_type": "markdown", "metadata": {}, "source": [ + "Note that the Web Mercator projection is only one of many possible projections used when working with geospatial data. If you need to work with these different projections, you can use the [GeoViews](http://geoviews.org) extension to HoloViews that makes elements aware of the projection they are defined in and automatically projects into whatever coordinates are needed for display. \n", + "\n", + "\n", "#### Exercise\n", "\n", "Import and use different tiles. \n", @@ -537,7 +552,7 @@ "metadata": {}, "outputs": [], "source": [ - "rasterized_pop = cleaned_ds.hvplot.image(rasterize=True)\n", + "rasterized_pop = cleaned_ds.hvplot.image(rasterize=True).opts(logz=True) # .opts() is explained in the next notebook\n", "rasterized_pop" ] }, @@ -615,7 +630,7 @@ "source": [ "As you can see, the HoloViews objects returned by hvPlot are in no way dead ends -- you can flexibly compare, combine, lay out, and overlay them to reveal complex interrelationships in your data.\n", "\n", - "In the [next notebook](06_Custom_Interactivity.ipynb) we'll learn more about how to use holoviews effectively to keep building up plots with custom interactivity." + "In the next notebook we'll learn how to flexibly customize how HoloViews plots interact with each other, to show such relationships on the fly." ] } ], diff --git a/examples/tutorial/06_Custom_Interactivity.ipynb b/examples/tutorial/06_Interlinked_Plots.ipynb similarity index 72% rename from examples/tutorial/06_Custom_Interactivity.ipynb rename to examples/tutorial/06_Interlinked_Plots.ipynb index a0fdb795..c5ca4bc7 100644 --- a/examples/tutorial/06_Custom_Interactivity.ipynb +++ b/examples/tutorial/06_Interlinked_Plots.ipynb @@ -6,28 +6,27 @@ "source": [ "\n", "\n", - "

Tutorial 6. Custom Interactivity

" + "

Tutorial 6. Interlinked Plots

" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "Using hvplot allows you to generate a number of different types of plot\n", - "quickly by building [HoloViews](https://holoviews.org) objects as discussed in the previous\n", + "Using hvPlot allows you to generate a number of different types of plot\n", + "quickly from a standard API by building [HoloViews](https://holoviews.org) objects, as discussed in the previous\n", "notebook. These objects are rendered with Bokeh which offers a number of\n", "standard ways to interact with your plot, such as panning and zooming\n", "tools.\n", "\n", "Many other modes of interactivity are possible when building an\n", "exploratory visualization (such as a dashboard) and these forms of\n", - "interactivity cannot be achieved using hvplot alone.\n", + "interactivity cannot be achieved using hvPlot alone.\n", "\n", "In this notebook, we will drop down to the HoloViews level of\n", - "representation to build a visualization consisting of linked plots that\n", + "representation to build a visualization directly that consists of linked plots that\n", "update when you interactivity select a particular earthquake with the\n", - "mouse. The goal is to show how more sophisticated forms or interactivity\n", - "can be introduced in the spirit of \"shortcuts\" not \"dead ends\"." + "mouse. The goal is to show how more sophisticated forms of interactivity can be built when needed, in a way that's fully compatible with all the examples shown in earlier sections." ] }, { @@ -116,20 +115,31 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "This object is an example of a HoloViews *Element*." + "This object is an example of a HoloViews *Element* which is an object that can display itself. These elements are *thin* wrappers around your data and the raw input data is always available on the `.data` attribute. For instance, we can look at the `head` of the `most_severe_projected` `DataFrame` as follows:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "high_mag_quakes.data.head()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "### HoloViews Elements\n", + "We will now learn a little more about `HoloViews` elements, including how to build them up from scratch so that we can control every aspect of them.\n", + "\n", + "### An Introduction to HoloViews Elements\n", "\n", "HoloViews elements are the atomic, visualizable components that can be\n", - "rendered by a plotting library such as Bokeh. We don't need to use\n", - "hvplot to create these element objects: we can create them directly by\n", + "rendered by a plotting library such as Bokeh. We don't actually need to use\n", + "hvPlot to create these element objects: we can create them directly by\n", "importing HoloViews (and loading the extension if we have not loaded\n", - "hvplot):" + "hvPlot):" ] }, { @@ -209,22 +219,29 @@ "For more information an HoloViews dimensions, see this [user guide](http://holoviews.org/user_guide/Annotating_Data.html)." ] }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### Exercise\n", + "\n", + "Visit the [HoloViews reference gallery](http://holoviews.org/reference/index.html) and browse\n", + "the available set of elements. Pick an element type and try running\n", + "one of the self-contained examples in the following cell." + ] + }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], - "source": [ - "### Exercise: Visit http://holoviews.org/reference/index.html and browse\n", - "### the available set of elements. Pick an element type and try running\n", - "### one of the self-contained examples in this cell." - ] + "source": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "### Help and Options\n", + "### Setting Visual Options\n", "\n", "The two `Points` elements above look quite different from the one\n", "returned by hvplot showing the earthquake positions. This is because\n", @@ -244,20 +261,31 @@ "height_v_weight.opts(color='fitness', size=8, colorbar=True, aspect='square')" ] }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### Exercise\n", + "\n", + "Copy the line above into the next cell and try changing the points to\n", + "'blue' or 'green' or another dimension of the data such as 'height' or 'weight'.\n", + "\n", + "Are the results what you expect?" + ] + }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], - "source": [ - "### Exercise: Try changing the points to 'blue' or 'green' or another dimension of the data \n", - "### such as 'height' or 'weight'. Are the results what you expect?" - ] + "source": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ + "### The `help` system\n", + "\n", "You can learn more about the `.opts` method and the HoloViews options\n", "system in the [corresponding user\n", "guide](http://holoviews.org/user_guide/Applying_Customizations.html). To\n", @@ -271,9 +299,8 @@ "metadata": {}, "outputs": [], "source": [ - "### Exercise: Uncomment the line below and run this cell. Try inspecting\n", - "### the output for different element types.\n", - "# hv.help(hv.Scatter)" + "# Commented as there is a lot of help output!\n", + "# hv.help(hv.Scatter) " ] }, { @@ -281,7 +308,7 @@ "metadata": {}, "source": [ "At this point, we can have some insight to the sort of HoloViews object\n", - "hvplot is building behind the scenes for our earthquake example:" + "hvPlot is building behind the scenes for our earthquake example:" ] }, { @@ -297,7 +324,39 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "### Adding interactivity to Elements\n", + "#### Exercise\n", + "\n", + "Try using `hv.help` to inspect the options available for different element types such as the `Points` element used above. Copy the line above into the cell below and pick a `Points` option that makes sense to you and try using it in the `.opts` method." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "
Hint
\n", + "\n", + "If you can't decide on an option to pick, a good choice is `marker`. For instance, try:\n", + "\n", + " * `marker='+'` \n", + " * `marker='d'`.\n", + " \n", + " HoloViews uses [matplotlib's conventions](https://matplotlib.org/3.1.0/api/markers_api.html) for specifying the various marker types. Try finding out which ones are support by Bokeh.\n", + "\n", + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Custom interactivity for Elements\n", "\n", "When rasterization of the population density data via hvplot was\n", "introduced in the last notebook, we saw that the HoloViews object\n", @@ -314,7 +373,7 @@ "guide](http://holoviews.org/user_guide/Dimensioned_Containers.html).\n", "\n", "Now let us build a very simple `DynamicMap` that is driven by a *linked\n", - "stream* specifically a `PointerXY` stream that represents the position\n", + "stream* (specifically a `PointerXY` stream) that represents the position\n", "of the cursor over the plot:" ] }, @@ -353,28 +412,36 @@ ] }, { - "cell_type": "code", - "execution_count": null, + "cell_type": "markdown", "metadata": {}, - "outputs": [], "source": [ - "### Exercise: Look up the Ellipse, HLine and VLine elements in the\n", - "### reference guide (http://holoviews.org/reference/index.html) and see\n", - "### if they align with your intuitions.\n", + "#### Exercise\n", "\n", - "### Additional Exercise: If you have time, try running one of the examples in the\n", - "### 'Streams' section of the HoloViews reference guide\n", - "### (http://holoviews.org/reference/index.html) in this cell" + "Look up the `Ellipse`, `HLine`, and `VLine` elements in the\n", + "[HoloViews reference guide](http://holoviews.org/reference/index.html) and see\n", + "if the definitions of these elements align with your initial intuitions.\n", + "\n", + "#### Exercise (additional)\n", + "\n", + "If you have time, try running one of the examples in the\n", + "'Streams' section of the [HoloViews reference guide](http://holoviews.org/reference/index.html) in the cell below. All the examples in the reference guide should be relatively short and self-contained." ] }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [] + }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Selecting a particular earthquake with the mouse\n", "\n", - "Now we only need two more concept before we can set up the appropriate\n", - "mechanism to select a particular earthquake on the hvplot generated\n", + "Now we only need two more concepts before we can set up the appropriate\n", + "mechanism to select a particular earthquake on the hvPlot-generated\n", "Scatter plot we started with.\n", "\n", "First, we can attach a stream to an existing HoloViews element such as\n", @@ -473,16 +540,21 @@ "(esri * high_mag_quakes.opts(tools=['tap']) * labeller).opts(hv.opts.Scatter(tools=['hover']))" ] }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### Exercise\n", + "\n", + "Pick an earthquake point above and using the displayed index, display the corresponding row of the `most_severe` dataframe using the `.iloc` method in the following cell." + ] + }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], - "source": [ - "### Exercise: Pick an earthquake point above and using the displayed\n", - "### index, display the corresponding row of the `most_severe` dataframe\n", - "### using the `.iloc` method." - ] + "source": [] }, { "cell_type": "markdown", @@ -492,10 +564,10 @@ "\n", "Now we will build a visualization that achieves the following:\n", "\n", - "* The user shall select an earthquake with magnitude `>7` using the tap\n", + "* The user can select an earthquake with magnitude `>7` using the tap\n", " tool in the manner illustrated in the last section.\n", " \n", - "* In addition to the existing label, we will add a crosshair to further highlight the\n", + "* In addition to the existing label, we will add concentric circles to further highlight the\n", " selected earthquake location.\n", "\n", "* *All* earthquakes within 0.5 degrees of latitude and longitude of the\n", @@ -510,7 +582,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "The first step is to generate a crosshair using a very similar approach to the `labeller` above. We can write a function that uses `HLine` and `VLine` to mark a particular earthquake and pass it to a `DynamicMap`:" + "The first step is to generate a concentric-circle marker using a similar approach to the `labeller` above. We can write a function that uses `Ellipse` to mark a particular earthquake and pass it to a `DynamicMap`:" ] }, { @@ -524,17 +596,17 @@ " return hv.Overlay([])\n", " first_index = index[0] # Pick only the first one if multiple are selected\n", " row = most_severe_projected.iloc[first_index]\n", - " return (hv.HLine(y=row.northing).opts(color='white', alpha=0.5) \n", - " * hv.VLine(x=row.easting).opts(color='white', alpha=0.5))\n", + " return ( hv.Ellipse(row.easting, row.northing, 1.5e6).opts(color='white', alpha=0.5)\n", + " * hv.Ellipse(row.easting, row.northing, 3e6).opts(color='white', alpha=0.5))\n", "\n", - "quake_crosshair = hv.DynamicMap(mark_earthquake, streams=[selection_stream])" + "quake_marker = hv.DynamicMap(mark_earthquake, streams=[selection_stream])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "Now we can test this component by building an overlay of the `ESRI` tile source, the `>=7` magnitude points and `quake_crosshair`:" + "Now we can test this component by building an overlay of the `ESRI` tile source, the `>=7` magnitude points and `quake_marked`:" ] }, { @@ -543,7 +615,7 @@ "metadata": {}, "outputs": [], "source": [ - "esri* high_mag_quakes.opts(tools=['tap']) * quake_crosshair" + "esri* high_mag_quakes.opts(tools=['tap']) * quake_marker" ] }, { @@ -588,10 +660,10 @@ "metadata": {}, "outputs": [], "source": [ - "def index_to_selection(index, cache={}):\n", - " if not index: \n", + "def index_to_selection(indices, cache={}):\n", + " if not indices: \n", " return most_severe.iloc[[]]\n", - " index = index[0] # Pick only the first one if multiple are selected\n", + " index = indices[0] # Pick only the first one if multiple are selected\n", " if index in cache: return cache[index]\n", " row = most_severe.iloc[index]\n", " selected_df = earthquakes_around_point(cleaned_reindexed_df, row.latitude, row.longitude)\n", @@ -610,13 +682,59 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "### Linked plots" + "#### Exercise\n", + "\n", + "Test the `index_to_selection` function above for the index you picked in the previous exercise. Note that the stream supplied a *list* of indices and that the function above only uses the first value given in that list. Do the selected rows look correct?:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### Exercise\n", + "\n", + "Convince yourself that the selected earthquakes are within 0.5$^o$ distance of each other in both latitude and longitude." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "
Hint
\n", + "\n", + "For a given `chosen` index, you can see the distance difference using the following code:\n", + "\n", + "```python\n", + "chosen = 235\n", + "delta_long = index_to_selection([chosen]).longitude.max() - index_to_selection([chosen]).longitude.min()\n", + "delta_lat = index_to_selection([chosen]).latitude.max() - index_to_selection([chosen]).latitude.min()\n", + "print(\"Difference in longitude: %s\" % delta_long)\n", + "print(\"Difference in latitude: %s\" % delta_lat)\n", + "```\n", + "\n", + "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ + "### Linked plots\n", + "\n", "So far we have overlayed the display updates on top of the existing\n", "spatial distribution of earthquakes. However, there is no requirement\n", "that the data is overlaid and we might want to simply attach an entirely\n", @@ -650,20 +768,32 @@ "elements explicitly. In this example, `.hvplot.hist` is used." ] }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The exact same principles can be used to build the scatter callback and `temporal_distribution` `DynamicMap`:" + ] + }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ - "## Exercise: Test this function by supplying a list of indices found using the labeller visualization above" + "def scatter_callback(index):\n", + " title = 'Temporal distribution of all magnitudes within half a degree of selection '\n", + " selected_df = index_to_selection(index)\n", + " return selected_df.hvplot.scatter('time', 'mag', color='green', title=title)\n", + "\n", + "temporal_distribution = hv.DynamicMap(scatter_callback, streams=[selection_stream])\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "The exact same principles can be used to build the scatter callback and `temporal_distribution` `DynamicMap`:" + "Lastly, let us define a `DynamicMap` that draws a `VLine` to mark the time at which the selected earthquake occurs so we can see which tremors may have been aftershocks immediately after that major earthquake occurred:" ] }, { @@ -672,18 +802,12 @@ "metadata": {}, "outputs": [], "source": [ - "def scatter_callback(index):\n", - " title = 'Temporal distribution of all magnitudes within half a degree of selection '\n", - " selected_df = index_to_selection(index)\n", - " return selected_df.hvplot.scatter('time', 'mag', color='green', title=title)\n", - "\n", "def vline_callback(index):\n", " if not index:\n", " return hv.VLine(0).opts(alpha=0)\n", " row = most_severe.iloc[index[0]]\n", " return hv.VLine(row.time).opts(line_width=2, color='black')\n", "\n", - "temporal_distribution = hv.DynamicMap(scatter_callback, streams=[selection_stream])\n", "temporal_vline = hv.DynamicMap(vline_callback, streams=[selection_stream])" ] }, @@ -694,6 +818,22 @@ "We now have all the pieces we need to build an interactive, linked visualization of earthquake data." ] }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### Exercise\n", + "\n", + "Test the `histogram_callback` and `scatter_callback` callback functions by supplying your chosen index, remembering that these functions require a list argument in the following cell." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [] + }, { "cell_type": "markdown", "metadata": {}, @@ -714,7 +854,7 @@ "metadata": {}, "outputs": [], "source": [ - "((esri * high_mag_quakes.opts(tools=['tap']) * labeller * quake_crosshair)\n", + "((esri * high_mag_quakes.opts(tools=['tap']) * labeller * quake_marker)\n", " + histogram + temporal_distribution * temporal_vline).cols(1)" ] }, @@ -731,13 +871,15 @@ "source": [ "## Conclusion\n", "\n", - "When exploring data it can be convenient to use the `.plot` API to quickly visualize a particular dataset. By calling `.plot` to generate different plots over the course of a session, it is possible to build up a mental model of how a high dimensional dataset is structured. While this can work well for relatively simple datasets, it can be more efficient to gain insights by building a linked visualization that supports direct user interaction.\n", + "When exploring data it can be convenient to use the `.plot` API to quickly visualize a particular dataset. By calling `.plot` to generate different plots over the course of a session, it is possible to gradually build up a mental model of how a particular dataset is structured. While this works well for simple datasets, it can be more efficient to build a linked visualization with support for direct user interaction as a tool for more rapidly gaining insight.\n", "\n", "In the workflow presented here, building such custom interaction is relatively quick and easy and does not involve throwing away prior code used to generate simpler plots. In the spirit of 'short cuts not dead ends', we can use the HoloViews output of `hvplot` that we used in our initial exploration to build rich visualizations with custom interaction to explore our data at a deeper level. \n", "\n", + "These interactive visualizations not only allow for custom interactions beyond the scope of `hvplot` alone, but they can display visual annotations not offered by the `.plot` API. In particular, we can overlay our data on top of tile sources, generate interactive textual annotations, draw shapes such a circles, mark horizontal and vertical marker lines and much more. Using HoloViews you can build visualizations that allow you to directly interact with your data in a useful and intuitive manner.\n", + "\n", "In this notebook, the earthquakes plotted were either filtered early on by magnitude (`>=7`) or dynamically to analyse only the earthquakes within a small geographic distance. This allowed us to use Bokeh directly without any special handing and without having to worry about the performance issues that would be occur if we were to try to render the whole dataset at once.\n", "\n", - "In the next section we will see how such large datasets can be directly visualized using datashader." + "In the next section we will see how such large datasets can be visualized directly using Datashader." ] } ], diff --git a/examples/tutorial/07_Large_Data.ipynb b/examples/tutorial/07_Large_Data.ipynb index 05f14665..bc283dc3 100644 --- a/examples/tutorial/07_Large_Data.ipynb +++ b/examples/tutorial/07_Large_Data.ipynb @@ -6,16 +6,16 @@ "source": [ "\n", "\n", - "

Tutorial 7. Large datasets

" + "

Tutorial 7. Large data

" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "HoloViews supports even high-dimensional datasets easily, and the standard mechanisms discussed already work well as long as you select a small enough subset of the data to display at any one time. However, some datasets are just inherently large, even for a single frame of data, and cannot safely be transferred for display in any standard web browser. Luckily, HoloViews makes it simple for you to use the separate [Datashader](http://datashader.readthedocs.io) library together with any of the plotting extension libraries, including Bokeh and Matplotlib. Datashader is designed to complement standard plotting libraries by providing faithful visualizations for very large datasets, focusing on revealing the overall distribution, not just individual data points.\n", + "hvPlot and HoloViews support even high-dimensional datasets easily, and the standard mechanisms discussed already work well as long as you select a small enough subset of the data to display at any one time. However, some datasets are just inherently large, even for a single frame of data, and cannot safely be transferred for display in any standard web browser. Luckily, HoloViews makes it simple for you to use the separate [Datashader](http://datashader.readthedocs.io) library together with any of the plotting extension libraries it supports, including Bokeh and Matplotlib. Datashader is designed to complement standard plotting libraries by providing faithful visualizations for very large datasets, focusing on revealing the overall distribution, not just individual data points.\n", "\n", - "Datashader uses computations accelerated using [Numba](http://numba.pydata.org), making it fast to work with datasets of millions or billions of datapoints stored in [Dask](http://dask.pydata.org/en/latest/) dataframes. Dask dataframes provide an API that is functionally equivalent to Pandas, but allows working with data out of core and scaling out to many processors across compute clusters. Here we will use Dask to load a large Parquet-format file of taxi coordinates.\n", + "Datashader uses computations accelerated using [Numba](http://numba.pydata.org), making it fast to work with datasets of millions or billions of datapoints stored in [Dask](http://dask.pydata.org/en/latest/) dataframes. Dask dataframes provide an API that is functionally equivalent to Pandas, but allows working with data out of core and scaling out to many processors across compute clusters. Here we will use Dask to load and visualize the entire earthquake dataset.\n", "\n", "
\n", "\n", @@ -80,7 +80,7 @@ "source": [ "## Load the data\n", "\n", - "As a first step we will load a large dataset using Dask. If you have followed the setup instructions you will have downloaded a large Parquet-format file containing 2 million earthquake events. Let's load this data using dask to create a dataframe ``ddf``:\n", + "As a first step we will load the earthquake dataset, with 2.1 million seismological events. Let's load this data using Dask to create a dataframe ``df``:\n", "\n", "
\n", " If you are low on memory (less than 8 GB) or have not downloaded the full data, you can load only a subset of the data by changing 'data/' to 'data/.data_stubs/'.\n", @@ -103,9 +103,9 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "## Create a dataset\n", + "## Create a HoloViews object\n", "\n", - "In previous sections we have already seen how to declare a set of [``Points``](http://holoviews.org/reference/elements/bokeh/Points.html) from a pandas dataframe. Here we do the same for a Dask dataframe passed in with the desired key dimensions:" + "In previous sections we have already seen how to declare a set of HoloViews [``Points``](http://holoviews.org/reference/elements/bokeh/Points.html) from a Pandas dataframe. Here we do the same for a Dask dataframe passed in with the desired key dimensions:" ] }, { @@ -140,14 +140,14 @@ "metadata": {}, "outputs": [], "source": [ - "datashade(points).opts(width=700, height=500, bgcolor=\"black\")" + "datashade(points).opts(width=700, height=500, bgcolor=\"lightgray\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "If you zoom in you will note that the plot rerenders depending on the zoom level, which allows the full dataset to be explored interactively even though only an image of it is ever sent to the browser. The way this works is that ``datashade`` is a dynamic operation that also declares some linked streams. These linked streams are automatically instantiated and dynamically supply the ``plot_size``, ``x_range``, and ``y_range`` from the Bokeh plot to the operation based on your current viewport as you zoom or pan:" + "The results are the same as if you pass `datashade=True` to hvPlot. If you zoom in you will note that the plot rerenders depending on the zoom level, which allows the full dataset to be explored interactively even though only an image of it is ever sent to the browser. The way this works is that ``datashade`` is a dynamic operation that also declares some linked streams. These linked streams are automatically instantiated and dynamically supply the ``plot_size``, ``x_range``, and ``y_range`` from the Bokeh plot to the operation based on your current viewport as you zoom or pan:" ] }, { @@ -208,7 +208,7 @@ "source": [ "## Aggregating with a variable\n", "\n", - "So far we have simply been counting earthquakes, but our dataset is much richer than that. We have information about a number of variables, as listed above. Datashader provides a number of ``aggregator`` functions, which you can supply to the datashade operation. Here use the ``ds.mean`` aggregator to compute the average magnitude at each location, for events with a depth below zero:" + "So far we have simply been counting earthquakes, but our dataset is much richer than that. We have information about a number of variables, as listed above. Datashader provides a number of ``aggregator`` functions, which you can supply to the datashade operation. Here we use the ``ds.mean`` aggregator to compute the average magnitude at each location, for events with a depth below zero:" ] }, { @@ -267,12 +267,14 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "As you can see, datashader requires taking some extra steps into consideration, but it makes it practical to work with even quite large datasets on an ordinary laptop. On a 16GB machine, datasets 100X or 1000X the one used here should be very practical, as illustrated at the [datashader web site](https://datashader.org).\n", + "As you can see, Datashader requires taking some extra steps into consideration, but it makes it practical to work with even quite large datasets on an ordinary laptop. On a 16GB machine, datasets 100X or 1000X the one used here should be very practical, as illustrated at the [datashader web site](https://datashader.org).\n", + "\n", + "Here the examples all use point data, but Datashader also supports raster data as shown in earlier sections, along with many other data types (lines, time series, trajectories, areas, trimeshes, quadmeshes, networks, etc.) \n", "\n", "\n", "# Onwards\n", "\n", - "* The [HoloViews Large Data](http://holoviews.org/user_guide/Large_Data.html) user guide explains in more detail how to work with large datasets using datashader.\n", + "* The [HoloViews Large Data](http://holoviews.org/user_guide/Large_Data.html) user guide explains in more detail how to work with large datasets using Datashader.\n", "* HoloViews also contains a [sample bokeh app](http://holoviews.org/gallery/apps/bokeh/nytaxi_hover.html) using this dataset and an additional linked stream that works well as a starting point." ] } diff --git a/examples/tutorial/08_Advanced_Dashboards.ipynb b/examples/tutorial/08_Advanced_Dashboards.ipynb index ea7f0085..67cbbcd9 100644 --- a/examples/tutorial/08_Advanced_Dashboards.ipynb +++ b/examples/tutorial/08_Advanced_Dashboards.ipynb @@ -4,7 +4,16 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "At this point we have learned how to build interactive apps and dashboards with Panel, how to quickly build visualizations with hvPlot and add custom interactivity by using HoloViews. In this section we will work on putting all of this together to build complex, and efficient data processing pipelines, controlled by Panel widgets." + "\n", + "\n", + "

Tutorial 8. Advanced Dashboards

" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "At this point we have learned how to build interactive apps and dashboards with Panel, how to quickly build visualizations with hvPlot, and add custom interactivity by using HoloViews. In this section we will work on putting all of this together to build complex, and efficient data processing pipelines, controlled by Panel widgets." ] }, { @@ -20,7 +29,7 @@ "import panel as pn\n", "import xarray as xr\n", "\n", - "import hvplot.pandas\n", + "import hvplot.pandas # noqa: API import\n", "import hvplot.xarray # noqa: API import\n", "\n", "pn.extension()" @@ -68,7 +77,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "In the previous section we built a little function to cache the closest earthquakes since the computation can take a little while. An alternative to this approach is to start building a pipeline in HoloViews to do this very thing. Instead of writing a function that operates directly on the data we rewrite the function to accept a Dataset and the index. This function again filters the closest earthquakes within the region and returns a new Dataset:" + "In the previous sections we built a little function to cache the closest earthquakes since the computation can take a little while. An alternative to this approach is to start building a pipeline in HoloViews to do this very thing. Instead of writing a function that operates directly on the data, we rewrite the function to accept a Dataset and the index. This function again filters the closest earthquakes within the region and returns a new Dataset:" ] }, { @@ -195,7 +204,7 @@ "source": [ "## Connecting widgets to the pipeline\n", "\n", - "At this point you may be thinking that we haven't done anything we haven't already seen in the previous section. However, apart from automatically handling the caching of computations, building visualization pipelines in this way provides one major benefit - we can inject parameters at any stage of the pipeline. These parameters can come from anywhere including from panel widgets, this way we can expose control over any aspect of our pipeline. \n", + "At this point you may be thinking that we haven't done anything we haven't already seen in the previous sections. However, apart from automatically handling the caching of computations, building visualization pipelines in this way provides one major benefit - we can inject parameters at any stage of the pipeline. These parameters can come from anywhere including from Panel widgets, allowing us to expose control over any aspect of our pipeline. \n", "\n", "You may have noticed that the ``earthquakes_around_point`` function takes two arguments, the ``index`` of the point **and** the ``degrees_dist``, which defines the size of the region around the selected earthquake we will select points in. Using ``.apply`` we can declare a ``FloatSlider`` widget and then inject its ``value`` parameter into the pipeline (ensure that an earthquake is selected in the map above):" ] @@ -220,7 +229,7 @@ "source": [ "When the widget value changes the pipeline will re-execute the part of the pipeline downstream from the function and update the plot. This ensures that only the parts of the pipeline that are actually needed are re-executed.\n", "\n", - "The ``.apply`` method can also used to apply options depending on some widget value, e.g. we can create a colormap selector and then use ``.apply.opts`` to connect it to the ``rasterized_pop`` plot:" + "The ``.apply`` method can also be used to apply options depending on some widget value, e.g. we can create a colormap selector and then use ``.apply.opts`` to connect it to the ``rasterized_pop`` plot:" ] }, { @@ -288,9 +297,9 @@ "source": [ "## Connecting panels to streams\n", "\n", - "At this point we have learned how to connect parameters on Panel objects to a pipeline. Earlier we also learned how we can use parameters to declare dynamic Panel components, so this section should be nothing new, we will simply try to connect the index parameter of the selection stream to a panel to try to compute the amount of people in the region around an earthquake.\n", + "At this point we have learned how to connect parameters on Panel objects to a pipeline and we earlier learned how we can use parameters to declare dynamic Panel components. So, this section should be nothing new;, we will simply try to connect the index parameter of the selection stream to a panel to try to compute the number of people in the region around an earthquake.\n", "\n", - "Since we have a population density dataset we can approximate how many people are affected by a particular earthquake (note this a very rough approximation and not quite valid since the population density reflects a particular date). " + "Since we have a population density dataset we can approximate how many people are affected by a particular earthquake. Of course, this value is only a rough approximation, as it ignores the curvature of the earth, assumes isotropic spreading of the earthquake, and assumes that the population did not change between the measurement and the earthquake. " ] }, { @@ -322,7 +331,7 @@ "\n", "dynamic_bounds = hv.DynamicMap(bounds, streams=[index_stream, dist_slider.param.value])\n", "\n", - "pn.Column(pn.panel(affected_population, width=400), rasterized_pop * high_mag_points * dynamic_bounds, dist_slider)" + "pn.Column(pn.panel(affected_population, width=400), rasterized_pop * high_mag_points * dynamic_bounds, dist_slider).servable()" ] } ], diff --git a/examples/tutorial/exercises/Advanced_Dashboarding.ipynb b/examples/tutorial/exercises/Advanced_Dashboarding.ipynb index 1cd82a7e..9b00a81e 100644 --- a/examples/tutorial/exercises/Advanced_Dashboarding.ipynb +++ b/examples/tutorial/exercises/Advanced_Dashboarding.ipynb @@ -12,8 +12,63 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "
WORK IN PROGRESS: We are in the progress of updating these materials in anticipation of a tutorial at the 2019 SciPy conference. Work will be complete by the morning of July 8th 2019. Check out this tag to access the materials as they were before these changes started. For the latest version of the tutorial, visit holoviz.org.\n", - "
" + "This exercise is entirely freeform. Get into groups of 3-4 people and start building a dashboard, with everything you have learned in this tutorial. By the end of the exercise you should have a dashboard that:\n", + "\n", + "* Uses datashading to render the whole dataset\n", + "* Builds a pipeline using the ``.apply`` method \n", + "* Uses a widget to filter the data either by cross-filtering with a linked stream (e.g. a BoundsXY stream) or using a widget (e.g. a RangeSlider)\n", + "* Uses a widget to control some aspect of the styling of the plot (e.g. to select a colormap, color, or size)\n", + "* Is servable by running ``panel serve Advanced_Dashboarding.ipynb`` in the exercise directory" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "import colorcet as cc # noqa\n", + "import holoviews as hv # noqa\n", + "import numpy as np # noqa\n", + "import dask.dataframe as dd\n", + "import panel as pn\n", + "import xarray as xr\n", + "\n", + "import hvplot.pandas # noqa: API import\n", + "import hvplot.xarray # noqa: API import\n", + "\n", + "pn.extension()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "As a starting point we will load the data; everything else is up to you:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "df = dd.read_parquet('../../data/earthquakes.parq').repartition(npartitions=4).persist()\n", + "\n", + "ds = xr.open_dataarray('../../data/gpw_v4_population_density_rev11_2010_2pt5_min.nc')\n", + "cleaned_ds = ds.where(ds.values != ds.nodatavals).sel(band=1)\n", + "cleaned_ds.name = 'population'" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "You don't really know what to build? Here are some ideas:\n", + " \n", + "* Build a dashboard with a pipeline that filters the data on one or more of the columns (e.g. magnitude using a ``RangeSlider`` or time using a ``DateRangeSlider``) and then datashades it\n", + "* Build a dashboard with multiple views of the data (e.g. longitude vs. latitude, magnitude vs. depth etc.) and cross-filters the data using ``BoundsXY`` streams (see the [Glaciers notebook](https://github.com/pyviz-demos/glaciers/blob/master/glaciers.ipynb) for reference)\n", + "* Build a dashboard that allows you to select multiple earthquakes using a 'box_select' tool and a ``Selection1D`` stream and compute statistics on them." ] } ], diff --git a/examples/tutorial/exercises/Building_a_Dashboard.ipynb b/examples/tutorial/exercises/Building_a_Dashboard.ipynb index 96427674..453ba231 100644 --- a/examples/tutorial/exercises/Building_a_Dashboard.ipynb +++ b/examples/tutorial/exercises/Building_a_Dashboard.ipynb @@ -8,14 +8,6 @@ "

Exercises 1-2: Building a Dashboard

" ] }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "
WORK IN PROGRESS: We are in the progress of updating these materials in anticipation of a tutorial at the 2019 SciPy conference. Work will be complete by the morning of July 8th 2019. Check out this tag to access the materials as they were before these changes started. For the latest version of the tutorial, visit holoviz.org.\n", - "
" - ] - }, { "cell_type": "code", "execution_count": null, @@ -46,7 +38,7 @@ "source": [ "#### The data\n", "\n", - "Throughout this tutorial we will be working with one core dataset, a collection of earthquakes recorded between 2000-2018 provided by the US Geological Survey (USGS). The data is provided as a parquet file as part of the tutorial and we will load it using dask and persist it. We will return to this later, for now we will focus on building a dashboard and you don't know any of the details about the dataset or the dask or pandas API." + "Throughout this tutorial we will be working with one core dataset, a collection of earthquakes recorded between 2000-2018 provided by the US Geological Survey (USGS). The data is provided as a Parquet file as part of the tutorial and we will load it using Dask and persist it. We will return to this later; for now we will focus on building a dashboard and you don't know any of the details about the dataset or the Dask or Pandas API." ] }, { @@ -68,7 +60,7 @@ "source": [ "#### The logo\n", "\n", - "The first component of the dashboard is an image of the US Geological Survey logo, start by declaring a pane containing the logo and assign it to the ``logo`` variable. Also set a width to ensure the logo doesn't appear to big." + "The first component of the dashboard is an image of the US Geological Survey logo. Start by declaring a pane containing the logo and assign it to the ``logo`` variable. Also set a width to ensure the logo doesn't appear too big." ] }, { @@ -216,9 +208,9 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "#### A plot\n", + "#### A plot [challenge]\n", "\n", - "If you are up to it create a custom plot from the ``year_df`` dataframe defined below, create a panel component and assign it to the ``plot`` variable.\n", + "If you are up to it, create a custom plot from the ``year_df`` dataframe defined below, create a Panel component, and assign it to the ``plot`` variable.\n", "\n" ] }, @@ -253,8 +245,10 @@ "source": [ "
Solution
\n", "\n", + "This example solution uses concepts covered in the plotting section of the tutorial:\n", + "\n", "```python\n", - "plot = hv.Violin(year_df, 'type', 'mag').opts(ylim=(0, None))\n", + "plot = hv.Violin(year_df, 'type', 'mag').opts(ylim=(0, None), xrotation=90)\n", "```\n", "\n", "
" @@ -414,7 +408,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "#### The Plot\n", + "#### The Plot [challenge]\n", "\n", "In case you defined a plot above make the plot dynamic by wrapping it in a function which depends on the year." ] @@ -436,7 +430,7 @@ "@pn.depends(year_slider.param.value)\n", "def plot_fn(year)\n", " year_df = df[df.time.dt.year == year].compute()\n", - " return hv.Violin(year_df, 'type', ('mag', 'Magnitude'))\n", + " return hv.Violin(year_df, 'type', ('mag', 'Magnitude')).opts(xrotation=90)\n", "```" ] }, @@ -486,7 +480,7 @@ " \"\"\".format(lat=lat, lon=lon), height=300, width=300)\n", "\n", "@pn.depends(year_slider.param.value)\n", - "def plot_fn(year)\n", + "def plot_fn(year):\n", " year_df = df[df.time.dt.year == year].compute()\n", " return hv.Violin(year_df, 'type', ('mag', 'Magnitude'))\n", " \n", diff --git a/examples/tutorial/exercises/Plotting.ipynb b/examples/tutorial/exercises/Plotting.ipynb index 9271f435..59ccaad1 100644 --- a/examples/tutorial/exercises/Plotting.ipynb +++ b/examples/tutorial/exercises/Plotting.ipynb @@ -5,35 +5,112 @@ "metadata": {}, "source": [ "\"HoloViz\n", - "

Exercises 3-5: Plotting

" + "

Exercises 3-4: Plotting

" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "
WORK IN PROGRESS: We are in the progress of updating these materials in anticipation of a tutorial at the 2019 SciPy conference. Work will be complete by the morning of July 8th 2019. Check out this tag to access the materials as they were before these changes started. For the latest version of the tutorial, visit holoviz.org.\n", - "
" + "### Exercise 3\n", + "\n", + "In this exercise we will explore hvplot some more which we will build on in Exercise 4 to create a custom linked visualization.\n", + "\n", + "#### Loading the data as before\n", + "\n", + "We will be building a new visualization based on the same data we have cleaned and filtered in the rest of the tutorial. First we load the `DataFrame` of the `>=7` earthquakes:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "import xarray as xr\n", + "import dask.dataframe as dd\n", + "import holoviews as hv\n", + "from holoviews import streams # noqa\n", + "import hvplot.dask # noqa\n", + "import hvplot.pandas # noqa\n", + "import hvplot.xarray # noqa: adds hvplot method to xarray objects\n", + "\n", + "df = dd.read_parquet('../../data/earthquakes.parq')\n", + "cleaned_df = df.copy()\n", + "cleaned_df['mag'] = df.mag.where(df.mag > 0)\n", + "cleaned_reindexed_df = cleaned_df.set_index(cleaned_df.time)\n", + "cleaned_reindexed_df = cleaned_reindexed_df.persist()\n", + "most_severe = cleaned_reindexed_df[cleaned_reindexed_df.mag >= 7].compute()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "### Exercise 3\n", + "And next we load the population density raster data:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "ds = xr.open_dataarray('../../data/gpw_v4_population_density_rev11_2010_2pt5_min.nc')\n", + "cleaned_ds = ds.where(ds.values != ds.nodatavals).sel(band=1)\n", + "cleaned_ds.name = 'population'" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### Visualizing the raster data\n", + "\n", + "Start by using `hvplot.image` to visualize the whole of the population density data using the Datashader support in hvPlot. Grab a handle on this `Image` HoloViews object called `pop_density` and customize it using the `.opts` method, enabling a logarithmic color scale and a blue colormap (specifically the `'Blues'` colormap). At the end of the cell, display this object.\n", + "\n", + "
Hint
\n", + "\n", + "Don't forget to include `rasterize=True` in the `hvplot.image` call. You can use `logz=True` and `cmap='Blues'` in the `.opts` method call\n", + "\n", + "
" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "pop_density = ... # Use hvplot here to visualize the data in cleaned_ds and customize it with .opts\n", + "pop_density # Display it" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "
Solution
\n", "\n", - "Add some visualizations via the `.plot` API to the dashboard." + "```python\n", + "pop_density = cleaned_ds.hvplot.image(rasterize=True).opts(logz=True, cmap='Blues') \n", + "pop_density\n", + "```\n", + "\n", + "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "Start by reading in the data from the parquet file.\n", + "#### Visualizing the earthquake positions\n", "\n", - "
Hint
\n", + "Now visualize the tabular data in `most_severe` by building a `hv.Points` object directly. This will be very similar to the approach shown in the tutorial but this time we want all the earthquakes to be marked with red crosses of size 100 (no need to use the `.opts` method this time). As above, get a handle on this object and display it where the handle is now called `quake_points`:\n", "\n", - "Use tab complete to find the `read_parquet` method in the `dask.dataframe` module (`dd`)\n", + "
Hint
\n", + "\n", + "Don't forget to map the longitude and latitude dimensions to `x` and `y` in the call to `hvplot.points`.\n", "\n", "
" ] @@ -43,7 +120,10 @@ "execution_count": null, "metadata": {}, "outputs": [], - "source": [] + "source": [ + "quake_points = ... # Use hvplot here to visualize the data in most_severe and customize it with .opts\n", + "quake_points" + ] }, { "cell_type": "markdown", @@ -52,20 +132,53 @@ "
Solution
\n", "\n", "```python\n", - "import dask.dataframe as dd\n", + "quake_points = most_severe.hvplot.points(x='longitude', y='latitude', marker='+', size=100, color='red')\n", + "quake_points \n", + "```\n", "\n", - "df = dd.read_parquet('../../data/earthquakes.parq')\n", + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### Enabling the `box_select` tool\n", + "\n", + "Now use `.opts` method to enable the Bokeh `box_select` tool on quake points.\n", + "\n", + "
Hint
\n", + "\n", + "The option is called `tools` and takes a list of tool names, in this case `'box_select'`.\n", + "\n", + "
" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "
Solution
\n", + "\n", + "```python\n", + "quake_points.opts(tools=['tap'])\n", "```\n", "\n", "
\n", "\n", - "Make a datashaded scatter of all the earthquakes. Call this plot `earthquake_scatter`.\n", "\n", - "
Hint
\n", "\n", - "Don't forget to include `datashade=True` and to import `hvplot.dask`\n", "\n", - "
" + "#### Composing these visualizations together\n", + "\n", + "Now overlay `quake_points` over `pop_density` to check everything looks correct. Make sure the box select tool is working as expected." ] }, { @@ -82,19 +195,49 @@ "
Solution
\n", " \n", "```python\n", - "import hvplot.dask\n", - " \n", - "earthquake_scatter = df.hvplot.scatter(x='longitude', y='latitude', datashade=True)\n", - "earthquake_scatter\n", + "pop_density * quake_points\n", "```\n", + "\n", + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Exercise 4\n", + "\n", + "Using `Selection1D`, define a HoloViews stream called `selection_stream` using `quake_points` as a source." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "selection_stream = ..." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "
Solution
\n", " \n", + "```python\n", + "selection_stream = streams.Selection1D(source=quake_points)\n", + "```\n", + "\n", "
\n", "\n", - "Include this plot in the panel layout that you build in Exercises 1 and 2.\n", + "#### Highlighting earthquakes with circles\n", + "\n", + "Now we want to create a circle around *all* the selected points chosen by the box select tool where each circle is centered at the latitude and longitude of a selected earthquake (10`^o` in diameter). A `hv.Ellipse` object can create a circle using the format `hv.Ellipse(x, y, diameter)` and we can build an overlay of circles from a list of `circles` using `hv.Overlay(circles)`. Using this information, complete the following callback for the `circle_marker` `DynamicMap`.\n", "\n", - "
Hint
\n", + "
Hint
\n", "\n", - "The panel object is indexable just like a list or a DataFrame, so just replace the item in. Call `print(dashboard)` to inspect the hierarchy\n", + "Each `circle` needs to be a `hv.Ellipse(longitude, latitude, 10)` where longitude and latitude correspond to the current earthquake row.\n", "\n", "
" ] @@ -104,7 +247,21 @@ "execution_count": null, "metadata": {}, "outputs": [], - "source": [] + "source": [ + "def circles_callback(indices):\n", + " circles = []\n", + " if len(indices) == 0:\n", + " return hv.Overlay([])\n", + " \n", + " for index in indices:\n", + " row = most_severe.iloc[index] # noqa\n", + " circle = ... # Define the appropriate Ellipse element here\n", + " circles.append(circle)\n", + " return hv.Overlay(circles)\n", + "\n", + "# Uncomment when the above function is complete\n", + "# circle_marker = hv.DynamicMap(circles_callback, streams=[selection_stream])" + ] }, { "cell_type": "markdown", @@ -113,23 +270,77 @@ "
Solution
\n", "\n", "```python\n", - "dashboard[0] = earthquake_scatter\n", - "dashboard\n", + "def circles_callback(indices):\n", + " circles = []\n", + " if len(indices) == 0:\n", + " return hv.Overlay([])\n", + " \n", + " for index in indices:\n", + " row = most_severe.iloc[index]\n", + " circle = hv.Ellipse(row.longitude, row.latitude, 10) \n", + " circles.append(circle)\n", + " \n", + " return hv.Overlay(circles)\n", + "\n", + "circle_marker = hv.DynamicMap(circles_callback, streams=[selection_stream])\n", "```\n", "\n", "
\n", "\n", - "You should now have a datashaded scatter plot that shows up in your dashboard.\n", "\n", - "
I don't
\n", + "Now test this works by overlaying `pop_density`, `quake_points`, and `circle_marker` together." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "
Solution
\n", + " \n", + "```python\n", + "pop_density * quake_points * circle_marker\n", + "```\n", + "
\n", + "\n", + "\n", + "#### Depth and magnitude scatter of selected earthquakes\n", + "\n", + "Now let us generate a scatter plot of depth against magnitude for the selected earthquakes. Define a `DynamicMap` called `depth_magnitude_scatter` that uses a callback called `depth_magnitude_callback`. This is a two-line function that returns a `Scatter` element generated by `.hvplot.scatter`.\n", + "\n", + "
Hint
\n", + "\n", + "The `index` argument of the callback can be passed straight to `most_severe.iloc` to get a filtered dataframe corresponding to the selected earthquakes.\n", + "\n", + "
\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "\n", + "
Solution
\n", "\n", "```python\n", - "import dask.dataframe as dd\n", - "import hvplot.dask\n", + "def depth_magnitude_callback(index):\n", + " selected = most_severe.iloc[index]\n", + " return selected.hvplot.scatter(x='mag', y='depth')\n", "\n", - "earthquake_scatter = df.hvplot.scatter(x='longitude', y='latitude', datashade=True)\n", - "dashboard = pn.Column('## Earthquakes', earthquake_scatter)\n", - "dashboard\n", + "depth_magnitude_scatter = hv.DynamicMap(depth_magnitude_callback, streams=[selection_stream])\n", "```\n", "\n", "
" @@ -139,18 +350,69 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "### Exercise 4\n", + "### Final visualization\n", + "\n", + "Now overlay `pop_density`, `quake_points` and `circle_marker` and put this in a one column layout together with the linked plot, `depth_magnitude_scatter`." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "
Solution
\n", + "\n", + "```python\n", + "((pop_density * quake_points * circle_marker) + depth_magnitude_scatter).cols(1)\n", + "```\n", "\n", - "Add a linked visualization with holoviews." + "
\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "### Exercise 5\n", + "
Overall Solution
\n", "\n", - "Add a datashaded visualization to the dashboard." + "After loading the data, the following (slightly cleaned up) version generates the whole visualization:\n", + "\n", + "```python\n", + "pop_density = cleaned_ds.hvplot.image(rasterize=True).opts(logz=True, cmap='Blues') \n", + "quake_points = most_severe.hvplot.points(x='longitude', y='latitude', marker='+', size=100, color='red')\n", + "quake_points.opts(tools=['box_select'])\n", + "\n", + "selection_stream = streams.Selection1D(source=quake_points)\n", + "\n", + "def circles_callback(index):\n", + " circles = []\n", + " if len(index) == 0:\n", + " return hv.Overlay([])\n", + " for ind in index:\n", + " row = most_severe.iloc[ind]\n", + " circle = hv.Ellipse(row.longitude, row.latitude, 10) # Define the appropriate Ellipse element here\n", + " circles.append(circle)\n", + " \n", + " return hv.Overlay(circles)\n", + "\n", + "circle_marker = hv.DynamicMap(circles_callback, streams=[selection_stream])\n", + "\n", + "def depth_magnitude_callback(index):\n", + " selected = most_severe.iloc[index]\n", + " return selected.hvplot.scatter(x='mag', y='depth')\n", + "\n", + "depth_magnitude_scatter = hv.DynamicMap(depth_magnitude_callback, streams=[selection_stream])\n", + "\n", + "((pop_density * quake_points * circle_marker) + depth_magnitude_scatter).cols(1)\n", + "```\n", + "\n", + "
" ] } ], diff --git a/examples/tutorial/index.ipynb b/examples/tutorial/index.ipynb index 052484eb..b91371fa 100644 --- a/examples/tutorial/index.ipynb +++ b/examples/tutorial/index.ipynb @@ -9,14 +9,6 @@ "

Tutorial Index

" ] }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "
WORK IN PROGRESS: We are in the progress of updating these materials in anticipation of a tutorial at the 2019 SciPy conference. Work will be complete by the morning of July 8th 2019. Check out this tag to access the materials as they were before these changes started. For the latest version of the tutorial, visit holoviz.org.\n", - "
" - ] - }, { "cell_type": "markdown", "metadata": {}, @@ -32,7 +24,7 @@ "[Param](http://param.pyviz.org), and \n", "[Colorcet](http://colorcet.pyviz.org).\n", "\n", - "\n", + "\n", "\n", "These tools were previously part of [PyViz.org](http://pyviz.org), but have been pulled out into [HoloViz.org](http://holoviz.org) to allow PyViz to be fully neutral and general.\n", "\n", @@ -40,7 +32,7 @@ "\n", "\n", "\n", - "This notebook serves as the homepage of the tutorial, including a table of contents listing each tutorial section." + "This notebook serves as the homepage of the tutorial, including a table of contents letting you launch each tutorial section." ] }, { @@ -51,38 +43,36 @@ "\n", "- **Introduction and setup**\n", " *   **5 min**  [Setup](./00_Setup.ipynb): Setting up the environment and data files.\n", - " * **15 min**  [Overview](./01_Overview.ipynb): Overview of the main HoloViz tools, philosophy, and approach.\n", - "\n", + " * **20 min**  [Overview](./01_Overview.ipynb): Overview of the HoloViz tools, philosophy, and approach.\n", "\n", "- **Building dashboards using Panel**\n", - " * **20 min**  [Panel](./02_Introduction_to_Panel.ipynb): How to make apps and dashboards from Python objects.\n", + " * **15 min**  [Building_Panels](./02_Building_Panels.ipynb): How to make apps and dashboards from Python objects.\n", " *   **5 min**  [*Exercise 1*](./exercises/Building_a_Dashboard.ipynb#Exercise-1): Using a mix of visualizable types, create a panel and serve it.\n", - " * **10 min**  [Panel_Interactivity](./03_Panel_Interactivity.ipynb): Connecting code to widgets using `@depends`.\n", + " * **10 min**  [Interlinked Panels](./03_Interlinked_Panels.ipynb): Customizing linkages between widgets and displayable objects.\n", " *   **5 min**  [*Exercise 2*](./exercises/Building_a_Dashboard.ipynb#Exercise-2): Add widgets to control your dashboard.\n", " * **10 min**  *Break*\n", " \n", " \n", "- **The `.plot` API: a data-centric approach to visualization** \n", - " * **30 min**  [Using .plot](./04_Using_.plot.ipynb): Quick introduction to the `.plot` interface.\n", + " * **30 min**  [Basic Plotting](./04_Basic_Plotting.ipynb): Quick introduction to the `.plot` interface.\n", " * **10 min**  [*Exercise 3*](./exercises/Plotting.ipynb#Exercise-3): Add some `.plot` or `.hvplot` visualizations to your dashboard.\n", - " * **10 min**  [Composing Plots](./05_Composing_Holoviews_Plots.ipynb): Overlaying and laying out `.hvplot` outputs to show relationships.\n", + " * **10 min**  [Composing Plots](./05_Composing_Plots.ipynb): Overlaying and laying out `.hvplot` outputs to show relationships.\n", " * **10 min**  *Break*\n", "\n", " \n", "- **Custom interactivity**\n", - " * **25 min**  [Custom Interactivity](./06_Custom_Interactivity.ipynb): Connecting HoloViews \"streams\" for customizing behavior.\n", + " * **25 min**  [Interlinked Plots](./06_Interlinked_Plots.ipynb): Connecting HoloViews \"streams\" to customize behavior.\n", " * **10 min**  [*Exercise 4*](./exercises/Plotting.ipynb#Exercise-4): Add a linked visualization with HoloViews.\n", "\n", "\n", "- **Working with large datasets**\n", - " * **20 min**  [Working with Large Datasets](./07_Large_Datasets.ipynb): Using Datashader to pre-render data in Python\n", - " * **10 min**  [*Exercise 5*](./exercises/Plotting.ipynb#Exercise-5): Add a datashaded visualization to your dashboard.\n", + " * **20 min**  [Large Data](./07_Large_Data.ipynb): Using Datashader to pre-render data in Python\n", " * **10 min**  *Break*\n", "\n", "\n", "- **Building advanced dashboards**\n", - " * **15 min**  [Advanced Dashboards](./08_Advanced_Dashboards.ipynb): How to use Panel to create an advanced dashboard with linked plots and streams.\n", - " * **20 min**  [*Exercise 6*](./exercises/Advanced_Dashboarding.ipynb): Build a new dashboard using everything you've learned so far." + " * **15 min**  [Advanced Dashboards](./08_Advanced_Dashboards.ipynb): Using Panel to create an advanced dashboard with linked plots and streams.\n", + " * **30 min**  [*Exercise 5*](./exercises/Advanced_Dashboarding.ipynb): Build a new dashboard using everything you've learned so far." ] }, {