sktime_worksheet_TSC.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# sktime worksheet - time seriesclassification\n",
    "\n",
    "This is an exercise style worksheet introducing the [sktime](https://github.com/alan-turing-institute/sktime) time series classification module, for self-study and user testing.\n",
    "\n",
    "This worksheet contains exercises for you to attempt after:\n",
    "* going through the basic `sktime` demo, live or [on youtube](https://www.youtube.com/watch?v=wqQKFu41FIw)\n",
    "* having had a look at the tutorial notebooks yourself.\n",
    "\n",
    "For this sheet, you are actively encouraged to seek out the sktime documentation and tutorial notebooks to solve the exercises.\n",
    "\n",
    "We would very much appreciate if you could leave feedback, especially critical feedback, in the markdown fields meant for this. You can also raise issues on any bugs or improvement suggestions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise 1: loading datasets\n",
    "\n",
    "(a) use `sktime`'s inbuilt data handling functionality to load the inline skates data set from timeseriesclassification.com\n",
    "\n",
    "(b) convert the feature data frame into 3d numpy format\n",
    "\n",
    "(c) convert the feature data frame into long pandas data frame format\n",
    "\n",
    "(d) convert the feature data frame into nested pandas data frame format"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "source": [
    "Questions:\n",
    "* was it easy/difficult to find the function to load the dataset?\n",
    "* was it easy/difficult to find the inline skates data set, and how to load it?\n",
    "* were the data type conversions easy/difficult to find and carry out?"
   ],
   "cell_type": "markdown",
   "metadata": {}
  },
  {
   "source": [
    "### Exercise 2: basic time series classification\n",
    "\n",
    "(a) randomly split the inline skates data set into a uniformly random 80/20 train/test split\n",
    "\n",
    "(b) train a HIVE-COTE classifier with default parameters on the training set, predict the classes on the test set\n",
    "\n",
    "(c) evaluate the result using classification accuary and balanced F1 scores"
   ],
   "cell_type": "markdown",
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "source": [
    "Questions:\n",
    "* was it easy/difficult to find the classifier?\n",
    "* was it easy/difficult to apply the classifier?\n",
    "* was it easy/difficult to understand how to evaluate the classifier?"
   ],
   "cell_type": "markdown",
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise 3: composite time series classifiers\n",
    "\n",
    "(a) build the following time series classifier: a k-nearest neighbor classifier, where both time series distance and the parameter k are tuned using cross-validation grid search\n",
    "\n",
    "(b) use the classifier in (a) and run the evaluation from exercise 2\n",
    "\n",
    "(c) build the following time series classifier: a pipeline consisting of a feature extractor that computes mean, variance, and five quartiles for each time series. A grid search tuned support vector classifier (for tabular data, from `sklearn`) is then used to predict the label.\n",
    "\n",
    "(d) use the classifier in (c) to obtain probabilistic class predictions on a uniformly random 80/20 split, and evaluate using the logartithmic loss\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "source": [
    "Questions:\n",
    "* was it easy/difficult to find out how to tune classifiers?\n",
    "* was it easy/difficult to find out how to chain feature extraction and a sklearn classifier?\n",
    "* was it easy/difficult to understand how to obtain and evaluate probabilistic predictions?"
   ],
   "cell_type": "markdown",
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise 4: Multivariate time series classification\n",
    "\n",
    "(a) load any multivariate time series classification set from timeseriesclassification.com which is not the basic motions dataset.\n",
    "\n",
    "(b) find, build, and evaluate a multivariate time series classifier on that data set\n",
    "\n",
    "(c) use grid search to tune which column-wise classifiers are used in a column ensemble classifier"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "source": [
    "Questions:\n",
    "* was it easy/difficult to find out material on multivariate time series classification?\n",
    "* was it easy/difficult to find out how to load multivariate time series classification data sets?\n",
    "* was it easy/difficult to tune over classifier choice?"
   ],
   "cell_type": "markdown",
   "metadata": {}
  },
  {
   "source": [
    "### General feedback:\n",
    "* What was the most difficult part to understand?\n",
    "* Did you get stuck anywhere or had to skip? \n",
    "* How can we improve the documentation? \n",
    "* What additional features would you like to see?"
   ],
   "cell_type": "markdown",
   "metadata": {}
  }
 ],
 "metadata": {
  "kernelspec": {
   "name": "python3",
   "display_name": "Python 3.7.10 64-bit ('sktime': conda)",
   "metadata": {
    "interpreter": {
     "hash": "1d08445f682e67118652b541d82c54c8d542bbf9d181dd3acc4827a948ac2085"
    }
   }
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}