r/lecture2-discovery.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": []
   },
   "source": [
    "This notebook is part of a course on Applied Process Mining. The collection of notebooks is a *living document* and subject to change.\n",
    "\n",
    "# Lecture 2 - 'Process Discovery with the Heuristics Miner' (R / bupaR)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Setup\n",
    "\n",
    "<img src=\"http://bupar.net/images/logo_text.PNG\" alt=\"bupaR\" style=\"width: 200px;\"/>\n",
    "\n",
    "In this notebook, we are going to need the `tidyverse` and the `bupaR` packages."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "## Perform the commented out commands below in a separate R session\n",
    "# install.packages(\"tidyverse\")\n",
    "# install.packages(\"bupaR\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# for larger and readable plots\n",
    "options(jupyter.plot_scale=1.25)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# the initial execution of these may give you warnings that we can safely ignore\n",
    "suppressPackageStartupMessages(library(tidyverse)) \n",
    "suppressPackageStartupMessages(library(bupaR))\n",
    "library(xesreadR)\n",
    "library(processanimateR)\n",
    "suppressPackageStartupMessages(library(heuristicsmineR))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Process Discovery"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Process Models 🚧\n",
    "\n",
    "bupaR does not provide an option to load BPMN models yet. Please have a look at the PM4Py instructions."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Quality Dimensions\n",
    "\n",
    "The quality dimensions `fitness`, `precision`, `simplicity`, and `generalisation` are best illustrated by using a small example event log.\n",
    "We are using an example event log in XES format that is used in the book `Process Mining - Data Science in Action` by Wil van der Aalst, which is downloaded and stored in the `../data` directory with the code below:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# ignore the warnings, the package needs to be updated and no 'activity instance identifier' is required in this example\n",
    "example_log <- xesreadR::read_xes(\"../data/Lfull.xes\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let us have a look at the event log in tabular form. The mapping of the activity labels to actual activities is:\n",
    "\n",
    "* a = register request, \n",
    "* b = examine thoroughly, \n",
    "* c = examine casually, \n",
    "* d = check ticket, \n",
    "* e = decide, \n",
    "* f = reinitiate request, \n",
    "* g = pay compensation, and \n",
    "* h = reject request."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "example_log %>% head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let us discover a process map as we have seen in Lecture 1:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "example_log %>% process_map()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Not really very insightful the directly-follows based process map visualization."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Heuristics Miner"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### L_heur_1 Event Log\n",
    "We are using an example event log that is suited to introduce the Heuristics Miner algorithm. This event log is already included with the `heuristicsmineR` package in bupaR."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "L_heur_1 %>% head(9)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The naive process map drawing reveals some weird behaviour between the activities `b` and `c`. There seems to be a loop between both activities even though they never occur more than once in each trace."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "L_heur_1 %>%\n",
    "    as_tibble() %>%\n",
    "    mutate(activity_id = as.character(activity_id)) %>%\n",
    "    mutate(activity_id = if_else(activity_id == \"b\" | activity_id == \"c\", \"cb\", activity_id)) %>%\n",
    "    simple_eventlog(case_id = \"CASE_concept_name\", activity_id = \"activity_id\", timestamp = \"timestamp\") %>%\n",
    "    process_map()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "L_heur_1 %>% process_map()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Dependency Graphs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "L_heur_1 %>% precedence_matrix(type = \"absolute\") %>%  plot()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Based on the precedence matrix, we can follows the formula for the dependency relation:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "mat_pre <- L_heur_1 %>% precedence_matrix(type = \"absolute\") %>% as.matrix()\n",
    "mat_pre"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Since, we want to compute how often activities follow each other in either direction, we need the transposed matrix:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "t_mat_pre <- t(mat_pre)\n",
    "t_mat_pre"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And, then it is basic math:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "(mat_pre - t_mat_pre) / (mat_pre + t_mat_pre + 1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Of course, this has already been implemented in the `heuristicsmineR` package. There are also some more details of the algorithm that deal with the detection of loops as well as making sure that all activities are connected to each other. Please consult the original [Heuristics Miner paper](https://is.ieis.tue.nl/staff/aweijters/WP334_FHMv3.pdf) and the documentation of `heuristicsmineR` for more details."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "L_heur_1 %>% dependency_matrix(threshold = 0) %>% plot()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A dependency graph can be "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "L_heur_1 %>% \n",
    "   dependency_matrix(threshold = 0.8) %>% \n",
    "   render_dependency_matrix()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Have a look at the parameters (via `?dependency_matrix`) and try to change some of them to see what happens."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "L_heur_1 %>% \n",
    "   dependency_matrix(threshold = 0.9) %>% \n",
    "   render_dependency_matrix()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "sepsis %>% precedence_matrix() %>% plot"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "sepsis %>% \n",
    "  dependency_matrix(threshold = 0.7) %>% \n",
    "  render_dependency_matrix()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "sepsis %>% \n",
    "  dependency_matrix(threshold = 0.9) %>% \n",
    "  render_dependency_matrix()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Causal nets"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "L_heur_1 %>% \n",
    "  causal_net(threshold = 0.8) %>%\n",
    "  render_causal_net()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "sepsis %>%\n",
    "  act_unite(Release = c(\"Release A\", \"Release B\", \"Release C\", \"Release D\", \"Release E\")) %>%\n",
    "  causal_net(all_connected = TRUE) %>%\n",
    "  render_causal_net()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "example_log %>% \n",
    "  causal_net() %>% \n",
    "  render_causal_net()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Visualise / Convert as BPMN 🚧"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In bupaR there is currently no support for BPMN visualizations. However, it is possible to convert the Causal net into a Petri net. For simple process models, the mapping between BPMN and Petri nets is easy to understand. Thus, we are using Petri nets here."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "L_heur_1 %>% \n",
    "    causal_net() %>%\n",
    "    as.petrinet() %>%\n",
    "    petrinetR::render_PN()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "example_log %>% \n",
    "    causal_net() %>%\n",
    "    as.petrinet() %>%\n",
    "    petrinetR::render_PN()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**TODO** we could use the discovered Petri net with PM4Py to do further processing 🚧"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "R",
   "language": "R",
   "name": "ir"
  },
  "language_info": {
   "codemirror_mode": "r",
   "file_extension": ".r",
   "mimetype": "text/x-r-source",
   "name": "R",
   "pygments_lexer": "r",
   "version": "4.2.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}