short_path.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "<a id='short-path'></a>\n",
    "<div id=\"qe-notebook-header\" align=\"right\" style=\"text-align:right;\">\n",
    "        <a href=\"https://quantecon.org/\" title=\"quantecon.org\">\n",
    "                <img style=\"width:250px;display:inline;\" width=\"250px\" src=\"https://assets.quantecon.org/img/qe-menubar-logo.svg\" alt=\"QuantEcon\">\n",
    "        </a>\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Shortest Paths\n",
    "\n",
    "\n",
    "<a id='index-0'></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Contents\n",
    "\n",
    "- [Shortest Paths](#Shortest-Paths)  \n",
    "  - [Overview](#Overview)  \n",
    "  - [Outline of the Problem](#Outline-of-the-Problem)  \n",
    "  - [Finding Least-Cost Paths](#Finding-Least-Cost-Paths)  \n",
    "  - [Solving for Minimum Cost-to-Go](#Solving-for-Minimum-Cost-to-Go)  \n",
    "  - [Exercises](#Exercises)  \n",
    "  - [Solutions](#Solutions)  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Overview\n",
    "\n",
    "The shortest path problem is a [classic problem](https://en.wikipedia.org/wiki/Shortest_path) in mathematics and computer science with applications in\n",
    "\n",
    "- Economics (sequential decision making, analysis of social networks, etc.)  \n",
    "- Operations research and transportation  \n",
    "- Robotics and artificial intelligence  \n",
    "- Telecommunication network design and routing  \n",
    "- etc., etc.  \n",
    "\n",
    "\n",
    "Variations of the methods we discuss in this lecture are used millions of times every day, in applications such as\n",
    "\n",
    "- Google Maps  \n",
    "- routing packets on the internet  \n",
    "\n",
    "\n",
    "For us, the shortest path problem also provides a nice introduction to the logic of **dynamic programming**.\n",
    "\n",
    "Dynamic programming is an extremely powerful optimization technique that we apply in many lectures on this site.\n",
    "\n",
    "The only scientific library we’ll need in what follows is NumPy:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "hide-output": false
   },
   "outputs": [],
   "source": [
    "import numpy as np"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Outline of the Problem\n",
    "\n",
    "The shortest path problem is one of finding how to traverse a [graph](https://en.wikipedia.org/wiki/Graph_%28mathematics%29) from one specified node to another at minimum cost.\n",
    "\n",
    "Consider the following graph\n",
    "\n",
    "<img src=\"https://s3-ap-southeast-2.amazonaws.com/python.quantecon.org/_static/lecture_specific/short_path/graph.png\" style=\"\">\n",
    "\n",
    "  \n",
    "We wish to travel from node (vertex) A to node G at minimum cost\n",
    "\n",
    "- Arrows (edges) indicate the movements we can take.  \n",
    "- Numbers on edges indicate the cost of traveling that edge.  \n",
    "\n",
    "\n",
    "(Graphs such as the one above are called **weighted directed graphs**)\n",
    "\n",
    "Possible interpretations of the graph include\n",
    "\n",
    "- Minimum cost for supplier to reach a destination.  \n",
    "- Routing of packets on the internet (minimize time).  \n",
    "- Etc., etc.  \n",
    "\n",
    "\n",
    "For this simple graph, a quick scan of the edges shows that the optimal paths are\n",
    "\n",
    "- A, C, F, G at cost 8  \n",
    "\n",
    "\n",
    "<img src=\"https://s3-ap-southeast-2.amazonaws.com/python.quantecon.org/_static/lecture_specific/short_path/graph4.png\" style=\"\">\n",
    "\n",
    "  \n",
    "- A, D, F, G at cost 8  \n",
    "\n",
    "\n",
    "<img src=\"https://s3-ap-southeast-2.amazonaws.com/python.quantecon.org/_static/lecture_specific/short_path/graph3.png\" style=\"\">"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Finding Least-Cost Paths\n",
    "\n",
    "For large graphs, we need a systematic solution.\n",
    "\n",
    "Let $ J(v) $ denote the minimum cost-to-go from node $ v $, understood as the total cost from $ v $ if we take the best route.\n",
    "\n",
    "Suppose that we know $ J(v) $ for each node $ v $, as shown below for the graph from the preceding example\n",
    "\n",
    "<img src=\"https://s3-ap-southeast-2.amazonaws.com/python.quantecon.org/_static/lecture_specific/short_path/graph2.png\" style=\"\">\n",
    "\n",
    "  \n",
    "Note that $ J(G) = 0 $.\n",
    "\n",
    "The best path can now be found as follows\n",
    "\n",
    "1. Start at node $ v = A $  \n",
    "1. From current node $ v $, move to any node that solves  \n",
    "\n",
    "\n",
    "\n",
    "<a id='equation-spprebell'></a>\n",
    "$$\n",
    "\\min_{w \\in F_v} \\{ c(v, w) + J(w) \\} \\tag{1}\n",
    "$$\n",
    "\n",
    "where\n",
    "\n",
    "- $ F_v $ is the set of nodes that can be reached from $ v $ in one step.  \n",
    "- $ c(v, w) $ is the cost of traveling from $ v $ to $ w $.  \n",
    "\n",
    "\n",
    "Hence, if we know the function $ J $, then finding the best path is almost trivial.\n",
    "\n",
    "But how can we find the cost-to-go function $ J $?\n",
    "\n",
    "Some thought will convince you that, for every node $ v $,\n",
    "the function $ J $ satisfies\n",
    "\n",
    "\n",
    "<a id='equation-spbell'></a>\n",
    "$$\n",
    "J(v) = \\min_{w \\in F_v} \\{ c(v, w) + J(w) \\} \\tag{2}\n",
    "$$\n",
    "\n",
    "This is known as the *Bellman equation*, after the mathematician Richard Bellman.\n",
    "\n",
    "The Bellman equation can be thought of as a restriction that $ J $ must\n",
    "satisfy.\n",
    "\n",
    "What we want to do now is use this restriction to compute $ J $."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Solving for Minimum Cost-to-Go\n",
    "\n",
    "Let’s look at an algorithm for computing $ J $ and then think about how to\n",
    "implement it."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### The Algorithm\n",
    "\n",
    "The standard algorithm for finding $ J $ is to start an initial guess and then iterate.\n",
    "\n",
    "This is a standard approach to solving nonlinear equations, often called\n",
    "the method of **successive approximations**.\n",
    "\n",
    "Our initial guess will be\n",
    "\n",
    "\n",
    "<a id='equation-spguess'></a>\n",
    "$$\n",
    "J_0(v) = 0 \\text{ for all } v \\tag{3}\n",
    "$$\n",
    "\n",
    "Now\n",
    "\n",
    "1. Set $ n = 0 $  \n",
    "1. Set $ J_{n+1} (v) = \\min_{w \\in F_v} \\{ c(v, w) + J_n(w) \\} $ for all $ v $  \n",
    "1. If $ J_{n+1} $ and $ J_n $ are not equal then increment $ n $, go to 2  \n",
    "\n",
    "\n",
    "This sequence converges to $ J $.\n",
    "\n",
    "Although we omit the proof, we’ll prove similar claims in our other lectures\n",
    "on dynamic programming."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Implementation\n",
    "\n",
    "Having an algorithm is a good start, but we also need to think about how to\n",
    "implement it on a computer.\n",
    "\n",
    "First, for the cost function $ c $, we’ll implement it as a matrix\n",
    "$ Q $, where a typical element is\n",
    "\n",
    "$$\n",
    "Q(v, w)\n",
    "=\n",
    "\\begin{cases}\n",
    "   & c(v, w) \\text{ if } w \\in F_v \\\\\n",
    "   & +\\infty \\text{ otherwise }\n",
    "\\end{cases}\n",
    "$$\n",
    "\n",
    "In this context $ Q $ is usually called the **distance matrix**.\n",
    "\n",
    "We’re also numbering the nodes now, with $ A = 0 $, so, for example\n",
    "\n",
    "$$\n",
    "Q(1, 2)\n",
    "=\n",
    "\\text{ the cost of traveling from B to C }\n",
    "$$\n",
    "\n",
    "For example, for the simple graph above, we set"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "hide-output": false
   },
   "outputs": [],
   "source": [
    "from numpy import inf\n",
    "\n",
    "Q = np.array([[inf, 1,   5,   3,   inf, inf, inf],\n",
    "              [inf, inf, inf, 9,   6,   inf, inf],\n",
    "              [inf, inf, inf, inf, inf, 2,   inf],\n",
    "              [inf, inf, inf, inf, inf, 4,   8],\n",
    "              [inf, inf, inf, inf, inf, inf, 4],\n",
    "              [inf, inf, inf, inf, inf, inf, 1],\n",
    "              [inf, inf, inf, inf, inf, inf, 0]])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Notice that the cost of staying still (on the principle diagonal) is set to\n",
    "\n",
    "- np.inf for non-destination nodes — moving on is required.  \n",
    "- 0 for the destination node — here is where we stop.  \n",
    "\n",
    "\n",
    "For the sequence of approximations $ \\{J_n\\} $ of the cost-to-go functions, we can use NumPy arrays.\n",
    "\n",
    "Let’s try with this example and see how we go:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "hide-output": false
   },
   "outputs": [],
   "source": [
    "nodes = range(7)                              # Nodes = 0, 1, ..., 6\n",
    "J = np.zeros_like(nodes, dtype=np.int)        # Initial guess\n",
    "next_J = np.empty_like(nodes, dtype=np.int)   # Stores updated guess\n",
    "\n",
    "max_iter = 500\n",
    "i = 0\n",
    "\n",
    "while i < max_iter:\n",
    "    for v in nodes:\n",
    "        # minimize Q[v, w] + J[w] over all choices of w\n",
    "        lowest_cost = inf\n",
    "        for w in nodes:\n",
    "            cost = Q[v, w] + J[w]\n",
    "            if cost < lowest_cost:\n",
    "                lowest_cost = cost\n",
    "        next_J[v] = lowest_cost\n",
    "    if np.equal(next_J, J).all():\n",
    "        break\n",
    "    else:\n",
    "        J[:] = next_J   # Copy contents of next_J to J\n",
    "        i += 1\n",
    "\n",
    "print(\"The cost-to-go function is\", J)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This matches with the numbers we obtained by inspection above.\n",
    "\n",
    "But, importantly, we now have a methodology for tackling large graphs."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercises\n",
    "\n",
    "\n",
    "<a id='short-path-ex1'></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise 1\n",
    "\n",
    "The text below describes a weighted directed graph.\n",
    "\n",
    "The line `node0, node1 0.04, node8 11.11, node14 72.21` means that from node0 we can go to\n",
    "\n",
    "- node1 at cost 0.04  \n",
    "- node8 at cost 11.11  \n",
    "- node14 at cost 72.21  \n",
    "\n",
    "\n",
    "No other nodes can be reached directly from node0.\n",
    "\n",
    "Other lines have a similar interpretation.\n",
    "\n",
    "Your task is to use the algorithm given above to find the optimal path and its cost.\n",
    "\n",
    "Note: You will be dealing with floating point numbers now, rather than\n",
    "integers, so consider replacing `np.equal()` with `np.allclose()`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "hide-output": false
   },
   "outputs": [],
   "source": [
    "%%file graph.txt\n",
    "node0, node1 0.04, node8 11.11, node14 72.21\n",
    "node1, node46 1247.25, node6 20.59, node13 64.94\n",
    "node2, node66 54.18, node31 166.80, node45 1561.45\n",
    "node3, node20 133.65, node6 2.06, node11 42.43\n",
    "node4, node75 3706.67, node5 0.73, node7 1.02\n",
    "node5, node45 1382.97, node7 3.33, node11 34.54\n",
    "node6, node31 63.17, node9 0.72, node10 13.10\n",
    "node7, node50 478.14, node9 3.15, node10 5.85\n",
    "node8, node69 577.91, node11 7.45, node12 3.18\n",
    "node9, node70 2454.28, node13 4.42, node20 16.53\n",
    "node10, node89 5352.79, node12 1.87, node16 25.16\n",
    "node11, node94 4961.32, node18 37.55, node20 65.08\n",
    "node12, node84 3914.62, node24 34.32, node28 170.04\n",
    "node13, node60 2135.95, node38 236.33, node40 475.33\n",
    "node14, node67 1878.96, node16 2.70, node24 38.65\n",
    "node15, node91 3597.11, node17 1.01, node18 2.57\n",
    "node16, node36 392.92, node19 3.49, node38 278.71\n",
    "node17, node76 783.29, node22 24.78, node23 26.45\n",
    "node18, node91 3363.17, node23 16.23, node28 55.84\n",
    "node19, node26 20.09, node20 0.24, node28 70.54\n",
    "node20, node98 3523.33, node24 9.81, node33 145.80\n",
    "node21, node56 626.04, node28 36.65, node31 27.06\n",
    "node22, node72 1447.22, node39 136.32, node40 124.22\n",
    "node23, node52 336.73, node26 2.66, node33 22.37\n",
    "node24, node66 875.19, node26 1.80, node28 14.25\n",
    "node25, node70 1343.63, node32 36.58, node35 45.55\n",
    "node26, node47 135.78, node27 0.01, node42 122.00\n",
    "node27, node65 480.55, node35 48.10, node43 246.24\n",
    "node28, node82 2538.18, node34 21.79, node36 15.52\n",
    "node29, node64 635.52, node32 4.22, node33 12.61\n",
    "node30, node98 2616.03, node33 5.61, node35 13.95\n",
    "node31, node98 3350.98, node36 20.44, node44 125.88\n",
    "node32, node97 2613.92, node34 3.33, node35 1.46\n",
    "node33, node81 1854.73, node41 3.23, node47 111.54\n",
    "node34, node73 1075.38, node42 51.52, node48 129.45\n",
    "node35, node52 17.57, node41 2.09, node50 78.81\n",
    "node36, node71 1171.60, node54 101.08, node57 260.46\n",
    "node37, node75 269.97, node38 0.36, node46 80.49\n",
    "node38, node93 2767.85, node40 1.79, node42 8.78\n",
    "node39, node50 39.88, node40 0.95, node41 1.34\n",
    "node40, node75 548.68, node47 28.57, node54 53.46\n",
    "node41, node53 18.23, node46 0.28, node54 162.24\n",
    "node42, node59 141.86, node47 10.08, node72 437.49\n",
    "node43, node98 2984.83, node54 95.06, node60 116.23\n",
    "node44, node91 807.39, node46 1.56, node47 2.14\n",
    "node45, node58 79.93, node47 3.68, node49 15.51\n",
    "node46, node52 22.68, node57 27.50, node67 65.48\n",
    "node47, node50 2.82, node56 49.31, node61 172.64\n",
    "node48, node99 2564.12, node59 34.52, node60 66.44\n",
    "node49, node78 53.79, node50 0.51, node56 10.89\n",
    "node50, node85 251.76, node53 1.38, node55 20.10\n",
    "node51, node98 2110.67, node59 23.67, node60 73.79\n",
    "node52, node94 1471.80, node64 102.41, node66 123.03\n",
    "node53, node72 22.85, node56 4.33, node67 88.35\n",
    "node54, node88 967.59, node59 24.30, node73 238.61\n",
    "node55, node84 86.09, node57 2.13, node64 60.80\n",
    "node56, node76 197.03, node57 0.02, node61 11.06\n",
    "node57, node86 701.09, node58 0.46, node60 7.01\n",
    "node58, node83 556.70, node64 29.85, node65 34.32\n",
    "node59, node90 820.66, node60 0.72, node71 0.67\n",
    "node60, node76 48.03, node65 4.76, node67 1.63\n",
    "node61, node98 1057.59, node63 0.95, node64 4.88\n",
    "node62, node91 132.23, node64 2.94, node76 38.43\n",
    "node63, node66 4.43, node72 70.08, node75 56.34\n",
    "node64, node80 47.73, node65 0.30, node76 11.98\n",
    "node65, node94 594.93, node66 0.64, node73 33.23\n",
    "node66, node98 395.63, node68 2.66, node73 37.53\n",
    "node67, node82 153.53, node68 0.09, node70 0.98\n",
    "node68, node94 232.10, node70 3.35, node71 1.66\n",
    "node69, node99 247.80, node70 0.06, node73 8.99\n",
    "node70, node76 27.18, node72 1.50, node73 8.37\n",
    "node71, node89 104.50, node74 8.86, node91 284.64\n",
    "node72, node76 15.32, node84 102.77, node92 133.06\n",
    "node73, node83 52.22, node76 1.40, node90 243.00\n",
    "node74, node81 1.07, node76 0.52, node78 8.08\n",
    "node75, node92 68.53, node76 0.81, node77 1.19\n",
    "node76, node85 13.18, node77 0.45, node78 2.36\n",
    "node77, node80 8.94, node78 0.98, node86 64.32\n",
    "node78, node98 355.90, node81 2.59\n",
    "node79, node81 0.09, node85 1.45, node91 22.35\n",
    "node80, node92 121.87, node88 28.78, node98 264.34\n",
    "node81, node94 99.78, node89 39.52, node92 99.89\n",
    "node82, node91 47.44, node88 28.05, node93 11.99\n",
    "node83, node94 114.95, node86 8.75, node88 5.78\n",
    "node84, node89 19.14, node94 30.41, node98 121.05\n",
    "node85, node97 94.51, node87 2.66, node89 4.90\n",
    "node86, node97 85.09\n",
    "node87, node88 0.21, node91 11.14, node92 21.23\n",
    "node88, node93 1.31, node91 6.83, node98 6.12\n",
    "node89, node97 36.97, node99 82.12\n",
    "node90, node96 23.53, node94 10.47, node99 50.99\n",
    "node91, node97 22.17\n",
    "node92, node96 10.83, node97 11.24, node99 34.68\n",
    "node93, node94 0.19, node97 6.71, node99 32.77\n",
    "node94, node98 5.91, node96 2.03\n",
    "node95, node98 6.17, node99 0.27\n",
    "node96, node98 3.32, node97 0.43, node99 5.87\n",
    "node97, node98 0.30\n",
    "node98, node99 0.33\n",
    "node99,"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Solutions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise 1\n",
    "\n",
    "First let’s write a function that reads in the graph data above and builds a distance matrix."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "hide-output": false
   },
   "outputs": [],
   "source": [
    "num_nodes = 100\n",
    "destination_node = 99\n",
    "\n",
    "def map_graph_to_distance_matrix(in_file):\n",
    "\n",
    "    # First let's set of the distance matrix Q with inf everywhere\n",
    "    Q = np.ones((num_nodes, num_nodes))\n",
    "    Q = Q * np.inf\n",
    "\n",
    "    # Now we read in the data and modify Q\n",
    "    infile = open(in_file)\n",
    "    for line in infile:\n",
    "        elements = line.split(',')\n",
    "        node = elements.pop(0)\n",
    "        node = int(node[4:])    # convert node description to integer\n",
    "        if node != destination_node:\n",
    "            for element in elements:\n",
    "                destination, cost = element.split()\n",
    "                destination = int(destination[4:])\n",
    "                Q[node, destination] = float(cost)\n",
    "        Q[destination_node, destination_node] = 0\n",
    "\n",
    "    infile.close()\n",
    "    return Q"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In addition, let’s write\n",
    "\n",
    "1. a “Bellman operator” function that takes a distance matrix and current guess of J and returns an updated guess of J, and  \n",
    "1. a function that takes a distance matrix and returns a cost-to-go function.  \n",
    "\n",
    "\n",
    "We’ll use the algorithm described above.\n",
    "\n",
    "The minimization step is vectorized to make it faster."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "hide-output": false
   },
   "outputs": [],
   "source": [
    "def bellman(J, Q):\n",
    "    num_nodes = Q.shape[0]\n",
    "    next_J = np.empty_like(J)\n",
    "    for v in range(num_nodes):\n",
    "        next_J[v] = np.min(Q[v, :] + J)\n",
    "    return next_J\n",
    "\n",
    "\n",
    "def compute_cost_to_go(Q):\n",
    "    J = np.zeros(num_nodes)      # Initial guess\n",
    "    next_J = np.empty(num_nodes)  # Stores updated guess\n",
    "    max_iter = 500\n",
    "    i = 0\n",
    "\n",
    "    while i < max_iter:\n",
    "        next_J = bellman(J, Q)\n",
    "        if np.allclose(next_J, J):\n",
    "            break\n",
    "        else:\n",
    "            J[:] = next_J   # Copy contents of next_J to J\n",
    "            i += 1\n",
    "\n",
    "    return(J)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We used np.allclose() rather than testing exact equality because we are\n",
    "dealing with floating point numbers now.\n",
    "\n",
    "Finally, here’s a function that uses the cost-to-go function to obtain the\n",
    "optimal path (and its cost)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "hide-output": false
   },
   "outputs": [],
   "source": [
    "def print_best_path(J, Q):\n",
    "    sum_costs = 0\n",
    "    current_node = 0\n",
    "    while current_node != destination_node:\n",
    "        print(current_node)\n",
    "        # Move to the next node and increment costs\n",
    "        next_node = np.argmin(Q[current_node, :] + J)\n",
    "        sum_costs += Q[current_node, next_node]\n",
    "        current_node = next_node\n",
    "\n",
    "    print(destination_node)\n",
    "    print('Cost: ', sum_costs)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Okay, now we have the necessary functions, let’s call them to do the job we were assigned."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "hide-output": false
   },
   "outputs": [],
   "source": [
    "Q = map_graph_to_distance_matrix('graph.txt')\n",
    "J = compute_cost_to_go(Q)\n",
    "print_best_path(J, Q)"
   ]
  }
 ],
 "metadata": {
  "date": 1584334757.781868,
  "filename": "short_path.rst",
  "kernelspec": {
   "display_name": "Python",
   "language": "python3",
   "name": "python3"
  },
  "title": "Shortest Paths"
 },
 "nbformat": 4,
 "nbformat_minor": 2
}