lq_inventories.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "<a id='inventory-sales-smoothing-v6'></a>\n",
    "<div id=\"qe-notebook-header\" align=\"right\" style=\"text-align:right;\">\n",
    "        <a href=\"https://quantecon.org/\" title=\"quantecon.org\">\n",
    "                <img style=\"width:250px;display:inline;\" width=\"250px\" src=\"https://assets.quantecon.org/img/qe-menubar-logo.svg\" alt=\"QuantEcon\">\n",
    "        </a>\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Production Smoothing via Inventories"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Contents\n",
    "\n",
    "- [Production Smoothing via Inventories](#Production-Smoothing-via-Inventories)  \n",
    "  - [Overview](#Overview)  \n",
    "  - [Example 1](#Example-1)  \n",
    "  - [Inventories Not Useful](#Inventories-Not-Useful)  \n",
    "  - [Inventories Useful but are Hardwired to be Zero Always](#Inventories-Useful-but-are-Hardwired-to-be-Zero-Always)  \n",
    "  - [Example 2](#Example-2)  \n",
    "  - [Example 3](#Example-3)  \n",
    "  - [Example 4](#Example-4)  \n",
    "  - [Example 5](#Example-5)  \n",
    "  - [Example 6](#Example-6)  \n",
    "  - [Exercises](#Exercises)  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In addition to what’s in Anaconda, this lecture employs the following library:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "hide-output": true
   },
   "outputs": [],
   "source": [
    "!pip install --upgrade quantecon"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Overview\n",
    "\n",
    "This lecture can be viewed as an application of the [quantecon lecture](https://python.quantecon.org/lqcontrol.html).\n",
    "\n",
    "It formulates a discounted dynamic program for a firm that\n",
    "chooses a production schedule to balance\n",
    "\n",
    "- minimizing costs of production across time, against  \n",
    "- keeping costs of holding inventories low  \n",
    "\n",
    "\n",
    "In the tradition of a classic book by Holt, Modigliani, Muth, and\n",
    "Simon [[HMMS60]](https://python.quantecon.org/zreferences.html#holt-modigliani-muth-simon), we simplify the\n",
    "firm’s problem by formulating it as a linear quadratic discounted\n",
    "dynamic programming problem of the type studied in this [quantecon](https://python.quantecon.org/lqcontrol.html).\n",
    "\n",
    "Because its costs of production are increasing and quadratic in\n",
    "production, the firm wants to smooth production across time provided\n",
    "that holding inventories is not too costly.\n",
    "\n",
    "But the firm also prefers to sell out of existing inventories, a\n",
    "preference that we represent by a cost that is quadratic in the\n",
    "difference between sales in a period and the firm’s beginning of period\n",
    "inventories.\n",
    "\n",
    "We compute examples designed to indicate how the firm optimally chooses\n",
    "to smooth production and manage inventories while keeping inventories\n",
    "close to sales.\n",
    "\n",
    "To introduce components of the model, let\n",
    "\n",
    "- $ S_t $ be sales at time $ t $  \n",
    "- $ Q_t $ be production at time $ t $  \n",
    "- $ I_t $ be inventories at the beginning of time $ t $  \n",
    "- $ \\beta \\in (0,1) $ be a discount factor  \n",
    "- $ c(Q_t) = c_1 Q_t + c_2 Q_t^2 $, be a cost of production\n",
    "  function, where $ c_1>0, c_2>0 $, be an inventory cost function  \n",
    "- $ d(I_t, S_t) = d_1 I_t + d_2 (S_t - I_t)^2 $, where\n",
    "  $ d_1>0, d_2 >0 $, be a cost-of-holding-inventories function,\n",
    "  consisting of two components:  \n",
    "  - a cost $ d_1 t $ of carrying inventories, and  \n",
    "  - a cost $ d_2 (S_t - I_t)^2 $ of having inventories deviate\n",
    "    from sales  \n",
    "- $ p_t = a_0 - a_1 S_t + v_t $ be an inverse demand function for a\n",
    "  firm’s product, where $ a_0>0, a_1 >0 $ and $ v_t $ is a\n",
    "  demand shock at time $ t $  \n",
    "- $ \\pi\\_t = p_t S_t - c(Q_t) - d(I_t, S_t) $ be the firm’s\n",
    "  profits at time $ t $  \n",
    "- $ \\sum_{t=0}^\\infty \\beta^t \\pi_t $\n",
    "  be the present value of the firm’s profits at\n",
    "  time $ 0 $  \n",
    "- $ I_{t+1} = I_t + Q_t - S_t $ be the law of motion of inventories  \n",
    "- $ z_{t+1} = A_{22} z_t + C_2 \\epsilon\\_{t+1} $ be the law\n",
    "  of motion for an exogenous state vector $ z_t $ that contains\n",
    "  time $ t $ information useful for predicting the demand shock\n",
    "  $ v_t $  \n",
    "- $ v_t = G z_t $ link the demand shock to the information set\n",
    "  $ z_t $  \n",
    "- the constant $ 1 $ be the first component of $ z_t $  \n",
    "\n",
    "\n",
    "To map our problem into a linear-quadratic discounted dynamic\n",
    "programming problem (also known as an optimal linear regulator), we\n",
    "define the **state** vector at time $ t $ as\n",
    "\n",
    "$$\n",
    "x_t = \\begin{bmatrix} I_t \\cr z_t \\end{bmatrix}\n",
    "$$\n",
    "\n",
    "and the **control** vector as\n",
    "\n",
    "$$\n",
    "u_t =  \\begin{bmatrix} Q_t \\cr S_t \\end{bmatrix}\n",
    "$$\n",
    "\n",
    "The law of motion for the state vector $ x_t $ is evidently\n",
    "\n",
    "$$\n",
    "\\begin{aligned}\n",
    " \\begin{bmatrix} I_{t+1} \\cr z_t \\end{bmatrix} = \\left[\\begin{array}{cc}\n",
    "1 & 0\\\\\n",
    "0 & A_{22}\n",
    "\\end{array}\\right] \\begin{bmatrix} I_t \\cr z_t \\end{bmatrix}\n",
    "             + \\begin{bmatrix} 1 & -1 \\cr\n",
    "             0 & 0 \\end{bmatrix} \\begin{bmatrix} Q_t \\cr S_t \\end{bmatrix}\n",
    "             + \\begin{bmatrix} 0 \\cr C_2 \\end{bmatrix} \\epsilon_{t+1} \\end{aligned}\n",
    "$$\n",
    "\n",
    "or\n",
    "\n",
    "$$\n",
    "x_{t+1} = A x_t + B u_t + C \\epsilon_{t+1}\n",
    "$$\n",
    "\n",
    "(At this point, we ask that you please forgive us for using $ Q_t $\n",
    "to be the firm’s production at time $ t $, while below we use\n",
    "$ Q $ as the matrix in the quadratic form $ u_t' Q u_t $ that\n",
    "appears in the firm’s one-period profit function)\n",
    "\n",
    "We can express the firm’s profit as a function of states and controls as\n",
    "\n",
    "$$\n",
    "\\pi_t =  - (x_t' R x_t + u_t' Q u_t + 2 u_t' H x_t )\n",
    "$$\n",
    "\n",
    "To form the matrices $ R, Q, H $, we note that the firm’s profits at\n",
    "time $ t $ function can be expressed\n",
    "\n",
    "$$\n",
    "\\begin{equation}\n",
    "\\begin{split}\n",
    "\\pi_{t} =&p_{t}S_{t}-c\\left(Q_{t}\\right)-d\\left(I_{t},S_{t}\\right)  \\\\\n",
    "    =&\\left(a_{0}-a_{1}S_{t}+v_{t}\\right)S_{t}-c_{1}Q_{t}-c_{2}Q_{t}^{2}-d_{1}I_{t}-d_{2}\\left(S_{t}-I_{t}\\right)^{2}  \\\\\n",
    "    =&a_{0}S_{t}-a_{1}S_{t}^{2}+Gz_{t}S_{t}-c_{1}Q_{t}-c_{2}Q_{t}^{2}-d_{1}I_{t}-d_{2}S_{t}^{2}-d_{2}I_{t}^{2}+2d_{2}S_{t}I_{t}  \\\\\n",
    "    =&-\\left(\\underset{x_{t}^{\\prime}Rx_{t}}{\\underbrace{d_{1}I_{t}+d_{2}I_{t}^{2}}}\\underset{u_{t}^{\\prime}Qu_{t}}{\\underbrace{+a_{1}S_{t}^{2}+d_{2}S_{t}^{2}+c_{2}Q_{t}^{2}}}\\underset{2u_{t}^{\\prime}Hx_{t}}{\\underbrace{-a_{0}S_{t}-Gz_{t}S_{t}+c_{1}Q_{t}-2d_{2}S_{t}I_{t}}}\\right) \\\\\n",
    "    =&-\\left(\\left[\\begin{array}{cc}\n",
    "I_{t} & z_{t}^{\\prime}\\end{array}\\right]\\underset{\\equiv R}{\\underbrace{\\left[\\begin{array}{cc}\n",
    "d_{2} & \\frac{d_{1}}{2}S_{c}\\\\\n",
    "\\frac{d_{1}}{2}S_{c}^{\\prime} & 0\n",
    "\\end{array}\\right]}}\\left[\\begin{array}{c}\n",
    "I_{t}\\\\\n",
    "z_{t}\n",
    "\\end{array}\\right]+\\left[\\begin{array}{cc}\n",
    "Q_{t} & S_{t}\\end{array}\\right]\\underset{\\equiv Q}{\\underbrace{\\left[\\begin{array}{cc}\n",
    "c_{2} & 0\\\\\n",
    "0 & a_{1}+d_{2}\n",
    "\\end{array}\\right]}}\\left[\\begin{array}{c}\n",
    "Q_{t}\\\\\n",
    "S_{t}\n",
    "\\end{array}\\right]+2\\left[\\begin{array}{cc}\n",
    "Q_{t} & S_{t}\\end{array}\\right]\\underset{\\equiv N}{\\underbrace{\\left[\\begin{array}{cc}\n",
    "0 & \\frac{c_{1}}{2}S_{c}\\\\\n",
    "-d_{2} & -\\frac{a_{0}}{2}S_{c}-\\frac{G}{2}\n",
    "\\end{array}\\right]}}\\left[\\begin{array}{c}\n",
    "I_{t}\\\\\n",
    "z_{t}\n",
    "\\end{array}\\right]\\right)\n",
    "\\end{split}\n",
    "\\end{equation}\n",
    "$$\n",
    "\n",
    "where $ S_{c}=\\left[1,0\\right] $.\n",
    "\n",
    "**Remark on notation:** The notation for cross product term in the\n",
    "QuantEcon library is $ N $ instead of $ H $.\n",
    "\n",
    "The firms’ optimum decision rule takes the form\n",
    "\n",
    "$$\n",
    "u_t = - F x_t\n",
    "$$\n",
    "\n",
    "and the evolution of the state under the optimal decision rule is\n",
    "\n",
    "$$\n",
    "x_{t+1} = (A - BF ) x_t + C \\epsilon_{t+1}\n",
    "$$\n",
    "\n",
    "Here is code for computing an optimal decision rule and for analyzing\n",
    "its consequences."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "hide-output": false
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import quantecon as qe\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "hide-output": false
   },
   "outputs": [],
   "source": [
    "class smoothing_example:\n",
    "    \"\"\"\n",
    "    Class for constructing, solving, and plotting results for\n",
    "    inventories and sales smoothing problem.\n",
    "    \"\"\"\n",
    "\n",
    "    def __init__(self,\n",
    "                 β=0.96,           # Discount factor\n",
    "                 c1=1,             # Cost-of-production\n",
    "                 c2=1,\n",
    "                 d1=1,             # Cost-of-holding inventories\n",
    "                 d2=1,\n",
    "                 a0=10,            # Inverse demand function\n",
    "                 a1=1,\n",
    "                 A22=[[1,   0],    # z process\n",
    "                      [1, 0.9]],\n",
    "                 C2=[[0], [1]],\n",
    "                 G=[0, 1]):\n",
    "\n",
    "        self.β = β\n",
    "        self.c1, self.c2 = c1, c2\n",
    "        self.d1, self.d2 = d1, d2\n",
    "        self.a0, self.a1 = a0, a1\n",
    "        self.A22 = np.atleast_2d(A22)\n",
    "        self.C2 = np.atleast_2d(C2)\n",
    "        self.G = np.atleast_2d(G)\n",
    "\n",
    "        # Dimensions\n",
    "        k, j = self.C2.shape        # Dimensions for randomness part\n",
    "        n = k + 1                   # Number of states\n",
    "        m = 2                       # Number of controls\n",
    "\n",
    "        Sc = np.zeros(k)\n",
    "        Sc[0] = 1\n",
    "\n",
    "        # Construct matrices of transition law\n",
    "        A = np.zeros((n, n))\n",
    "        A[0, 0] = 1\n",
    "        A[1:, 1:] = A22\n",
    "\n",
    "        B = np.zeros((n, m))\n",
    "        B[0, :] = 1, -1\n",
    "\n",
    "        C = np.zeros((n, j))\n",
    "        C[1:, :] = C2\n",
    "\n",
    "        self.A, self.B, self.C = A, B, C\n",
    "\n",
    "        # Construct matrices of one period profit function\n",
    "        R = np.zeros((n, n))\n",
    "        R[0, 0] = d2\n",
    "        R[1:, 0] = d1 / 2 * Sc\n",
    "        R[0, 1:] = d1 / 2 * Sc\n",
    "\n",
    "        Q = np.zeros((m, m))\n",
    "        Q[0, 0] = c2\n",
    "        Q[1, 1] = a1 + d2\n",
    "\n",
    "        N = np.zeros((m, n))\n",
    "        N[1, 0] = - d2\n",
    "        N[0, 1:] = c1 / 2 * Sc\n",
    "        N[1, 1:] = - a0 / 2 * Sc - self.G / 2\n",
    "\n",
    "        self.R, self.Q, self.N = R, Q, N\n",
    "\n",
    "        # Construct LQ instance\n",
    "        self.LQ = qe.LQ(Q, R, A, B, C, N, beta=β)\n",
    "        self.LQ.stationary_values()\n",
    "\n",
    "    def simulate(self, x0, T=100):\n",
    "\n",
    "        c1, c2 = self.c1, self.c2\n",
    "        d1, d2 = self.d1, self.d2\n",
    "        a0, a1 = self.a0, self.a1\n",
    "        G = self.G\n",
    "\n",
    "        x_path, u_path, w_path = self.LQ.compute_sequence(x0, ts_length=T)\n",
    "\n",
    "        I_path = x_path[0, :-1]\n",
    "        z_path = x_path[1:, :-1]\n",
    "        𝜈_path = (G @ z_path)[0, :]\n",
    "\n",
    "        Q_path = u_path[0, :]\n",
    "        S_path = u_path[1, :]\n",
    "\n",
    "        revenue = (a0 - a1 * S_path + 𝜈_path) * S_path\n",
    "        cost_production = c1 * Q_path + c2 * Q_path ** 2\n",
    "        cost_inventories = d1 * I_path + d2 * (S_path - I_path) ** 2\n",
    "\n",
    "        Q_no_inventory = (a0 + 𝜈_path - c1) / (2 * (a1 + c2))\n",
    "        Q_hardwired = (a0 + 𝜈_path - c1) / (2 * (a1 + c2 + d2))\n",
    "\n",
    "        fig, ax = plt.subplots(2, 2, figsize=(15, 10))\n",
    "\n",
    "        ax[0, 0].plot(range(T), I_path, label=\"inventories\")\n",
    "        ax[0, 0].plot(range(T), S_path, label=\"sales\")\n",
    "        ax[0, 0].plot(range(T), Q_path, label=\"production\")\n",
    "        ax[0, 0].legend(loc=1)\n",
    "        ax[0, 0].set_title(\"inventories, sales, and production\")\n",
    "\n",
    "        ax[0, 1].plot(range(T), (Q_path - S_path), color='b')\n",
    "        ax[0, 1].set_ylabel(\"change in inventories\", color='b')\n",
    "        span = max(abs(Q_path - S_path))\n",
    "        ax[0, 1].set_ylim(0-span*1.1, 0+span*1.1)\n",
    "        ax[0, 1].set_title(\"demand shock and change in inventories\")\n",
    "\n",
    "        ax1_ = ax[0, 1].twinx()\n",
    "        ax1_.plot(range(T), 𝜈_path, color='r')\n",
    "        ax1_.set_ylabel(\"demand shock\", color='r')\n",
    "        span = max(abs(𝜈_path))\n",
    "        ax1_.set_ylim(0-span*1.1, 0+span*1.1)\n",
    "\n",
    "        ax1_.plot([0, T], [0, 0], '--', color='k')\n",
    "\n",
    "        ax[1, 0].plot(range(T), revenue, label=\"revenue\")\n",
    "        ax[1, 0].plot(range(T), cost_production, label=\"cost_production\")\n",
    "        ax[1, 0].plot(range(T), cost_inventories, label=\"cost_inventories\")\n",
    "        ax[1, 0].legend(loc=1)\n",
    "        ax[1, 0].set_title(\"profits decomposition\")\n",
    "\n",
    "        ax[1, 1].plot(range(T), Q_path, label=\"production\")\n",
    "        ax[1, 1].plot(range(T), Q_hardwired, label='production when  $I_t$ \\\n",
    "            forced to be zero')\n",
    "        ax[1, 1].plot(range(T), Q_no_inventory, label='production when \\\n",
    "            inventories not useful')\n",
    "        ax[1, 1].legend(loc=1)\n",
    "        ax[1, 1].set_title('three production concepts')\n",
    "\n",
    "        plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Notice that the above code sets parameters at the following default\n",
    "values\n",
    "\n",
    "- discount factor β=0.96,  \n",
    "- inverse demand function: $ a0=10, a1=1 $  \n",
    "- cost of production $ c1=1, c2=1 $  \n",
    "- costs of holding inventories $ d1=1, d2=1 $  \n",
    "\n",
    "\n",
    "In the examples below, we alter some or all of these parameter values."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Example 1\n",
    "\n",
    "In this example, the demand shock follows AR(1) process:\n",
    "\n",
    "$$\n",
    "\\nu_t = \\alpha + \\rho \\nu_{t-1} + \\epsilon_t,\n",
    "$$\n",
    "\n",
    "which implies\n",
    "\n",
    "$$\n",
    "z_{t+1}=\\left[\\begin{array}{c}\n",
    "1\\\\\n",
    "v_{t+1}\n",
    "\\end{array}\\right]=\\left[\\begin{array}{cc}\n",
    "1 & 0\\\\\n",
    "\\alpha & \\rho\n",
    "\\end{array}\\right]\\underset{z_{t}}{\\underbrace{\\left[\\begin{array}{c}\n",
    "1\\\\\n",
    "v_{t}\n",
    "\\end{array}\\right]}}+\\left[\\begin{array}{c}\n",
    "0\\\\\n",
    "1\n",
    "\\end{array}\\right]\\epsilon_{t+1}.\n",
    "$$\n",
    "\n",
    "We set $ \\alpha=1 $ and $ \\rho=0.9 $, their default values.\n",
    "\n",
    "We’ll calculate and display outcomes, then discuss them below the\n",
    "pertinent figures."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "hide-output": false
   },
   "outputs": [],
   "source": [
    "ex1 = smoothing_example()\n",
    "\n",
    "x0 = [0, 1, 0]\n",
    "ex1.simulate(x0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The figures above illustrate various features of an optimal production\n",
    "plan.\n",
    "\n",
    "Starting from zero inventories, the firm builds up a stock of\n",
    "inventories and uses them to smooth costly production in the face of\n",
    "demand shocks.\n",
    "\n",
    "Optimal decisions evidently respond to demand shocks.\n",
    "\n",
    "Inventories are always less than sales, so some sales come from current\n",
    "production, a consequence of the cost, $ d_1 I_t $ of holding\n",
    "inventories.\n",
    "\n",
    "The lower right panel shows differences between optimal production and\n",
    "two alternative production concepts that come from altering the firm’s\n",
    "cost structure – i.e., its technology.\n",
    "\n",
    "These two concepts correspond to these distinct altered firm problems.\n",
    "\n",
    "- a setting in which inventories are not needed  \n",
    "- a setting in which they are needed but we arbitrarily prevent the\n",
    "  firm from holding inventories by forcing it to set $ I_t=0 $\n",
    "  always  \n",
    "\n",
    "\n",
    "We use these two alternative production concepts in order to shed light on the baseline model."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Inventories Not Useful\n",
    "\n",
    "Let’s turn first to the setting in which inventories aren’t needed.\n",
    "\n",
    "In this problem, the firm forms an output plan that maximizes the expected\n",
    "value of\n",
    "\n",
    "$$\n",
    "\\sum_{t=0}^\\infty \\beta^t \\{ p_t Q_t - C(Q_t) \\}\n",
    "$$\n",
    "\n",
    "It turns out that the optimal plan for $ Q_t $ for this problem also\n",
    "solves a sequence of static problems\n",
    "$ \\max_{Q_t}\\{p_t Q_t - c(Q_t)\\} $.\n",
    "\n",
    "When inventories aren’t required or used,  sales always equal\n",
    "production.\n",
    "\n",
    "This simplifies the problem and the optimal no-inventory production\n",
    "maximizes the expected value of\n",
    "\n",
    "$$\n",
    "\\sum_{t=0}^{\\infty}\\beta^{t}\\left\\{ p_{t}Q_{t}-C\\left(Q_{t}\\right)\\right\\}.\n",
    "$$\n",
    "\n",
    "The optimum decision rule is\n",
    "\n",
    "$$\n",
    "Q_{t}^{ni}=\\frac{a_{0}+\\nu_{t}-c_{1}}{c_{2}+a_{1}}.\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Inventories Useful but are Hardwired to be Zero Always\n",
    "\n",
    "Next, we turn to a distinct problem in which inventories are useful –\n",
    "meaning that there are costs of $ d_2 (I_t - S_t)^2 $ associated\n",
    "with having sales not equal to inventories – but we arbitrarily impose on the firm\n",
    "the costly restriction that it never hold inventories.\n",
    "\n",
    "Here the firm’s maximization problem is\n",
    "\n",
    "$$\n",
    "\\max_{\\{I_t, Q_t, S_t\\}}\\sum_{t=0}^{\\infty}\\beta^{t}\\left\\{ p_{t}S_{t}-C\\left(Q_{t}\\right)-d\\left(I_{t},S_{t}\\right)\\right\\}\n",
    "$$\n",
    "\n",
    "subject to the restrictions that $ I_{t}=0 $ for all $ t $ and\n",
    "that $ I_{t+1}=I_{t}+Q_{t}-S_{t} $.\n",
    "\n",
    "The restriction that $ I_t = 0 $ implies that $ Q_{t}=S_{t} $\n",
    "and that the maximization problem reduces to\n",
    "\n",
    "$$\n",
    "\\max_{Q_t}\\sum_{t=0}^{\\infty}\\beta^{t}\\left\\{ p_{t}Q_{t}-C\\left(Q_{t}\\right)-d\\left(0,Q_{t}\\right)\\right\\}\n",
    "$$\n",
    "\n",
    "Here the optimal production plan is\n",
    "\n",
    "$$\n",
    "Q_{t}^{h}=\\frac{a_{0}+\\nu_{t}-c_{1}}{c_{2}+a_{1}+d_{2}}.\n",
    "$$\n",
    "\n",
    "We introduce this $ I_t $ **is hardwired to zero** specification in\n",
    "order to shed light on the role that inventories play by comparing outcomes\n",
    "with those under our two other versions of the problem.\n",
    "\n",
    "The bottom right panel displays an production path for the original\n",
    "problem that we are interested in (the blue line) as well with an\n",
    "optimal production path for the model in which inventories are not\n",
    "useful (the green path) and also for the model in which, although\n",
    "inventories are useful, they are hardwired to zero and the firm pays\n",
    "cost $ d(0, Q_t) $ for not setting sales $ S_t = Q_t $ equal to\n",
    "zero (the orange line).\n",
    "\n",
    "Notice that it is typically optimal for the firm to produce more when\n",
    "inventories aren’t useful. Here there is no requirement to sell out of\n",
    "inventories and no costs from having sales deviate from inventories.\n",
    "\n",
    "But “typical” does not mean “always”.\n",
    "\n",
    "Thus, if we look closely, we notice that for small $ t $, the green\n",
    "“production when inventories aren’t useful” line in the lower right\n",
    "panel is below optimal production in the original model.\n",
    "\n",
    "High optimal production in the original model early on occurs because the\n",
    "firm wants to accumulate inventories quickly in order to acquire high\n",
    "inventories for use in later periods.\n",
    "\n",
    "But how the green line compares to the blue line early on depends on the\n",
    "evolution of the demand shock, as we will see in a\n",
    "deterministically seasonal demand shock example to be analyzed below.\n",
    "\n",
    "In that example,  the original firm optimally accumulates inventories slowly\n",
    "because the next positive demand shock is in the distant future.\n",
    "\n",
    "To make the green-blue model production comparison easier to see, let’s\n",
    "confine the graphs to the first 10 periods:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "hide-output": false
   },
   "outputs": [],
   "source": [
    "ex1.simulate(x0, T=10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Example 2\n",
    "\n",
    "Next, we shut down randomness in demand and assume that the demand shock\n",
    "$ \\nu_t $ follows a deterministic path:\n",
    "\n",
    "$$\n",
    "\\nu_t = \\alpha + \\rho \\nu_{t-1}\n",
    "$$\n",
    "\n",
    "Again, we’ll compute and display outcomes in some figures"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "hide-output": false
   },
   "outputs": [],
   "source": [
    "ex2 = smoothing_example(C2=[[0], [0]])\n",
    "\n",
    "x0 = [0, 1, 0]\n",
    "ex2.simulate(x0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Example 3\n",
    "\n",
    "Now we’ll put randomness back into the demand shock process and also\n",
    "assume that there are zero costs of holding inventories.\n",
    "\n",
    "In particular, we’ll look at a situation in which $ d_1=0 $ but\n",
    "$ d_2>0 $.\n",
    "\n",
    "Now it becomes optimal to set sales approximately equal to\n",
    "inventories and to use inventories to smooth production quite well, as\n",
    "the following figures confirm"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "hide-output": false
   },
   "outputs": [],
   "source": [
    "ex3 = smoothing_example(d1=0)\n",
    "\n",
    "x0 = [0, 1, 0]\n",
    "ex3.simulate(x0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Example 4\n",
    "\n",
    "To bring out some features of the optimal policy that are related to\n",
    "some technical issues in linear control theory, we’ll now temporarily\n",
    "assume that it is costless to hold inventories.\n",
    "\n",
    "When we completely shut down the cost of holding inventories by setting\n",
    "$ d_1=0 $ and $ d_2=0 $, something absurd happens (because the\n",
    "Bellman equation is opportunistic and very smart).\n",
    "\n",
    "(Technically, we have set parameters that end up violating conditions\n",
    "needed to assure **stability** of the optimally controlled state.)\n",
    "\n",
    "The firm finds it optimal to set\n",
    "$ Q_t \\equiv Q^* = \\frac{-c_1}{2c_2} $, an output level that sets\n",
    "the costs of production to zero (when $ c_1 >0 $, as it is with our\n",
    "default settings, then it is optimal to set production negative,\n",
    "whatever that means!).\n",
    "\n",
    "Recall the law of motion for inventories\n",
    "\n",
    "$$\n",
    "I_{t+1} = I_t + Q_t - S_t\n",
    "$$\n",
    "\n",
    "So when $ d_1=d_2= 0 $ so that the firm finds it optimal to set\n",
    "$ Q_t = \\frac{-c_1}{2c_2} $ for all $ t $, then\n",
    "\n",
    "$$\n",
    "I_{t+1} - I_t = \\frac{-c_1}{2c_2} - S_t < 0\n",
    "$$\n",
    "\n",
    "for almost all values of $ S_t $ under our default parameters that\n",
    "keep demand positive almost all of the time.\n",
    "\n",
    "The dynamic program instructs the firm to set production costs to zero\n",
    "and to **run a Ponzi scheme** by running inventories down forever.\n",
    "\n",
    "(We can interpret this as the firm somehow **going short in** or\n",
    "**borrowing** inventories)\n",
    "\n",
    "The following figures confirm that inventories head south without limit"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "hide-output": false
   },
   "outputs": [],
   "source": [
    "ex4 = smoothing_example(d1=0, d2=0)\n",
    "\n",
    "x0 = [0, 1, 0]\n",
    "ex4.simulate(x0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let’s shorten the time span displayed in order to highlight what is\n",
    "going on.\n",
    "\n",
    "We’ll set the horizon $ T =30 $ with the following code"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "hide-output": false
   },
   "outputs": [],
   "source": [
    "# shorter period\n",
    "ex4.simulate(x0, T=30)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Example 5\n",
    "\n",
    "Now we’ll assume that the demand shock that follows a linear time trend\n",
    "\n",
    "$$\n",
    "v_t = b + a t  , a> 0, b> 0\n",
    "$$\n",
    "\n",
    "To represent this, we set\n",
    "$ C_2 = \\begin{bmatrix} 0 \\cr 0 \\end{bmatrix} $ and\n",
    "\n",
    "$$\n",
    "A_{22}=\\left[\\begin{array}{cc}\n",
    "1 & 0\\\\\n",
    "1 & 1\n",
    "\\end{array}\\right],x_{0}=\\left[\\begin{array}{c}\n",
    "1\\\\\n",
    "0\n",
    "\\end{array}\\right],\n",
    "G=\\left[\\begin{array}{cc}\n",
    "b & a\\end{array}\\right]\n",
    "$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "hide-output": false
   },
   "outputs": [],
   "source": [
    "# Set parameters\n",
    "a = 0.5\n",
    "b = 3."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "hide-output": false
   },
   "outputs": [],
   "source": [
    "ex5 = smoothing_example(A22=[[1, 0], [1, 1]], C2=[[0], [0]], G=[b, a])\n",
    "\n",
    "x0 = [0, 1, 0] # set the initial inventory as 0\n",
    "ex5.simulate(x0, T=10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Example 6\n",
    "\n",
    "Now we’ll assume a deterministically seasonal demand shock.\n",
    "\n",
    "To represent this we’ll set\n",
    "\n",
    "$$\n",
    "A_{22} = \\begin{bmatrix}  1 & 0 & 0 & 0 & 0  \\cr 0 & 0 & 0 & 0  & 1 \\cr\n",
    "    0 & 1 & 0 & 0 & 0 \\cr\n",
    "    0 & 0 & 1 & 0 & 0 \\cr\n",
    "    0 & 0 & 0 & 1 & 0 \\end{bmatrix},\n",
    "  C_2 = \\begin{bmatrix} 0 \\cr 0 \\cr 0 \\cr 0 \\cr 0 \\end{bmatrix},  G' = \\begin{bmatrix} b \\cr a \\cr 0 \\cr 0 \\cr 0\n",
    "  \\end{bmatrix}\n",
    "$$\n",
    "\n",
    "where $ a > 0, b>0 $ and\n",
    "\n",
    "$$\n",
    "x_0 = \\begin{bmatrix} 1 \\cr 0 \\cr 1 \\cr 0 \\cr 0 \\end{bmatrix}\n",
    "$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "hide-output": false
   },
   "outputs": [],
   "source": [
    "ex5 = smoothing_example(A22=[[1, 0, 0, 0, 0],\n",
    "                             [0, 0, 0, 0, 1],\n",
    "                             [0, 1, 0, 0, 0],\n",
    "                             [0, 0, 1, 0, 0],\n",
    "                             [0, 0, 0, 1, 0]],\n",
    "                        C2=[[0], [0], [0], [0], [0]],\n",
    "                        G=[b, a, 0, 0, 0])\n",
    "\n",
    "x00 = [0, 1, 0, 1, 0, 0] # Set the initial inventory as 0\n",
    "ex5.simulate(x00, T=20)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we’ll generate some more examples that differ simply from the\n",
    "initial **season** of the year in which we begin the demand shock"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "hide-output": false
   },
   "outputs": [],
   "source": [
    "x01 = [0, 1, 1, 0, 0, 0]\n",
    "ex5.simulate(x01, T=20)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "hide-output": false
   },
   "outputs": [],
   "source": [
    "x02 = [0, 1, 0, 0, 1, 0]\n",
    "ex5.simulate(x02, T=20)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "hide-output": false
   },
   "outputs": [],
   "source": [
    "x03 = [0, 1, 0, 0, 0, 1]\n",
    "ex5.simulate(x03, T=20)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercises\n",
    "\n",
    "Please try to analyze some inventory sales smoothing problems using the\n",
    "`smoothing_example` class."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise 1\n",
    "\n",
    "Assume that the demand shock follows AR(2) process below:\n",
    "\n",
    "$$\n",
    "\\nu_{t}=\\alpha+\\rho_{1}\\nu_{t-1}+\\rho_{2}\\nu_{t-2}+\\epsilon_{t}.\n",
    "$$\n",
    "\n",
    "You need to construct $ A22 $, $ C $, and $ G $ matrices\n",
    "properly and then to input them as the keyword arguments of\n",
    "`smoothing_example` class. Simulate paths starting from the initial\n",
    "condition $ x_0 = \\left[0, 1, 0, 0\\right]^\\prime $.\n",
    "\n",
    "After this, try to construct a very similar `smoothing_example` with\n",
    "the same demand shock process but exclude the randomness\n",
    "$ \\epsilon_t $. Compute the stationary states $ \\bar{x} $ by\n",
    "simulating for a long period. Then try to add shocks with different\n",
    "magnitude to $ \\bar{\\nu}_t $ and simulate paths. You should see how\n",
    "firms respond differently by staring at the production plans."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise 2\n",
    "\n",
    "Change parameters of $ C(Q_t) $ and $ d(I_t, S_t) $.\n",
    "\n",
    "1. Make production more costly, by setting $ c_2=5 $.  \n",
    "1. Increase the cost of having inventories deviate from sales, by\n",
    "  setting $ d_2=5 $.  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Solution 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "hide-output": false
   },
   "outputs": [],
   "source": [
    "# set parameters\n",
    "α = 1\n",
    "ρ1 = 1.2\n",
    "ρ2 = -.3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "hide-output": false
   },
   "outputs": [],
   "source": [
    "# construct matrices\n",
    "A22 =[[1,  0,  0],\n",
    "          [1, ρ1, ρ2],\n",
    "          [0,  1, 0]]\n",
    "C2 = [[0], [1], [0]]\n",
    "G = [0, 1, 0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "hide-output": false
   },
   "outputs": [],
   "source": [
    "ex1 = smoothing_example(A22=A22, C2=C2, G=G)\n",
    "\n",
    "x0 = [0, 1, 0, 0] # initial condition\n",
    "ex1.simulate(x0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "hide-output": false
   },
   "outputs": [],
   "source": [
    "# now silence the noise\n",
    "ex1_no_noise = smoothing_example(A22=A22, C2=[[0], [0], [0]], G=G)\n",
    "\n",
    "# initial condition\n",
    "x0 = [0, 1, 0, 0]\n",
    "\n",
    "# compute stationary states\n",
    "x_bar = ex1_no_noise.LQ.compute_sequence(x0, ts_length=250)[0][:, -1]\n",
    "x_bar"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the following, we add small and large shocks to $ \\bar{\\nu}_t $\n",
    "and compare how firm responds differently in quantity. As the shock is\n",
    "not very persistent under the parameterization we are using, we focus on\n",
    "a short period response."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "hide-output": false
   },
   "outputs": [],
   "source": [
    "T = 40"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "hide-output": false
   },
   "outputs": [],
   "source": [
    "# small shock\n",
    "x_bar1 = x_bar.copy()\n",
    "x_bar1[2] += 2\n",
    "ex1_no_noise.simulate(x_bar1, T=T)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "hide-output": false
   },
   "outputs": [],
   "source": [
    "# large shock\n",
    "x_bar1 = x_bar.copy()\n",
    "x_bar1[2] += 10\n",
    "ex1_no_noise.simulate(x_bar1, T=T)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Solution 2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "hide-output": false
   },
   "outputs": [],
   "source": [
    "x0 = [0, 1, 0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "hide-output": false
   },
   "outputs": [],
   "source": [
    "smoothing_example(c2=5).simulate(x0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "hide-output": false
   },
   "outputs": [],
   "source": [
    "smoothing_example(d2=5).simulate(x0)"
   ]
  }
 ],
 "metadata": {
  "date": 1584334743.6048598,
  "filename": "lq_inventories.rst",
  "kernelspec": {
   "display_name": "Python",
   "language": "python3",
   "name": "python3"
  },
  "title": "Production Smoothing via Inventories"
 },
 "nbformat": 4,
 "nbformat_minor": 2
}