A.1 Solution of linear equations.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Numerical algorithms by examples\n",
    "\n",
    "## Contents\n",
    "   - [LU decomposition](#sec1)\n",
    "   - [Jacobi method](#sec2)\n",
    "   - [Gauss-Seidel method](#sec3)\n",
    "   - [SOR method](#sec4)\n",
    "   - [Thomas algorithm](#sec5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import scipy as scp\n",
    "from scipy.linalg import solve_triangular\n",
    "from FMNM.Solvers import Thomas\n",
    "\n",
    "from IPython.display import display\n",
    "import sympy\n",
    "\n",
    "sympy.init_printing()\n",
    "\n",
    "\n",
    "def display_matrix(m):\n",
    "    display(sympy.Matrix(m))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The problem we want to solve in this notebook is very simple:\n",
    "$$ A x = b$$\n",
    "where A is a $n\\times n$ square matrix and b is a vector of dimension $n$. **Our goal is to find the $x$!**\n",
    "\n",
    "It is well known that the solution of this problem is simply:\n",
    "$$ x = A^{-1} b $$\n",
    "however, inverting a matrix is not (in general) a good idea.  \n",
    "Matrix inversion is a slow operation and there are plenty of algorithms that permit to solve the matrix equation with no matrix inversion.\n",
    "\n",
    "I will review the main methods by examples.   \n",
    "Let us choose the following values of $A$,$x$ and $b$: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Matrix A:\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAHgAAABkCAYAAABNcPQyAAAACXBIWXMAAA7EAAAOxAGVKw4bAAALDUlEQVR4Ae2dX47VNhTG71Q8V3Qq9b2XHVBYAdMd0LICYAdFPMEbgh0AKyiwA2AFCHYAC6hUGHUF/X6ZOHJynXvtxP/mYkueOE7iz+d89rFjn5s5efTo0fXNZvNR0RXePH78+A/XhZZXhwbEz2fVZOuqja6dXLEuPFOam+3wxT5p6So18NRRq9+Vd5t8m+DnYrxKQlWv64qfbEF0flXnp7XW2a7rkrTkoleeKb5S+nyuDF17Mb2mPLJ2CJ7eNzrvAR/0mTd0/Kr4QPkjxY8einfyXjgQarBIE367OKT72+M+tBDAfq38d1ZeiiRD53OisFzlnyv/J9cFO8/uwXb+KK2CaE0A0fW7oDSm4SN5iqmFpTEREBor80bxiXBnW7auxwpPhXPfLkznEHxVkXqkCjdVMHp1WVV6tulse/G9CFYJkDkVkt57T/mvFQ+2JN2zJnwSVvbJXi+fawJ6V8K8V0xJ8Eb4Q4cyylNeN6HS0Qv7B/PggSMt5rMKNabR3E4LoyV3oCbziI7XJMuOkjPJ92EGZ8eizNzXZfsS3JkKETlnEqfE78O8TNdQ8m3J/XbSuLFojI/JgvB2eqjywH0SAuplolXwnHlkTNzoupn8hGAH3SsMhgMa0s+KWAzG4KS4Kp91ABTNjPSb0ox79GoI3yFA+cmC8JCZtwmvsddUxLcHm/uHo4AgF9AgwKGAsATE8rrwrBcQTCZ4DB1JgzBo3OZVhB4EZtKGNSMQ2MSgsJhgoTC5ooWzQJI0CIOZ+jA8KM3MkmEjqZlEKGHRe8Gm54JJo2Y+0r1nKp08CAvMMx3BDwqLCBYQiv2i45zpDqrEwpshedsLv7CI/Y+pbIYFGhdvDMjLhMvI/FLnWJYcgTcY1+vSQexggiUUQrOClGV2KRzGO9erihEupZIxiaMhSHVh7KU3g5t8iBAGAWth1gK6DN8/QQRLOICu6Wha8UbppL1IeDcUXSSeIqTwk4yHKhdMXgHPwbGD8uhNEN3Vwb4WO93XY6tyd+rhg+VNsICYVN3UcdSilbe4dflUUPe8ECY9ZhroPcFj0rSQuXNholCWA1GuK9AAkuFbgDRwwqIefOXi2f1/eyGZVL1TejqxYfBPOdFiiZQ4rKQp/Vdf48GS7Jdg8VXKZ1nyluLQg5RmmGLoWDQuBtaGhkQY8C9O/f56Eayi3irSkhFsGpKYSAOCEhVZvTENC7NIa/5VeYuENmUfOqp8GvRd3ceEyu5BOTYbTPXQL3LOrWyZ+5xHL4IlnMtEOgtMkSl8esrQg1NgzJUpbBSc2lLMwW962Rev9XuPwbM1aBeq1kAjuGp61leuEbxeh1WX0Aiump71lWsEr9dh1SU0gqumZ33lGsHrdVh1CY3gqulZX7lG8HodVl2CN8FaUWGzYRSUd8wOdyNZL+uJ11JlL1xJ5/Ot6mB2sdhdYV04i9O9GnFJbDYaHiqawHnQOngIwWaxnZ5s9kOTO5/3Ci7idF8Su2d0x0VWdQpyug8huIjzuQTFq2K00SAhczndF8OWjOzcuTxZ2N3ydrr3HoNVaKlwJuBSTvclsdnBW+0WdRkIxmuCPeG5vV/GpVShJHYUp/sQE73pzQYKzel8PrcX283qVSf2a5MElV0SO4rTfUgPhtgizudT9qR4yLVnt9Nbkp3nxO4b2Cqne2+CBVbM+dzBVjan+5LY0jkOjQxNjMcMFzTqIKd7CP5RkWCOF2d+f3ldSu02O6qJhC7mdJ8TW1jMopc63f9ilObVgwVW0vnc1HXTC53N6X4AVqIAdhSnewj+rxfEHG25TJrVI9ds9ZQbJHyyiY6pgDAwV7md7jv43NjCQ9drnO7/MXrz6sG6uYjzuamkBGZSVcLpflMCW5iMu1Gc7n1fk4o5n0vYrYQt4nRfElsyR3G69yJYghZzPpegxZzuS2JL51Gc7r0IlqAbSNZhtCZMfuog3GJO9yWx0avwmdvMLbZ4qd53DPYqrN1UnwYawfVxErVGjeCo6qyvsEZwfZxErVEjOKo66yusEVwfJ1FrBMHm9edr1JJbYSU1wM5T90oLwVtFdmi6dWUdW7j8GjiTCN0XEZqJvvxk7pXAi2CtqLDne0/RtaO0FyDGReGy2TAK1EUR69PCHg34LlWiYLo8mw6u4tj5WPwdCVeBk7xiTvemHpKP/Vk7/K285NukAK7B9iX4pnA6D0Nbwj6NvTe/OnBcjpJlJoA0tGxO99RcysVKsJuFLzY6II9zYtJ18hjYvgQj1I6Pbl8BrqX+tG4pp3v4hEh6a0cuGQoMVTS01GE1ti/BH2Yk4acVq3Y7ZsqtIluy4UWC1bhlV0j5O43dvh4jHQvbd5K100NVAcakJzGEqbgM3iWZX+BhkTtEwfbtwSPhJDDjUvDXx0eFBJ4IEy9DTGM2p3th4YuGswO9+I7iv4qMu0G/8NP9S0IU7EUEq7b03umscokQvs9A7PAPovoGhn8wbqX22Ohbnu994BJuCGeYSCrN5/3vKu5Ytovbo/yNgu1lou3qSih675mOKRVrQ26Eld3pXphGwVgq8+sCU69XSvD9SnOPyY9ytMpdjR1MsCRgbMgxgzykLOqQw+neJSs/64RczGjKsBp7CcHMLM17aUrhurLVmos43Qv3vBfOHF2yYs2ih5jYQQT3pgOh9gkdW2B6Cb1lGk7JUJ1SriYxDLmwTV1cPcxcW3uMgh1EsGpsTFK2HizMkk73LM+6ein/FJPXp5TzkCjYoQSb1pyzB3dO91LoEKTYLF98Fw6zZPyThzcGpdHBn4p8SiFZiIUd+pqEOYTcuZWt6AJL0JJO9xvhM4Nnxa7bX5WADA184j/l0NDpMQb2ifUv3vlhV8oxJTr5rUC3BsQji0JYvpNQE+0useVWq4FGcLXUxKlYIziOHqstpRFcLTVxKtYIjqPHaktpBFdLTZyKNYLj6LHaUhrB1VITp2KN4Dh6rLYUb4K1KoLbyigorzmfjzRS30nIWnRx53Ojvr6x3dcR54OsISc2HUjCPbQE5DzIHyyEYLNFSE9mzZqdluRffBeGK+AvnHyx3wWsvJzYbHKMGrHOj+6L7yM9S0CzVTjKz3GSE1tY380X3wfuJDTWg+1KYtZQABv33NUO9t6TrKzanAe7I0VPPRzn7457JTc2e+63JS8+aYy9JuB8YPamTd7sMWQM3giohPN5V3lhY5q9BZuVeMGFEtjC/H6++C5ht+IFH6jsDgmFsfndl7FY9NwzxaDJpbeJlqDZnc8ljAm8EhlBTV6uYzFsyYyLMvMNxmMc/GjowV981zOLAz0qqfO5hOzcTxbXcMWDFWDTqfhdMn5pTLjMLzm9f1Xh1YNVeCnnc1osq2WlTHMR7L5NYpKH30ORJz2w9kBvZtKFuT4YfCdZ+EObhQ67UDwMAQ4aF+wCDqQhmA+Bs7hgB16XsBzk07pHirBvXJEuhi15IJDGhXkeBeUhL0R3uh9ddJz4EozzuUuJtKJkzt/CpOyd8pX/jXwdjclyiLYuqzA2E0oijdhlvWgAO3pxSexlovVgMedzV6WV17XwmWups3Nh03i7ZUlbIBHOnIQh00W8fWuX9vaLpjXpCdOLMQ+YbCYAO2akKznBH2HxHkw9zPiDqfqg/GcJ4EZFlsAWJkMRmw328Hhws0HPdRNTHU+8CR5J206q1oBNsK+JrlqgVrl5DTSC53VzFFcawUdB47wQjeB53RzFlUbwUdA4L4S90MEi9vROtqySLSZMwdp5uAbEz2c9tZ17EoJ5YR75/Vg3e71MW/e3ZH4NDF8fcEH/Dx/JpF05sGX3AAAAAElFTkSuQmCC",
      "text/latex": [
       "$\\displaystyle \\left[\\begin{matrix}2 & 5 & 8 & 7\\\\5 & 2 & 2 & 8\\\\7 & 5 & 6 & 6\\\\5 & 4 & 4 & 8\\end{matrix}\\right]$"
      ],
      "text/plain": [
       "⎡2  5  8  7⎤\n",
       "⎢          ⎥\n",
       "⎢5  2  2  8⎥\n",
       "⎢          ⎥\n",
       "⎢7  5  6  6⎥\n",
       "⎢          ⎥\n",
       "⎣5  4  4  8⎦"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Vector x:\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAABoAAABkCAYAAACYYiB/AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAD/ElEQVRoBe2a0VHcMBCGj0yeM5fLTAowHRDo4NJBmHQAHYTJ090r6QAoATqADhLSARSQB8KkgvyfsZw9WbZW5iYPjHdGSFqv/t9aydJKx85qtdqbzWa3Sim5Wq/Xh6kHsU52d9JVsZ66nu28Ng++qYyxlXtbyZRPE88/SvcJvSU6E3MJ8Aau2p5vKFSRDlWHCGVW1BhXXyp9UPkx26AxsD3qbSPAuR5eKD0o7Sslx0L6XvES8eb1pBDpF5XpVZG8KrJ+hvFENNp5k+sm17UeGDMZ3jWtFy2Ko+BagsDR0sNCiiyfstmldKz218o7K3dj02YlRK4NsEWOCmNcF0H4qhORz08Jq8l1Caf4VJPrfH5KWJUsQZXanzQYxHbEeCda5342usHMRSQwSAiZiaVrUZlY+xad0k2j7s28kwHQY4sicHpHYBlWdfu4U/YSsTXcCZzQ2Ao9mUtPjwfFSwTgvQDpQUriF+jYeMeoby+qY3C9QHZCeHvUeUOBQ4LLwkzs2FjFaCKBMAk4enJSzApEbxqrkGcbCfxMRoxZn0sDxvtQKO6RwI/UeKG8/aYC2FAO0Z/GIOS99gLnPLpre6JyRepp9Cvo3T0SGIN/oDwefMgfAmBf7p3evDGDfyMixsfKUrrshHARCfVaCTLGJ5bsN0QDF5HeeDdGL627x6gUOLafiGKPuOsv03Ucptims1+320//DNkw6xAA11VKfO0LpW3LUoD1SvLyxsi1BOFPLUO4OIRcBCPUT6XPxnSyc691ABOVBiKI2R44kR8qXQE2JN4xYtU+asADXujJ16AYyr1EbAXEdKRaRNqWg24od42RQHn7txbI9C7eCK1ZW/b2qG1AQSR8H3U8rnL21oQ2rh5hiAiUuAGSAyXc+UPJJaVEgJMgZdZxbNnqrAN7Q0TAlGZCcPk033iYqLjGSEB7pET74DrcOSguIiHwsw9uyr55H5t3jHARMR25lf2mEj5e+2yj7CWKo9OZSJkM9PA48QIbJFRcRAI6V1oq2Y+zUnvXQdlNhKFIcE/WRdimxDsZUm2LdBNRkbus8eQ6642i8uS6IndZ49Gu05LEHmXXPovbKY8mEhLXAe6DwSgi9YSfsYukmAiXiYENMN4EB4mLiYT2WWSuWM4yFxE1LnNPgFFEIqnU8FH5qP/iKOkRsUGxy0KvXEQi4NgyymVuosZl3G2Pclkg8kRBjA0XgvGNPdOc20f03K92QrJAQp4lEkAy+pH+t9oTVOYucGs+1xjVlt0/BI8kl2R7FKOoB0wK3IkQVOK678oHrzvHELUn85rK+ec5rnNSPJlNREXussb/zXV21vHTmn0Jylv5j0GAIAo3kNRjKVnfUv8x2OL9BQezDxkTjeNuAAAAAElFTkSuQmCC",
      "text/latex": [
       "$\\displaystyle \\left[\\begin{matrix}1\\\\2\\\\3\\\\4\\end{matrix}\\right]$"
      ],
      "text/plain": [
       "⎡1⎤\n",
       "⎢ ⎥\n",
       "⎢2⎥\n",
       "⎢ ⎥\n",
       "⎢3⎥\n",
       "⎢ ⎥\n",
       "⎣4⎦"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Vector b:\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAACUAAABkCAYAAAABmvnKAAAACXBIWXMAAA7EAAAOxAGVKw4bAAAGPklEQVRoBe2bT44VNxDGm4g1IhMpe15uwJ8T8LhBgBMANwBlNbMbwQ2AEwS4AeQEhLkBsEdKGOUE+X49XcbtttvlnvcGIbUlj+1yuerrcrXtas+7dHh4eL3rug/KufTm6Ojobq7jPDTJ/Kjxm5wM9V26HHU8Ux3mOH2KGzusP83IuiPa79BjUM+Fcl8gRhik58WIoIZokCagIFaTBqdP+adoJ6WB6sM9Hql8VOJJ6bGl0r5RW0LxgdfKT1R/R6dK2uTfaBcS/UXQuTE/5YgFGsKxSg9o4Lmqsjjl4n1ckDVLdllKwplrpuF2LE10nDOb1Af/6ZCzPCWi11L4w6kUocSb7ot/4tCewS5LSdBN5U/D099X/R9l/Oi1aPF0itT7GtP2vG8s+OMFhe+QbgrEk7Nqr/yr2g+U30Q0XgisWvQ14y2V1emTcAN0XfV0Ol5J8MuIBz28/ilfSX+WXgUVjco9OdsToJneTmAeqlg8bcggVUFJkTm3lWcjx3834mParqrMgR9zV1pen8KZUVpKAKH/lkCxnsWJpQHQ0HlZgk/GTHHdC4opSZUh54YyTm1voJX09Ul9X1V5p9J92qhOH5IlkLcLwWHfUx1fuqf8QHkuwUd2J6+lAHYHUMrmyAfSclvt7L428NmUb9XG0u9VckSaTW5QSJHAqj+YNvG6TwU2xkrX9BnzRZUrKK+lV0utlvJawMu3+pTXUk0rek2oVvGNeLbKr1SfO+rMiloMSko5kqRBJjT2RqJtFZPEieLnCTUhLAYlOWyw6WZ8SzSOL7mDHhZ07Z2LQOlpi0Gm+iaxoGhMa6cyBBi0S6kZlAQzRfhLzmfeFxRx5NntIS9RVAwyc5YQjYPhcSJjttm0TklBU5ApfqaN0Cz1vd2AGhS0BplYKRyhZ5FEnS2Wagoyh4fgGDwJJiL92Sqgrgw9Vk4YJXhJkMlxOLc0TOQPhF+to2qp4YmXBJl8PvrXFLWULAn/DQOsTMfjrE1Bph6EkIpxLQ7+xRRX16nBJyZ+IfpckNl/W5CSRZaqTp+hz5RYg5xLRj/NddZoVUulAmQhNlymhlQKMpk2AJVWeMYW0xJQ1SBTwHnrqqeBEqrzTF9J5rnpKyivCVdL/fCW4vXlNV+0+nqf3sHHrtEvN/jURpkF8UD5e6atlPdfCX9sR9cqTcAwSqJxpMHSO00t28xfgJB2O47Ypstn61ESXz8NAxG34P7GvTm3TB8vAoKxGICI4W7EygCtzOX4R5UWPR+r/Vltt0VbLHUiwbXY7aUAHIgvfJZWnXF/i471JoGqaJPUYqnJ4AyBI3DuXM6Uc8yxKc8M/UbaGahIYW6949KSZCfSs1bhb8v0dVJMVMPT/qKMjxyL1ju+SmJCkbLrHfwkxlRTi6UAw3enZ8p8PSF/UH0bacH5c4ptOdnt9Ek5dzOnBkB1fIetIX79+8ujGKjqALJxOX8zkaFssVQYFFVQYheQnQCg/JryXdUfKxOyYzk7q7tAuXxKwt9KMK/6ZKEUnRSmRTwAG53jB3DwuUB5LcVbExQjfUis1p2U9s5uxEzJFHJfCOBqcllKUl5IYO7TIE4eAlXxsE6xgF4zACp5GPhKVlbXOHktxYfV2KE7te0TY7zK4z/pOsW3UbacmjUDMpelJJAL6fRWFOXBIkgUD8sFa9IfKm26GResCV8tuUAhRIJx0pED54SLLzfNOdYizTt9RQH76FhBea26Wmq1lNcCXr7Vp3ZuKa3UdnoMskUbBaNqc7Z6CD0wLai4txnJ9gSjAGfjvrCbUdv9Ucw+yHmcwCE+I134zagnGO0EchJwirbRA9DHg1RTy/RVhYnBzuIp795vRlOFoZ2zhGjNN6NNlpKCYjAakEUV8TNt3Iw2nbFaFk9e81owGkHqq/u9GdXTeoLRAGqw0lZl01EYAS2WCgqjCktDCEYjOlWOzq44LxnnA6Wnfatc+hkLMnMrOOGWrW2p3tm211JNwageAJA4ebywzgKJO72gCEZzP67YSljOZ3gI0l4t5Q1Gz6B8m85FlnKtU7KSKxg1RCpPlAFUWuEj1mnVBYphAFNRDUYj3vVmFGPsNXnfvr2CSIWvoFKLlNrx28d9Ssp34b+EBACg5l71RRtq+mSZ9uw/ev0PRGQak36D6VMAAAAASUVORK5CYII=",
      "text/latex": [
       "$\\displaystyle \\left[\\begin{matrix}64\\\\47\\\\59\\\\57\\end{matrix}\\right]$"
      ],
      "text/plain": [
       "⎡64⎤\n",
       "⎢  ⎥\n",
       "⎢47⎥\n",
       "⎢  ⎥\n",
       "⎢59⎥\n",
       "⎢  ⎥\n",
       "⎣57⎦"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "A = np.array([[2, 5, 8, 7], [5, 2, 2, 8], [7, 5, 6, 6], [5, 4, 4, 8]])\n",
    "x = np.array([[1], [2], [3], [4]])\n",
    "b = A @ x\n",
    "print(\"Matrix A:\")\n",
    "display_matrix(A)\n",
    "print(\"Vector x:\")\n",
    "display_matrix(x)\n",
    "print(\"Vector b:\")\n",
    "display_matrix(b)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The matrix A of this example is invertible!  \n",
    "We can check it is a full rank matrix and calculate (just for curiosity) its determinant."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Rank: 4\n",
      "Determinant: 194.0\n"
     ]
    }
   ],
   "source": [
    "print(\"Rank:\", np.linalg.matrix_rank(A))\n",
    "print(\"Determinant:\", np.linalg.det(A).round(2))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The first method I'm presenting, is forbidden!!   \n",
    "So... read it and then forget it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The inverse of A is:\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAARgAAABkCAYAAABdGS+CAAAACXBIWXMAAA7EAAAOxAGVKw4bAAAWyklEQVR4Ae2dXZIcNRLHeybmecPgCN53uIE/ToC5AV5OYLgBG36CNwe+AfgEBm4APoHBN4B93wjMxF6A/f9qKivUaqla1VJXV/VkRqhLUukj81+p1EeXShdff/31g81m85tcin765ptvnqZueJwj4Ag4ArIPvwuF6xQSundxFdx4KT+JQ/ojDLjfEXAEHIEIgW+jMMFP5T7DExqY72Rx3KCAitOqEZAeP5B7Fwqh8D2FP3QdD1Gp9wvP7+NSFEfUjoGJ0zUPq2Kzdn+q8I/lvp3ywEvyKw2K9DxgnvCPiv8liJvNW8LzPmZUBtPYH+Ueyn+TSq/474L4D+V/lksbpJvdK54O1gHlZSj+757pR7q+J6z4LWOiuDeK47lbPH7o4e3lPH8bYPtljwx4gTXts6rdXARrMB+rsKONYFQ26zwvdP0JIXRFCOI+lX9vvUpTlF/pGIkZUFS1UZjG+VrXru4ucoYf1VfEc4oV5QWfV3I0IhoTRuYDxd/oOlCfjnqQm2nuRlfSvpHDIO3FljxzkHipwQOFR0aG3x3Jj7H6Sg4dGhqC/Ez1MbJgiPw8d3RvCzvFnQ1JthpswQljMrQb+RmB0G6eyj+p3Sj9F8rHs7q4lOfo1Fd4L2RUfh42jIc9b5KX0vx9OoCO6ZkiwlFNfL95uJTnXMXgI8fD5aG/zqVTPEaIoX9nXEgnPz33r3J7sSX9HCSeULqDdUB5MSZDA4BnlcloBj2iIYT0TvcwxhdydJyMcs7ZuNRiS/4vhFE3remBNINd1W5mMTBimH+ibLja899d3ur3iQTDgo5RaX6mXUMPN1bgDPdKea5lBaVIjVLAuwTb2vpL89fi8UQV/Z7QFRoChuu6lJEzTFeLLbqCAR6MsPAc/DV4zWVgUI73CUatYXB/jErzY7A+Ezg/R4pI7zd3b17K85jco/cCGVPY/tlnZnq1BKrFA0Pyx4ji7+ukloDBsXiowlaY/iLHiM9GLRv5bTRT1W6ujiWxlStGSx488+UkTcmvtLy3w7QLcP6SnyE0oxoMzqR5pPIcTKqrSubSilUP0yiSp/C735dz8p69BR4qg146Raw3bXSfXngghbtpgyLAAQxYg9lKMyResUcyNdc1lYnB6qak8u/8SzQFrjlGMKb8NyOMjYE0Kb8AQRENFEACrLkVaxLPI7iU3MJwpoxI1/B0bwzbkvJbpDkKHnrWyIjs9s+S8YrMP+j+Sznu4X6TH104N2qGLXjKsWjOWhdthnW8KprDwJQwaL1tSdpUmiG/AGL0gjFj5MKQDwVk7m5DPgUXQQPPldywgL2RfEPjkZ+GBwaQTUNvQ8v9PQQPFncZtQ4L3IinMP8qmfyEwQBdqBruU/ZKqQhb4cTiOEaZTvq1HEa5qt1clQCmSugR+NuTaynxDwhW8P1IBrO+tl6QSlqcX/UxLOavWfu3AUUDIBTxlfzMNQfFS1XWKK6Y59r6kEfunyqHvxkxLCgTa1HdepSuSzAwzfGQrBgL1mRyUyfd3iJwYNH7Wm4JmAzMiZ9Ttq+Bj9AjnjDctBXeIdt5PSJMO+YvNTBU9HCsoNw9mJTjNiDGZHHZBz4xP1MiGttAyg9QjGZ4N4Je/uhrMRN5Hng91EN9ymtGtStGcWABZbG9vX3839Z4qDw6Ev6a3/nHUHE/9/dy+mo6d3zBC2sAHyXN8TtaCnnlSJOSy+JGdUD56Zg2ur7jGhBTJNrMwe1mrimSTVUC3juvjWCG1es4QR/em1/gAOY9XXlYW6Q4AMawWH1b948U2Mvzkeq1YlGauUZsVufYtQkeepaMSHm3ZRi5yM+ohKkw9EjOGlYX0f90z17p4kYUplmrvxZb3h1jOpTCrQqTuQwMUxQefExYbeZ9O0YhSrg3f18G1twULSqiUzoexFy0l+cWjEhe/pbnH7NBOXo/vU68+NmiykPLqMZDcmE0H+say4XRsWnY97rPiDUm8Jjz+cf1HzNciy3tz6ZEIZ/WZg/GbRYDowfOvzrvdUUROpKfBvEvuW6Rkkji5P6W23obV+Gi/CqCXo0549DY+nIZUvNX9ehQkbStqJRneJXbkTni434fTo3AMKjWuCwbCvelyl1Mb12Lh/IjJ3KBF6+hD66XlUYCdfG33ttfpeWfEWgY9dwGz+NX8hW1D6XL6RoGm6nlQEpLW6UdoUeG7XC/1DPnXiSYZV0AZlnUfSy3826ChGGthKF9vKZQmp9e7rlc2OhOstlRMpTynJOZBgXR+1IWBgMjibG0v+I38tt6C2mgk8h7W3X+twYP5QWj3OiUUfCwhiE/6WyUg1FGF859u0CtrqFjoQEGw4M2Owp/OvRuL9JsBkYVOjkCjsAdQCA0MJd3QF4X0RFwBE6EgBuYEwHv1ToCdwEBNzB34Sm7jI7AiRBwA3Mi4L1aR+AuIOAG5i48ZZfRETgRAm5gTgS8V+sI3AUE3MDchafsMjoCJ0LADcyJgPdqHYG7gIAbmLvwlF1GR+BECFydqN6jVNu/Is4rz3zN7OYolXihjoAjUIzArAZGjd72zLAXiR2v7HUo2oDYGw/bX8IuT9tfwv4cI/Yh8SEi9kFYXHi9UfwHYcQS/eLxYJyQR/nDL7exF+eZ4lZrcGvwUN5ryb9Pb4BtcSTe0Wf2o2UP3EsxXSqz0rF/iX17RoSb7mObzcBIGHZIs7mx++BTLxzfoOCrc6NGRvdREozG8IEh+WmElt+2k7OBEn+qPEY2pmjyLpMkVw1OKAj5wSo8hO0/Ci/qELZS9CvxKNWbUnaOnk7y8gzDA/eQoZiUf4rMW4etUYnyd18j0LXJh9kuizmvSChm2V3JVvGBaflvFEc47G1ztWBM4t3VGAvKsB3HXV6Vi8Fii/ngdIP87NAe6u8SL+xH/NXitIpD2Ephb4BHsd6U8nTsdJKZUXbJgXs5Vopk7rGlM4qJz6eEo5r4/qTwLAZGHLENPJzKGJNv5XkiYbHaY8Too+TQLcpL0Y6lTiVaQFwtTnzDIzV6A/sSnBcAwRYLtXiU6s1WpSsPlMrMEsUwIziWzHMZGIRmzSQmawzcH6Nu2oN1zyTqDJTu74xQFIdFf5HJt7Tog3EKjHQK5z97QVm7WhMdjEcvZJHerAmQAl5LZaYzPvohhVcFDFclCRR/rBwWIrOkMujJUsQi2Eb3U6Mj4q91m7Ne1rD2sm8Uh6hZnCQjQ+tcGvsiHnisgiRLFR4IqTIO0ptVAJRhslRmpeMTmXTIjHqPdkjhHCMYaxS50QdQlSgT6QYSOBgXGsyY8WD0glsDtcAJhUkZEbCCJuN8m+0kvy3w2GG8UG928q05Iiez4jHA9mVE2gkjxmRnfaj8cxiYEt6shy1Ja2lY3MUKbx26ZTcVT0Nj3YEh47nQPpy67xtLZhSlI/kxLmbcbUra3139ZR8eKQFH9SaV4QzikjJLNxi9oBusx9BOaDNNDylkivQPOciut6HgV4zQ872Rm9IDtjp4LeDk1it++Odp36Fb/Os0a4M6NU6qn2nSYg5hOzUeseKInxK9ibOtOpyTWfH8Y3msQwo/MtAwMHtJzGDlHu5NmEhAXjnupIyTxRUbgh6Y5KFbUfW5f1SiZO2CS8Cp5yH+S9+micU4t0BlCXiYHBP0xrKs/rpHZnTi6IcUYmD+1yNp12MAa8OvuGybZxdNYwQYRmPn0C0K1b2h8ciP4WK413Q+ST1HpiY4JXhkmrSkQ9gSLCajmuBRqjdJDlYaOSazRHovd09pGDhskeKYGbCWZ21z635h4L+W7tI8R74yB3yUqINRUcnBaxsJTSPZd+iWVWF1AeSaqAonYbSWQ9hKn0kVHlQyUW9K+Vp0un0y6z6GhZkFnXCK6KCLOv1U5jCuaIoUZjjEL0E4bY9zaWgA3bsquiIEB699YmX2cX8pnDrnBmWjF47f/GUhN17opWwIIFdDkqMKJwmKwsRGFdx4q3lto7lNLR7KDx5T9EbJF0W2iM1oYkuXJRs6XtNW+AeJbQGfyA1ly8/aTLNDCmcxMGIYYrTCG7WPdeXFL64INyg+gsox1flVLiROnUNZED6mIX9wgzhAexvErcV7ME7C7qUcSvlcVxQQAvMmvdFtcbP/HoyHOJ2qN7MLl6pQzwujCD25vXSGgHYxHLinNFVtBZ2Q41/HV7q+7+vh0nSzox+8FiDrXkfAEahHQAaLgQAbbi8u64vzEhwBR8ARSCPgBiaNi8c6Ao5AAwTcwDQA0YtwBByBNAJuYNK4eKwj4Ag0QMANTAMQvQhHwBFII+AGJo2LxzoCjkADBDAw/L/O3pXwv/AGRXsRjoAjcEcR4L2rbj8cBuZajrdja/YeKLuTI+AIOAIdArwg2L1x71Mk1whHwBE4GgJzbhWoFkJvBjLawjr6wWrVaHoBdxGBudtQUwMj5u27I+w18oPVMhpcg1NcpMpil3l3REt8by3hGjyUl07HPpvKLnrWEtlYy360s6UKzNAXpi+8yp/Chz1O3eGEurKf7XmQiPCkvUrNDIyY4YyVF7qGu6XtYLThWy0Bs4NXebp1IF2HYxTkx1hZftusxwZJ/KnyGNmYosm7TJJcB+OUkYiNcattTDV4KG+p3mSgW2d0DWaSeEob2jnuR3VPOpitiYFRpWxuuqdrZ1x4bPJjCQljLQfDwb0EYUzir7DRC1EuDWg47lVxO2UpDkXb6DrUT3hpJP5qcdoSSeV9tRWxskADPIr1ZmXQZNltgNlGZextQ309dIYxPVMEn88tamuXce4Dw0+VL9WL8rkEvtdinw7IFc/oww9W24/TgJ8wZah707shfmWeufRmZbCMsluLGW0yRfFopcnBbK0MDAaCuW9MNpXh/hh10x41GhpMijoDpfs7VlNx9GIvUpkWGFeLUyjS55LdjpwI49fkr8WjSG/WBEgBr1WYTWhDGKLqg9muCgQaTSKG941OyD/6jo3KwCqniF56o/up0RHxTI3uxMFqITiSm6lR955BGL8mv2Q4md6sCaeQ1xaYheXhV5nJNqT4JgeztRjBmPHIjT6Qo0SZSDeQBMS4IPzYwi2jF9waqAlOvULY18zWIHeOxyZ4xIUX6k2cbS3hY2CWbUPCko7fRsmkY/SU7OxzALYwMLmyw3g+4ziVWNz1g9V2UeMvaXvou3fPK6a53pwXPElpijHrOyvWSO1f2q0CFc8pHgwcDj6YrXqKpMpTay/GqFlc3ospJgnG8H9xB6sVC5BOWI2TcOk+RZgufnWx1XjEEhfqTZxt1rB4ZDS/lEMM+efW1km3cOh1rfpgtmoDI0YYrsNcahpkcUkhtiTqA71gizxYLcVvaVwtTsrPdJFXAYqxLOXtFOlq8Yh5VnkY3xK9ibPOGkZuVciHzCdTa8zEACOUnD4xJao+mK3awPQoMcSiAcRkI5jkECxOLAAR2A9Wi4G5DYMv50IxdQzpgQLXfTyjvrE1qzDfEvyz6s0SBG7AQyvM6PzRqZ01FekQ9+jMMIZbpLhJB7O1MjAofWqxFUvd4mC1eM2BV8KhsWH2bYpl/R6Mkx4sirVjqBXP2TgcQZH7J25ZCGxzczAeVozkxsDmDuSL9cayrflajVkvfLYNCVNmJTg6rtQIBwO0o4spUC9TkVPjxAQP8r2ujEA6kh8mOFiNN/86Ik7ub7mtNwQVxpICHPfZIzE4xbGoGVtSyobi+NvYhf5KjiqcMmKBheGRSbLM6Fo8lH+q3iwTiAlc1WIWVGU6k2tDdFjdtoAgz0b1MxUtPpit1QgGHhit+MFqIDFONTgNJetBsxBOA4P4JwAD/VbX+JTLLsGCf2rwWOXBag2eRQ1mVj1TI4zLW4sIr9KjJgez+cFrIarudwQcgWoEZJy6fzt1vbisLs0LcAQcAUcgg4AbmAwwHu0IOAL1CLiBqcfQS3AEHIEMAm5gMsB4tCPgCNQj4AamHkMvwRFwBDIIuIHJAOPRjoAjUI+AG5h6DL0ER8ARyCDgBiYDjEc7Ao5APQJuYOox9BIcAUcgg8DqDIzeDmRz2xYpjj1M11uRHnAEMgi4DmWAOUJ0y71IGz0421HNB6YmHbyGbIX53ygdG7Vsm7lt2mJ/xuKoUKZRvlUGRpW9RnwAaGdzmuIwrnbsC3gQZl9Y0Y5XpV0MiecqHUIQlTGKl5IsXodqcZiSP0hrevBacda+LO6gazMDI4bYIX3QwWtwPiG/faIBJWIrOScNUO9Ow1P8SWmCTDt8Ki+G4pUc8rK1PjlC69NxhpQZmI387Gpnx+tTuZ2TGHRvkSReD9Yh5S3Cqxd80TpUgwPyleZXOnSKjgv96TojXQnjGCBUUxMDI6bY3MQ0ZVBm+W/68N6D1ybm5/syi//2yUSZdh6k8mMwOznl5xSBnalhnwnsv1AaDIrhbyOX57pncX3yZV7Ee60OleIFAIvVoQY4TMERQ8JoxfQFbDDUqW/AcG8yXU7Okc5AQ0gNqd4q/okEgOkxqs0/Vvap7s0lE7jTuHAdCe/Bb3EruM6F19KhqMWhKL90hFEundbWR7kU/ymuFUitDMwTMWTDzpA3s4TcH6Pa/GNln+reLDJJGfhuxwdcTVD57cNfjB7XQrPgtQIwanEozc+UmlnGUTujq1rAxeC9gjI+zKU5JL/ydMNAlXlfjnkkazCpEVSu2qPGHyJTK4ZUNwrGQilfAtzqnVrV0bqcU+ClOhenQ7U4TMzPuh7f12UU87mc/THDV+yGzqr2WVcbGDFgxmPMEo4Zoan5KesHgdDVpysGhnOtGdo1A6YS2KkyVVY3/HOCcXksh7H9tbrQ+QqYG6+l6lAtDlPyW5t8pHYzfChe/r/knsk1WbtrNUXap4qMNGpoyC/BMSSdcaFA+ZmGYVjWNB2A9UEmArUkHFi4fCnHHPy13G/y21Sptvgl5G+Gl3BZsw7V4nBf8ptx4djleJT7gx72qyBN1bNvMYJJrb0YU2ZRGX7lqDY/5WJkWEzOfQU9V/ex4lvIdDBvwoETMTHCDHdZnxkM8sGF7smoOlDapRwotofb5O0l6FCt3kzNb2ukISC8KsD0kSlU9Yyg2sCgvHLiZWNWEb+RxaUE6dJMya+0fOSZw7UeWgXR1eqLoucNTpGpljPV9YAydH0XlcUUiSkTrslwNyp/K4jMisg9l620cYC8ckSnnp/FZXUoLm8srHoWq0O1OJTmD9LxzHLE0kM1tZoiYelSDNkIZp8lLM2PVTWFC4Xv6hFwcSML08ztL5Wpli96HKZDKVxqy54z/1x4LV2HanEozU+6MZ1pYtBbGRhe2OHBxUSPxtrAmKUkT2n+71VW6g1DemkAWxKVylTLM9jalCgsy57H0nAJeQz9c+G1dB2qxaE0P2uWqUEBbZYRZRO9aWJgxEzVgWKl+SV4dyBbqJXKy1uuEIubi6FSmZTuntzOYXSRIPf7sI0Iw9v8A8CwfyCV95kC9E78Vb3PuA/5TukRn1U6FPE+hteidagWhwn5mTbzDpXt/drIj85sHZYY4To52OxcpJ45mEWhWdTl79Kd91OU7nfFI9iwd0bhzYT8WF37W40Gx8IWeykW15AmyJTDhN4IYoTGw2cKyNCVbQHD6r/83A8NLBitbrPjjHgtWoca4ICu7G2LSrNRXaQjPUR72mmz3Z0JPyqTRWIM+UUzAzOhfk/qCDgCZ4xAaGCaTJHOGCsXzRFwBCoQcANTAZ5ndQQcgXEE3MCM4+N3HQFHoAIBNzAV4HlWR8ARGEfADcw4Pn7XEXAEKhC4CvKyIzkIdl5e4Ar//ozve9gRcATuMAKyD7xicZ2DAAPDexVb76QEiZu8LhyU515HwBE4LwSGF/VSYv0fYdcMy82DwNcAAAAASUVORK5CYII=",
      "text/latex": [
       "$\\displaystyle \\left[\\begin{matrix}-0.08 & 0.19 & 0.25 & -0.3\\\\-0.27 & -0.9 & -0.2 & 1.28\\\\0.27 & 0.4 & 0.2 & -0.78\\\\0.05 & 0.13 & -0.15 & 0.06\\end{matrix}\\right]$"
      ],
      "text/plain": [
       "⎡-0.08  0.19  0.25   -0.3 ⎤\n",
       "⎢                         ⎥\n",
       "⎢-0.27  -0.9  -0.2   1.28 ⎥\n",
       "⎢                         ⎥\n",
       "⎢0.27   0.4    0.2   -0.78⎥\n",
       "⎢                         ⎥\n",
       "⎣0.05   0.13  -0.15  0.06 ⎦"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The value of x, given by matrix multiplication 'x = inv(A) @ b' is:\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAACoAAABkCAYAAADwkaJHAAAACXBIWXMAAA7EAAAOxAGVKw4bAAAGSElEQVR4Ae2cW44VNxCGDxHP0YRIWcBhBwOzAoYdcFlBYAdBeZp5Q7ADYAUIdgA7IMwOYAGRMhllA8n/ma4+bh/f+nomokuy7C6Xy3+X3Xa5jmdunJ2dHW82m89KMXp/fn7+MFYxNU/9fJHObUyv6m7c9CpeqoywT1/9h5nLLyL674v3AL4P9JWQLwmsg0t9v+4w9CAerD2gMIukxkyVd0p3VL4qNvAEJG9W+0vs20ovxKsyjm9RT2W3KGVH4rxRulS6qxSdS+InSTr4Dp4rf49Qo/Oz8vtKRbA/JDV7FVJ0pfRQ6anYb72qqqLaPZHgkXIHkkYqMxo8v+K5RFVAS0oq6lk5LiJyn8Q7FWhGLEtLAT0VCqZNSDbk1GdpdqA11hLCW1mUqpwdqAcit0Jcm6EvGeznksASFo3NTcNlQ866mqXZgTbLECBiw2s8+6iSYGcH2vT8Ufk2gsIsSn2WlgLKlsuOFtIdMS48q4f17fMQoDbxzRqtMpYipX+VOm6jnnE4LpU7B4MGKjPsj5R+5blEVXs9SqQYq0C2OL8Tj7n1QbnzfJSz1cL7A8GAsB5OyIlyPh7ye3qO7Viq6lIfoFUOtDrGK9oj8a/ExFcYREOGflBHYxutQMdaMGy/WjS0yNjn1aJjLRi2Xy0aWmTs8//Gon220K2s8qyxDJ4QDvGz2r2adpIdHICosqg6ACQhn6dNcu6ZeAQQzEkBS5Ikh0f1STkvR5yLl8ahQXeRqoBKC5boOBR0KB6OhnlVKsZJsosFILDaF3VoRwdDhGeOD1qyymIBCAB9FSAsGKPwBUIZXjR2yLOzUnH63Aw1xp4FMOWLHiOv+qTzq7rSS6Bi77QA06faOeq3cWUBACRDbivBnkzDMBCp0UCs+DKDgUo5HxGhc77gsWTnsKQegP7Y1FqeFLYKgSNUyJxNTQkTJY/NTas3a6cCEL+YYG+LChxLzS3lxNeLJDkb8tjwGs8+qqQ+gP7T1FqeFFanHHdvK28tqfKWlGz0rWJoAOJP01ttUYE5VqMT5eHHA/jc8NLX6ABE7fKExejso4CGoexT8dwHpZyh/FuJ6AfbrCOVXyuxdT5Q8mP4BCDuNWLZrAqoNHxQAizzM6QLYwjEYQMQAhANKhhAP0/J8hKS6/gLfrtSuXqOlhTNXb8CndrCq0W/a4uyz7JslHaXqY1Uo4+t1y1pzNGtEruNeTIqXhvC83c74foxTT0mtXv9RlsgU8S2QJwPnvnxgHlURZKdPQABMLwfSwBm7hBAaH+SyaGV3CIBCLymJwEos+TvOYDUqd1iAQhcObwfkiN13paNl8lHByCq5qhAYb2ffCDi2ZC75cOvi5RZZvauC4lnZyXq2/smkfbDLhYIJIpdPErlGIC2L9Uzv0tUXMOrLGq9qFPOTYA8UWI6xH5KFLtDBiI3VYov0xco4EgbgWboCTtyzSg7bMgXqCoAUdARr27AYSV+vM1Z5DKuwXHN2qkARNu0agsVkGNS22pXsKFnOkRJ7WzIYy9jPPuoojpgVgGVHIs1w2yKaduHWDXYyUIyi9qaHNa3z7VAsQoBMXKf7jYPpY5GByBqgbp4u49QoPmYsDBxffcCWFzpcDcg1DmRDiIi/uLOUHJTsbWmyocNQGDJBlALCl6MJBcNVvASkjfvK9Y0y6sd+qySJSpXoFNbebXoatGpLTC1vnWOfrcW7eXh+1bSloh/6i4a+PxcWW3mDUAkOsd1M38yIbJjC+QiAYhdjyqp0986jMKD5BcLQLRQ1ClDjifkfNC2Il8YHYAYsjw9FtjsWT6CmTNV7JBnZ6Xkmct09QLaDLnvPJueZK42R8nKXUVxrlcDVYdb6TUPftdFuWQgclOl+DLVQIWHpajvkJdf45vENAEIAeSr7TXkHsLY3LRqs/b4AEQz5JwubeJbJ1W52tmQx4bXeEXdNTsTc5MLBSzwPrFMcfsBPvdLwgsHviyHQvSEZBYtHhqLQAUAJXuKxOcCARcN2msbIQrvmZex7dNjb7h8MNufYFhHDJsNneMJ9GEDEIaMXGD4qGwYCUpgLW4xvlSy5cuCZzQxwnrz/wmG9SYw2SCC6tcAhBnrWud9dqaDvsgKdGrz+ws+V4ND/dfiPxYACqDss6klp7gHo2Qiiu1crer/AP5TPXUImJZvAAAAAElFTkSuQmCC",
      "text/latex": [
       "$\\displaystyle \\left[\\begin{matrix}1.0\\\\2.0\\\\3.0\\\\4.0\\end{matrix}\\right]$"
      ],
      "text/plain": [
       "⎡1.0⎤\n",
       "⎢   ⎥\n",
       "⎢2.0⎥\n",
       "⎢   ⎥\n",
       "⎢3.0⎥\n",
       "⎢   ⎥\n",
       "⎣4.0⎦"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "inv_A = scp.linalg.inv(A)\n",
    "print(\"The inverse of A is:\")\n",
    "display_matrix(inv_A.round(2))\n",
    "print(\"The value of x, given by matrix multiplication 'x = inv(A) @ b' is:\")\n",
    "display_matrix((inv_A @ b).round(2))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id='sec1'></a>\n",
    "## LU decomposition\n",
    "\n",
    "For more info about the theory of the LU decomposition, have a look at the wiki page [link](https://en.wikipedia.org/wiki/LU_decomposition)\n",
    "\n",
    "Let us have a look at the LU decomposition.  \n",
    "The matrix A is decomposed into the product of three matrices:\n",
    "$$ A = P L U $$\n",
    "where L and U are lower and upper triangular matrices, and P is a permutation matrix, which, reorders the rows of L@U. \n",
    "\n",
    "If you are not familiar with these concepts, they will be immediately clear after seeing them in practice.  \n",
    "Scipy offers 3 modules:\n",
    "- [scipy.linalg.lu](https://docs.scipy.org/doc/scipy/reference/generated/scipy.linalg.lu.html)\n",
    "- [scipy.linalg.lu_factor](https://docs.scipy.org/doc/scipy/reference/generated/scipy.linalg.lu_factor.html)\n",
    "- [scipy.linalg.lu_solve](https://docs.scipy.org/doc/scipy/reference/generated/scipy.linalg.lu_solve.html)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Lower triangular matrix:\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPgAAABkCAYAAACrdf0zAAAACXBIWXMAAA7EAAAOxAGVKw4bAAAPMklEQVR4Ae1dW5LdthG9o5rvlCJV+T+jHcx4VmBpB5a9AsU7iCtf0p/K2oHtFdjWDqysYOLZgex/V0VSZQPJORw2B5cXIJsECQJEowoXYKPx6AM0APLicfby5cvLw+HwG6zPvH316tVzX4DRDAFDYHsEoJ/vUYoLX0kQdnbuBLyBn8yu+d19ML8hYAhkh8B3nhI9A+1L0l0F/x4abwrtQctIhkCuCEBnf+iXDTSSThS8z6d+RoKc5v8CewX/J3VEMIJfeqD/4PEJ7Heg7a6jqUXOKXXv460Np7XlfeADWUNDwR7C/gL7Pfh/hPW+Bwylhbh897+B+y0sXxG+hf0V/slpDeWzdVgtcsbiXBtOKeSNUfBPKOBz2G9QsT9NrVzE+zvisJN4K3Hh5+jPZ3YauzC1yBlbWbXhlEre2QoeW6GIz6/zt550bkB7CgAeesJKJNUiZ2zd1IZTEnm3VPCnaBEfPK1C3r8ZvgdTi5yxdVUbTknk3UTBlaPzo9gWs3X8WuSMxbk2nFLKu4mCo0GI8g59cd/DFL0WOWN1vDacksm7lYJrGsRjDdMOeGqRM7aqasNpEXm3UnDfu7c0AOnd+L946aYWOWPrqTacksm7iYLjHUSm5r5puNDkY1ts49ksfi1yxgJcG04p5d1EwdsG8Q7uhadxyAjO8D2YWuSMravacEoi75YKzqWtn3taxRVot04v52EpilSLnLGVUhtOSeRdSsHlg4CMvl1lQ1G5Wu1/sEdbUvHMRfIf4DaL4hkBfk7Pv4J9wec9mFrkjK2r2nBKJe95TMWgkOyFaPinPQ3XpvPdmevJm10ucLmklbR/k6FnOFpzc8k1XH5Uo/sFnm/h7snUImdsndWG0+rynjkHPjxpFTG2kiy+IWAIbIgA9Jj7PLj9+2ypKfqG4ljWhoAhEELAFDyEjNENgR0gYAq+g0o0EQyBEAKm4CFkjG4I7AABU/AdVKKJYAiEEDAFDyFjdENgBwiYgu+gEk0EQyCEgCl4CBmjGwI7QMAUfAeVaCIYAiEETMFDyBjdENgBApPWomPp2+xLChD3Anjx3HMa7iLjpneeh36y7hw099hkbmB5AZrsIcdjngZlvETJuD7fLoCYWUXAcHYbm5nlptHWllc9gqMgsy8pQFwqN9fGftNaLrKnYv+GZ9mocoCfO894P9p7uA0v/K9h/8Az08jOtGW2CyAWqBlgObuNLZB98iRSyKtScBQk9pIC9sq8IKEzSJOjOUdl2ZHGMN6Q8ghhvOWkMfCzI+BONHdUvwvM4Bfl4245uwAisi6AYWwbiyxB2uip5FUpOESPPaSdozRHZTmOSdDkqRYctWV05t5w31FNVPI9XYYg8tONxdZNq2R/bTgkkVer4FTQD57WI8rYTbM9PCRRkX+HIofeo6nkovy+fLhXnMZ3AsxdSLm/sdiWK/lxyWvDIYm858cYnz45incaeE85OcnlPqg5qYW9lc/wo9QBeXCEpkvHl9ZjBsDISH/3VPgv5JVObUgSHx5D/MWF1YZDSnk1I7g0sNDoywalaahHDQ9CUrmpsPJlneG8eNCnxOSlmZzPXbRsf1fBNltpwwWrDYdk8moUPFwt9yEywt5Txn38uPYWit59UMNzcxYbaJy+NKbtCKRzkVcCCa7BnYPtHnGpDYdF5OUU/S9taxC33zh878TCIz2RvCMLfdCF0vKLON/Jj6bueOYX6b8hjOe0XcKlkDetDX2AQ3CxZnFsC0WiNhzWlvczaQejIziVrmX2TY+Fph5ZkR7/DuFfYc+kEK7L/GD5H/gbWC6E4bRdejN1Pm6aufoh26LY5irnWLlqwyGlvFTw/7YVIK6vPhY5pB2CcRTm4Y7dyA3/Ba0vU4fG0fwd+EQhnKDivYtgWzwKd/+0+NrBo1Y24rQns2a9/ylAjY7gLSPfl31/UV2BrrqkAMpJJb2G635UY/JU+mbKgrAvYT/Cyszg0Pr5Tt6Px7h7MNHY7gEEyFAbDknkVSk4lEx1SQGVEdZ3yQF7ZgrEcC5Z7SxonI7LyEy+/vsJ45HnFm7uRl4lZNTpyovyh7BRYdsltFMP8KkKh1Ty8iOb1nC0HrykAIXm+zPfk/uXHPwKGpWX79990yku4vK9m0ryT7gyijPPrKdnKB87IRr5+m8XQNzhMfV3tI1NTTBz/tXltYsPMm8BVjxDYCoCGHA4kHKWfKaaok/NwPgNAUMgDwRMwfOoByuFIbAKAqbgq8BqiRoCeSBgCp5HPVgpDIFVEDAFXwVWS9QQyAMBU/A86sFKYQisggAVnP9b8zil/gKTVTK0RA0BQ2B1BLhupDkijQp+AcvdXSerr0AzYwgYAuUhwAVXzRmGNkUvr/KsxIaAGoEpS1XVifYZsaKGswT2Kj/DL+vO+2z2bAgYAgsjMEnBoZxzD6XnTjJOGbh8zicC17D/1Q3AM+NwjffkSwTcdNbwR+DQFWdMPoSzU5QddNzJx28k3osiukQL8YzJPiTGEtgPpZ86bG151AqOgvBQ+tdweQDDAS43g/DigmewYwcxXIOXL/4+Po7sTUNu0+TZ6GzMbNRs5FkZlHE2Dlr5wEe52Rl2h2LAz85V8M56842vwrSy++IKDWnMxl7SyMlNIY9KwVEQ76H0oFPZOTJ3DTEEIHhPeEBrFLhN5wCX0/fmMAj4/wE/R/FsDMoUhcME+bwXRbT5c1ZzNNvJBqCBgkyQ3ZtKLPbeRDckppLngVJGKl23rdOJcwP/UxRWtnY6QUde8vkMt4Ie3XjiY8qIFouDVhTOajQXRWjT2wNfKuxTYZVEHq2Cs8H5/ieXKTfDgwZK3EzrXQbQOEq9dmkF+KNwmCBf8zoDjEIfJMc61AlZFcOaCvtUgCSR53xMGjQyTWOa9B860uTU/BKufEQaK8bm4WvgEBIKebF395nmlQXhvtmUj38XtJTYpwAspTyaEVyUNzSaEBNNJ+Bix9GbtiSzBg5q+dEoqNzsGIvpFNXCjTNuiv148SZzJJNHo+Ca0vOYJZVBQ2Uj5Xt7cV+CFQKqcVCk1Wfhx7X+RRF9npqf18R+C1wXkUej4L53bxFYeqIpFx/wo5q8u0s6JbhL46CWGZ0h/6k4uShCnUD5jJthvxJ0yeQZVXA0Lpma+6bhQpuisN0xySuBt0qyK+CgKify5V9zwYsiVIkUzrQV9mvBllKeUQVvheR0+sIjsIzgquk2BGOHwHSk0/AkmTVpERy0EgIvdoZzLorQZlESX1LsEwCTRB6tgvP9jyvL+uYKBNXFB21ESWNoitLPI6fnpXAYlQnKfQmmwYsiRhPZF0My7BPBlkQelYKjsakOpecIDXty8YEDmEzpNSO4fGSQWYKTzDbeBXGgAEH5kA9nOWwAxJNLVjsLmntRBB6LNEOye9uQFvtS0Eglz+j/4A5gHK3nXnwgydzCQ+W+EULfheBs2DRcCEBzconAHXmz3ygclPKpLorYDIGZGWtkBw83HvGbTv/yDOY6iv3Mom0VbXV57OKDrarW8jUEVkIAHaRdfLAStpasIZAVAqp38KxKbIUxBAwBNQKm4GqojNEQKA8BU/Dy6sxKbAioETAFV0NljIZAeQiYgpdXZ1ZiQ0CNgCm4GipjNATKQ8AUvLw6sxIbAmoETMHVUBmjIVAeAlOWqs6WDitrLhCZS0/t4oPZKFpEQ2A6ApMUHIoqxyzxgIcnsFybrtkLzp1Ru7n4YAjmCIxOkkVaxI2bSwZPntXynWSwISElTk5eIvFPoHFfRBLT1g/3WEy+xMMp+1Sda2RTKzgyijl0/hq5cf+rrzPgyF7MxQcNaoGfSIx8qbJRaBqils+XR3JaKpyQD2eOxIY3wrD9HeDymZYD1GoG+XDnZNQlHkgjRuca2VQKjoyiDvxnTkij+IsPGsQCP0tg5CaN9Hjxw6jR8o0mlIghMU5UZI7WjXK3IlLxfAPNogggT+6abE7HbeuIszG1WQon7Ue22EPabwKScYo/OP0MxMuRHItRJxMwYWNgA6ENGi1fMIFtApLgBGx4Gg5x5FkGnQGdV22dDDYdQz6eRXDSKjin0b5TWKQnZHjQANC9XHwQlBEBURj1Ev4amB01zF64PGr5hD8HNxVOHDi4t3ywk8wBkEAZFsHpPJB4RwZAD7uHsOdROOg0BGny3aioiw9OpbinLIkR0uLUnB8kB42WbzCRxIGJceLxYDyJlqP417DykYoHiLhT9sQojGe3JE6jCo7iiPIO9YSaTsCVrMSLD9zy9/2LYISKZccnJ5r08+ietXxdhHw8KXGSNvk58Ooui4D/I+wL2JNZZT4wLadz2in6mOyPxxgkHMCyEe/14gMR0+dqMOJfYpqpuZbPV47cadE4AUNRbs4S+3j+DAB+dHhyxyNUPg1OB80I7nv3lkylR+b0R2uyvPigrfB/QQhpHBp5niMe/8aKxgjpNMfsjGWq5RtLZ254KTi18sk3Ildc/vVErDmFz3WqHt2eROBRBUeFcspIfl/DF5oPSMmj7/Lr5hT+fvxVniknEr6ak3gsRojPWc1DuIO4aPnmyKCNQ1nBmztO0mZZ1pAh5lkaYgzLsol+ueUU2mBbkQijCt4ysqfzASIjuKonRKFZOKbDUW9vJgYjYsIz0Pm/rWsu8XDR0lmhPG11lA/83Tunm1gm/tVxauUP5SMwqBREmDdwQ+WfpHNaBWfDk2WqrqzsyWu6+MCVve+fjREaJCuT9siA/hGEd3D5n6gYLZ/w5+amwon/RPQ7TGLBNssR8gRHBmZkZuPkyvDAfQj5AQY/VHyAy+l1Y+DnaPwV7Is7SrNajdPM3V58IHL63AUxcpMnxjIlc+l9v5avHy/5cyqckA+/krNz7AYm+E/abCIAHrf5yOjbZcsywZ7oDGgqnesSCni0Izijs+fjyrNruPyoRvcLPHfTbfjZM3Lq4zu0HuRmav4J7g0ffAbxpdd92obndvGBr9hCWwKjAzDg6COvRPzHgZjcwH0jGdHV8rlxMvEnwQn4cNUa26ysK6CCHbXZNfFAvqNtGTxDOjOK01j57eKDMYQs3BAoDAF0GnbxQWF1ZsU1BGYhoHoHn5WyRTIEDIHNETAF37wKrACGwHoImIKvh62lbAhsjoAp+OZVYAUwBNZDwP2b7D2+vvVzeguau8iiH27PhoAhsCEC0M/3yP4iVAQqOP+3Dp2qkvtyvpBcRjcEakGgW8jjE/j/eVxgzinYqVAAAAAASUVORK5CYII=",
      "text/latex": [
       "$\\displaystyle \\left[\\begin{matrix}1.0 & 0 & 0 & 0\\\\0.29 & 1.0 & 0 & 0\\\\0.71 & 0.12 & 1.0 & 0\\\\0.71 & -0.44 & -0.46 & 1.0\\end{matrix}\\right]$"
      ],
      "text/plain": [
       "⎡1.0     0      0     0 ⎤\n",
       "⎢                       ⎥\n",
       "⎢0.29   1.0     0     0 ⎥\n",
       "⎢                       ⎥\n",
       "⎢0.71  0.12    1.0    0 ⎥\n",
       "⎢                       ⎥\n",
       "⎣0.71  -0.44  -0.46  1.0⎦"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Upper triangular matrix:\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAOcAAABkCAYAAAB90CdWAAAACXBIWXMAAA7EAAAOxAGVKw4bAAATFUlEQVR4Ae2dW7LdNBaGd07luStAVb9zmEEuIyDMAJoRJMwAiifyloIZACNIYAbACBIyg9DvXdXJqZ4A/X/eXi7Zlm3Z1vbtLFVp62pprV9a0pIsa9/57rvv7p9Opz9lY+bXZ8+efRFL8DhHwBGYh4Bk661KuI6VorQ7d4OEH+Qnc2j+CgPudwQcgawIfB8p7TPFfU58KJw/Slqjwqh4pPux7Ev5b3hw70Z83Jd9E/Kh8D2FP5QbxSHM635HYC4C6mc/NctQHFEt4WzmC8Oovj9iy4fDNPw3iv+gGdkMK4+NFP9V2iey3ytuLUH4Q3UjjCag+DEPzs7w78b4GSa4J0fAi+V6oTjDxuKibvDsFto1SmNqZMCLPbIaDuHMacTE3EeK/F02JkjMqN/EHgrjxDTr2udyfyVeLsLwp9zPZGPlku2S5l1ZOAMP9UMX9N2U8b2O8m2Nn156uxLFB1rRL7LfyE8bn+QSxjKA9hrldRyE0CVwSBVOKkcXrhnFFYtZuYXA1RKDgNKfKngvzCc/sy3PMSO3yg4ev5T3jeqftNml57bIz1ScEEJmh0Iwy0IYOAcHTMfhjNalcLgqG2PIedWRAbX0q460MBohiKlIlPtYZZhKGT6zZf8h+BHurG3QHGprH8WjzaQMmI7DuZdeBIck4VRDtWZGxbF+fH6mbfAX1dfUyDCzjc6k78kchR8GVjSYJFU+0kCOwxmUi+BwNwL4YJQaE3WW3c6UtWbKrPjhYKUXyCD6C/VURX8kC0+sOWMzfFW70jfLT0Vkuuehsv4lnpg9v5S1DZ1fFBequa0SHYczJJfEYZJwiixmTdt5bTVcI8IEr290TunwjWJnB6mzejUkkBHOt3JR6fo65lb5mQKI4f5QPFcDrfzvZZ/ItjSmoBLH4QzGxXBIUmuDBjmpwejErBP7OnD4SIqfmWtRI/oRwmrAkB8VG57YoJprFudnLMHi1wQTDai25lRZL2V/DvKMLd7yOw5nJCbhMFo4VRfrFFsrWiP0ubG1puW3UQd1agsGvq7VKRmAusye+OniIYyPtSWvRxBe1N4u4zickbkYDlOEkx2+PoJqjamObrOTjdRhusXFOkiYL6tfNP0mSwfsMkZXK32L/LSITIgI+LD2iT3VOUgFz8ewsrhF2zXGwFBcwMfmcBglnGIE0GmwPkZieKAuxhraZs6cKnKs/mYcM4J1oDCtoEd8vgkjI/6t8RMhMSkKPmI42MNDwuU4nJG6CA6jhFN0mJqTPHOWrcyLbnu2jCqcB/rlMMBYYQ/LmOL/SXXGTr+wJQ7QQ2Zr/AzR25XO+jo2aNIuvGIZwsJxOCN7ERzGCqeNslFhUmNyCuhv2ZrKqDAbDu/kohIXRn7K+pfsk3PMor+cEa5t/Cj8dUkBL5QLA42ye+DHSB7lijd2Y3+XW+28w7Piau1CnKzjsDAOY1+loO4hmK9kW0YNyGiLKvS6lXg+UM6JokdKYwMI91OFh1TISFHzoqBRFlpMQFFn0QY+Vlw18OCX3Tw/c9AQf+xaN7GotYvjcEZ4aRzuBB9bf6LKh9YYc/qBP+sIOAIDCEgGORiDZndnrFo7ULQnOwKOQC4EXDhzIenlOAKZEXDhzAyoF+cI5ELAhTMXkl6OI5AZARfOzIB6cY5ALgRcOHMh6eU4ApkRcOHMDKgX5wjkQsCFMxeSXo4jkBkBF87MgHpxjkAuBFw4E5DUaY37zWyK47zptcXjl31KvMW5u00E1EaD7bkFyseere2lWUzbAWrOzq59aXRFq+hCiOyWQISHMOdJh766sDJSLqCmwTmrO+vibavwkm7QTlbNC8UNnnEucbTrTPjKiPPI3HfbelZxdm6ZOji7zLUn1bllIlc0Ke3ZSd5SOGQTThHMlyhbujS6AFd0IYx0IBPOk/x8HcMH11/I9t2TU5ShHzohBgHk/DHPNC+g5iA/wh47n8ynaNap5V3HiFcGJT5vGn2BdPksA091Zab8DMZ2MXgx0CkOvOkL5OX/d05ywe3fch/IxvAh25ImpT2j9Ih+MFwEhyzCKYK3fMkytKFuIowmiDZjfqs0i4s2RhmZdAG1yq86rhWmOBrzFNRtSWu4CCazpPEPDQhTisAgiNUAx4MqByEHX8q1v+P4WX7+b6YQTPnJB36v5WU2bWFEnoVNUnt20LQYDlcdBIyN5hvIlmqjOD4te6yGoQOsZaDrprQFDaKHcG4DrzGD+lzr1LFMl44TDWgLzGC1y7wUzydjKQLD7M/thM22RNDD9Tf1xISddli7L4iE2WYxHLLMnGIXgmuNXkJgjUR6ygxVPpbPUWei89ioXhSsODoQJlwXnWMm/qrMFn+KY5R9PrHI3I8xQPB96tSBCRy5qa/reQTUBNfUxpAH9iEwD2XDmbuI3NHPYjjMFs6gQfrwZUNgE0b0MlAUqon8sQElSqfyFqq7ErnmEFW19wJq5SdP0sXbyreEQSgmXSANceIH7ShmmI1JLzQnuQRj7W3XQ4LL6kZ0jmpPI1jPLYbDbOEU0dYQXSMqfNmIajwu7gpUOhGCycYNHYk1UKqB/rEXUDMAYLdirA2mXCAd5aHEFGELN7vQIMC5aQohVqTR0UxfMjylPTvpuxQOV5015k2wUTNvqSNKE4BsAvwgy8j3QpZdRlNve0tSvlEXUCs/HZb11SbUN9FhApH7Amk2gn5V+dXmj8JPAFNxlYDKj2DeEC9jS51zaIVf0TOqPRNIvAgOCOc/ysrNTaClluVdLVQP2Kxq64166kohNQ6jO52F/wSxjjuWGjpZ1wXUrO9W74QRhmI08doDDFB7k41wY72OmlxT8xQG149leU31tSzaA4OVbZjFaFDy6qavPTuJE3+5cfinVTZbraUxZCkv1sktbrUGEW2FOiUXVTY0qLWM7liENWr03G9K4NUA10XGjPEYpjEjr8ZzSAh+0W5thOB0GQQoyag81mtgEt3lpT6l13aoFWcq/qq4iI4p7RnFRWVdFAdmzv+VNZsbJWQgEvUt1rg2c66p3jEzoMLGhGiArSKZGSX2bMGbyq0JfVkPWPQJQkq9ufPQBjE+rJ4koRF/DDxcBlfNmPJ3aRBWNu59Wa7hXBuXUe0ZMhD6L4jDf6ye2TNnWRA6t42MVjYusw1rvTUbhLpjnYJGwgwNHFxAHW54nJ86z7ixZ63cPnXfyljSRf2inZqGNmJmjfFSy6s8CNijCB4IbLHzrTT8HESorhlVHIMCGkqX9qGkxczY9mwRthQOWYRTxBYM0zCyhYoolwbhcuJPW9wtG9ESLOgUCdD3lfwI76mk9728DCZhJ+KoFrZS0+RvXUBNGaWhXExR7tm7/q9oZuOmuEBaboGJXGhtXSCtuBYOyos2gHBTRvP9MJtftilEvubAxHNgXdMyFLeGSWpP0Qo2q+KQ7d7akhlmTzolG0C8suh9F6j0RYxoY9Su1DD56UCtg+/K91bxdL5KEBU+KUx+E3LUWTofR9daAljmRZXmoHfnWlbpqxjRRBvR8TDw0moj5WnhUMaBQ8zUBrSyDvJZPYN/xhsr9FJxoi+pPdfAQXWyjmUAuZNNOC8FpJfrCNwmBELhvLpNjDuvjsCeEHDh3FNrOa23CgEXzlvV3M7snhBw4dxTazmttwoBF85b1dzO7J4QcOHcU2s5rbcKAYSTY1u812u+OL5VQDizjsBGEOCkVvGeHeG8luXEBy+k3TgCjsC6CHBgpjiB5Wrtug3htTsCnQi4cHZC4wmOwLoIZDn4bizo6JF9mcLZ2s1cKm30jXWPxk8q/+Kbr084rM49s63zw33lTMGsrI+D8bUzzX31zElTPSzlrC7O/xJunbXuqyOFT+Wh7G+DcggnnzPOJpwihMPeHKIOv0qxC4eTvhUMmFjdezR+hgAtOxKferExyGdvdNhRZgZmDASLfLFS8jnrkvERfLauRdWzxe0bcgc/isii1qqi4iazsEL5GXEhoPl50agGXyPz0fhJwZD2kuVqEWaUFynPhHmmYqbn7PO7sLhL+umrT1Uvnw2asW9Zw1nO0mpuKp9lPiaspuGOpcF6eCiLcKocPseKjXyvFM+3fkznezJH42cJ7Edjpn6B+swgjl3K0E9rdYqOMfWn8smyLnqNSyqjuYST7V/UoaYxdZb0PZmj8bME9lMw+1KCkXx3cA4mVB/f636Aa+XJb7NoipaXyicTE5cP8Dcg96wuuezLpNRzuhs8NMnbqLirjN28Qz0aP10NkjN+CmZ6BnU2qZPmpLVZluhA2BAYNqR6B4oxfCovN0+wrEPw38vPx/rMpuF/9ijYbXLMnCZ4fapBOHJ0U7ONlKPxswSqozBTR70WUaxxTbNagsZaHaqbO3wZIFhjo+q+rmWIB8byiQpsAs8AwEAQW/5Fa8shnNGCG5EfNcJ7Dx6NnyXaI8RscJa6NEESzMmXjA/QVvGpOpg1mbSYMVGjGZT4MyhToxXsNrPVWhUdW2tajTbS8N5zL2aX/KjB0U7+kB2jpbA7mzyS9zRgMmaqr7gjp6esxZNEEyooQsRrDtaj+GNmLJ+8J7b3qdwyj1Dy2uhn+Vn7dtVT1D1bOKlAlsJincLiVlNfCi5H/OyVn7KhH4xgNVvWVMyUj5njntzV+oPqvg/jcpuDEmotaieWtWLL6JkxfR01lpvvK6PnGQSYRblArbMee+CueWa6NmU3i7GZs9oZa2bYaPho/CwBcwpmD0UI994ye4QGgeFiauL5iwe76TDMk8tfvHtUHX0zZF9dg3yqbCYlBqHWzKg4+EP4TTY668olnIDKSNE0jORrXyrdpCklfDR+UniemycFMzo2tmbUWbkfFjWPDZRLGwQmplIycGBa9J2jq98UPk/ihVmWASemJSC8Q/XkOYQgAtiReie3WujKDwG1C4sr9jbuORo/E+C2TY3W6E67yv4tWzv9MhMz+gp2CcOszP+lVEa002+pn42qGxLkzuWTgaY4qkd5ZlQua25ep8SE1rIVbrZ7a2FGJTJ7whwbQI9kOWvb1O0VvX1zNH5SEBfPzAoY1kO0J21HJ6Iz2SuBk/ytS6eVh/hRfUD5ec/JOpT6MKh7rxRvt8cXkbl/VD71hbM0NLQOvivfLD71POr6t7LhRhIC2zlrKq3YMJPrl0oLODeOwGYQCIXzajNUOSGOgCNQQ8CFswaHBxyB7SDgwrmdtnBKHIEaAi6cNTg84AhsBwEXzu20hVPiCNQQcOGsweEBR2A7CLhwbqctnBJHoIaAC2cNDg84AttBwIVzO23hlDgCNQRyHXyvFeoBR2CrCOgEzrVo4/jeS/k5arpZk1U4xax9mcLZWr5b47zi4AHfraJzNH6m4nwwHDjvypneH8VXDBK+JvkgltAVp/yUyaF5+7C6llXxJhcW/0Jxg2fOswmnKuMrBQ66Fx+qyuUQtF8qbc2xU/do7apm4IMMDp7HJg1m1CnfkvLBQEvYhB2zNGlcYl0cdpdLGMvk1WuyCKcq5CQ9n9hUX5DLzwhEmFFq1v2dvRxcIPFo/EyF6Kg4iK9Wf1QcgnSSW/XhFNyUv+9SbISQWTL8CoVJKzYwtKrLtSHE5zetkUNxr2QfizgI2pM5Gj9TsT8iDvTJmGEJFlVLY5mJU37UWdat2JpRGt+Ikl59akcGxXOXUGtwIK1pcgkn6kD4zZrVYyME6XsyR+NnKvaHw0GC0ZoZFcea8PkEkPouxUbQ0R5bgptaz93UjF35VHnKrPhh1/Nbiz8aP1PxvS04iE/UWe6wHbXWVP6hS7G59oT7gpg9v5S1TdLej62VrzKzhVMlmeD1jRApAlwRtbLnaPxMhfO24MCs2dxN7cWsFGhmRdMMY/mtzz9Uvkrw5ef29yeyrRm8WUgutbZZbjNsd9I04/caPho/U9th1zhIQJg12RMJN2xSsOC1SW0tGT6kNBNMZuRmvpfKy721lid8tObPIZyxtaZVYqMvU/pezNH4mYr7bcCBdWHf7NfCTkI15lLsWNm8ckQwUXt7zWzhFLE3ZQ2xkcDiYkT2ErZW4tH4mYrjLcGBHdW+QagGnzBhpuWVYW9/DrAz2aiVUwYoq9fkWHNSAWpBrDKbOceqDb1EL5B4NH6mQnZYHCRATBz02TcjwCF/6qXYXdhZdb0CTqZcwsnL1tii+oHi/VJpkN6nOVq7hq1gauWYmROBw9aMBD12KTaHb8CvaZAJNpNa5TQzXjUjpoRVEYtev1R6CngbfuZo7dqA2pZcUdVTvKO+ti7PbpRhQcqy8oo4PctuLDfLV5MWZSou+aL1XDMnBDEicMqCs4tsAOF+qvAYtUGPbMYcjZ+pwB4VB/olghk9MaR+a69KXncBpzzMjracY9eXmbK6FFthTgMhE+TDsMxLlolsN74XVfuPI+AIzEJAglzd+J5FrZ1FjT/sCDgCUQRcOKOweKQjsD4CLpzrt4FT4AhEEXDhjMLikY7A+gi4cK7fBk6BIxBFIHyV8lY7Rc1M/Ic9H9y6cQQcgcwISLb4/8/rrmIRTo4RdX0BPnjEqKtgj3cEHIFBBKoDCrGc/wejVe5wER60mQAAAABJRU5ErkJggg==",
      "text/latex": [
       "$\\displaystyle \\left[\\begin{matrix}7.0 & 5.0 & 6.0 & 6.0\\\\0 & 3.57 & 6.29 & 5.29\\\\0 & 0 & -1.04 & 3.08\\\\0 & 0 & 0 & 7.46\\end{matrix}\\right]$"
      ],
      "text/plain": [
       "⎡7.0  5.0    6.0   6.0 ⎤\n",
       "⎢                      ⎥\n",
       "⎢ 0   3.57  6.29   5.29⎥\n",
       "⎢                      ⎥\n",
       "⎢ 0    0    -1.04  3.08⎥\n",
       "⎢                      ⎥\n",
       "⎣ 0    0      0    7.46⎦"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Permutation matrix:\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAALgAAABkCAYAAAA1z/qTAAAACXBIWXMAAA7EAAAOxAGVKw4bAAAIHElEQVR4Ae1d7W0cNxCVAv0OjBhIAXIHUlJB7A7idJC4BP+U/jodKKkgsDqIO3CiDpQCAtgw0kDyRr45XE68JXc5Qw5vH4HV3pFc7ntv3u3x9mN0enV1dXFycvInllS5vb6+fplqYB0ViKAA/HkPHOcpLGg7Pdtp+BmvpfNu+Wv3DV9TgYAKvElgeoG676V+1+A3cDwNnVCLVXEVgGd/2UeHOql6ZPD9frPfY2D9NH3Axs+wvIn0oQEWmY69xXKJ15+wLi6RuUXGViKwJ/4vSgCU9AFImce/x/o1FpnuvMbyO14n50clY1r0wf6fYHmL5Qbj/YplNh5sG5Kb6BMZm+DLFW/8JgYHyJ9ARIx0q4TwWo6Q8l6M1a0IDiwvsbwCiN/mAsF2kbmFxVaicwttTQwOMnKm5S5B6j3qnoPIk0TbKFWRuUXGVhJfd/xWBn8ONh8TjPRHq7SPWiJzi4ytJN7u+KsNXnh0/qqEbbQ+kblFxlYSx1b4qw0OMmreqbMSo05RInOLjK3E403wWxi8hMzTkk6D9onMLTK2knBX47cweGrureD1UyrnxUcskblFxlYS6yb4qw2OuZROTVLTEK3TH5slxMP0icwtMraSALbCX23wDZl3WJ8niOkRXNpHLZG5RcZWEm93/FYGl8vf3yQYXaLubufTmugSvioyt8jYSgLrjt/E4DCw3PDyEeuHG1yEGV7L9OQHLD/K+yBFf7ToN8sWluDF8i8WuSy/LXgflltkbFsBJ160wH82sf+5TXK0lpurvsVaflTK+ju8v8O6awEGOVJIkQsLUuTeFPldIPfKPNyNhrVc0pe6P6TDXgnLDTgjY9uTMfnWFf/pzgMPzzYBTqJgJRUYRQH4WO7Rkdu/T02mKKMQJ871KUCDry/mq2JMg68q3OsjS4OvL+arYkyDryrc6yNLg68v5qtiTIOvKtzrI0uDry/mq2JMg68q3OsjS4OvL+arYmx5L8oJLo2GTvwzFVlin1Knvg36XmAUuSeoadIlsyM4CIRNjpMLD7HnFFrWDl3lDs2uSZdMDA4SwyagIfZl5i3ZCtp2T7pkYnCQdU/gUiLowj7EvlC4BptVx8bK4O4JXBzFJHZHcSuHro5NtcHxNaQPFk9xefQEzVTnVm3E3krp+fuxik21wQFdzftpgkbJh2Bic7cmYneTtnpgk9hYGLyEydOSTkH7EHvQwABWNjZi8C83+HU9l06TBC5zQRX2J/ZCoTp0q4nN14q3+ggup4I2g6WmIVoXMvEPsasN4q2tYiMG/2dDT9dL2LoncFkCqnAbYi8UqkO3pbH5W7FWH8E3A7kncFHADmtidxDVaMjq2JgYHF8nYZPj5IQm9pxCZu36g1DPjmwHRgzcki5Z3mx1CcQhE/9slTz8gtgPa1PVAvPKUViKXLSR0jTpEhP/fBadf49IAXyomPjniOJJKhMKmMzBJ8ZnExXoqgAN3lV+7txbARrcW2GO31UBGryr/Ny5twI0uLfCHL+rAmJwuU9E/o/71M0tXUFy51RgpgJyiV88fSIGP8dyg+XRFSbUsVCBERWQi0ri6QeDj0iAmKlAkQKcgxfJxE6jKjD7XhRcBr0AWbm/oGkCF2+BwYtJi7xFPjC+p/ZFR3AA6J7A5YA2JtXgx6RFJkrOH8Rb+1KDd0/gMl+6si0gMJMWlUll3quF9kUGN2BWncDFAMOhISJjO4RZ60fGLhzc8bcyuJy2SZ1n12c19V5hDVzLdWRsOR1Gxi7c3PG7GxxfQ/rg8VSwupyDj4xtSixpGxl7S/zuBgcZNa8+fS/89kvJh2B/G4v3kbHl+I2MXbg1wd/C4LlASbs+r1fSt3WfyNhyWoyMXbhV429h8NTcWwOjn+IPWtF4HRlbToqRsQu3JvjdDY65ok5NUtMQrdMfm7mgmrZHxpYjOjJ24dYKv7vBN4FamsAlF2eL9sjYcvxGxi7c3PG3Mnh1ApdcpCvaI2PL0RoZu3Bzx7/E4Drx1/nzNgj42nFL4LLdifELYGbSImNNS4drof3ZDDDyaZMiJ+elNE3g8nmXbn8vMTKTFrnJOzmwq/ZM/DOpPRtHVADfDEz8M2LgiHm+Akvm4PP3wi2oQCcFaPBOwnO3bRSgwdvozL10UoAG7yQ8d9tGARq8jc7cSycFaPBOwnO3bRSgwdvozL10UoAG7yQ8d9tGARq8jc7cSycFiu9FKcGHS6RMnlMi1II+0PYCm8n9QEeVcEmk8PSN2REcIJk8R6JlWKDpUSdcEqm8fWNicIBk8hxDY+tQ0PVoEy5tzO3uGxODA6x7AhcNusN6ZOw5OaJzc8dnZXD3BC65SFa0j4w9Rzs6N3d81QbH16g+ODwl9qOnf6Y6t2obGXtOo+jcWuGrNjiEVvPq0/Mp7Us+BKntvOtGxp7TJjq3JvgsDJ4TWtr1Oc6SvtH6jIw9p2V0btX4LAzeJIFLLlIL20fGnqMcnVsTfNUGx1xKpyapaYjWdUnsk3PAyNhH59ZK+2qDb4R2T+CSC2hF+8jYc7Sjc3PHZ2Vw9wQuuUhWtI+MPUc7Ojd3fCYGx9cNk+fkrFbfrj+49OzDdkToP1zCJQHfwjdnW5XqX1xiCCbPqdfxfyPABHKUkyIXRaQcU8Il4ePqGyb+EYlZjkoBHBSY+OeoIkoyBxUwmYMfHJ0NVKCzAjR45wBw974K0OC++nL0zgrQ4J0DwN37KrB7mvAevz7393aLOrkpnYUKhFQA/rwHsPND4MTgcp/IqwMdQt5DcgArq9epgD7onmT/H90+ecZwa5WxAAAAAElFTkSuQmCC",
      "text/latex": [
       "$\\displaystyle \\left[\\begin{matrix}0 & 1.0 & 0 & 0\\\\0 & 0 & 0 & 1.0\\\\1.0 & 0 & 0 & 0\\\\0 & 0 & 1.0 & 0\\end{matrix}\\right]$"
      ],
      "text/plain": [
       "⎡ 0   1.0   0    0 ⎤\n",
       "⎢                  ⎥\n",
       "⎢ 0    0    0   1.0⎥\n",
       "⎢                  ⎥\n",
       "⎢1.0   0    0    0 ⎥\n",
       "⎢                  ⎥\n",
       "⎣ 0    0   1.0   0 ⎦"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Have we obtained the correct decomposition? True\n"
     ]
    }
   ],
   "source": [
    "P, L, U = scp.linalg.lu(A)\n",
    "print(\"Lower triangular matrix:\")\n",
    "display_matrix(L.round(2))\n",
    "print(\"Upper triangular matrix:\")\n",
    "display_matrix(U.round(2))\n",
    "print(\"Permutation matrix:\")\n",
    "display_matrix(P)\n",
    "print(\"Have we obtained the correct decomposition?\", np.allclose(P @ L @ U, A))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Ok... But what is the Permutation matrix?    \n",
    "For more information on this subject have a look at the [wiki link](https://en.wikipedia.org/wiki/Permutation_matrix)\n",
    "\n",
    "We can see that the product L@U is almost equal to A, but the rows have a different order.  For this reason we need to multiply it by P which produces the desired permutation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAALgAAABkCAYAAAA1z/qTAAAACXBIWXMAAA7EAAAOxAGVKw4bAAAPiklEQVR4Ae2dX5LcthHGZ1N6TtlKVd69voGkPYHlGzjxCWzfIC4/SW8q+waWTxDbN5B1Av25gZz3VEWeygWS74dlUyAIDsEZckDOdldxAQIY4OOHRqNJgtirJ0+ePNjtdm905OTXp0+f/i2X4WnOwBoYkH6+E47rHBblXd2LMn5QnMKx/B6feNwZWCED32cwfa60L0iPFfxHaXxWoZXOCHms42fF9/xwSyLMD3S8jTHr/COd31eYvea4rMfXy4D673mKTmkk9RQ8LRef48b8yNH8OM4jvlf6x2lieq4yNtr+o7xPdXyvtHMo2Eu1g0KbkhNHHt4G438rYg/govYN7D+VZtdjadkw+u25eW/xRBgs7Sz4YwtuDefCGyX+piOnjFj2b3M/itN0gfj5zxT+SrpClOyNws915Oql2FzyvqmIgUpbYADLvkk/GKhcNexqm9nzFx3fKk4f7BRyzoGROCgqWw07wNR+VfylCg5Q/JqONODJC0rbyYxOlP+1Tj+KyymO1ed3zAy9uqOfzxF9q7aOulnW72pjR5GxdkG5GzIwDqNGYQXYgVsV/58awsaCVwMFcDG+GciLk1Gu3HRKvY9Vh7kM8W/WEq+GXbzgRzLrdPxMpTPrlRiFatjpvDXgL1JwAe1ZaKXhTz/jQgoEN8bchLi4WSHy1yo1sWM8mOmKXKkMgTWxA6c6/nsZUkaTRDh+FU8mSnzvEut8f7TREwsIa3A1VM1fdIAfHzw3q7QtKb829kcC87twYMW/1GE3ib8oLXZZWswWWQF2oFTHf5SCCzjW256IcCGHxJT3kBUqUaRDbYzlUX/7iFOdj4K/U8hUf0hRamM3Xh4JZ2tMFP9Dx1c6ejNrRERt7ECpjr/IRYlI24lUlAO/+ZBixD8piWNVFxNhRZH31oDiuEbg5wb3VFkEuzCacjBTdnxwAf5Zx09RmWOvYRHsgImwVcU/WcGFHb/KfGeuZUxyvrf9xqwMU++5hWu4VkcwYIdkDdhzXPPojwGACzAka8AOtqr4j1Fw7uwPkdchXApkltMsUpxvaTkS4nJHx9X+Cx0oxJAYhl5+TexR28ZfD58SBgdn9Pvc9Vnakrwbbgur4J+k4CINYiD1EOjcheAO5DrDLPic7k7aPlYO3KmEtnVNb9OM5LwmdtrOYTeIYwpaEzsYq+OfpOACbFNisQVveoKH/fbbJikED/WXlzBTB0xcx1j8uerPvfHjERodMCY1sXOPkDMM8MbjwzH8NbHDa3X8UxXcrElWIUU4byv/p6PjEuicm6T3CnFvgihOXX/X8dVtymJ/WT/TuZnU+T+a1tq3m+DRsSrswsNTkt8Utk+swKm0Dm+k6VgVdvgVpur4pz4mZDpHuXkD2RNdEFaFafN1L/N2YRNvPm+Ux00l4Wc6H3MRMlWVJ4FHB+2akuOaMAN9orR2oBLXsSrsXKUw8QQoxd/hTfmrxL4G/FfRBw+fiqgxnw7MLs7AqhmQHvNSj5n7aqqLsuoLc3DOQMqAK3jKiJ9fFAOu4BfVnX4xKQOu4Ckjfn5RDLiCX1R3+sWkDLiCp4z4+UUx4Ap+Ud3pF5My4AqeMuLnF8WAK/hFdadfTMrAnVBwvdF6kF640li/cZ2m+/l8DMCvjq/her5ap9VUvBalUQb7bIqVgaznYK+O4rUkKmuLhliL4hv/FPRVoxzfRUVRltFvMqPyu4q8Y1hYA1Rtw6giBRdBWDpAtlsVNKTZxj1jyzZ3Ks8KQ9/4RyRMlN7WHOISBWcGOvRNZmimMu8sqEM3cmucWK5sBjNgzf05FX+RgqthLC+fqrWihrHeLGphzfHBbduacp0OURor4OggRng7cBRfQja58U/DW2fpcUMOS4xf6jio4CvgfScMvb5VWnANFS6Ov9QHZ7TxFXrqSzE6S3xZ1l3nXBmW3fIBc1qvklcjNbHjxvUUZAIzNbEDM7usWum9WWngmk7GX6rgYZqRIu4HgIwpKAMEnz0Vm7rIX6vUxI6CfCHe+a405pgZ1da3H+KtJvadMPcstNLA/uwQ6CjvZPz3osoGowLFSMpJeDqh/Jx1DuWVF3dMrg7S+AhhURGOzW38I8z8AwKUhC+h2AsFnxWrjsL3lEfprSh/Fby3gBQRJlyTs24YVWrBY5whLrAoN4DHbhRMefe9Sj4klHTGh9LTY9TPxj8/6AAvBzfIWIhDUh27MGJcbF8UrB+YBw1KdDHVsUdYLAp+jhKZBf/RCi6E3FxiYfjPEKfKYhvQAEwYN7fxjxEq7FhvjAOWG1cRo8L9UPt9q86PlUV5j0EJL7i53+Ia5pJR/Cj4n5vWLBxtXCDx//jWcch1iet4H58kcRulPBc/t+D/r3rjH/GLW8Xg5IkVfHPDaZyP7Wy1Nt7PuWHUX02ZUPBJ0pB+vyF79LcqZ65Jzg2xNLvZHK1vagG1v8mNf5rrZDrvuIC6HnxvrDncDbpYtXkXtlSYcQ4Nuk75ufCj4P9taraw01B8okYBycfJZkV2io9ZQaqwqTWujrhZ8DmnrbSNR0qwgRTnhbaF/22cmIlXwS5cYOYR7D7FpDQMAopu/KVF7LwKdmvcwuZarnXeuxYrMxAei//fVl+xBRfIB/rRjcKORVFaycjEX0fRUmEDG9/4J2VF5+IZZeBlGIqREwbAmGGoyXuM2fq+2II3Pz4Zf5GCNyTTGBYlbKRjodK+UTyMTIXkr20DmoA3ZlsYN7HxjzAzU4bX8gl+fHNcr+DarZT3GDKDEQl6chv98HdJ/EXPwQXlhQ4sCcSm0k7xAorFgfTXaSGdY615g3WjkJtKws4GNjqfXcCjI904B0uy+o1/hJtdrXgtzw1lbP06i62Utzrek45ER1DuV0l6OF0Sv2/8k2Pc0zbNgAYMhpiZ+6rIRdn01Tr4O82AK/id7v7Lv3hX8Mvv4zt9ha7gd7r7L//iXcEvv4/v9BW6gt/p7r/8i0fBeW7NQpj4OevlX7lf4SUzwBve8IklCn6tg9WBY+saVMTFGdgEAyxCC188uYuyif5ykMcyUKzgeivEYquOKI21J8wALs7AKhkoXYsC+JcotELWFSC2gIY1JkWi39vnSqxFYU0za0TCgqGiCmYqpDYZrCwSC35aSbWOvYSlfhnxhp58F+Vw3llLE+Vlo6dwX2zB1TI3oXsdKAcgWY/8UI2TNioqx/4erxTydQqfubHslhVxNWYAVkYW33M4drF1vGDE6HM7MCoYF5ZZj8qp3E9RcNZtf6zjSgcfPQC4VLlZ/II7w6AI0vyW83Az0CQvHqhdWypb1JbKO/YipvqFGu4wbKmwQjK26ml+OJ+D+ykKngVRmMi6ZnNt4p+wfJIPUZkRFhe1w+zDoCwamA0gx358z+CGVt246FwKzmOb3HN287/JP4d8KSW3LRhK23PspUz1y2HAqm5cNOUmcyflCNO1QPO5/rUONtPMWWZl3YryS6xzsT9s9U4NhQPXZJI75NinstwtL/6qb1w0xYKjqFvdPIfBaF+9dHvh8JkNvEMuTckAPtzKgVwpyWaxc1nCj4tnsyZP0ZgRDxpF5SOzcF+s4AK62c1zRBZ37UZyYG/GP8xmS8qWsaPgPC3BQFTZuKhYwQd6EB96bNuInO9t1dko5bn4IiKCw+dLR1bu2I8kjp813FfduKhIwQV0k5vnCDfTO48n7WZ2Unfpd+aa5NwQSzuq7jEgW8YeXVv1jYtKbzLZ1yJnzYIFVmeM+VTHbuAScXVUFAVnLxde7MTyQCfMPKTz1X2610tc1rHHbBTGxSkGAONiRqL9pdLgnHcgNoO3eUnkZO5LFfy5AOWUgBsGQIwJimSv6eOyvOZfbOMfYQZbD5/S/yBdITdAY+LYxxjK5Itbbuo5MCS5WY4B0OubpKqTuS9yUdToljfPSTgLp8G6xBnqBKzN2jYtiiFafEvYMSBVNy4q3heFkSiwZsWZWnBZeq/rVe6d0rGOnYVMOqdjsOJ7HdxU3ugYfY6uMrOJMPAcnOtg5kGYJlkfE7aAVujYAy3z/RGnuIO8lo9d3N5iqzm5V13hwYLCq2IFn++SvSZnYFkGYgUvdVGWReS1OwMLMeAKvhCxXu06GHAFX0c/OIqFGHAFX4hYr3YdDLiCr6MfHMVCDLiCL0SsV7sOBlzB19EPjmIhBlzBFyLWq10HA67g6+gHR7EQA8UKrrdDvHLtiNJYv3HdSfQTZ2BFDJSuJgTyZjf+aQahraOxpb+soxlb5tt2lcraakjW0Zxt06ItY4c84WcNUrxFBOe9tSiUHZJTuC+24GqcxTIslMKSA5KFSqvf+KdREFZD8ukXR1iiK+xvFLdFVzodFpVjbw8WZTEozrZpkdpidtwk9ohN3/gnImOJKJY3XdmINWewst74oEjJam78s2Xsu4Y7jEMqvvFPysgJ51jpdyKbWScWFtuX3EOwpjnnyrxS+tKbFm0ZO1z7xj+wIClyFW6LTv6LIvOJFBY7J6nip2XAFq9ltnz7SsWxGyP9ECPgG/+IBD6gWESk2EOfpYWnQsrPWeeAJWP1cxgde44VpYk/3/in4WbMig5QeFyyiEe5uYGzJytDFZnyDll/fufYh9hTurjGwNieNNxTMOMNGhXlmczCffFTFAHd8sY/RpqF3FxiXcKnapZ4ZLj0xj8prE1hF8e+8Y96kGfLZxERzneZ+ORDrkuMI+d7W75ZGMdujCShOOYJlG/8IxLshi2haN7ThvD7Cou29FU5c01yboilOfbhbsIl6biB4pT3Jzxdgb/BG/S5uC91UXj7Zx2qaCvBignMmE/Fkwx83lTMCo7tj5H+bvK5MDJVsnF/a7kVH9t2jnYc+2S22zeYPIbdpz9XGkbhLBv/lCo4G/8w6lJhBJYoJ34jgySVRTf+scaE/YHi7HDVsSZKQ+kPuSFU4dhhYaKIaxR7rzBn2KgNgzmmOydzX7RtRAOS19TtG0HF2W+bNQafKB5GqUJAs2sUu1WhvK3onD1HqIORu1NI2X/p+EzxsRmAnxwlqhuCX+jIkflY+WHgNngc+1Es538kTjGAuCn0cdARSipub4dtP5pZ9aapnyUOV/docExUkJsy1hRwg4bgWmD5WuUmUfmMWKaf15wngsJTx41CbswIF1Vu1Y+g3Cg5pKbSDizHnlJz+rk4ZQMoXsv/pDCeKTuLrZbkvsiCn36pXoMzcD4GNGAwZsGCl/rg50PnLTkDMzLgCj4jmV7V+hhwBV9fnziiGRlwBZ+RTK9qfQy4gq+vTxzRjAzEjwn5KCCtmgVJ7Zu/NNPPnYHaDEg/eb9yPYQDBee5dfsCJyl4lnUWSZt+6gxMYYAXSYPyf+t0M7KuqVpQAAAAAElFTkSuQmCC",
      "text/latex": [
       "$\\displaystyle \\left[\\begin{matrix}7.0 & 5.0 & 6.0 & 6.0\\\\2.0 & 5.0 & 8.0 & 7.0\\\\5.0 & 4.0 & 4.0 & 8.0\\\\5.0 & 2.0 & 2.0 & 8.0\\end{matrix}\\right]$"
      ],
      "text/plain": [
       "⎡7.0  5.0  6.0  6.0⎤\n",
       "⎢                  ⎥\n",
       "⎢2.0  5.0  8.0  7.0⎥\n",
       "⎢                  ⎥\n",
       "⎢5.0  4.0  4.0  8.0⎥\n",
       "⎢                  ⎥\n",
       "⎣5.0  2.0  2.0  8.0⎦"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "display_matrix(L @ U)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's have a deeper look.  \n",
    "The coordinates of the ones in the matrix P indicate the relation between the rows of A and the rows of L@U:  \n",
    "- (0,1) The first row of A corresponds to the second row of L@U; \n",
    "- (1,3) the second row of A correspond to the fourth of L@U; \n",
    "- (2,0) the third row of A corresponds to the first of L@U;\n",
    "- (3,2) the fourth of A corresponds to the third of L@U.  \n",
    "\n",
    "This is clear by looking at the columns of the matrix ```np.where(P==1)```."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAHQAAAAzCAYAAABGxyzzAAAACXBIWXMAAA7EAAAOxAGVKw4bAAAFoElEQVR4Ae2c2XEUMRCGFxcBGMjAZMARASYDmxDsECie7DcKMgAioEwGkIGxM8AZYAiB/xumtxZZ0mgOzYVUJUurq9X9S60eqeHe2dnZ/sYTzs/Pf3uKS9FMJCB8vLjtaX5Xir+c+G4m8y7TCEvgwsEMDC/uaYf+UOZUiH8L9y01c5eA8DvRHI/ZoSWsSAL3U3gR+qaCf6r9Y8V3KrtJ6TtEG9F6onFQMU+VH+VsF50D0Xtdz/+Z0lt+q/y6LsuW1LRPawKclcwFmTdq0UZANQhn7FulXyCgFAJXSl8qZgO1pvNJtBAkAoWpUYJoQ+sDPBpB5VnUxnejYK1f27Tmm4VjgG6UP9I4X5UeK1Y4hMbdC1VQrs7o5f3dQZRnhzDoB8VsATqKMABjn7MR8g8MeFuB0kTzYLfCO5oiZ0DmJ6IHiBZsAb2xglAaBVSdjhV9KuZS5Yciym5dYzgUUz88/CFYFjg7OFdA3iwcYhVEb5u3slDapHJh7KOns6la6qMqwNN3CUUA9yQiyGwLWTSh/WBXSCqz3dqoFYOAapCUST/cJbyWvHhHM/kCxtlG9T6t5Wvfu0y02DTVEaC8b3P9QyMIqFoZWLHtngL6PwSX+kPCBExUrVm+WVmp6QHmc0UW0PcUgjFAU/o/Smm0kjYYQ18k6Pdj8CM6gFhpAuVRuVjYvazc28jEbffyXbr6IEFydt0g0CmYFV3sFDTlhfJRrRi0ctXRVK1vACsz42gKPkehKTnwGfFQ6fabNCdh0cEYq85qh46pXNRwMAQBrXtgcflMdNuh1K82SLCousdKtztT+QNiRqa5yEG92qZpRaoJUM6NZ54Rn6rsWkRtF3uaLLtIvLFLnit1jSBAjh1HfRlHppzVrmwNh+gmihpFGvQjDCkeKVbfm0pZOa8UX/SdeYv+ZnyhGVxGWwyT1lQ8sgNZzN+Ud7/9uFDJaRi5C2gjeiwi5M6rWJT/KKAagMBu5GIY8xkjiPSFflcWmPLZgmggVIKdGxgFnNvcazZ+k1U9u/35qm6Ayvnphqx8w5cii2Z3ITEX7s6ju5OJlvdQF66F/hbYLL7yHrpQ/ILT3gvWlIpFSqAAukjYwpMugIZls8iaAugiYQtPugAals0iawqgi4QtPOkCaFg2i6wpgC4StvCkC6Bh2SyyJuUut2JMV0u8PnC3Orazs7lTcjl9oJjkcKx2vYN4xpeHwB32aA7mogufdknPKwuvO0lO3lFANTBCnMrZGdqdHY7Vt1cQ77xLTuFgDpidnbyjKldMTens3MvhuA+a4hvakziYi24vJ+8ooH2EMkBfnql4+yNWgQVm+cwpHgq+Z7JLled2MOepsLOTd1TlZhZadHiBx9tfZ4fj6ODNlQjV995qPlTUVw/+zUO1bgHfnZ28ZwuoKwYBjBCTHY7d/qm/RYezuymYT1VTu9b1or/1X3I6Y5RuVO/THNumswdUDMAIYOIpATPm/aZslmBgxdR7CuiDTa6WAcaSWb7BsZcAKCBWq1KM4VuT5HAc5HiYCvNxGma05lH4XExy8p6zUXSHTQHKucXOaXQ4vtM5veA20tR2L9+lowTx3MrJe7aAomaIHqmZykUNDx5E01StT61amRlHg9PfHVBz4fOplZP3bAEVI3zYd3Y43hVMhzyWJmeWG2yHNnrfuR3b/haYHC+tnbznDCg7pbPDcVsBOu05s7hyc8MoDua1Zurk5N3GKDJDgFVqaslleMjfdyy6etWi9hodjvtMRHQmczAXbTQDC6qTk3cjoCLA4AQ7s0Zxdq6F2tnh+O+Ue/2dysG8l5N3cbTuhfl8OmsDFEfr+cAx3EzmbBQNx+V/NFIBdGVgF0ALoCuTwMrYsc8W/pm5XWtVLOr3GN+aKxPneOy4eIlyhZ+pXC6Ad/8TZHOOGm+GhVJbCXA/cAezP2MyDDBRe3G5AAAAAElFTkSuQmCC",
      "text/latex": [
       "$\\displaystyle \\left[\\begin{matrix}0 & 1 & 2 & 3\\\\1 & 3 & 0 & 2\\end{matrix}\\right]$"
      ],
      "text/plain": [
       "⎡0  1  2  3⎤\n",
       "⎢          ⎥\n",
       "⎣1  3  0  2⎦"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "After the permutation the resulting matrix is equal to A:\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAALgAAABkCAYAAAA1z/qTAAAACXBIWXMAAA7EAAAOxAGVKw4bAAAQHUlEQVR4Ae2d35HdthXG72b0nJGUmbxn1cFKW0GUDhS7AlkdxKMn6U1jdWC7gtjqwHYFstWBnPfMSNpJA873w/JQIAgS4L3kguQCM1j8IS7w4cPhwSEBYs9evHhxcTgcfpOPuTcvX778Z+xCzasMrIEByed74TiPYdG1szvehdeKU9h3v/uJGq8MrJCBbyKY/qG8J+T7Av6tJH6XAq1+Xci/84lQ+q7S9/faZ7+vpeLiFs36WP4Hxa+WwKF6vwvrVR5ZPQEPy3XSDdivm8xHCj/Kf638juB0fhQkVNbutg+69ED+G+XdxE31i9pBoA0rcdzD6yD9txT2BvdzDyHYf1T+z17eaLQUdoHC/P0WLwwKeu5K+fd6uUHGKfh9DR5U+zmpBrgTAYnqd65p9Dfy5JNkqwx2/iuFb6hAIQNlv19ayLkZcRBOW2AAy5XCpFO5kthRAs98kEoj4HflHZf+tTCuMiWxXwoPshEbXzS7KcwQdps+FX+WgKs1NG9IMtr7K+X/KD96FzblOgOiPO5eBog7vL1xFF/CvVNbRz0sl8TetI2Ahu6pMn6RHxXwktgNsDD0xlZ57qFQ4eL4/2RAEiF323sBsqndinN3IrgOsGVGQoTLzAP/8lslHkfq9cuUjpfEjhnXE5AJhJTEDkzGN+Z6s1KskPJOxp8r4G6akSAOTemh4Id4uUHMTPCv2dTF9bW6ktgRkCfi/adACTCjMvOlXEnsB2HuaWjlgf1VCnhz/WT8d3IaEijupJjDpqUjMe3syutaSvgpd98VXvCPcGBOgeUv8sw42OCDuHX9UBq72mcdAiHhjcAnxbFZ0eoIfE94lN86XV8F7y0gRYQJ3nmjlWN7z4I/V4P7OF1cIBFuAKfAmvBe9Sr5nJHTmc+lp8eon1dVrxtywcwDLhpizBXHLowoF3sVhvYD8+iN2XSoOPYGhx+AH5/jZsF/tIALIQ+XaBgWiE51aNXFnDDypqe9wRTHNMLsypnmU7iWxo72BjuaG8woFZ6H3HtexU9xi2L3gQkvuHneog9zuST+owRcIBGM3xUOmS5+B2K2t123u/SDZdxgiJCfN8QPNVsUu7BhVnFz8sYKvnngNM6/V3ps5iuKPUIob+HsmStyuZc1C/7JAi5SIZ0VwKyne5UzzRkbDMub0vEeE2MZah97NfaqzX5mGCzdhqWxCwjTeccEFCZsb7Q5uAdNrBVgF7yOY8YZE9pO4bnwTxJwNQrIBwpNixwUT2lBgNvU2umEEqbB55y2wjYeKSMmxK5t4X8X/iBIF8EuXGDmFexVgOegPBQCgm78hUUsXQS7NW5h05dzpXt9sTID4cn4swVcIC8E4lJhR6MoL+fOxF5H0EL3UBkswkzteFjPWPo71Y/GCx3aL+fGKoK94YTFMAQj5rgBUviLYI+AtbHP1uBNHSfjzxLwhmQaQ6OwZN965T1T2gmoQq7/Id8xCZTmLcBHhdwMzinOAH0h//Q6Z7G/Dqtfu9r+V5P2Z6I1YgefW5YP8GMmYno50w4u5dfGuw+ZscY5ObmOfv67JP47n5sZjf2kq2gSiA1dO8UL6JU8pP8aFlIabc0K1qVCHioJ/650+3ulZ3fgkadde2PCtI4m+ZvyWsKJy68N+8/ChALggdLXfp3NVmvEHgwkYwzXb4N8l1wS/5n3wQO2tdMIMRA1rzKwFQYkxyhiZu6zLBNlKx2rOCsDIQNVwENGanpXDFQB39Vw1s6EDFQBDxmp6V0xUAV8V8NZOxMyUAU8ZKSmd8VAFfBdDWftTMhAFfCQkZreFQNVwHc1nLUzIQPZAq5VITZbdZzy2ANx3smsicrAihjI3YsC5C0fnsNNaLsg2dnGvo5NHFrUKJBNYhfHB5SggufEG0e6s5fGLgyFqsM+c2MPEztD2VuUta0kW4OrUoSCDTNockCyH/mhGiIv6VSOHYZvFSJYfObGoLEjbtEZoKmffQnsesS7LbpqO+ebTBVzg1SxOyaO+oMwMubm+bKHcWh3lo7VqnIncT9FwNm3fU/+TJ6NWQDOFW42v2DOcFM41/yWtO3ya67MHnD39w4tUh7Y2QI86oSzYh9laPhiwx0CGjp2SPpaPbzu0nNwP0XAoyAyM9nXzJbJ0LF9kg9RmRGWcnzYsNVDi7aMnfHEnMj6tJHCEXey3NyUgDNQmDihMzuK60s5vnphT/jQbJO6uSr240cGBVb04KI7U7DblKHf8Ln+ufwr5cU0c1ttpnbmI4RFnNpHC8Sceys0hr9ij9GWnyf+ih9cNEWDo+k2eXhOOCQiHuHmBrW3E2ERS9uNN6T9KZeaAayuWcKtYRdeFEyxg4uyBVxAN3t4TkSyeLjcxKFFW8cuueFtCQoCe9y+kr+xg4sQ8D/L4yy8TuX9xYZOHRsRs72tdtOQHyxj6VCE89Zmk4cWbQ278PIGqsTBRX81OcrS4AK65cNzrK+HhvDNHFrUAldko9iLH1yEgP+vIdJCn1eLs/oXszWdBhb5ow+a+q1NTVafhabBU+d7WPmjQ2FkqtzaoUWuv1vELszIC2sfmCcdpzxmftZAbPw7173EsXLzX6sjS4Or8CYPz7FOitALxbd4aNFhq9iFG8HmKI5zG4cg5AZIKTaelVCuocs+MCrr2IgGJCuX7Yqg4hyew2pUe76I8gD9SZ5VT0C0Tmn+RSF1uNXMpux/lLfo2ShqB4I51yVGJotM7tSrBk/FLqLmcuKUNQTMFMa41eSK2+qwO5l4bu6b+t2xEVnvwfWDzR6eI3I3e2jRxrEfJDfFDy7K0uAiurrKwGYY8DV4rg2+mc5VoJUBn4Eq4D4bNb47BqqA725Ia4d8BqqA+2zU+O4YqAK+uyGtHfIZqALus1Hju2MAAWfZlAWcsU1Ru+t47dCuGWBRzy1KIuDn8uywS+0LUJHqKgObYIAVVGT6UE2UTYxXBXksA1kCrpUh9nx/Jc9ek8054WazVcfRF3lmr+p2zEDWXhT1HwFB5bOBRUHPsWvsXi83yFCZow9wCaqamtzsoUXWUY87y/q38lLblF1Z77d8WDLp4Bxr7NTQw2BV3Qj+XAG/FCoMd/sK3kASYu+kvm08qIOcj8FHyv5uQg7f4YuPWL3UPZezB2huVNuLDJZ2h9tYQypXDLvaZpZh2yg7Md2OSIWk8W4npMJBVxI7oNR+Ufy5Ag7Q3vkWDXiuOaEdYlnXbXtkW055aH3SzAy9uofqOjKf7btDX9ePVrkC7Agy2s7f7oupmFQKK8AOt0XxZ9ngAvkWpBHHsVztHvHIdctCuGLTKfUuffCPYTg2LIZd3PIVErOOfZXu+qB8Zr0cpVAMO0DXgD9LwAW01byO4Wvw2NOvLJ0IMWPMTPCLmhbi+lpdSewoD2a6LFMqQmBJ7MApjv9OhJRklgjHrrpQmGN757x5uZ9s9MQCwurMJFWzmUOLhJXPtfjYBC3+pbw9JCZPZ9Vv1sB7cfxHCbiIRnvbGxFFR50J75gWyhmM0UYSF6mfQ4scBoXcoJzNwVTv27ZhNaWxGy+PhLNVJop/kn8q35tZvQ6Uxg6U4vizTBSPtINIRTiwm8cEw/9JThytupgTVgTZCTeNKI5pBH632kXeCW4R7MJowsFM2bHBhfUHef5/vZU5Fv4i2AHjYSuKf7KACzt2ldnO9CXlYra3/ca0DFPvTTv6sIVDi2Jc89oS4Y59cW48roX3oviPEXCe7MfIM4JdqDvZNGdM21hejIROPccm1P4mDy3yeDP+YhQwm0ad93vj2C9neUvybrgt9Nu3+OL4Jwm4SIMYQI2BNvB+iDkQ64xp8DnNHb9d4mg5cIfOta0+vQsvBOmS2Gk7ht0gpgS0JHYwFsc/ScAF2KbEbA3ejAQv++23TZYLHuovizBTbxi/jlR8y4cW8YwQUwzwxutDBGjMleQdXMXxTxVw0yZRgUTDy/8hj43YOqV5SPqoEPPGOcWp6wv5p9c5i/1l/0znYVJpDi3Ctaub4JFfFXbh4S0JZ4u0b6zAqbwOb+TJrwo75ApTcfxTXxMynSPc0ZVNdQitwrT5q3zo0DqsfF4q5KGScNFTrVT/ATzytGtCjmnCDNSeyNWUWx32BhdvgEL8Hd50fZXY14C/HvzDKFS3KwZ0w7Oo545um2qi7IqI2pn9M1AFfP9jfKt7WAX8Vg///jtfBXz/Y3yre1gF/FYP//47XwV8/2N8q3tYBfxWD//+O18FfP9jfKt7WAX8Vg///jufLeBaHboI6VAeeyDOw/yargyshYEpe1E2f3iOkd7crM8U8vFGllNZ2/DEPpoih+cAdGvYhZfNYc/B3jjSyW9KrTCh6jia+2wNrnbYoMRGKzQ5INkp9lCNk5d0KscOw7cKOcCGfx/HN4Z8jFBiBmAbqdsPrjDpKvYkRWMF2CjGmJtHqaBc2p2lYz8+lfspAs6+7XvyZ/IP5AGcK9zRg3/UMW4S2+U31s/ZrgmzbZXNqlPlK/YspvqFGu46W6ebUmyR9rV6/8fKmYP7KQIeBZGZWfQAGsMowph9uCmzbszmdxW7ETg9xJTLOaBoqOaTub8pAX+sHsS+ArJPrrh+E+5LCXn4hXqq3Yo9xdDwdb4beCLOMUXvesWwqXNm7pO5v+M1mowK5BYPz3H9EnZMkxxSWx6CQWnzg0i2LR/8Lju5VezC/QavjmJvc5YLz11odQSe/EGn6/4NMVQuyf0UDU6DHJ7zugEKWE6HTWlfAzFmFuR0ZqiTyXxhPFch++olWd4rULF7ZBwTFfeYGTZrormRF74MS7lZuM8WcAHd3OE5HoM8tRvJXvYs0cUOz2nQbRn7QbyjvVFuaG77yp9TxbLeojQcDAVJ7rMFfKAFbOhVH54jIt3nSwP4U9mx5wb7jWkY3osv4raMHUIa/ChG3rjxbSwPnGh03PdKj83cs3CfJeACgs0Ue91zDXXk7A79zkyTWGcszx42rb5ZQrWNacJq61H1V+wnDwMmSXumIrWJU2xvtDljP2jezsV97kPmI4GJ3VFOiwlMyqayqUnVdJxpwdT5Hp0fTUgg4JfCx8KO7y6UYOYhH83SGQS/oOIVe0BITlKcIsAol6uwvPLgHEG38Q+LWPpk7nMFnMNzYkLAHZgjnAiSLbcaeEKOkljs4B9hBlsPn/I/ka/QpkslB13FPkjN8AVxy0M9HkUSm0G5AXpjE9R4MvdZJooa3ezhOQFhlnTaxRKEGgS0zeoOz/ExNvEtYUeBsO8EzK1TmucizF4n+FyXX4T77HNRBIDp3rQ4UwsmS2+5XuXeKx/t2NnIpDSdRItfyfNgdin/Svkp80bF5nFqi/fg9MNsP6ZJ9sewN+agsGKHiBmdOMUcZFneN3F7m63m5F51uRcLCs+yBXzGPteqKgOLMuALeK6JsiigWnllYCkGqoAvxWytdxUMVAFfxTBUEEsxUAV8KWZrvatgoAr4KoahgliKAX+hhw0wYTtsd8xZDAl/V9OVgRthQPLJq93zocYQcF62d95Ze4VjK1De5RqtDBRnILZC3oL6P/oGM7J+M+zkAAAAAElFTkSuQmCC",
      "text/latex": [
       "$\\displaystyle \\left[\\begin{matrix}2.0 & 5.0 & 8.0 & 7.0\\\\5.0 & 2.0 & 2.0 & 8.0\\\\7.0 & 5.0 & 6.0 & 6.0\\\\5.0 & 4.0 & 4.0 & 8.0\\end{matrix}\\right]$"
      ],
      "text/plain": [
       "⎡2.0  5.0  8.0  7.0⎤\n",
       "⎢                  ⎥\n",
       "⎢5.0  2.0  2.0  8.0⎥\n",
       "⎢                  ⎥\n",
       "⎢7.0  5.0  6.0  6.0⎥\n",
       "⎢                  ⎥\n",
       "⎣5.0  4.0  4.0  8.0⎦"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "display_matrix(np.where(P == 1))\n",
    "print(\"After the permutation the resulting matrix is equal to A:\")\n",
    "display_matrix((L @ U)[[1, 3, 0, 2]])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The previous code is useful to obtain the decomposition.  \n",
    "Now we can solve the problem \n",
    "\n",
    "$$ PLU x = b $$\n",
    "\n",
    "using both forward (for L) and backward (for U) substitution.  \n",
    "Since permutation matrices are orthogonal matrices (i.e., $PP^T = I$) the inverse matrix exists and can be written as \n",
    "$$ P^{-1} = P^T $$\n",
    "and there is no need to perform a matrix inversion.\n",
    "\n",
    "The method **solve_triangular** performs the forward/backward substitution (it is a wrapper of the LAPACK function *trtrs*).  \n",
    "We can write:\n",
    "\n",
    "$$ L y = P^T b $$\n",
    "and\n",
    "$$ U x = y $$\n",
    "\n",
    "and solve it:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAACoAAABkCAYAAADwkaJHAAAACXBIWXMAAA7EAAAOxAGVKw4bAAAGSElEQVR4Ae2cW44VNxCGDxHP0YRIWcBhBwOzAoYdcFlBYAdBeZp5Q7ADYAUIdgA7IMwOYAGRMhllA8n/ma4+bh/f+nomokuy7C6Xy3+X3Xa5jmdunJ2dHW82m89KMXp/fn7+MFYxNU/9fJHObUyv6m7c9CpeqoywT1/9h5nLLyL674v3AL4P9JWQLwmsg0t9v+4w9CAerD2gMIukxkyVd0p3VL4qNvAEJG9W+0vs20ovxKsyjm9RT2W3KGVH4rxRulS6qxSdS+InSTr4Dp4rf49Qo/Oz8vtKRbA/JDV7FVJ0pfRQ6anYb72qqqLaPZHgkXIHkkYqMxo8v+K5RFVAS0oq6lk5LiJyn8Q7FWhGLEtLAT0VCqZNSDbk1GdpdqA11hLCW1mUqpwdqAcit0Jcm6EvGeznksASFo3NTcNlQ866mqXZgTbLECBiw2s8+6iSYGcH2vT8Ufk2gsIsSn2WlgLKlsuOFtIdMS48q4f17fMQoDbxzRqtMpYipX+VOm6jnnE4LpU7B4MGKjPsj5R+5blEVXs9SqQYq0C2OL8Tj7n1QbnzfJSz1cL7A8GAsB5OyIlyPh7ye3qO7Viq6lIfoFUOtDrGK9oj8a/ExFcYREOGflBHYxutQMdaMGy/WjS0yNjn1aJjLRi2Xy0aWmTs8//Gon220K2s8qyxDJ4QDvGz2r2adpIdHICosqg6ACQhn6dNcu6ZeAQQzEkBS5Ikh0f1STkvR5yLl8ahQXeRqoBKC5boOBR0KB6OhnlVKsZJsosFILDaF3VoRwdDhGeOD1qyymIBCAB9FSAsGKPwBUIZXjR2yLOzUnH63Aw1xp4FMOWLHiOv+qTzq7rSS6Bi77QA06faOeq3cWUBACRDbivBnkzDMBCp0UCs+DKDgUo5HxGhc77gsWTnsKQegP7Y1FqeFLYKgSNUyJxNTQkTJY/NTas3a6cCEL+YYG+LChxLzS3lxNeLJDkb8tjwGs8+qqQ+gP7T1FqeFFanHHdvK28tqfKWlGz0rWJoAOJP01ttUYE5VqMT5eHHA/jc8NLX6ABE7fKExejso4CGoexT8dwHpZyh/FuJ6AfbrCOVXyuxdT5Q8mP4BCDuNWLZrAqoNHxQAizzM6QLYwjEYQMQAhANKhhAP0/J8hKS6/gLfrtSuXqOlhTNXb8CndrCq0W/a4uyz7JslHaXqY1Uo4+t1y1pzNGtEruNeTIqXhvC83c74foxTT0mtXv9RlsgU8S2QJwPnvnxgHlURZKdPQABMLwfSwBm7hBAaH+SyaGV3CIBCLymJwEos+TvOYDUqd1iAQhcObwfkiN13paNl8lHByCq5qhAYb2ffCDi2ZC75cOvi5RZZvauC4lnZyXq2/smkfbDLhYIJIpdPErlGIC2L9Uzv0tUXMOrLGq9qFPOTYA8UWI6xH5KFLtDBiI3VYov0xco4EgbgWboCTtyzSg7bMgXqCoAUdARr27AYSV+vM1Z5DKuwXHN2qkARNu0agsVkGNS22pXsKFnOkRJ7WzIYy9jPPuoojpgVgGVHIs1w2yKaduHWDXYyUIyi9qaHNa3z7VAsQoBMXKf7jYPpY5GByBqgbp4u49QoPmYsDBxffcCWFzpcDcg1DmRDiIi/uLOUHJTsbWmyocNQGDJBlALCl6MJBcNVvASkjfvK9Y0y6sd+qySJSpXoFNbebXoatGpLTC1vnWOfrcW7eXh+1bSloh/6i4a+PxcWW3mDUAkOsd1M38yIbJjC+QiAYhdjyqp0986jMKD5BcLQLRQ1ClDjifkfNC2Il8YHYAYsjw9FtjsWT6CmTNV7JBnZ6Xkmct09QLaDLnvPJueZK42R8nKXUVxrlcDVYdb6TUPftdFuWQgclOl+DLVQIWHpajvkJdf45vENAEIAeSr7TXkHsLY3LRqs/b4AEQz5JwubeJbJ1W52tmQx4bXeEXdNTsTc5MLBSzwPrFMcfsBPvdLwgsHviyHQvSEZBYtHhqLQAUAJXuKxOcCARcN2msbIQrvmZex7dNjb7h8MNufYFhHDJsNneMJ9GEDEIaMXGD4qGwYCUpgLW4xvlSy5cuCZzQxwnrz/wmG9SYw2SCC6tcAhBnrWud9dqaDvsgKdGrz+ws+V4ND/dfiPxYACqDss6klp7gHo2Qiiu1crer/AP5TPXUImJZvAAAAAElFTkSuQmCC",
      "text/latex": [
       "$\\displaystyle \\left[\\begin{matrix}1.0\\\\2.0\\\\3.0\\\\4.0\\end{matrix}\\right]$"
      ],
      "text/plain": [
       "⎡1.0⎤\n",
       "⎢   ⎥\n",
       "⎢2.0⎥\n",
       "⎢   ⎥\n",
       "⎢3.0⎥\n",
       "⎢   ⎥\n",
       "⎣4.0⎦"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "y = solve_triangular(L, P.T @ b, lower=True)\n",
    "x_solution = solve_triangular(U, y)  # by default it considers upper triangular matricies as input\n",
    "display_matrix(x_solution.round(2))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Another LU solver\n",
    "A faster method to solve the linear equation using the LU decomposition is to use the **lu_factor** method (it is a wrapper of the LAPACK function getrf).   \n",
    "It returns two matrices\n",
    "- The first, LU,  is a matrix from which it is possible to derive easily the L and U matrices\n",
    "- The second, piv, corrsponds to the pivot indices of the permutation matrix P:  \n",
    "   In the matrix P, changing the row i with the row piv[i] (for all $0\\leq i\\leq 3$), produces the identity    matrix. The same permutation transforms A into L@U.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Construction of the matrix L:\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPgAAABkCAYAAACrdf0zAAAACXBIWXMAAA7EAAAOxAGVKw4bAAAPMklEQVR4Ae1dW5LdthG9o5rvlCJV+T+jHcx4VmBpB5a9AsU7iCtf0p/K2oHtFdjWDqysYOLZgex/V0VSZQPJORw2B5cXIJsECQJEowoXYKPx6AM0APLicfby5cvLw+HwG6zPvH316tVzX4DRDAFDYHsEoJ/vUYoLX0kQdnbuBLyBn8yu+d19ML8hYAhkh8B3nhI9A+1L0l0F/x4abwrtQctIhkCuCEBnf+iXDTSSThS8z6d+RoKc5v8CewX/J3VEMIJfeqD/4PEJ7Heg7a6jqUXOKXXv460Np7XlfeADWUNDwR7C/gL7Pfh/hPW+Bwylhbh897+B+y0sXxG+hf0V/slpDeWzdVgtcsbiXBtOKeSNUfBPKOBz2G9QsT9NrVzE+zvisJN4K3Hh5+jPZ3YauzC1yBlbWbXhlEre2QoeW6GIz6/zt550bkB7CgAeesJKJNUiZ2zd1IZTEnm3VPCnaBEfPK1C3r8ZvgdTi5yxdVUbTknk3UTBlaPzo9gWs3X8WuSMxbk2nFLKu4mCo0GI8g59cd/DFL0WOWN1vDacksm7lYJrGsRjDdMOeGqRM7aqasNpEXm3UnDfu7c0AOnd+L946aYWOWPrqTacksm7iYLjHUSm5r5puNDkY1ts49ksfi1yxgJcG04p5d1EwdsG8Q7uhadxyAjO8D2YWuSMravacEoi75YKzqWtn3taxRVot04v52EpilSLnLGVUhtOSeRdSsHlg4CMvl1lQ1G5Wu1/sEdbUvHMRfIf4DaL4hkBfk7Pv4J9wec9mFrkjK2r2nBKJe95TMWgkOyFaPinPQ3XpvPdmevJm10ucLmklbR/k6FnOFpzc8k1XH5Uo/sFnm/h7snUImdsndWG0+rynjkHPjxpFTG2kiy+IWAIbIgA9Jj7PLj9+2ypKfqG4ljWhoAhEELAFDyEjNENgR0gYAq+g0o0EQyBEAKm4CFkjG4I7AABU/AdVKKJYAiEEDAFDyFjdENgBwiYgu+gEk0EQyCEgCl4CBmjGwI7QMAUfAeVaCIYAiEETMFDyBjdENgBApPWomPp2+xLChD3Anjx3HMa7iLjpneeh36y7hw099hkbmB5AZrsIcdjngZlvETJuD7fLoCYWUXAcHYbm5nlptHWllc9gqMgsy8pQFwqN9fGftNaLrKnYv+GZ9mocoCfO894P9p7uA0v/K9h/8Az08jOtGW2CyAWqBlgObuNLZB98iRSyKtScBQk9pIC9sq8IKEzSJOjOUdl2ZHGMN6Q8ghhvOWkMfCzI+BONHdUvwvM4Bfl4245uwAisi6AYWwbiyxB2uip5FUpOESPPaSdozRHZTmOSdDkqRYctWV05t5w31FNVPI9XYYg8tONxdZNq2R/bTgkkVer4FTQD57WI8rYTbM9PCRRkX+HIofeo6nkovy+fLhXnMZ3AsxdSLm/sdiWK/lxyWvDIYm858cYnz45incaeE85OcnlPqg5qYW9lc/wo9QBeXCEpkvHl9ZjBsDISH/3VPgv5JVObUgSHx5D/MWF1YZDSnk1I7g0sNDoywalaahHDQ9CUrmpsPJlneG8eNCnxOSlmZzPXbRsf1fBNltpwwWrDYdk8moUPFwt9yEywt5Txn38uPYWit59UMNzcxYbaJy+NKbtCKRzkVcCCa7BnYPtHnGpDYdF5OUU/S9taxC33zh878TCIz2RvCMLfdCF0vKLON/Jj6bueOYX6b8hjOe0XcKlkDetDX2AQ3CxZnFsC0WiNhzWlvczaQejIziVrmX2TY+Fph5ZkR7/DuFfYc+kEK7L/GD5H/gbWC6E4bRdejN1Pm6aufoh26LY5irnWLlqwyGlvFTw/7YVIK6vPhY5pB2CcRTm4Y7dyA3/Ba0vU4fG0fwd+EQhnKDivYtgWzwKd/+0+NrBo1Y24rQns2a9/ylAjY7gLSPfl31/UV2BrrqkAMpJJb2G635UY/JU+mbKgrAvYT/Cyszg0Pr5Tt6Px7h7MNHY7gEEyFAbDknkVSk4lEx1SQGVEdZ3yQF7ZgrEcC5Z7SxonI7LyEy+/vsJ45HnFm7uRl4lZNTpyovyh7BRYdsltFMP8KkKh1Ty8iOb1nC0HrykAIXm+zPfk/uXHPwKGpWX79990yku4vK9m0ryT7gyijPPrKdnKB87IRr5+m8XQNzhMfV3tI1NTTBz/tXltYsPMm8BVjxDYCoCGHA4kHKWfKaaok/NwPgNAUMgDwRMwfOoByuFIbAKAqbgq8BqiRoCeSBgCp5HPVgpDIFVEDAFXwVWS9QQyAMBU/A86sFKYQisggAVnP9b8zil/gKTVTK0RA0BQ2B1BLhupDkijQp+AcvdXSerr0AzYwgYAuUhwAVXzRmGNkUvr/KsxIaAGoEpS1XVifYZsaKGswT2Kj/DL+vO+2z2bAgYAgsjMEnBoZxzD6XnTjJOGbh8zicC17D/1Q3AM+NwjffkSwTcdNbwR+DQFWdMPoSzU5QddNzJx28k3osiukQL8YzJPiTGEtgPpZ86bG151AqOgvBQ+tdweQDDAS43g/DigmewYwcxXIOXL/4+Po7sTUNu0+TZ6GzMbNRs5FkZlHE2Dlr5wEe52Rl2h2LAz85V8M56842vwrSy++IKDWnMxl7SyMlNIY9KwVEQ76H0oFPZOTJ3DTEEIHhPeEBrFLhN5wCX0/fmMAj4/wE/R/FsDMoUhcME+bwXRbT5c1ZzNNvJBqCBgkyQ3ZtKLPbeRDckppLngVJGKl23rdOJcwP/UxRWtnY6QUde8vkMt4Ie3XjiY8qIFouDVhTOajQXRWjT2wNfKuxTYZVEHq2Cs8H5/ieXKTfDgwZK3EzrXQbQOEq9dmkF+KNwmCBf8zoDjEIfJMc61AlZFcOaCvtUgCSR53xMGjQyTWOa9B860uTU/BKufEQaK8bm4WvgEBIKebF395nmlQXhvtmUj38XtJTYpwAspTyaEVyUNzSaEBNNJ+Bix9GbtiSzBg5q+dEoqNzsGIvpFNXCjTNuiv148SZzJJNHo+Ca0vOYJZVBQ2Uj5Xt7cV+CFQKqcVCk1Wfhx7X+RRF9npqf18R+C1wXkUej4L53bxFYeqIpFx/wo5q8u0s6JbhL46CWGZ0h/6k4uShCnUD5jJthvxJ0yeQZVXA0Lpma+6bhQpuisN0xySuBt0qyK+CgKify5V9zwYsiVIkUzrQV9mvBllKeUQVvheR0+sIjsIzgquk2BGOHwHSk0/AkmTVpERy0EgIvdoZzLorQZlESX1LsEwCTRB6tgvP9jyvL+uYKBNXFB21ESWNoitLPI6fnpXAYlQnKfQmmwYsiRhPZF0My7BPBlkQelYKjsakOpecIDXty8YEDmEzpNSO4fGSQWYKTzDbeBXGgAEH5kA9nOWwAxJNLVjsLmntRBB6LNEOye9uQFvtS0Eglz+j/4A5gHK3nXnwgydzCQ+W+EULfheBs2DRcCEBzconAHXmz3ygclPKpLorYDIGZGWtkBw83HvGbTv/yDOY6iv3Mom0VbXV57OKDrarW8jUEVkIAHaRdfLAStpasIZAVAqp38KxKbIUxBAwBNQKm4GqojNEQKA8BU/Dy6sxKbAioETAFV0NljIZAeQiYgpdXZ1ZiQ0CNgCm4GipjNATKQ8AUvLw6sxIbAmoETMHVUBmjIVAeAlOWqs6WDitrLhCZS0/t4oPZKFpEQ2A6ApMUHIoqxyzxgIcnsFybrtkLzp1Ru7n4YAjmCIxOkkVaxI2bSwZPntXynWSwISElTk5eIvFPoHFfRBLT1g/3WEy+xMMp+1Sda2RTKzgyijl0/hq5cf+rrzPgyF7MxQcNaoGfSIx8qbJRaBqils+XR3JaKpyQD2eOxIY3wrD9HeDymZYD1GoG+XDnZNQlHkgjRuca2VQKjoyiDvxnTkij+IsPGsQCP0tg5CaN9Hjxw6jR8o0mlIghMU5UZI7WjXK3IlLxfAPNogggT+6abE7HbeuIszG1WQon7Ue22EPabwKScYo/OP0MxMuRHItRJxMwYWNgA6ENGi1fMIFtApLgBGx4Gg5x5FkGnQGdV22dDDYdQz6eRXDSKjin0b5TWKQnZHjQANC9XHwQlBEBURj1Ev4amB01zF64PGr5hD8HNxVOHDi4t3ywk8wBkEAZFsHpPJB4RwZAD7uHsOdROOg0BGny3aioiw9OpbinLIkR0uLUnB8kB42WbzCRxIGJceLxYDyJlqP417DykYoHiLhT9sQojGe3JE6jCo7iiPIO9YSaTsCVrMSLD9zy9/2LYISKZccnJ5r08+ietXxdhHw8KXGSNvk58Ooui4D/I+wL2JNZZT4wLadz2in6mOyPxxgkHMCyEe/14gMR0+dqMOJfYpqpuZbPV47cadE4AUNRbs4S+3j+DAB+dHhyxyNUPg1OB80I7nv3lkylR+b0R2uyvPigrfB/QQhpHBp5niMe/8aKxgjpNMfsjGWq5RtLZ254KTi18sk3Ildc/vVErDmFz3WqHt2eROBRBUeFcspIfl/DF5oPSMmj7/Lr5hT+fvxVniknEr6ak3gsRojPWc1DuIO4aPnmyKCNQ1nBmztO0mZZ1pAh5lkaYgzLsol+ueUU2mBbkQijCt4ysqfzASIjuKonRKFZOKbDUW9vJgYjYsIz0Pm/rWsu8XDR0lmhPG11lA/83Tunm1gm/tVxauUP5SMwqBREmDdwQ+WfpHNaBWfDk2WqrqzsyWu6+MCVve+fjREaJCuT9siA/hGEd3D5n6gYLZ/w5+amwon/RPQ7TGLBNssR8gRHBmZkZuPkyvDAfQj5AQY/VHyAy+l1Y+DnaPwV7Is7SrNajdPM3V58IHL63AUxcpMnxjIlc+l9v5avHy/5cyqckA+/krNz7AYm+E/abCIAHrf5yOjbZcsywZ7oDGgqnesSCni0Izijs+fjyrNruPyoRvcLPHfTbfjZM3Lq4zu0HuRmav4J7g0ffAbxpdd92obndvGBr9hCWwKjAzDg6COvRPzHgZjcwH0jGdHV8rlxMvEnwQn4cNUa26ysK6CCHbXZNfFAvqNtGTxDOjOK01j57eKDMYQs3BAoDAF0GnbxQWF1ZsU1BGYhoHoHn5WyRTIEDIHNETAF37wKrACGwHoImIKvh62lbAhsjoAp+OZVYAUwBNZDwP2b7D2+vvVzeguau8iiH27PhoAhsCEC0M/3yP4iVAQqOP+3Dp2qkvtyvpBcRjcEakGgW8jjE/j/eVxgzinYqVAAAAAASUVORK5CYII=",
      "text/latex": [
       "$\\displaystyle \\left[\\begin{matrix}1.0 & 0 & 0 & 0\\\\0.29 & 1.0 & 0 & 0\\\\0.71 & 0.12 & 1.0 & 0\\\\0.71 & -0.44 & -0.46 & 1.0\\end{matrix}\\right]$"
      ],
      "text/plain": [
       "⎡1.0     0      0     0 ⎤\n",
       "⎢                       ⎥\n",
       "⎢0.29   1.0     0     0 ⎥\n",
       "⎢                       ⎥\n",
       "⎢0.71  0.12    1.0    0 ⎥\n",
       "⎢                       ⎥\n",
       "⎣0.71  -0.44  -0.46  1.0⎦"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Construction of the matrix U:\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAOcAAABkCAYAAAB90CdWAAAACXBIWXMAAA7EAAAOxAGVKw4bAAATFUlEQVR4Ae2dW7LdNBaGd07luStAVb9zmEEuIyDMAJoRJMwAiifyloIZACNIYAbACBIyg9DvXdXJqZ4A/X/eXi7Zlm3Z1vbtLFVp62pprV9a0pIsa9/57rvv7p9Opz9lY+bXZ8+efRFL8DhHwBGYh4Bk661KuI6VorQ7d4OEH+Qnc2j+CgPudwQcgawIfB8p7TPFfU58KJw/Slqjwqh4pPux7Ev5b3hw70Z83Jd9E/Kh8D2FP5QbxSHM635HYC4C6mc/NctQHFEt4WzmC8Oovj9iy4fDNPw3iv+gGdkMK4+NFP9V2iey3ytuLUH4Q3UjjCag+DEPzs7w78b4GSa4J0fAi+V6oTjDxuKibvDsFto1SmNqZMCLPbIaDuHMacTE3EeK/F02JkjMqN/EHgrjxDTr2udyfyVeLsLwp9zPZGPlku2S5l1ZOAMP9UMX9N2U8b2O8m2Nn156uxLFB1rRL7LfyE8bn+QSxjKA9hrldRyE0CVwSBVOKkcXrhnFFYtZuYXA1RKDgNKfKngvzCc/sy3PMSO3yg4ev5T3jeqftNml57bIz1ScEEJmh0Iwy0IYOAcHTMfhjNalcLgqG2PIedWRAbX0q460MBohiKlIlPtYZZhKGT6zZf8h+BHurG3QHGprH8WjzaQMmI7DuZdeBIck4VRDtWZGxbF+fH6mbfAX1dfUyDCzjc6k78kchR8GVjSYJFU+0kCOwxmUi+BwNwL4YJQaE3WW3c6UtWbKrPjhYKUXyCD6C/VURX8kC0+sOWMzfFW70jfLT0Vkuuehsv4lnpg9v5S1DZ1fFBequa0SHYczJJfEYZJwiixmTdt5bTVcI8IEr290TunwjWJnB6mzejUkkBHOt3JR6fo65lb5mQKI4f5QPFcDrfzvZZ/ItjSmoBLH4QzGxXBIUmuDBjmpwejErBP7OnD4SIqfmWtRI/oRwmrAkB8VG57YoJprFudnLMHi1wQTDai25lRZL2V/DvKMLd7yOw5nJCbhMFo4VRfrFFsrWiP0ubG1puW3UQd1agsGvq7VKRmAusye+OniIYyPtSWvRxBe1N4u4zickbkYDlOEkx2+PoJqjamObrOTjdRhusXFOkiYL6tfNP0mSwfsMkZXK32L/LSITIgI+LD2iT3VOUgFz8ewsrhF2zXGwFBcwMfmcBglnGIE0GmwPkZieKAuxhraZs6cKnKs/mYcM4J1oDCtoEd8vgkjI/6t8RMhMSkKPmI42MNDwuU4nJG6CA6jhFN0mJqTPHOWrcyLbnu2jCqcB/rlMMBYYQ/LmOL/SXXGTr+wJQ7QQ2Zr/AzR25XO+jo2aNIuvGIZwsJxOCN7ERzGCqeNslFhUmNyCuhv2ZrKqDAbDu/kohIXRn7K+pfsk3PMor+cEa5t/Cj8dUkBL5QLA42ye+DHSB7lijd2Y3+XW+28w7Piau1CnKzjsDAOY1+loO4hmK9kW0YNyGiLKvS6lXg+UM6JokdKYwMI91OFh1TISFHzoqBRFlpMQFFn0QY+Vlw18OCX3Tw/c9AQf+xaN7GotYvjcEZ4aRzuBB9bf6LKh9YYc/qBP+sIOAIDCEgGORiDZndnrFo7ULQnOwKOQC4EXDhzIenlOAKZEXDhzAyoF+cI5ELAhTMXkl6OI5AZARfOzIB6cY5ALgRcOHMh6eU4ApkRcOHMDKgX5wjkQsCFMxeSXo4jkBkBF87MgHpxjkAuBFw4E5DUaY37zWyK47zptcXjl31KvMW5u00E1EaD7bkFyseere2lWUzbAWrOzq59aXRFq+hCiOyWQISHMOdJh766sDJSLqCmwTmrO+vibavwkm7QTlbNC8UNnnEucbTrTPjKiPPI3HfbelZxdm6ZOji7zLUn1bllIlc0Ke3ZSd5SOGQTThHMlyhbujS6AFd0IYx0IBPOk/x8HcMH11/I9t2TU5ShHzohBgHk/DHPNC+g5iA/wh47n8ynaNap5V3HiFcGJT5vGn2BdPksA091Zab8DMZ2MXgx0CkOvOkL5OX/d05ywe3fch/IxvAh25ImpT2j9Ih+MFwEhyzCKYK3fMkytKFuIowmiDZjfqs0i4s2RhmZdAG1yq86rhWmOBrzFNRtSWu4CCazpPEPDQhTisAgiNUAx4MqByEHX8q1v+P4WX7+b6YQTPnJB36v5WU2bWFEnoVNUnt20LQYDlcdBIyN5hvIlmqjOD4te6yGoQOsZaDrprQFDaKHcG4DrzGD+lzr1LFMl44TDWgLzGC1y7wUzydjKQLD7M/thM22RNDD9Tf1xISddli7L4iE2WYxHLLMnGIXgmuNXkJgjUR6ygxVPpbPUWei89ioXhSsODoQJlwXnWMm/qrMFn+KY5R9PrHI3I8xQPB96tSBCRy5qa/reQTUBNfUxpAH9iEwD2XDmbuI3NHPYjjMFs6gQfrwZUNgE0b0MlAUqon8sQElSqfyFqq7ErnmEFW19wJq5SdP0sXbyreEQSgmXSANceIH7ShmmI1JLzQnuQRj7W3XQ4LL6kZ0jmpPI1jPLYbDbOEU0dYQXSMqfNmIajwu7gpUOhGCycYNHYk1UKqB/rEXUDMAYLdirA2mXCAd5aHEFGELN7vQIMC5aQohVqTR0UxfMjylPTvpuxQOV5015k2wUTNvqSNKE4BsAvwgy8j3QpZdRlNve0tSvlEXUCs/HZb11SbUN9FhApH7Amk2gn5V+dXmj8JPAFNxlYDKj2DeEC9jS51zaIVf0TOqPRNIvAgOCOc/ysrNTaClluVdLVQP2Kxq64166kohNQ6jO52F/wSxjjuWGjpZ1wXUrO9W74QRhmI08doDDFB7k41wY72OmlxT8xQG149leU31tSzaA4OVbZjFaFDy6qavPTuJE3+5cfinVTZbraUxZCkv1sktbrUGEW2FOiUXVTY0qLWM7liENWr03G9K4NUA10XGjPEYpjEjr8ZzSAh+0W5thOB0GQQoyag81mtgEt3lpT6l13aoFWcq/qq4iI4p7RnFRWVdFAdmzv+VNZsbJWQgEvUt1rg2c66p3jEzoMLGhGiArSKZGSX2bMGbyq0JfVkPWPQJQkq9ufPQBjE+rJ4koRF/DDxcBlfNmPJ3aRBWNu59Wa7hXBuXUe0ZMhD6L4jDf6ye2TNnWRA6t42MVjYusw1rvTUbhLpjnYJGwgwNHFxAHW54nJ86z7ixZ63cPnXfyljSRf2inZqGNmJmjfFSy6s8CNijCB4IbLHzrTT8HESorhlVHIMCGkqX9qGkxczY9mwRthQOWYRTxBYM0zCyhYoolwbhcuJPW9wtG9ESLOgUCdD3lfwI76mk9728DCZhJ+KoFrZS0+RvXUBNGaWhXExR7tm7/q9oZuOmuEBaboGJXGhtXSCtuBYOyos2gHBTRvP9MJtftilEvubAxHNgXdMyFLeGSWpP0Qo2q+KQ7d7akhlmTzolG0C8suh9F6j0RYxoY9Su1DD56UCtg+/K91bxdL5KEBU+KUx+E3LUWTofR9daAljmRZXmoHfnWlbpqxjRRBvR8TDw0moj5WnhUMaBQ8zUBrSyDvJZPYN/xhsr9FJxoi+pPdfAQXWyjmUAuZNNOC8FpJfrCNwmBELhvLpNjDuvjsCeEHDh3FNrOa23CgEXzlvV3M7snhBw4dxTazmttwoBF85b1dzO7J4QcOHcU2s5rbcKAYSTY1u812u+OL5VQDizjsBGEOCkVvGeHeG8luXEBy+k3TgCjsC6CHBgpjiB5Wrtug3htTsCnQi4cHZC4wmOwLoIZDn4bizo6JF9mcLZ2s1cKm30jXWPxk8q/+Kbr084rM49s63zw33lTMGsrI+D8bUzzX31zElTPSzlrC7O/xJunbXuqyOFT+Wh7G+DcggnnzPOJpwihMPeHKIOv0qxC4eTvhUMmFjdezR+hgAtOxKferExyGdvdNhRZgZmDASLfLFS8jnrkvERfLauRdWzxe0bcgc/isii1qqi4iazsEL5GXEhoPl50agGXyPz0fhJwZD2kuVqEWaUFynPhHmmYqbn7PO7sLhL+umrT1Uvnw2asW9Zw1nO0mpuKp9lPiaspuGOpcF6eCiLcKocPseKjXyvFM+3fkznezJH42cJ7Edjpn6B+swgjl3K0E9rdYqOMfWn8smyLnqNSyqjuYST7V/UoaYxdZb0PZmj8bME9lMw+1KCkXx3cA4mVB/f636Aa+XJb7NoipaXyicTE5cP8Dcg96wuuezLpNRzuhs8NMnbqLirjN28Qz0aP10NkjN+CmZ6BnU2qZPmpLVZluhA2BAYNqR6B4oxfCovN0+wrEPw38vPx/rMpuF/9ijYbXLMnCZ4fapBOHJ0U7ONlKPxswSqozBTR70WUaxxTbNagsZaHaqbO3wZIFhjo+q+rmWIB8byiQpsAs8AwEAQW/5Fa8shnNGCG5EfNcJ7Dx6NnyXaI8RscJa6NEESzMmXjA/QVvGpOpg1mbSYMVGjGZT4MyhToxXsNrPVWhUdW2tajTbS8N5zL2aX/KjB0U7+kB2jpbA7mzyS9zRgMmaqr7gjp6esxZNEEyooQsRrDtaj+GNmLJ+8J7b3qdwyj1Dy2uhn+Vn7dtVT1D1bOKlAlsJincLiVlNfCi5H/OyVn7KhH4xgNVvWVMyUj5njntzV+oPqvg/jcpuDEmotaieWtWLL6JkxfR01lpvvK6PnGQSYRblArbMee+CueWa6NmU3i7GZs9oZa2bYaPho/CwBcwpmD0UI994ye4QGgeFiauL5iwe76TDMk8tfvHtUHX0zZF9dg3yqbCYlBqHWzKg4+EP4TTY668olnIDKSNE0jORrXyrdpCklfDR+UniemycFMzo2tmbUWbkfFjWPDZRLGwQmplIycGBa9J2jq98UPk/ihVmWASemJSC8Q/XkOYQgAtiReie3WujKDwG1C4sr9jbuORo/E+C2TY3W6E67yv4tWzv9MhMz+gp2CcOszP+lVEa002+pn42qGxLkzuWTgaY4qkd5ZlQua25ep8SE1rIVbrZ7a2FGJTJ7whwbQI9kOWvb1O0VvX1zNH5SEBfPzAoY1kO0J21HJ6Iz2SuBk/ytS6eVh/hRfUD5ec/JOpT6MKh7rxRvt8cXkbl/VD71hbM0NLQOvivfLD71POr6t7LhRhIC2zlrKq3YMJPrl0oLODeOwGYQCIXzajNUOSGOgCNQQ8CFswaHBxyB7SDgwrmdtnBKHIEaAi6cNTg84AhsBwEXzu20hVPiCNQQcOGsweEBR2A7CLhwbqctnBJHoIaAC2cNDg84AttBwIVzO23hlDgCNQRyHXyvFeoBR2CrCOgEzrVo4/jeS/k5arpZk1U4xax9mcLZWr5b47zi4AHfraJzNH6m4nwwHDjvypneH8VXDBK+JvkgltAVp/yUyaF5+7C6llXxJhcW/0Jxg2fOswmnKuMrBQ66Fx+qyuUQtF8qbc2xU/do7apm4IMMDp7HJg1m1CnfkvLBQEvYhB2zNGlcYl0cdpdLGMvk1WuyCKcq5CQ9n9hUX5DLzwhEmFFq1v2dvRxcIPFo/EyF6Kg4iK9Wf1QcgnSSW/XhFNyUv+9SbISQWTL8CoVJKzYwtKrLtSHE5zetkUNxr2QfizgI2pM5Gj9TsT8iDvTJmGEJFlVLY5mJU37UWdat2JpRGt+Ikl59akcGxXOXUGtwIK1pcgkn6kD4zZrVYyME6XsyR+NnKvaHw0GC0ZoZFcea8PkEkPouxUbQ0R5bgptaz93UjF35VHnKrPhh1/Nbiz8aP1PxvS04iE/UWe6wHbXWVP6hS7G59oT7gpg9v5S1TdLej62VrzKzhVMlmeD1jRApAlwRtbLnaPxMhfO24MCs2dxN7cWsFGhmRdMMY/mtzz9Uvkrw5ef29yeyrRm8WUgutbZZbjNsd9I04/caPho/U9th1zhIQJg12RMJN2xSsOC1SW0tGT6kNBNMZuRmvpfKy721lid8tObPIZyxtaZVYqMvU/pezNH4mYr7bcCBdWHf7NfCTkI15lLsWNm8ckQwUXt7zWzhFLE3ZQ2xkcDiYkT2ErZW4tH4mYrjLcGBHdW+QagGnzBhpuWVYW9/DrAz2aiVUwYoq9fkWHNSAWpBrDKbOceqDb1EL5B4NH6mQnZYHCRATBz02TcjwCF/6qXYXdhZdb0CTqZcwsnL1tii+oHi/VJpkN6nOVq7hq1gauWYmROBw9aMBD12KTaHb8CvaZAJNpNa5TQzXjUjpoRVEYtev1R6CngbfuZo7dqA2pZcUdVTvKO+ti7PbpRhQcqy8oo4PctuLDfLV5MWZSou+aL1XDMnBDEicMqCs4tsAOF+qvAYtUGPbMYcjZ+pwB4VB/olghk9MaR+a69KXncBpzzMjracY9eXmbK6FFthTgMhE+TDsMxLlolsN74XVfuPI+AIzEJAglzd+J5FrZ1FjT/sCDgCUQRcOKOweKQjsD4CLpzrt4FT4AhEEXDhjMLikY7A+gi4cK7fBk6BIxBFIHyV8lY7Rc1M/Ic9H9y6cQQcgcwISLb4/8/rrmIRTo4RdX0BPnjEqKtgj3cEHIFBBKoDCrGc/wejVe5wER60mQAAAABJRU5ErkJggg==",
      "text/latex": [
       "$\\displaystyle \\left[\\begin{matrix}7.0 & 5.0 & 6.0 & 6.0\\\\0 & 3.57 & 6.29 & 5.29\\\\0 & 0 & -1.04 & 3.08\\\\0 & 0 & 0 & 7.46\\end{matrix}\\right]$"
      ],
      "text/plain": [
       "⎡7.0  5.0    6.0   6.0 ⎤\n",
       "⎢                      ⎥\n",
       "⎢ 0   3.57  6.29   5.29⎥\n",
       "⎢                      ⎥\n",
       "⎢ 0    0    -1.04  3.08⎥\n",
       "⎢                      ⎥\n",
       "⎣ 0    0      0    7.46⎦"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "piv vector:\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAABoAAABkCAYAAACYYiB/AAAACXBIWXMAAA7EAAAOxAGVKw4bAAADYUlEQVRoBe2awW3cMBBF5SDnYONDCpA7MOIONh3ELdglBDntXp0O7BbsDuwS4nRgF5BDEKSC/CdrBIqkpKGwCAKDA9AkR+T/5OeIy+X6aLfbnTZN86iUs7v9fn+eexD71O5Jvjb2U9ezo7fBg28q0zi057CyUL7KPP8k32f8IdG1mEuAR7jqezNyqCIfroQI56SpE7J86Rt8VP6Luvw/et9sFs5osmFPwoyRojOVkeoRn9JD757M3kw+GT8A9DJ0CZzZ/Va6Df1TZS/RVgBPAt9EQMxkI3822sK2XiIAnwXIDHIWDyBp412jqXeJd7DRABYDwjujZIQChwTJLBKTNqFjNZFACAJ2Dl70RVtFJPBrIbNmU5ImxMVEAr8QyrHy4Z1KUDOOIiKBs52chDNRuSVlsEcuN5HAWPwz5fHiQ852NGve8GbELP6DiFif0LbyLQaEi0io90qQsT6xLb5DdHARacQnMXpp3b1GpcBx+0oUK+KuV+ncUsUNq3SxIu66awsCTdtQq8x27nqAbLxRt5Vs9QBJDA3mlW7oYAVF4f97gHzXj9JyG/Rkrtl4D5AfDKRYOpGsPkD+6Vktt0EkuUhKD5A/DcQ9o37x6wHSlKsHyEGJ4oI76oqRow6VKBLEX0U6rs6451n8euiHHVpy49LdIUHUKrEbHysd2jgCdF9FX18wlB4g7c6O2ywkv9KuvngpyHq4iAQGMNeaRtSozGfTvfJzpTvA5sy7RnyqXvTghmcz+WqOudxLxF0Cl4KkzkQ6lM03l3ulY/TvQ6Bgdl34hs9yZe+MRn1FwvvRXeiqnNx3jxr3FdeMrKNAOTRCcqaEnN+VXFZKBDipESlRx733QaMO7JGJgJAmIG5V3oweZiquNRLQKSnT36RDzllzEQmBn32QaXHkU2zeNUIiLgXJQ+O7LGYv70st89dLZF+SBwiREgzM8DIzgKGdFVxEArpR2iqFL2crENcvLZC5iGgoEuRZlIi2OfMGQ65vka8SFckVNq7ShWoUlat0RXKFjUu2oFYd7VzHZkq9HiBDNbPleoBMZHFHHT314VcPkIOErr0OyXrZho59oR4gY0WSej1AJpK4oi7ptcJRiVaI9tLln0kX7t78tBaP+CD/MQgoRHYDGZNQL/nHvtx/DA6YfwHQLFvPoi3s5AAAAABJRU5ErkJggg==",
      "text/latex": [
       "$\\displaystyle \\left[\\begin{matrix}2\\\\2\\\\3\\\\3\\end{matrix}\\right]$"
      ],
      "text/plain": [
       "⎡2⎤\n",
       "⎢ ⎥\n",
       "⎢2⎥\n",
       "⎢ ⎥\n",
       "⎢3⎥\n",
       "⎢ ⎥\n",
       "⎣3⎦"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "LU, piv = scp.linalg.lu_factor(A)\n",
    "print(\"Construction of the matrix L:\")\n",
    "display_matrix((np.tril(LU, k=-1) + np.eye(4)).round(2))\n",
    "print(\"Construction of the matrix U:\")\n",
    "display_matrix(np.triu(LU).round(2))\n",
    "print(\"piv vector:\")\n",
    "display_matrix(piv)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Product of L@U:\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAALgAAABkCAYAAAA1z/qTAAAACXBIWXMAAA7EAAAOxAGVKw4bAAAPiklEQVR4Ae2dX5LcthHGZ1N6TtlKVd69voGkPYHlGzjxCWzfIC4/SW8q+waWTxDbN5B1Av25gZz3VEWeygWS74dlUyAIDsEZckDOdldxAQIY4OOHRqNJgtirJ0+ePNjtdm905OTXp0+f/i2X4WnOwBoYkH6+E47rHBblXd2LMn5QnMKx/B6feNwZWCED32cwfa60L0iPFfxHaXxWoZXOCHms42fF9/xwSyLMD3S8jTHr/COd31eYvea4rMfXy4D673mKTmkk9RQ8LRef48b8yNH8OM4jvlf6x2lieq4yNtr+o7xPdXyvtHMo2Eu1g0KbkhNHHt4G438rYg/govYN7D+VZtdjadkw+u25eW/xRBgs7Sz4YwtuDefCGyX+piOnjFj2b3M/itN0gfj5zxT+SrpClOyNws915Oql2FzyvqmIgUpbYADLvkk/GKhcNexqm9nzFx3fKk4f7BRyzoGROCgqWw07wNR+VfylCg5Q/JqONODJC0rbyYxOlP+1Tj+KyymO1ed3zAy9uqOfzxF9q7aOulnW72pjR5GxdkG5GzIwDqNGYQXYgVsV/58awsaCVwMFcDG+GciLk1Gu3HRKvY9Vh7kM8W/WEq+GXbzgRzLrdPxMpTPrlRiFatjpvDXgL1JwAe1ZaKXhTz/jQgoEN8bchLi4WSHy1yo1sWM8mOmKXKkMgTWxA6c6/nsZUkaTRDh+FU8mSnzvEut8f7TREwsIa3A1VM1fdIAfHzw3q7QtKb829kcC87twYMW/1GE3ib8oLXZZWswWWQF2oFTHf5SCCzjW256IcCGHxJT3kBUqUaRDbYzlUX/7iFOdj4K/U8hUf0hRamM3Xh4JZ2tMFP9Dx1c6ejNrRERt7ECpjr/IRYlI24lUlAO/+ZBixD8piWNVFxNhRZH31oDiuEbg5wb3VFkEuzCacjBTdnxwAf5Zx09RmWOvYRHsgImwVcU/WcGFHb/KfGeuZUxyvrf9xqwMU++5hWu4VkcwYIdkDdhzXPPojwGACzAka8AOtqr4j1Fw7uwPkdchXApkltMsUpxvaTkS4nJHx9X+Cx0oxJAYhl5+TexR28ZfD58SBgdn9Pvc9Vnakrwbbgur4J+k4CINYiD1EOjcheAO5DrDLPic7k7aPlYO3KmEtnVNb9OM5LwmdtrOYTeIYwpaEzsYq+OfpOACbFNisQVveoKH/fbbJikED/WXlzBTB0xcx1j8uerPvfHjERodMCY1sXOPkDMM8MbjwzH8NbHDa3X8UxXcrElWIUU4byv/p6PjEuicm6T3CnFvgihOXX/X8dVtymJ/WT/TuZnU+T+a1tq3m+DRsSrswsNTkt8Utk+swKm0Dm+k6VgVdvgVpur4pz4mZDpHuXkD2RNdEFaFafN1L/N2YRNvPm+Ux00l4Wc6H3MRMlWVJ4FHB+2akuOaMAN9orR2oBLXsSrsXKUw8QQoxd/hTfmrxL4G/FfRBw+fiqgxnw7MLs7AqhmQHvNSj5n7aqqLsuoLc3DOQMqAK3jKiJ9fFAOu4BfVnX4xKQOu4Ckjfn5RDLiCX1R3+sWkDLiCp4z4+UUx4Ap+Ud3pF5My4AqeMuLnF8WAK/hFdadfTMrAnVBwvdF6kF640li/cZ2m+/l8DMCvjq/her5ap9VUvBalUQb7bIqVgaznYK+O4rUkKmuLhliL4hv/FPRVoxzfRUVRltFvMqPyu4q8Y1hYA1Rtw6giBRdBWDpAtlsVNKTZxj1jyzZ3Ks8KQ9/4RyRMlN7WHOISBWcGOvRNZmimMu8sqEM3cmucWK5sBjNgzf05FX+RgqthLC+fqrWihrHeLGphzfHBbduacp0OURor4OggRng7cBRfQja58U/DW2fpcUMOS4xf6jio4CvgfScMvb5VWnANFS6Ov9QHZ7TxFXrqSzE6S3xZ1l3nXBmW3fIBc1qvklcjNbHjxvUUZAIzNbEDM7usWum9WWngmk7GX6rgYZqRIu4HgIwpKAMEnz0Vm7rIX6vUxI6CfCHe+a405pgZ1da3H+KtJvadMPcstNLA/uwQ6CjvZPz3osoGowLFSMpJeDqh/Jx1DuWVF3dMrg7S+AhhURGOzW38I8z8AwKUhC+h2AsFnxWrjsL3lEfprSh/Fby3gBQRJlyTs24YVWrBY5whLrAoN4DHbhRMefe9Sj4klHTGh9LTY9TPxj8/6AAvBzfIWIhDUh27MGJcbF8UrB+YBw1KdDHVsUdYLAp+jhKZBf/RCi6E3FxiYfjPEKfKYhvQAEwYN7fxjxEq7FhvjAOWG1cRo8L9UPt9q86PlUV5j0EJL7i53+Ia5pJR/Cj4n5vWLBxtXCDx//jWcch1iet4H58kcRulPBc/t+D/r3rjH/GLW8Xg5IkVfHPDaZyP7Wy1Nt7PuWHUX02ZUPBJ0pB+vyF79LcqZ65Jzg2xNLvZHK1vagG1v8mNf5rrZDrvuIC6HnxvrDncDbpYtXkXtlSYcQ4Nuk75ufCj4P9taraw01B8okYBycfJZkV2io9ZQaqwqTWujrhZ8DmnrbSNR0qwgRTnhbaF/22cmIlXwS5cYOYR7D7FpDQMAopu/KVF7LwKdmvcwuZarnXeuxYrMxAei//fVl+xBRfIB/rRjcKORVFaycjEX0fRUmEDG9/4J2VF5+IZZeBlGIqREwbAmGGoyXuM2fq+2II3Pz4Zf5GCNyTTGBYlbKRjodK+UTyMTIXkr20DmoA3ZlsYN7HxjzAzU4bX8gl+fHNcr+DarZT3GDKDEQl6chv98HdJ/EXPwQXlhQ4sCcSm0k7xAorFgfTXaSGdY615g3WjkJtKws4GNjqfXcCjI904B0uy+o1/hJtdrXgtzw1lbP06i62Utzrek45ER1DuV0l6OF0Sv2/8k2Pc0zbNgAYMhpiZ+6rIRdn01Tr4O82AK/id7v7Lv3hX8Mvv4zt9ha7gd7r7L//iXcEvv4/v9BW6gt/p7r/8i0fBeW7NQpj4OevlX7lf4SUzwBve8IklCn6tg9WBY+saVMTFGdgEAyxCC188uYuyif5ykMcyUKzgeivEYquOKI21J8wALs7AKhkoXYsC+JcotELWFSC2gIY1JkWi39vnSqxFYU0za0TCgqGiCmYqpDYZrCwSC35aSbWOvYSlfhnxhp58F+Vw3llLE+Vlo6dwX2zB1TI3oXsdKAcgWY/8UI2TNioqx/4erxTydQqfubHslhVxNWYAVkYW33M4drF1vGDE6HM7MCoYF5ZZj8qp3E9RcNZtf6zjSgcfPQC4VLlZ/II7w6AI0vyW83Az0CQvHqhdWypb1JbKO/YipvqFGu4wbKmwQjK26ml+OJ+D+ykKngVRmMi6ZnNt4p+wfJIPUZkRFhe1w+zDoCwamA0gx358z+CGVt246FwKzmOb3HN287/JP4d8KSW3LRhK23PspUz1y2HAqm5cNOUmcyflCNO1QPO5/rUONtPMWWZl3YryS6xzsT9s9U4NhQPXZJI75NinstwtL/6qb1w0xYKjqFvdPIfBaF+9dHvh8JkNvEMuTckAPtzKgVwpyWaxc1nCj4tnsyZP0ZgRDxpF5SOzcF+s4AK62c1zRBZ37UZyYG/GP8xmS8qWsaPgPC3BQFTZuKhYwQd6EB96bNuInO9t1dko5bn4IiKCw+dLR1bu2I8kjp813FfduKhIwQV0k5vnCDfTO48n7WZ2Unfpd+aa5NwQSzuq7jEgW8YeXVv1jYtKbzLZ1yJnzYIFVmeM+VTHbuAScXVUFAVnLxde7MTyQCfMPKTz1X2610tc1rHHbBTGxSkGAONiRqL9pdLgnHcgNoO3eUnkZO5LFfy5AOWUgBsGQIwJimSv6eOyvOZfbOMfYQZbD5/S/yBdITdAY+LYxxjK5Itbbuo5MCS5WY4B0OubpKqTuS9yUdToljfPSTgLp8G6xBnqBKzN2jYtiiFafEvYMSBVNy4q3heFkSiwZsWZWnBZeq/rVe6d0rGOnYVMOqdjsOJ7HdxU3ugYfY6uMrOJMPAcnOtg5kGYJlkfE7aAVujYAy3z/RGnuIO8lo9d3N5iqzm5V13hwYLCq2IFn++SvSZnYFkGYgUvdVGWReS1OwMLMeAKvhCxXu06GHAFX0c/OIqFGHAFX4hYr3YdDLiCr6MfHMVCDLiCL0SsV7sOBlzB19EPjmIhBlzBFyLWq10HA67g6+gHR7EQA8UKrrdDvHLtiNJYv3HdSfQTZ2BFDJSuJgTyZjf+aQahraOxpb+soxlb5tt2lcraakjW0Zxt06ItY4c84WcNUrxFBOe9tSiUHZJTuC+24GqcxTIslMKSA5KFSqvf+KdREFZD8ukXR1iiK+xvFLdFVzodFpVjbw8WZTEozrZpkdpidtwk9ohN3/gnImOJKJY3XdmINWewst74oEjJam78s2Xsu4Y7jEMqvvFPysgJ51jpdyKbWScWFtuX3EOwpjnnyrxS+tKbFm0ZO1z7xj+wIClyFW6LTv6LIvOJFBY7J6nip2XAFq9ltnz7SsWxGyP9ECPgG/+IBD6gWESk2EOfpYWnQsrPWeeAJWP1cxgde44VpYk/3/in4WbMig5QeFyyiEe5uYGzJytDFZnyDll/fufYh9hTurjGwNieNNxTMOMNGhXlmczCffFTFAHd8sY/RpqF3FxiXcKnapZ4ZLj0xj8prE1hF8e+8Y96kGfLZxERzneZ+ORDrkuMI+d7W75ZGMdujCShOOYJlG/8IxLshi2haN7ThvD7Cou29FU5c01yboilOfbhbsIl6biB4pT3Jzxdgb/BG/S5uC91UXj7Zx2qaCvBignMmE/Fkwx83lTMCo7tj5H+bvK5MDJVsnF/a7kVH9t2jnYc+2S22zeYPIbdpz9XGkbhLBv/lCo4G/8w6lJhBJYoJ34jgySVRTf+scaE/YHi7HDVsSZKQ+kPuSFU4dhhYaKIaxR7rzBn2KgNgzmmOydzX7RtRAOS19TtG0HF2W+bNQafKB5GqUJAs2sUu1WhvK3onD1HqIORu1NI2X/p+EzxsRmAnxwlqhuCX+jIkflY+WHgNngc+1Es538kTjGAuCn0cdARSipub4dtP5pZ9aapnyUOV/docExUkJsy1hRwg4bgWmD5WuUmUfmMWKaf15wngsJTx41CbswIF1Vu1Y+g3Cg5pKbSDizHnlJz+rk4ZQMoXsv/pDCeKTuLrZbkvsiCn36pXoMzcD4GNGAwZsGCl/rg50PnLTkDMzLgCj4jmV7V+hhwBV9fnziiGRlwBZ+RTK9qfQy4gq+vTxzRjAzEjwn5KCCtmgVJ7Zu/NNPPnYHaDEg/eb9yPYQDBee5dfsCJyl4lnUWSZt+6gxMYYAXSYPyf+t0M7KuqVpQAAAAAElFTkSuQmCC",
      "text/latex": [
       "$\\displaystyle \\left[\\begin{matrix}7.0 & 5.0 & 6.0 & 6.0\\\\2.0 & 5.0 & 8.0 & 7.0\\\\5.0 & 4.0 & 4.0 & 8.0\\\\5.0 & 2.0 & 2.0 & 8.0\\end{matrix}\\right]$"
      ],
      "text/plain": [
       "⎡7.0  5.0  6.0  6.0⎤\n",
       "⎢                  ⎥\n",
       "⎢2.0  5.0  8.0  7.0⎥\n",
       "⎢                  ⎥\n",
       "⎢5.0  4.0  4.0  8.0⎥\n",
       "⎢                  ⎥\n",
       "⎣5.0  2.0  2.0  8.0⎦"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The matrix A after the permutations indicated by piv:\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAHgAAABkCAYAAABNcPQyAAAACXBIWXMAAA7EAAAOxAGVKw4bAAAKe0lEQVR4Ae2cX47VNhTG71R9ruhU6nsvO6CwAoYd0LICYAdFfYI3BDuArqDADoAVINgBLKBSAXUF/X4hjhwnmevc+NiZjC35JnYSfz7n878cn5uThw8fXtvtdu8Vx8KrR48e/TZ2oeatQwPi56Nqsh+rja6dfO9deKpzbvbDJz9Rz1epgScjtbqlvNvk+wQ/E+OjhCqfFnKm+ELnX3kwZxDmNcUPPqbSV5Q+1XG0zv69Wz6X/M9D+ZRH1oDg8D4/zTD+jNg+7F/j/KvyfwwzE6bfqnwIdSRzTvj128H+V/hhT/lbea4+phVYgu334PMqeUMX3yiO9RZ69oPzHk5w7XNbBg2NOrxSfCzBzUcTYTB6vVR8oHN0sNORNPEqaauQAjuWYIRiXO+FtgJcQ+GW4YMwSi32IJLe2pDbCskIMtbYU+tgMXYswe8mav6koOInqpQuW7IxjzFq3PRLVf6gsfvXU5ynwv4upjICG/RQ5TEnPY55/gLfc191Z31hPhWM6CgJdmwP7uFLYOYlVrbWc2+HK6x7SjA0/qQIPnOw9SLnunA+CYdefEfxX0Xm3ZfK84dsZSUPSbCPIlii0HvDVWVyCb0CIbZ7RZNyIfijjreMFQ0u4bpwusas8y+KdxUHI9u325P8JsGOGqL96koolHumo3UL7mCFBZFfXYbOWeCAz6ubSRCGUzAjVfiu+UKgf3n3JK2DV+5i7NkESxLmhhwryENKow57KYMGZxnGZMW0SwNgGLUMi7GPIZiVpXsvtRSuKVsEvlacspVzj+tpSesiTDdiuONY+SaNKyX2LIIFjDIR6jyhxxSxJI9eAm4YTslQnSwXWkwDY9iuLmM9zF1bekyCPYtg1dgNSdl6sDCfi8Qxi9GZrlmvA5jjx3opJlJenyzxk2DPJdi15pw9GPt3bzGl9B9SMMHUuiUcVslvdOzeGHSODn5XvKtoFlJhz31NYjiE3CnLVnKBJSjvoVjMHMkMzYwgvyjPvKEJgxV8iH9TeZZTQ6PHFNgn3ob/VRVoOackJ78WOK4B8YhRiJHvZO4QPV5izV2tBirBq6UmTcUqwWn0uNpSKsGrpSZNxSrBafS42lIqwaulJk3FKsFp9LjaUirBq6UmTcUqwWn0uNpSLgTBssjgMtMLyruiuO9lbiiBbIr3kHOJWNG26FaZzm2FXSXswfgKm9tkhVPM8b1V8J+eklF4Dp8sGjX290V/NogiuCUXoM5dVOfssLwnT9Fy20wwnYMBQmMvZ5cni+O7cNhowIulC0pDMCOIpU9Wkj8bRBEsySAzFJLei1Eb52zLv62o+F0Rx/dWvjFvErYK3ypaErwTftehUAJBec20pGMUduwcfKay8WIM54PG68CBNjXY1g+OBgMlZxJxakt2MKKcV59YgiGSfdmp/deQ+PMwL9I1lHxbcuMX5svIiOb2p03kEd6ghyoP3MdzAKOGaBU85TnRrG513XyhJYzsju/C5A/wKBpHQ3yhWWTSqyF8QIDyzYLwGJpn/9kgtgcPKi5AyAXUrawH9yTMoPfg+P5UETwiCzymDtMgDBq384umB4Fp3qBHhAKbOCscTbBQWFzRwvkygGkQRnbHdyeQsOm9TE30XKYqGjXrEfKzBGGBedSfDSD4h7aW7niw0gJk/mFOnhq6D5aR4AZel0wd3yUf0wKNizcG5GXB5WQ2+2fDiG7m/tngZ1cGBM8KrdCnrbCznj3mZuEUcXxv68qQ2JuCVB/mXnoz04b5FCEMAqPF5+Zs5g8E/9c+446TRUg4gHDOc614p3PTXiS864r+CtbV75QT4ZvMhyoXTIwZDM+9oDxnbGnq0LuYONHWY69iB/U4B+ofdy26Bwvomh66oWOvRSvv6NblKnHgWMTxXXKiUJzbUe5YoAFYW/DApYETjurBsa9JCMmiCifw8P2Pyd9yodU4vgujs6TpPIvju+RlpMIsiR9014N0ztzM1JHDzZiGROjwvyXjfqMIVlGvFSEZwcJgMkQ6EJSoGDqe05rNHd+FS4PGLMmCyu9BOTYbnArQL+ROWbbcfaPHKIIlHIuKYkH49JSuB+esiLBRcLfmyIkNViv70bb+6Dk4t2AVL40GKsFp9LjaUirBq6UmTcUqwWn0uNpSKsGrpSZNxSrBafS42lIg2L2C+O95q61wrViUBrCwNa+VELxXxDplblcVRg15NMAmSGNxrEN0HoUXQ4kmWBYVNht6QXmbdj7vCXtBE1Gmyla2Ys7noW7bxnZfx+zmy5zYdCDJ/qcnP+lZdvA5BLtFGD3Z7Yfmcj73ZGxO2dnCRlwi5MQeuMiK9FlO93MILuJ8HjIoAd1WYXjJPJ0TW1js3C12uo+eg821FwEgoRk92DojZg0FsNnBW+x0f6EIlsB3pGjnwpqV4ALY7P8udrqfM0TvpFyGDSb6nF9dF1yzL8rQHHqTNNesfyR3dmxhJnG6n9ODIbaU8/le2PhH5XCR6bUXYZbExtHAjVhHOd1HEyxBizmfS0heiZygPQIyJIphS2YcGllvHO10H03whCLpUaZusxKSaaHU0Fwae7HTfRTBUnIR53PhMjxiLSs1NBfBbjtTEqf72EUWvrnO0NHiN4dmg0IEWBkdIBhfbIwLfuB1iZGDfLwuQ19t/95jz4thSx7WOzQuhudeUB7y8u+KRve9iyOJWIJxPh9T4pnKNHP+FiZlD8pX/hfydTTzdiyMzYKSSCMeG71oAAO9KG8QooZoPdU4n/tPC9hZlMyU7OMF500LD/JyJXNho9fGLOkLJr2zLoh2uo/+IDitSQW7XszwwJDNv+4Gw4jyTYKwWGxRD0YOAkPVO+Vb/rOiASqBLUymIjYb/Onx4GaDnmsWhzqeRBPcSFl/LoQGfIJjh+gLIVit5FADleChTjaVUwneFJ1DYSrBQ51sKqcSvCk6h8JUgoc62VROJXhTdA6FqQQPdbKpnErwpugcChNNcGs265WgPHY89r3MmliVBmJ3k6h0Mcf3thE5O7jbusQObrVN2ZFUGJuNDWzRLpA+aIt2N3OcQ7AzeGMAZwsLQ7+543ur4CJfmy+JLd0SLoXjO54Nvb+oSPH0XnZM2PA/+gs0evZQKIbdyncpHN/ZGiz1tfmS2JfG8R3PBdxUpvadmZesQknsy+H4LmKnPEZYC+x03WyhVRj78ji+h91TiodcXs/cyjq8xSydE7ttYM4ffNOO7yFhLK6yfG0+BFY6G7YI3r7je6hgCY1fFnPy1NAdPpIsnRNbWLwlbNvxPWSmFTrb1+Z9/ALYSRzfY02VWI/GVqunKEHCmy10nJKFwXCV+2vzDXxubOGh60nHd12LdnyPJbjIV9cb7epHArOoKvG1+SLYkpdXwsbx3ekgONIAohzfY02Vxb66LmFZLbOwyf61+ZLYktc5vi/62nwUwRKURU2Rr65L0GJfmy+JLX0n+dp8FMESdAfJOvRswuRbB+EW+9p8SWz0KnzWNoveFmLnYGsea/lGGqgEGyl2LcVWgtfChFE9KsFGil1LsZXgtTBhVA9/Fc2megiDQX/RKi4ssKbTakD8fFSJ+6lSIfi815+xzwdMlVXzy2gAm/Vk+B+cXaRdAACE1wAAAABJRU5ErkJggg==",
      "text/latex": [
       "$\\displaystyle \\left[\\begin{matrix}7 & 5 & 6 & 6\\\\2 & 5 & 8 & 7\\\\5 & 4 & 4 & 8\\\\5 & 2 & 2 & 8\\end{matrix}\\right]$"
      ],
      "text/plain": [
       "⎡7  5  6  6⎤\n",
       "⎢          ⎥\n",
       "⎢2  5  8  7⎥\n",
       "⎢          ⎥\n",
       "⎢5  4  4  8⎥\n",
       "⎢          ⎥\n",
       "⎣5  2  2  8⎦"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Permutation of A in order to obtain L@U:\n",
    "temp_A = list(A)\n",
    "for i, j in enumerate(piv):\n",
    "    temp_A[i], temp_A[j] = temp_A[j], temp_A[i]  # swap row i with row j=piv[i]\n",
    "\n",
    "print(\"Product of L@U:\")\n",
    "display_matrix(L @ U)\n",
    "print(\"The matrix A after the permutations indicated by piv:\")\n",
    "display_matrix(np.array(temp_A))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Solution\n",
    "\n",
    "The solution of the linear system of equation can be obtained by feeding the output of lu_factor into the method **lu_solve** (which is a wrapper of the LAPACK function getrf)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The value of x is:\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAACoAAABkCAYAAADwkaJHAAAACXBIWXMAAA7EAAAOxAGVKw4bAAAGSElEQVR4Ae2cW44VNxCGDxHP0YRIWcBhBwOzAoYdcFlBYAdBeZp5Q7ADYAUIdgA7IMwOYAGRMhllA8n/ma4+bh/f+nomokuy7C6Xy3+X3Xa5jmdunJ2dHW82m89KMXp/fn7+MFYxNU/9fJHObUyv6m7c9CpeqoywT1/9h5nLLyL674v3AL4P9JWQLwmsg0t9v+4w9CAerD2gMIukxkyVd0p3VL4qNvAEJG9W+0vs20ovxKsyjm9RT2W3KGVH4rxRulS6qxSdS+InSTr4Dp4rf49Qo/Oz8vtKRbA/JDV7FVJ0pfRQ6anYb72qqqLaPZHgkXIHkkYqMxo8v+K5RFVAS0oq6lk5LiJyn8Q7FWhGLEtLAT0VCqZNSDbk1GdpdqA11hLCW1mUqpwdqAcit0Jcm6EvGeznksASFo3NTcNlQ866mqXZgTbLECBiw2s8+6iSYGcH2vT8Ufk2gsIsSn2WlgLKlsuOFtIdMS48q4f17fMQoDbxzRqtMpYipX+VOm6jnnE4LpU7B4MGKjPsj5R+5blEVXs9SqQYq0C2OL8Tj7n1QbnzfJSz1cL7A8GAsB5OyIlyPh7ye3qO7Viq6lIfoFUOtDrGK9oj8a/ExFcYREOGflBHYxutQMdaMGy/WjS0yNjn1aJjLRi2Xy0aWmTs8//Gon220K2s8qyxDJ4QDvGz2r2adpIdHICosqg6ACQhn6dNcu6ZeAQQzEkBS5Ikh0f1STkvR5yLl8ahQXeRqoBKC5boOBR0KB6OhnlVKsZJsosFILDaF3VoRwdDhGeOD1qyymIBCAB9FSAsGKPwBUIZXjR2yLOzUnH63Aw1xp4FMOWLHiOv+qTzq7rSS6Bi77QA06faOeq3cWUBACRDbivBnkzDMBCp0UCs+DKDgUo5HxGhc77gsWTnsKQegP7Y1FqeFLYKgSNUyJxNTQkTJY/NTas3a6cCEL+YYG+LChxLzS3lxNeLJDkb8tjwGs8+qqQ+gP7T1FqeFFanHHdvK28tqfKWlGz0rWJoAOJP01ttUYE5VqMT5eHHA/jc8NLX6ABE7fKExejso4CGoexT8dwHpZyh/FuJ6AfbrCOVXyuxdT5Q8mP4BCDuNWLZrAqoNHxQAizzM6QLYwjEYQMQAhANKhhAP0/J8hKS6/gLfrtSuXqOlhTNXb8CndrCq0W/a4uyz7JslHaXqY1Uo4+t1y1pzNGtEruNeTIqXhvC83c74foxTT0mtXv9RlsgU8S2QJwPnvnxgHlURZKdPQABMLwfSwBm7hBAaH+SyaGV3CIBCLymJwEos+TvOYDUqd1iAQhcObwfkiN13paNl8lHByCq5qhAYb2ffCDi2ZC75cOvi5RZZvauC4lnZyXq2/smkfbDLhYIJIpdPErlGIC2L9Uzv0tUXMOrLGq9qFPOTYA8UWI6xH5KFLtDBiI3VYov0xco4EgbgWboCTtyzSg7bMgXqCoAUdARr27AYSV+vM1Z5DKuwXHN2qkARNu0agsVkGNS22pXsKFnOkRJ7WzIYy9jPPuoojpgVgGVHIs1w2yKaduHWDXYyUIyi9qaHNa3z7VAsQoBMXKf7jYPpY5GByBqgbp4u49QoPmYsDBxffcCWFzpcDcg1DmRDiIi/uLOUHJTsbWmyocNQGDJBlALCl6MJBcNVvASkjfvK9Y0y6sd+qySJSpXoFNbebXoatGpLTC1vnWOfrcW7eXh+1bSloh/6i4a+PxcWW3mDUAkOsd1M38yIbJjC+QiAYhdjyqp0986jMKD5BcLQLRQ1ClDjifkfNC2Il8YHYAYsjw9FtjsWT6CmTNV7JBnZ6Xkmct09QLaDLnvPJueZK42R8nKXUVxrlcDVYdb6TUPftdFuWQgclOl+DLVQIWHpajvkJdf45vENAEIAeSr7TXkHsLY3LRqs/b4AEQz5JwubeJbJ1W52tmQx4bXeEXdNTsTc5MLBSzwPrFMcfsBPvdLwgsHviyHQvSEZBYtHhqLQAUAJXuKxOcCARcN2msbIQrvmZex7dNjb7h8MNufYFhHDJsNneMJ9GEDEIaMXGD4qGwYCUpgLW4xvlSy5cuCZzQxwnrz/wmG9SYw2SCC6tcAhBnrWud9dqaDvsgKdGrz+ws+V4ND/dfiPxYACqDss6klp7gHo2Qiiu1crer/AP5TPXUImJZvAAAAAElFTkSuQmCC",
      "text/latex": [
       "$\\displaystyle \\left[\\begin{matrix}1.0\\\\2.0\\\\3.0\\\\4.0\\end{matrix}\\right]$"
      ],
      "text/plain": [
       "⎡1.0⎤\n",
       "⎢   ⎥\n",
       "⎢2.0⎥\n",
       "⎢   ⎥\n",
       "⎢3.0⎥\n",
       "⎢   ⎥\n",
       "⎣4.0⎦"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "x = scp.linalg.lu_solve((LU, piv), b)\n",
    "print(\"The value of x is:\")\n",
    "display_matrix(x.round(2))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id='sec2'></a>\n",
    "## Jacobi method\n",
    "\n",
    "This is an iterative method.   \n",
    "The idea is to write the matrix $A = D + R$ as the sum of a diagonal matrix D and a \"reminder\" R (a matrix with only zeros in the diagonal).\n",
    "\n",
    "$$ (D+R) x = b $$\n",
    "\n",
    "$$ Dx = b - Rx$$\n",
    "\n",
    "Using this idea, we can start with a guess solution $x^0$ and then iterate until $||x^{n+1}-x^n|| < \\epsilon$.\n",
    "\n",
    "$$ x^{n+1} = D^{-1} (b - Rx^{n})$$\n",
    "\n",
    "The Jacobi and Gauss–Seidel methods converge if the matrix A is strictly diagonally dominant:\n",
    "\n",
    "$$ |a_{i,i}| > \\sum_{i \\not= j} |a_{i,j}| $$\n",
    "where $a_{i,j}$ are the entries of A.\n",
    "\n",
    "For more info on the Jacobi method have a look at the wiki page [link](https://en.wikipedia.org/wiki/Jacobi_method).\n",
    "\n",
    "#### New example:\n",
    "Since the matrix A in the previous example is not diagonally dominant, let us define a new A, and therefore a new b as well. I like x=[1,2,3,4]."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Matrix A with Rank:  4\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAJcAAABkCAYAAACLmaLOAAAACXBIWXMAAA7EAAAOxAGVKw4bAAALU0lEQVR4Ae2dW5LUNhSGZ1I8pyaTqryn2QGXFQA7CNlBYAekeII3CnYArCCBHQArILADWECqMpnKCvJ/Gsuofel2W7Ilu8+pcutit3X06/eRbOnYp0+ePLlxcnLySVuXvH369On9rh2WZwiIG1+EwqYLCe07vRbseKE4B4fyNUxY3BBoIPC8kSZ5T9svREJyvRTbVkkm1euGts9U2IvSZ4qfr7XOvp5ThsLuVfP8yiOrRa7mcZ1p/Zlu9I22m4pfdh2kfM/of7T/urbnystJ3A8qHzJ5ghFHbl4F+X+lH93L75UmtxRekFa+17naNW+g8ve2d59GoeXqO+ZEBdAYr7VRYSre2c8qn2MZvz1T+LZK899PSt/Tlotg6I0AFDqgGzpeKswu0gM86TnoUpwozgXqcXtfZc8SqOzB7b1LoaHkohHcwF4FP1KcRmqJ9j1Q5plCRywOUPyySr9UsgaPfTPKZ+lQ8o0JRHoY4iF9sVrgSS/xQ7hv6rjKHdTe+/T4bt8BB+6nAbvM+Efl35XSXBEmbQTuKutLBz5YLC7W3p6ifapyclKTC5B8FxTW0neH7DdpIwCJvlYWo71XBOvKLD1vULc4pBIdV13X3867MufIk36uy1ZZP2rDEjDm6rKyc6izVYb06Ouy3fCjFD23lB6QSGm5PHHor/sk1xVIuX+qkV5o446MjcFysZZUukEsLgJ/B6nosiQluYbUHKsxu6ihuFOtSa843TRdETcZpQoDeWZIeLi9SElJrq6xlgfFWzWee5UiEGyjxsM6FCXSCdIzBuvrLovSt0+ZZOQSEN4ydHV9Ps8P7Pv0SZ4vvd5p65s7pTyvW/Kyx5xQujI2ZOYg12ObMWp3/icZuaqz09V0WQJvudg/t/DQt4tATic1YhGDekCRLkybXFdYWyzFi7Su6LtPUpOLcQKN2RSmWXiQedncMUP6lcplCqopDOZzkL2ph0tLRwbwtxU2B/AQbteQo/N8JWSOIZcflHtrVNdDwDCReaEQQJwojtX4VdtvVzmz/zKtsjVwV5pZBqS2EFfJPL/SB2vPhckDU6evD5X3UPEcF6UHo7e9/QF94eDnXKoglUf87fsb5TGGYkwTzo5jpZiovq2QATzhHaWzdD/oqA19PMG4KLAEPysvZ6NJhVreKQbBGG81JRduQ9u7qW+dPg0WC9LXzz7grjWxyCoQEIe4QLC+p2O6xVWAYJWYHgEj1/QYH20JRq6jbfrpK27kmh7joy3ByHW0TT99xY1c02N8tCUYuY626aevuJFreoyPtgQj19E2/fQVP0py6ekxk8RborzFOkJsVaSgxOC5xS6d1SAb5ftZfFZDMGe3BEfO4pxkS8Oy0se7u7H4gLZmjnbwSpLR5KoKX6ojJxcBggVjPjWrk2yhWGIkPLlOFGelC4sU7mur/VKV1yujyaUzLtmRszQn2aKwVNsy+fxAJIJMnkjeYj3WPp+naL/EjLlYerM6R85+qCbdUxqWLPNhOVK9JEkkq+NDkYixXDCZt8f0FXo2VAk7zq2ILQZLtSltu/UKAeX5BaB+XdzeZhtNLhXWt4rT3Ylpf5ZFbntrXB0g/TD9XACstGSwms1JtnQspR+W1XXdiocLQ5XdL6PJ1XVKFQyxwjvIrsNKyINUOMk6q6sQneni8W/0Y4usekqP7FhWOkAsVhNjLP46BJSk5FLBLI0t3pETEoUgKc1SaEiFye9y5ggPnyueHUthAqFcD6Q43SJe6oPvFhnQf1+h5cMqeVigQmmYJTty8kiiCDeuErGUTtwhYunxncDy98lPfgfkihYVxvhlEY6c0rVoJ9kSsJQO3Fy4sXODHL5bpKvcK5Drv+ooH+79U3iAlMBcLsmR85b07bryzqmX6uO6AeJzS0FY4qFOF9iF0z5Y/vYHRFkuFQ67l+bIWaSTbGFY0v0xdiYMhQsTGXTTM3pAr4I3KoRB53vFm88+eItgzrezhI6cTYC802k4tZHVSbZALP18MURyIh3pobBkg510R5NLhSzSkVMglegkWxSWwgjrjoEIjQbG5KBHNeYUK8RM0iEgQppTbDo47Ux9CEQN6PtOavmGAAgYuYwHkyFg5JoMWjuxkcs4MBkCRq7JoLUTQy4mbHmgeGFwGAIJEODpvXtADbk22nhY5ubWFJoYAjEIMKntHr5atxgDo/13JwJGrp3w2M4YBGLmFuty9cif1RFMYvd+PbY+OENE+jHh+jgomjSL3gbN7gf/Sxrdh5v2M2TxE+zoTPogx9RYhffpuOv8o8mlQqnsoK/H7lJgpn00iG8kV6TSbkWlwkE+eKn0HIpbdVy0Y+oYvYfquO/cMeS61MmdB5CUYckK1qs4kW5MpHZ9noX34n/QNje5huKG3tGOqTrHwSLMhuq489zHMObC4WLLIWMnIuXsZEUsjczmpGp0nyw+HG25iq/ZNwU/KvpIDcOaKTxXfGPhhxeuV/r2jwJi0pPxYLRjas6qrJ5caiSW69L1sZLyX8VZZYk1C9+DoGTZIr3vSsODHVNz1uoYusUTNQxjQ+8pTAPRUNkcMQ5pcOmOJw5jWm5I0Nl74BxymizHHgu5sFp0h1gsuhtu6fGwJr9okY68kYfPJ3OB/KENr5zi9QbU1ZNLDcFdF2u/ua1n/TyDe3eXq/C10mcKFyHSle6di2SfY2oR9Vk9uYQy3eCWN0vVSFgxiEUXWZxIxySOqTkrtmpyVVaJd51ytW+J8vwbBUudsI9xTN2qa67E2skFqS5FpE0PwFiurFNAPXqRje7Rjqk7zj/5rlTkCp1QJ1f6wAIYX7XGKCIcYzEeR2DBcsku3OjKeTZXi3RlIM8FMdgxtf7z+MguHXeeNcpvUZVlshph3EKluVWmsWg0f+uvZF6RLkxNMXF9EWiSbeJ6KG46Dlz9zQeqY4FnmbgeqiNKhaL/cdG6j3lGkSs8qcUNARAIyZWqWzRkDYEWAkauFiSWkQoBI1cqJO08LQSMXC1ILCMVAkauVEjaeVoIGLlakFhGKgSMXKmQtPO0EDBytSCxjFQIGLlSIWnnaSFg5GpBYhmpELgWcyI96t/o/36t1C3FmbvL/qXYZp0qPb3fInOg6D3LHF1Tl31p6VqEg7H0AKcoR+LR5KoarJgvxfY1WgVSFufSPp2a+ZWOpTkYRzsSx3SLzhMlBEogYcUutfnVEuHuXHHvXBquO/druMIrM5d+J8KNNWe4vWFdWSefVaQHmPU5Eg/GLIZcLAdZwpdiWQYE4dmc0Jg+bmEnAkkciUd3i1KJq5913n0Nddap9syZ0g89F+1cOjNkFJfEkXg0udRo4SK2sP7unRHaj8UoTqQXFtd16YoXs6CxJKCESxJH4phusYWHlIJY3In5O8jWMbky0E3bIp1Lc2AmrDAe/uLjYuSiPMhgJCWXCmcgD+tzflRKKrRFOi3WubRdm+lzhBc3QAx5RjsSJyOXlHkpRRbxpVjpuijn0umptF2C8OFuMdqROAm5KmXOFRb3qiLptHjn0u2mnyVFN7g1tBGOXJAHORJHk0uFYj5L/lLs4p1LZ6FTVYjak7v8JI7EUeTCKkiR0r8Uy7hh0c6lVbvPEqhNwSuJI3HMo4iNlGAAX+KXYsOG2DLv7BCAWFuu0DmdSyl6iIROqDR0DvGOxHcqsjkdFGcsNtiReDS5KEQbBKPAphx0y9r8c8q0AEny1dOUOnWdS3r6KTNu+RGcdrM4GKtcDAbvjOUtQBdOm6ufgxyJzSk2QM6i8QiIjBgb53EdNeaKV8XOsGYEjFxrbt3MdTNyZW6ANRdv5Fpz62aum5ErcwOsufjwUQQL/5p15eFj39Ka5rGWPjIExI0vqvKmr9qQi2cp3nmheVzOt+41dbF0eQgwB9kr/wPy6H2AJPoLMgAAAABJRU5ErkJggg==",
      "text/latex": [
       "$\\displaystyle \\left[\\begin{matrix}10 & 5 & 2 & 1\\\\2 & 15 & 2 & 3\\\\1 & 8 & 13 & 1\\\\2 & 3 & 1 & 8\\end{matrix}\\right]$"
      ],
      "text/plain": [
       "⎡10  5   2   1⎤\n",
       "⎢             ⎥\n",
       "⎢2   15  2   3⎥\n",
       "⎢             ⎥\n",
       "⎢1   8   13  1⎥\n",
       "⎢             ⎥\n",
       "⎣2   3   1   8⎦"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "b:\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAACUAAABkCAYAAAABmvnKAAAACXBIWXMAAA7EAAAOxAGVKw4bAAAGpUlEQVRoBe2b3Y0cNwzH10Geg3MC5D2bDs6+CnzuII4rcNyBjTzdvRl2B04qiO0O4lTgjw7sAgLEPqSC/H+CqONoNCPO3t4mAYaAIImiSA5JfXBm98bZ2dnxZrN5p9KCV+fn5/daA1fBiecHzd+2eGjsxpdu4JnaEHv46Dt7bD9t8Lor3A/gvVLPpeV1KTHQQXJ+GSDUEQ7USCmQk6BJmPthJjhSTf+p8K8zrlTCmSX+EvL7TBd+4C8Kp5mGhKDEY9VWUO65yu/Cpaez6eoTn28yLSHxONM1Y8jm+TqklCb8RKkUMAv9bAw1Dt2R6lcOd6E2fR4iBFGl3osbzCkJJLi0DaealQptDW+EONWco3qg1feB3hpPODHDKjc9gXDmNm+BU9GMglg4iyfGixU9P9+OWsrP2UghmBPMD9VOSqiOWOHrAaOJTshSNleC2WhR6EQFN71VMTCBLbcaTUTxwT5lEydrKYUiKWbUxn3vVN9T6bokM/1mkrkb2Ml9zM+KYJWXamOBT+AnwKzIvtWFkFISekxpcDP3sbLMbS0XGc4CvsHqEhVSSuRsiLjKmF9yGLZYpdshKvXMUra3NUguUVGlsAI3BmoPt3PHhL1U33Ce7pY67xvzPU1pR5VKR0WZpYYEEOhYjm0hKaua7eFTHlMz0UHzo8oD+hEIbQkIUyFu/EaJm+4KZ1YyeViFg5ptg8CmvqN+WrVqdyGkFFyy8FqBkQDRYTW7TYzGI4io+yK89kazKhU15Wqp1VJRC0Tp1pjau6W0U4+uLsKRuWyjwqJ04WNGDP9ACdV2htEGOOsGILrrT0azRG6WnGtYDIW4At+SAuAKqH/lZHSJpbgPkddNgsabyajwPAA3DF5idGHfqw+lzb1e+KJkdN9KnUoT3FyD3c0Z78IS923MPeJKqsSqeyJcsoxqC/w5oXZXn6PZLLEUQl9I+DMVrscUkgl7ehM4CPxKekTxuFISztW3CFQbl3AT9VfkSodRFwt3AUt9lams7k5yBCi2lYK4shVLRmpWnEtGvzXikPsklJdj7D9TwM5uVmy5yHAW8FN8Eh6l/s4UVrcmkMsZYz+eLCCFUrBrAHditRrMUnOJx582KWQpEZNi8e6yBoLcCzpoMsqb40FAq/8oa1h2eeEOmox+lEASTFMMdxDY3wlnsZR1TAf0wZJRgrSbZGYlu3T2BK06GlOtudeGW5WKmna11P/eUrbU5w7U6ENehY6TIW0lxNRWhU3Rzic1/xXgyEqb8xroUfsvuqPDVMeIJZom4zfh7OqScI6GS93iL6NhpSSI2ONqwtfRdF1RTZ9SrjXCcRkkoUjfa1RzD+Muz3U6fMnTnBAgHKv4+xMCiyCNNZNR0VgyGhIUspSE8SL/WOWO5yp8nfHOJaOPRO+vzZ7VoB1dfewfF2Ja350GzNRhWbf2O7Mm410IWUpcuKNz0cNa91UsgPmsZvGFK3sQ2gujSpnA21KCJDSB2p9VHqgQMyZwzprGJ3NoV133SaAx4ptf/dH6hdj+6mjaUi6x4WT0csp8y+LCU7H8URr3tmLJaM2KuL0LEUuZO6xuMSVLtnGUrMFwrQeracPvEghmYzxiIoQJg45NtgazlN/japrS71oqU3J6t4TxvpOtwoSxweLKGqDb75dRCWV1vVZdzj21sdzgi6dwh0tGeWwJ5OyqE9LWF0+scphkNCtW9ij6LZDiF8KvyWjLOHvHRVff3gXPMVyVmrOOH1st5a0x114tNWcdP/aftFT0OuwfJLV1nByrwc+UBkeK+lvweQKHNn3OQrtJ5KHpamelxJJrSp0ZowTJqim1UZv0jC8W4R8U7uQ+CbB36JI3gNDPeQczGp3FSkkh3MZNgFIDlhuMib5FV88b9Hdx330JGrjIOApP3ER+zmtTmvUiS0kobrOvDk2GHin6U/UHP+f141PtsKUkgFXEfdyShCmeG9HgYhQ6UcGlb1XCEFZKHFn+3ZsnkkWHImllqs3qW/Rz3pD7xJhVFXabaAtoLkkHwW4/5y1jU42uUmKK23iFE3Kb6HBdDeY+XNqFiPtQ6kTC2Cw9IJzMGDxvZHAtafxG7Zsqi7cC5gJdpcScZT46IoT/DF51+QipPoqAqxWyBHXER/Qj6Co1mnGJOFKT4mG0EKQggQ4dC6VW1s8t7cVKibFP4U/Vx338JYUfUSz5OW9Rom7solQ5bGtm9KVY090t2ilcd/VNTbxO/KpU1LqrpaKW8qvvg1ZOPe/g/4REAZTiTJta5t3zDiY7QHkj2Jr7D+gIUg7iDgy8AAAAAElFTkSuQmCC",
      "text/latex": [
       "$\\displaystyle \\left[\\begin{matrix}30\\\\50\\\\60\\\\43\\end{matrix}\\right]$"
      ],
      "text/plain": [
       "⎡30⎤\n",
       "⎢  ⎥\n",
       "⎢50⎥\n",
       "⎢  ⎥\n",
       "⎢60⎥\n",
       "⎢  ⎥\n",
       "⎣43⎦"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "D:\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAANcAAABkCAYAAAAVI6VuAAAACXBIWXMAAA7EAAAOxAGVKw4bAAALgElEQVR4Ae1dW5LUNhTtSc13ioSq/KfZwRBWkGEHkB0AOwiVL/hLwQ6AFRDYAWQFA7MDWECqIFPZQHKOx7ejdsu2/LhuXeuqSiNZD/ueI13r0fKdkydPnpxtNpuP8DH39unTp/djGZ7mDJTOAHTjEzjYxnhA3slpkPEccRYO3efwwuPOgDOwx8Czvavri7sI7jEaKtcLaJsr0zVB/tcZ6GUA+vKyWQhpTDpQrma56DUqcxr5Bv424lexQkgXjf6C/Fvwz5CWrLhT68dk0k6zKPNQTtaIURPTNykEQ4Ab8G/gX6D8K/joPJP3Qhmu3y4QPobnVPMx/DvEW+sgf+em1t/daMGIRZmH0rNGjNqYUpXrCoLch3+ERnnd1jDIf4g8KuJbKYM4RzdeUzE73dT6nTdXyrQo81Aq1ohxCUxJyjWgMbizeBkpf4G0cwC6EckLk6bWD++1VNyizEO5WSNGdUxzK9c5Wu1rpOVkvcX8Lje1fte9tfIsyjyUizViVMc0m3IljEps0O/bWnVq/bb7aqZblHkoH2vEuBSm2ZQLjSaKE91BrBu1a1o4tf7QfjNHeYsyD8W9RoyLYJpTuVIa7WZKoY4yU+t33Foty6LMQ8lYI8bJmOZUrthaSxpJ3hT83avNTa3fdl/NdIsyD+VjjRgXwTSbcmEeK9PB2NRP0mRj46CBp9Y/uOECCRZlHkrLGjEuhWk25aob7T3CbaQBZeRifpebWr/r3lp5FmUeysUaMapjmlu5eCzqp0jL3UbaZfDGiBSpkqbWb7uvZrpFmYfysUaM6pjGKJcs9GQ02jUUlIcHGb8irA4uMgNxTgl/gX/Aazqmwf8Lv/epC66T6l/fJY+/FmUeytwaMS6B6TSVaAhDTafjj290PGvINRTPDYangzlK8aDuHYTcwGD4M64vEVYOcR6nYt0PdVIY9NYPC2cStyjzUOrWiFEV00nwseStusMPJd3LOwPOQM0AdIjna/n51smYaaET6Qw4AwkMuHIlkORFnIExDLhyjWHN6zgDCQy4ciWQ5EWcgTEMuHKNYc3rOAMJDLhyJZDkRZyBMQy4co1hzes4AwkMuHIlkORFnIExDLhyjWHN6zgDCQy4ciWQ1CyCX9/PImk8L7ltpvt1uQwkny0cSxE63CQDoWOfO6VerTw8S9lm+PRPlOGBZDkvKd+r8axakjPKi7m27GsMzXZQHbkg+CQDoX3EzJlPZYFPMnyK5/JL1it4jmBULNplbFNEZO07S7yI5BZlFtnbQm1MaiMXBI8aCEW6GAilwfpsHOSistCW3QbxXxEcTP2YVzt+m1aVlYTUEPVM8UJcFmXua48lMGmOXOx8Mm0KsV7g4hzgZCoV5pUQt8iLRZn7+pI6Jk3l4ndfnD41ndjRkO/Cmvlrv7bIi0WZ+/qROqbTPgnG5CeOSgdfMo951rHqyLQCz78Jv4X/HWmxkXonokVeLMq8I7wlshQmrZFLFIfrmDZneVpI2f9AIz2H539xof+ION+GXc4iLxZl7moD5i2CSUu5+sAxn298kw5KdBd+9+JAnFNdWhPq/U8uCYAt8mJR5r6mmIyJyvVt/RQJ+x6akh9ba0k9eWt0GQiVspZCKtgWisYpYpuzyItFmdv4l3RNTD/IQ1RGruCtHpv6SZpsbIgsJkJgo0GePatVDcEFXyO52tKW0S5WRtKy4mWNbbkUJirXP3UvkPCgU4xMUDe6OFKuqdVol1EUIbxXNSKj4S7DxEjcIi8WZY5Qv5ekhekveYrKyFXfXN3oooBYOHwJBeL/eW46bmawwfqcRV4synz0dlBTLnRAcwY+g9aQxaysD4OsymzW3sYFsPJEBx1/mKwc0tzwqZCRYbhE/1T5nSvg8jbinQZCg7JHj4JwvqHpZEv9wPApynyGJyZRMCogF8g/Ik3WVBvG4bl+csOnICFTp9o/3Shopq3uYtlkAC9UNwpqs+lcaksMqK25LJHgsjoDGgy4cmmw6vd0BsCAK5d3A2dAiQFXLiVi/bbOgCuX9wFnQIkBKhd/i3kE33WYUenxfltnYHUM8JQO9alac20R8gfR2GkElnHnDDgD6QzwAEJ1wMCnhemkeUlnYBADrlyD6PLCzkA6A9pnC3nGbnWGJGP0WsQJmc+AhecpozYXkc8lQ7V+QMjPbHjNc5Upp/9RtPqGLev212w3VeWC4PyokIZbaKtwg5ANRFsT/Ew+q48CKd9YByxmcNZt8ApYuYHFb9OoMAeuLvcYoSjXBvF7KMiPRe/DV216UDFIQJmsedGWT21aCMGjxi/BvRgFDZrBbtQaTsjL0/pUDirN6w7m2X4PUY4KJU5GrN8koS1Evazbfwn51JQLpKsbXWxr2IXT14qTX1TzE5q9z2gGcJs7L+ryaU4LuSXJDyabTqaDzO+dWjQrZ3i9Spx4s3OU+i7kOxjF5Fu2MLsZz50XdflURi40AtdWfc7872ql4GRDAis7IzcnHiEee2myWOVy52Up+U6FkJlDUZzdlCJy/xQFjFTLKmn1ONERuaNIxboDz6niB/g+lzsvi8inpVx95DNf7FSklLVcxjROKBcVin6DODc3uNubtFvIOh0ud14my6cyLQShXztIlbfGl44yVrJKwVm1B5SKa2TORmhbpGvmkTsvi8inolwgXqaDsQaQNNnYsKJIB3KuGSewndEfgP5/WsipYtTlzstS8qkoV804d5u2EfZl5JLfTCJFTCWtFSd/AOYUUF6GQxsld17U5dNUrjUakox1sLXi5OzjbfCWF+w81UHX93LMnRd1+dSUC41i2SjodfdJ+GscpyzaZTYRIua/RXoXJgArNzQ4knE7nsq3QWjS+CnkVu+f2ruFt8G/GaOg7CwjnSmc6Fh8a9PJuilm/JRmu8/hwx+MOc3nudDdqIW4ZeOnqu3mRkGrPuZ/nIF5GMDLxo2CzkOl38UZaGdAbc3V/kjPcQbKYMCVq4x2dpRHYMCV6wik+yPLYMCVq4x2dpRHYMCV6wik+yPLYMCVq4x2dpRHYMCV6wik+yPLYMCVq4x2dpRHYMCV6wik+yPLYED7bOEGx0GyNgo5VzOXgNMaRsjLQ8ahGThe8xzl7mxkX/tPwaw6ckEwfhN0gZDGJZ8jXp20RnzbB8pSfgk4jWLkoXH2PfG01cgT/aEtxtauNhWzmnJBsKyNQrYyOjCjBJwWMdYy8+XedA+QEI5mzfzqeg7MasoFCdWNLkZZWT6xBJwWMd5CV7g7oTtMxqypXPxWKGYIRGxnyLdEE/BnUbUEnBYxXqB33MMIRNv2XGuJ4x5A+I2apDfDyZhPm3ec47oBpu2Wsa9f28pmmV4CTqsYITdNFNBaFddXfyPO9T5HMypbp6Vn5IfKiCpR19t/tUYuefBVVKzrxBQAHdWzyCoBp1mMUBJO7cQ6MEcsjkaXCT1nFsxaypUgvxsFTSHJSBmxxZGVuFAujlp8wXPEEmtPn+r0qbL2YtZSrthaS8DIW8GNggojeYcm2xIKxN1q2vvgNvxneG5ucCSje4XrrpnTLJhVlAuCy3QwBkDSZGPjGq7BvyXgNIyR00Cus3YOWLjW4ijGPsgpYtTNhVlFuWqJZRhuApCRK/lX8uYNMrsuAacpjFAOKg9Nvl01+wrS+FKnkkk/bBaR68mYNZVL3eiisHDksAScpjDWSkWTb9uWvkHl63u5T8asplwApm50sYW4RZNLwGkUI9dXB/8wAli4FuN2fLUsQahm1FTld66gd6saXQyec+xoCThNYYTSvIfnUSduXoQbFHsHd5GnZtTUjYIeWy39+atiAMrqRkFX1aIOJksG1NZcWaJ1oZyBBRlw5VqQbH9UWQy4cpXV3o52QQZcuRYk2x9VFgPhVjwPNDbR89i+nMdq5vm1M1A0A9CNTyBg20YClYs/ptG2QMyZP/8XA+VpzsBMDIjxpejt/gNZcY7TRsUfqgAAAABJRU5ErkJggg==",
      "text/latex": [
       "$\\displaystyle \\left[\\begin{matrix}10.0 & 0 & 0 & 0\\\\0 & 15.0 & 0 & 0\\\\0 & 0 & 13.0 & 0\\\\0 & 0 & 0 & 8.0\\end{matrix}\\right]$"
      ],
      "text/plain": [
       "⎡10.0   0     0     0 ⎤\n",
       "⎢                     ⎥\n",
       "⎢ 0    15.0   0     0 ⎥\n",
       "⎢                     ⎥\n",
       "⎢ 0     0    13.0   0 ⎥\n",
       "⎢                     ⎥\n",
       "⎣ 0     0     0    8.0⎦"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "R:\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAALgAAABkCAYAAAA1z/qTAAAACXBIWXMAAA7EAAAOxAGVKw4bAAAOH0lEQVR4Ae2dX3LdthnFJY+fO7E7k/fKO5DsFdTZQZId1N5BO3my3zLxDmKvIE124HQFTrQDp++diaPpBtLzo/mhFC5IgpfEBah8mIGIf5c4ODgAARCEzl+8eHF5dnb2s2zK/PDy5csvUhEe5gy0wID0+V44LlJYFHd+fxDxSm4SD80vQ4+7nYEGGfgmgekzhX1O+FDg30rxd1LQKtel7PWQCPk/kf/hXS3zsKx32a36ex2XT2EEHQg8TrfYrxtba/pVP34k+00jAvqXcCBoEzluzNXHy/zfWmVTvjx+/9EjfKzrB/wKt7L0UeOXWtiHiISBofD3sldy3wzj5txr8N+bu3luvEAwjn+nK+Qz3KFS3sqdHB/l3nejdIgCUiEZcf8gm010rbL13PFkfd5bGiTC/ln+p7rOGqWrVi/K+xPZ72W/FdA3sou1sBb/JgIXiGcCT2EQTmfkRlD4KVxtcy08D2TPZR/J0gizehGlq1k2nojPh+SBXX6w0xtOmsrYz5T/jewXspThu0mwiUj9bjX3mwhc2FhpST0y3yn8qYDSa+7V1CwbvfT7BH8/KpwOZa5HrIl9i/pejX8rgVMRDANiY5PWrMdp/ONG/DXLhpB/kZDHnjZzHUdN7FtU32r899eiSPQuqVs+TAWeMkw4u8ed8vyzLD3f1wpLPXUCrNplU/70YCnDXOJsCn9t7CnQS8K2wr9FD27iHetlKNdcT7Ok7MekJf9/irRXsoxhsTkTtebKJvyImwZKGaZMc9inwCbiNsG/hcAT2A6C6DWrGYniM9nQAOVm6MTjf4sJ8KnLxuSSN8ysVK01p8a+Fm/8+1n8Wwg8NfY2INYKWRdvzSDyCwmF3nDMNFU2YaVBMiYfG7oMy9EU9iGwTPcm+FcLXGRbz5gahliYTTYzy7ZdMuFjLZ614DFjGA/iWyqbsDCH4M0rr6FnTUvYZ8EmEmyFf7XAe2w87lM9ofXgxNcyvP1LibjDJiKvZ4BVL5sw8tqZ9fvQc8s99/ShWNWxz3A7F70a/1YCZ1yIkGJzpQBestzEESf0v1b+bBuIDUtQOQ2vatmE/VI4n+gaTyoR/dRjnPJWxQ6AlWY1/k0ELvLZ8PJBV0jvjNz0ml/K/u1jSLW/vOq+NZmU/+89mmGPyIuT32VvDWfkr1Y25c1TkUoGW1cOuyqM1/ddx6Frc9h7focXmxDaUz3ElcS/eh08oPy4cYnNVU8UxqSS61/lnxsCDG6xvVP5MykDl4kcgun5/qKw8GTBLctc4acECp5ENcr2VvkicsbfsQm8Noq9wytsNFAMT0wMe1PgmblRtxOwJP7zwQcPjPGqTQa7ovsfZ2ADBqRjOgSeeOebDFE2wOS3cAaKMOACL0Kr37QVBlzgrdSE4yjCgAu8CK1+01YYcIG3UhOOowgDLvAitPpNW2HABd5KTTiOIgy4wIvQ6jdthQEXeCs14TiKMOACL0Kr37QVBrL3oui154VA2442dg6yn2N3B9CMEa/yNXlo0d557/Hb0RdswENH7OvJ2cnZVdeausnqwXuQuz2AZkzUFq7ysYOwuUOL9s678CNoOkGzCJ1Nb2y0CjtP5R81SreqbrIErtx3fQDNKHuKEIHd1/a6tnho0d55h9tn4nYoZuu5v5qqF+K2qJtcgbPV8a4eQMOe8LD1FGJ7807X2ocW7Z13eGVL8q1tyT2/OZfVdZM7BqfVcUJrABqh41E0Zaiog1NAFWbbc4kPPejUjQrEtYxt17xLL+B/MKwzhVlvbvvzh9Gxe3Xd3I/vmPILFC0pZfic6kzxqR6wS6+4OfGT7uArj+7Hhf+0jI2iC9+d4l3lQbDdsEvuVIcXalzxm+gmS+Ah14FDABD3haytrAxibzlNvGO9P4lzCnPrpht5WsaWLOIeee8xI+4nsnSGqa+m4vJuUjdHC1xo+BTpj3AAjX1LGFdALf/ueJfAEXX3lJebIQqninHq7Nph6WzdMMn8U19Tdu294xcBY/x0Fw6g+TBeyjBs4vvSJsxd4L0XNU9zvs2cenKvqZtPrcJyV1Es/ZlAsfRzJw6gUVls2JQi2sJsIhw4qOHYI+/CzMIEQ9nY2BCFYUvSbFU3CPy/fQ52TWZIoDLl8XLXDqBhps9cIjY2BiS+qtkx77ykYThincVSHo+tm/9YRtk9uEDSEu/iATSMaR8bIYMrR0XUPrTobOe884RknsZ1aIzvuc5jdd1kHRshgPRwnNGRAsTLkO7kKF1pqb/JIgwEEoz8/ItCXtl2E4s+7b8V1sLZKU1iE0e75l34Gc6e6RqWBOVmFIBwObjIzkXZVDe6L/l2x0bkrqLs/gAaFXjK0BhrHOwzhYm4XfOOgGXpAIcvdWi0HGcdOku5ix26lNWDz9WCxzsDLTGgBhN68HstAXMszsDWDLjAt2bU79cUAy7wpqrDwWzNgAt8a0b9fk0x4AJvqjoczNYMuMC3ZtTv1xQDCJy9FnwrN7W5pSnQDsYZmGGANfbuQ2cEfiHLQrztvZDTjTOwawbYxNW9XPIhyq7r0cHPMeACn2PI43fNQO5elFBIvQZlVyGbZa7YQxAiMhxKX+VwHeXLZp6vBhDxs+E+7IcYxCWdtbAnwUSBLWMzqMJYRTdZAhc4BPFGlokoWx0Zty8yugd7g7/WdbibkL3CbLwp/VEBG6nsdKUOt/zdFyW6zn42pTQ1sU/y3Di26rrJGqKIRHZ78Q0dIvlukvFEpH7H5hf+l2MQk9w3CsM/3GmW+PW6oD5vBBob/n/nsFeP4zt/TexJQIPAlrEBU/iq6yZL4ANOj3WuPsDl2Iz1O/aqZ/1/95E8amIfgRSCW8YWQK5wrC7fqQTOsk1qnd2GJsSXMu9048/Vm3AeHo9MM8wHcp4eNbEb1rFry9jGMC8JX12++0tyOyZtJKqxWxRbg1f+fDLFUIgvSX6Tm3Nc6NURfBgyyX9gFD9sEAfxfUAx7GMZEt4ytincuXFble8UPbgJgDH3mMkR0thvZ8NFFo86+2yKnpue4Xr2h/9/+VUN+wTG6rxOYNsiapPynULgOYWdPcAl5yZjaSRwem9ESs/N0uCFLIeJEr7WFMW+ElzL2FYWrfv5bPlOIfDU2NsKZ630VwvY+ioRs4LDUiQfPHNYERNOenTMG/mnnh5VsX+EOPq3ZWyjoBdEbFK+4gKXgOzxnhKShdlkc0H5s5MyJLl1fqIwMfamNyd/hitJ0wD2JC4CW8Y2CnpBxFblKy7wvkw2LIiLaD149hvF+AZTfpGEgFl/v4nTKYxGhdANQ5zE/FWwW+Yz15axzUDPil5dvlMJfPUBLll0RIl6YfOy4SKKMi8NYK5xVcFuAGeuLWObgZ4Vvbp8xwjcBvYHPZ+ERG/5u+ytN4fys4LxQdcwqZMbcX0pyxvFkobx9sFBj8qfsTlLhd3wCDyyrWGf5EV4a/I6iS0RWUU32evgIpPWhLExK6JBHIjETiiityTsJxJG5kr+kx+uIzw/ytKImFAOJy63NlsprjnsEX9j3iq8joGJw8VrVd34wT9xjbh/9wyoUfnBP7uvRS9AFgPHjMGzbuyJnIEWGHCBt1ALjqEYAy7wYtT6jVtgwAXeQi04hmIMuMCLUes3boEBF3gLteAYijHgAi9Grd+4BQZc4C3UgmMoxoALvBi1fuMWGFiyF+VCgG1f9WO52dfBRwTXuQVRWvZmY/jAgf3Y7E3pNjsRWMooD7DbuShs8sJP3nM7CQOkWtgNgPK/lJt9Hbs5cMmwz11LcpslcAFAEPxbtnD8Qg/KDu6ZFYrSs8Pw5Af/KF8ETUM0gZ/Jza5GNoll/b90pauJfc8HLonmaVOa23vT2YdYet4gEEIFjN78RtZ2ixGcNEpb7eAfASLvZ8IQturKbw2y6YN/hJkdjrs8cCkphCjwFLrIFThbZPlIl95waBAK+6jp4afM6gNcpm4+E8cQioaI7YzwBreFTVxrYp+AlRXVOvbi+LKGKKISIV9OCCMWfsw+DcSObRjG2fib+MkzSoY/WuIWZrA/GP5GYdab5x78UwX7EPOR7mq8Z+Itju9+DhAJgpaWMkx8zhQ/OtFU3Jz4ucXB10EEljDCA6ndkEvulHBDtq1hD8AyHK1jPxW+LIGn+BRAxM3QxFZWUskIM/FODQtyGsHY/bPCe7yI+4ksDTL11VF8ryawx6Ay/a1jPwm+owUukplccizaq0zCp5LZ93pTaVbFCSei7p40cjNEYQUoaxVlJuPi2GfyXxPdOvbV+HInmbdIlDAYu3KIztjQZZj+w9ATua0Vsy5+MiPcjPd5ohx8jByBaA57hG/K2zr2k+BbLHCJg2W3h7qGNfEplpXOhiapYYiF2WRz6lZHxSl/JseXiR/bEIVhS9LUxp4ElRnYOvZT4VskcIHi0f5I19Bzy32BneGdlYxUGuvBiS9leEnDcMQa09J8amJfijVO3zr24viyBS6B0As+0TWeVCL6qccNpDNef4wjMlfyX+ue1stH0Zt4uTdzhTgPwzPXuGpiX0tA69iL48sSuMRB7wsYXurwyj5YhT038ehKfGuH59Ag38oGI4w0Snr01rEHzHLYhMueeiGuUd4DvjGHcLNMW/RAqNxVFASCyBl/x+baAgSYV8utHfzzWpieyg5f6lAWTpwNvbfczWGHV+GiY8HYXGEXBy59hJz1l6d4sQOh/OCfrDrwRHtiQJ0CHTGjjPN7ewLuWJ2BpQy4wJcy5ul3xYALfFfV5WCXMuACX8qYp98VAy7wXVWXg13KwHCZkA8a4t/zgiS8tYwj3e8M1GZA+nwvDBdjOBA469a3PkcbJC62R2SQhzudgTUM2IfsyXv8D1yzWetxHxn7AAAAAElFTkSuQmCC",
      "text/latex": [
       "$\\displaystyle \\left[\\begin{matrix}0 & 5.0 & 2.0 & 1.0\\\\2.0 & 0 & 2.0 & 3.0\\\\1.0 & 8.0 & 0 & 1.0\\\\2.0 & 3.0 & 1.0 & 0\\end{matrix}\\right]$"
      ],
      "text/plain": [
       "⎡ 0   5.0  2.0  1.0⎤\n",
       "⎢                  ⎥\n",
       "⎢2.0   0   2.0  3.0⎥\n",
       "⎢                  ⎥\n",
       "⎢1.0  8.0   0   1.0⎥\n",
       "⎢                  ⎥\n",
       "⎣2.0  3.0  1.0   0 ⎦"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "A = np.array([[10, 5, 2, 1], [2, 15, 2, 3], [1, 8, 13, 1], [2, 3, 1, 8]])\n",
    "print(\"Matrix A with Rank: \", np.linalg.matrix_rank(A))\n",
    "display_matrix(A)\n",
    "x = np.array([[1], [2], [3], [4]])\n",
    "print(\"b:\")\n",
    "b = A @ x\n",
    "display_matrix(b)\n",
    "\n",
    "D = np.eye(A.shape[0]) * np.diag(A)  # diagonal\n",
    "D_inv = np.eye(A.shape[0]) * 1 / np.diag(A)  # inverse of D\n",
    "R = A - D  # remainder\n",
    "print(\"D:\")\n",
    "display_matrix(D)\n",
    "print(\"R:\")\n",
    "display_matrix(R)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Comment\n",
    "Notice that we do not need to invert the matrix D.  \n",
    "Since D is a diagonal matrix, we can just invert the elements of the diagonal. This is a much more efficient method."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Convergence in 56 iterations:\n",
      "x = \n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAACoAAABkCAYAAADwkaJHAAAACXBIWXMAAA7EAAAOxAGVKw4bAAAGSElEQVR4Ae2cW44VNxCGDxHP0YRIWcBhBwOzAoYdcFlBYAdBeZp5Q7ADYAUIdgA7IMwOYAGRMhllA8n/ma4+bh/f+nomokuy7C6Xy3+X3Xa5jmdunJ2dHW82m89KMXp/fn7+MFYxNU/9fJHObUyv6m7c9CpeqoywT1/9h5nLLyL674v3AL4P9JWQLwmsg0t9v+4w9CAerD2gMIukxkyVd0p3VL4qNvAEJG9W+0vs20ovxKsyjm9RT2W3KGVH4rxRulS6qxSdS+InSTr4Dp4rf49Qo/Oz8vtKRbA/JDV7FVJ0pfRQ6anYb72qqqLaPZHgkXIHkkYqMxo8v+K5RFVAS0oq6lk5LiJyn8Q7FWhGLEtLAT0VCqZNSDbk1GdpdqA11hLCW1mUqpwdqAcit0Jcm6EvGeznksASFo3NTcNlQ866mqXZgTbLECBiw2s8+6iSYGcH2vT8Ufk2gsIsSn2WlgLKlsuOFtIdMS48q4f17fMQoDbxzRqtMpYipX+VOm6jnnE4LpU7B4MGKjPsj5R+5blEVXs9SqQYq0C2OL8Tj7n1QbnzfJSz1cL7A8GAsB5OyIlyPh7ye3qO7Viq6lIfoFUOtDrGK9oj8a/ExFcYREOGflBHYxutQMdaMGy/WjS0yNjn1aJjLRi2Xy0aWmTs8//Gon220K2s8qyxDJ4QDvGz2r2adpIdHICosqg6ACQhn6dNcu6ZeAQQzEkBS5Ikh0f1STkvR5yLl8ahQXeRqoBKC5boOBR0KB6OhnlVKsZJsosFILDaF3VoRwdDhGeOD1qyymIBCAB9FSAsGKPwBUIZXjR2yLOzUnH63Aw1xp4FMOWLHiOv+qTzq7rSS6Bi77QA06faOeq3cWUBACRDbivBnkzDMBCp0UCs+DKDgUo5HxGhc77gsWTnsKQegP7Y1FqeFLYKgSNUyJxNTQkTJY/NTas3a6cCEL+YYG+LChxLzS3lxNeLJDkb8tjwGs8+qqQ+gP7T1FqeFFanHHdvK28tqfKWlGz0rWJoAOJP01ttUYE5VqMT5eHHA/jc8NLX6ABE7fKExejso4CGoexT8dwHpZyh/FuJ6AfbrCOVXyuxdT5Q8mP4BCDuNWLZrAqoNHxQAizzM6QLYwjEYQMQAhANKhhAP0/J8hKS6/gLfrtSuXqOlhTNXb8CndrCq0W/a4uyz7JslHaXqY1Uo4+t1y1pzNGtEruNeTIqXhvC83c74foxTT0mtXv9RlsgU8S2QJwPnvnxgHlURZKdPQABMLwfSwBm7hBAaH+SyaGV3CIBCLymJwEos+TvOYDUqd1iAQhcObwfkiN13paNl8lHByCq5qhAYb2ffCDi2ZC75cOvi5RZZvauC4lnZyXq2/smkfbDLhYIJIpdPErlGIC2L9Uzv0tUXMOrLGq9qFPOTYA8UWI6xH5KFLtDBiI3VYov0xco4EgbgWboCTtyzSg7bMgXqCoAUdARr27AYSV+vM1Z5DKuwXHN2qkARNu0agsVkGNS22pXsKFnOkRJ7WzIYy9jPPuoojpgVgGVHIs1w2yKaduHWDXYyUIyi9qaHNa3z7VAsQoBMXKf7jYPpY5GByBqgbp4u49QoPmYsDBxffcCWFzpcDcg1DmRDiIi/uLOUHJTsbWmyocNQGDJBlALCl6MJBcNVvASkjfvK9Y0y6sd+qySJSpXoFNbebXoatGpLTC1vnWOfrcW7eXh+1bSloh/6i4a+PxcWW3mDUAkOsd1M38yIbJjC+QiAYhdjyqp0986jMKD5BcLQLRQ1ClDjifkfNC2Il8YHYAYsjw9FtjsWT6CmTNV7JBnZ6Xkmct09QLaDLnvPJueZK42R8nKXUVxrlcDVYdb6TUPftdFuWQgclOl+DLVQIWHpajvkJdf45vENAEIAeSr7TXkHsLY3LRqs/b4AEQz5JwubeJbJ1W52tmQx4bXeEXdNTsTc5MLBSzwPrFMcfsBPvdLwgsHviyHQvSEZBYtHhqLQAUAJXuKxOcCARcN2msbIQrvmZex7dNjb7h8MNufYFhHDJsNneMJ9GEDEIaMXGD4qGwYCUpgLW4xvlSy5cuCZzQxwnrz/wmG9SYw2SCC6tcAhBnrWud9dqaDvsgKdGrz+ws+V4ND/dfiPxYACqDss6klp7gHo2Qiiu1crer/AP5TPXUImJZvAAAAAElFTkSuQmCC",
      "text/latex": [
       "$\\displaystyle \\left[\\begin{matrix}1.0\\\\2.0\\\\3.0\\\\4.0\\end{matrix}\\right]$"
      ],
      "text/plain": [
       "⎡1.0⎤\n",
       "⎢   ⎥\n",
       "⎢2.0⎥\n",
       "⎢   ⎥\n",
       "⎢3.0⎥\n",
       "⎢   ⎥\n",
       "⎣4.0⎦"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "x0 = np.ones_like(b)  # initial guess\n",
    "eps = 1e-10  # tolerance error\n",
    "N_max = 100  # max number of iterations\n",
    "\n",
    "for i in range(1, N_max + 1):\n",
    "    x_new = D_inv @ (b - R @ x0)\n",
    "    if scp.linalg.norm(x_new - x0) < eps:\n",
    "        print(\"Convergence in {} iterations:\".format(i))\n",
    "        break\n",
    "    x0 = x_new\n",
    "    if i == N_max:\n",
    "        print(\"Fail to converge in {} iterations\".format(i))\n",
    "print(\"x = \")\n",
    "display_matrix(x_new.round(2))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id='sec3'></a>\n",
    "## Gauss-Seidel method\n",
    "\n",
    "This method is very similar to the Jacobi method. It is again an iterative method.   \n",
    "The idea is to write the matrix $A = L + U$ as the sum of a lower triangular matrix L and a strictly upper triangular matrix U.\n",
    "\n",
    "$$ (L+U) x = b $$\n",
    "\n",
    "$$ Lx = b - Ux$$\n",
    "\n",
    "Like in the Jacobi method we look for an approximate solution $x^{n+1}$, and iterate until $||x^{n+1}-x^n|| < \\epsilon$.\n",
    "\n",
    "$$ x^{n+1} = L^{-1} (b - Ux^{n})$$\n",
    "\n",
    "Again, given the specific form of the matrices L and U, we don't need to invert any matrix.  \n",
    "We will just use the forward substitution.\n",
    "\n",
    "For more info on the Gauss-Seidel method have a look at the wiki page [link](https://en.wikipedia.org/wiki/Gauss%E2%80%93Seidel_method).\n",
    "\n",
    "Let us consider the same A,x,b as in the previous example:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "U = np.triu(A, k=1)  # Matrix U\n",
    "L = np.tril(A)  # Matrix L"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Convergence in 15 iterations:\n",
      "x = \n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAACoAAABkCAYAAADwkaJHAAAACXBIWXMAAA7EAAAOxAGVKw4bAAAGSElEQVR4Ae2cW44VNxCGDxHP0YRIWcBhBwOzAoYdcFlBYAdBeZp5Q7ADYAUIdgA7IMwOYAGRMhllA8n/ma4+bh/f+nomokuy7C6Xy3+X3Xa5jmdunJ2dHW82m89KMXp/fn7+MFYxNU/9fJHObUyv6m7c9CpeqoywT1/9h5nLLyL674v3AL4P9JWQLwmsg0t9v+4w9CAerD2gMIukxkyVd0p3VL4qNvAEJG9W+0vs20ovxKsyjm9RT2W3KGVH4rxRulS6qxSdS+InSTr4Dp4rf49Qo/Oz8vtKRbA/JDV7FVJ0pfRQ6anYb72qqqLaPZHgkXIHkkYqMxo8v+K5RFVAS0oq6lk5LiJyn8Q7FWhGLEtLAT0VCqZNSDbk1GdpdqA11hLCW1mUqpwdqAcit0Jcm6EvGeznksASFo3NTcNlQ866mqXZgTbLECBiw2s8+6iSYGcH2vT8Ufk2gsIsSn2WlgLKlsuOFtIdMS48q4f17fMQoDbxzRqtMpYipX+VOm6jnnE4LpU7B4MGKjPsj5R+5blEVXs9SqQYq0C2OL8Tj7n1QbnzfJSz1cL7A8GAsB5OyIlyPh7ye3qO7Viq6lIfoFUOtDrGK9oj8a/ExFcYREOGflBHYxutQMdaMGy/WjS0yNjn1aJjLRi2Xy0aWmTs8//Gon220K2s8qyxDJ4QDvGz2r2adpIdHICosqg6ACQhn6dNcu6ZeAQQzEkBS5Ikh0f1STkvR5yLl8ahQXeRqoBKC5boOBR0KB6OhnlVKsZJsosFILDaF3VoRwdDhGeOD1qyymIBCAB9FSAsGKPwBUIZXjR2yLOzUnH63Aw1xp4FMOWLHiOv+qTzq7rSS6Bi77QA06faOeq3cWUBACRDbivBnkzDMBCp0UCs+DKDgUo5HxGhc77gsWTnsKQegP7Y1FqeFLYKgSNUyJxNTQkTJY/NTas3a6cCEL+YYG+LChxLzS3lxNeLJDkb8tjwGs8+qqQ+gP7T1FqeFFanHHdvK28tqfKWlGz0rWJoAOJP01ttUYE5VqMT5eHHA/jc8NLX6ABE7fKExejso4CGoexT8dwHpZyh/FuJ6AfbrCOVXyuxdT5Q8mP4BCDuNWLZrAqoNHxQAizzM6QLYwjEYQMQAhANKhhAP0/J8hKS6/gLfrtSuXqOlhTNXb8CndrCq0W/a4uyz7JslHaXqY1Uo4+t1y1pzNGtEruNeTIqXhvC83c74foxTT0mtXv9RlsgU8S2QJwPnvnxgHlURZKdPQABMLwfSwBm7hBAaH+SyaGV3CIBCLymJwEos+TvOYDUqd1iAQhcObwfkiN13paNl8lHByCq5qhAYb2ffCDi2ZC75cOvi5RZZvauC4lnZyXq2/smkfbDLhYIJIpdPErlGIC2L9Uzv0tUXMOrLGq9qFPOTYA8UWI6xH5KFLtDBiI3VYov0xco4EgbgWboCTtyzSg7bMgXqCoAUdARr27AYSV+vM1Z5DKuwXHN2qkARNu0agsVkGNS22pXsKFnOkRJ7WzIYy9jPPuoojpgVgGVHIs1w2yKaduHWDXYyUIyi9qaHNa3z7VAsQoBMXKf7jYPpY5GByBqgbp4u49QoPmYsDBxffcCWFzpcDcg1DmRDiIi/uLOUHJTsbWmyocNQGDJBlALCl6MJBcNVvASkjfvK9Y0y6sd+qySJSpXoFNbebXoatGpLTC1vnWOfrcW7eXh+1bSloh/6i4a+PxcWW3mDUAkOsd1M38yIbJjC+QiAYhdjyqp0986jMKD5BcLQLRQ1ClDjifkfNC2Il8YHYAYsjw9FtjsWT6CmTNV7JBnZ6Xkmct09QLaDLnvPJueZK42R8nKXUVxrlcDVYdb6TUPftdFuWQgclOl+DLVQIWHpajvkJdf45vENAEIAeSr7TXkHsLY3LRqs/b4AEQz5JwubeJbJ1W52tmQx4bXeEXdNTsTc5MLBSzwPrFMcfsBPvdLwgsHviyHQvSEZBYtHhqLQAUAJXuKxOcCARcN2msbIQrvmZex7dNjb7h8MNufYFhHDJsNneMJ9GEDEIaMXGD4qGwYCUpgLW4xvlSy5cuCZzQxwnrz/wmG9SYw2SCC6tcAhBnrWud9dqaDvsgKdGrz+ws+V4ND/dfiPxYACqDss6klp7gHo2Qiiu1crer/AP5TPXUImJZvAAAAAElFTkSuQmCC",
      "text/latex": [
       "$\\displaystyle \\left[\\begin{matrix}1.0\\\\2.0\\\\3.0\\\\4.0\\end{matrix}\\right]$"
      ],
      "text/plain": [
       "⎡1.0⎤\n",
       "⎢   ⎥\n",
       "⎢2.0⎥\n",
       "⎢   ⎥\n",
       "⎢3.0⎥\n",
       "⎢   ⎥\n",
       "⎣4.0⎦"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "x0 = np.ones_like(b)  # initial guess\n",
    "eps = 1e-10  # tolerance error\n",
    "N_max = 100  # max number of iterations\n",
    "\n",
    "for i in range(1, N_max + 1):\n",
    "    x_new = solve_triangular(L, (b - U @ x0), lower=True)\n",
    "    if scp.linalg.norm(x_new - x0) < eps:\n",
    "        print(\"Convergence in {} iterations:\".format(i))\n",
    "        break\n",
    "    x0 = x_new\n",
    "    if i == N_max:\n",
    "        print(\"Fail to converge in {} iterations\".format(i))\n",
    "print(\"x = \")\n",
    "display_matrix(x_new.round(2))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Comment:\n",
    "The Gauss-Seidel algorithm converges much faster than the Jacobi method (15 iterations agains 56)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id='sec4'></a>\n",
    "## SOR (successive over-relaxation) \n",
    "\n",
    "\n",
    "This is another improvement of the previous methods.   \n",
    "Let's write the matrix $A = L + U + D$ as the sum of a strictly lower triangular matrix L, a strictly upper triangular matrix U and a diagonal matrix D.\n",
    "\n",
    "$$ (L+U+D) x = b $$\n",
    "\n",
    "$$ w (L+U+D) x = w b $$\n",
    "\n",
    "$$ w L x = w b - w D x - w U x \\pm D x $$\n",
    "\n",
    "$$ (w L + D) x = w b - \\bigl( w U + (w - 1) D \\bigr)  x $$\n",
    "\n",
    "As in the previous methods, given the specific form of the matrices, we don't need to invert any matrix.  The iterative method is the following:\n",
    "\n",
    "$$ x^{n+1} = (w L + D)^{-1} \\biggl( w b - \\bigl( w U + (w - 1) D \\bigr)  x^n \\biggr) $$\n",
    "\n",
    "For more info on the SOR method have a look at the wiki page [link](https://en.wikipedia.org/wiki/Successive_over-relaxation)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "D = np.eye(A.shape[0]) * np.diag(A)  # diagonal\n",
    "U = np.triu(A, k=1)  # Strict U\n",
    "L = np.tril(A, k=-1)  # Strict L"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Convergence in 34 iterations:\n",
      "x = \n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAACoAAABkCAYAAADwkaJHAAAACXBIWXMAAA7EAAAOxAGVKw4bAAAGSElEQVR4Ae2cW44VNxCGDxHP0YRIWcBhBwOzAoYdcFlBYAdBeZp5Q7ADYAUIdgA7IMwOYAGRMhllA8n/ma4+bh/f+nomokuy7C6Xy3+X3Xa5jmdunJ2dHW82m89KMXp/fn7+MFYxNU/9fJHObUyv6m7c9CpeqoywT1/9h5nLLyL674v3AL4P9JWQLwmsg0t9v+4w9CAerD2gMIukxkyVd0p3VL4qNvAEJG9W+0vs20ovxKsyjm9RT2W3KGVH4rxRulS6qxSdS+InSTr4Dp4rf49Qo/Oz8vtKRbA/JDV7FVJ0pfRQ6anYb72qqqLaPZHgkXIHkkYqMxo8v+K5RFVAS0oq6lk5LiJyn8Q7FWhGLEtLAT0VCqZNSDbk1GdpdqA11hLCW1mUqpwdqAcit0Jcm6EvGeznksASFo3NTcNlQ866mqXZgTbLECBiw2s8+6iSYGcH2vT8Ufk2gsIsSn2WlgLKlsuOFtIdMS48q4f17fMQoDbxzRqtMpYipX+VOm6jnnE4LpU7B4MGKjPsj5R+5blEVXs9SqQYq0C2OL8Tj7n1QbnzfJSz1cL7A8GAsB5OyIlyPh7ye3qO7Viq6lIfoFUOtDrGK9oj8a/ExFcYREOGflBHYxutQMdaMGy/WjS0yNjn1aJjLRi2Xy0aWmTs8//Gon220K2s8qyxDJ4QDvGz2r2adpIdHICosqg6ACQhn6dNcu6ZeAQQzEkBS5Ikh0f1STkvR5yLl8ahQXeRqoBKC5boOBR0KB6OhnlVKsZJsosFILDaF3VoRwdDhGeOD1qyymIBCAB9FSAsGKPwBUIZXjR2yLOzUnH63Aw1xp4FMOWLHiOv+qTzq7rSS6Bi77QA06faOeq3cWUBACRDbivBnkzDMBCp0UCs+DKDgUo5HxGhc77gsWTnsKQegP7Y1FqeFLYKgSNUyJxNTQkTJY/NTas3a6cCEL+YYG+LChxLzS3lxNeLJDkb8tjwGs8+qqQ+gP7T1FqeFFanHHdvK28tqfKWlGz0rWJoAOJP01ttUYE5VqMT5eHHA/jc8NLX6ABE7fKExejso4CGoexT8dwHpZyh/FuJ6AfbrCOVXyuxdT5Q8mP4BCDuNWLZrAqoNHxQAizzM6QLYwjEYQMQAhANKhhAP0/J8hKS6/gLfrtSuXqOlhTNXb8CndrCq0W/a4uyz7JslHaXqY1Uo4+t1y1pzNGtEruNeTIqXhvC83c74foxTT0mtXv9RlsgU8S2QJwPnvnxgHlURZKdPQABMLwfSwBm7hBAaH+SyaGV3CIBCLymJwEos+TvOYDUqd1iAQhcObwfkiN13paNl8lHByCq5qhAYb2ffCDi2ZC75cOvi5RZZvauC4lnZyXq2/smkfbDLhYIJIpdPErlGIC2L9Uzv0tUXMOrLGq9qFPOTYA8UWI6xH5KFLtDBiI3VYov0xco4EgbgWboCTtyzSg7bMgXqCoAUdARr27AYSV+vM1Z5DKuwXHN2qkARNu0agsVkGNS22pXsKFnOkRJ7WzIYy9jPPuoojpgVgGVHIs1w2yKaduHWDXYyUIyi9qaHNa3z7VAsQoBMXKf7jYPpY5GByBqgbp4u49QoPmYsDBxffcCWFzpcDcg1DmRDiIi/uLOUHJTsbWmyocNQGDJBlALCl6MJBcNVvASkjfvK9Y0y6sd+qySJSpXoFNbebXoatGpLTC1vnWOfrcW7eXh+1bSloh/6i4a+PxcWW3mDUAkOsd1M38yIbJjC+QiAYhdjyqp0986jMKD5BcLQLRQ1ClDjifkfNC2Il8YHYAYsjw9FtjsWT6CmTNV7JBnZ6Xkmct09QLaDLnvPJueZK42R8nKXUVxrlcDVYdb6TUPftdFuWQgclOl+DLVQIWHpajvkJdf45vENAEIAeSr7TXkHsLY3LRqs/b4AEQz5JwubeJbJ1W52tmQx4bXeEXdNTsTc5MLBSzwPrFMcfsBPvdLwgsHviyHQvSEZBYtHhqLQAUAJXuKxOcCARcN2msbIQrvmZex7dNjb7h8MNufYFhHDJsNneMJ9GEDEIaMXGD4qGwYCUpgLW4xvlSy5cuCZzQxwnrz/wmG9SYw2SCC6tcAhBnrWud9dqaDvsgKdGrz+ws+V4ND/dfiPxYACqDss6klp7gHo2Qiiu1crer/AP5TPXUImJZvAAAAAElFTkSuQmCC",
      "text/latex": [
       "$\\displaystyle \\left[\\begin{matrix}1.0\\\\2.0\\\\3.0\\\\4.0\\end{matrix}\\right]$"
      ],
      "text/plain": [
       "⎡1.0⎤\n",
       "⎢   ⎥\n",
       "⎢2.0⎥\n",
       "⎢   ⎥\n",
       "⎢3.0⎥\n",
       "⎢   ⎥\n",
       "⎣4.0⎦"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "x0 = np.ones_like(b)  # initial guess\n",
    "eps = 1e-10  # tolerance error\n",
    "N_max = 100  # max number of iterations\n",
    "w = 1.4  # relaxation factor\n",
    "\n",
    "for i in range(1, N_max + 1):\n",
    "    x_new = solve_triangular((w * L + D), (w * b - w * U @ x0 - (w - 1) * D @ x0), lower=True)\n",
    "    if scp.linalg.norm(x_new - x0) < eps:\n",
    "        print(\"Convergence in {} iterations:\".format(i))\n",
    "        break\n",
    "    x0 = x_new\n",
    "    if i == N_max:\n",
    "        print(\"Fail to converge in {} iterations\".format(i))\n",
    "print(\"x = \")\n",
    "display_matrix(x_new.round(2))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "By changing the *relaxation parameter* we can see how the number of iterations changes.  \n",
    "The relaxation must satisfy $0<w<2$.  \n",
    "It seems that in this case the optimal value of $w$ is 1 and the minimum number of iterations is 15.  \n",
    "The Gauss-Seidel algorithm is a special case of the SOR algorithm, with w=1!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id='sec5'></a>\n",
    "## Thomas algorithm\n",
    "\n",
    "The Thomas algorithm is a simplified form of Gaussian elimination that is used to solve tridiagonal systems of equations.   \n",
    "Have a look at the wiki page: [TDMA](https://en.wikipedia.org/wiki/Tridiagonal_matrix_algorithm)\n",
    "\n",
    "The idea is simply to transform the matrix A into a triagonal matrix and then use backward substitution.\n",
    "\n",
    "Here we consider the single coefficients of the linear system instead of performing matrix operations like in the previous algorithms.  \n",
    "\n",
    "$$ a_i x_{i-1} + b_i x_{i} + c_i x_{i+1} = d_i $$\n",
    "\n",
    "The algorithm we implement is the following (taken from wikipedia):\n",
    "\n",
    "$$ w_i = \\frac{a_i}{b_{i-1}} $$\n",
    "\n",
    "$$ b_i = b_i - w_i c_{i-1} $$\n",
    "\n",
    "$$ d_i = d_i - w_i d_{i-1} $$\n",
    "\n",
    "and then we perform backward substitution:\n",
    "\n",
    "$$ x_n = \\frac{d_n}{b_n} $$\n",
    "\n",
    "$$ x_i = \\frac{d_i - c_i x_{i+1}}{b_i} $$\n",
    "\n",
    "Let us introduce a new matrix A wich is tridiagonal. Of course we like to keep the ```x = [1,2,3,4]``` "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Matrix A with Rank:  4\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAJcAAABkCAYAAACLmaLOAAAACXBIWXMAAA7EAAAOxAGVKw4bAAALc0lEQVR4Ae2dXZLcNBSFJ1SeqRCqeKfZwYSsILCDwA6AHUDxlLxRsAPICvjZAWEFIdlBWABVwBQr4HyOb5dGLbdtWbZk+t4qj/Vn6eroWJIl3ek7T548ub66unqpKyW/PH369JNUhIc5AuLGa6FwSCGhuDt3g4jv5CZxKH+EHnc7AhEC30Z+vB/reowjJNf3Ytv/kkyq17WuV1TYRP57ct//v9bZ6rnmXdj9EOevMIJOyBWnS/r1MMPoz7oeyH2TSqRwY/Rfiv9A17cKq0nc31Q+ZDKC4UYevLm18bc13JbqE/Zcgwj3DfNMCf7W9aGu5DhLBkrL/O0b3X/p/TTkS/k/1lWLYOiN8GKgA7qhY/LlUNzmIl2awq2EPlPJRSN0E3sV+qXcNNKJKO5zBd7TvSMWCeS+6f3fy8t4XENeSYdmP0ykW1O4ldLnrcItTQPa0BNm/UKej6S0DUdhnLvfvLgt4VakHUuT6yMxxYagkDQ2HBLvcopAa7gV0efuaT3zQib2Svfzcl/+lHX1yuldXcwZmXOleovlhc3IoTXcSupTsucy4pybJNcaFin3JwH3na6v5ObiI6OFnrQ13IrpU5JcU95Xeo3NRSTiS/VIerkZpp/r4iNjD1IFtzPATNKnJLlScy3Tz94G1r1aEQh2ENEYImtKa7gV06cYuYKeITX0WZhN7DdrTOn1qy7WkIbEdBuKXzW8NdxK6lOMXH0LMNSkegLruYjfWlj0TRGo00lgVp/US7/WcCuiT2lysS1EY8byQAEsZN7EERv4f1C5bEHFwmS+BtljPfC3hlsRfXLIZZM5642OYKkR2cj8W/du45IIuek1PtX1Gf4Kwob8rYm7/OwyIE2s2kufpnArpc/kdS4VCJsR+3z/WWHMoZjThLvj9FJsVD/UnQk890fyVxl+0FEX+hjBeCmYtL6vsBo9qYpOSlO4ScPF+twJDgt+QEMkq+2BjsBEBMQh9kkZLe7kDIsTi/Fkl46Ak+vSGbBi/Z1cK4J76Vk7uS6dASvW38m1IriXnrWT69IZsGL9nVwrgnvpWTu5Lp0BK9bfybUiuJee9UWSS6vH13HDKwyrpUMc7v58BCbvLaaK6BuDI8MIpyHYs/tK4VX2EVECUfmQh73QIcPd5oxkW8RSOi0ybs4mVw8Ge0hHW8ReGTOA3fQ4i8rm9MUkw12l4yVAICH7qVWNZFvDEmCk02Ij3WxyqXxY/QWKmEghei02Luk13rHwLe4qlxMO3REauQcNd3tdWjOSbQ3LIka6S+ZcHL15rYakxwiFHsvnLyEi4+7WsOQlTU1tXih8snHzEnJBIs5KDZ2Jikk3DvHlpmgNS8huU4ewVexIFvGjcnc0xUACkaobghLR3ZeY4lPMTySvEyT9uq5fpXOylq/EakayLWEpXaZ0CventNqSnuskfykGsWgo+4I8SdNIAAC2aiTbQVQRSyPO0IiEflMIeFWUXCqUiTz/6pL/UtisSL89GMm2jCW9/ahArrf7VHYffSiVQA3GGXXmYEPDZeqxlsKYT7RgJHtVGcvUXMvayXq1c8bN71niIj2XwGD+wr+APK55WQGt3aVj60ayVbEUPjYcpoY+C7OJ/dnmhVz/9insfvaBOFLKYEaGccexx5K7iR4g1rX3s5NgIIVJurdSulf7EGkIS75eDyE4vdt6LuKH5E+LWNRzCYxrZfRQ93gCD+HOda9Wfo17k0ayjWHJfI+XMBbMzSYbNy9ZioDZKPFcwJhNoCnDQlvNSb1NOHnTrJs33TojWel33F2Qu6qRrMpvCkvpwwvIbstjXeH/tv1UID4yIMfu2eRSxr/qAhTmCLFUGVoEBGRHbJHvxHBXaVo0km0OS2FIL7XIuNmNYqGiSzEE9PLS2bhRbDFEPaMkAosm9MkcPdAR6BFwcjkVVkPAybUatJ6xk8s5sBoCTq7VoPWMIRf7RCwotrqi7q20LwTYGuoWqCHXQRcr7LZvJKeLI5CNAAvY3Y6ND4vZGPqDYwg4ucYQ8vhsBJbsLXaFarl/keFktuYzHpSOHLH5OngEP/uO546OBMnXcap8TpWwH5o03lU8UxbbYEdn/Oz3bab3mI7SZ1AWkUsFLzacHNSsbAQNYo3U5Sw/5Lr1w6Nli0znRpmKGTXe7dNxMuGot9wcZeKw4ye6jj+Ymi4pP1R5T9JxrITsYVEKsEF5q3EUdqMwKh0fwRnTY7X4Xk9eglj4v/hhbxbHr+IHI12QA9L8eKYQ8P1c6SCUifVYq+o9Q0fTK3nPJpdyK2I4mdSqbCC/ntH88etElTm2xMvK1QmNbu493JcMi3xyhj9uYPW189XEr9Z1W2ET7i+U5ks1DGem6DGsgZgrNtPDxvWQnvRSt/4lgsKsF2tW77AeWeRSJe+FmQy4m1g3k66YukFyGuYfuTmSTW/G3KUF8kuVcZGuvKzd/5SQO/VSj2eycYrcYdGIY71ASu0pBEw9VzxMjcEQbg1CA9FQVU7Lzq2cdL/WxTFs5mjo/PvcPGqlzyXXFH3tHPuUtKumUePQa/Ei0GMx3Bx08U9UbJiRt02RjhhE8PPJvCB8APAvqprXGzRzyXVuH9J6tXOGk5S9iagh+OrCwprPes7PM7k3M7hn8jfTw44BIl0ZxnlJumWUsfS147PIpUpSQSTVMBZmE/s3Kev9ZRi8ZfrWNxK9GLoyRDYn0pHhkEXWWGxYbFLvUNkscvUZ2PAS5ofbei5bk4njN/OrcSAPa3E3caEKg/z0BKZvnKS2n7U5hkB7WWvrM7v8JeRi2+LDRImzDCcTzxcL6knFouVhIFMarvpLMKAbLwRfuvGLYZi3qvexOtnkUqX5+mrtV2GPFQsczK9O5ijSn7kYyxE1h2/76En1ngzlrM0dRboykeeF+ELumHTHdIUd53Q8W9Qiu0VVkIoyp6GiTOAf6qr2T9RUdlKkJ3MXtkzCD5FqG9fSJzTeBUOWGCA5ZLclkyu5mVfZx4ec221cT9URpULRc7y0nd3iInKFmbrbEQCBkFzZw6JD6QiMIeDkGkPI47MRcHJlQ+cPjiHg5BpDyOOzEXByZUPnD44h4OQaQ8jjsxFwcmVD5w+OIeDkGkPI47MRcHJlQ+cPjiHg5BpDyOOzEbib/WT/oJb72VtE2FvkjBQ2gjU3g9HlRHakJ/ugg4ayJxVbKUB4secZmrDhn7Ufu4hcUoAzR2xUcy7qqlfIfim2GYK1rmeP26ihLBhvKIsNibOHRQHC7vdejGKb1lNYcuZsiqHsJtzq25aOI5ZZhsTZ5FKpHAXhqEgs2AlO/jXR+OEV/HvRc4WqZ2fJ9GaxIfEScnHWKDwfZTWx4ZD4FmQveraAlelAB8GvZ3C+jLmWCfPryQa5WXOuqEArOL6nTlfGaVb170XPVUHIyFy4FTEkzu25jDg3Z3QPGX8m2apRe9FzVRByMhfBmE7YqVh6LEaA1DRoMPtccg1mGETY2esgqEnnXvTcFDyRi/P6dB7ZhsS55ErNtazy1luw7lVb9qJnbZxulS9iFTEkziKXCrfhMDX0WZhN7G8pvqVnL3puicnEshgGFxsSZ5GrV7B5o9id6Tmx3ddNpheSzoF1QetAjgUqjA5jsiHxEnKxRWEGmkcF5Higa/KviYYPruTei54rVX9etj2pWNQ9DDwJ+SYZ5GaTS4Xvwih2L3oGDWkfGDZ3DaI2c/KluNiQOGudK6givRR7UA91ZwLP/ZH8sz5Z9cza0ryewoweFuGTH6FxGYZuGcp2MSv/Ubn8tDRbPc90Dz+KZm1cu1Hsyg11admLjEeL6+xh8dJA8/rOR8DJNR8zf2IiAk6uiUB5svkIOLnmY+ZPTETAyTURKE82H4FwKYL/bhznwNEL1jxcHIETBMSN1wo8nET0AZCLtZTjjxdFCavvD0b6uLctBMw4J6nVf+wOgUnYwWePAAAAAElFTkSuQmCC",
      "text/latex": [
       "$\\displaystyle \\left[\\begin{matrix}10 & 5 & 0 & 0\\\\2 & 15 & 2 & 0\\\\0 & 8 & 13 & 1\\\\0 & 0 & 1 & 8\\end{matrix}\\right]$"
      ],
      "text/plain": [
       "⎡10  5   0   0⎤\n",
       "⎢             ⎥\n",
       "⎢2   15  2   0⎥\n",
       "⎢             ⎥\n",
       "⎢0   8   13  1⎥\n",
       "⎢             ⎥\n",
       "⎣0   0   1   8⎦"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "b:\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAACUAAABkCAYAAAABmvnKAAAACXBIWXMAAA7EAAAOxAGVKw4bAAAHKklEQVRoBe2bXY4UNxDHm2if0WQj5Z3hBgt7AoYbADdguQHRPu2+IbgBcIIEbgA5AR83AN4jhaxygvx/psuUPe7u6mVG0Updktd2uVz173K1P9qz187Ozo66rvug1KLX5+fn91sNP8KTzk/qv27pUNu1A9fwTGWEPX32lR2WnzZ03RXvHnwP6rlQ7gtEgUF2XhQMVcSDtQUK5iCpE+7+rRe4rfwrdfE/9ryciWee+FvMm0pPxQs/8E9Z00ihB4QnH/XplsQB80H1je+qOvH5TjmACQke5I3KzRjyfa0cAiVhnvyRdSLHqLILpVfUIfFOlK2Uv06MbzxkqD833lQeBYU3PsnYqlL4VnVAmBd4U7eGU7x3SptGf7G3KQoK45+llKdukYEFPLFWk8VTMdS1kNUPrDCWC8zQXMUc16n9o5IBG1N1ONZobVFPmXzOBQJADJu9kWZwyJv0jQDvLg1KBghwZnzesCj9EhG8FCgB4U0ixvywtmLJMJgXmbcmaTYoAeG1P1TOspBJdRu21hAZzwI+92sVZoGSYZaBm8qzh1Rek3rlvKVW9vbMU7RPUhiUDBPYx8otsE05QG3oiDOWoJrSCqC+5s26vahHpwSeHoNvpbiemZkUU7Arf6HE8nJPKc3qyhm6B0p3lEIUAiVNb5QARjzVVM/geIUF+Fg5gU1+R/VaTuw2hUBJISt9iCTLEBXrZKijEwrHlOuz9+ICKurixVOLp6IeiMotMXWlPRVaZnhCLR9rZbZ8sMhSZ40rtiOq03aqZET9VS1nja08BKo3xOpvoDqV2bJwyLyvlM954gE0y2FUdUAV50H4QxQNdHYHJ1IMECPzUPaK2pHjhFzTQzGyXN1Y16Og2Haw+pMSCUAuG085u4lim+zawsXo8OGVn71W5zW/6eMk/Fht7L8YVgPOsd/LeVVb5RCoupeMbcRL3xdUzp91VObIRXwxzP+ozNYZ7xF7Pu7EGqbo8CUNUnyk9FgVApkhfZ8a3B+1c6gwoADnAcK7TlTNBcXx/Flv+Hf151OQD/6urzNseIhhXyvxcaSQE2+QAHW9b7V8UNg3yAjDgfH0utMmHm/fXeVMHxxWCXo7jr1UfYXcAP1q/JCnpIxh44hVkw0fQwQxXMURTP0Aj9cAZHIqDhOg/u2bLW9JM/cwVINP2rcxQeK9gsTjZAy4w6KhrPxl1ZCnJIwh3qza4O1eEedB2i6Ur3tena3EsAm3bivqUVAMCXNPJhkncDHEd1ADS/zkGDNhtRNrTAuhbwkH1nEslzJOvpyE/QSIRwjq/PSUlVhSCOqvTufuF2SUY1BZBuAMFkXJMSfZG1e0RSvR4Yvq24ncAirqxsVTV95TTGhsRfy8En2oXcox3aS9PTG1VmJSHFuX1Lx3YrFOk/PVDnTN1Ee1r8TzN1h186XrobWv1/4nIFRmGYEoQ3x4LUhyfo0kLB6KZ4t2IduqzBk+XgQU4zEAsT+65Y0BWonLcba/6RZV5SdKX1QndkM0x1Psz6cW2peyyhVJvkRSmX7sUPFe6Ew4x1ORp2SP1dozMeRsfWzIR3XtDJQz2Jrv7PbKdqqjoOYMXyfD7CB5Wu7tiJEn4qXAV85WWKzmfIc8RJ9JmuMpwPwhw5z72B6T6p8AEPwtwzad7Hb4BISt74WMJlKZ2GFp8K8/W+FObRtySGUAWb9WvCU5/2eOp3w/K2Mk3/cJAMZvKPFxgw8dnAPxHB8+oBCoUExJOScZXvWtiTKZ+j6RdpIBWFpY+zZ4gINCoKKe4q1ZJbXlH2ZrjKZgL5uKGkNoZ8OioVUJeUod0+ViQwGxk084Asc8xQR6Q+UUR8p5GOSGvKymkqKe4sc3PqA71fkkBPlZnvip5yluVFlypryJrkQhT0khX1D4wGrAGDaMZ4+gTe1MF8xJp8ptuLe+ICM7RiFQKJARgrQI4JZiyRVfXVoyU7zo8E3p2Wn7AirqzsVTi6eiHojKLTH1v3lKMzo7goLEm3VonbPMrGXJlhnWNeqtdS18aC2Qu0oIFE+qPtGbUdsl4DHWS/btHDAulIcoBEqaOMWcSLG/IrN91KnaMGwUObSabDOPvn3shXjS/LRznrxpeYQZ8pQA4JXIzeiIqXhTCFStTiA34nEYYEdpF45ZTLzBQ2sWGinMAiVjBC+AjpUYUj5c1MRLwaE1DbVy3lK+whRXJnUnX58LCiCkTkY4JHBCLn6XgHHajVRnK22HVu79Jika6FuKZIg3Dm9s3VptCX+bGvKhtdFesEKgBCB0Myo5pozWjyXMKEM7SSFQ0jJ5M9pb+tFDa1ITBcUwjd6M9qA4tLbihpfDJttedDiLgorejEYPrcOI1BJ6+/T00ZvR0KF1FFEUFEoEDPdPDoHkQodWdA5RdPiG+u+Fv4CKunXxVNRTfkpgJa/7MWH6j2J1+6Xq0jn4n5AoBNTYK0zbPsg+zDZ1/wc7opJIjhkhqgAAAABJRU5ErkJggg==",
      "text/latex": [
       "$\\displaystyle \\left[\\begin{matrix}20\\\\38\\\\59\\\\35\\end{matrix}\\right]$"
      ],
      "text/plain": [
       "⎡20⎤\n",
       "⎢  ⎥\n",
       "⎢38⎥\n",
       "⎢  ⎥\n",
       "⎢59⎥\n",
       "⎢  ⎥\n",
       "⎣35⎦"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "A = np.array([[10, 5, 0, 0], [2, 15, 2, 0], [0, 8, 13, 1], [0, 0, 1, 8]])\n",
    "print(\"Matrix A with Rank: \", np.linalg.matrix_rank(A))\n",
    "display_matrix(A)\n",
    "print(\"b:\")\n",
    "x = np.array([[1], [2], [3], [4]])\n",
    "b = A @ x\n",
    "display_matrix(b)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "## TDMA:  Tri-Diagonal-Matrix-Algorithm solver\n",
    "def TDMA(A, B):\n",
    "    N = len(B)  # number of equations\n",
    "    a = A.diagonal(offset=-1).copy()  # need to copy because the array is not writable\n",
    "    b = A.diagonal(offset=0).copy()  # see A.diagonal(offset=0).flags\n",
    "    c = A.diagonal(offset=1).copy()\n",
    "    d = B.copy()\n",
    "    x = np.ones_like(d)  # initialize x\n",
    "\n",
    "    # Overwright coefficients\n",
    "    for i in range(1, N):\n",
    "        w = a[i - 1] / b[i - 1]\n",
    "        b[i] = b[i] - w * c[i - 1]\n",
    "        d[i] = d[i] - w * d[i - 1]\n",
    "\n",
    "    # backward substitution\n",
    "    x[-1] = d[-1] / b[-1]\n",
    "    for i in range(N - 2, -1, -1):\n",
    "        x[i] = (d[i] - c[i] * x[i + 1]) / b[i]\n",
    "\n",
    "    return x"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "x= \n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAABoAAABkCAYAAACYYiB/AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAD/ElEQVRoBe2a0VHcMBCGj0yeM5fLTAowHRDo4NJBmHQAHYTJ090r6QAoATqADhLSARSQB8KkgvyfsZw9WbZW5iYPjHdGSFqv/t9aydJKx85qtdqbzWa3Sim5Wq/Xh6kHsU52d9JVsZ66nu28Ng++qYyxlXtbyZRPE88/SvcJvSU6E3MJ8Aau2p5vKFSRDlWHCGVW1BhXXyp9UPkx26AxsD3qbSPAuR5eKD0o7Sslx0L6XvES8eb1pBDpF5XpVZG8KrJ+hvFENNp5k+sm17UeGDMZ3jWtFy2Ko+BagsDR0sNCiiyfstmldKz218o7K3dj02YlRK4NsEWOCmNcF0H4qhORz08Jq8l1Caf4VJPrfH5KWJUsQZXanzQYxHbEeCda5342usHMRSQwSAiZiaVrUZlY+xad0k2j7s28kwHQY4sicHpHYBlWdfu4U/YSsTXcCZzQ2Ao9mUtPjwfFSwTgvQDpQUriF+jYeMeoby+qY3C9QHZCeHvUeUOBQ4LLwkzs2FjFaCKBMAk4enJSzApEbxqrkGcbCfxMRoxZn0sDxvtQKO6RwI/UeKG8/aYC2FAO0Z/GIOS99gLnPLpre6JyRepp9Cvo3T0SGIN/oDwefMgfAmBf7p3evDGDfyMixsfKUrrshHARCfVaCTLGJ5bsN0QDF5HeeDdGL627x6gUOLafiGKPuOsv03Ucptims1+320//DNkw6xAA11VKfO0LpW3LUoD1SvLyxsi1BOFPLUO4OIRcBCPUT6XPxnSyc691ABOVBiKI2R44kR8qXQE2JN4xYtU+asADXujJ16AYyr1EbAXEdKRaRNqWg24od42RQHn7txbI9C7eCK1ZW/b2qG1AQSR8H3U8rnL21oQ2rh5hiAiUuAGSAyXc+UPJJaVEgJMgZdZxbNnqrAN7Q0TAlGZCcPk033iYqLjGSEB7pET74DrcOSguIiHwsw9uyr55H5t3jHARMR25lf2mEj5e+2yj7CWKo9OZSJkM9PA48QIbJFRcRAI6V1oq2Y+zUnvXQdlNhKFIcE/WRdimxDsZUm2LdBNRkbus8eQ6642i8uS6IndZ49Gu05LEHmXXPovbKY8mEhLXAe6DwSgi9YSfsYukmAiXiYENMN4EB4mLiYT2WWSuWM4yFxE1LnNPgFFEIqnU8FH5qP/iKOkRsUGxy0KvXEQi4NgyymVuosZl3G2Pclkg8kRBjA0XgvGNPdOc20f03K92QrJAQp4lEkAy+pH+t9oTVOYucGs+1xjVlt0/BI8kl2R7FKOoB0wK3IkQVOK678oHrzvHELUn85rK+ec5rnNSPJlNREXussb/zXV21vHTmn0Jylv5j0GAIAo3kNRjKVnfUv8x2OL9BQezDxkTjeNuAAAAAElFTkSuQmCC",
      "text/latex": [
       "$\\displaystyle \\left[\\begin{matrix}1\\\\2\\\\3\\\\4\\end{matrix}\\right]$"
      ],
      "text/plain": [
       "⎡1⎤\n",
       "⎢ ⎥\n",
       "⎢2⎥\n",
       "⎢ ⎥\n",
       "⎢3⎥\n",
       "⎢ ⎥\n",
       "⎣4⎦"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "x_sol = TDMA(A, b)\n",
    "print(\"x= \")\n",
    "display_matrix(x_sol)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Ok, I know the previous code is not very efficient! \n",
    "I just wanted to prove that the algorithms work. \n",
    "\n",
    "In order to show how slow is python code compared to routines that call optimized C/fortran, I wrote a wrapper for the LAPACK function **dgtsv** which implements the Thomas algorithm. I called this function *\"Thomas\"*.  \n",
    "You can see that it is about **8 times much faster** than the function I wrote above.\n",
    "\n",
    "On this topic, I wrote an entire notebook about code optimization and speed up. (notebook **A2**)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Solution of Ax=b with the LAPACK function dgtsv. \n",
      " x= \n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAACoAAABkCAYAAADwkaJHAAAACXBIWXMAAA7EAAAOxAGVKw4bAAAGSElEQVR4Ae2cW44VNxCGDxHP0YRIWcBhBwOzAoYdcFlBYAdBeZp5Q7ADYAUIdgA7IMwOYAGRMhllA8n/ma4+bh/f+nomokuy7C6Xy3+X3Xa5jmdunJ2dHW82m89KMXp/fn7+MFYxNU/9fJHObUyv6m7c9CpeqoywT1/9h5nLLyL674v3AL4P9JWQLwmsg0t9v+4w9CAerD2gMIukxkyVd0p3VL4qNvAEJG9W+0vs20ovxKsyjm9RT2W3KGVH4rxRulS6qxSdS+InSTr4Dp4rf49Qo/Oz8vtKRbA/JDV7FVJ0pfRQ6anYb72qqqLaPZHgkXIHkkYqMxo8v+K5RFVAS0oq6lk5LiJyn8Q7FWhGLEtLAT0VCqZNSDbk1GdpdqA11hLCW1mUqpwdqAcit0Jcm6EvGeznksASFo3NTcNlQ866mqXZgTbLECBiw2s8+6iSYGcH2vT8Ufk2gsIsSn2WlgLKlsuOFtIdMS48q4f17fMQoDbxzRqtMpYipX+VOm6jnnE4LpU7B4MGKjPsj5R+5blEVXs9SqQYq0C2OL8Tj7n1QbnzfJSz1cL7A8GAsB5OyIlyPh7ye3qO7Viq6lIfoFUOtDrGK9oj8a/ExFcYREOGflBHYxutQMdaMGy/WjS0yNjn1aJjLRi2Xy0aWmTs8//Gon220K2s8qyxDJ4QDvGz2r2adpIdHICosqg6ACQhn6dNcu6ZeAQQzEkBS5Ikh0f1STkvR5yLl8ahQXeRqoBKC5boOBR0KB6OhnlVKsZJsosFILDaF3VoRwdDhGeOD1qyymIBCAB9FSAsGKPwBUIZXjR2yLOzUnH63Aw1xp4FMOWLHiOv+qTzq7rSS6Bi77QA06faOeq3cWUBACRDbivBnkzDMBCp0UCs+DKDgUo5HxGhc77gsWTnsKQegP7Y1FqeFLYKgSNUyJxNTQkTJY/NTas3a6cCEL+YYG+LChxLzS3lxNeLJDkb8tjwGs8+qqQ+gP7T1FqeFFanHHdvK28tqfKWlGz0rWJoAOJP01ttUYE5VqMT5eHHA/jc8NLX6ABE7fKExejso4CGoexT8dwHpZyh/FuJ6AfbrCOVXyuxdT5Q8mP4BCDuNWLZrAqoNHxQAizzM6QLYwjEYQMQAhANKhhAP0/J8hKS6/gLfrtSuXqOlhTNXb8CndrCq0W/a4uyz7JslHaXqY1Uo4+t1y1pzNGtEruNeTIqXhvC83c74foxTT0mtXv9RlsgU8S2QJwPnvnxgHlURZKdPQABMLwfSwBm7hBAaH+SyaGV3CIBCLymJwEos+TvOYDUqd1iAQhcObwfkiN13paNl8lHByCq5qhAYb2ffCDi2ZC75cOvi5RZZvauC4lnZyXq2/smkfbDLhYIJIpdPErlGIC2L9Uzv0tUXMOrLGq9qFPOTYA8UWI6xH5KFLtDBiI3VYov0xco4EgbgWboCTtyzSg7bMgXqCoAUdARr27AYSV+vM1Z5DKuwXHN2qkARNu0agsVkGNS22pXsKFnOkRJ7WzIYy9jPPuoojpgVgGVHIs1w2yKaduHWDXYyUIyi9qaHNa3z7VAsQoBMXKf7jYPpY5GByBqgbp4u49QoPmYsDBxffcCWFzpcDcg1DmRDiIi/uLOUHJTsbWmyocNQGDJBlALCl6MJBcNVvASkjfvK9Y0y6sd+qySJSpXoFNbebXoatGpLTC1vnWOfrcW7eXh+1bSloh/6i4a+PxcWW3mDUAkOsd1M38yIbJjC+QiAYhdjyqp0986jMKD5BcLQLRQ1ClDjifkfNC2Il8YHYAYsjw9FtjsWT6CmTNV7JBnZ6Xkmct09QLaDLnvPJueZK42R8nKXUVxrlcDVYdb6TUPftdFuWQgclOl+DLVQIWHpajvkJdf45vENAEIAeSr7TXkHsLY3LRqs/b4AEQz5JwubeJbJ1W52tmQx4bXeEXdNTsTc5MLBSzwPrFMcfsBPvdLwgsHviyHQvSEZBYtHhqLQAUAJXuKxOcCARcN2msbIQrvmZex7dNjb7h8MNufYFhHDJsNneMJ9GEDEIaMXGD4qGwYCUpgLW4xvlSy5cuCZzQxwnrz/wmG9SYw2SCC6tcAhBnrWud9dqaDvsgKdGrz+ws+V4ND/dfiPxYACqDss6klp7gHo2Qiiu1crer/AP5TPXUImJZvAAAAAElFTkSuQmCC",
      "text/latex": [
       "$\\displaystyle \\left[\\begin{matrix}1.0\\\\2.0\\\\3.0\\\\4.0\\end{matrix}\\right]$"
      ],
      "text/plain": [
       "⎡1.0⎤\n",
       "⎢   ⎥\n",
       "⎢2.0⎥\n",
       "⎢   ⎥\n",
       "⎢3.0⎥\n",
       "⎢   ⎥\n",
       "⎣4.0⎦"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "print(\"Solution of Ax=b with the LAPACK function dgtsv. \\n x= \")\n",
    "display_matrix(Thomas(A, b))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "15.2 µs ± 188 ns per loop (mean ± std. dev. of 7 runs, 100,000 loops each)\n"
     ]
    }
   ],
   "source": [
    "%%timeit\n",
    "TDMA(A, b)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1.4 µs ± 4.48 ns per loop (mean ± std. dev. of 7 runs, 1,000,000 loops each)\n"
     ]
    }
   ],
   "source": [
    "%%timeit\n",
    "Thomas(A, b)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}