III_06_Linear_transformations.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "+ This notebook is part of lecture 30 *Linear transformations and their matrices* in the OCW MIT course 18.06 by Prof Gilbert Strang [1]\n",
    "+ Created by me, Dr Juan H Klopper\n",
    "    + Head of Acute Care Surgery\n",
    "    + Groote Schuur Hospital\n",
    "    + University Cape Town\n",
    "    + <a href=\"mailto:juan.klopper@uct.ac.za\">Email me with your thoughts, comments, suggestions and corrections</a> \n",
    "<a rel=\"license\" href=\"http://creativecommons.org/licenses/by-nc/4.0/\"><img alt=\"Creative Commons Licence\" style=\"border-width:0\" src=\"https://i.creativecommons.org/l/by-nc/4.0/88x31.png\" /></a><br /><span xmlns:dct=\"http://purl.org/dc/terms/\" href=\"http://purl.org/dc/dcmitype/InteractiveResource\" property=\"dct:title\" rel=\"dct:type\">Linear Algebra OCW MIT18.06</span> <span xmlns:cc=\"http://creativecommons.org/ns#\" property=\"cc:attributionName\">IPython notebook [2] study notes by Dr Juan H Klopper</span> is licensed under a <a rel=\"license\" href=\"http://creativecommons.org/licenses/by-nc/4.0/\">Creative Commons Attribution-NonCommercial 4.0 International License</a>.\n",
    "\n",
    "+ [1] <a href=\"http://ocw.mit.edu/courses/mathematics/18-06sc-linear-algebra-fall-2011/index.htm\">OCW MIT 18.06</a>\n",
    "+ [2] Fernando Pérez, Brian E. Granger, IPython: A System for Interactive Scientific Computing, Computing in Science and Engineering, vol. 9, no. 3, pp. 21-29, May/June 2007, doi:10.1109/MCSE.2007.53. URL: http://ipython.org"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<link href='http://fonts.googleapis.com/css?family=Alegreya+Sans:100,300,400,500,700,800,900,100italic,300italic,400italic,500italic,700italic,800italic,900italic' rel='stylesheet' type='text/css'>\n",
       "<link href='http://fonts.googleapis.com/css?family=Arvo:400,700,400italic' rel='stylesheet' type='text/css'>\n",
       "<link href='http://fonts.googleapis.com/css?family=PT+Mono' rel='stylesheet' type='text/css'>\n",
       "<link href='http://fonts.googleapis.com/css?family=Shadows+Into+Light' rel='stylesheet' type='text/css'>\n",
       "<link href='http://fonts.googleapis.com/css?family=Philosopher:400,700,400italic,700italic' rel='stylesheet' type='text/css'>\n",
       "\n",
       "<style>\n",
       "\n",
       "@font-face {\n",
       "    font-family: \"Computer Modern\";\n",
       "    src: url('http://mirrors.ctan.org/fonts/cm-unicode/fonts/otf/cmunss.otf');\n",
       "}\n",
       "\n",
       "#notebook_panel { /* main background */\n",
       "    background: #ddd;\n",
       "    color: #000000;\n",
       "}\n",
       "\n",
       "\n",
       "\n",
       "/* Formatting for header cells */\n",
       ".text_cell_render h1 {\n",
       "    font-family: 'Philosopher', sans-serif;\n",
       "    font-weight: 400;\n",
       "    font-size: 2.2em;\n",
       "    line-height: 100%;\n",
       "    color: rgb(0, 80, 120);\n",
       "    margin-bottom: 0.1em;\n",
       "    margin-top: 0.1em;\n",
       "    display: block;\n",
       "}\t\n",
       ".text_cell_render h2 {\n",
       "    font-family: 'Philosopher', serif;\n",
       "    font-weight: 400;\n",
       "    font-size: 1.9em;\n",
       "    line-height: 100%;\n",
       "    color: rgb(200,100,0);\n",
       "    margin-bottom: 0.1em;\n",
       "    margin-top: 0.1em;\n",
       "    display: block;\n",
       "}\t\n",
       "\n",
       ".text_cell_render h3 {\n",
       "    font-family: 'Philosopher', serif;\n",
       "    margin-top:12px;\n",
       "    margin-bottom: 3px;\n",
       "    font-style: italic;\n",
       "    color: rgb(94,127,192);\n",
       "}\n",
       "\n",
       ".text_cell_render h4 {\n",
       "    font-family: 'Philosopher', serif;\n",
       "}\n",
       "\n",
       ".text_cell_render h5 {\n",
       "    font-family: 'Alegreya Sans', sans-serif;\n",
       "    font-weight: 300;\n",
       "    font-size: 16pt;\n",
       "    color: grey;\n",
       "    font-style: italic;\n",
       "    margin-bottom: .1em;\n",
       "    margin-top: 0.1em;\n",
       "    display: block;\n",
       "}\n",
       "\n",
       ".text_cell_render h6 {\n",
       "    font-family: 'PT Mono', sans-serif;\n",
       "    font-weight: 300;\n",
       "    font-size: 10pt;\n",
       "    color: grey;\n",
       "    margin-bottom: 1px;\n",
       "    margin-top: 1px;\n",
       "}\n",
       "\n",
       ".CodeMirror{\n",
       "        font-family: \"PT Mono\";\n",
       "        font-size: 100%;\n",
       "}\n",
       "\n",
       "</style>\n",
       "\n"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from IPython.core.display import HTML, Image\n",
    "css_file = 'style.css'\n",
    "HTML(open(css_file, 'r').read())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sympy import init_printing, Matrix, symbols, sqrt, Rational\n",
    "from warnings import filterwarnings"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "init_printing(use_latex = 'mathjax')\n",
    "filterwarnings('ignore')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Linear transformations"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Mappings / transformations / projections"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "+ We'll begin the chapter with a familiar example: Projections"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "+ Let's look at a mapping / transformation / projection\n",
    "    + T: &#8477;<sup>2</sup> &#8594; T: &#8477;<sup>2</sup>\n",
    "    + No axes are required for this mapping\n",
    "+ Another example using matrices\n",
    "    $$ T\\left({v}\\right)={A}{v} $$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "+ For these to be a linear mapping, the following must apply\n",
    "$$  {T}\\left({v}+{w}\\right)={T}\\left({v}\\right)+{T}\\left({w}\\right) \\\\ {T}\\left({cv}\\right)={c}{T}\\left({v}\\right) $$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "+ Example that are not linear transformations\n",
    "    + Shifting the axes\n",
    "    + The transformation that turns a vector into its length\n",
    "    + Transformation involving power or transcendental function"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Example"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "+ Let's create and example which accomplishes the following\n",
    "    + T: &#8477;<sup>3</sup> &#8594; &#8477;<sup>2</sup>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "+ Without axes we could look at something like this\n",
    "    $$ {f}\\left({x},{y},{z}\\right)=\\left({x}+{y}+{z},\\quad{3x}-{2y}+{z}\\right) $$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "+ With axes we naturally turn to matrices\n",
    "    + We require **v** in 3-space\n",
    "    $$ \\begin{bmatrix}{v}_{1}\\\\{v}_{2}\\\\{v}_{3}\\end{bmatrix} $$\n",
    "    + To have a resultant vector in 2-space we require a matrix A of size 2&times;3\n",
    "    $$ T\\left( v \\right) =Av\\\\ { \\begin{bmatrix} { a }_{ 11 } & { a }_{ 12 } & { a }_{ 13 } \\\\ { a }_{ 21 } & { a }_{ 22 } & { a }_{ 23 } \\end{bmatrix} }_{ 2\\times 3 }{ \\begin{bmatrix} { v }_{ 1 } \\\\ { v }_{ 2 } \\\\ { v }_{ 3 } \\end{bmatrix} }_{ 3\\times 1 }={ \\begin{bmatrix} { a }_{ 11 }{ v }_{ 1 }+{ a }_{ 12 }{ v }_{ 2 }+{ a }_{ 13 }{ v }_{ 3 } \\\\ { a }_{ 21 }{ v }_{ 1 }+{ a }_{ 22 }{ v }_{ 2 }+{ a }_{ 23 }{ v }_{ 3 } \\end{bmatrix} }_{ 2\\times 1 } $$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "+ Notice if we know what a linear transformation does to a single vector, we know what it does to constant multiples of that vector (from the property T(c**v**)=cT(**v**))\n",
    "    + In 2- or 3-space this would represent a line\n",
    "+ If we knew what T does to two (linearly independent or basis) vectors, we know what it does to the subspace created by those two vectors (the whole plane in &#8477;<sup>2</sup> or a plane in &#8477;<sup>3</sup>), i.e. linear combinations of these two vectors"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Coordinates / basis"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "+ Coordinates originate from a basis\n",
    "$$ {v}={c}_{1}{v}_{1}+{c}_{2}{v}_{2}+\\dots+{c}_{n}{v}_{n} $$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "+ This need not be the standard basis, though\n",
    "$$ v=\\begin{bmatrix} 2 \\\\ 3 \\\\ -1 \\end{bmatrix}=2\\begin{bmatrix} 1 \\\\ 0 \\\\ 0 \\end{bmatrix}+3\\begin{bmatrix} 0 \\\\ 1 \\\\ 0 \\end{bmatrix}-\\begin{bmatrix} 0 \\\\ 0 \\\\ 1 \\end{bmatrix} $$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "+ Let's look then at the following transformation: T: &#8477;<sup>n</sup>&#8594;&#8477;<sup>m</sup>\n",
    "+ We need a basis for the input (*v*<sub>1</sub>,*v*<sub>2</sub>,...,*v*<sub>n</sub> in &#8477;<sup>n</sup>) and the output (*w*<sub>1</sub>,*w*<sub>2</sub>,...,*w*<sub>m</sub> in &#8477;<sup>m</sup>)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "+ So what is the rule for finding A that will transform **v** into **w**?\n",
    "+ Rule finding A given basis for **v** and **w**\n",
    "    + The first column of A: apply T(*v*<sub>1</sub>)=*a*<sub>11</sub>*w*<sub>1</sub>+*a*<sub>21</sub>*w*<sub>2</sub>+...+*a*<sub>m1</sub>*w*<sub>m</sub>, with the first column entries of A being *a*<sub>11</sub>, *a*<sub>21</sub>, ..., *a*<sub>m1</sub>\n",
    "    + The second column of A: apply T(*v*<sub>2</sub>)=*a*<sub>12</sub>*w*<sub>1</sub>+*a*<sub>22</sub>*w*<sub>2</sub>+...+*a*<sub>m2</sub>*w*<sub>m</sub>, with the second column entries of A being *a*<sub>12</sub>, *a*<sub>22</sub>, ..., *a*<sub>m2</sub>\n",
    "    + ... and so on until *v*<sub>n</sub>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Example"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "+ Let's look at polynomials with the following transformation\n",
    "$$ T=\\frac{d}{dx} $$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "+ The input might be\n",
    "$$ {c}_{1}+{c}_{2}{x}+{c}_{3}{x}^{2} $$\n",
    "+ The output will be\n",
    "$$ {c}_{2}+2{c}_{3}{x} $$\n",
    "+ This gives use the following basis for each\n",
    "$$ {1},{x},{x}^{2} \\\\ {1},{x} $$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "+ We need the following\n",
    "$$ {A}\\begin{bmatrix}{c}_{1}\\\\{c}_{2}\\\\{c}_{3}\\end{bmatrix}=\\begin{bmatrix}{c}_{2}\\\\2{c}_{2}\\end{bmatrix} $$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Following the rule above we will have to have the following\n",
    "$$ {A}=\\begin{bmatrix}0&1&0\\\\0&0&2\\end{bmatrix} $$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "## Example problems"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Example problem 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "+ Consider the 2&times;2 matrix A and let T(A)=A<sup>T</sup>\n",
    "    + Why is T linear and what is T<sup>-1</sup>\n",
    "    + Write the matrix of T in:\n",
    "    $$ { v }_{ 1 }=\\begin{bmatrix} 1 & 0 \\\\ 0 & 0 \\end{bmatrix},{ v }_{ 2 }=\\begin{bmatrix} 0 & 1 \\\\ 0 & 0 \\end{bmatrix},{ v }_{ 3 }=\\begin{bmatrix} 0 & 0 \\\\ 1 & 0 \\end{bmatrix},{ v }_{ 4 }=\\begin{bmatrix} 0 & 0 \\\\ 0 & 1 \\end{bmatrix}\\\\ { w }_{ 1 }=\\begin{bmatrix} 1 & 0 \\\\ 1 & 0 \\end{bmatrix},{ w }_{ 2 }=\\begin{bmatrix} 0 & 0 \\\\ 0 & 1 \\end{bmatrix},{ w }_{ 3 }=\\begin{bmatrix} 0 & 1 \\\\ 1 & 0 \\end{bmatrix},{ w }_{ 4 }=\\begin{bmatrix} 0 & 1 \\\\ -1 & 0 \\end{bmatrix} $$\n",
    "    + What are the eigenvalues and eigenvectors of T?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Solution"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "+ From the properties of linear transformation we have the following\n",
    "$$ {T}\\left({A+B}\\right)={\\left({A+B}\\right)}^{T}={A}^{T}+{B}^{T}={T}\\left({A}\\right)+{T}\\left({B}\\right) \\\\ {T}\\left({cA}\\right)={\\left({cA}\\right)}^{T}={c}{T}\\left({A}\\right) $$\n",
    "    + A transpose turns a row into a columns and *vice versa*, from which we infer the following\n",
    "    $$ {T}^{2}={I} \\\\ \\therefore\\quad{T}^{-1}={T} $$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "+ For the next question we will have to see what T does to each basis matrix\n",
    "$$ {T}\\left({v}_{1}\\right)={v}_{1} \\\\ {T}\\left({v}_{2}\\right)={v}_{3} \\\\ {T}\\left({v}_{3}\\right)={v}_{2} \\\\ {T}\\left({v}_{4}\\right)={v}_{4} $$\n",
    "    + From this we have to form a matrix\n",
    "        + Think of the columns each being T*v*<sub>1</sub>, T*v*<sub>2</sub>, T*v*<sub>3</sub>, and T*v*<sub>4</sub>\n",
    "        + We see that transforming *v*<sub>1</sub> takes 1 *v*<sub>1</sub>, and none of the rest\n",
    "        $$ {M}_{T}=\\begin{bmatrix}1&0&0&0\\\\0&0&1&0\\\\0&1&0&0\\\\0&0&0&1\\end{bmatrix} $$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "+ For the *w*<sub>i</sub> we will have the following matrix\n",
    "$$ {M}_{T}=\\begin{bmatrix}1&0&0&0\\\\0&1&0&0\\\\0&0&1&0\\\\0&0&0&-1\\end{bmatrix} $$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "+ For the last question we will have the following\n",
    "$$ {T}\\left({v}\\right)={\\lambda}{v} $$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.4.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}