Merge branch 'master' of github.com:zackchase/mxnet-the-straight-dope

chapter02_supervised-learning/perceptron.ipynb

@@ -271,10 +271,10 @@
     "Here the first equality follows from the definition of the weight updates. The next inequality follows from the fact that $(w^*, b^*)$ separate the problem with margin at least $\\epsilon$, and the last inequality is simply a consequence of iterating this inequality $t+1$ times. Growing alignment between the 'ideal' and the actual weight vectors is great, but only if the actual weight vectors don't grow too rapidly. So we need a bound on their length:\n",
     "\n",
     "$$\\begin{eqnarray}\n",
-    "\\|(w_{t+1}, b_{t+1}\\|^2 \\geq & \\|(w_t, b_t)\\|^2 + 2 y_t x_t^\\top w_t + 2 y_t b_t + \\|(x_t, 1)\\|^2 \\\\\n",
+    "\\|(w_{t+1}, b_{t+1})\\|^2 \\leq & \\|(w_t, b_t)\\|^2 + 2 y_t x_t^\\top w_t + 2 y_t b_t + \\|(x_t, 1)\\|^2 \\\\\n",
     "= & \\|(w_t, b_t)\\|^2 + 2 y_t \\left(x_t^\\top w_t + b_t\\right) + \\|(x_t, 1)\\|^2 \\\\\n",
-    "\\geq & \\|(w_t, b_t)\\|^2  + R^2 + 1 \\\\\n",
-    "\\geq & (t+1) (R^2 + 1)\n",
+    "\\leq & \\|(w_t, b_t)\\|^2  + R^2 + 1 \\\\\n",
+    "\\leq & (t+1) (R^2 + 1)\n",
     "\\end{eqnarray}$$\n",
     "\n",
     "Now let's combine both inequalities. By Cauchy-Schwartz, i.e. $\\|a\\| \\cdot \\|b\\| \\geq a^\\top b$ and the first inequality we have that $t \\epsilon \\leq (w_t, b_t)^\\top (w^*, b^*) \\leq \\|(w_t, b_t)\\| \\sqrt{2}$. Using the second inequality we furthermore get $\\|(w_t, b_t)\\| \\leq \\sqrt{t (R^2 + 1)}$. Combined this yields\n",