PROD_Edge_Machine_Learning.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "PROD_Edge_Machine_Learning.ipynb",
      "version": "0.3.2",
      "provenance": [],
      "collapsed_sections": [
        "4h6NFsfUOaUc",
        "djgR1ScbkQZv",
        "omSTDQwaCftQ"
      ],
      "include_colab_link": true
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/noahgift/managed_ml_systems_and_iot/blob/master/PROD_Edge_Machine_Learning.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "metadata": {
        "id": "Hat17x80OVRW",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "# Managed Machine Learning Systems and Internet of Things\n",
        "\n",
        "![hero](https://user-images.githubusercontent.com/58792/45260894-08e67a00-b3a8-11e8-941f-e539cb10f8e1.jpg)"
      ]
    },
    {
      "metadata": {
        "id": "4h6NFsfUOaUc",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "## Pragmatic AI Labs\n",
        "\n"
      ]
    },
    {
      "metadata": {
        "id": "kaAAnCxZTerK",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "![alt text](https://paiml.com/images/logo_with_slogan_white_background.png)\n",
        "\n"
      ]
    },
    {
      "metadata": {
        "id": "ELi2P3zcBWu-",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "This notebook was produced by [Pragmatic AI Labs](https://paiml.com/).  You can continue learning about these topics by:\n",
        "\n",
        "*   Buying a copy of [Pragmatic AI: An Introduction to Cloud-Based Machine Learning](http://www.informit.com/store/pragmatic-ai-an-introduction-to-cloud-based-machine-9780134863917)\n",
        "*   Reading an online copy of [Pragmatic AI:Pragmatic AI: An Introduction to Cloud-Based Machine Learning](https://www.safaribooksonline.com/library/view/pragmatic-ai-an/9780134863924/)\n",
        "*  Watching video [Essential Machine Learning and AI with Python and Jupyter Notebook-Video-SafariOnline](https://www.safaribooksonline.com/videos/essential-machine-learning/9780135261118) on Safari Books Online.\n",
        "* Purchasing video [Essential Machine Learning and AI with Python and Jupyter Notebook- Purchase Video](http://www.informit.com/store/essential-machine-learning-and-ai-with-python-and-jupyter-9780135261095)\n",
        "*   Viewing more content at [noahgift.com](https://noahgift.com/)"
      ]
    },
    {
      "metadata": {
        "id": "jVxkbCuXOZbg",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "u491aIb_3cWE",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "## Edge Machine Learning"
      ]
    },
    {
      "metadata": {
        "id": "BPZbWOdb8QWe",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "\n",
        "\n",
        "*   Low Latency (Role of 5G?)\n",
        "*   Available Offline\n",
        "*   Privacy\n",
        "*   ASIC (Application Specific Integrated Circuits) play key role\n",
        "\n"
      ]
    },
    {
      "metadata": {
        "id": "pvcq8U9VSsXX",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "### Key Industry Players"
      ]
    },
    {
      "metadata": {
        "id": "OfcSnJYhSycx",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "All have \"inference\" based solutions\n",
        "\n",
        "*   AWS\n",
        "*   Google\n",
        "*   Apple\n",
        "*   Intel\n",
        "\n"
      ]
    },
    {
      "metadata": {
        "id": "yylxGK6_3c7u",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "**Sections**\n",
        "\n",
        "\n",
        "\n",
        "*   [Intel Movidius](https://colab.research.google.com/drive/1Q1UY2gK2qfx9aMPGEoRv7sqlzk7zbIJR#scrollTo=LpQ25u7y6wkU)\n",
        "*   [iOS CoreML](https://colab.research.google.com/drive/1Q1UY2gK2qfx9aMPGEoRv7sqlzk7zbIJR#scrollTo=2AjmMkHtDrLC)\n",
        "\n"
      ]
    },
    {
      "metadata": {
        "id": "LpQ25u7y6wkU",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "### Intel Movidius"
      ]
    },
    {
      "metadata": {
        "id": "0ExLFWf165Vl",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "#### Overview of Movidius\n",
        "\n"
      ]
    },
    {
      "metadata": {
        "id": "tM7aVgGn78to",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "\n",
        "[Intel Movidius Neural Compute Stick](https://developer.movidius.com/)\n",
        "\n",
        "\"tiny fanless deep learning device for...AI programming at the edge\"\n",
        "\n",
        "\n",
        "![movidius workflow](https://user-images.githubusercontent.com/58792/45309910-b73e0c80-b4d9-11e8-8c72-e3aa17d61ab9.png)\n",
        "\n",
        "\n",
        "**How To**\n",
        "\n",
        "*  Using an Ubuntu VM is most straightforward path to install SDK\n",
        "\n",
        "![alt text](https://user-images.githubusercontent.com/58792/45322711-f5e5be00-b4fd-11e8-9a19-894d73450f70.png)\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "**Resources:**\n",
        "\n",
        "*   [Neural Network Prototyping Video](https://www.youtube.com/watch?v=FDguqBh3V6g)\n",
        "*   [Getting Started Guide](https://developer.movidius.com/start)\n",
        "* [NCSDK](https://github.com/movidius/ncsdk)\n"
      ]
    },
    {
      "metadata": {
        "id": "qNRF0IP78A54",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "#### Movidius Hello World"
      ]
    },
    {
      "metadata": {
        "id": "j7ZKmDlyPH91",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "Checkout sample applications and run hello world (hello_ncs_py):  https://github.com/movidius/ncappzoo/blob/master/README.md\n",
        "\n",
        "\n",
        "```shell\n",
        "\n",
        "parallels@parallels-vm:~/workspace/ncappzoo/apps/hello_ncs_py$ make run\n",
        "\n",
        "making run\n",
        "python3 hello_ncs.py;\n",
        "Error - no NCS devices detected, verify an NCS device is connected.\n",
        "parallels@parallels-vm:~/workspace/ncappzoo/apps/hello_ncs_py$ make run\n",
        "\n",
        "making run\n",
        "python3 hello_ncs.py;\n",
        "Hello NCS! Device opened normally.\n",
        "Goodbye NCS! Device closed normally.\n",
        "NCS device working.\n",
        "```\n"
      ]
    },
    {
      "metadata": {
        "id": "RlCe2zGx1hCa",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "#### Movidius Real-time Image Classification\n",
        "\n",
        "Example using Caffe model from [ncappzoo Github repo](https://github.com/movidius/ncappzoo)\n",
        "\n"
      ]
    },
    {
      "metadata": {
        "id": "FkwHx8cS1kwm",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "![Movidius](https://user-images.githubusercontent.com/58792/45392092-a1fcd700-b5da-11e8-85d3-35e792fd216f.png)"
      ]
    },
    {
      "metadata": {
        "id": "2AjmMkHtDrLC",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "### iOS CoreML"
      ]
    },
    {
      "metadata": {
        "id": "W_r2CsFMDrLE",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "#### Overview of CoreML\n",
        "\n",
        "\n",
        "![CoreML](https://docs-assets.developer.apple.com/published/7e05fb5a2e/4b0ecf58-a51a-4bfa-a361-eb77e59ed76e.png)\n",
        "\n",
        "[CoreML Docs](https://developer.apple.com/documentation/coreml)\n",
        "\n"
      ]
    },
    {
      "metadata": {
        "id": "BQorH943DrLF",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "#### Key Concepts\n",
        "\n",
        "*  Optimized for on-device performance\n",
        "*  Minimal memory footprint\n",
        "*  Can convert trained models from third-party machine learning framework\n",
        "\n",
        "\n"
      ]
    },
    {
      "metadata": {
        "id": "F-DcJbU177Vj",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "#### Converting Trained Models\n",
        "\n",
        "Example of conversion process:\n",
        "\n",
        "*Install python package:*\n",
        "\n",
        "```python\n",
        "pip install -U coremltools\n",
        "```\n",
        "\n",
        "```python\n",
        "\n",
        "#import coremltools\n",
        "import coremltools\n",
        "\n",
        "#convert model\n",
        "coreml_model = coremltools.converters.caffe.convert('my_caffe_model.caffemodel')\n",
        "\n",
        "#save model to Core ML model format\n",
        "coremltools.utils.save_spec(coreml_model, 'my_model.mlmodel')\n",
        "```\n",
        "\n",
        "\n",
        "**Convertors**\n",
        "\n",
        "\n",
        "\n",
        "*   [MXNet Convertor](https://github.com/apache/incubator-mxnet/tree/master/tools/coreml)\n",
        "*   [TensorFlow Convertor](https://github.com/tf-coreml/tf-coreml)\n",
        "\n",
        "\n",
        "**Resources:**\n",
        "\n",
        "\n",
        "*   [Convert trained Models docs](https://developer.apple.com/documentation/coreml/converting_trained_models_to_core_ml)\n",
        "*   [coremltools](https://pypi.org/project/coremltools/)\n"
      ]
    },
    {
      "metadata": {
        "id": "i2sO5hiqzTNN",
        "colab_type": "code",
        "outputId": "2e4651d4-9c5d-48f9-89c2-2aa4132f5dbb",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 86
        }
      },
      "cell_type": "code",
      "source": [
        "!pip -q install -U coremltools"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "\u001b[31mgoogle-colab 0.0.1a1 has requirement six~=1.11.0, but you'll have six 1.10.0 which is incompatible.\u001b[0m\n",
            "\u001b[31mcufflinks 0.14.6 has requirement plotly>=3.0.0, but you'll have plotly 1.12.12 which is incompatible.\u001b[0m\n",
            "\u001b[31mpymc3 3.6 has requirement joblib<0.13.0, but you'll have joblib 0.13.0 which is incompatible.\u001b[0m\n",
            "\u001b[31mfeaturetools 0.4.1 has requirement pandas>=0.23.0, but you'll have pandas 0.22.0 which is incompatible.\u001b[0m\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "O_mtYudXzc29",
        "colab_type": "code",
        "outputId": "aaffeff7-ad22-4af6-c617-a7f02b3b3b30",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 51
        }
      },
      "cell_type": "code",
      "source": [
        "import coremltools"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "WARNING:root:Keras version 2.2.4 detected. Last version known to be fully compatible of Keras is 2.1.6 .\n",
            "WARNING:root:TensorFlow version 1.12.0 detected. Last version known to be fully compatible is 1.5.0 .\n"
          ],
          "name": "stderr"
        }
      ]
    },
    {
      "metadata": {
        "id": "XN4O6Q7Izj_y",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "coremltools.converters.sklearn.convert?"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "v9oZIawL9q-3",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "#### CoreML Image Classification Demo\n",
        "\n",
        "\n",
        "\n",
        "*   [Classifying Images with Vision and Core ML](https://developer.apple.com/documentation/vision/classifying_images_with_vision_and_core_ml)\n",
        "\n",
        "\n",
        "*Mobilenet.mlmodel (could have been converted)*\n",
        "\n",
        "![Mobilenet.mlmodel](https://user-images.githubusercontent.com/58792/45366919-e1083980-b594-11e8-9b4e-9900da6c452f.png)\n",
        "\n",
        "iOS Simulator Doing Image Classification\n",
        "\n",
        "![iOS Simulator](https://user-images.githubusercontent.com/58792/45366912-dea5df80-b594-11e8-8b67-81571f0f8b44.png)\n",
        "\n",
        "To use MXNet models:\n",
        "\n",
        "https://github.com/apache/incubator-mxnet/tree/master/tools/coreml\n",
        "\n",
        "\n",
        "\n"
      ]
    },
    {
      "metadata": {
        "id": "djgR1ScbkQZv",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "## FAQ/Homework"
      ]
    },
    {
      "metadata": {
        "id": "Pra79jnAkUjo",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "Do you need Apple to convert to CoreML?"
      ]
    },
    {
      "metadata": {
        "id": "alPh481Fkgd9",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "If a neural net can be trained in the cloud and then run on a phone in an edge scenario, then what is moved to the edge device? What gets transferred after training and placed on the device for use?"
      ]
    },
    {
      "metadata": {
        "id": "FNiX_Tf5k3PX",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "Can you please point me to the best available models for agriculture using edge based ML?"
      ]
    },
    {
      "metadata": {
        "id": "ZKF3Vh-Bl3e9",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "What is stored in the file transferred from the cloud training model to the edge device. In other words, what is the nature of the model output that would allow functionality on the edge device? Is it a tensor with a legend? "
      ]
    },
    {
      "metadata": {
        "id": "omSTDQwaCftQ",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "## References\n",
        "\n",
        "\n",
        "\n"
      ]
    },
    {
      "metadata": {
        "id": "r6Nz9zVH8nWO",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "\n",
        "*   [Pragmatic AI: An Introduction to Cloud-Based Machine Learning-Physical Book](http://www.informit.com/store/pragmatic-ai-an-introduction-to-cloud-based-machine-9780134863917)\n",
        "*   [Pragmatic AI: Pragmatic AI: An Introduction to Cloud-Based Machine Learning-SafariOnline Book](https://www.safaribooksonline.com/library/view/pragmatic-ai-an/9780134863924/)\n",
        "*  [Essential Machine Learning and AI with Python and Jupyter Notebook-Video-SafariOnline](https://www.safaribooksonline.com/videos/essential-machine-learning/9780135261118)\n",
        "* [Essential Machine Learning and AI with Python and Jupyter Notebook- Purchase Video](http://www.informit.com/store/essential-machine-learning-and-ai-with-python-and-jupyter-9780135261095)\n",
        "*  [noahgift.com](https://noahgift.com/)\n",
        "*  [paiml.com](https://paiml.com/)\n",
        "*  https://storage.googleapis.com/nexttpu/index.html\n",
        "* https://riscv.org/wp-content/uploads/2017/05/Mon0915-RISC-V-50-Years-Computer-Arch.pdf\n",
        "* https://software.intel.com/en-us/ai-academy/devcloud\n"
      ]
    }
  ]
}