This tutorial introduces the programming language Julia and the Knet deep learning framework. By the end, the reader should be able to define, train, evaluate, and visualize basic MLP, CNN, and RNN models. Each notebook is written to work stand-alone but they rely on concepts introduced in earlier notebooks, so I recommend reading them in order. Every Knet function outside of the standard Julia library is defined or explained before use.
To run the notebooks on your computer, install and run IJulia by typing the following at the
julia> prompt (see IJulia.jl for more
information):
julia> using Pkg; Pkg.add("IJulia"); Pkg.add("Knet")
julia> using IJulia, Knet
julia> notebook(dir=Knet.dir("tutorial"))
To run the notebooks in the cloud you can use Google Colab, or services like AWS. To run on Colab copy the notebooks to the "Colab Notebooks" folder in your Google Drive, then open and follow the instructions in the JuliaOnColab.ipynb notebook. To run on AWS follow the instructions in the Knet Installation Section.
Contents:
- Julia is fast: comparison of Julia's speed to C, Python and numpy.
- Getting to know Julia: basic Julia tutorial from JuliaBox.
- Quick start: if you are familiar with other deep learning frameworks and want to see a quick Julia example.
- The MNIST dataset: introduction to the MNIST handwritten digit recognition dataset.
- Julia iterators: iterators are useful for generating and training with data.
- Creating a model: define, train, visualize simple linear models, introduce gradients, SGD, using the GPU.
- Multilayer perceptrons: multi layer perceptrons, nonlinearities, model capacity, overfitting, regularization, dropout.
- Convolutional networks: convolutional neural networks, sparse and shared weights using conv4 and pool operations.
- Recurrent networks: introduction to recurrent neural networks.
- IMDB sentiment analysis: a simple RNN sequence classification model for sentiment analysis of IMDB movie reviews.
- Language modeling: a character based RNN language model that can write Shakespeare sonnets and Julia programs.
- Sequence to sequence: a sequence to sequence RNN model typically used for machine translation.