Implementation of Deep learning model

Study materials

• How to: How to use markdown (for using github)
• How to: Jump to Python (Korean)
• How to: Jupyter Notebook
• How to: Jupyter Notebook Shortcuts
• How to: Matplotlib
• [Tips: Python tips]
• Fastcampus lecture notes by Gunho Choi
• Deep Learning for beginners by Gunho Choi
• Distill (visualizing)
• 라온피플 블로그 (Korean)

Hierarchy of deep learning milestones

1 Pytorch basics

• Python basics
• Pytorch basics
• matplotlib

2 Neural Network (NN)

• Perceptron
  • Activation functions (sigmoid, softmax, & ReLU)
• Multi-Layer Perceptron
  • Backpropagation
• Deep Neural Network

3 Basic concepts for NN modeling

• Data Segmentation: Train/Validation/Test
• Overfitting & Underfitting
  • Weight decay (Regularization)
  • Dropout
  • Input data transform
  • Learning rate decay
• Convergence
  • Initialization
  • Batch Normalization
• Optimization algorithm (Momentum, Nasterov, SGD, & Adam)

4 Datasets

• Dataset: MNIST (10 classes, 28x28x1 handwriting images)
• Dataset: ILSVRC (1,000 classes, 224x224x3 object images)

5 Convolutional Neural Network (CNN) series

• Naive CNN
  • Convolution
  • Pooling
• AlexNet
• ZFNet
• VGGNet
• GoogLeNet
  • Inception module
  • Network In Network
• ResNet
  • ResNet module
  • Bottleneck Architecture
• DenseNet
• ShuffleNet
  • Channel shuffle
  • Depthwise Seperable Convolution
• Xception
• MobileNet
• MobileNetV2

6 Recurrent Neural Network (RNN) series

• Naive RNN
• Long Short-Term Memory (LSTM)
• GRU
• Dynamic RNN
• Bidirectional RNN

7 Recurrent Convolutional Neural Network (RCNN) series

8 Autoencoder series

• Restricted Boltzman Machine (RBM)
• Deep Beilief Network (DBN)
• Convolutional Autoencoder (CAE, CNN + Autoencoder)
• Denoising Convolutional Autoencoder
• Variational Autoencoder (VAE)

9 GAN

• GAN, DCGAN 개념
• Generative Adversarial Nets (GAN) 구현
• DCGAN