deepdirtycodes

Experimental and exercising codes for deep learning with TensorFlow.

MNIST on CNN

Tutorial codes for learning the basics of nueral networks and training a simple CNN for image classification.

The code is from the tutorial blogs of Python TensorFlow Tutorial – Build a Neural Network and of Convolutional Neural Networks Tutorial in TensorFlow.

The official tutorial for building CNN with TensorFlow is available here.

Related scripts

• mnist_simple.py - A simple feed-forward neural network for MNIST image classification (%96-97 accuracy).
• mnist_cnn.py - A CNN model with two convolution layers (about 99% accuracy).

Word2Vec

Tutorial codes for training word2vec embeddings with the Skip-Gram model.

Run the script - python word2vec.py

The code is from the Tensorflow Word2Vec Tutorial (Github repo), with minor adaptions.

A very detailed explanation of the code, also a more noob-friendly tutorial of word2vec word embedding, can be found here.

RNN and LSTM

A review of RNN models can be found in this arXiv paper. An introduction of LSTM networks can be found in this blog artical, which is referenced in many tutorials.

A serie of detailed technical tutorials of RNN and LSTM is found here, based on which I experimented the networks. The older tensorflow api used in those articals have been changed in my scripts.

Related scripts

• rnn_basic.py - A simple recurrent neural network learning a time-series of numbers echoing with fixed time delay.
• rnn_lstm.py - The same echo time-serie learned with single/multi-layer lstm network, based on tensorflow api.

Dilated CNN

The network strucure is based on this arXiv paper, where a 1D diated CNN is used to predict financial time series.

Related scripts

• The network is implemented in the tcn module.
• tcn_sp500.py gives a demo of training the network (4 layers with kernel size of 4 and filter size of 1) to do a 1-step-ahead prediction of a single time sequence. The demo also produces log files for tensorboard visualization.