This was my final project for Introduction to Machine Learning at the University of Vermont. I wanted to prove to myself that I thoroughly understood the calculus and linear algebra underlying neural networks (and especially backpropagation) so I implemented this from scratch. This may be a good resource if you are trying to do the same.
The project is tailored to the MNIST dataset of handwritten digits. Logic is separated into three classes, Data.py, NeuralNetwork.py, and Trainer.py. The Main.py file conducts the interaction between the aforementioned three classes.
Sample output:
INITIALIZING DATA
Importing training data...
Importing testing data...
Normalizing data
Successfully loaded normalizedData.npz
Converting target data to one-hot encoding...
DATA INITIALIZATION COMPLETE
*******************************
* INITIALIZING NEURAL NETWORK *
*******************************
Input layer size: 784
Hidden layer size: 128
Output layer size: 10
Number of hidden layers: 2
Total number of synapses: 101760
***************************
* TRAINING NEURAL NETWORK *
***************************
Epochs: 12
Iterations per epoch: 937
Batch size: 64
Learning rate: 0.02
epoch training testing
0 15.23% 16.26%
1 74.31% 74.51%
2 89.61% 89.5%
3 92.39% 92.1%
4 94.06% 93.54%
5 95.15% 94.51%
6 95.83% 95.17%
7 96.41% 95.41%
8 96.85% 95.69%
9 97.29% 95.92%
10 97.56% 96.13%
11 97.88% 96.25%