In this project, we train our own ANN with one hidden layer using backpropagation algorithm on MNIST dataset which later can help our machine to identify a given numeric digit.
Information and documentation of the dataset can be found on: http://yann.lecun.com/exdb/mnist/
Arrangement like this is done so as to maintain degree of randomness in our dataset thus weight distribution becomes uniform for all types of digits
The backpropagation algorithm was originally introduced in the 1970s, but its importance wasn't fully appreciated until a famous 1986 paper by David Rumelhart, Geoffrey Hinton, and Ronald Williams. That paper describes several neural networks where backpropagation works far faster than earlier approaches to learning, making it possible to use neural nets to solve problems which had previously been insoluble. Today, the backpropagation algorithm is the workhorse of learning in neural networks.
This chapter is more mathematically involved than the rest of the book. If you're not crazy about mathematics you may be tempted to skip the chapter, and to treat backpropagation as a black box whose details you're willing to ignore. Why take the time to study those details?
Link for step-by-step backpropagation execution. The contents of the blog are also present in the theory folder. (Just in case ¯＼(º_o)/¯ )
The above is the squared error function used to calculate the error. For each iteration, we check the error obtained in the previous iteration and the current iteration and the difference is then compared with a momentum value. If this goes below a value, we stop the process.
Classwise accuracies achieved are:
Accuracy for class 0 is 98.469388 percent
Accuracy for class 1 is 98.149780 percent
Accuracy for class 2 is 95.930233 percent
Accuracy for class 3 is 95.049505 percent
Accuracy for class 4 is 94.602851 percent
Accuracy for class 5 is 93.609865 percent
Accuracy for class 6 is 96.764092 percent
Accuracy for class 7 is 94.844358 percent
Accuracy for class 8 is 96.201232 percent
Accuracy for class 9 is 94.449950 percent
The overall accuracy obtained is 95.85%
DataSet/mnist_all.mat: Matfile containing processed MNIST dataset containing 10 classes (0-9) having 60000 training images and 10000 testing images.
DataSet/DigitRecognition.mat: Matfile containing the final weight matrices after completely training the model after the absolute error is unity. When tested using these weight matrices, we get the accuracy discussed above.
DataSet/trainSet.mat: Refer cotent under the heading Initializing The Dataset.
Code/imageProjection: Visualizing the dataset.
Code/init: Initializing the dataset as explained under the heading Initialing The Dataset.
Code/backPropagation: Declaring the weight matrix and creating the neural network. Calls the feedForward.m method and feeds the input using wait matrix to the hidden layer which further feeds forward to the output layer. Takes the total error at the output layer and corrects the error.In short, performs the backpropagation algorithm.
Code/feedForward: Method that uses the formula input*weight + bias and later uses the sigmoidal function to normalise the value.
Code/Test: Testing the 10000 test images over the trained weights obtained from the module backPropagation.m.
Code/DigitRecognition: Main Method that triggers all the other method.
Refer summary and statistics file for more details.
I would like to show my deepest and humble gratitude towards my mentor, Professor Akshay Rajput who helped me in all sorts. Much thanks! ಥ‿ಥ
Feel free to drop any suggestions at parthendo@geu.ac.in