AlexNet

Tensorflow 2.x Implementation of the original AlexNet Paper
Paper Link · Architecture · Data Loader · Model Trainer

Table of Contents

• • Features
  • Hardware Requirements
    • Hardware used for Development and Testing
    • Minimum Hardware Requirements
  • Dataset Download & Setup
    • Dataset Download
    • Raw/Source Dataset Directory Structure
    • Processed/Destination Dataset Directory Structure
  • References
  • Contributing
  • License
  • Contact
    • Animikh Aich

About The Project

AlexNet is a Deep Learning Paper published in the year 2012 by Alex Krizhevsky (Hence, the name). It is one of the pioneer Deep Learning Publications which kick started the Deep Learning Research and proved its importance.

This implementation is a part of my learning where I take an attempt to implement Key Deep Learning Papers using Tensorflow or PyTorch.

The Original Literature Can be found here: AlexNet Paper

Paper Explained Here: ImageNet Classification with Deep Convolutional Neural Networks - YouTube

Quick Links

• The Original Literature Can be found here: AlexNet Paper
• Paper Explained Here: ImageNet Classification with Deep Convolutional Neural Networks - YouTube
• Data Loader: GitHub | nbViewer
• Model Architecture: GitHub | nbViewer
• Model Trainer: GitHub | nbViewer
• Trained Weights: Epoch 35, Categorical Accuracy 0.48

Training/Validation Resources

Trained Weights

The final Trained Weights after automatic early stopping can be found here: Epoch 35, Categorical Accuracy 0.48

Legend

• Orange = Training Curves
• Blue = Validation Curves

Training Curves

Features

• Tf.Data - Optimized Tensorflow Data Pipelining using Tf.Data and Tensorflow Datasets
• Vanilla Implementation - I have taken an attempt to implement the original AlexNet Architecture as resembling the original literature as closely as possible.
• Fused Implementation - Since the Original Implementation was not very efficient for a single GPU training, hence, I have also implementation a Fused Model which closely resembles the original, but is a single straight network designed to run on a single GPU.
• Mixed Precision Training - The Implementation uses FP16 - Mixed Precision Training since that uses the Tensor Cores of the Nvidia GPU with Compute Capability 7.0 or higher.

Hardware Requirements

Nvidia GPU for Training is recommended, However, it can work with CPUs as well (Not Recommended, ImageNet is Huge. It will probably take over a year to Train)

Hardware used for Development and Testing

• CPU: AMD Ryzen 7 3700X - 8 Cores 16 Threads
• GPU: Nvidia GeForce RTX 2080 Ti 11 GB
• RAM: 32 GB DDR4 @ 3200 MHz
• Storage: 1 TB NVMe SSD
• OS: Ubuntu 20.04

Minimum Hardware Requirements

• CPU: AMD/Intel 4 Core CPU (CPU will become a bottleneck here)
• GPU: Nvidia GeForce GTX 1660 6 GB (You can go lower, but I would not recommend it)
• RAM: 16 GB
• Storage: Minimum of 500 GB SSD (HDD is Not Recommended)
• OS: Any Linux Distribution

Dataset Download & Setup

Enough HDD/SSD space is required for the following:

• Downloading Raw Dataset - 156.8 GB
• Convert to TFRecord and Store - 155.9 GB
• Total Storage Required - 312.7 GB

An SSD is recommended and a Mechanical HDD should be avoided since it will slow down the data loader significantly.

Dataset Download

ImageNet Download Link: Download ImageNet Dataset

• Download Train Images (Required): ILSVRC2012_img_train.tar - Size 137.7 GB
• Download Val Images (Required): ILSVRC2012_img_val.tar - Size 6.3 GB
• Download Train Images (Optional): ILSVRC2012_img_test.tar - Size 12.7 GB

Raw/Source Dataset Directory Structure

Download the dataset from the above link and put it in the folder like shown:

imagenet2012/
├── ILSVRC2012_img_test.tar
├── ILSVRC2012_img_train.tar
└── ILSVRC2012_img_val.tar

Processed/Destination Dataset Directory Structure

Create another folder and create the folders data, download & extracted like shown:

imagenet/
├── data/
├── downloaded/
└── extracted/

References

• Source Literature: AlexNet Paper
• Paper Explanation: ImageNet Classification with Deep Convolutional Neural Networks - YouTube
• Download the Dataset by Signing up with your Institution Email Addreess: Download ImageNet Dataset
• Tensorflow Datasets ImageNet2012 Dataset: API Docs
• Efficient Data Pipelining using: Tf.Data
• Tf.Data Training and Testing Split: Slice & Split Docs
• Data Loader Template taken from my own Repository: Zero Coding Tf Classifier

Contributing

Contributions are what make the open source community such an amazing place to be learn, inspire, and create. Any contributions you make are greatly appreciated.

1. Fork the Project
2. Create your Feature Branch (git checkout -b feature/AmazingFeature)
3. Commit your Changes (git commit -m 'Add some AmazingFeature')
4. Push to the Branch (git push origin feature/AmazingFeature)
5. Open a Pull Request

License

Distributed under the GNU AGPL V3 License. See LICENSE for more information.

Contact

Animikh Aich

• Website: Animikh Aich - Website
• LinkedIn: animikh-aich
• Email: animikhaich@gmail.com
• Twitter: @AichAnimikh