Introduction to the architecture of Alex-Net. The 101 Guide to Alex-Net Neural Network
It’s a type of Deep Convolutional Neural Network that has a large impact on Machine Learning and specifically in application of deep learning to computer Vision.
It was a research work of Alex Krizhesvsky, Geoffrey Hinton and IIya Sutskever at the University of Toronto.
It had architecture very similar to the LeNet proposed by Yann LeCun in 1998 .
WHAT MAKES IT DIFFERENT ?
- Earlier before 2000s mainly two activation functions were used which were sigmoid and tanh ,but here it used ReLu activation Function as it solved the problem of vanishing Gradient .
- Implementation of techniques like Dropout in order to reduce over fitting .
- Used Multiple GPU’s (use two GTX 580 GPU of 3GB RAM ) by parallelism .
- Data Augmentation .
- LRM ( local response normalization ) – But it didn’t made much difference
HOW IT GAINED POPULARITY ?
After it famously won the ImageNet ILSVRC-2012 competition by a very large margin .
The Image-Net (http://www.image-net.org/) is a dataset of over 15 million labelled high resolution images belonging to roughly 22,000 categories.
The dataset was collected by human labelers using Amazon’s Mechanical Turk crowd-sourcing tool.
Number of convolutional Layer – 5
Number of Max-Pool Layer – 3
Number of Fully connected Layers (dense) – 2
Output layer with softmax activation function.
You can use any Weight Initializer , But it’s advisable to use to use :-
- glorot _nomal or glorot_uniform with tanh and sigmoid function.
- He_normal and He_uniform with ReLu function.
Epochs = 90
Used SGD with Momentum with following hyperparameter values .
SGD = 0.01 Learning rate
Momentum = 0.9
It was trained for 6 days continuously on two Nvidia Geforce 580 GPUs.
AlexNet achieved top-1 and top-5t est set error rates of 37.5% and 17.0%5. The best performance achieved during the ILSVRC-2010 competition was 47.1% and 28.2%.
Which shows the winning with great margin .
- Original Research paper – (https://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks.pdf)
- Architecture Image- (https://www.shutterstock.com/image-vector/vector-tech-icon-alexnet-convolutional-neural-1330331141)
- Code in Keras – (http://euler.stat.yale.edu/~tba3/stat665/lectures/lec18/notebook18.html).
ABOUT THE AUTHOR
If You find it interesting!! we would really like to hear from you.
Ping us at Instagramemail@example.com
IF YOU WANT ARTICLES ON ANY TOPICS DM US ON INSTA.
Thanks for reading!!