WHAT IS ALEX-NET ? BEGINNERS GUIDE TO ALEX-NET

WHAT IS ALEX-NET ? BEGINNERS GUIDE TO ALEX-NET

Introduction to the architecture of Alex-Net. The 101 Guide to Alex-Net Neural Network

It’s a type of Deep Convolutional Neural Network that has a large impact on Machine Learning and specifically in application of deep learning to computer Vision.

It was a research work of Alex Krizhesvsky, Geoffrey Hinton and IIya Sutskever at the University of Toronto.

It had architecture very similar to the LeNet proposed by Yann LeCun in 1998 .


WHAT MAKES IT DIFFERENT ?

  1. Earlier before 2000s mainly two activation functions were used which were sigmoid and tanh ,but here it used ReLu activation Function as it solved the problem of vanishing Gradient .
  2. Implementation of techniques like Dropout in order to reduce over fitting .
  3. Used Multiple GPU’s (use two GTX 580 GPU of 3GB RAM ) by parallelism .
  4. Data Augmentation .
  5. LRM ( local response normalization ) – But it didn’t made much difference

HOW IT GAINED POPULARITY ?

After it famously won the ImageNet ILSVRC-2012 competition by a very large margin .

DATASET

The Image-Net (http://www.image-net.org/) is a dataset of over 15 million labelled high resolution images belonging to roughly 22,000 categories.

The dataset was collected by human labelers using Amazon’s Mechanical Turk crowd-sourcing tool.


ARCHITECTURE

Source ShutterStock

Number of convolutional Layer – 5

Number of Max-Pool Layer – 3

Number of Fully connected Layers (dense) – 2

Output layer with softmax activation function.

You can use any Weight Initializer , But it’s advisable to use to use :-

  1. glorot _nomal or glorot_uniform with tanh and sigmoid function.
  2. He_normal and He_uniform with ReLu function.

TRAINING

Epochs = 90

Used SGD with Momentum with following hyperparameter values .

SGD = 0.01 Learning rate

Momentum = 0.9

It was trained for 6 days continuously on two Nvidia Geforce 580 GPUs.


RESULTS

AlexNet achieved top-1 and top-5t est set error rates of 37.5% and 17.0%5. The best performance achieved during the ILSVRC-2010 competition was 47.1% and 28.2%.

Which shows the winning with great margin .


REFERENCES


ABOUT THE AUTHOR

Atharv Shah , is an undergrad pursuing Btech in computer science , he is fascinated by the artificial intelligence and it’s application used to solve real world problems .Atharv Shah


ATHARV SHAH

REVIEWS

If You find it interesting!! we would really like to hear from you.

Ping us at Instagram/@the.blur.code

IF YOU WANT ARTICLES ON ANY TOPICS DM US ON INSTA.

Thanks for reading!! Happy Coding

Leave a Reply

Your email address will not be published. Required fields are marked *

Related Posts

Begin typing your search term above and press enter to search. Press ESC to cancel.

Back To Top
Hey, wait!

Before you dive into the exciting Article, wanna join our newsletter list, We promise we will not spam