ImageNet Classification with Deep Convolutional Neural Networks

Alex Krizhevsky, Ilya Sutskever, Geoffrey E Hinton

Introduction

Image classification plays an important role in MIR, and there have been several works to do that. This paper proposed a method that using deep convolutional neural networks to do image classification, and got a great improvement.

Method

The author applied Rectified Linear Units (ReLU) to every neurons. The formulation of this function is simple :

f(x) = max(0,x),

then applied local response normalization after ReLU :

The overall architecture looks like this :

To reduce overfitting, the author applied data augmentation and dropout. For data augmentation, they simply do some editing on the original image, such as translation and horizontal reflection. Dropout means the neurons have a certain probability no to propagate to the next layer. 

In the learning step, the author used the following formulation :

The variable v in the first equation is the  momentum, and we add it to w to update our weight. 

Performance

We can see the error rates on the dataset is lower than other methods.

Result