Scribd Logo
Upload

Welcome to Scribd!

  • Artificial Intelligence: Alexnet

    Uploaded by

    Thành Cao Đức
    20 views20 pages
    The document describes AlexNet, which was the winner of the ImageNet LSVRC-2012 competition for image classification. AlexNet used a neural network architecture with 5 convolutional layers and 3 fully-connected layers. It achieved top-5 and top-1 error rates on the ImageNet test set that were significantly lower than other models at the time, demonstrating the effectiveness of deep learning for image classification tasks.

    Original Description:

    Original Title

    Alexnet

    Copyright

    © © All Rights Reserved

    Available Formats

    PDF, TXT or read online from Scribd

    Did you find this document useful?

    Is this content inappropriate?

    Report this Document
    The document describes AlexNet, which was the winner of the ImageNet LSVRC-2012 competition for image classification. AlexNet used a neural network architecture with 5 convolutional layers and 3 fully-connected layers. It achieved top-5 and top-1 error rates on the ImageNet test set that were significantly lower than other models at the time, demonstrating the effectiveness of deep learning for image classification tasks.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Flag for inappropriate content
    Download as pdf or txt
    20 views20 pages

    Artificial Intelligence: Alexnet

    Uploaded by

    Thành Cao Đức
    The document describes AlexNet, which was the winner of the ImageNet LSVRC-2012 competition for image classification. AlexNet used a neural network architecture with 5 convolutional layers and 3 fully-connected layers. It achieved top-5 and top-1 error rates on the ImageNet test set that were significantly lower than other models at the time, demonstrating the effectiveness of deep learning for image classification tasks.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Flag for inappropriate content
    Download as pdf or txt
    of 20

    Artificial Intelligence

    AlexNet

    Pham Viet Cuong


    Dept. Control Engineering & Automation, FEEE
    Ho Chi Minh City University of Technology
    AlexNet

    Pham Viet Cuong - Dept. Control Eng. & Automation, FEEE, HCMUT 2
    AlexNet
    ✓ Winner of ImageNet LSVRC-2012
    ✓ Image classification problem
    ❖ 1000 classes
    ❖ 650K neurals
    ❖ 62M parameters
    ❖ Training: 1.2M images
    ❖ Validation: 50K images
    ❖ Test: 150K images

    Pham Viet Cuong - Dept. Control Eng. & Automation, FEEE, HCMUT 3
    Architecture
    ✓ 5 convolutional layers
    ✓ 3 fully-connected layers
    ✓ Softmax

    Pham Viet Cuong - Dept. Control Eng. & Automation, FEEE, HCMUT 4
    Architecture
    ✓ Training on Multiple GPUs
    ❖ Smaller error

    Pham Viet Cuong - Dept. Control Eng. & Automation, FEEE, HCMUT 5
    Architecture
    ✓ Pooling
    ❖ Overlapping pooling
    ❖ Window: 3x3
    ❖ Stride: 2

    Pham Viet Cuong - Dept. Control Eng. & Automation, FEEE, HCMUT 6
    Architecture
    ✓ Activation function
    ❖ ReLU
    ❖ ReLU vs tanh

    Pham Viet Cuong - Dept. Control Eng. & Automation, FEEE, HCMUT 7
    Overfitting
    ✓ Data augmentation
    ❖ Without data augmentation: substantial overfitting
    ❖ Image translation & horizontal reflection
    ➢ 256x256 image → 224x224 patches
    ➢ 2048 times
    ❖ Altering intensities of RGB channels
    ✓ Test time: 1 images → 5 patches → 10 patches (reflection)

    Pham Viet Cuong - Dept. Control Eng. & Automation, FEEE, HCMUT 8
    Overfitting
    ✓ Dropout
    ❖ Without dropout: substantial overfitting
    ❖ Used in the first two fully-connected layers
    ❖ p = 0.5
    ❖ Roughly doubles # iterations

    Pham Viet Cuong - Dept. Control Eng. & Automation, FEEE, HCMUT 9
    Training
    ✓ Stochastic gradient descent
    ✓ Actually: mini-batch gradient descent
    ✓ With momentum
    ✓ Batch size: 128
    ❖ Momentum: 0.9
    ❖ Weight decay: 0.0005 (for regularization?)

    Pham Viet Cuong - Dept. Control Eng. & Automation, FEEE, HCMUT 10
    Results
    ✓ Example:

    Pham Viet Cuong - Dept. Control Eng. & Automation, FEEE, HCMUT 11
    Training

    Pham Viet Cuong - Dept. Control Eng. & Automation, FEEE, HCMUT 12
    Results
    ✓ Evaluation criteria
    ❖ Top-1 & top-5 error rate
    ✓ ILSVRC-2010

    ✓ ILSVRC-2012

    Pham Viet Cuong - Dept. Control Eng. & Automation, FEEE, HCMUT 13
    Results
    ✓ ImageNet 2009
    ❖ 10.184 classes
    ❖ 8.9M images
    ❖ 6th convolutional layer added
    Model Top 1 error rate (%) Top 5 error rate (%)
    AlexNet 67.4 40.9
    Other 78.1 60.9

    Pham Viet Cuong - Dept. Control Eng. & Automation, FEEE, HCMUT 14
    Results

    Pham Viet Cuong - Dept. Control Eng. & Automation, FEEE, HCMUT 15
    Results
    ✓ Kernel: 96, 11x11x3

    Pham Viet Cuong - Dept. Control Eng. & Automation, FEEE, HCMUT 16
    Results

    Pham Viet Cuong - Dept. Control Eng. & Automation, FEEE, HCMUT 17
    Results

    Pham Viet Cuong - Dept. Control Eng. & Automation, FEEE, HCMUT 18
    Results

    Pham Viet Cuong - Dept. Control Eng. & Automation, FEEE, HCMUT 19
    Results

    Pham Viet Cuong - Dept. Control Eng. & Automation, FEEE, HCMUT 20

    You might also like