Professional Documents
Culture Documents
Res Net
Res Net
● Problem Domain
● Related Works
● Conclusions
Background
Introduction of CNN
● Deep Convolutional Neural Networks
○ Breakthrough for Image Classification
○ Integrates low/mid/high-level features and classifiers in an end-to-end
multilayer fashion
○ “...Network depth is of crucial importance”
Benefits of Deeper Networks
Table 8: Object Detection Map on COCO Validation set COCO Results from ICCV15 Slides
Accomplishments
● ILSVRC 2015 and COCO 2015 Competitions
● 1st place in Classification Competition
● 1st place in ImageNet Detection
● 1st place in Imagenet Localization
● 1st place in COCO Detection
● 1st place in COCO Segmentation
● Most cited paper in Google Scholar Metrics 2018 (over
10k citations)
Insight of ResNet
Identity shortcut
● If optimal mapping is
closer to identity, easier
to capture small
fluctuations
https://gkioxari.github.io/Tutorials/eccv2018/index.html
From vision to general machine learning tasks
● Visual Recognition
● Image Generation
● Natural Language
Processing
● Speech Recognition
● Advertising, User
Prediction
● AlphaGo Zero: 40
Residual Blocks
Conclusions
● Residual Learning!
○ Degradation problem is addressed!
○ Deeper networks are easier to train via residual learning
○ Less error when increasing network depth
○ More accuracy gained from depth!
● Experiments
○ Image Recognition (ImageNet, CIFAR-10)
○ Object Detection (MS COCO)
● Model and Code:
○ https://github.com/KaimingHe/deep-residual-networks
Reference
“Very Deep Convolutional Networks for Large-Scale Image Recognition”, Simonyan &
Zisserman. ICLR 2015
“Deep Residual Learning for Image Recognition”, Kaiming He et al. CVPR 2016
“Identity Mappings in Deep Residual Networks”, Kaiming He et al ECCV 2016
“Deep Networks with Stochastic Depth”, G. Huang et al, ECCV 2016
“Residual Networks are Exponential Ensembles of Relatively Shallow Networks” ,
Andreas Veit et al, NIPS 2016
“Learning Deep Representations for Visual Recognition”, Kaiming He, Tutorial on Visual
Recognition and Beyond, ECCV 2018
Any Questions?