Deep Learning for Vision

Dr. Đinh Viết Sang

Hanoi University of Science and Technology

Hanoi 2016
 Contact Info

 Dr. Đinh Viết Sang

Department of Computer Science
School of Information and Communication Technology
Hanoi University of Science and Technology
 Research Interests: Machine Learning, Deep Learning,
Computer Vision
 Website: soict.hust.edu.vn/~khmt/sangdv
 Email: sangdv@soict.hust.edu.vn

2
Computer Vision

3
Computer Vision

4
Computer Vision

5
Computer Vision

6
 Computer Vision
Traditional machine perception

7
Computer Vision

8
Computer Vision

9
 Deep Learning

Deep learning
approach

10
Deep Learning

11
Deep Learning

12
 Deep Learning vs Human

DeepFace 2014
Closing the Gap to Human Level Performance in Face Verification

Accuracy
DeepFace: 97.35%
Human: 97.5%
13
 Deep Learning vs Human

The competition is fierce, with the ImageNet Large Scale Visual Recognition
Challenge doing the judging for the 2015 championship on December 17.
Between now and then expect to see a stream of papers claiming they have one-
upped humans too. For instance, only 5 days after Microsoft announced it had
beat the human benchmark of 5.1% errors with a 4.94% error grabbing neural
network, Google announced it had one-upped Microsoft by 0.04%.

14
 Deep Learning vs Human

Deep Speech 2015

Baidu has developed a voice system that can recognize English
and Mandarin speech better than people, in some cases.

“For short phrases, out of context, we seem to be surpassing human

levels of recognition”- Andrew Ng
15
Deep Learning vs Human

AlphaGo vs Human: 9-1 16

Deep Learning vs Human

17
 What is Deep Learning?
Deep learning is a branch of machine learning based on a set
of algorithms that attempt to model high-level abstractions in data by using
a deep graph with multiple processing layers, composed of multiple non-
linear transformations.

18
History of DL

19
History of DL

20
History of DL

21
History of DL

22
History of DL

23
History of DL

24
History of DL

25
ConvNets

26
ConvNets

27
ConvNets

28
ConvNets

29
ConvNets

30
ConvNets

31
ConvNets

32
ConvNets

33
ConvNets

34
Neuron

35
Neural Networks

36
Neural Networks

37
Neural Networks

38
Local Connectivity

39
Local Connectivity

40
ConvNets

41
ConvNets

42
ConvNets

43
ConvNets

44
ConvNets

45
ConvNets

46
ConvNets

47
ConvNets

48
ConvNets

49
ConvNets

50
ConvNets

51
ConvNets

52
ConvNets

53
ConvNets

Share
weights

54
ConvNets

55
ConvNets

56
ConvNets

57
ConvNets

58
ConvNets

59
ConvNets

60
ConvNets

61
ConvNets

62
ConvNets

63
ConvNets

64
ConvNets

65
 ConvNets

INPUT -> [[CONV -> RELU]*N -> POOL?]*M -> [FC -> RELU]*K -> FC
where the * indicates repetition, and the POOL? indicates an optional
pooling layer.
N >= 0 (and usually N <= 3), M >= 0, K >= 0 (and usually K < 3).

66
ConvNets

67
Training ConvNets

68
Training ConvNets

69
Data Augmentation

70
Data Augmentation

71
Data Augmentation

72
Data Augmentation

73
Data Augmentation

74
Transfer learning

75
Transfer learning

76
Transfer learning

77
Transfer learning

78
GPU Training

79
GPU Training

80
GPU Training

81
Distributed Training

82
Compressed DNNs

83
Compressed DNNs

84
Compressed DNNs

85
Compressed DNNs

86
87
88
Software Packages

89
Software Packages

90
Software Packages

91
Software Packages

92
Software Packages

93
Software Packages

94
Software Packages

95
Software Packages

96
Software Packages

97
Software Packages

98
 Thank
CÁM ƠN SỰyou!
THEO DÕI!

99