q

se
i@
e u

zh
@s
Fundamentals of Computer

qi
i
zh
Vision
qi

i
se

zh
i@

qi
Lecture 5：More about filters
u
se

zh
i@

qi

u
2023 spring

se
zh

i@
eu

zh
@s

qi
hi
 Recall

se
i@
u
Iˆ(i, j )  I (i, j )  n(i, j )

zh
@s

qi
i
zh ( f  g ) i , j    f i  k , j  l g k , l 
qi

u
k ,l

i
se

zh
i@

qi
u
se

zh
i@

qi

u
se
zh

Q: given the above conv operation that removes the

i@
eu

noise for pixel(i,j), how to remove the noise in the

entire image? zh
@s

qi
hi

Q: what decides the denoising effect of a filter?

 Contents

se
i@
u
Generation of image noise

zh
l

@s
Image sampling and quantisation

qi
i
l zh
Definition of convolution
qi

u
Examples of denoising filters

i
l

se

zh
l Average filter: boxing, Gaussian

i@

qi
u

Order statistics filter: median

se

l zh
Examples of sharpening filters
i@

qi
l

u
se
Laplacian filter
zh

i@
Extend the convolution into deep learning
eu

l
zh
@s

qi
hi
 q

se
Median filtering – order statistics filter

i@
u
• A median filter operates over a window by

zh
@s
selecting the median intensity in the window

qi
i
zh
qi

i
se

zh
i@

qi
u
se

zh
i@

qi

u
se
zh

i@
eu

zh
@s

Q: what kind of pixel will be replaced by the median

qi
hi

with a higher proberbility?

 q

se
Averaging Filter Vs. Median Filter Example

i@
u
• What advantage does median filtering have

zh
@s
over averaging filtering?

qi
i
• Good at reducing a certain type of random noise, eg., salt
zh
and pepper noise.
qi

i
se

zh
i@

qi
u
se

zh
i@

qi

u
se
zh

i@
eu

zh
@s

qi
hi
 zh
i@
se
u
hi qi
@s zh
eu i@s
qi e u
zh
i@ q
se
qi
zh u
i@ qi
se zh
u i@
qi se
Example: Averaging Filter Vs. Median Filter Example

zh
i
 zh
i@
se
u
hi qi
@s zh
eu i@s
qi e u
zh
i@ q
se
qi
zh u
i@ qi
se zh
u i@
Averaging Filter Vs. Median Filter Example

qi se
zh
i
 zh
i@
se
u
hi qi
@s zh
eu i@s
qi e u
zh
i@ q
se
qi
zh u
i@ qi
se zh
u
Averaging Filter Vs. Median Filter Example

i@
qi se
zh
i
 q

se
i@
e u

zh
Summary of smoothing operation:

@s

qi
i
zh
• Replace each pixel by a weighted average of its neighbors
or by the mean/max/min of its neighbor.
qi

i
se
• Images become more smoother while “high-frequency”

zh
components are removed from the images.

i@

qi
u

• These filters are low-pass filters.

se

zh
• Useful for reducing noise and unimportant details.
i@

qi
• The size of the mask or the sigma of Gaussian controls

u
se
the amount of smoothing.
zh

i@
• Trade-off: noise removal vs. blurring and loss of details.
eu

zh
@s

qi
hi
 q

se
Example : noise vs. blurring and loss of detail

i@
u
•This is an image from Hubble Space Telescope.

zh
@s
•By smoothing the original image we get rid of lots of the

qi
finer detail which leaves only the gross features for

i
zh
thresholding.
qi

u
•Compare the result before and after thresholding, we find

i
se

zh
that only dominant objects are preserved.

i@

qi
u
se

zh
i@

qi

u
se
zh

i@
eu

zh
@s

qi
hi

Original Image Smoothed Image Thresholded Image

 q

se
i@
e u

zh
@s

qi
Except the smoothing, what else can filters do?

i
zh
qi

i
se

zh
i@
Really important!

qi
u
se

• Enhance images zh
– Denoise, resize, increase contrast, etc.
i@

qi

u
• Extract information from images

se
zh

– Texture, edges, distinctive points, etc.

i@
eu

• Detect patterns
– Template matching zh
@s

qi
hi
 Contents

se
i@
u
Generation of image noise

zh
l

@s
Image sampling and quantisation

qi
i
l zh
Definition of convolution
qi

u
Examples of denoising filters

i
l

se

zh
l Average filter: boxing, Gaussian

i@

qi
u

l Order statistics filter: mean

se

l
zh
Laplacian filter
i@

qi

u
se
zh

i@
eu

zh
@s

qi
hi
 Contents

se
i@
u
Generation of image noise

zh
l

@s
Image sampling and quantisation

qi
i
l zh
Definition of convolution
qi

u
Examples of denoising filters

i
l

se

zh
l Average filter: boxing, Gaussian

i@

qi
u

l Order statistics filter: median

se

l zh
Examples of sharpening filters
i@

qi

u
Laplacian filter

se
l
zh

l Extend the convolution into deep learning

i@
eu

zh
@s

qi
hi
 q

se
Sharpening Spatial Filters

i@
u
•Previously we have looked at smoothing filters

zh
@s
which remove fine detail

qi
i
•However, sometimes details are required.
zh
• What are those details in the image?
qi

i
se

zh
• High frequency contents

i@

qi
• region presenting huge intensity variance
u
se

• For example: edges

zh
i@

qi

u
se
zh

i@
eu

zh
@s

qi
hi
 q

se
Sharpening Spatial Filters

i@
u
•Sharpening spatial filters seek to highlight fine detail

zh
@s
• Remove blurring from images

qi
i
zh
• Highlight edges
qi

u
• This operation is called image enhancement.

i
se

zh
•Sharpening filters are based on spatial differentiation

i@

qi
u
se

zh
i@

qi

u
se
zh

i@
eu

zh
@s

qi
hi
 q

se
Spatial Differentiation

i@
e u

zh
@s
•Differentiation measures the rate of change of a

qi
function

i
zh
qi

i
se

zh
i@

qi
u
se

zh
i@

qi

u
se
zh

i@
eu

zh
@s

qi
hi
 q

se
1st Derivative （one of the options）

i@
e u

zh
@s

qi
f
i
zh  f ( x  1)  f ( x)
x
qi

i
se

zh
i@

qi
u
se

zh
i@

qi

u
se
zh

i@
eu

zh
@s

qi
hi
 q

se
2nd Derivative

i@
u
Horizontal second derivative

zh
@s
2 f

qi
 f ( x  1)  f ( x  1)  2 f ( x )
i
2
zh
 x
qi

i
se
Vertical second derivative

zh
i@
2 f

qi
u

 f ( y  1)  f ( y  1)  2 f ( y )
se

2
 y zh
i@

qi

u
se
zh

The 2nd derivative is more useful for image enhancement

i@
than the 1st derivative
eu

zh
• Stronger response to fine detail
@s

• Simpler implementation
qi
hi
 q

se
Using Second Derivatives For Image Enhancement

i@
e u

zh
@s

qi
i
The first sharpening filter we will look at is the
zh
Laplacian
qi

i
• Isotropic

se

zh
• One of the simplest sharpening filters

i@

qi
u
se

zh
i@

qi

u
se
zh

i@
eu

zh
@s

qi
hi
 q

se
The Laplacian

i@
u
The Laplacian is defined as follows:

zh
@s
2 2
 f  f

qi
i
2
zh  f  2  2
 x  y
qi

i
se

zh
where the partial 2nd order derivative in the x

i@

qi
direction is defined as follows:
u
se

2 f zh
 f ( x  1)  f ( x  1)  2 f ( x )
i@

qi

u
2
 x

se
zh

and in the y direction as follows:

i@
eu

2
 f zh
 f ( y  1)  f ( y  1)  2 f ( y )
@s

2
 y
qi
hi
 q

se
The Laplacian filter

i@
u
0 0 0

zh
2
 f

@s
 f ( x  1)  f ( x  1)  2 f ( x ) 1 -2 1

qi
2
 x 0 0 0

i
zh +
qi

i
0 1 0

se

zh
2
 f
 f ( y  1)  f ( y  1)  2 f ( y ) 0 -2 0

i@

qi
2
 y
u

0 1 0
se

zh
i@

qi

u
se
2 2
 f  f
zh

2
 f  2  2 0 1 0

i@
eu

 x  y 1 -4 1
zh
@s

qi
0 1 0
hi
 q

se
i@
u
2 2
 f  f

zh
2
 f  2  2 0 1 0

@s
 x  y

qi
i
1 -4 1
zh 0 1 0
qi

i
se

zh
i@

qi
Q5: Given the Laplacian filter above, can you explain what
u
se

does the filter below do?

zh
i@

qi

u
se
1 1 1
zh

i@
1 -8 1
eu

1 1 1 zh
@s

qi
hi
 q

se
The Laplacian

i@
u
Applying the Laplacian to an image we get a

zh
@s
new image that highlights edges and other

qi
discontinuities

i
zh
qi

i
0 1 0

se

zh
1 -4 1

i@

qi
u
se

zh 0 1 0
i@

qi

u
se
zh

Original Image Laplacian Filtered Image

i@
eu

Q6: Look at the filtered image zh

@s

(1) What contributes to the dark background?

qi
hi

(2) What does the brighter region stand for?

 q

se
But That Is Not Very Enhanced!

i@
eu

zh
•The result of a Laplacian filtering is not an

@s

qi
enhanced image

i
zh
•We have to do more work in order to get our
qi

i
final image

se

zh
•Add the Laplacian result to the original image

i@

qi
u

to generate our final sharpened enhanced

se

image
zh
i@

qi

u
se
zh

i@
eu

g ( x, y )  f ( x, y )   fzh 2
@s

qi
hi
 q

se
Laplacian Image Enhancement

i@
e u

zh
@s

qi
i
zh + =
qi

i
se

zh
i@

qi
u
se

Original zhLaplacian Sharpened

i@

Image Filtered Image Image

qi

u
se
zh

i@
eu

zh
@s

qi
hi
 q

se
Laplacian Image Enhancement

i@
e u

zh
@s

qi
i
zh
qi

i
se

zh
i@

qi
u
se

zh
i@

qi

u
se
zh

i@
eu

zh
@s

qi
In the final sharpened image edges and fine details
hi

are much more obvious

 zh
i@
se
u
hi qi
@s zh
eu i@s
qi e u
zh
i@ q
se
qi
zh u
i@ qi
se zh
u i@
se
Sharpening revisited – another sharpening filter

qi
zh
i
 q

se
Sharpening revisited

i@
u
What does blurring take away?

zh
@s

qi
i
zh – =
qi

i
se

zh
smoothed (5x5) detail

i@
original

qi
u
se

zh
-
i@

0 0 0 1 1 1
qi

u
se
0 1 0 1 1 1
zh

0 0 0 1 1 1

i@
eu

zh
@s

qi
hi
 q

se
Sharpening revisited

i@
ue

zh
Let’s add it back:

@s

qi
i
zh
qi

i
+α =

se

zh
i@

qi
u

original detail sharpened

se

zh
i@

qi

u
-

se
0 0 0 0 0 0 1 1 1
+ (
zh

0 1 0 0 1 0 1 1 1 )

i@
eu

0 0 0 0 0 0 1 1 1
zh
@s

qi
(Note that filter sums to 1 with α =1 )
hi
 Contents

se
i@
u
Generation of image noise

zh
l

@s
Definition of convolution

qi
l

i
l zh
Examples of denoising filters
qi

u
Average filter: boxing, Gaussian

i
l

se

zh
l Order statistics filter: median

i@

qi
Laplacian filter
u

l
se

zh
i@

qi

u
se
zh

i@
eu

zh
@s

qi
hi
 Contents

se
i@
u
Generation of image noise

zh
l

@s
Image sampling and quantisation

qi
i
l zh
Definition of convolution
qi

u
Examples of denoising filters

i
l

se

zh
l Average filter: boxing, Gaussian

i@

qi
u

l Order statistics filter: median

se

l zh
Examples of sharpening filters
i@

qi

u
Laplacian filter

se
l
zh

l Extend the convolution into deep learning

i@
eu

zh
@s

qi
hi
 zh
i@
se
u
hi qi
@s zh
eu i@s
qi e u
zh
i@ q
se
qi
zh u
i@ qi
se zh
u i@
qi se
zh
i
 zh
i@
se
u
hi qi
@s zh
eu i@s
qi e u
zh
i@ q
se
qi
zh u
i@ qi
se zh
u i@
qi se
zh
i
 zh
i@
se
u
hi qi
@s zh
eu i@s
qi e u
zh
i@ q
se
qi
zh u
i@ qi
se zh
u i@
qi se
zh
i
 zh
i@
se
u
hi qi
@s zh
eu i@s
qi e u
zh
i@ q
se
qi
zh
Q: what’s the functions of these models？
u
i@ qi
se zh
u i@
qi se
zh
i
 zh
i@
se
u
hi qi
@s zh
eu i@s
qi e u
zh
i@ q
se
qi
zh u
i@ qi
se zh
u i@
qi se
zh
i
 zh
i@
se
u
hi qi
@s zh
eu i@s
qi e u
zh
i@ q
se
qi
zh u
i@ qi
se zh
u i@
qi se
zh
i
 Arch structure of AlexNet

se
i@
e u

zh
@s

qi
i
zh
qi

i
se

zh
i@

qi
u
se

zh
i@

qi

u
se
zh

i@
eu

zh
@s

qi
Krizhevsky, Alex; Sutskever, Ilya; Hinton, Geoffrey E. (2017-05-24). "ImageNet classification with deep convolutional neural networks" (PDF).
hi

Communications of the ACM. 60 (6): 84–90.

 q

se
Separable convolution in deep learning

i@
e u

zh
@s

qi
i
zh
qi

i
se

zh
i@

qi
u
se

zh
i@

qi

u
se
zh

i@
eu

zh
Q: 1) what are the two operators of convolution？
@s

2) what’s the size of the output, if we ignor the boundray effect?

qi
3) To finish the output calculation, how many add+muls do I need?
hi
 q

se
Separable convolution in deep learning

i@
e u

zh
@s

qi
i
zh
qi

i
se

zh
i@

qi
u
se

zh
i@

qi

u
se
zh

i@
eu

https://blog.csdn.net/u010349092/article/details/81607819

zh
@s

qi
MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications. Andrew G. Howard, Menglong Zhu, Bo Chen, Dmitry
hi

Kalenichenko, Weijun Wang, Tobias Weyand, Marco Andreetto, Hartwig Adam. arxiv:1704.04861
 zh
i@
se
u
hi qi
@s zh
eu i@s
qi e u
zh
i@ q
se
qi
zh u
i@ qi
se zh
u i@
qi se
zh
i