Filtering and Edge Detection 1

CNG 483
Introduction to Computer Vision
Pixels and Filters
Asst. Prof. Dr. Meryem Erbilek
Slide partially based on Stanford U. CS131
METU Lecture 4- 1 CNG483

What we will learn today?
• Image sampling and quantization
• Image histograms
• Images as functions
• Linear systems (filters)
• Convolution and correlation
Some background reading:
Forsyth and Ponce, Computer Vision: A Modern Approach, Part II: Early Vision: Just One Image, Chp.4
Richard Szeliski, Computer Vision: Algorithms and Applications, Chp.3.


Types of Images

Binary image representation
Slide partially based on Stanford U. CS131 Slide credit: Ulas Bagci

Grayscale image representation

Color Image - one channel

Color image representation

Images are sampled
What happens when we zoom into the images?

Errors due Sampling

Resolution
is a sampling parameter, defined in dots per inch
(DPI) or equivalent measures of spatial pixel
density.
- Resolution of the camera and the resolution of
the screen can both affect the quality.

Images are Sampled and Quantized
• An image contains discrete number of pixels
– A simple example
75
– Pixel value:
• “grayscale”
(or “intensity”): [0,255]
231
148

Images are Sampled and Quantized
• An image contains discrete number of pixels
– A simple example
– Pixel value: [90, 0, 53]
• “grayscale”
(or “intensity”): [0,255]
• “color”
– RGB: [R, G, B]
[249, 215, 203]
– Lab: [L, a, b]
– HSV: [H, S, V]
[213, 60, 67]

With this loss of information (from
sampling and quantization),
Can we still use images for useful

tasks?


Histogram
• Histograms measure the frequency of
brightness within the image.

Histogram
• Histogram captures the distribution of gray levels in
the image: how many times does a particular pixel
value appear in an image.

Histogram
Slide partially based on Stanford U. CS131 Slide credit: Dr. Mubarak Shah

Histogram
• Histograms can be used to provide a
quantifiable description of what things look
like.
• For instance; can help us detect particular

features in images!
• Quite useful ☺

Histogram – use case: tissue classification
Slide partially based on Stanford U. CS131 Slide credit: Dr. Mubarak Shah

Histogram - use case: viewfinder
Slide partially based on Stanford U. CS131 Slide credit: Dr. Bernd Girod


Images as discrete functions
• Images are usually digital (discrete):

– Sample the 2D space on a regular grid
• Represented as a matrix of integer values

pixel

Images as coordinates
Cartesian coordinates
Notation for
discrete
functions

Images as functions
• An Image as a function f from R2 to RM:
• f( x, y ) gives the intensity at position ( x, y )
• Defined over a rectangle, with a finite range:
f: [a,b] x [c,d ] → [0,255]
Domain range
support

Images as functions
• An Image as a function f from R2 to RM:
• f( x, y ) gives the intensity at position ( x, y )
• Defined over a rectangle, with a finite range:
f: [a,b] x [c,d ] → [0,255]
Domain range
support
• A color image:

Histograms are a type of image
function
● Histogram can be seen as another f: region → vector mapping.
● In recognition tasks for computer vision, image representation typically plays

a critical role.
● Throughout the course, we'll progressively discuss more powerful

representation and extraction techniques.


Systems and Filters
Filtering:
– Forming a new image whose pixel values are
transformed from original pixel values.
Goals:
• Goal is to extract useful information (e.g.
Edge detection) from images, or transform
images into another domain where we can
modify/enhance image properties (e.g. de-
noising).

Super-resolution
De-noising
In-painting
Bertamio et al

System and Filters
• We define a system as a unit that converts an
input function f[n,m] into an output (or
response) function g[n,m], where (n,m) are
the independent variables.
– In the case for images, (n,m) represents the
spatial position in the image.
(Filter)

2D discrete-space systems (filters)
A digital filter is a system that performs mathematical
operations on a sampled, discrete-time signal. [Wikipedia]
● S is the system operator, defined as a mapping or
assignment of a member of the set of possible outputs
g[n,m] to each member of the set of possible inputs
f[n,m].
Equivalent notations:

Filter example #1: Moving Average
2D discrete space moving average over a 3 × 3
window of neighborhood:
h
1 1 1
1 1 1
1 1 1

0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0
0 0
0 0
0 90
90 0
0 90
90 90
90 90
90 0
0 0
0
0
0 0
0 0
0 90
90 90
90 90
90 90
90 90
90 0
0 0
0
0
0 0
0 0
0 0
0 0
0 0
0 0
0 0
0 0
0 0
0
Courtesy of S. Seitz
0
0 0
0 90
90 0
0 0
0 0
0 0
0 0
0 0
0 0
0
0
0 0
0 0
0 0
0 0
0 0
0 0
0 0
0 0
0 0
0

0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 10
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 0 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 0 0 0 0 0 0 0
0 0 90 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
Slide partially based on Stanford U. CS131 35

0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 10 20
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 0 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 0 0 0 0 0 0 0
0 0 90 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 10 20 30
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 0 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 0 0 0 0 0 0 0
0 0 90 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 10 20 30 30
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 0 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 0 0 0 0 0 0 0
0 0 90 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 10 20 30 30 30 20 10
0 0 0 90 90 90 90 90 0 0 0 20 40 60 60 60 40 20
0 0 0 90 90 90 90 90 0 0 0 30 60 90 90 90 60 30
0 0 0 90 90 90 90 90 0 0 0 30 50 80 80 90 60 30
0 0 0 90 0 90 90 90 0 0 0 30 50 80 80 90 60 30
0 0 0 90 90 90 90 90 0 0 0 20 30 50 50 60 40 20
0 0 0 0 0 0 0 0 0 0 10 20 30 30 30 30 20 10
0 0 90 0 0 0 0 0 0 0 10 10 10 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
Slide partially based on Stanford U. CS131 Source: S. Seitz

In summary:
• This filter “Replaces” each
pixel with an average of its 1 1 1
neighborhood. 1 1 1
1 1 1
• Achieve smoothing effect
(remove sharp features)


Filter example #2: Thresholding
• Image segmentation based on a simple

threshold:
255,

Properties of linear systems
• Additivity
• Homogeneity
• Superposition
System is a linear system (function), iff it satisfies Superposition!

Linear Systems (filters)
• Is the moving average a linear system?

YES!
• Is thresholding a linear system?

No!


1D Discrete convolution (symbol: )
We are going to convolve a function f with a filter h
f[k] h[k]

1D Discrete convolution (symbol: )
We are going to convolve a function f with a filter h
We first need to calculate the shifted filter h[n-k]
h[k] h[-k] h[n-k]

Discrete convolution (symbol: )
We are going to convolve a function f with a filter h.
h[-2-k]
f[k] g[-2]
METU Lecture 4- 48 6-Oct-16

h[-1-k]
f[k] g[-1]
METU Lecture 4- 49 6-Oct-16

h[-k]
f[k] g[0]

h[1-k]
f[k] g[1]

h[2-k]
f[k] g[2]

h[n-k]
f[k] g[n]

In summary, the steps for discrete convolution are:
• Fold h[k,l] about origin to form h[−k]
• Shift the folded results by n to form h[n − k]
• Multiply h[n − k] by f[k]
• Sum over all k
• Repeat for every n
Slide partially based on Stanford U. CS131 n

2D convolution
2D convolution is very similar to 1D.
• The main difference is that we now have to iterate over 2 axis instead of 1.
Assume we have a filter(h[,]) that

is 3x3 and an image (f[,]) that is
7x7.

2D convolution

is 3x3. and an image (f[,]) that is
7x7.

2D convolution

7x7.

2D convolution

7x7.

2D convolution

7x7.

2D convolution

7x7.

2D convolution example
Slide Slide partially

credit: Song Ho based
Ahn on Stanford U. CS131













Convolution in 2D - examples
?
•0 •0 •0
* •0 •1 •0
•0 •0 •0
=
Original
Courtesy of D Lowe

•0 •0 •0
* •0 •1 •0
•0 •0 •0
=
Original Filtered
Courtesy of D Lowe
(no change)

?
•0 •0 •0
* •0 •0 •1
•0 •0 •0
=
Original
Courtesy of D Lowe

•0 •0 •0
* •0 •0 •1
•0 •0 •0
=
Original Shifted right
Courtesy of D Lowe
By 1 pixel

= ?
•1 •1 •1
* •1 •1 •1
•1 •1 •1
Original
Courtesy of D Lowe

•1 •1 •1
* •1 •1 •1
•1 •1 •1
=
Original Blur (with a
Courtesy of D Lowe
box filter)

-
•0 •0 •0 •1 •1 •1
•0 •2 •0
•0 •0 •0
•1 •1 •1
•1 •1 •1
=?
(Note that filter sums to 1)
Original
“details of the image”
Courtesy of D Lowe
•0 •0 •0 •0 •0 •0 •1 •1 •1
•0 •1 •0 + •0 •1 •0 •1 •1 •1
•0 •0 •0 •0 •0 •0 •1 •1 •1

• What does blurring take away?
– =
original smoothed (5x5) detail
• Let’s add it back:
+a =
original detail sharpened


Convolution in 2D –
Sharpening filter
-
•0 •0 •0 •1 •1 •1
•0 •2 •0
•0 •0 •0
•1 •1 •1
•1 •1 •1 =
Original
Courtesy of D Lowe
Sharpening filter: Accentuates differences with local average

Image support and edge effect
• A computer will only convolve finite support
signals.
• That is: images that are zero for n,m outside some
rectangular region
• numpy’s convolution performs 2D DS convolution of
finite-support signals.
* = (N1 + N2 − 1) × (M1 +M2 − 1)

N2 ×M2
N1 ×M1

Image support and edge effect
• A computer will only convolve finite support
signals.
• What happens at the edge?
• zero “padding”
f • edge value replication
h • mirror extension
• more (beyond the scope of this class)
-> Matlab conv2 uses

zero-padding

(Cross) correlation (symbol: )
Cross correlation of two 2D signals f[n,m] and g[n,m]
(k, l) is called the lag
• Equivalent to a convolution without the flip
(g* is defined as the complex conjugate of g. In this class, g(n,m) are real numbers, hence g*=g.)

(Cross) correlation – example
Left Right
scanline
Norm. cross corr. score

Cross Correlation Application:
Vision system for TV remote
control
- uses template matching
Figure from “Computer Vision for Interactive Computer Graphics,” W.Freeman et al, IEEE Computer Graphics and Applications,
1998 copyright 1998, IEEE
Slide partially based on Stanford U. CS131 Forsyth and Ponce, Computer Vision: A Modern Approach, Fig. 4.16

Convolution vs. (Cross) Correlation
f*h f**h

Convolution vs. (Cross) Correlation
• A convolution is an integral that expresses the amount
of overlap of one function as it is shifted over another
function.
– convolution is a filtering operation
• Correlation compares the similarity of two sets of

data. Correlation computes a measure of similarity of
two input signals as they are shifted by one another.
The correlation result reaches a maximum at the time
when the two signals match best .
– correlation is a measure of relatedness of two signals

Filtering and Edge Detection 1

Uploaded by

Copyright:

Available Formats

You might also like

Filtering and Edge Detection 1

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Filtering and Edge Detection 1

Uploaded by

Copyright:

Available Formats

CNG 483

Introduction to Computer Vision

Pixels and Filters

Asst. Prof. Dr. Meryem Erbilek

Slide partially based on Stanford U. CS131

METU Lecture 4- 1 CNG483

Slide partially based on Stanford U. CS131

METU Lecture 4- 2 CNG483

Slide partially based on Stanford U. CS131

METU Lecture 4- 3 CNG483

Slide partially based on Stanford U. CS131

METU Lecture 4- 4 CNG483

Slide partially based on Stanford U. CS131 Slide credit: Ulas Bagci

METU Lecture 4- 5 CNG483

Slide partially based on Stanford U. CS131 Slide credit: Ulas Bagci

METU Lecture 4- 6 CNG483

Slide partially based on Stanford U. CS131 Slide credit: Ulas Bagci

METU Lecture 4- 7 CNG483

Slide partially based on Stanford U. CS131 Slide credit: Ulas Bagci

METU Lecture 4- 8 CNG483

Slide partially based on Stanford U. CS131

METU Lecture 4- 9 CNG483

Slide partially based on Stanford U. CS131 Slide credit: Ulas Bagci

METU Lecture 4- 10 CNG483

the screen can both affect the quality.

Slide partially based on Stanford U. CS131 Slide credit: Ulas Bagci

Slide partially based on Stanford U. CS131

METU Lecture 4- 12 CNG483

[213, 60, 67]

Slide partially based on Stanford U. CS131

METU Lecture 4- 13 CNG483

Can we still use images for useful

Slide partially based on Stanford U. CS131

METU Lecture 4- 14 CNG483

Slide partially based on Stanford U. CS131

METU Lecture 4- 15 CNG483

Slide partially based on Stanford U. CS131

METU Lecture 4- 16 CNG483

Slide partially based on Stanford U. CS131

METU Lecture 4- 17 CNG483

METU Lecture 4- 18 CNG483

• For instance; can help us detect particular

Slide partially based on Stanford U. CS131

METU Lecture 4- 19 CNG483

METU Lecture 4- 20 CNG483

METU Lecture 4- 21 CNG483

Slide partially based on Stanford U. CS131

METU Lecture 4- 22 CNG483

• Images are usually digital (discrete):

• Represented as a matrix of integer values

Slide partially based on Stanford U. CS131

METU Lecture 4- 23 CNG483

Slide partially based on Stanford U. CS131

METU Lecture 4- 24 CNG483

Slide partially based on Stanford U. CS131

METU Lecture 4- 25 CNG483

Slide partially based on Stanford U. CS131

METU Lecture 4- 26 CNG483

● In recognition tasks for computer vision, image representation typically plays