Chapter 05

Digital Image Processing, 2nd ed. www.imageprocessingbook.
com
Chapter 5
Image Restoration
• Image Degradation Model (Linear/Additive)
g ( x, y ) = h ( x , y ) ∗ f ( x, y ) + η ( x, y )
G ( u, v ) = H ( u, v ) ⋅ F ( u, v ) + N ( u, v )
© 2002 R. C. Gonzalez & R. E. Woods

Digital Image Processing, 2nd ed. www.imageprocessingbook.com
Chapter 5
Image Restoration
• Source of noise
– Image acquisition (digitization)
– Image transmission
• Spatial properties of noise
– Statistical behavior of the gray-level values of pixels
– Noise parameters, correlation with the image
• Frequency properties of noise
– Fourier spectrum
– Ex. white noise (a constant Fourier spectrum)

Chapter 5
Image Restoration
• Noise Model
⎛ ( z − µ )2 ⎞
1 ⎛ ( z − a )2 ⎞
p(z) = exp ⎜ − ⎟ 2
p ( z ) = ( z − a ) exp ⎜ − ⎟u ( z − a)
2πσ 2 ⎜ 2σ 2
⎟ b ⎜ b ⎟
⎝ ⎠ ⎝ ⎠
a b z b −1 − az
p( z) = e u ( z) p ( z ) = ae − az u ( z )
( )
b − 1 !
1
p( z) = (u ( z − a ) − u ( z − b))
b−a p ( z ) = Paδ ( z − a ) + Pbδ ( z − b )

Chapter 5
Image Restoration
• Test Pattern

Chapter 5
Image Restoration

Chapter 5
Image Restoration

Chapter 5
Image Restoration
• Periodic Noise:
– Electronic Devices

Chapter 5
Image Restoration
• Periodic noise
– Observe the frequency spectrum
• Random noise with PDFs
– Case 1: imaging system is available
• Capture images of “flat” environment
– Case 2: noisy images available
• Take a strip from constant area
• Draw the histogram and observe it
• Measure the mean and variance

Chapter 5
Image Restoration
• Noise Estimation:
– Shape: Histogram of a subimage (Background)

Chapter 5
Image Restoration
• Noise Estimation:
– Parameters:
µ = ∑ zi p ( zi ) ⎫
zi ∈S ⎪
⎬ ⇒ {θ } : PDF Parameters
σ 2 = ∑ ( zi − µ ) p ( zi ) ⎪
2
zi ∈S ⎭
ML method:
⎧ NS ⎫
θ = arg max ⎨∏ p ( zi ,θ ) ⎬
⎩ i =1 ⎭
– Spectral Estimation (several subimages)

Chapter 5
Image Restoration
• Noise-only spatial filter:

– g(x,y)=f(x,y)+η(x,y)
• Mean Filters:
– A m×n mask centered at (x,y): Sxy
– An algebraic Operation on mask members!

Chapter 5
Image Restoration
• Mean Filter:
– Arithmetic mean:
1
fˆ ( x, y ) = ∑ g ( s, t )
mn ( s ,t )∈S xy
Noise Reduction vs. Blurring

– Geometric mean:
1
⎡ ⎤ mn
fˆ ( x, y ) = ⎢ ∏ ( g ( s, t ) ) ⎥
⎢⎣ ( s ,t )∈S xy ⎥⎦
– ≈ same smoothing with less detail degradation:

Chapter 5
Image Restoration
• Mean Filter:
– Harmonic mean:
mn
fˆ ( x, y ) =
1
∑
( s ,t )∈S xy g ( s, t )
• Salt Reduction (Pepper will increase)/ Good for

gaussian like
– Contra-harmonic mean:
∑ ( g ( s, t ) )
Q +1
fˆ ( x, y ) =
( s ,t )∈S xy
(Q>0: Pepper, Q<0:Salt)
∑ g ( s, t )
Q
( s ,t )∈S xy

Chapter 5
Image Restoration
G+Noise
Original
Arithmetic
Geometric
More-Less
Blurring
Chapter 5
Image Restoration
Pepper Salt
Q=+1.5 Q=-1.5

Contra Harmonic Correct Uses
Chapter 5
Image Restoration
Q=-1.5 Q=+1.5
Contra Harmonic Incorrect Uses

Chapter 5
Image Restoration

• Order Statistic Filtera:
– A m×n mask centered at (x,y): Sxy
– An Ordering/Ranking operation on mask members!

Chapter 5
Image Restoration
• Median: Most Familiar OS filter:

fˆ ( x, y ) = median ( g ( s, t ) )
( s ,t )∈S xy
– Little Blurring/Single/Double valued noises

• Max/min:
fˆ ( x, y ) = Max ( g ( s, t ) ) , fˆ ( x, y ) = min ( g ( s, t ) )
( s ,t )∈S xy ( s ,t )∈S xy
– Find Brightest/Darkest Points (Pepper/Salt Reducer)

• Midpoint:
1
fˆ ( x, y ) = ⎛⎜ Max ( g ( s, t ) ) + min ( g ( s, t ) ) ⎞⎟
2 ⎝ ( s ,t )∈S xy ( s ,t )∈S xy ⎠
– OS+Mean Properties (Gaussian, Uniform)

Chapter 5
Image Restoration
• Alpha Trimmed mean:

– Delete (d/2) lowest and highest samples
1
fˆ ( x, y ) =
mn − d
∑
( s ,t )∈S xy
g r ( s, t )
– D = 0 → Mean Filter
– D = mn-1 → Median Filter
– Others: Combinational Properties

Chapter 5
Image Restoration
One pass 3×3 Median Three pass 3×3 Median

Salt and Pepper
Two pass 3×3 Median

A little Blurring But less than mean
Chapter 5
Image Restoration
3×3 Max Filter 3×3 min Filter

Chapter 5
Image Restoration
First, additive Then, additive Salt-

Uniform Pepper
5×5 Arithmetic mean 5×5 Geometric mean
5×5 Median 5×5 Alpha Trimmed

Chapter 5
Image Restoration

• Adaptive, local noise reduction:
– If σ2η is small, return g(x,y)
– If σ2L>> σ2η , return value close to g(x,y)
– If σ2L≈ σ2η , return the arithmetic mean mL
σ 2
fˆ ( x, y ) = g ( x, y ) − 2 ( g ( x, y ) − mL )
η
σL

Chapter 5
Image Restoration
Original Noisy A- Mean
G- Mean Local

Chapter 5
Image Restoration
• Adaptive Median: Z min = Minimum Gray Level in the Mask

Z max = Maximum Gray Level in the Mask
Z med = Median Gray Level in the Mask
Z xy = Center of Mask Gray Level
Smax = Maximum Size of the Mask
A1 = Z med − Z min , A2 = Z med − Z max

if ( A1 > 0 and A2 < 0 ) then
⎧⎪ B1 = Z xy − Z min , B2 = Z xy − Z max ⎫⎪
⎨ ⎬
⎪⎩if ( B1 > 0 and B2 < 0 ) then Z xy = Z xy else Z xy = Z med
ˆ ˆ
⎪⎭
else
⎧increase Mask size ( ≤ S max ) and ⎫
⎨ ⎬
⎩ Check " A " con dition. ⎭

Chapter 5
Image Restoration
Noisy Median Adaptive Median

Chapter 5
Image Restoration
• Band Reject Filter (Periodic (sin) Noises)

⎧ W
⎪1 D ( u , v ) < D 0 −
2
⎪
⎪ W W
H I ( u , v ) = ⎨0 D0 − ≤ D ( u, v ) ≤ D0 +
⎪ 2 2
⎪ W
⎪1 D ( u , v ) > D 0 +
⎩ 2
1
H Bn ( u, v ) = 2n
⎡ D ( u, v )W ⎤
1+ ⎢ 2 2⎥
⎣ D ( u , v ) − D 0 ⎦
⎛ 1 ⎡ D 2 ( u, v ) − D 2 ⎤ 2 ⎞
H G ( u , v ) = 1 − exp ⎜ − ⎢ 0
⎥ ⎟
⎜ 2 ⎣ D ( u, v ) W ⎦ ⎟
⎝ ⎠

Chapter 5
Image Restoration
• Band Reject Filter (Periodic Noises)

IBRF BBRF(1) GBRF

Chapter 5
Image Restoration
Spectrum
Noisy
BBRF(1)
De Noisy

Chapter 5
Image Restoration
• Band Pass Filter (Extract Periodic Noises)

– Analysis of Spectrum in different bands
H bp ( u, v ) = 1 − H br ( u, v )

Chapter 5
Image Restoration
• Notch Filter (Single Frequency Removal):

12
D1 ( u , v ) = ⎡( u − M 2 − u0 ) + ( v − N 2 − v0 ) ⎤
2 2
⎣ ⎦
12
D2 ( u , v ) = ⎡( u − M 2 + u0 ) + ( v − N 2 + v0 ) ⎤
2 2
⎣ ⎦
⎧0 D1 ( u , v ) ≤ D0 and D2 ( u , v ) ≤ D0
H I ( u, v ) = ⎨
⎩1 O.W.
1
H B ( u, v ) =
⎡ D02 ⎤
1+ ⎢ ⎥
D
⎣ 1 ( u , v ) D2 ( u , v ) ⎦
⎛ 1 ⎛ D ( u , v ) D2 ( u , v ) ⎞ ⎞
H G ( u , v ) = 1 − exp ⎜ − ⎜ 1 ⎟ ⎟⎟
⎜ 2 D 2
⎝ ⎝ 0 ⎠⎠

Chapter 5
Image Restoration
• Notch Reject Filter
INRF
BNRF
GNRF
NPF=1-NRF
Chapter 5
Image Restoration
•Horizontal Pattern in space

•Vertical Pattern in Frequency
Spectrum Notch in Vertical lines
Notch Notch
Passed Rejected
Spectrum Spectrum
Vertical
Chapter 5
Image Restoration
• Several Single Frequency Noises:
1) Images 2) Spectrum

Chapter 5
Image Restoration
• Multiple Notch will disturb the image:

– Extract interference pattern using Notch Pass
N (u, v) = H (u , v)G (u , v) ⇒ η ( x, y ) = ℑ−1 { H (u, v)G (u, v)}
– Now Perform Filter Function in Spatial Domain as

An Adaptive-Local Point Process:
fˆ ( x, y ) = g ( x, y ) − w ( x, y )η ( x, y )
– Optimization Criteria: minimum variance of

filtered images
{ }
min σ 2fˆ

Chapter 5
Image Restoration
• Optimum Notch Filter:

+d +d 2
1 ⎡ fˆ x + s, y + t ) − fˆ ( x, y ) ⎤
2 ∑ ∑ ⎢ (
σ ( x, y ) =
2
fˆ
( 2d + 1) s =− d t =− d ⎣ ⎦⎥
+d +d
1
fˆ ( x, y ) = ∑ ∑ fˆ ( x + s, y + t )
( 2d + 1)
2
s =− d t =− d
∴
+d +d
1
σ 2
( x, y ) = ∑ ∑ {⎡⎣ g ( x + s, y + t ) − w ( x + s, y + t )η ( x + s, y + t )⎤⎦ −
fˆ
( 2d + 1)
2
s =− d t =− d
}
2
L ⎡ g ( x , y ) − w ( x , y )η ( x , y ) ⎤
⎣ ⎦
Smoothness of w( x, y ):
w ( x + s , y + t ) ≈ w ( x , y ) , w ( x , y )η ( x , y ) ≈ w ( x , y )η ( x , y )

Chapter 5
Image Restoration
• Optimum Notch Filter:

+d +d
1
σ 2
( x, y ) ∑ ∑ {⎡⎣ g ( x + s, y + t ) − w ( x, y )η ( x + s, y + t ) − ⎤⎦
fˆ
( 2d + 1)
2
s =− d t =− d
}
2
L ⎡⎣ g ( x, y ) − w ( x, y )η ( x, y ) ⎤⎦
∂σ 2fˆ ( x, y ) g ( x , y )η ( x , y ) − g ( x , y )η ( x , y )
= 0 ⇒ w ( x, y ) =
∂w ( x, y ) η 2 ( x, y ) − η 2 ( x, y )
• Computed for each pixel using surrounded

mask

Chapter 5
Image Restoration
Former images Spectrum (without fftshift)

Chapter 5
Image Restoration
N(u,v) η(x,y)

Chapter 5
Image Restoration
Filtered Image

Chapter 5
Image Restoration
• Linear Degradation:
L-System:
g ( x, y ) = H ⎡⎣ f ( x, y ) ⎤⎦ + η ( x, y )
+∞ +∞
H ⎡⎣ f ( x, y ) ⎤⎦ = ∫ ∫ f (α , β )H ⎡⎣δ ( x − α , y − β )⎤⎦ dα d β
−∞ −∞
h ( x, y, α , β ) = H ⎡⎣δ ( x − α , y − β ) ⎤⎦
LSI-System:
+∞ +∞
H ⎡⎣ f ( x, y ) ⎤⎦ = ∫ ∫ f (α , β )h ( x − α , y − β ) dα d β
−∞ −∞
g ( x, y ) = f ( x, y ) ∗∗h ( x, y ) + η ( x, y )
G ( u , v ) = F ( x, y ) H ( x, y ) + N ( x, y )

Chapter 5
Image Restoration
• Degradation Estimation:
– Image Observation:
• Look at the image and
– Experiments:
• Acquire image using well defined object (Flat, pinhole,
and etc.)
– Modeling:
• Introduce certain model for certain degradation using
physical knowledge.

Chapter 5
Image Restoration
• Degradation (Using Observation/PSF)

Original Object Degraded Object
H s ( u, v )
H s ( u, v ) =
Fˆ ( u , v )
s
G ( u, v )
H s ( u, v ) =
APSF
Chapter 5
Image Restoration
• Atmospheric Turbulence:

Chapter 5
Image Restoration
• Modeling of turbulence in atmospheric images:
(
H ( u , v ) = exp − k ( u 2 + v 2 )
−5 6
)

Chapter 5
Image Restoration
• Motion Blurring:
– Camera Limitation;
– Object movement.

Chapter 5
Image Restoration
• Motion Blurring Modeling: T

g ( x, y ) = ∫ f ( x − x0 ( t ) , y − y0 ( t ) )dt
0
ℑ
⎛T
+∞ +∞
⎞ − j 2π ( ux + vy )
G ( u, v ) = ∫ ∫ ⎜ ∫ f ( x − x0 ( t ) , y − y0 ( t ) )dt ⎟e dxdy
−∞ −∞ ⎝ 0 ⎠
T
− j 2π ( ux0 ( t ) + vy0 ( t ) )
G ( u, v ) = F ( u, v ) ∫ e dt = F ( u , v ) H ( u, v )
0
• Linear one/Two dimensional motion blurring:

T
x0 ( t ) = at T , tMax = T ⇒ H ( u, v ) = sin (π ua ) e − jπ ua
π ua
T
x0 ( t ) = at T , y0 ( t ) = bt T ⇒ H ( u , v ) = sin (π ( ua + vb ) ) e − jπ (ua + vb )
π ( ua + vb )

Chapter 5
Image Restoration
• Motion Blurring Example:

Chapter 5
Image Restoration
• Motion Blurring Discrete Modeling:

Chapter 5
Image Restoration
• Out of Focus Blurring:

1 ⎛ x2 + y2 ⎞
h ( x, y ) = circ ⎜ ⎟
πR ⎜ R ⎟
⎝ ⎠

Chapter 5
Image Restoration
• Inverse Filtering:
– Without Noise:
G ( u, v ) F ( u, v ) H ( u, v )
F ( u, v ) =
ˆ = ≈ F ( u, v )
Hˆ ( u , v ) Hˆ ( u , v )
Problem of division by zero!
– With Noise:
G ( u, v ) F ( u, v ) H ( u, v ) + N ( u, v ) N ( u, v )
Fˆ ( u , v ) = = ≈ F ( u, v ) +
Hˆ ( u , v ) Hˆ ( u , v ) Hˆ ( u , v )
Problem of division by zero!
Impossible to recover even if H(.,.) is known!!

Chapter 5
Image Restoration
• Pseudo Inverse (Constrained) Filtering:

– Set infinite (large) value to zero;
– Multiply H(u,v) by a I/G/B LPF
⎧ G ( u, v ) ⎧ G ( u, v )
⎪ ˆ Hˆ ( u, v ) ≥ H THR
H ( u, v ) Hˆ ( u, v ) ≥ H THR
⎪ ⎪⎪ Hˆ ( u, v )
Fˆ ( u, v ) = ⎨ or F ( u, v ) = ⎨
ˆ
⎪ G ( u, v ) Hˆ ( u, v ) < H THR ⎪T Hˆ ( u, v ) < H THR
⎪ H ⎪⎩
⎩ THR
– HTHR may be zero

Chapter 5
Image Restoration
Full Band 60 Band
70 Band 85 Band

Chapter 5
Image Restoration
Ghost
Threshold
Effect

Chapter 5
Image Restoration
• Phase Problem:
– Look at this formulation:
⎧ 1
⎪ Hˆ u , v Hˆ ( u , v ) ≥ H THR
⎪ ( )
Fˆ ( u, v ) = G ( u , v ) ⎨
⎪ 1 Hˆ ( u , v ) < H THR
⎪⎩ H THR
– We preserve the Correct Phase!

Chapter 5
Image Restoration
• Phase Problem:

Chapter 5
Image Restoration
• Wiener Filtering (Optimal Linear Filter):

– Preliminary (1D continues case):
u ( t ) :Desired Signal
r ( t ) : Known Degradation Mechanism
n ( t ) : Uncorrelated Noise
c ( t ) : Observation
c (t ) = s (t ) + n (t )
+∞
s (t ) = ∫ r ( t − τ ) u (τ ) dτ
−∞
S ( f ) = R ( f )U ( f )
C( f )
We seek for a optimal Filtering:U% ( f ) = W(f)
R( f )
Chapter 5
Image Restoration

– Preliminary (1D continues case):
+∞ +∞
2
Optimality Criteria: ∫ u% ( t ) − u ( t ) dt = U% ( f ) − U ( f ) df
2
−∞
∫
−∞
2
+∞
2
+∞
S( f )+ N( f ) S( f )
∫ U% ( f ) − U ( f ) df = ∫ W ( f )−
2
E = df
−∞ −∞
R( f ) R( f )
{ } df
+∞
−2
R( f ) S ( f ) 1−W ( f ) + N ( f ) W ( f )
2 2 2 2
∫
2
E =
−∞
We have uses uncorrelated condition for S and N

S( f )
2 2
∂E
= 0 ⇒W ( f ) =
∂W ( f ) S( f ) + N( f )
2 2

Chapter 5
Image Restoration

C ( f ) C ( f ) S ( f )
2
⎡ 1 S ( f )
2
⎤
U(f)=
% W(f )= =⎢ ⎥C( f )
R( f ) R( f ) S ( f ) 2 + N ( f ) 2 ⎢R( f ) S ( f ) 2 + N ( f ) 2 ⎥
⎣14444244443⎦
H(f)
– Note that:
• Wiener-Hopf Filter is Anti Causal!
• We Need Spectrum of Noise (N) and

Chapter 5
Image Restoration
• Wiener Filtering As An Causal FIR Adaptive Filter:

– d(n): Desired output
– x(n): Input signal
– w(n): Adaptive Filter Coefficients
– Y(k): Output of Adaptive Filter
– e(n): Error
W(z) ~ w(n) +
x(n) y(n) + e(n)
-
d(n)
e (n) = d (n) − y (n) = d ( n) − x ( n) ∗ w( n)

Chapter 5
Image Restoration
• Wiener Filtering As An FIR Adaptive Filter:

e (n) = d (n) − y (n) = d (n) − x (n) ∗ w(n)
E {e 2 ( n )} = E {d 2 ( n ) − 2d ( n ) y ( n ) + y 2 ( n )}
xi ( n ) x (n − i)
N
y ( n ) = ∑ wi ( n ) xi ( n ) = w T ( n ) x ( n ) = x T ( n ) w ( n )
i =0
w ( n ) = ( w0 ( n ) wo ( n ) L wN ( n ) )
T
x ( n ) = ( x0 ( n ) xo ( n ) L xN ( n ) )
T
∴
E {e 2 ( n )} = E {d 2 ( n )} − 2 w T ( n ) P + w T ( n ) Rw ( n )
P E {d ( k ) x ( k )}
R E { x ( k ) x T ( k )}
Chapter 5
Image Restoration
• Wiener Filtering as a FIR Adaptive Filter:

if we know P and R :
1 ⎧ ∂e ( n ) ⎫
2
∇ w E {e 2 ( n )} = E ⎨e ( n )
∂
{ }
⎬ = E {e ( n ) x ( n )} = E ( d ( n ) − x ( n ) ) x ( n ) = 0
14444244443
⎩ w ⎭
Orthogonality Principle
⇒ −2 P + 2 Rw ⇒ w = R −1 P
If we discard causality and FIR cinditions:
Pdx ( z )
W (z) =
Pxx ( z )

Chapter 5
Image Restoration
• Wiener Filtering in 2D case:
Degradation De Degradation
f(x,y) g(x,y) g(x,y) f(x,y)
g ( x, y ) = s ( x, y ) + η ( x, y ) ⇔ G ( u , v ) = S ( u , v ) + N ( u , v )
Fˆ ( u , v ) = W ( u, v ) .G ( u, v )
E ( u, v ) = F ( u , v ) − Fˆ ( u , v ) = F ( u, v ) − W ( u , v ) .G ( u, v )
{
E E ( u, v )
2
} = E {( F − WG )( F − WG ) } ∗

Chapter 5
Image Restoration

{
E E ( u, v ) } = P − WP W − W P − WP
2
FF GG
∗ ∗
FG GF
∂E { E ( u, v ) }
2
P ( u, v )
= 0 ⇒ W ( u, v ) = FG
∂W PGG ( u , v )
PXX ( u, v ) = E X { } : Spectral Estimation
2
PXY ( u , v ) = E { XY ∗ } : Cross Spectral Estimation

PXY ( u , v ) = PYX∗ ( u, v )

Chapter 5
Image Restoration

– Special Cases:
• Noise Only:
g ( x, y ) = f ( x, y ) + η ( x, y ) ⇔ G ( u , v ) = F ( u , v ) + N ( u , v )
Independent Noise and Image:
PFF ( u, v )
W ( u, v ) =
PFF ( u , v ) + PNN ( u , v )

Chapter 5
Image Restoration

– Special Cases:
• Degradation plus Noise:
g ( x, y ) = f ( x, y ) ∗ h ( x, y ) + η ( x, y ) ⇔ G ( u , v ) = F ( u , v ) H ( u , v ) + N ( u , v )
Independent Noise and Image:
2 2
PFF H ∗ H∗ 1 H 1 H
W ( u, v ) = = = = −1
PFF H + PNN
2
2 PNN H H 2 + PNN H ⎛ PFF ⎞
H + 2
H +⎜
PFF PFF ⎟
P
⎝12NN ⎠
3
• White Noise SNR
2
1 H
H H2+ K
{
Select Interavtively

Chapter 5
Image Restoration
• Wiener Filter is known as:

– Wiener-Hopf
– Minimum Mean Square Error
– Least Square Error
• Problems with Wiener:
– PFF
– PNN

Chapter 5
Image Restoration
• Phase in Wiener Filter:

⎧ H
⎪ W =
2 P
1 H
2
⎪⎪ H + NN
W= ⇒⎨ PFF
H H 2 + PNN ⎪
⎪ W = − H = ⎛⎜ ⎞⎟
PFF 1
⎪⎩ ⎝H⎠
• No Phase compensation!

Chapter 5
Image Restoration
• Wiener Filter vs. Inverse Filter:

⎧1
H∗ H∗ ⎪ H ≠0
W= ⇒ lim W = = ⎨H
2P PNN →∞
H
2
⎪⎩0
H + NN H =0
PFF
• Other Wiener Related Filter:
1− S
⎛ ⎞
S ⎜ ⎟
⎛1⎞ ⎜ H∗
WS = ⎜ ⎟ ⎟ , 0 ≤ S ≤1
⎝ H ⎠ ⎜ H 2 + PNN ⎟
⎜ PFF ⎟
⎝ ⎠
1 ⎛ ⎛ 1 ⎞⎞
W0.5 = exp ⎜ − j ⎜ ⎟ ⎟
2 PNN ⎝ ⎝ H ⎠⎠
H +
PFF

Chapter 5
Image Restoration
• Nonlinear Filter:
α α −1
Fˆ ( u, v ) = G ( u, v ) e j G ( u ,v )
= G ( u, v ) G ( u, v )
⎧α 1 HP like
⎨
⎩α >> 1 LP like
Fˆ ( u, v ) = ⎨
( )
⎧⎪log G ( u , v ) e j G ( u ,v )
G ( u , v ) > Gmin
⎪⎩0 O.W.

Chapter 5
Image Restoration
Full Inverse Pseudo Inverse Wiener

Chapter 5
Image Restoration
Inverse
Motion Blurring +Noise Wiener

Noise Decrease

Chapter 5
Image Restoration
• Algebraic Approach
– A Matrix Review:
• Matrix Diagonalization: Each square matrix A can be
decomposed into the very special form:
A = PDP -1
• P: Matrix composed of the eigenvectors of .
• D: Diagonal matrix constructed from the corresponding
eigenvalues of A.

Chapter 5
Image Restoration
• Algebraic Approach:
– Matrix Representation of Circular Convolution:
Periodicity: g ( i ) = g ( i ± N )
N −1
y ( n ) = f ( n ) ∗ g ( n ) = ∑ f ( j )g ( n − j )
j =0
In Matrix Form:
⎡ y0 ⎤ ⎡ g 0 g N −1 g N − 2 K g1 ⎤ ⎡ f 0 ⎤
⎢ y ⎥ ⎢g g g K g ⎥⎢ f ⎥
⎢ 1 ⎥ ⎢ 1 0 N −1 2⎥⎢ 1 ⎥
⎢ y2 ⎥ = ⎢ g 2 g1 g0 K g3 ⎥ ⎢ f 2 ⎥
⎢ ⎥ ⎢ ⎥⎢ ⎥
⎢ M ⎥ ⎢ M M M O M ⎥⎢ M ⎥
⎢⎣ yN −1 ⎥⎦ ⎢⎣ g N −1 g N − 2 g N −3 K g 0 ⎥⎦ ⎢⎣ f N −1 ⎥⎦
1444442444443
C
C : A Circulant Matrix!
C (k, j ) = g (k − j )
Chapter 5
Image Restoration
– Diagonalization of Circular Convolution:
C = WDW∗ = WDW −1 , W∗ = W −1
2π
jm n
W ( m, n ) = e N
⎡G ( 0 ) 0 L 0 ⎤
⎢ ⎥
0 G ( 0) L 0
D=⎢ ⎥
⎢ M M O M ⎥
⎢ ⎥
⎢⎣ 0 0 L G ( N − 1) ⎥⎦
G ( k ) = DFT ( g ( n ) ) K

Chapter 5
Image Restoration
– Diagonalization of Circular Convolution:
⎛ ⎞
⎜ ⎛ ∗ ⎞⎟
∴ f ∗ g = ( WDW ) f = W ⎜ D ⎜ {
∗
W f ⎟⎟
⎜ 14⎝ DFT3⎠ ⎟
⎜ 24 ⎟
⎝
144244
Multiply
3⎠
IDFT
– Extension to 2D case is similar!

Chapter 5
Image Restoration
• Algebraic Approach to Noise Reduction:

– Least Square Approach:
• Matrix Representation of Degradation Model:
g = Hf + n ⇒ n = g − Hf
fˆ = arg min n
f
{ } 2
{
= arg min g − Hf
2
}
∂ ( g − Hf )( g − Hf )
2 T
∂ g − Hf
= =0
∂f ∂f
∂ ( g − Hf )( g − Hf )
T
= −2HT ( g − Hf ) = 0
∂f
∴ fˆ = ( HT H ) HT g
−1
14 4244 3
Pseudo Inverse (Left)
( HT H ) HT H = I
−1

Chapter 5
Image Restoration
• Least Square Approach:

– If H is square and H-1 exist then: fˆ = H −1g
– This is Inverse Filter!
⎧⎪ H * ( u, v ) G ( u, v ) ⎫⎪
−1 ⎪ G ( u , v ) ⎪
⎧ ⎫
f ( x, y ) = ℑ ⎨
−1
⎬=ℑ ⎨ ⎬
⎪⎩ H ( )
u , v
2
⎪⎭ ⎪
⎩ H ( u , v ) ⎪⎭
ˆf = ( HT H )−1 HT g = H −1g

Chapter 5
Image Restoration
• Constrained Least Square Approach:

f
{
fˆ = arg min Qf
2
} Constrained to g - Hf
2
= n
2
• Q is a linear operator (High pass components of

image, Laplacian and etc.)
– Using Lagrange Multiplier:
J ( f ) = Qf + λ ( g - Hf − n )
2 2 2
∂J ( f )
= 2QTQf - 2λ H T ( g - Hf )
∂f
−1
⎛ 1 ⎞
∴ fˆ = ⎜ H T H + Q TQ ⎟ H T g = ( H T H + γ Q TQ ) H T g
−1
⎝ λ ⎠
2 2
λ : Lagrange Multiplier which should be be satisfy: g - Hf = n

Chapter 5
Image Restoration
• CLS in frequency domain:

– Diagonalization:
H = WD H W −1 = WD H W* ⎫⎪ ⎪⎧H H = W D H W
T 2 *
T * −1 * *⎬
⇒⎨ 2
H = WD H W = WD H W ⎭⎪ ⎪ QT Q = W DQ W*
⎩
⎛ T
⎜
⎝
H H +
1 T ⎞
λ
Q Q ⎟
⎠
f = H T
g ⇒ W D H
2
+ γ D Q
2
(
W*f = WD*H W*g )
W ×
⎯⎯⎯
−1
(
→ D H + γ DQ
2 2
) (
W*f = D*H W*g ⇒ H ( u , v ) + γ C ( u , v )
2 2
) Fˆ (u, v ) = H (u, v ) G (u, v )
*
C ( u , v ) : Fourier Transform of Q (Linear Operator!)

H * ( u, v )
⇒ Fˆ ( u, v ) = G ( u, v )
H ( u, v ) + γ C ( u, v )
2 2

Chapter 5
Image Restoration
• CLS in frequency domain:

H * ( u, v )
Fˆ ( u, v ) = G ( u, v )
H ( u, v ) + γ C ( u, v )
2 2
−1
⎛ 1 ⎞
fˆ = ⎜ H T H + Q TQ ⎟ H T g
⎝ λ ⎠

Chapter 5
Image Restoration
• Wiener Filter vc. CLS filter:

⎧R f = E {ff } ⎫ ⎧ H *
( u, v ) ⎫
( ) ( )⎪
T
ˆ
F u , v = G u , v
⎪ ⎪ ⎪ H ( u, v ) + γ C ( u, v )
2 2
⎪ T ⎪ ⎪ ⎪
⎨ nR = E { }⎬ ⎨
nn ⇔
−1
⎬
⎪ T −1 ⎪ ⎪ˆ ⎛ T 1 T ⎞ ⎪
⎪⎩Q Q = R R n⎪ f = H H + Q Q H T
g
f
⎭ ⎪⎩ ⎜⎝ λ ⎟
⎠
⎪
⎭
γ = 0 ↔ Inverse Filter
γ = 0 ↔ Wiener Filter

Chapter 5
Image Restoration
• For Laplacian as a measure of smoothness:

⎡C0 CM −1 CM − 2 K C1 ⎤
⎡ 0 −1 0 ⎤ ⎢C C0 CM −1 K C2 ⎥⎥
P ( x, y ) = ⎢⎢ −1 4 −1⎥⎥ ⎢ 1
Q = ⎢ C2 C1 C0 K C3 ⎥
⎢⎣ 0 −1 0 ⎥⎦ ⎢ ⎥
⎢M M M O M ⎥
⎢⎣CM −1 CM − 2 CM −3 K C0 ⎥⎦
⎡ Pe ( j , 0 ) Pe ( j , N − 1) L Pe ( j ,1) ⎤
⎢ ⎥
M M M M
Cj = ⎢ ⎥
⎢M M O M ⎥
⎢ ⎥
⎢⎣ Pe ( j , N − 1) L L Pe ( j , 0 ) ⎥⎦

Chapter 5
Image Restoration
• How to estimate γ:
Residual Vector: r = g - Hfˆ
It can be shown that ϕ ( r ) = r = rT r is monotically increasing with r.
2
2 2
We seek for γ such that r = n ± δ , δ is our accuracy
1. Initial Guess on γ
2
2. Calculate r = g - Hfˆ
2
3. If criteria ( r = n ± δ ) is not reached :

2 2
3.a if ( r < n − δ ) increase γ .

2 2
3.b if ( r > n + δ ) decrease γ .

2 2
4. Re-calculate new filter and repeat until criteria is meeted

Chapter 5
Image Restoration
• How to estimate γ:
– Computational Tips:
2 2
We need r and n .
M −1 N −1
.R ( u , v ) = G ( u , v ) − H ( u , v ) Fˆ ( u , v ) ⇒ r = ∑ ∑ r 2 ( x, y )
2
x =0 y =0
M −1 N −1 2
1
σ n2 =
MN
∑ ∑ n ( x, y ) − m
x =0 y =0
n
M −1 N −1
1
mn =
MN
∑ ∑ n ( x, y )
x =0 y =0
M −1 N −1
n = ∑ ∑ n 2 ( x, y )
2
x =0 y =0
∴ n = MN (σ n2 + mn2 )
2
Need only σ n2 and mn2

Chapter 5
Image Restoration
High Noise Mid Noise Low Noise

Chapter 5
Image Restoration
Correct Noise Parameter Incorrect Noise Parameter

Chapter 5
Image Restoration
• Computation Efforts:
g = Hf + n ⇒ n = g − Hf
fˆ = ( HT H ) HT g
−1
H is size of MN × MN (65536 × 65536)!!!!

• Gradient Descent Methods of function minimization:
∂Φ ( x )
min Φ ( x ) ⎯⎯→ x( k +1) = x( k ) − µ
GD
= x( k ) − µ∇ Φ ( x( k ) )
∂x (k )
2
Convergence Condition: 0 ≤ µ ≤
max { λ }
λ : Eignvalue of E {xxT }

Chapter 5
Image Restoration
• Iterative LS or CLS (Tikhonov-Miller):

LS :
g = Hf + n ⇒ n = g − Hf
2
1 ∂ g − Hf
Φ ( f ) = g − Hf = ∇ Φ = − HT ( g − Hf )
2
⇒
2 ∂f
1. f( 0) = µ HT g
( )
2. f( k +1) = f( k ) + µ HT g − Hf( k ) = µ HT g + ( I − µ HT H ) f( k ) = f( 0) + ( I − µ HT H ) f( k )
CLS :
∂J ( f )
J (f ) =
1
2
(Qf
2
(
+ λ g - Hf
2
− n
2
)) ⇒
∂f
= Q T Qf - λ H T ( g - Hf )
1. f( 0) = µ HT g
( ( ) ) (
2. f( k +1) = f( k ) + µ λ H T g - Hf( k ) − Q T Qf( k ) = µλ HT g + I − µ ( λ HHT + Q T Q ) f( k ) )

Uniform Gaussian
Blurring Blurring
GIF GIF
Thr=1e-3 Thr=1e-3

Uniform Gaussian
Blurring Blurring
GIF GIF
Thr=1e-1 Thr=1e-1

Gaussian
Uniform Blurring
Blurring
CLS CLS
γ=1 γ=1

Gaussian
Uniform Blurring
Blurring
CLS CLS
γ=1e-4 γ=1e-4

Chapter 5
Image Restoration

Chapter 5
Image Restoration
• Wiener!

Chapter 5
Image Restoration
• Iterative Wiener Filter:

– Wiener in Matrix form (Spatial Filter):
g = Hf + n
( HR H + R n ) , T is Hermitian Transform!
−1
W = Rf H T
f
T
fˆ = Wf
R f : Autocorrelation of f
R n : Autocorrelation of n

Chapter 5
Image Restoration

– Conditions:
1. The original image and noise are statistically
uncorrelated.
2. The power spectral density (or autocorrelation) of the
original image and noise are known.
3. Both original image and the noise are zero mean.

Chapter 5
Image Restoration

– Iterative Algorithm:
1. R f ( 0 ) = R g
2. W ( i + 1) = R f ( i ) HT ( HR f ( i ) HT + R n )
−1
3. fˆ ( i + 1) = W ( i + 1) g
{ }
4. R f ( i + 1) = E fˆ ( i + 1) fˆ T ( i + 1)
5. Repeat 2,3,4 until convergence.


Chapter 5
Image Restoration
• Adaptive Wiener Filter:

– Image are Non Stationary!
– Need Adaptive WF which is locally optimal.
– Assume small region which image are stationary
• Image Model in each region:
f ( x, y ) = µ f ( x, y ) + σ f η ( x, y )
η : zero-mean white noise with unit variance!
µ f , σ f : Constant over each region.
• Noise Image:
g ( x, y ) = f ( x, y ) + v ( x, y ) , σ v : Constant over each region

Chapter 5
Image Restoration
• Local Wiener Filter in each region:

Pff σ 2f
Wa ( u, v ) = = 2
Pff + Pvv σ f + σ v2
σ 2f
wa ( x, y ) = 2 δ ( x, y )
σ f +σv 2
fˆ ( x, y ) = ( g ( x, y ) − µ f ) ∗ wa ( x, y ) + µ f
σ 2
fˆ ( x, y ) = ( g ( x, y ) − µ f ) 2 + µf
f
σ f + σ v2
µ f ( x, y ) : Low-pass filtered on noisy image.
µ f ( x, y ) = µ g ( x, y ) , Zero mean assumption
g ( x, y ) − µ f ( x, y ) : Hi-pass filtered on noisy image.
σ f ( m, n )
2
∴ fˆ ( x, y ) = HP ( m, n ) 2 + LP ( m, n )
σ f ( m, n ) + σ v
2

Chapter 5
Image Restoration
• Parameter Estimation:
σ g2 = Local Noisy Image Variance
σ v2 = Variance in a smooth image region or background
σ 2f ( m, n ) = σ g2 ( m, n ) − σ v2

Chapter 5
Image Restoration
• Results:

Chapter 5
Image Restoration
• Results:

Chapter 5
Image Restoration
• Results:

Chapter 5
Image Restoration
• Results:

Chapter 5
Image Restoration

Chapter 5
Image Restoration

Chapter 5
Image Restoration

Chapter 5
Image Restoration

Chapter 5
Image Restoration

Chapter 5
Image Restoration

Chapter 5
Image Restoration

Chapter 5
Image Restoration

Chapter 05

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Chapter 05

Uploaded by

Copyright:

Available Formats

Digital Image Processing, 2nd ed. www.imageprocessingbook.

• Image Degradation Model (Linear/Additive)

© 2002 R. C. Gonzalez & R. E. Woods

© 2002 R. C. Gonzalez & R. E. Woods

© 2002 R. C. Gonzalez & R. E. Woods

© 2002 R. C. Gonzalez & R. E. Woods

© 2002 R. C. Gonzalez & R. E. Woods

© 2002 R. C. Gonzalez & R. E. Woods

© 2002 R. C. Gonzalez & R. E. Woods

© 2002 R. C. Gonzalez & R. E. Woods

© 2002 R. C. Gonzalez & R. E. Woods

– Spectral Estimation (several subimages)

© 2002 R. C. Gonzalez & R. E. Woods

• Noise-only spatial filter:

© 2002 R. C. Gonzalez & R. E. Woods

Noise Reduction vs. Blurring

– ≈ same smoothing with less detail degradation:

© 2002 R. C. Gonzalez & R. E. Woods

• Salt Reduction (Pepper will increase)/ Good for

© 2002 R. C. Gonzalez & R. E. Woods

© 2002 R. C. Gonzalez & R. E. Woods

© 2002 R. C. Gonzalez & R. E. Woods

• Noise-only spatial filter:

© 2002 R. C. Gonzalez & R. E. Woods

• Median: Most Familiar OS filter:

– Little Blurring/Single/Double valued noises

– Find Brightest/Darkest Points (Pepper/Salt Reducer)

© 2002 R. C. Gonzalez & R. E. Woods

• Alpha Trimmed mean:

© 2002 R. C. Gonzalez & R. E. Woods

One pass 3×3 Median Three pass 3×3 Median

© 2002 R. C. Gonzalez & R. E. Woods

3×3 Max Filter 3×3 min Filter

© 2002 R. C. Gonzalez & R. E. Woods

First, additive Then, additive Salt-

5×5 Arithmetic mean 5×5 Geometric mean

5×5 Median 5×5 Alpha Trimmed

© 2002 R. C. Gonzalez & R. E. Woods

• Noise-only spatial filter:

© 2002 R. C. Gonzalez & R. E. Woods

Original Noisy A- Mean

© 2002 R. C. Gonzalez & R. E. Woods

• Adaptive Median: Z min = Minimum Gray Level in the Mask

A1 = Z med − Z min , A2 = Z med − Z max

© 2002 R. C. Gonzalez & R. E. Woods

Noisy Median Adaptive Median

© 2002 R. C. Gonzalez & R. E. Woods

• Band Reject Filter (Periodic (sin) Noises)

© 2002 R. C. Gonzalez & R. E. Woods

• Band Reject Filter (Periodic Noises)

© 2002 R. C. Gonzalez & R. E. Woods

© 2002 R. C. Gonzalez & R. E. Woods

• Band Pass Filter (Extract Periodic Noises)

© 2002 R. C. Gonzalez & R. E. Woods

• Notch Filter (Single Frequency Removal):

© 2002 R. C. Gonzalez & R. E. Woods

• Notch Reject Filter

•Horizontal Pattern in space

Spectrum Notch in Vertical lines

• Several Single Frequency Noises:

© 2002 R. C. Gonzalez & R. E. Woods

• Multiple Notch will disturb the image:

– Now Perform Filter Function in Spatial Domain as

– Optimization Criteria: minimum variance of