Dip08 Restoration

17/11/2020
Gianni Ramponi Gianni Ramponi

http://www.units.it/ramponi Digital Image Processing http://www.units.it/ramponi Digital Image Processing
Image Restoration Image Restoration

Image degradation model Noise models
Typical pdf’s of i.i.d. noise
Note 1: In the additive model,
zero-mean Gaussian and
Uniform noises are typically
used
Note 2: In real cases, noise is
often signal-dependent, e.g.
multiplicative (with mean=1) 
does not comply with the model
we are using
g ( x, y )  h ( x, y )  f ( x, y )  n ( x, y )
G(u , v )  H (u, v) F (u , v)  N (u , v)
Note: even small amounts of noise can lead
to large errors in restoration  dip08_0.m
Copyright notice: Most images in these slides are

© Gonzalez and Woods, Digital Image Processing, Prentice-Hall


Noise models Noise models
Gaussian noise is typical in sensors, especially in low lighting conditions

Impulse (salt-and-pepper) noise comes from disturbed switching devices
Silver halide grains in photographic films yield lognormal distributions
Rayleigh noise (multipl.) arises in range images (n-look SAR images)
Exponential noise (multipl.) is present in laser imaging (1-look SAR)
For simulation purposes, a noise having a specified cdf Fz can be

generated starting from a uniform random field w by the inverse function
mapping: z  F 1 ( w)
z
(like in the case of histogram specification)
(Matlab: use rand and randn, or imnoise  [NoiseGeneration.m])
17/11/2020


Note:
noise with
mean>0
was used
in these
images
black, gray1, gray2
In general, the noise distribution is not easily recognizable

by image inspection.


- black+
pepper
- gray 1
- gray 2
- salt
17/11/2020

Image Restoration Nonlinear mean filters

Noise models
g ( x, y )  f ( x, y )  n ( x, y ) ; G (u, v)  F (u , v)  N (u , v )
We are looking for the best compromise between noise attenuation and
Noise estimation from a locally uniform area of the image detail preservation.
Let Sxy be an mxn neighborhood of (x,y); define various mean filters:
Arithmetic mean 1
fˆ ( x, y )   g ( s, t )
mn ( s ,t )Sxy
1
Geometric mean   mn
(always smaller than the fˆ ( x, y )    g ( s, t ) 

arithm.mean)  ( s ,t )Sxy 
[Contraharmonic mean] fˆ ( x, y )   g ( s, t )
( s ,t )Sxy
Q 1
 g ( s, t )
( s ,t )Sxy
Q
Q=0  arithmetic mean;

Q>0  good for dark impulse noise
Q<0  good for light impulse noise

Order statistics filters Order statistics filters

Median filter
Max filter Min filter

17/11/2020

Weighted Median (WM) filters depend also on the position of the

gray levels within the window
WM filter of span N associated with integer weights w= [w1,w2,...,wN]: Alpha-trimmed mean filter:
fˆ ( x)  median{gW (s )} Sort the pixels in Sxy and delete the first d/2 and the last d/2; let S’xy
s  Sx
where gW is obtained from g by duplication of its elements: be the set of the remaining pixels. The output is:
k
 
gW (s )  [ w1 g (1), w2  g (2), ... , wN  g ( N )] , and k  g  g , g , ... , g 1
fˆ ( x, y ) 
mn  d
 g ( s, t )
( s ,t )S ' xy
I.e.: duplicate each sample g(i) by the number of the corresponding
weight wi, and choose the median value from the new sequence. Defaults to the arithmetic mean filter when d=0 and to the median
Example: length-5 WM filter with integer weights [1, 2, 3, 2, 1]. Input filter when d=mn-1.
sequence g= [-1, 5, 8, 11, -2], window centered at sample value 8. Good for mixed short- and long-tailed noise
After sorting and duplication: gW=[11, 11, 8, 8, 8, 5, 5, -1, -2]. Output
is 8, whereas the 5-point median would output 5.
Note: Central WM (CWM) filters are particularly important in this class

 NonlinearFilters.m
c d e f
a a
b b
17/11/2020

Adaptive linear filters Adaptive linear filters
The response of the filter changes according to the local properties of

the image. Used features:
mL ,  L2 local average, local variance of the pixels in Sxy
 2
n
variance of the noise corrupting f(x,y) (to be estimated!)
The output is (Wiener filter-like approach, see [Lim p.538]):
 2
fˆ ( x, y )  g ( x, y )  [ g ( x , y )  mL ]
n
 2
L
Note1: it is reasonable to suppose that  L2   n2 everywhere

but for safety truncate  / 1   1
2 2
n L
Note2: good noise estimate:  n2  mean( L2 ) (Matlab’s wiener2)

Adaptive median filters Adaptive median filters
Define:
mi  min {g ( s, t )} ; ma  max {g (s, t )} ; md  median{g (s, t )} ;

( s ,t )Sxy ( s ,t )Sxy ( s ,t )Sxy
Algorithm:
window size = minsize:
while (window size < maxsize) do
if (md=mi or md=ma) increase window size;
end do;
if (mi<g(x,y)<ma) output g(x,y), else output md;
Useful for images corrupted by heavy impulse noise.

17/11/2020

Rational filter Rational filter

for speckle noise
On a 3x3 support:
A linear lowpass filter can be combined with an edge 1 2 3

sensor to achieve edge-preserving noise smoothing 8 0 4
a  c  2b d  e  2b 7 6 5
b'  b  
k ( a  c )  5  W k ( d  e) 2  5  W
2
k: edge protection factor 1

0.8
W: lowpass filter control factor 0.6
0.4
• k = 0, W = 0: b’ = (a+b+c+d+e) / 5
0.2
• k = 0, W > 0: b’ = [(a+c+d+e-4b)+(5+W)b] / (5+W)

0 1 2 3
1
0.8
k: edge protection factor, range [700, 1100]
= [(a+c+d+e)+(1+W)b] / (5+W) 0.6 A: lowpass filter control factor, range [4.0, 5.0]
• k > 0: smoothing only along edges 0.4
no. of passes: [6, 10]
0.2
Note 1: filter is meant to operate iteratively 0 1 2 3

• note k=0; A=9  3x3 average filter
Note 2: this is a sort of Anisotropic Diffusion • if k>0 the denominator is "locally larger" than A
Note 3: asymptotic output is a uniform image
zz08_speckle_spl97

Bilateral filter BM3D filter – Noise modelling

Combined domain and range filtering:
The closeness function and
the similarity function
can be Gaussian functions of the Euclidean distance d or absolute

difference d between their arguments.
• d,r are the geometric (domain) spread and the photometric

Foi07 & Rakhshanfar16
(range) spread parameters
• Fast realization techniques exist zz08_FastBilateral_ellet06

17/11/2020

Noise smoothing Noise smoothing

ISO/ASA is a sensitivity measure for photographic film, i.e. for
signal amplification in a digital camera Noise smoothing in a professional camera (JPEG output)
High ASA setting  small signal  large amplification  large noise
100 ASA, 6 s
• Noise smoothing in a consumer camera:
3200 ASA, 1/8 s
100 ASA, 1.3 s 1600 ASA, 0.1 s

Frequency-selective filters Frequency-selective filters
Let D(u,v) be the distance from the origin of a given frequency.   D(u , v) W  2 n 
Band-reject filters (reject center frequency D0, reject bandwidth W):
Butterworth H (u, v)  1 1   2  
2 
  D (u, v)  D0  
1 if D(u,v)  D0  W / 2 and Gaussian
Ideal   1  D 2 (u, v)  D 2  2 
H (u , v)  0 if D0  W / 2  D(u,v)  D0  W / 2 band-reject filters: H (u, v)  1  exp   0
 
1  2  D(u , v) W  
 if D(u,v)  D0  W / 2
17/11/2020

Band-pass filter: H BP (u, v)  1  H BR (u, v)

Notch filters
Let
D1 (u, v)  (u  M / 2  u0 ) 2  (v  N / 2  v0 ) 2
D2 (u, v )  (u  M / 2  u0 ) 2  (v  N / 2  v0 ) 2
distances of center-shifted freq. (u,v) from freqs. (u0 ,v0) and (-u0 ,-v0)
0 if D1(u,v)  D0 or D2(u,v)  D0
Ideal H (u, v)  
1 otherwise
  D02  
n
Note: for the
Butterworth H (u , v)  1 1     filter coeff.
  D1 (u, v) D2 (u, v)   to be real,
notch areas
 1  D (u , v) D (u , v)   must always
Gaussian H (u , v)  1  exp   1 2
  be defined in
 2  D02   symmetric
pairs
17/11/2020

Frequency-selective filters Image Restoration

Image degradation model
“Notch-pass” filter: Linear, space-invariant degradation model with impulse response

h (“Point-spread function”, PSF):
Hnp(u,v)=1-Hnr(u,v) g ( x, y )  h( x, y )  f ( x, y )  n( x, y )
G (u, v)  H (u , v) F (u , v)  N (u , v)
Note that space invariance is an often unrealistic constraint (think of

haze in outdoor scenes, motion of different objects, distance from
moving objects...)
Note: The notch-pass filter here is simply a vertical highpass filter

Estimating the PSF Estimating the PSF

By mathematical modelling:
1. lens defocus NOTE:
By image observation: Defocus normally
Look at a detail of the degraded image, in a high SNR area, and is shift-variant
construct its estimated version f est .Then, H (u , v)  G (u , v) Fest (u , v )
By experimentation:
Thin-lens eq. for
Suppose that equipment similar to the one used for acquisition be defocused object
available; then it is possible to obtain an accurate estimation of the
degradation by imaging an impulse of amplitude A using the same f/# = F / A
system settings. Then, H (u , v )  G (u , v ) A f-number =
focal length / aperture
diameter
Useful also for

depth-from-focus
algorithms
17/11/2020

Estimating the PSF Estimating the PSF
2. atmospheric turbulence:
(similar to a Gaussian) H (u, v)  exp[k (u 2  v 2 )5 / 6 ]
3. linear planar motion, described by the components x(t) and y(t),

in the acquisition interval [0,T]:
Fig.5.25
T
g ( x, y )   f [ x0  x (t ), y0  y (t )] dt
0
T
G (u, v )  F (u , v)  exp[ j 2 [ux(t )  vy(t )]] dt  F (u, v ) H (u , v )
0
e.g. uniform horizontal motion:
x (t )  at T ; y (t )  0
Note: H should be space-
variant to take into T T
H (u , v)   exp[ j 2 u at T ] dt  sin( ua) exp[ j ua]
account the different 0  ua
distances of the objects

Estimating the PSF Inverse filter
Let G (u, v)  H (u, v) F (u , v)  N (u , v)

Suppose we know H. The most obvious restoration operator is
G (u , v ) N (u , v)
 Fˆ (u, v)   F (u , v) 
H (u , v) H (u, v)
Note: if H has zeros on the unit circle or is nonminimum-phase,

one can try to cap the amplitude of 1/H
Note: where the degradation is small, noise can dominate. If H
has zeros close to the unitary circle, 1/H has large peaks. This
often happens at high frequencies  set 1/H = 1 above a given
distance from the origin
E.g., starting from Fig.5.25b, (image size 480x480, Butterworth

response used to cap 1/H at various frequencies):
17/11/2020

Inverse filter Wiener filter
Wiener filter: 1-D derivation [Jain]
Given two zero-mean, stationary random sequences f(x) and g(x), it is

desired to obtain from g(x) a linear estimate of f(x) that minimizes the
MSE: 
fˆ ( x)  h
s  
W ( x  s) g (s) , e 2  E{[ f ( x)  fˆ ( x )]2 }  min
It can be proved (orthogonality principle) that the best estimate is

obtained from a filter hw(k) such that:

 x, y 0  E{[ f ( x)  fˆ ( x)] g ( y )}  E{[ f ( x )  h
s  
W ( x  s ) g ( s )] g ( y )}
Using the definition of cross-correlation between two real-valued

stationary sequences a(x), b(y): rab ( x  y )  E{a ( x )b( y )}
and substituting from above, the orthogonality condition becomes the
Wiener filter equation: 
h
s  
W ( x  s ) rgg ( s  y )  rfg ( x  y )

Wiener filter Wiener filter
For nonzero-mean sequences and zero-mean noise: subtract the

average from the available data, perform the filtering, and then add
Taking the Fourier transform we get the [cross] power spectral
the average back.
densities and we can solve for Hw:
1
H W (u ) S gg (u )  S fg (u )  H W (u )  S fg (u ) S gg (u )
In the 2-D case:
2
H * (u , v) S ff (u, v) 1 H (u , v)
Now, if the usual model is taken for the corruption mechanism: H W (u , v)  
2
H (u , v) S ff (u, v )  S nn (u, v) H (u , v) H (u, v ) 2  S nn (u , v) S ff (u , v)
g ( x)  h( x)  f ( x)  n( x)  G (u )  H (u ) F (u )  N (u )
the needed power spectra are obtained from: Note: if the noise is zero, the Wiener filter becomes the inverse filter
2
S gg (u )  H (u ) S ff (u )  S nn (u ) ; S fg (u )  H (u ) S ff (u )
* Note: when both noise and blur are present, the Wiener filter seeks a
compromise between lowpass noise smoothing and highpass
deblurring. The result typically is a bandpass filter.
Note: if the spectrum of the original image is not known, one can
H * (u ) S ff (u ) use:
And the Wiener filter is expressed as: H W (u )  2
H (u , v)
2
H (u ) S ff (u )  S nn (u ) 1
H W (u , v) 
H (u , v) H (u , v) 2  K
17/11/2020



17/11/2020

Constrained Least Squares filter Constrained Least Squares filter
In the Fourier domain: minimize J  || P(u, v) Fˆ (u, v) || 2

An extension to the simplified Wiener filter seen last. Suppose we know
the noise mean and variance. subject to || G (u, v )  H (u , v) Fˆ (u , v) || 2  || N (u, v ) || 2
Approach: (~Jain) minimize a cost function J  || p( x, y ) * f ( x, y ) ||
ˆ 2
subject to the constraint || g ( x, y )  h( x, y )  fˆ ( x, y ) || 2  || n( x, y ) || 2 A solution can be obtained via

1 H (u , v )
2
the Lagrange multiplier method: H LS (u, v) 

H (u , v ) H (u , v ) 2   P (u , v ) 2
Normally, p is an operator that measures the “roughness” of the
estimated image, e.g. its Laplacian: Gamma can be set iteratively: compute D  || G (u , v)  H (u, v) Fˆ (u , v) || 2
 0 1 0 
and increase gamma if D  || N (u , v) || 2 , decrease it if D  || N (u, v) || 2
p ( x, y )   1 4  1
-- Notes:
 0  1 0 
|| N (u, v) || 2 is the noise power (the variance if the noise is zero-mean)
I.e.: we want the output image to be as smooth as possible, but not The LS filter is equivalent to the Wiener filter if we specify
too different from an estimate of the original S nn (u, v)   and S ff (u , v)  1 | P(u, v) |2

Constrained Least Squares filter Constrained Least Squares filter
5.25-b
•  values set manually

Similar to Fig.5.28(c)
• LS is better than Wiener when the noise is high or medium
17/11/2020
Gianni Ramponi
http://www.units.it/ramponi Digital Image Processing

Dip08 Restoration

Uploaded by

Copyright:

Available Formats

You might also like

Dip08 Restoration

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Dip08 Restoration

Uploaded by

Copyright:

Available Formats

17/11/2020

Gianni Ramponi Gianni Ramponi

Image Restoration Image Restoration

Copyright notice: Most images in these slides are

Gianni Ramponi Gianni Ramponi

Image Restoration Image Restoration

Gaussian noise is typical in sensors, especially in low lighting conditions

For simulation purposes, a noise having a specified cdf Fz can be

Gianni Ramponi Gianni Ramponi

Image Restoration Image Restoration

black, gray1, gray2

In general, the noise distribution is not easily recognizable

Gianni Ramponi Gianni Ramponi

Image Restoration Image Restoration

Gianni Ramponi Gianni Ramponi

Image Restoration Nonlinear mean filters

(always smaller than the fˆ ( x, y )    g ( s, t ) 

Q=0  arithmetic mean;

Gianni Ramponi Gianni Ramponi

Order statistics filters Order statistics filters

Max filter Min filter

Gianni Ramponi Gianni Ramponi

Order statistics filters Order statistics filters

Weighted Median (WM) filters depend also on the position of the

Gianni Ramponi Gianni Ramponi

Order statistics filters Order statistics filters

Gianni Ramponi Gianni Ramponi

Adaptive linear filters Adaptive linear filters

The response of the filter changes according to the local properties of

Note1: it is reasonable to suppose that  L2   n2 everywhere

Note2: good noise estimate:  n2  mean( L2 ) (Matlab’s wiener2)

Gianni Ramponi Gianni Ramponi

Adaptive median filters Adaptive median filters

mi  min {g ( s, t )} ; ma  max {g (s, t )} ; md  median{g (s, t )} ;

Useful for images corrupted by heavy impulse noise.

Gianni Ramponi Gianni Ramponi

Rational filter Rational filter

A linear lowpass filter can be combined with an edge 1 2 3

k: edge protection factor 1

• k = 0, W > 0: b’ = [(a+c+d+e-4b)+(5+W)b] / (5+W)

Note 1: filter is meant to operate iteratively 0 1 2 3

Gianni Ramponi Gianni Ramponi

Bilateral filter BM3D filter – Noise modelling

The closeness function and

the similarity function

can be Gaussian functions of the Euclidean distance d or absolute

• d,r are the geometric (domain) spread and the photometric

• Fast realization techniques exist zz08_FastBilateral_ellet06

Gianni Ramponi Gianni Ramponi

Noise smoothing Noise smoothing

Gianni Ramponi Gianni Ramponi

Frequency-selective filters Frequency-selective filters

Gianni Ramponi Gianni Ramponi

Frequency-selective filters Frequency-selective filters

Band-pass filter: H BP (u, v)  1  H BR (u, v)

Gianni Ramponi Gianni Ramponi

Frequency-selective filters Frequency-selective filters

Gianni Ramponi Gianni Ramponi

Frequency-selective filters Image Restoration