23

Image Degradation and Restoration

and Model of Image Degradation
and Restoration process
© Dr. Dafda
❖ What is Image Degradation and Restoration?
• Image restoration is to restore a degraded image back to the original image while
image enhancement is to manipulate the image so that it is suitable for a specific
application.
• Image restoration attempts to restore images that have been degraded
– Identify the degradation process and attempt to reverse it
– Similar to image enhancement, but more objective

Periodic noise
Periodic noise can be reduced by
looks like dots setting frequency
In the frequency components
domain corresponding to
noise to zero.
 ❖ Causes of Image degradation:
1. Noise due to image sensor
2. Blurring due to miss-focus
3. Blurring due to motion
4. Noise from the transmission channel
5. Blur due to transmission channel
6. Atmospheric turbulence
7. Improper opening of closing of camera shutter

Restoration is the process of recovering an

image that has been degraded. It uses
mathematical or probabilistic models.
❖ Comparison between Image Enhancement & Restoration:
Sr. Image Enhancement Image Restoration
No:
1. The output image is enhanced for The output image is restored, as if it would
specific application. have been recorded without degradation.
2. It makes image look better without It tries to fix the image to get back to the real,
regard to how it truly should look. true image.
3. This means improving image to show This means improving image to match the
some hidden details. original image.
4. This is purely subjective. This is purely objective.
5. This is a cosmetic procedure of This is a reverse procedure of getting an
improving quality by working with optimal estimate of the desired result.
existing data.
❖ Model of Image degradation and restoration process

where h(x,y) is a system that causes image degradation and ƞ(x,y) is noise.
 24

Noise Models with examples

in Digital Image Processing
© Dr. Dafda
❖ Model of Image degradation and restoration process

where h(x,y) is a system that causes image degradation and ƞ(x,y) is noise.
❖ Noise and Images
• The sources of noise in digital images arise during image acquisition
(digitization) and transmission

– Imaging sensors can be affected by ambient conditions

– Interference can be added to an image during transmission

• To understand the restoration process, it is important to understand the

noise function ƞ(x,y).

• Noise cannot be predicted but can be approximately described in statistical

way using the probability density function (PDF).
❖ Noise Types
There are many different models for the image noise term η(x, y):
– Gaussian
• Most common model
– Rayleigh
– Erlang (Gamma)
– Exponential
– Uniform
– Impulse
• Salt and pepper noise
 Rayleigh noise
Gaussian noise:
 2
( z − a )e −( z − a ) / b for z  a
2

1 − ( z −  ) / 2 p( z ) =  b
p( z ) =
2 2
e
2  0 for z  a

Erlang (Gamma) noise Exponential noise

 a b z b−1 p( z ) = ae− az
 ( z − a )e −az for z  0
p( z ) =  (b − 1)!
 0 for z  0

Uniform noise Impulse (salt & pepper) noise

 1  Pa for z = a
for a  z  b 
p( z ) =  b - a p( z ) =  Pb for z = b
 0 otherwise 0
 otherwise
PDF tells how much each z value occurs.
❖ Noise Example
• The test pattern to the right is
ideal for demonstrating the
addition of noise
• The following slides will show the
result of adding noise based on
various models to this image
 25
Periodic Noise, Noise Estimation
and Band Pass/Band Reject
filters and its implementation in
MATLAB © Dr. Dafda
Periodic Noise Reduction by Freq. Domain Filtering
Degraded image DFT

Periodic noise
Periodic noise can be reduced by
looks like dots setting frequency
In the frequency components
domain corresponding to
noise to zero.

Band reject filter Restored image

❖ What is Periodic Noise
• Periodic noise is sinusoidal at multiples of specific frequencies.
• A common source of periodic noise in an image is from
electrical/electromechanical interference during the image capturing process.
• An image affected by periodic noise will look like a repeating pattern has been
added on top of the original image. It can be reduced significantly via frequency
domain filtering
❖ Estimation of Noise parameters
• One way of estimation of periodic noise is by inspection of the Fourier spectrum
of the image.
• The parameters of the noise pdf can be estimated partially from sensor
specifications.
• Also sensor noise can be estimated from small patches of reasonably constant
background intensity.
❖ Band Reject Filters
Use to eliminate frequency components in some bands
❖ Band Pass Filters

Can be obtained from band reject filters.

❖ Notch Reject Filters
A notch reject filter is used to eliminate frequency components in predefined
neighborhood about a center frequency.
 26

Restoration in the Presence of Noise

Only - Spatial Filtering
© Dr. Dafda
❖ Model of Image degradation and restoration process

where h(x,y) is a system that causes image degradation and ƞ(x,y) is noise.
 Spatial filters

Sharpening spatial filters (HPF)

Smoothing spatial filters (LPF)

Linear filters Non-linear(Order statistics) spatial filter Laplacian linear filter

Averaging filter Weighted averaging filter

Minimum filter Maximum filter Median filter

Mean Filters:
Arithmetic mean filter or moving average filter (from Chapter 3)

ˆf ( x, y ) = 1
 g ( s, t )
mn ( s ,t )S xy
Noise is reduced due to blurring

mn = size of moving window

Geometric mean filter

1
  mn
fˆ ( x, y ) =   g ( s, t ) 
Here also noise is reduced,
 ( s ,t )S  but it loses less image details
 xy 
Geometric Mean Filter: Example

Image
Original
corrupted
image
by AWGN

Image Image
obtained obtained
using a 3x3 using a 3x3
arithmetic geometric
mean filter mean filter
Harmonic and Contraharmonic Filters
Harmonic mean filter
mn Works well for salt noise
fˆ ( x, y ) =
1 but fails for pepper noise

( s ,t )S xy g ( s, t )

Contraharmonic mean filter mn = size of moving window

 g ( s , t ) Q +1
Positive Q is suitable for
fˆ ( x, y ) =
( s ,t )S xy
eliminating pepper noise.
 g ( s , t ) Q
Negative Q is suitable for
( s ,t )S xy eliminating salt noise.
Q = the filter order For Q = 0, the filter reduces to an arithmetic mean filter.
For Q = -1, the filter reduces to a harmonic mean filter.
Contraharmonic Filters: Example
Image Image
corrupted corrupted
by pepper by salt
noise with noise with
prob. = 0.1 prob. = 0.1

Image Image
obtained obtained
using a 3x3 using a 3x3
contra- contra-
harmonic harmonic
mean filter mean filter
With Q = 1.5 With Q=-1.5
 ❖ Non-Linear Order statistics Spatial Filtering:
• Median filtering is a nonlinear method used to remove noise from images.
• It is widely used as it is very effective at removing noise while preserving edges.
• It is particularly effective at removing ‘salt and pepper’ type noise.
• The median filter works by moving through the image pixel by pixel, replacing
each value with the median value of neighboring pixels.

Input image Output image

R = median{Zk | k =1,2,3,…..9}
Filter mask
Order-Statistics Filters
Median filter

fˆ ( x, y ) = median g ( s, t )
( s ,t )S xy

Max filter

fˆ ( x, y ) = max g ( s, t ) Reduce “dark” noise

(pepper noise)
( s ,t )S xy

Min filter

fˆ ( x, y ) = min g ( s, t )
Reduce “bright” noise
(salt noise)
( s ,t )S xy

Midpoint filter

ˆf ( x, y ) = 1  max g ( s, t )+ min g ( s, t )

2  ( s ,t )S xy ( s ,t )S xy 
Median Filter : Example
1 2
Image
corrupted
by salt-and-
pepper
noise with
pa=pb= 0.1

3 4

Images obtained using a 3x3 median filter

Max and Min Filters: Example
Image Image
corrupted corrupted
by pepper by salt
noise with noise with
prob. = 0.1 prob. = 0.1

Image Image
obtained obtained
using a 3x3 using a 3x3
max filter min filter
 Alpha-trimmed Mean Filter
Formula:

1
fˆ ( x, y ) =
mn − d
 g ( s, t )
( s ,t )S xy
r

Alpha-trimmed mean filter removes the d/2 highest and d/2

lowest intensity values. The average of these remaining mn-d
values is called an alpha-trimmed mean filter.

This filter is useful in situations involving multiple types

of noise such as a combination of salt-and-pepper and
Gaussian noise.
Alpha-trimmed Mean Filter: Example
1 2
Image
additionally
Image corrupted
corrupted by additive
by additive salt-and-
uniform pepper
noise noise

Image 2 Image 2
obtained obtained
using a 5x5 using a 5x5
arithmetic geometric
mean filter mean filter

(Images from Rafael C. Gonzalez and Richard E.

Wood, Digital Image Processing, 2nd Edition.
Alpha-trimmed Mean Filter: Example (cont.)
1 2
Image
additionally
Image corrupted
corrupted by additive
by additive salt-and-
uniform pepper
noise noise

Image 2
obtained
Image 2 using a 5x5
obtained alpha-
using a 5x5 trimmed
median filter mean filter
with d = 5
Alpha-trimmed Mean Filter: Example (cont.)
Image Image
obtained obtained
using a 5x5 using a 5x5
arithmetic geometric
mean filter mean filter

Image
obtained
Image using a 5x5
obtained alpha-
using a 5x5 trimmed
median filter mean filter
with d = 5
 27

Estimating the Degradation function

in Digital Image Processing
© Dr. Dafda
 Degradation function Estimation

Estimation by Observation Estimation by Experimentation Estimation by modelling

Finding information from the Finding information from the Finding information from the
observed image. Instrument used for image constructed model
❖ Estimation by Observation:

• Identify the good portion of the image in which the signal content is strong.

• Let the observed sub-image be 𝑔𝑠(𝑥, 𝑦) and let the processed sub-image be
𝑓መ𝑠 𝑥, 𝑦 .

• Assuming that the noise is negligible,

Hs(u,v) = Gs(u,v)/ˆFs(u,v)

• Now we can apply inverse Fourier transform to the above equation and estimate
the degradation function.
 ❖ Estimation by Experimentation:

• Identify the instrument used for obtaining the image and study the device settings.
• Obtain the impulse response of the degradation by imaging an small white
dot(impulse) using the same system settings.
• But the Fourier transform of an impulse is constant. So,
H(u,v) = G(u,v)/A
• Now as the effect of noise on impulse is negligible, we can apply inverse Fourier
transform to the above equation and estimate the degradation function.
❖ Estimation by Modelling:
• Using the mathematical models, the degraded function like motion blur,
atmospheric turbulence etc. can be estimated.
• Now there are three possibilities:
1. Complete knowledge about the blur is available. Then we can apply Inverse
filtering(deconvolution).
2. Partial knowledge about the blur is available. Weiner filter can be used in this
case.
3. No knowledge about the blur is available. Blind deconvolution is applied in this
case.
 28
Inverse Filtering in Digital Image
Processing and its
implementation in MATLAB
© Dr. Dafda
❖ Estimation by Modelling:
• Using the mathematical models, the degraded function like motion blur,
atmospheric turbulence etc. can be estimated.
• Now there are three possibilities:
1. Complete knowledge about the blur is available. Then we can apply Inverse
filtering(deconvolution).
2. Partial knowledge about the blur is available. Weiner filter can be used in this
case.
3. No knowledge about the blur is available. Blind deconvolution is applied in this
case.
❖ Model of Image degradation and restoration process

where h(x,y) is a system that causes image degradation and ƞ(x,y) is noise.
 ❖ Inverse filtering:
• If we want to get rid of the blurring and noise, the simplest method is by applying
inverse filtering (deconvolution). Blurring is smoothing/LPF, so de-blurring is
sharpening/HPF and hence inverse filter acts as a HPF (also noise will increase).
• The input image gets convolved by a blurring function h(x,y) and changes to
g(x,y). So to retrieve f(x,y) from g(x,y), we take the inverse of h(x,y).
• Assuming that the noise is zero,
F(u,v) = G(u,v) / H(u,v)
Let HI(u,v) = 1 / H(u,v),
So, F(u,v) = HI(u,v) . G(u,v)
• Taking the inverse Fourier transform of the above equation,
f(x,y) = hI(x,y) * g(x,y)
• This shows that if we convolve the blurred image with the inverse of blurring
function, we get back the original image.
• But if the noise term is also present,

• If H(u,v) is zero, the second term N(u,v)/H(u,v) can have large values resulting in
amplification of noise as it tends to infinity.
• Therefore inverse filtering is not used in its original form and we have to go for
Wiener filter.
 29
Wiener Filter in Digital Image
Processing and its
implementation in MATLAB
© Dr. Dafda
❖ Model of Image degradation and restoration process

where h(x,y) is a system that causes image degradation and ƞ(x,y) is noise.
 ❖ Wiener Filter:
• Wiener filter uses the statistical properties of the image and can be
used to restore images in the presence of blur as well as noise.
• Wiener filter was proposed by N. Wiener in 1942. It is also called
Minimum Mean Square Error Filter or Least Square Error Filter.
• Here the goal is to find an estimate of the restored image such that the
mean square error between restored image and uncorrupted image is
minimized.
• This means it minimizes the overall mean square error in the process
of inverse filtering and noise smoothing.
• Wiener filter gives an linear estimation of the original uncorrupted
image.
❖ Wiener Filter: Minimum Mean Square Error Filter
Objective: optimize mean square error:
Wiener Filter Formula:

e2 = E ( f − fˆ )2 
 H *
( u , v ) S ( u , v ) 
Fˆ (u, v ) =  f
G ( u , v )
 S f (u, v ) H (u, v ) + S (u, v ) 
2

where
 H * ( u, v )  f = uncorrupted image
= G ( u , v )
 H (u, v ) + S (u, v ) / S f (u, v ) 
2
fˆ = estimate of f
e = error measure
 1 H ( u, v )
2

= G ( u , v ) E{·} = expected value
 H (u, v ) H (u, v ) + S (u, v ) / S f (u, v ) 
2
H(u,v) = Degradation function
H*(u,v) = complex conjugate of H(u,v)
‫׀‬H(u,v)‫׀‬2 = H*(u,v). H(u,v)
Sn(u,v) = ‫׀‬N(u,v)‫׀‬2 = Power spectrum of noise
Sf(u,v) = ‫׀‬F(u,v)‫׀‬2 Power spectrum of undegraded
image
❖ Approximation of Wiener Filter
Wiener Filter Formula:
 1 H ( u , v )
2

Fˆ (u, v ) =  G (u, v )
 H (u, v ) H (u, v ) + S (u, v ) / S f (u, v ) 
2

Difficult to estimate
Approximated Formula:

 1 H ( u , v )
2

Fˆ (u, v ) =  G ( u , v )
 H (u, v ) H (u, v ) + K 
2

Practically, K is chosen manually to obtained the best visual result!

If K=0, it will behave like Inverse filter.
❖ Problems with Wiener Filters:
1. Wiener filter is not optimal when no information on the power spectra is
available.
2. The performance of the Wiener filter depends on the correct estimation of the
value of K.
3. In minimizing the mean square error, the Wiener filter tends to smooth the
image more than what the human eyes would prefer.
4. Standard Wiener filters cannot handle rotational motion blur.
5. Wiener filters perform poorly if both, signal and noise are non-stationary.
 30

Constrained Least Squares Filtering

in Digital Image Processing and its
implementation in MATLAB
© Dr. Dafda
❖ Model of Image degradation and restoration process

where h(x,y) is a system that causes image degradation and ƞ(x,y) is noise.
❖ Constrained Least Squares Filtering (CLSF):
• Wiener filter is not optimal when no information on the power spectra is
available.
• For Wiener filters, the power spectra of the undegraded image(Sf) and
noise(Sn) must be known/assumed.
• CLSF does not make any assumption about original image, and only makes
use of mean and variance of the noise.
• The method to improve the Wiener filter is to impose a constraint on the
mean square error term. Hence CLSF is an advanced version of the Wiener
filter, where we do not need the knowledge of Sf and Sn; and some
constraint is added to the least square filtering.
• The CLSF algorithm yields optimal result for each image to which it is
applied.
Constrained Least Squares Filter
Degradation model: Written in a matrix form
g ( x, y ) = f ( x, y )  h ( x, y ) +  ( x, y ) g = Hf + η
Objective: to find the minimum of a criterion function

 
M −1 N −1
C =   f ( x, y ) 2 2

x =0 y =0
Subject to the constraint,
2 where w = wT w
2
g − Hfˆ = η
2

We get a constrained least square filter

 H *
( u , v ) 
Fˆ (u, v ) =  2
G (u, v )
 H (u, v ) +  P(u, v ) 
2

where  0 −1 0 
 
P (u,v) = Fourier transform of p(x,y) = − 1 4 − 1
 
 0 − 1 0 
Constrained Least Squares Filter: Example
Result of the Result of the
Image Constrained Wiener filter
degraded Least square
filter
by motion
blur +
2=650
AWGN

2=325

(Images from Rafael C.

Gonzalez and Richard E.
Wood, Digital Image
Processing, 2nd Edition.)

2=130
Constrained Least Squares Filter: Adjusting 
Define It can be shown that
 ( ) = r T r = r
2
r = g − Hfˆ
We want to adjust gamma so that
r = η a
2 2
1

where a = accuracy factor

1. Specify an initial value of 
2
2. Compute r

3. Stop if 1 is satisfied
Otherwise return step 2 after increasing  if r  η −a
2 2

or decreasing  if r  η +a
2 2

Use the new value of to recompute

 H *
( u , v ) 
Fˆ (u, v ) =  2
G (u, v )
 H (u, v ) +  P(u, v ) 
2
Constrained Least Squares Filter: Adjusting  (cont.)
 H *
( u, v ) 
ˆ
F ( u, v ) =  2
G (u, v )
 H (u, v ) +  P(u, v ) 
2

For computing 2
R(u, v) = G(u, v) − H (u, v ) Fˆ (u, v) r

M −1 N −1
1
r =  ( x, y )
2 2
r
MN x =0 y =0

M −1 N −1
1
m =
MN
 ( x, y )
x =0 y =0

  ( x, y ) − m 
M −1 N −1
1 For computing 2
 =
2 2
η
MN x =0 y =0


η = MN  2 − m
2

Constrained Least Squares Filter: Example
Use correct noise
Original image Correct parameters:
parameters
Initial  = 10-5
Correction factor = 10-6
a = 0.25
2 = 10-5

Blurred image Use wrong noise

Wrong noise parameter
Due to Turbulence parameters 2 = 10-2

(Images from Rafael C.

Gonzalez and Richard E.
Wood, Digital Image
Processing, 2nd Edition.

Results obtained from constrained least square filters

 ❖ CLSF vs MMSE:

• Wiener(MMSE) is optimal in the average sense while CLSF yields optimal result
to each image to which it is applied.

• When noise is low, both MMSE and CLSF generate equal results, but when
noise is high CLSF is advantageous if proper  is selected. Because it is easier to
select scalar value of  rather than to approximate the K which is never constant.