Lecture 7

Unit II: Image Preprocessing

Pixel brightness transformations and Gray scale
transformations

Prof. Mohammed Usman

Image preprocessing
• Performed on the intensity images
• Input and output of preprocessing are brightness images
• Preprocessing decreases entropy
• Objective of preprocessing
• To suppress irrelevant information and improve desired (or more relevant)
information useful for further processing
• Four categories of image preprocessing
• Pixel brightness transformations
• Geometric transformations
• Methods based on local neighborhood
• Methods that require knowledge of entire image (global neighborhood)
• Images have considerable redundant information
• Preprocessing yields statistical understanding of image information
Pixel brightness transformations
• Modifies pixel brightness
• Two types:
• Brightness corrections
• Gray scale transformations
• Brightness correction
• Based on original pixel brightness and position of pixel in the image
• Gray scale transformation
• Modifies brightness without regard to pixel position
Position dependent brightness correction
• Pixel brightness is affected due to non uniform sensitivity of image
sensor – causes image degradation
• Degradation also caused by uneven illumination
• Degradation represented by a multiplicative error coefficient e(i, j)

• g(i, j) is original undegraded image, f(i, j) is the degraded version

• To perform brightness correction, use a reference image of constant
brightness ‘c’ – the degraded version of this constant brightness
image is fc(i, j)
Position dependent brightness correction
• The original undegraded image can be obtained as follows:
Constant brightness image x degraded image of
interest
Degraded version of constant brightness image

• Constant brightness reference image should have brightness level

that is far from the lower and upper limits of brightness of the test
image
• For eg. if the test image has 256 brightness levels, the reference
(ideal) image should have a brightness level of 128
Gray scale transformation
• Does not depend on pixel position
• Transforms brightness p having scale [p0, pk]
to brightness q having scale [q0, qk]: 𝑞 =
𝑇(𝑝) where 𝑇 denotes the transformation
• 3 types of gray scale transformations
• Contrast enhancement: brightness values
between p1 and p2 are enhanced in
accordance with a piecewise linear function –
improves contrast
• Brightness Thresholding: pixels below a
certain brightness threshold are made black
and those above the threshold are made
white– results in a black and white image
• Negative transformation: brightness levels
are flipped – low brightness is made high and
vice versa
Gray scale transformation – Linear
transformation
• Two types of linear transformation:

• where ’s’ is the output pixel value and ‘r’ is the input pixel value
Gray scale transformation – Piecewise Linear
transformation
• Two types of piecewise linear
transformation:
• Contrast stretching: brightness values
between p1 = (r1, s1)and p2 = (r2, s2) are
enhanced in accordance with a piecewise
linear function

• Gray level slicing: pixel values between a

range [A, B] are highlighted (enhanced);
other pixel values are either darkened (left
diagram) or preserved (right diagram)
Logarithmic gray scale transformation
• Simulates logarithmic sensitivity of
human eye to light intensity
• 𝑠 = 𝑐 log(𝑟 + 1)
• where ’s’ is the output pixel value and
‘r’ is the input pixel value
• 1 is added to ‘r’ to avoid log(0)
• ‘c’ is a scaling factor, that adjusts the
enhancement
• Darker (lower) pixels are enhanced
more compared to brighter (higher)
pixel values
• Inverse log transform is the opposite
of log transform
Histogram equalization for contrast
enhancement
• Objective is to create uniformly distributed brightness levels across the
entire brightness scale

• Enhances contrast for brightness values close to histogram maxima and

decreases contrast for brightness values close to histogram minima
 Lecture 8
Unit II: Image Preprocessing
Histogram Equalization and Geometric Transformations
Histogram equalization for contrast
enhancement
• Objective is to create uniformly distributed brightness levels across the
entire brightness scale

• Enhances contrast for brightness values close to histogram maxima and

decreases contrast for brightness values close to histogram minima
Histogram equalization for contrast
enhancement
• Note: the equalized histogram is
not equalized ideally – this is due to
discretization
• Input histogram H(p) with
brightness range [p0, pk]
• Transformation 𝑞 = 𝑇(𝑝) such that
the desired output histogram
𝐺 𝑞 is uniformly distributed over
the entire output brightness range
[q0, qk]
Histogram equalization for contrast
enhancement
• The transformation 𝑇 is monotonic, i.e.

• For an N x N image, G(q) corresponds to uniform probability density

function f defined as

• The idealized continuous probability density is

• The transform 𝑇 is
Histogram equalization for contrast
enhancement
• The integral in the pixel brightness transform equation is called the
cumulative histogram

• In digital images, the integral is approximated by a sum – so the

resulting histogram is not equalized ideally.
• The discrete approximation of the pixel brightness transformation for
histogram equalization is
!! "!"
𝑞=Τ 𝑝 = ∑&$%&" 𝐻(𝑖) + 𝑞'
##
Geometric transformations
• Geometric transform maps a pixel coordinate 𝑥, 𝑦 to a new position
𝑥′, 𝑦′
𝑥 ( = 𝑇) (𝑥, 𝑦) and 𝑦 ( = 𝑇* (𝑥, 𝑦)
• 𝑇) and 𝑇* are transformation functions which may be known in
advance for operations such as rotation, translation, scaling – or can
be determined from known original and transformed images
• Several pixels in both images with known correspondences are used
to derive the unknown transformation.
• Geometric transformations performed in two steps:
• Pixel co-ordinate transformation
• Brightness interpolation (from several points in the neighborhood)
Geometric transformations - Rotation
• Rotation by an angle ∅ about the origin
𝑥 ( = 𝑥 cos ∅ + 𝑦 sin ∅
𝑦 ( = −𝑥 sin ∅ + 𝑦 cos ∅
• In matrix form, the rotation transformation is
𝑥′ cos ∅ sin ∅ 𝑥
=
𝑦′ − sin ∅ cos ∅ 𝑦

• Note: Determinant of the transformation matrix is |J| = 1

• |J| is called Jacobian determinant. If |J| = 1, the area of the image is
invariant under the transformation
• If |J| = 0, the transformation has no inverse (singular)
Geometric transformations - Scaling
• Change of scale a in the x-axis and b in the y-axis
𝑥′ = 𝑎𝑥
𝑦′ = 𝑏𝑦
• In matrix form, the scaling transformation is

𝑥′ 𝑎 0 𝑥
=
𝑦′ 0 𝑏 𝑦

• Determinant of the transformation matrix is |J| = 𝑎𝑏

Geometric transformations - Skewing
• Skewing by the angle ∅
𝑥 ( = 𝑥 + 𝑦 tan ∅
𝑦( = 𝑦
• In matrix form, the skewing transformation is

𝑥′ 1 tan ∅ 𝑥
= 𝑦
𝑦′ 0 1
• Determinant of the transformation matrix is |J| = 1
Geometric transformations – Bilinear
transform
• Bilinear transform – transforms input coordinates 𝑥, 𝑦 to output
coordinates 𝑥′, 𝑦′ as follows:

• 𝑎' , 𝑎+ , 𝑎, , 𝑎- are the transformation coefficients

• To compute the transformation coefficients in bilinear transform, 4
pairs of corresponding points 𝑥, 𝑦 , 𝑥′, 𝑦′ from the original and
transformed images are sufficient
Geometric transformations –Affine
transformation
• Affine transformation – transforms input coordinates 𝑥, 𝑦 to output
coordinates 𝑥′, 𝑦′ as follows:

• 𝑎! , 𝑎" , 𝑎# , are the transformation coefficients

• Affine transformation produces a linear combination of translation,
rotation, scaling and skewing
• To compute the transformation coefficients in affine transform, 3 pairs of
corresponding points 𝑥, 𝑦 , 𝑥′, 𝑦′ from the original and transformed
images are sufficient
Brightness interpolation
• Pixel coordinate transformation transforms pixel points 𝑥, 𝑦 to
𝑥′, 𝑦′
• 𝑥′, 𝑦′ may not lie on the integer grid as required
• Due to this, inverse transform from 𝑥′, 𝑦′ to 𝑥, 𝑦 does not fit the
discrete raster (solid lines)
𝑥, 𝑦 of original image

𝑥, 𝑦 obtained from inverse

transform of 𝑥′, 𝑦′

• How to assign brightness level for this new 𝑥, 𝑦 position?

 Lecture 9
Brightness interpolation, Image Smoothing - Averaging

Prof. Mohammed Usman

Brightness interpolation
• Pixel coordinate transformation transforms pixel points 𝑥, 𝑦 to
𝑥′, 𝑦′
• 𝑥′, 𝑦′ may not lie on the integer grid as required
• Due to this, inverse transform from 𝑥′, 𝑦′ to 𝑥, 𝑦 does not give the
original 𝑥, 𝑦
𝑥, 𝑦 of original image

𝑥, 𝑦 obtained from inverse

transform of 𝑥′, 𝑦′

• How to assign brightness level for this new 𝑥, 𝑦 position?

Brightness interpolation
• How to assign brightness level for this new 𝑥, 𝑦 position?
• The only information available about the original continuous image
function 𝑓(𝑥, 𝑦) is its sampled version 𝑔𝑠(𝑙∆𝑥, 𝑘∆𝑦)
• The result of brightness interpolation in the transformed image is, in
general, denoted by the convolution equation

• where 𝒏 distinguishes the different interpolation methods

• ℎ𝑛 is called the interpolation kernel - defined only for a small
neighborhood outside which ℎ𝑛 = 0
• For simplicity, ∆𝑥 = ∆𝑦 = 1 is adopted
Brightness interpolation
• How to assign brightness level for this new 𝑥, 𝑦 position?
• 3 methods commonly used to assign brightness level

• Nearest neighbour interpolation

• Linear interpolation – also called bilinear interpolation as image is 2D

• Bicubic interpolation
Brightness interpolation – Nearest neighbor
interpolation
• Assigns to the point 𝑥, 𝑦 the brightness value of the
nearest point 𝑔 in the discrete raster
• Solid lines show raster of original input image
• Dashed lines represent the raster of input image obtained
from inverse transformation of output image to input image

• Position error of NNI is at most half a pixel

• This error is perceptible on objects with straight line
boundaries that may appear step-like after the
transformation
Interpolation kernel for NNI
Brightness interpolation – Linear interpolation
• Brightness value is assigned based on 4 points 𝑙, 𝑘 + 1 𝑙 + 1, 𝑘 + 1
neighboring the point 𝑥, 𝑦
• Assumes linearity of brightness function in the
neighborhood 𝑙, 𝑘 𝑙 + 1, 𝑘

• Step-like straight boundaries as in NNI are reduced

• Small loss of resolution and blurring can occur in
linear interpolation due to its averaging nature

Interpolation kernel for linear interpolation

Brightness interpolation – Bicubic interpolation
• Brightness value is assigned based on 16 points neighboring the point
𝑥, 𝑦
• Step-like stright boundries as in NNI are avoided
• Blurring as in linear interpolation is also reduced with bicubic
interpolation
• Fine details of image are preserved very well with bicubic
interpolation

Interpolation kernel for bicubic interpolation

Local preprocessing
• Local preprocessing methods are based on local neighborhood of a
pixel – these methods are also called filtration
• Based on goal of preprocessing, there are 2 categories of local
preprocessing
• Smoothing – aims to suppress noise or small fluctuations in the image
• Noise is high frequency content – smoothing suppresses high frequencies
• Side effect of smoothing – causes blurring of sharp edges
• Gradient operators – based on local derivatives of image function
• Gradient is high where the image changes rapidly
• Gradient operators indicate locations of high gradient in the image
• Gradient operators suppress low frequencies
• Suppression of low frequencies has the effect of increasing noise level
• Smoothing and gradient operators have conflicting aims
 Local preprocessing
• Another classification of local preprocessing is based on transformation
properties – linear and non linear transformations
• Linear – output image pixel f(i, j) is a linear combination of pixels in a
local neighborhood ‘O’ of input image pixel g(i, j)
‘h’ denotes the weight coefficients

• Rectangular neighborhoods ‘O’ with odd number of rows/columns are

used to enable specification of central pixel of the neighborhood
Image smoothing
• Image smoothing suppresses noise – high frequency components
• New pixel value is average of pixels in some neighborhood ‘O’
• Edge preserving methods are used to avoid blurring of sharp edges
• In edge preserving methods, average is computed only from those
points in the neighborhood which have similar properties to the
processed point
• Image smoothing can effectively eliminate impulsive noise or
degradations appearing as thin stripes
• Smoothing not effective for large blobs or thick stripes
 Image smoothing - averaging
• Assumes each pixel is corrupted by zero mean noise ‘v’
• Multiple images of same scene are captured
g1 g2 gn
v1 v2 Multiple input images vn

Output image
Each pixel is average of
corresponding input image
Smoothing without pixels
Equals zero since
blurring can be achieved noise is zero mean
Image smoothing - averaging
• In most situation multiple images of the same scene are NOT
available – only one noise corrupted image is available
• In such cases, averaging is performed based on local neighborhood
• For a 3 x 3 neighborhood, the averaging mask ‘h’ is

• Weightage of central pixel or 4-neighborhood pixels can be increased

for better approximation of noise properties
Image smoothing - averaging
Image smoothing – averaging with limited
data validity
• Variant implementations of averaging
• Define a brightness interval of invalid data [min,max], i.e. invalid
brightness levels
• Only values of pixels with invalid gray-levels are replaced with an
average of their neighborhoods, and only valid data contribute to the
averages

Neighborhood pixels contribute

to averaging only if they are not
elements of the invalid range
Image smoothing – averaging with limited
data validity
 Lecture 10
Image Smoothing – Averaging – inverse gradient, rotating mask, median
filtering, Edge detectors

Prof. Mohammed Usman

Averaging according to inverse gradient
• Inverse gradient 𝛿 of a point (i, j) with respect to the central pixel (m,n)
is computed as

• If g(m,n) = g(i,j), then we define δ(i,j) = 2, so δ is in the interval (0,2]

• The averaging mask h(i,j) is defined as

(i,j) Shaded region is the neighborhood ‘O’

(m,n)

Averaging mask coefficient for central

pixel is h(i,j) = 0.5
Averaging using rotating mask – non linear
averaging method
• Avoids edge blurring by searching for the homogeneous part of the
current pixel neighborhood and the resulting image is sharpened
• The brightness average is calculated only within the homogeneous
region
• A brightness dispersion 𝜎2 is used as the region homogeneity measure
‘n’ is the number of pixels in a region ‘R’
of input image ‘g’

• 8 possible 3 x 3 rotation masks of a 5 x 5 neighborhood of a current pixel

are Choose the mask with smallest 𝜎2 value

Assign to the output pixel f(i,j) the

average brightness in the chosen mask
Median filtering – non linear smoothing
method
• Replace the current pixel brightness g(i,j) by the median of the brightness
in its neighborhood
• Recall: Median ‘M’ is the central member of a sorted list
• Advantages of median filtering
• Reduces impulse noise
• Avoids blurring of edges
• Computational complexity can be high due to repetative sorting of pixels in
a rectangular window
• Efficient algorithm for sorting can be implemented noting that as the
window slides, one row (column) is removed and a new row (column) gets
included
Median filtering

Impulse noise is
significantly reduced
with median filtering
Edge detectors
• Collection of local preprocessing methods used
to locate changes in intensity function – edges
are pixels where brightness changes rapidly
• Edges are important for image perception
• Edge is described as the gradient of the image
function g(x,y)
• Edge is a vector property – has magnitude and
direction
• Edge magnitude is the magnitude of the
gradient
• Edge direction is the gradient direction rotated
by -90o i.e. edge direction is perpendicular to
gradient direction
Edge detectors – gradient magnitude and
direction

• Partial derivatives are used for gradient since image function g(x,y)
depends on two variables
• When only the magnitude of the gradient is required (and not the
direction), a linear differential operator called Laplacian is used

• Laplacian is invariant to rotation

 Edge detectors – Gradient operators
• Gradient (derivatives) are approximated using differences
• Edge detection is achieved by applying gradient operators in a small
local neighborhood
• Different gradient operators are used for edge detection – expressed
as convolution masks
• Roberts operator
• Uses 2 x 2 neighborhood of current pixel
• Edge magnitude is computed as

Roberts operator is sensitive to noise as very few pixels are used to approximate the gradient
Edge detectors – Gradient operators
• Laplace operator ∇2 – approximates the second derivative and
computes the edge magnitude only
• 3 x 3 convolution mask is often used

4-neighborhood mask 8-neighborhood mask

• Operators invariant to rotation use a single convolution mask – others

use several masks
Edge detectors – Gradient operators
• Prewitt operator
• Approximates the first derivative
• Gradient is estimated in 8 possible directions
• 3 x 3 convolution masks for first 3 directions are shown below

• Other masks can be obtained by simple rotation

• Mask that gives the greatest magnitude for gradient indicates the
direction of gradient
Edge detectors – Gradient operators
• Sobel operator – similar to Prewitt operator
• Computes the first derivative
• Convolution mask assigns some weight to certain pixels

• Sobel operator usually used to detect horizontal and vertical edges –

masks h1 and h3 are used in this case
• Other masks can be obtained by simple rotation
Edge detectors – Gradient operators
• Kirsch operator – similar to Prewitt and Sobel operators
• Computes the first derivative
• Convolution mask assigns some weight to certain pixels

• Other masks can be obtained by simple rotation

Edge detection based on zero crossings of the
second derivative – Marr Hildreth edge detection
• A step edge corresponds to abrupt change in image function
• First derivative will have an extremum at the position of the edge –
second derivative will therefore be zero at the same position

Position of edge
in image function

Zero crossing of
second derivative

Extremum of first
derivative

1-D edge profile of zero crossing

Zero crossings of the second derivative
• Easier and more precise to find a zero crossing position than an
extremum
• Position of the zero crossing of the second derivative of image
function corresponds to edge position
• How to compute the second derivative robustly?
• Step 1: Smooth the image using a smoothing filter
• Step 2: Compute second derivative using Lapalace operator ∇2
Zero crossings of the second derivative
• Step 1: Smooth the image using a smoothing filter – Gaussian filter is
used
• 2-D Gaussian operator is
• 𝜎 is the only parameter of the Gaussian filter
• 𝜎 is proportional to the size of the neighborhood on which the filter operates
• Pixels further from the center of operator have smaller influence
• Pixels farther than 3𝜎 from the center have negligible influence
• Apply Gaussian operator with specific 𝜎 to the image function f(x,y)
to obtain the smoothed image
• * denotes convolution operation
Zero crossings of the second derivative
• Step 2: Compute second derivative using Lapalace operator ∇2
• Apply the Laplace operator ∇2 to the Gaussian smoothed image

• We get the Laplacian of image f(x,y) smoothed by the Gaussian G(x,y,𝜎)

• i.e. à Laplacian of Gaussian (LoG)
• Since differentiation and convolution are linear operators, order can be
interchanged =
• Derivative of Gaussian filter ∇2G can be precomputed since it is
independent of the image under consideration
Zero crossings of the second derivative
• Laplacian operator ∇2 is defined as
• For the Gaussian filter
• and

• Similarly, 2nd derivative w.r.t 𝑦 is

• = +

• Therefore,
Zero crossings of the second derivative
• A normalizing multiplicative coefficient ‘c’ is introduced in the
expression for ∇2G to get the convolution mask of the LoG operator

• The 2nd derivative of smoothed image function 𝑓(𝑥, 𝑦) can be

obtained by convolution of 𝑓(𝑥, 𝑦) with the convolution mask ℎ(𝑥, 𝑦)
• = ℎ 𝑥, 𝑦 ∗ 𝑓(𝑥, 𝑦)
• After image convolution with ∇2G, the locations in the convolved
image where the zero level is crossed correspond to the positions of
edges
• Advantage: Larger area around the current pixel is taken into account
to compute the edges
 Lecture 11
Scale space image, Canny edge detection, Parametric edge models, Edges in
multispectral images, local preprocessing in frequency domain, homomorphic filtering

Prof. Mohammed Usman

Scale in image processing
• Image processing techniques work locally at the Noisy curve

pixel level and its neighborhood

• What is the right size of the neighborhood?
• Neighborhood size depends on the object under
investigation
• Knowing the object size requires proper image Processing criteria for scale
interpretation – not known at the preprocessing
• The longer the curve segment,
stage the better
• Image is processed at a range of scales to find • Change of curvature should be
minimal
the structure of the scene
• For example, Gaussian smoothing applied with
a range of 𝝈 values
Scale space image
• Signal 𝑓 𝑥 is smoothed by convolution with a Gaussian filter

• The smoothed function for different values of 𝜎 is

• 𝐹 𝑥, 𝜎 represents a surface on the 𝑥, 𝜎 plane and is called the
scale-space image
• Qualitative description of the signal 𝑓 𝑥 is obtained from the
inflection points of the curve 𝐹 𝑥, 𝜎0 for a particular value 𝜎0
• Inflection point – location in 𝐹 𝑥, 𝜎0 where 2nd derivative is zero and
3rd derivative not equal to zero
Canny edge detection
• It is an optimal edge detection method for step edges corrupted with
white noise
• Edge detection must satisfy 3 criteria
• Detection criterion: important edges should not be missed and there should
be no spurious edges
• Localization criterion: distance between actual and located position of edge
should be minimal
• One response criterion: same edge must not be detected multiple times -
partly covered by first criterion
• This method has strong similarity to LoG based Marr-Hildreth edge
detection
Canny edge detection
• Step 1: Smooth the image using a Gaussian filter
• If the original image is 𝑓 the smoothed image is G ∗ 𝑓.
• Step 2: Estimate the gradient of the smoothed image – Robert,
Prewitt or Sobel operators can be used for this purpose
• Sobel operator is commonly used to estimate the x-direction (h3 mask) and y-
direction (h1 mask) gradients

Gy Gx

$% &!
• Magnitude of the gradient is 𝐺!" + 𝐺#" and direction 𝑛 = 𝑡𝑎𝑛 &"
Canny edge detection
$% &!
• Estimating the gradient direction 𝑛 = 𝑡𝑎𝑛 is analytically
&"
equivalent to where ∇ operator denotes first
derivative

• Step 3: Non-maximum suppression - Once the gradient direction 𝑛 is

estimated, compute the first derivative Gn of the Gaussian filter G, in
the direction 𝑛
(eq. 1)
• Convolve 𝐺𝑛 with the image function 𝑓, i.e. compute Gn ∗ 𝑓
• Edge location is the local maxima of Gn ∗ 𝑓, i.e. the location at which
• (eq. 2)
Canny edge detection
• Substituting eq.1 in eq. 2, we get, which denotes the
local maxima in the gradient direction – which corresponds to the
zero crossing of the second derivative
• Only the local maxima in the gradient image are considered – this operation is
called non-maximum suppression
• NOTICE the similarity with Marr-Hildreth edge detector – the
difference is that in ’Canny edge detection’ the 1st and 2nd
derivatives are computed w.r.t the gradient direction ‘n’. Secondly, the
Laplacian operator in Marr-Hildreth edge detector does not give
information about edge direction
 Canny edge detection
• Step 4: Thresholding – After the non-maximum suppression, the edge
pixels are still marked with their gradient strength, pixel-by-pixel
• Thresholding is applied to declare each of these pixels as ‘edge pixel’
or not Double thresholding using
High and Low thresholds.
Thresholds are chosen
based on estimated SNR.
Gradient magnitude

If gradient at a pixel is
• Above High, declare it as edge pixel
• Below Low, declare it as non-edge pixel
• Between Low and High, consider its
neighbors and declare it as ’edge pixel’
if it is connected to another edge pixel
directly or via other pixels between low
and high
Parametric edge models
• Discrete image intensity function 𝑔𝑠(𝑙∆𝑥, 𝑘∆𝑦) is a sampled and noisy
approximation of the continuous image intensity function 𝑓(𝑥, 𝑦)
• 𝑓(𝑥, 𝑦) is not known but can be estimated from 𝑔𝑠(𝑙∆𝑥, 𝑘∆𝑦)
• Modeling the discrete image as a continuous function 𝑓(𝑥, 𝑦)
requires higher order functions in 𝑥 𝑎𝑛𝑑 𝑦 – hence practically
impossible to represent the discrete image function using a single
continuous function
• Solution: piecewise continuous functions called facets are used to
represent a pixel and its neighborhood – such a representation is
called facet model
Parametric edge models
• Applications of facet model
• Peak noise removal Edge detectors based on
• Segmentation into constant gray-level regions parametric models are more
precise than convolution
• Gradient edge detection and zero-crossing edge detection based edge detectors
• Line detection and corner detection
• Types of facet models:
• Flat facet model – uses piecewise constant functions
• Linear (sloped) facet model – uses piecewise linear functions
• Quadratic facet model – uses piecewise quadratic functions (of 2 variables: x
and y)
• Cubic facet model – uses piecewise cubic functions (of 2 variables: x and y)
• Example of a cubic facet model: Coefficients c1, c2, … can be
estimated using least
squares method or singular
value decomposition (SVD)
Edges in multispectral images

Spectral band 1 Spectral band 2 Spectral band 3 Spectral band n

1 2 3 n Each pixel in multispectral image is

Pixel 1
described by an n-dimensional vector

• Recall: Multispectral image is a collection of ‘n’ images each representing a

‘spectral band
• Edges may be detected separately in each spectral band image and the
results can be combined
• Multispectral edge detection can be used – uses brightness information
from all ‘n’ spectral bands for each pixel location - more complex
Local preprocessing in frequency domain
• An input image 𝑓 is filtered by a filter having impulse response ℎ
• The output of the filter is 𝑔 = 𝑓 ∗ ℎ, where ∗ denotes convolution
• Performing convolution operation is easier in the frequency domain
• Convolution of 2 functions is equivalent to the product of their Fourier Transforms
• Let 𝐹 denote the Fourier transform of 𝑓 and 𝐻 denote the Fourier
transform of ℎ, then 𝐺 = 𝐹.×𝐻, where .× denotes element-by-element
mutiplication
• 𝐻 is called the frequency response or frequency transfer function of the filter
• The filtered image 𝑔 can be obtained by inverse Fourier transform of 𝐺
• Performing the image operations (filtering) in frequency domain is called
spatial filtering
Local preprocessing in frequency domain –
low pass filtering
• Examples of spatial filtering – low-pass, high-pass and band-pass
filters
• Low pass filtering – preserves low spatial frequencies and suppresses
high spatial frequencies
• Noise is high frequency component in image – low pass filtering is
used to suppress noise
• Sharp edges and lines also constitute high frequency components –
these also get suppressed due to low pass filtering
• Low pass filtering causes blurring of image
Local preprocessing in frequency domain –
high pass and band pass filtering
• High pass filtering - preserves high spatial frequencies and
suppresses low spatial frequencies
• High pass filter is useful to enhance edges and lines by suppressing
the low spatial frequencies
• Band pass filtering – preserves a certain band of frequencies and
suppresses all other frequencies that lie outside the band
H(f) H(f) H(f) Band pass filter
Low pass filter High pass filter

fco f fco f fCL fCU f

Illustration of low-pass, high-pass and band-
pass filtering

Low pass filtered image High pass filtered image Band pass filtered image
 Local preprocessing in frequency domain –
Homomorphic filtering
• Homomorphic filtering is used to remove multiplicative noise
• It improves image contrast and normalizes image intensity across the
image
• Homomorphic filtering procedure:
Factorize image function Apply logarithmic transform Apply Fourier transform
𝑓 𝑥, 𝑦 = 𝑖 𝑥, 𝑦 . 𝑟(𝑥, 𝑦) 𝑧 𝑥, 𝑦 = log 𝑓(𝑥, 𝑦) = log 𝑖(𝑥, 𝑦) + log 𝑟(𝑥, 𝑦) 𝑍 𝑢, 𝑣 = 𝐼 𝑢, 𝑣 + 𝑅(𝑢, 𝑣)

Inverse logarithmic transform Inverse Fourier transform Apply filter H 𝑢, 𝑣

𝑔 𝑥, 𝑦 = 𝑒𝑥𝑝 𝑠(𝑥, 𝑦) 𝑠 𝑥, 𝑦 = 𝐹 !" 𝑆 𝑢, 𝑣 S = H. x 𝑍 = 𝐻. x 𝐼 + 𝐻. x 𝑅

𝑔 𝑥, 𝑦 is the homomorphic filtered version of 𝑓 𝑥, 𝑦

Homomorphic filtering illustration

• Details of tunnel surface at the top and right are visible after
homomorphic filtering
 Lecture 12
Line detection, Image restoration – Inverse filtering, Weiner filtering

Prof. Mohammed Usman

Line detection
• Line finding operators are used to find very thin curves in an image
• Curves that do not bend sharply or are straight lines
• Used for detecting railways, roads in satellite images, remote sensing, and
document processing
• Convolve the image 𝑓 with convolution kernels (masks) which serve as line patterns
• Examples of 3x3 convolution kernels (masks) used for line detection are:
Horizontal -45o Vertical +45o

ℎ4 = ?

• These kernels respond to lines of single pixel width at a particular orientation

• These masks are tuned to detect light lines against dark background
• 5x5 kernels are also commonly used
Line detec*on – 5x5 convolu*on masks
• Examples of 5x5 convolution masks used for line finding

Procedure to find line pixel and its orientation

Convolve the image 𝒇 with each of the masks

ℎ1, ℎ2, …

Let the response of each convolution be 𝑅1, 𝑅2, …

For any particular point 𝑥, 𝑦 in the image, if 𝑅𝑖 >

𝑅𝑗 for all 𝑗 ≠ 𝑖, that point is likely to contain a line
whose orientation and width is the same as the
mask ℎ𝑖
Image restoration
• What is image restoration?
• Suppression of image degradation using knowledge of the degradation is
called image restoration
• Common causes of image degradation are:
• Optical lens defects, relative motion between object and camera, wrong
focus, atmospheric turbulence etc
Deterministic DegradaGon funcGon is Apply inverse of the degradation
techniques known and very liHle noise function to obtain the original image
Image
restoration
techniques
Stochastic Degradation function is Restoration is done based on
techniques NOT known statistical criterion–e.g. least squares
Inverse filtering for image restoration
• Assumes image is degraded by a linear function ℎ(𝑖, 𝑗) along with
additive noise 𝑛(𝑖, 𝑗)
• Degraded image 𝑔 𝑖, 𝑗 = [𝑓 𝑖, 𝑗 ∗ ℎ 𝑖, 𝑗 ] + 𝑛(𝑖, 𝑗)
• Applying Fourier Transform we get
𝐺 𝑢, 𝑣 = [𝐹 𝑢, 𝑣 . x 𝐻 𝑢, 𝑣 ] + 𝑁(𝑢, 𝑣)
• Degradation can be removed by using a restoration filter with
transfer function that is inverse to the degradation filter 𝐻 𝑢, 𝑣
• Original image 𝐹 𝑢, 𝑣 can be obtained from the degraded image
𝐺 𝑢, 𝑣 by using the restoration filter 𝐻!" 𝑢, 𝑣 as follows

• Original image 𝑓 𝑖, 𝑗 can be obtained by inverse Fourier transform of

𝐹 𝑢, 𝑣
Inverse filtering for image restoration

• Inverse filtering works well for images not corrupted by noise

• If noise is present, 2 problems arise:

• Effect of noise becomes more at high frequencies – noise dominates the
entire restoration result
• Not enough information about noise is known to determine 𝑁(𝑢, 𝑣)
sufficiently
Image restora*on – Weiner filtering
• Principle: Weiner filter gives an estimate 𝑓! of the original uncorrupted image
𝑓
• Optimal 𝑓! is in general, a non-linear function of the degraded image 𝑔
• The mean square error between the original and estimated images should be
minimized
!
!
𝑒 = 𝐸 𝑓 𝑖, 𝑗 − 𝑓(𝑖, 𝑗)
2

• Minimizing 𝑒2 is easy if 𝑓! is a linear combination of the values in the image 𝑔

• 𝑓! is equal to the theoretical optimum 𝑓 only if the stochastic processes
describing 𝑓, 𝑔 and noise 𝑛 are homogeneous and have Gaussian probability
density
• These conditions are not usually satisfied for typical images
Image restoration – Weiner filtering
• Weiner filtering can be achieved in the Fourier transform domain as
follows:
• 𝐻𝑤 𝑢, 𝑣 is the Fourier transform of the Weiner filter ℎ𝑤(𝑖, 𝑗),
• 𝐺(𝑢, 𝑣) is the Fourier transform of the degraded image 𝑔(𝑖, 𝑗)
7 𝑣) is the Fourier transform of the estimated image 𝑓(𝑖,
• 𝐹(𝑢, 8 𝑗)
8 𝑗) can be obtained by computing the inverse Fourier transform of
• 𝑓(𝑖,
7 𝑣)
𝐹(𝑢, • H is the transform function of
the degradation
• The transform function of Weiner filter is given as • H* is the complex conjugate of H
• Svv is the spectral density of
noise
• Sff is the spectral density of the
undegraded image
Image restoration – Weiner filtering
• Limitation of Weiner filtering
• Spatially variant degradations cannot be restored using Weiner filtering
• Most images are non-stationary (contain large homogeneous regions
separated by high contrast edges) - Weiner filtering cannot handle non
stationary signals and noise
• More advanced methods like power spectrum equalization and
geometric mean filtering are used in such cases