UNIT I LINEAR FILTERS Lecture 8Hrs

Introduction to Computer Vision, Linear Filters and Convolution, Shift

Invariant Linear Systems, Spatial Frequency and Fourier Transforms,
Sampling and Aliasing Filters as Templates, Technique: Normalized
Correlation and Finding Patterns, Technique: Scale and Image
Pyramids.

1.1 Introduction to Computer Vision

Computer vision is a multidisciplinary field that focuses on enabling

computers to interpret and understand visual information from images or
videos, much like human vision. It combines techniques from computer
science, machine learning, image processing, and artificial intelligence to
extract meaningful information from visual data.

The goal of computer vision is to develop algorithms and systems that can
analyze and interpret images or videos to perform tasks such as object
recognition, image classification, image segmentation, object tracking, and
scene reconstruction. These capabilities have various real-world
applications, including autonomous vehicles, facial recognition systems,
medical imaging, surveillance systems, augmented reality, robotics, and
much more.

Computer vision algorithms typically follow a pipeline of processes to

extract useful information from images. These processes may include:

1. Image Acquisition: Obtaining digital images or videos using cameras or

other sensors.

2. Preprocessing: Applying techniques to enhance the quality of images,

such as noise reduction, resizing, and color normalization.
3. Feature Extraction: Identifying and representing relevant visual features,
such as edges, corners, textures, or color histograms, that can be used to
characterize objects or regions of interest.
4. Image Representation: Converting images into mathematical
representations, such as matrices or feature vectors, that can be processed
by machine learning algorithms.

5. Machine Learning: Utilizing machine learning techniques, such as deep

learning, to train models that can recognize patterns or objects in images.
These models can learn from labeled examples and make predictions on
unseen data.

6. Object Detection and Recognition: Identifying and localizing specific

objects or classes of objects within images or videos.

7. Image Segmentation: Dividing an image into meaningful regions or

segments to understand the spatial distribution of objects or regions of
interest.

8. Tracking and Motion Analysis: Following objects or regions of interest

over time to analyze their motion patterns and interactions.

9. Scene Understanding and Interpretation: Extracting higher-level

semantic information from visual data, such as understanding the overall
context or scene category.

Advancements in deep learning, particularly with convolutional neural

networks (CNNs), have significantly improved the field of computer vision in
recent years. CNNs are capable of automatically learning and recognizing
complex patterns in images, making them highly effective in various
computer vision tasks.
 Computer vision has enormous potential to revolutionize industries and
impact our daily lives, enabling machines to perceive and understand the
visual world like humans do. As the field continues to evolve, new
algorithms and technologies are being developed, leading to exciting
advancements and applications in areas such as healthcare, robotics,
entertainment, and more.

1.2 Linear Filters and Convolution

Linear filters and convolution are fundamental concepts in computer vision
used for image processing and feature extraction. They play a crucial role
in tasks such as edge detection, blurring, sharpening, and feature
enhancement.

Linear Filters:

A linear filter is a mathematical operation applied to an input image to

modify its pixel values. It works by convolving a small matrix called a kernel
or filter mask with the image. Each element of the kernel represents a
weight that determines the contribution of the corresponding pixel and its
neighbors to the output pixel value. The filter mask is usually a small
square or rectangular matrix with odd dimensions.

Convolution:

Convolution is the process of applying a linear filter to an image. It involves

sliding the kernel over the image and computing the element-wise
multiplication of the kernel with the corresponding image pixels within the
neighborhood. The resulting values are summed up, and the sum is
assigned to the output pixel. This process is repeated for each pixel in the
image, generating a new filtered image.

Convolution Operation:
The convolution operation can be described using the

following steps: 1. Place the kernel on top of the first pixel in

the image.

2. Multiply each element of the kernel with the corresponding

pixel value. 3. Sum up the results.

4. Assign the sum to the output pixel.

5. Slide the kernel to the next pixel and repeat the process until all pixels are
processed.

The output image obtained through convolution can have various visual
effects depending on the kernel used. Common examples of linear filters
include:
1. Edge detection filters: These filters highlight the edges in an image by
detecting changes in pixel intensity. Examples include the Sobel, Prewitt,
and Roberts filters.

2. Blur filters: These filters reduce image noise and smooth out details by
averaging pixel values in a neighborhood. The Gaussian blur filter is a
commonly used blur filter.

3. Sharpening filters: These filters enhance the edges and details in an

image by emphasizing high-frequency components. The Laplacian and
unsharp mask filters are popular sharpening filters.

4. Emboss filters: These filters create a 3D embossed or engraved effect

on an image by enhancing the differences in pixel intensities. They give the
 illusion of depth by simulating highlights and shadows.

Linear filters and convolution are essential tools in computer vision and
form the basis for more advanced techniques, such as convolutional neural
networks (CNNs), which have revolutionized the field of image analysis and
recognition.

1.3 Shift Invariant Linear Systems

Shift-invariant linear systems play a significant role in computer vision

tasks, particularly in image processing and analysis. These systems
operate on images or signals and possess two essential properties: shift
invariance and linearity. Let's explore these concepts further.

1. Shift Invariance: A shift-invariant system produces the same output

regardless of the input's spatial location. In the context of computer vision,
this means that the system's response to an image remains unchanged
even if the image is translated (shifted) by some amount.

2. Linearity: A linear system follows the principles of superposition and

scaling. Superposition implies that the response to a sum of inputs is equal
to the sum of the
individual responses to each input. Scaling refers to the output being
proportionally scaled when the input is scaled.

Shift-invariant linear systems are commonly employed in various computer

vision tasks, including image filtering, feature extraction, and convolutional
neural networks (CNNs). Here are a few examples:

1. Image Filtering: Convolutional filters are widely used to perform

operations such as blurring, edge detection, and sharpening on images.
These filters are shift-invariant and linear. The filter's weights remain the
same regardless of the image's location, and the response to an image is a
linear combination of its pixel values.

2. Feature Extraction: Shift-invariant linear systems can be used to extract

features from images. For instance, the Scale-Invariant Feature Transform
(SIFT) algorithm detects keypoints and computes descriptors that are
invariant to translation, rotation, and scaling. SIFT achieves shift invariance
by constructing scale-space representations and applying
difference-of-Gaussian filters.

3. Convolutional Neural Networks (CNNs): CNNs are a class of deep

learning models widely used in computer vision. They consist of multiple
layers of convolutional filters, which are shift-invariant and linear. The filters
convolve with the input image, extracting hierarchical features. CNNs
leverage the spatial locality and shift-invariance of the filters to learn
powerful representations for tasks like object recognition, image
segmentation, and more.

The shift-invariant property allows these systems to capture spatial

relationships and patterns across an image, making them highly effective in
computer vision applications. By leveraging linearity, these systems can be
designed and optimized efficiently using mathematical techniques, enabling
sophisticated image processing and analysis capabilities.

1.4 Spatial Frequency and Fourier transforms

Spatial frequency and Fourier transforms are fundamental concepts in
computer vision used to analyze and process images. They play a crucial
role in various image processing tasks, such as image filtering, feature
extraction, and image compression. Let's explore these concepts in more
detail:

1. Spatial Frequency:

Spatial frequency refers to the variation of intensity or color in an image

as a function of position. It represents how rapidly the pixel values change
across space. High spatial frequencies correspond to rapid changes, such
as edges or fine details, while low spatial frequencies represent slower
variations, such as smooth regions or large-scale structures.

2. Fourier Transform:

The Fourier transform is a mathematical tool used to decompose a

function (or a signal) into a sum of sinusoidal components with different
frequencies. In the context of computer vision, the Fourier transform is
often applied to images.

In the 2D Fourier transform, an image is represented as a sum of

complex sinusoids, which are called Fourier coefficients or frequency
components. The transform maps the image from the spatial domain to the
frequency domain. Each frequency component represents a specific spatial
frequency and its corresponding amplitude and phase. The lower
frequencies are located at the center of the frequency domain, while the
higher frequencies are closer to the edges.

3. Fourier Transform in Image Processing:

The Fourier transform is widely used in image processing for various

purposes, including:

a. Image Filtering: Applying a Fourier transform to an image allows us to

analyze its frequency content. By modifying the Fourier coefficients, we can
selectively filter out
certain frequency components and then transform the filtered image back
to the spatial domain. This process is used in tasks like noise removal,
sharpening, and blurring.

b. Feature Extraction: The Fourier transform can also be used to extract

 features from an image. By examining the magnitudes and orientations of
frequency components, we can identify distinctive patterns, textures, or
structures in the image.

c. Image Compression: Fourier transform-based techniques, such as

the Discrete Cosine Transform (DCT), are at the heart of image
compression algorithms like JPEG. These algorithms exploit the property
that many images contain energy concentrated in the lower spatial
frequencies, allowing for efficient compression by discarding high-frequency
components.

d. Image Registration: Fourier transforms are employed in image

registration tasks, where the goal is to align two or more images. By
analyzing the frequency content of the images, registration algorithms can
estimate the necessary transformations to align the images properly.

Overall, spatial frequency and Fourier transforms are powerful tools that
enable the analysis and manipulation of image data in computer vision
applications. They provide insights into the frequency content of images
and allow for various processing operations that can enhance and extract
meaningful information from visual data.

1.5 Sampling and Aliasing Filters as Templates

Sampling and aliasing filters play crucial roles in computer vision,

particularly when dealing with image acquisition and processing. Let's
discuss them as templates in computer vision.

1. Sampling Filters:
Sampling is the process of converting a continuous signal (analog) into a
discrete signal (digital) by capturing its values at specific time intervals. In
computer vision, sampling filters are used to convert continuous images
into discrete pixel grids.
The most commonly used sampling filter is the box filter, also known as the
nearest neighbor filter. It simply replicates the nearest pixel value to sample
points. Other commonly used sampling filters include bilinear and bicubic
filters, which consider neighboring pixels to determine the sampled value
and provide smoother results.

Sampling filters are essential because they define the relationship between
continuous and discrete representations of an image. They affect the
quality of the digital image and can introduce artifacts if not chosen
carefully. Properly designed filters help preserve image details during
sampling and minimize aliasing effects.

2. Aliasing Filters:

Aliasing occurs when a signal (e.g., an image) is sampled at a rate

insufficient to accurately represent its true frequency content. This leads to
artifacts such as moiré patterns, jagged edges, and distortion in the
reconstructed image. Aliasing filters are used to mitigate these artifacts.

The most common aliasing filter is the anti-aliasing filter, which is typically
applied before sampling an image. It removes high-frequency components
that could cause aliasing by attenuating them. The anti-aliasing filter
effectively acts as a low-pass filter, allowing only frequencies below the
Nyquist limit (half the sampling rate) to pass through.

In computer vision, anti-aliasing filters are employed to prevent aliasing

during image downscaling or resizing operations. By reducing the
high-frequency components before sampling, they help preserve the visual
quality of the resulting image.

It's important to note that anti-aliasing filters can be implemented using

various techniques, such as Gaussian filtering, Lanczos filtering, or
Mitchell-Netravali filtering.
These filters have different frequency response characteristics and
computational costs, allowing for trade-offs between quality and efficiency.

In summary, sampling filters are used to convert continuous images into

discrete representations, defining the pixel grid. Aliasing filters, particularly
anti-aliasing filters, are employed to mitigate artifacts caused by insufficient
sampling rates. Both types of filters are crucial for maintaining image quality
and accuracy in computer vision tasks.

1.6 Technique: Normalized Correlation and Finding Patterns

Normalized correlation is a technique commonly used in computer vision to

find patterns or similarities between images or image regions. It is a
measure of similarity that quantifies the resemblance between two images
by comparing their pixel intensities.

Here's how the normalized correlation technique works:

1. Image Alignment: First, the images or image regions to be compared are

aligned. This is important to ensure that corresponding features in both
images are compared accurately. Techniques such as feature detection and
matching or geometric transformations like translation, rotation, and scaling
can be used for alignment.

2. Windowing: To compare two images, a sliding window is typically used to

examine different regions of the images. The size of the window depends
on the scale and complexity of the patterns you are looking for. The window
is moved across the images, comparing the pixel values within each
window.
 3. Pixel Intensity Comparison: For each window position, the pixel
intensities of the two images are compared. One common approach is to
calculate the correlation coefficient between the pixel intensities of the two
windows. The correlation coefficient measures the linear relationship
between the intensities of corresponding pixels.
4. Normalization: After obtaining the correlation coefficient, it is normalized
to eliminate the influence of the overall brightness and contrast differences
between the images. This normalization step ensures that the technique is
robust to variations in illumination or intensity.

5. Pattern Matching: Finally, a threshold is applied to the normalized

correlation coefficient to determine if a match or similarity exists. If the
correlation coefficient exceeds a certain threshold, it indicates a potential
match or similarity between the patterns in the images.

Normalized correlation is widely used in various computer vision

applications, such as object detection, template matching, and image
registration. It allows for robust pattern matching even in the presence of
noise, occlusions, and geometric transformations. However, it may not be
the most efficient technique for large-scale or real-time applications due to
its computational complexity. In such cases, more advanced techniques like
deep learning-based methods may be preferred.

1.7 Technique: Scale and Image Pyramids.

Scale and image pyramids are techniques commonly used in computer

vision for various tasks such as object detection, feature extraction, and
image matching. These techniques involve creating multiple scaled
versions of an image to capture information at different levels of detail.
A scale pyramid, also known as an octave, is a series of images that are
generated by progressively down scaling the original image. Each level of
the pyramid is obtained by reducing the size of the previous level, typically
by a factor of 2. This process can be repeated multiple times to create a
pyramid with different scales.

The purpose of the scale pyramid is to capture objects or features at

different scales. By down scaling the image, smaller details become more
prominent, while larger structures
are preserved. This allows algorithms to detect objects or extract features
at different sizes.

The image pyramid, on the other hand, involves creating a series of images
where each level is a down sampled version of the original image. Unlike
the scale pyramid, the image pyramid retains the original aspect ratio and
captures the complete information of the image at different resolutions.

The image pyramid is useful for tasks such as multi-scale image blending,
image compression, and image enhancement. It allows algorithms to
operate on images at different resolutions, facilitating the processing of
large images or handling images with varying levels of detail.

Both scale and image pyramids have various applications in computer

vision. Some common use cases include:

1. Scale-invariant feature detection: Scale pyramids are used to detect key

points or features at different scales. This is important for tasks such as
object recognition and image matching, where objects may appear at
different sizes in different images.
2. Image segmentation: Image pyramids can be employed for multi-scale
segmentation, where the image is analyzed at different resolutions to detect
and classify regions of interest.

3. Image fusion: By combining corresponding regions from different levels

of an image pyramid, it is possible to create a composite image that
captures the best details from each scale. This can be useful for tasks like
super-resolution or generating high-quality images from low-resolution
inputs.
4. Image alignment: Scale and image pyramids can be utilized for aligning
images with different resolutions or scales. By matching corresponding
regions across multiple levels, it is possible to align images with varying
sizes or perspectives.

Overall, scale and image pyramids are powerful techniques in computer

vision that allow for multi-scale analysis and processing of images. They
enable algorithms to capture information at different levels of detail,
improving the robustness and accuracy of various computer vision tasks.