Attaphongse Taparugssanagorn - Classical Signal Processing and Non-Classical Signal Processing - The Rhythm of Signals-Cambridge Scholars Publishing (2023)

Classical Signal
Processing
and Non-Classical
Signal Processing:
The Rhythm of Signals

Classical Signal
Processing
and Non-Classical
Signal Processing:
By
Attaphongse Taparugssanagorn
Classical Signal Processing and Non-Classical Signal Processing:
By Attaphongse Taparugssanagorn
This book first published 2023
Cambridge Scholars Publishing
Lady Stephenson Library, Newcastle upon Tyne, NE6 2PA, UK
British Library Cataloguing in Publication Data

A catalogue record for this book is available from the British Library
Copyright © 2023 by Attaphongse Taparugssanagorn
All rights for this book reserved. No part of this book may be reproduced,
stored in a retrieval system, or transmitted, in any form or by any means,
electronic, mechanical, photocopying, recording or otherwise, without
the prior permission of the copyright owner.
ISBN (10): 1-5275-2864-2

ISBN (13): 978-1-5275-2864-2
TABLE OF CONTENTS
Chapter I ..................................................................................................... 1
Introduction
1. Background and motivation: This chapter introduces the concept

of signals and their importance in various fields such as
communication, healthcare, and entertainment. It also discusses
the motivation behind the book, which is to provide a
comprehensive overview of classical and non-classical signal
processing.
2. Overview of signal processing: This section provides an overview
of signal processing, including the different types of signals, the
importance of signal processing, and the different signal
processing techniques available.
Chapter II .................................................................................................... 7
Classical Signal Processing
1. Basic signal concepts: This section covers the fundamental

concepts of signals, such as amplitude, frequency, and phase. It
also discusses different signal types such as analog and digital
signals.
2. Fourier analysis and signal spectra: This section introduces the
Fourier transform, which is a fundamental tool for analyzing
signals in the frequency domain. It also discusses different types
of signal spectra, such as power spectral density and energy
spectral density.
3. Sampling and quantization: This section covers the concepts of
signal sampling and quantization, which are crucial in digital
signal processing. It also discusses different sampling and
quantization techniques and their trade-offs.
4. Signal filtering and convolution: This section covers signal
filtering techniques such as low-pass, high-pass, band-pass, and
vi Table of Contents
band-stop filters. It also discusses convolution, which is an

essential operation in signal processing.
5. Time and frequency domain representations: This section
discusses the relationship between the time and frequency domains
of signals. It also covers different time and frequency domain
representations such as the time-frequency distribution and
spectrogram.
6. Statistical signal processing: The chapter on statistical signal
processing introduces key concepts and techniques for analyzing
signals using statistical methods. It emphasizes the role of
probability theory, random variables, and random processes in
understanding uncertainty. The chapter covers estimation,
detection, hypothesis testing, signal classification, and pattern
recognition. It is important to have a solid understanding of
probability theory before diving into advanced topics. These
foundational concepts provide a strong basis for comprehending
the subsequent chapters on statistical signal processing.
Chapter III ................................................................................................ 82

Non-Classical Signal Processing
1. Wavelet transforms and time-frequency analysis: This section

introduces wavelet transforms, which are useful in analyzing non-
stationary signals. It also covers time-frequency analysis
techniques such as the short-time Fourier transform and the Gabor
transform.
2. Compressed sensing and sparse signal processing: This section
covers compressed sensing, which is a technique for reconstructing
signals from fewer measurements than traditional methods. It also
covers sparse signal processing, which is a technique for
processing signals that have a sparse representation.
3. Machine learning and deep learning for signals: This section
discusses the application of machine learning and deep learning
techniques to signal processing. It covers different machine
learning techniques such as support vector machines and neural
networks.
4. Signal processing for non-Euclidean data: This section covers
signal processing techniques for non-Euclidean data, such as
graphs and networks.
Classical Signal Processing and Non-Classical Signal Processing: vii
Chapter IV .............................................................................................. 177

Applications of Signal Processing
1. Audio and speech processing: This section covers signal

processing techniques used in audio and speech applications, such
as audio coding, speech recognition, and speaker identification.
2. Image and video processing: This section covers signal processing
techniques used in image and video applications, such as image
and video compression, object recognition, and tracking.
3. Biomedical signal processing: This section covers signal
processing techniques used in biomedical applications, such as
electrocardiogram analysis, magnetic resonance imaging, and
brain-computer interfaces.
4. Communications and networking: This section covers signal
processing techniques used in communication and networking
applications, such as channel coding, modulation, and equalization.
5. Sensor and data fusion: This section covers signal processing
techniques used in sensor and data fusion applications, such as data
integration, feature extraction, and classification.
Chapter V ............................................................................................... 205

Future Directions in Signal Processing
1. Emerging signal processing techniques and applications: This

section discusses emerging signal processing techniques and
applications, such as quantum signal processing and signal
processing for blockchain.
2. Challenges and opportunities in signal processing research: This
section covers the challenges and opportunities in signal processing
research, such as developing more efficient algorithms and
addressing privacy and security concerns.
3. Concluding remarks: This section provides concluding remarks on
the importance of signal processing and the potential impact of
future developments in the field.
Chapter VI .............................................................................................. 210

Appendices
1. Mathematical and computational tools for signal processing.
CHAPTER I
INTRODUCTION
Background and motivation

Signals are like persons; their sexiness and significance cannot be
determined solely by their looks. In the world of signal processing, we delve
deep into the values and meaning that signals carry. Some signals may have
a beautiful pattern or a nice appearance, but lack meaningful information.
However, with the application of signal processing techniques, even the
most peculiar-looking signals can be transformed into something truly sexy
and valuable.
In “Classical Signal Processing and Non-Classical Signal Processing: The

Rhythm of Signals,” author Attaphongse Taparugssanagorn introduces the
concept of signals and their significance in various fields such as
communication, healthcare, and entertainment. This comprehensive
exploration highlights the crucial role signals play in our daily lives, from
making a simple phone call to analyzing complex medical data.
Motivated by the desire to provide a comprehensive overview of classical

and non-classical signal processing, the book dives into fundamental
concepts such as Fourier analysis, signal filtering, and time and frequency
domain representations. It goes beyond traditional approaches and explores
cutting-edge topics like wavelet transforms, compressed sensing, and
machine learning for signals.
What sets this book apart is its unique perspective on presenting these
concepts. It demonstrates how signals can be made sexy and valuable
through the application of diverse signal processing techniques. It
showcases signal processing as a powerful tool for extracting new
information, transforming signals from mundane to captivating.
Ideal for students, researchers, and industry professionals, “Classical Signal

Processing and Non-Classical Signal Processing: The Rhythm of Signals”
covers both theory and practice, providing readers with a comprehensive
2 Chapter I
understanding of classical and non-classical signal processing techniques.

The book offers a fresh and engaging approach, making the subject
accessible and relevant to those working in emerging fields.
Moreover, as a bonus, the author, known for their talent as a rap rhyme
composer, provides entertaining rap rhyme summaries at the end of each
chapter. This unique addition allows readers to relax and enjoy a rhythmic
recap after engaging with the complex material. Additionally, the author
provides layman's explanations throughout the book, ensuring that readers
without a technical background can grasp the concepts.
Overall, “Classical Signal Processing and Non-Classical Signal Processing:

The Rhythm of Signals” is a captivating and comprehensive book that takes
readers on a journey through the world of signals and signal processing. It
combines theory and application, inspiring and engaging anyone with an
interest in the science of signals.
Overview of signal processing

This section provides an overview of signal processing, including the
different types of signals, the importance of signal processing, and the
different signal processing techniques available.
Signal processing is a field of study that focuses on the analysis, synthesis,

and modification of signals. A signal is a representation of a physical
quantity that varies over time or space, such as sound waves, images, or
biological signals. There are various types of signals, including continuous-
time signals, discrete-time signals, and digital signals. In Figure 1-1, we can
observe the different types of signals, such as sound waves, images or two-
dimensional (2-D) signal, and biological signals, e.g., electrocardiograms
(ECG) signal. These signals play a significant role in various fields such as
communication, healthcare, and entertainment.
In order to carry out signal processing effectively, it is crucial to have a deep

understanding of the characteristics exhibited by these signals. These
techniques range from classical methods such as Fourier analysis and filters
to modern approaches like wavelet transforms and machine learning. Each
technique offers unique capabilities and is applied based on the specific
requirements of the signal processing task.
Introduction 3
By utilizing these various signal processing techniques, we can extract

valuable information from signals, remove noise or interference, compress
data for efficient storage or transmission, and enhance the quality or
intelligibility of signals. Signal processing has wide-ranging applications in
fields such as telecommunications, audio and video processing, biomedical
engineering, radar and sonar systems, and many more. Each technique
offers unique capabilities and is applied based on the specific requirements
of the signal processing task.
a)
b)
4 Chapter I
c)
Figure 1-1: Different types of signals, including a) sound waves, b) images or 2-D
signal, and c) biological signals, e.g., ECG signal (right).
Signal processing plays a crucial role in many fields, such as

communication, healthcare, entertainment, and scientific research. For
example, in communication systems, signal processing techniques are used
to encode and decode messages, reduce noise and interference, and improve
the quality of the received signal. In healthcare, signal processing is used
for the analysis and interpretation of medical signals, such as ECG signal
and electroencephalograms (EEG) signal. In entertainment, signal
processing is used to create and modify audio and visual signals, such as
music and movies.
There are various signal processing techniques available, ranging from

classical techniques such as Fourier analysis, filtering, and time and
frequency domain representations, to more recent techniques such as
wavelet transforms, compressed sensing, and machine learning. Each
technique has its own strengths and weaknesses, and the choice of technique
depends on the specific application and the requirements of the signal
processing task.
Introduction 5
Rhyme summary and key takeaways:

The introduction chapter is summarized as follows:
The overview of signal processing is critical. As it highlights the different

types of signals, their significance and why it is pivotal. Signals can be
sound waves, images or biological and signal processing techniques are
used to make them logical.
In communication, signals are encoded and decoded to transmit messages

clear. While in healthcare, medical signals are analyzed to give a diagnosis
and cure.
Signal processing is also used in entertainment to make music and movies

sound great. With classical and modern techniques, the results are first-rate.
Classical techniques like Fourier and filters are still essential and time and
frequency domains are signal processing fundamentals.
However, new techniques like wavelets and machine learning are emerging,
Making signals more exciting and the science behind it compelling.

“Classical Signal Processing and Non-Classical Signal Processing: The
Rhythm of Signals” is the book that makes signals captivating and engaging.
It is perfect for students, researchers and industry professionals, for
knowledge ranging.
With a fresh approach and a mix of theory and practice, it is the perfect
guide. To understand classical and non-classical signal processing
techniques and ride high.
Key takeaways from the introduction chapter are given as follows:
1. Signals are diverse and have significance in various fields such as

communication, healthcare, and entertainment. Understanding
signal processing is crucial to make sense of these signals.
2. Signal processing techniques are used to encode and decode
signals for clear message transmission, analyze medical signals for
diagnosis, and enhance the quality of music and movies.
3. Classical signal processing techniques like Fourier analysis and
filters are still fundamental and important in both time and
frequency domains.
6 Chapter I
4. Newer techniques such as wavelets and machine learning are

emerging, adding excitement and complexity to signal processing.
5. The book “Classical Signal Processing and Non-Classical Signal
Processing: The Rhythm of Signals” takes a fresh approach,
blending theory and practice, making it an engaging guide for
students, researchers, and industry professionals.
6. The book covers classical and non-classical signal processing
techniques, providing a comprehensive understanding of the
subject.
Overall, the introduction chapter establishes the importance of signal

processing, its applications in various domains, and sets the stage for an
intriguing exploration of the topic in the subsequent chapters of the book.
Layman’s guide:
In simple terms, the introduction chapter is all about signals and how they
are processed. Signals can be different types of things like sound waves,
images, or biological data. Signal processing is important because it helps
us understand and make sense of these signals.
Signal processing is used in different areas. In communication, signals are

encoded and decoded to send clear messages. In healthcare, medical signals
are analyzed to diagnose and treat patients. And in entertainment, signal
processing is used to make music and movies sound great.
There are classical techniques that have been used for a long time, like
Fourier analysis and filters. These techniques are still really important in
understanding signals in terms of time and frequency.
But there are also newer techniques like wavelets and machine learning that
are emerging. These techniques make signal processing more exciting and
add complexity to the science behind it.
The book “Classical Signal Processing and Non-Classical Signal

Processing: The Rhythm of Signals” is introduced as a guide that makes
signals interesting and engaging. It is a mix of theory and practice, which
makes it perfect for students, researchers, and industry professionals who
want to learn about both classical and newer signal processing techniques.
CHAPTER II
CLASSICAL SIGNAL PROCESSING1
Classical signal processing is a branch of electrical engineering and applied

mathematics that deals with the analysis, modification, and synthesis of
signals. It encompasses various techniques for transforming, filtering, and
analyzing signals to extract useful information or enhance their quality. The
main goal of classical signal processing is to improve the performance and
efficiency of systems that rely on signals, such as communication systems,
audio and video processing, and control systems.
Classical signal processing consists of several fundamental techniques,

including Fourier analysis, filtering, modulation, and digital signal
processing. Fourier analysis is used to represent a signal in the frequency
domain, allowing us to decompose it into its constituent frequencies.
Filtering is the process of selectively removing or attenuating certain
frequency components of a signal to extract or enhance specific features.
Modulation involves manipulating a signal's amplitude, frequency, or phase
to encode information or transmit it over a communication channel. Digital
signal processing involves the use of computers to process signals in a
discrete-time domain.
The applications of classical signal processing are widespread, and it has

revolutionized several fields, including telecommunications, audio and
video processing, and control systems. In telecommunications, signal
processing techniques are used for modulation, encoding, decoding, and
error correction in wireless communication systems, satellite communication
systems, and optical communication systems. In audio and video
processing, signal processing techniques are used for compression, noise
reduction, and enhancement of audio and video signals. In control systems,
1 Classical Signal Processing refers to the traditional methods of analyzing and
manipulating signals that are based on mathematical and engineering principles. It

consists of several techniques such as filtering, modulation, demodulation,
sampling, quantization, and signal reconstruction. These techniques are applied to
various types of signals such as audio, images, videos, and data to extract relevant
information and make them useful for different applications.
8 Chapter II
signal processing techniques are used for feedback control, system

identification, and fault diagnosis. Overall, classical signal processing plays
a vital role in modern technology and has enabled significant advancements
in various fields.
Basic signal concepts

This section covers the fundamental concepts of signals, such as amplitude,
frequency, and phase. It also discusses different signal types such as analog
and digital signals.
Fourier analysis and signal spectra

This section introduces the Fourier transform, which is a fundamental tool
for analyzing signals in the frequency domain. It also discusses different
types of signal spectra, such as power spectral density and energy spectral
density.
Signals are physical phenomena that vary over time or space and can be
represented mathematically as functions. Amplitude, frequency, and phase
are three fundamental concepts of signals. Amplitude refers to the
magnitude of a signal, which represents the strength of the signal.
Frequency is the number of cycles per unit of time, and it determines the
pitch of the signal. Phase is the position of a waveform relative to a fixed
reference point in time.
Signals can be classified into two main types: analog and digital signals.
Analog signals are continuous-time signals that vary smoothly over time
and can take any value within a range. On the other hand, digital signals are
discrete-time signals that have a finite set of possible values.
Fourier analysis is a mathematical technique used to represent a signal in

the frequency domain. It allows us to decompose a signal into its constituent
frequencies, which are represented as complex numbers. The Fourier
transform is the mathematical tool used to perform this decomposition. The
frequency domain representation of a signal provides valuable information
about the signal's spectral characteristics, such as its frequency components
and their relative strengths.
There are different types of signal spectra that can be derived from the
Fourier transform. Power spectral density (PSD) is a measure of the power
of a signal at each frequency. Energy spectral density (ESD) is a measure
Classical Signal Processing 9
of the energy of a signal at each frequency. The PSD and ESD are essential
tools for characterizing signals and are widely used in various fields,
including communication systems, audio processing, and image processing.
In summary, understanding the fundamental concepts of signals, such as

amplitude, frequency, and phase, and their different types, such as analog
and digital signals, is crucial for signal processing. Fourier analysis and the
different types of signal spectra provide valuable insights into the spectral
characteristics of signals, enabling us to analyze and process them
effectively.
The Fourier transform, denoted by F(Ȧ), is used to represent a signal in the

frequency domain. It is mathematically defined as
ஶ
‫ܨ‬ሺ߱ሻ = ‫ି׬‬ஶ ݂ሺ‫ݐ‬ሻ݁ ି௝ఠ௧ ݀‫ݐ‬, (1)
where ݂ሺ‫ݐ‬ሻ is the original signal, Ȧ is the angular frequency, and ݆ =

ξെ1 is the imaginary unit.
Figure 2-1: An example of a sinusoidal signal ݂ሺ‫ݐ‬ሻ = ‫ܣ‬sin(2ߨ݂‫)ݐ‬, where A=1 and
f= 10 Hz and its Fourier transform.
10 Chapter II
Amplitude, frequency, and phase are three fundamental concepts of signals.

Amplitude is represented by A, which refers to the magnitude of a signal
and represents the strength of the signal. Frequency is represented by ݂ or
߱, which is the number of cycles per unit of time, and it determines the
pitch of the signal. Phase is represented by ߠ, which is the position of a
waveform relative to a fixed reference point in time. An example of a
sinusoidal signal ݂(‫ܣ = )ݐ‬sin(2ߨ݂‫)ݐ‬, where A=1 and f= 10 Hz and its
Fourier transform is depicted in Figure 2.1. As can be seen, the Fourier
transform of such a period signal is represented by the unit impulse ߜ(‫)ݐ‬.
The concept of the Fourier transform is rooted in the idea that any periodic
signal can be expressed as a sum of sinusoidal components of different
frequencies. The Fourier transform allows us to analyze a signal in the
frequency domain, providing insight into its spectral content. In the case of
a sinusoidal signal, the Fourier transform simplifies to a unit impulse,
ߜ(‫)ݐ‬, which represents a pure frequency component at the specific
frequency of the sinusoid.
This example highlights the Dirichlet's condition for the existence of the
Fourier transform. According to Dirichlet's condition, a periodic function
must satisfy certain requirements in order for its Fourier transform to exist.
These conditions ensure that the function has a finite number of
discontinuities, finite number of extrema, and finite total variation within a
given period. In the case of the sinusoidal signal described, it satisfies
Dirichlet's condition, allowing its Fourier transform to be represented by the
unit impulse, indicating the presence of a single frequency component.
Overall, this example illustrates the connection between a sinusoidal signal,

its Fourier transform, and the concept of Dirichlet's condition, providing a
fundamental understanding of the relationship between time-domain and
frequency-domain representations of signals.
Signals can be classified into two main types: analog and digital signals.
Analog signals are continuous-time signals that vary smoothly over time
and can take any value within a range. Mathematically, they are represented
as functions of continuous variables. On the other hand, digital signals are
discrete-time signals that have a finite set of possible values. They are
represented as a sequence of numbers.
PSD is a measure of the power of a signal at each frequency and is defined

as
ଵ
ܲܵ‫ = )߱(ܦ‬lim |‫|)߱(ܨ‬ଶ , (2)
்՜ஶ ்
where ‫ )߱(ܨ‬is the Fourier transform of the signal and ܶ is the observation
time.
ESD is a measure of the energy of a signal at each frequency and is defined

as follows:
‫|)߱(ܨ| = )߱(ܦܵܧ‬ଶ . (3)
Both PSD and ESD are essential tools for characterizing signals and are
widely used in various fields, including communication systems, audio
processing, and image processing.

The Fourier analysis and signal spectra section is summarized as follows.
Let me break it down for you, listen close. We are diving into Fourier
analysis, a powerful dose. Signals and their spectra, we are going to explore,
in the frequency domain, we will uncover more.
First, let us talk signals, they are quite profound. Amplitude, frequency, and
phase, all around. Amplitude's the strength, the magnitude you see,
Frequency determines pitch, cycles per unit, it is key. Phase tells us the
position in time, it is neat. With these concepts, signals become complete.
Analog and digital, two signal types we know, Analog's continuous,

smoothly they flow. Digital's discrete, with a set of values, they are finite.
Represented as a sequence, clear and right.
Now, Fourier transform takes us on a ride. It represents signals in the

frequency stride. ‫ )߱(ܨ‬is the transform, symbol of the game. Integrating
݂(‫ )ݐ‬with exponential to the power, it is no shame.
Power spectral density, PSD, let us unveil. Measures signal power at each
frequency detail. ESD, energy spectral density, joins the parade. It measures
signal energy, frequencies displayed.
These spectra are crucial, you know they are grand, in communication,
audio, and image land. They help us analyze and process signals with flair.
Understanding their characteristics, beyond compare.
12 Chapter II
So, wrap it up, Fourier transform is the key. With amplitude, frequency,
phase, you see. Analog, digital, their differences profound, PSD and ESD,
spectra that astound.
Now you know the signals and their flow, Fourier analysis, it is time to let
it show. Understanding signals and their spectra is a must. With these tools,
you conquer, there is no doubt you will thrust.Top of Form
Key takeaways from the Fourier analysis and signal spectra section are
given as follows:
1. Signals can be characterized by their amplitude, frequency, and

phase, which together provide a complete description of the signal.
2. Fourier transform is a powerful tool that represents signals in the
frequency domain, allowing us to analyze and process them
effectively.
3. Analog signals are continuous and smoothly varying, while digital
signals are discrete and represented by a finite set of values.
4. The Fourier transform involves integrating the signal with
exponential functions to obtain its frequency representation.
5. PSD measures the power of a signal at each frequency, while ESD
measures the energy of a signal at different frequencies.
6. Spectral analysis is crucial in various fields like communication,
audio, and image processing, as it helps us understand the
characteristics of signals and enables effective signal processing.
7. By understanding Fourier analysis and signal spectra, you gain the
key tools to analyze and manipulate signals, enhancing your ability
to work with them effectively.
Layman’s guide:
Let me break it down for you in simple terms. We are going to explore
Fourier analysis, a powerful tool for understanding signals and their spectra
in the frequency domain.
Signals are quite interesting. They have three important properties:

amplitude, frequency, and phase. Amplitude represents the strength or
magnitude of a signal. Frequency determines the pitch and is measured in
cycles per unit. Phase tells us the position of the signal in time. These
concepts help us fully describe a signal.
There are two types of signals we commonly encounter: analog and digital.
Analog signals are continuous and flow smoothly, while digital signals are
discrete and have a set of specific values. Digital signals are often
represented as a sequence.
Now, let us talk about the Fourier transform. It is like a magical journey that
takes signals into the frequency domain. The Fourier transform represents
signals in terms of their frequency components. It involves integrating the
signal with exponential functions raised to a power.
We also have two important measures: power spectral density (PSD) and
energy spectral density (ESD). PSD measures the power of a signal at
different frequencies, providing detailed information about signal power.
ESD measures the energy of a signal at different frequencies, giving us
insights into its energy distribution.
These spectra are crucial in various fields like communication, audio, and
image processing. They help us analyze and process signals with expertise.
By understanding the characteristics of signals through their spectra, we
gain valuable insights that are unmatched.
To sum it up, the Fourier transform is the key tool in this journey. It allows
us to analyze signals using their amplitude, frequency, and phase. We also
learn about the differences between analog and digital signals, as well as the
significance of PSD and ESD.
Exercises of Fourier analysis and signal spectra

Problem 1: Identifying the Dominant Frequencies in a Music Signal
Question: How can we identify the dominant frequencies present in a music
signal using Fourier analysis?
Solution: Fourier analysis can be used to decompose a music signal into its
constituent frequencies. By applying the Fourier transform to the music
signal, we can obtain the frequency spectrum, which represents the
amplitudes of different frequencies present in the signal. By analyzing the
frequency spectrum, we can identify the dominant frequencies, which
correspond to the main musical notes or tones in the music signal.
14 Chapter II
MATLAB example:
Step 1: Load the music signal in MATLAB.
[y, Fs] = audioread('music.wav');
Step 2: Compute the Fourier transform of the music signal using fast
Fourier transform (FFT) command fft.
Note that FFT is a specific algorithm used to compute the Discrete Fourier
Transform (DFT) of a sequence or signal. The DFT is a mathematical
transformation that converts a discrete-time signal from the time domain
into the frequency domain. It reveals the frequency components present in
the signal and their respective magnitudes and phases.
While The DFT computation involves performing N complex

multiplications and N-1 complex additions for each frequency bin. This
direct calculation has a computational complexity of O(ܰ ଶ ), which can be
quite slow for large input sizes, the FFT algorithm, on the other hand, is a
fast implementation of the DFT that significantly reduces the computational
complexity to O(NlogN). It exploits the symmetry properties of the DFT
and divides the signal into smaller subproblems, recursively computing their
DFTs. By using this divide-and-conquer approach, the FFT algorithm
achieves a substantial speedup compared to the direct DFT calculation.
Y = fft(y);
Step 3: Compute the frequency axis.
L = length(y);
f = Fs*(0:(L/2))/L;
Step 4: Plot the single-sided amplitude spectrum.
P = abs(Y/L);
P = P(1:L/2+1);
plot(f, P)
title('Single-Sided Amplitude Spectrum of Music Signal')
xlabel('Frequency (Hz)')
ylabel('Amplitude') )
Step 5: Identify the dominant frequencies from the plot.
This code snippet demonstrates how to load a music signal, compute its
Fourier transform, and plot the single-sided amplitude spectrum. By
analyzing the resulting spectrum, you can identify the dominant frequencies
present in the music signal.
By performing Fourier analysis on music signals, we can understand the

frequency content, identify musical elements, and gain insights into the
composition, performance, and overall structure of the music.
Problem 2: How can we use Fourier analysis to remove background noise

from an audio recording? Provide solutions and illustrate the process using
PSD.
Solution:
Background noise can degrade the quality of an audio recording. Fourier

analysis, along with PSD, can be employed to remove background noise.
Here are two solutions using PSD:
Solution 1: Filtering in the Frequency Domain using PSD
Filtering in the frequency domain using PSD is a technique used to remove

background noise from an audio recording. The process involves analyzing
the frequency content of the noisy audio signal and the background noise
using Fourier analysis, estimating the noise power spectrum, and
subtracting it from the PSD of the noisy audio signal to obtain a cleaner
version of the audio.
Here is a step-by-step explanation of the process.
Step 1: Load the noisy audio signal and the background noise in
MATLAB.
MATLAB example:
[y, Fs] = audioread('noisy_audio.wav');

[noise, ~] = audioread('background_noise.wav');
16 Chapter II
Step 2: Compute the Fourier transforms of the noisy audio signal and
the background noise.
MATLAB example:
Y = fft(y);
N = fft(noise);
Step 3: Compute the power spectral densities (PSDs) of the noisy audio
signal and the background noise.
MATLAB example:
PSD_y = abs(Y).^2;
PSD_n = abs(N).^2;
Step 4: Estimate the noise power spectrum by averaging the PSD of the
background noise.
MATLAB example:
estimated_noise_PSD = mean(PSD_n, 2);
Step 5: Subtract the estimated noise power spectrum from the PSD of
the noisy audio signal.
MATLAB example:
clean_PSD = max(PSD_y - estimated_noise_PSD, 0);
Step 6: Reconstruct the clean audio signal using the inverse Fourier
transform.
MATLAB example:
clean_signal = ifft(Y .* sqrt(clean_PSD), 'symmetric');
Solution 2: Wiener Filtering in the Frequency Domain using PSD
Wiener filtering in the frequency domain using PSD is a technique used to

remove noise from an audio recording while preserving the desired signal.
The approach is based on the Wiener filtering theory, which utilizes the
statistical properties of the desired signal and the noise to perform optimal
noise reduction.
The theory behind Wiener filtering involves the statistical properties of the
desired signal and the noise. It assumes that both the desired signal and the
noise are stochastic processes and have certain statistical characteristics.
The Wiener filter aims to estimate the clean signal by considering the
statistical properties of both the desired signal and the noise. It computes a
filter transfer function that minimizes the mean square error between the
estimated clean signal and the desired signal. The filter transfer function is
computed based on the PSDs of the desired signal and the noise. The Wiener
filter assumes that the clean and noise signals are statistically uncorrelated.
The steps involved are as follows:
1. Calculate the signal-to-noise ratio (SNR):

x Subtract the estimated noise PSD from the PSD of the
observed noisy signal (PSD_y).
x Take the maximum between the difference and zero to
ensure a non-negative SNR.
x Divide the result by the PSD of the observed noisy signal.
The SNR represents the ratio of the signal power to the noise power
and provides a measure of the noise contamination in the observed
signal.
2. Calculate the clean PSD:

x Multiply the PSD of the observed noisy signal (PSD_y)
by the SNR.
x Divide the result by the sum of the SNR and 1.
This step applies the Wiener filter by weighting the PSD of the
observed signal based on the estimated SNR. The goal is to
enhance the clean signal components and suppress the noise
components.
It is important to note that the effectiveness of the Wiener filter depends on

the accuracy of the estimated noise PSD and the assumption that the clean
and noise signals are statistically uncorrelated. In practice, the noise PSD
estimation can be challenging, and deviations from the assumptions may
affect the filter's performance.
The key principle behind Wiener filtering is that it provides an optimal

trade-off between noise reduction and preservation of desired signal
components. By taking into account the statistical properties of the signal
and the noise, the Wiener filter adapts its filtering characteristics to different
frequency components of the signal.
18 Chapter II
Here is a step-by-step explanation of the process.
Step 1: Load the noisy audio signal and the background noise in
MATLAB.
MATLAB example:
[y, Fs] = audioread('noisy_audio.wav');

[noise, ~] = audioread('background_noise.wav');
Step 2: Compute the Fourier transforms of the noisy audio signal and
the background noise.
MATLAB example:
Y = fft(y);
N = fft(noise);
Step 3: Compute the power spectral densities (PSDs) of the noisy audio
signal and the background noise.
MATLAB example:
PSD_y = abs(Y).^2;
PSD_n = abs(N).^2;
Step 4: Estimate the power spectral density of the clean audio signal
using the Wiener filter.
MATLAB example:
SNR = max(PSD_y - estimated_noise_PSD, 0) ./ PSD_y;

clean_PSD = PSD_y .* SNR ./ (SNR + 1);
Step 5: Reconstruct the clean audio signal using the inverse Fourier
transform.
MATLAB example:
clean_signal = ifft(Y .* sqrt(clean_PSD), 'symmetric');
Both solutions utilize Fourier analysis to analyze the frequency content of

the audio signals and estimate the noise power spectrum, which is a common
approach in signal processing. However, it is important to note that noise
reduction techniques can also be applied in the time domain.
In the time domain approach, the noisy audio signal is directly processed in
the time waveform. Techniques such as temporal filtering, adaptive
filtering, and statistical modeling can be employed to estimate and suppress
the unwanted noise components in the signal. This approach operates on the
amplitude and temporal characteristics of the signal, making it suitable for
certain scenarios where time-domain processing is effective and simpler.
On the other hand, the frequency domain approach, as mentioned earlier,

utilizes Fourier analysis to convert the audio signal from the time domain to
the frequency domain. By examining the frequency content, the noise power
spectrum can be estimated and subtracted from the PSD of the noisy audio
signal. This process effectively attenuates the noise components and yields
a cleaner version of the audio signal.
The choice between time and frequency domain processing depends on

various factors, including the nature of the noise, the complexity of the
signal, computational efficiency, and the available signal processing
techniques. In some cases, time-domain processing may be more suitable
due to its simplicity and effectiveness in certain noise scenarios. However,
the frequency domain approach, with its ability to analyze and manipulate
the frequency components of the signal, offers a powerful toolset for noise
reduction and audio enhancement.
Ultimately, the decision to employ time or frequency domain processing

should be based on the specific requirements and constraints of the
application, as well as the most effective and efficient techniques available.
Both approaches have their merits and can be utilized to achieve high-
quality noise reduction and audio enhancement results.
The implementation complexity can vary depending on the specific

algorithms and techniques used within each domain. It is important to
consider factors such as computational efficiency, memory requirements,
and real-time processing capabilities when choosing the appropriate
implementation approach. Simpler implementations may sacrifice some
level of performance or adaptability compared to more advanced techniques
but can still provide satisfactory results in certain scenarios.
Both time and frequency domain noise reduction techniques can have
simpler implementations depending on the specific requirements and
constraints of the application. Here are some considerations for simpler
implementations in each domain:
20 Chapter II
Sampling and quantization

This section covers the concepts of signal sampling and quantization, which
are crucial in digital signal processing. Imagine you have an analog signal,
like a sound wave or a picture. If you want to use a computer to process,
store, or analyze that signal, you need to convert it into a digital form. It is
like building a bridge between the analog and digital worlds.
In digital signal processing, sampling and quantization are the fundamental

processes that make this conversion possible. Sampling is like taking
snapshots or pictures of the continuous analog signal at regular intervals. It
is similar to an artist making quick brushstrokes on a canvas to capture the
essence of a moving scene. Each snapshot becomes an important building
block in creating our digital representation of the signal.
Quantization, on the other hand, is the process of representing each snapshot

with a specific value. It is like rounding off the values to fit into a limited
set of possibilities. This step helps us store and process the signal using finite
numbers. However, it also introduces some trade-offs and compromises in
terms of the accuracy and quality of the digital representation.
This chapter explores the intricacies of sampling and quantization,

highlighting their significant role in digital signal processing. It also
discusses different techniques for sampling and quantization, along with the
trade-offs that come with each approach. By understanding these processes,
we can better appreciate how digital signals are created and manipulated in
the world of digital signal processing.
Sampling process
The potency of sampling lies in the careful determination of the sampling

frequency, a critical decision that significantly impacts the fidelity of the
resulting digital representation. The Nyquist-Shannon sampling theorem
emerges as a guiding principle, illuminating the path towards faithful signal
reconstruction. Mathematically, this theorem dictates that a bandlimited
continuous-time signal with a maximum frequency component of ݂୫ୟ୶ can
be perfectly reconstructed from its samples if the sampling rate ݂௦ is greater
than or equal to twice ݂୫ୟ୶ (i. e. , ݂௦ ൒ 2݂୫ୟ୶ ). Adhering to this theorem
ensures the faithful preservation of the intricate nuances inherent in the
analog symphony, allowing subsequent digital processing to unfold with
precision and accuracy. We mathematically represent an analog signal in a
digital form, i.e., as a sequence of numbers ሼ‫ݒ‬ሾ݊ሿሽ =

ሼ. . . , ‫ݒ‬ሾെ2ሿ, ‫ݒ‬ሾെ1ሿ, ‫ݒ‬ሾ0ሿ, ‫ݒ‬ሾ1ሿ, ‫ݒ‬ሾ2ሿ, . . . ሽ.
Figure 2-2: Sampling process.
For convenience, a sequence {‫ݒ‬ሾ݊ሿ} is normally written as v[n]. The signal

‫ݒ‬ሾ݊ሿ is referred to as a discrete-time signal whose values are taken from the
corresponding analog signal ‫ )ݐ(ݒ‬by ‫ݒ‬ሾ݊ሿ=‫)ܶ݊(ݒ‬, ݊ ‫ܼ א‬, where ܶ is the
sampling period while ݂௦ = 1/ܶ is the sampling frequency or sampling rate.
It is convenient to represent the sampling process in the two following
stages, as illustrated in Figure 2-2.
22 Chapter II
Figure 2-3: Fourier transforms of sampled signals.
1. Multiplication by a periodic impulse train with period T, i.e.,
‫ = )ݐ(ݏ‬σஶ
௡ୀିஶ ߜ(‫ ݐ‬െ ݊ܶ). (4)
With the sifting property2 of the unit impulse ߜ(‫)ݐ‬, multiplying ‫ )ݐ(ݒ‬by
‫ )ݐ(ݏ‬gives us the signal ‫ݒ‬௦ (‫ )ݐ‬as
2 The sifting property of the unit impulse ߜ(‫ )ݐ‬states that when the impulse function
ߜ(‫ )ݐ‬is integrated with another function ݂(‫)ݐ‬, it “sifts out” the value of ߜ(‫ )ݐ‬at t = 0.
‫ݒ‬௦ (‫ )ݐ(ݒ = )ݐ(ݏ)ݐ(ݒ = )ݐ‬σஶ

௡ୀିஶ ߜ(‫ ݐ‬െ ݊ܶ) = σ௡ୀିஶ ‫ ݐ(ߜ)ܶ݊(ݒ‬െ ݊ܶ).(5)
ஶ
2. Conversion of the impulse train to a sequence, i.e., discrete-time

signal.
The Fourier transform of the sampled signal ‫ݒ‬௦ (‫ )ݐ‬consists of periodically

repeated copies at ܸ(݂) equally spaced apart by ݂௦ as illustrated in Figure
2-3 and expressed as
ଵ
ܸ௦ (݂) = ܸ(݂) ‫ = )݂(ܵ כ‬σஶ
௡ୀିஶ ܸ(݂ െ ݂݇௦ ), (6)
்
ଵ
௡ୀିஶ ߜ(݂ െ ݂݇௦ ) and ‫ כ‬is the convolution integral.
where ܵ(݂) = σஶ
்
Sampling is not without its challenges, such as aliasing, where the signal
gets distorted due to inadequate sampling rates. To prevent this, anti-
aliasing filters are crucial as they protect against unwanted frequencies,
ensuring the accuracy of the captured samples. By navigating the
complexities of sampling and embracing quantization, we can transform
continuous analog signals into digital form, like sculpting, shaping them
into discrete values for manipulation in the digital realm.
Quantization process
Next, we discuss quantization of a single symbol produced from a source,

e.g., its sample value. A scalar quantizer with M levels partitions the set R
into M subsets ܴଵ , … , ܴெ called quantization regions. Each region ܴ௠ , m ‫א‬
{1,…,M}, is then represented by a quantization point ‫ݍ‬௠ ‫ܴ א‬௠ . If a symbol
u ‫ܴ א‬௠ is produced from the source, then u is quantized to ‫ݍ‬௠ . Our goal is
to treat the following problem. Let U be a random variable (RV) denoting a
source symbol with probability density function (PDF) ݂௎ (‫)ݑ‬. Let ‫ )ܷ(ݍ‬be
a RV denoting its quantized value. Given the number of quantization levels
M, we want to find the quantization regions ܴଵ , … , ܴெ and the quantization
points ‫ݍ‬ଵ , … , ‫ݍ‬ெ to minimize the following mean square error (MSE)
distortion expressed as
ଶ ஶ ଶ
‫ ܧ = ܧܵܯ‬ቂ൫ܷ െ ‫)ܷ(ݍ‬൯ ቃ = ‫ି׬‬ஶ൫‫ ݑ‬െ ‫)ݑ(ݍ‬൯ ݂௎ (‫ݑ݀)ݑ‬. (7)
In other words, the integral of ߜ(‫ )ݐ‬times ݂(‫ )ݐ‬is equal to ݂(0). In general, it can be
ஶ
mathematically expressed as ‫ି׬‬ஶ ݂(‫ ݐ(ߜ)ݐ‬െ ‫ݐ‬଴ ) = ݂(‫ݐ‬଴ ).
24 Chapter II
Figure 2-4: Example quantization regions and quantization points for M = 4.
Now, let us assume that ܴଵ , … , ܴெ are intervals, as shown in Figure 2-4. We

ask the two following simplified questions.
1. Given ‫ݍ‬ଵ , … , ‫ݍ‬ெ , how do we choose ܴଵ , … , ܴெ ?
2. Given ܴଵ , … , ܴெ , how do we choose ‫ݍ‬ଵ , … , ‫ݍ‬ெ ?
We first consider the problem of choosing ܴଵ , … , ܴெ given ‫ݍ‬ଵ , … , ‫ݍ‬ெ . For a

given u ‫ א‬R, the square error to ‫ݍ‬௠ is (‫ ݑ‬െ ‫ݍ‬௠ )ଶ . To minimize the MSE,
‫ ݑ‬should be quantized to the closest quantization point, i.e., ‫ݍ = )ݑ(ݍ‬௠ ,
where ݉ = arg min௝‫א‬ሼଵ,…,ெ} (‫ ݑ‬െ ‫ݍ‬௝ )ଶ . It follows that the boundary point
ܾ௠ between ܴ௠ and ܴ௠ାଵ must be the halfway point between ‫ݍ‬௠ and ‫ݍ‬௠ାଵ ,
i.e., ܾ௠ = (‫ݍ‬௠ + ‫ݍ‬௠ାଵ )/2. In addition, we can say that ܴଵ , … , ܴெ must be
intervals. We now consider the problem of choosing ‫ݍ‬ଵ , … , ‫ݍ‬ெ
given ܴଵ , … , ܴெ . Given ܴଵ , … , ܴெ , the MSE in (7) can be written as
௠ୀଵ ‫׬‬ோ (‫ ݑ‬െ ‫ݍ‬௠ ) ݂௎ (‫ݑ݀)ݑ‬.

‫ = ܧܵܯ‬σெ ଶ
(8)
೘
To minimize the MSE, we can consider each quantization region separately

from the rest. Define a RV ܸ such that ܸ = ݉ if ܷ ‫ܴ א‬௠ , and let ‫݌‬௠ =
Pr {ܸ = ݉}. The conditional PDF of ܷ given that ܸ = ݉ can be written as
௙ೆ,ೇ (௨,௠) ௙ೇ|ೆ (݉|‫) ݑ‬௙ೆ (௨)
݂௎|௏ (‫= )݉|ݑ‬ = = ݂௎ (‫)ݑ‬/݂௏ (݉) = ݂௎ (‫)ݑ‬/‫݌‬௠
௙ೇ (௠) ௙ೇ (௠)
in region ܴ௠ . In terms of ݂௎|௏ (‫)݉|ݑ‬, the contribution of region ܴ௠ to the

MSE can be written as
න (‫ ݑ‬െ ‫ݍ‬௠ )ଶ ݂௎ (‫݌ = ݑ݀)ݑ‬௠ න (‫ ݑ‬െ ‫ݍ‬௠ )ଶ ݂௎ (‫)ݑ‬/‫݌‬௠ ݀‫ݑ‬

ோ೘ ோ೘
= ‫݌‬௠ ‫׬‬ோ (‫ ݑ‬െ ‫ݍ‬௠ )ଶ ݂௎|௏ (‫݌ = ݑ݀)݉|ݑ‬௠ ‫ܧ‬ሾ(ܷ െ ‫ݍ‬௠ )ଶ |ܸ = ݉ሿ. (9)
೘
It is known that the value of a that minimizes ‫ܧ‬ሾ(ܺ െ ܽ)ଶ ሿ is the mean of
ܺ, i.e., ‫ܧ‬ሾܺሿ = arg minୟ‫ܧ ୖא‬ሾ(ܺ െ ܽ)ଶ ሿ. Therefore, the MSE is minimized
when we set ‫ݍ‬௠ equal to the conditional mean3 of ܷ given ܸ = ݉, i.e.,
‫ݍ‬௠ = ‫ܴ א ܷ|ܷ[ܧ = ]݉ = ܸ|ܷ[ܧ‬௠ ].
An array of quantization techniques exists, each offering a unique

amalgamation of precision and complexity. The most prevalent among
them, uniform quantization, partitions the continuous amplitude range into
equal intervals, establishing a uniform grid upon which analog values are
mapped. Mathematically, uniform quantization can be represented as
௫
ܳ(‫ = )ݔ‬οround ቀ ቁ, (10)
ο
where ‫ ݔ‬represents the continuous analog value, ο denotes the quantization

step size (equal interval), and round(‫ )ڄ‬rounds the value to the nearest
integer.
We now consider the special case of high-rate uniform scalar quantization.

In this scenario, we assume that ܷ is in a finite interval [‫ݑ‬୫୧୬ , ‫ݑ‬୫ୟ୶ ].
Consider using ‫ ܯ‬quantization regions of equal lengths, i.e., uniform
quantization. In addition, assume that M is large, i.e., high-rate quantization.
Let ο denote the length of each quantization region. Note that ο=
[‫ݑ‬୫୧୬ , ‫ݑ‬୫ୟ୶ ]/M. When ‫ ܯ‬is sufficiently large (and hence small ο), we can
approximate the PDF ݂௎ (‫ )ݑ‬as being constant in each quantization region.
More specifically,
௣೘
݂௎ (‫ )ݑ‬ൎ , ‫ܴ א ݑ‬௠ . (11)
ο
Under this approximation, the quantization point in each region is the

midpoint of the region. From (11), the corresponding MSE can be expressed as
ெ ெ
ο/ଶ
‫݌‬௠ ‫݌‬௠
‫ ܧܵܯ‬ൎ ෍ න (‫ ݑ‬െ ‫ݍ‬௠ )ଶ ݀‫ = ݑ‬෍ ቆන ‫ ݓ‬ଶ ݀‫ݓ‬ቇ ,
ο ோ೘ ο ିο/ଶ
௠ୀଵ ௠ୀଵ
௣೘ οయ οమ
= σெ
௠ୀଵ ቀଵଶቁ = ଵଶ, (12)
ο
3 To see why, we can write ‫ ܺ([ܧ‬െ ܽ)ଶ ] = ‫ ܺ[ܧ‬ଶ ] െ 2ܽ‫ ]ܺ[ܧ‬+ ܽଶ . Differentiating
the expression with respect to ܽ and setting the result to zero, we can solve for the
optimal value of ܽ. And the result is equal to the mean ‫]ܺ[ܧ‬.
26 Chapter II
ο/ଶ
where we use the fact that ‫׬‬ோ (‫ ݑ‬െ ‫ݍ‬௠ )ଶ ݀‫ି׬ = ݑ‬ο/ଶ ‫ ݓ‬ଶ ݀‫ ݓ‬for each length-
೘
ο quantization region with the quantization point in the middle. Therefore,
the approximate MSE does not depend on the form of ݂௎ (‫ )ݑ‬for a high-rate
uniform quantizer.
Alternatively, non-uniform quantization techniques, such as adaptive and

logarithmic quantization, employ ingenious strategies to allocate additional
quantization levels to regions of greater significance, thus affording
heightened resolution where it proves most consequential.
Adaptive Quantization: Adaptive quantization adjusts the quantization

step size according to the characteristics of the signal. It takes into account
the local amplitude variations, allocating more quantization levels to regions
with greater amplitude variations and fewer levels to regions with smaller
variations. This technique allows for a better representation of signals with
varying dynamics, allocating more bits to preserve fine details in high-
amplitude regions and fewer bits in low-amplitude regions. Adaptive
quantization can be implemented using techniques such as delta modulation,
where the quantization step size is dynamically adjusted based on the
differences between successive samples.
Logarithmic Quantization: Logarithmic quantization employs a

logarithmic transformation to allocate quantization levels non-uniformly.
This technique aims to provide improved resolution for lower-amplitude
signals while allowing coarser quantization for higher-amplitude signals.
The logarithmic mapping can be based on a specific logarithmic function,
such as a logarithmic companding law, which compresses the amplitude
range of the signal before quantization and expands it back afterward.
Logarithmic quantization is commonly used in applications where
preserving fine details in low-amplitude regions is crucial, such as audio or
image compression.
These non-uniform quantization techniques offer alternatives to uniform

quantization, providing greater flexibility in allocating quantization levels
based on the characteristics and requirements of the signals being processed.
By adapting the quantization process to the signal properties, these
techniques can achieve improved fidelity and better preservation of signal
details in specific regions.
Within the realm of digital signal processing, sampling and quantization

stand as formidable pillars, fostering the seamless transition from analog to
digital domains. Through meticulous consideration of sampling rates and
the judicious application of quantization techniques, we unlock the inherent

potential to faithfully capture and manipulate the symphony of signals,
propelling ourselves towards the virtuosity that defines our digital age.

The sampling and quantization section is summarized as follow.
In the realm of signals, sampling is the key, capturing snapshots, freezing

time, you see.
Nyquist-Shannon guides, fidelity it ensures. Sample twice the frequency,

restoration endures.
Beware of aliasing, the lurking distortion. Anti-aliasing filters, our trusted

protection.
Quantization steps in, amplitudes discretized. Uniform or non-uniform,

options realized.
Adaptive quantization, step size it adapts. Fine details preserved, where the
signal maps.
Logarithmic quantization, a logarithmic twist. Precision in low amplitudes,

a quantizer is gist.
Sampling and quantization, a symphony they create, bridging the analog

and digital state.
Capture and manipulate, with care and precision. In the realm of signals, the
ultimate mission.
Key takeaways from the sampling and quantization section are given as
follows:
1. Sampling and quantization are fundamental processes in digital

signal processing that bridge the gap between the continuous
analog domain and the discrete world of digital signals.
2. Sampling involves capturing intermittent snapshots of analog
waveforms, frozen in time as discrete samples, which serve as the
building blocks for digital representation.
3. The sampling frequency, determined by the Nyquist-Shannon
sampling theorem, plays a crucial role in maintaining fidelity
during signal reconstruction. Sampling at a rate at least twice the
28 Chapter II
frequency of the highest component ensures accurate restoration of

the signal.
4. Aliasing is a potential challenge in sampling caused by insufficient
sampling rates, leading to signal distortion. Anti-aliasing filters
help mitigate aliasing by removing unwanted frequencies before
sampling.
5. Quantization transforms the continuous amplitude of analog
signals into a finite set of discrete values, enabling digital
manipulation. Uniform quantization partitions the amplitude range
into equal intervals, while non-uniform quantization techniques,
such as adaptive and logarithmic quantization, allocate quantization
levels based on signal characteristics.
6. Adaptive quantization adjusts the step size dynamically based on
the local amplitude variations, offering enhanced resolution in
regions with greater variations and fewer levels in regions with
smaller variations.
7. Logarithmic quantization applies a logarithmic transformation to
allocate quantization levels non-uniformly, preserving fine details
in lower-amplitude regions while allowing coarser quantization in
higher-amplitude regions.
8. By carefully considering sampling rates and applying appropriate
quantization techniques, the symphony of signals can be faithfully
captured and manipulated in the digital domain.
Layman’s guide:
Imagine you have a signal, which could be anything from a sound wave to
an image. Sampling is the technique of capturing snapshots of that signal at
specific points in time. It is like taking freeze-frame pictures that allow us
to work with the signal in a digital form.
When sampling, there is an important rule called the Nyquist-Shannon

theorem. It tells us that to faithfully capture the signal, we need to sample it
at least twice as fast as the highest frequency it contains. This ensures that
we do not lose any important information and allows for accurate restoration
later.
However, there is a problem called aliasing that we need to be cautious

about. Aliasing can cause distortions in the sampled signal. To prevent this,
we use anti-aliasing filters, which act as our trusted protectors. These filters
remove unwanted high-frequency components before sampling, ensuring a
clean and accurate representation of the signal.
Once we have the samples, quantization comes into play. Quantization is

the process of discretizing the amplitudes of the samples. It is like assigning
values to each snapshot. We have options here: we can use uniform
quantization, where the steps between values are equal, or non-uniform
quantization, where the steps can vary based on the signal's characteristics.
Adaptive quantization is an interesting approach. It adapts the step size

based on the details of the signal. It helps preserve fine details, capturing
them with precision in areas where the signal carries important information.
Another technique called logarithmic quantization adds a twist. It focuses

on preserving precision in low amplitudes, where our human perception is
more sensitive. This logarithmic approach ensures that even small variations
in low amplitudes are accurately represented.
When we combine sampling and quantization, they create a symphony,

bridging the analog and digital world. They allow us to capture signals and
manipulate them with care and precision. It is like an ultimate mission to
bring signals into the digital realm, where we can analyze, process, and work
with them effectively.
So, remember, sampling freezes time and captures snapshots of signals,

while quantization discretizes the amplitudes of those snapshots. Together,
they enable us to capture, manipulate, and work with signals in the digital
domain, ensuring accuracy and fidelity throughout the process.
Exercises of sampling and quantization

Problem 1: What is the importance of sampling frequency in the process of
signal sampling?
Solution: The sampling frequency plays a crucial role in signal sampling as

it determines the fidelity of the resulting digital representation. According
to the Nyquist-Shannon sampling theorem, to accurately restore a signal, it
must be sampled at a rate at least twice the frequency of its highest
component. Sampling at a lower frequency can lead to aliasing and
distortion in the reconstructed signal. Therefore, selecting an appropriate
sampling frequency is essential for preserving the details and accuracy of
the original analog signal.
30 Chapter II
MATLAB example:
% Generate a continuous analog signal

t = 0:0.001:1; % Time vector
analog_signal = sin(2*pi*10*t) + 0.5*sin(2*pi*20*t);
% Sample the analog signal at different frequencies

sampling_freq_1 = 40; % Sampling frequency lower than Nyquist rate
sampling_freq_2 = 100; % Sampling frequency higher than Nyquist rate
% Perform signal reconstruction using low and high sampling frequencies

reconstructed_signal_1 = interp1(t, analog_signal, 0:1/sampling_freq_1:1);
reconstructed_signal_2 = interp1(t, analog_signal, 0:1/sampling_freq_2:1);
% Plot the original and reconstructed signals

figure;
subplot(2,1,1);
plot(t, analog_signal, '-b', 'LineWidth', 1.5);
title('Original Analog Signal');
xlabel('Time');
ylabel('Amplitude');
subplot(2,1,2);
hold on;
plot(0:1/sampling_freq_1:1, reconstructed_signal_1, 'ro-', 'LineWidth',
1.5);
plot(0:1/sampling_freq_2:1, reconstructed_signal_2, 'gs--', 'LineWidth',
1.5);
legend('Sampling Freq = 40', 'Sampling Freq = 100');
title('Reconstructed Signals');
xlabel('Time');
Figure 2-5 illustrates the results, showcasing both the original analog signal
and the corresponding reconstructed signals.
Figure 2-5: MATLAB example of the importance of sampling frequency.
Problem 2: What are the trade-offs between uniform and non-uniform

quantization techniques?
Solution: Uniform quantization partitions the amplitude range into equal

intervals, providing a straightforward and simple approach. However, it
may result in quantization errors and loss of fine details, especially in
regions with low amplitudes. On the other hand, non-uniform quantization
techniques, such as adaptive and logarithmic quantization, offer the
advantage of allocating more quantization levels to regions of greater
significance. This allows for better resolution in important areas while using
fewer levels in regions with less significance. The trade-off is that non-
uniform quantization techniques can be more complex to implement and
require additional processing. Choosing between uniform and non-uniform
quantization depends on the specific requirements of the application and the
desired balance between simplicity and accuracy.
32 Chapter II
MATLAB example:
% Generate a continuous analog signal

t = 0:0.001:1; % Time vector
analog_signal = sin(2*pi*10*t) + 0.5*sin(2*pi*20*t);
% Perform uniform quantization

num_levels = 8; % Number of quantization levels
uniform_quantized_signal = round(analog_signal * num_levels) /
num_levels;
% Perform non-uniform quantization (logarithmic quantization)

log_quantized_signal = round(log(1 + abs(analog_signal)) * num_levels) /
num_levels;
log_quantized_signal = log_quantized_signal .* sign(analog_signal);
% Plot the original and quantized signals

figure;
subplot(3,1,1);
plot(t, analog_signal, 'b-', 'LineWidth', 0.3);
title('Original Analog Signal');
xlabel('Time');
subplot(3,1,2);
stem(t, uniform_quantized_signal, 'r:', 'LineWidth', 0.3);
title('Uniform Quantization');
xlabel('Time');
subplot(3,1,3);
stem(t, log_quantized_signal, 'g--', 'LineWidth', 0.3);
title('Logarithmic Quantization');
xlabel('Time');
Figure 2-6 visually presents the outcome, depicting the trade-offs observed
when comparing uniform and non-uniform quantization techniques.
Figure 2-6: MATLAB example of the trade-offs between uniform and non-uniform
quantization techniques.
Signal filtering and convolution

This chapter delves into the realm of signal filtering techniques, specifically
focusing on the fundamental concepts and applications of low-pass, high-
pass, band-pass, and band-stop filters. These techniques play a pivotal role
in signal processing by enabling the manipulation and extraction of desired
frequency components from signals. Moreover, this chapter delves into the
foundational principles and techniques of convolution, a critical operation
employed extensively in the field of signal processing.
The section commences by elucidating the theoretical foundations of signal

filtering. It explores the underlying principles and design methodologies of
various types of filters, such as low-pass filters that allow signals with
frequencies below a specific cutoff to pass through, high-pass filters that
permit signals with frequencies above a designated threshold, band-pass
filters that facilitate the transmission of signals within a specific frequency
range, and band-stop filters that effectively suppress signals within a
defined frequency band. Detailed explanations are provided for the design
parameters, including filter order, cutoff frequencies, and attenuation
34 Chapter II
characteristics, ensuring a comprehensive understanding of their practical

implementation.
Additionally, the chapter presents practical examples and real-world

applications of signal filtering techniques. These applications span across
diverse domains, encompassing audio processing, image enhancement,
biomedical signal analysis, telecommunications, and many more. Detailed
case studies and illustrative examples elucidate the efficacy of each filtering
technique, showcasing their ability to enhance signal quality, eliminate
noise, and extract relevant information from complex waveforms.
Furthermore, convolution, an integral operation in signal processing, is

meticulously explored in this chapter. Convolution is an indispensable
technique for manipulating and analyzing signals, as it facilitates the
extraction of meaningful information by convolving signals with suitable
filters or impulse response functions. The theoretical foundations of
convolution, including the convolution theorem and its applications in linear
time-invariant systems, are elucidated in a concise yet rigorous manner.
In summary, this chapter provides a comprehensive overview of signal

filtering techniques, encompassing various filter types and their practical
applications. Furthermore, it delves into the theoretical underpinnings and
practical implications of convolution, underscoring its significance in the
realm of signal processing. Through detailed explanations, practical
examples, and insightful case studies, readers will acquire a profound
understanding of these fundamental concepts, enabling them to apply signal
filtering and convolution techniques effectively in their respective domains
of research and application.
Signal filtering
The theoretical foundations of signal filtering are explained, focusing on

different types of filters. Let us delve into some of the key concepts:
x Low-pass filters allow signals with frequencies below a specific

cutoff to pass through. They are designed to attenuate or remove
high-frequency components. A common example of a low-pass
filter is the ideal low-pass filter, which is defined as
1, if |߱| ൑ ߱௖ ,
‫ = )߱(ܪ‬൜ (13)
0, if |߱| > ߱௖ .
Here, ‫ )߱(ܪ‬represents the frequency response of the filter, and ߱

is the angular frequency. ߱௖ is the cutoff frequency, determining
the point beyond which the filter attenuates the signal.
x High-pass filters allow signals with frequencies above a

designated threshold to pass through, while attenuating low-
frequency components. An ideal high-pass filter can be
represented by
1, if |߱| ൒ ߱௖ ,
‫ = )߱(ܪ‬൜ (14)
0, if |߱| < ߱௖ .
Similar to the low-pass filter, ‫ )߱(ܪ‬represents the frequency

response, ߱௖ is the cutoff frequency, and ߱ is the angular
frequency.
x Band-pass filters facilitate the transmission of signals within a

specific frequency range. They selectively pass frequencies within
the desired band, while attenuating those outside the band. The
frequency response of a band-pass filter can be represented by
1, if ߱୪୭୵ ൑ |߱| ൑ ߱୦୧୥୦ ,
‫ = )߱(ܪ‬൜ (15)
0, otherwise.
Here, ߱୪୭୵ and ߱୦୧୥୦ represent the lower and upper cutoff
frequencies, respectively.
x Band-stop filters effectively suppress signals within a defined

frequency band while allowing others to pass. They are commonly
used to eliminate noise or unwanted interference. The frequency
response of a band-stop filter can be represented by
0, if ߱୪୭୵ ൑ |߱| ൑ ߱୦୧୥୦ ,
‫ = )߱(ܪ‬൜ (16)
1, otherwise.
Here, ߱୪୭୵ and ߱୦୧୥୦ represent the lower and upper cutoff
frequencies, respectively.
Detailed explanations are provided for important design parameters such as

filter order, cutoff frequencies, and attenuation characteristics, ensuring a
comprehensive understanding of their practical implementation. When
designing filters, several important parameters need to be considered. Here
are explanations of some key design parameters:
36 Chapter II
x Filter Order: The filter order determines the complexity and

performance of the filter. A higher order generally allows for
sharper frequency response characteristics but comes at the cost of
increased computational complexity. The filter order specifies the
number of poles or zeros in the filter transfer function, which
affects the sharpness of the filter's cutoffs and roll-off.
x Cutoff Frequencies: The cutoff frequencies determine the range
of frequencies that the filter allows to pass or attenuates. For
example, in a low-pass filter, the cutoff frequency represents the
point at which the filter starts attenuating high-frequency
components. In a high-pass filter, the cutoff frequency represents
the point at which the filter begins to attenuate low-frequency
components. The selection of cutoff frequencies depends on the
specific application and desired frequency range.
x Attenuation Characteristics: Attenuation refers to the reduction
in amplitude or energy of certain frequency components by the
filter. Different filter designs have different attenuation
characteristics. For instance, a steep roll-off filter exhibits rapid
attenuation beyond the cutoff frequency, while a gradual roll-off
filter has a gentler attenuation slope. The attenuation characteristics
determine the filter's ability to suppress unwanted frequencies and
preserve the desired signal components.
To achieve specific filter characteristics, various design methodologies can

be employed, such as Butterworth, Chebyshev, or elliptic filter designs.
These methodologies offer trade-offs between factors like filter steepness,
passband ripple, stopband attenuation, and phase response.
By understanding these design parameters and methodologies, engineers

can make informed decisions when implementing signal filtering
techniques. They can tailor the filter's behavior according to the specific
requirements of the application, ensuring that the filtered signal meets the
desired specifications in terms of frequency response, noise suppression,
and signal enhancement.
Convolution
Convolution is an essential operation in the field of signal processing. It is

a technique used to manipulate and analyze signals, enabling the extraction
of valuable information by convolving signals with appropriate filters or
impulse response functions.
Convolution involves combining two signals to produce a third signal. The

first signal is the input signal, which represents the data or information we
want to analyze. The second signal is typically a filter or an impulse
response function that defines how the input signal should be modified or
processed. The convolution theorem, which is a fundamental result in signal
processing, states that the Fourier transform of the convolution of two
signals is equal to the pointwise multiplication of their individual Fourier
transforms. The convolution theorem is a powerful tool that allows us to
analyze convolutions in the frequency domain, where they often become
simpler and more manageable. By taking advantage of the properties of the
Fourier transform, we can gain insights into the behavior of convolutions
and design more efficient signal processing algorithms.
The section also discusses the applications of convolution in linear time-

invariant (LTI) systems. LTI systems are widely used in signal processing
and communication engineering. They exhibit certain properties that make
them amenable to analysis using convolution. We also discuss how the
convolution theorem can be applied to understand and characterize LTI
systems, enabling us to predict their behavior and design suitable filters to
achieve desired signal processing objectives.
As for the equations explaining convolution, one of the key equations is the
mathematical expression of convolution:
ஶ
‫ି׬ = )ݐ(݄ כ )ݐ(ݔ = )ݐ(ݕ‬ஶ ‫ ݐ(݄ )߬(ݔ‬െ ߬)݀߬, (17)
where ‫ )ݐ(ݔ‬represents the input signal, ݄(‫ )ݐ‬represents the impulse response
function or filter, and ‫ )ݐ(ݕ‬represents the output signal resulting from the
convolution of ‫ )ݐ(ݔ‬and ݄(‫)ݐ‬. It is computed according to the following
steps:
1. Replace ‫ ݐ‬by ߬ to get ‫ )߬(ݔ‬and ݄(߬).

2. Reflect ݄(߬) around the origin to get ݄(െ߬).
3. For ‫ ݐ‬൒ 0, shift ݄(െ߬) to the right by ‫ ݐ‬to form ݄(‫ ݐ‬െ ߬). For ‫< ݐ‬
0, shift ݄(െ߬) to the left by െ ‫ ݐ‬to form ݄(‫ ݐ‬െ ߬).
4. Multiply ‫ )߬(ݔ‬and ݄(‫ ݐ‬െ ߬) and take the area under the resulting
curve as the value of the convolution integral at time ‫ݐ‬.
5. The complete result is obtained by repeating Steps 3 and 4 for all
‫ א ݐ‬R. As ‫ ݐ‬increases, ݄(‫ ݐ‬െ ߬) slides from left to right with respect
to ‫)߬(ݔ‬.
38 Chapter II
The convolution theorem can be expressed mathematically in frequency

domain as
ܻ(݂) = ܺ(݂)‫)݂(ܪ‬, (18)
where ܺ(݂) and ‫ )݂(ܪ‬represent the Fourier transforms of the input signal
and the impulse response function/filter, respectively, while ܻ(݂) represents
the Fourier transform of the output signal. These equations form the basis
for understanding and utilizing convolution in signal processing and are
essential tools in analyzing and designing signal processing systems.
Convolution has a wide range of applications beyond signal processing and

communication engineering. Here are a few additional areas where
convolution plays a crucial role:
1. Image Processing: Convolution is extensively used in image processing

tasks such as image filtering, edge detection, image enhancement, and
image recognition. By convolving images with appropriate filters or
kernels, we can extract features, remove noise, and perform various
transformations on images.
2. Computer Vision: In computer vision applications, convolutional
neural networks (CNNs) are commonly used. CNNs employ
convolutional layers that apply convolutions to input images, enabling
the network to automatically learn and extract relevant features for
tasks such as object recognition, object detection, and image
segmentation.
3. Natural Language Processing (NLP): In NLP, convolutional neural
networks and convolutional operations are utilized for tasks like text
classification, sentiment analysis, and language modeling.
Convolutional filters are applied to sequences of words or characters to
capture local patterns and relationships within the text.
4. Medical Imaging: Convolution is extensively used in medical imaging
techniques such as computed tomography (CT), magnetic resonance
imaging (MRI), and ultrasound imaging. It helps in tasks like noise
reduction, image reconstruction, feature extraction, and medical image
analysis.
5. Audio Processing: Convolution is employed in audio processing
applications such as audio filtering, room impulse response modeling,
and audio effects. It allows for tasks like noise cancellation, reverb
simulation, and audio equalization.
6. Radar and Sonar Systems: Convolution is used in radar and sonar
systems for target detection, range estimation, and signal processing. It
enables the analysis of echo signals and the extraction of meaningful

information from the received signals.
7. Financial Analysis: Convolution is applied in financial analysis to
perform tasks like time series analysis, pattern recognition, and
algorithmic trading. By convolving financial data with suitable filters,
meaningful patterns and trends can be identified for decision-making.
8. Biomedical Signal Processing: Convolution plays a vital role in
processing biomedical signals, such as electrocardiograms (ECGs) and
electroencephalograms (EEGs). It helps in filtering noise, identifying
abnormalities, and extracting relevant features for medical diagnosis
and monitoring.
These are just a few examples of the diverse range of applications where
convolution is utilized. The flexibility and effectiveness of convolution
make it a powerful tool in various fields that deal with data analysis, pattern
recognition, and information extraction.
Convolution in 2-D extends the concept of convolution from 1-D signals to

2-D signals, such as images. It involves the operation of combining two 2-
D signals to produce a third 2-D signal. Here is an explanation of
convolution in 2-D.
Let us consider two 2-D signals: the input signal, often referred to as the
image, denoted by ‫ݔ(ܫ‬, ‫ )ݕ‬and a 2-D filter, denoted by ‫ݑ(ܭ‬, ‫ )ݒ‬where ‫ݔ‬, ‫ݕ‬,
‫ݑ‬, and ‫ ݒ‬represent the spatial coordinates. The 2-D convolution operation is
defined as
‫ܫ‬Ԣ(‫ݔ‬, ‫ = )ݕ‬σ௨ σ௩ ‫ ݔ(ܫ‬െ ‫ݑ‬, ‫ ݕ‬െ ‫ݑ(ܭ)ݒ‬, ‫)ݒ‬. (19)
In this equation, ‫ܫ‬Ԣ(‫ݔ‬, ‫ )ݕ‬represents the output or convolved signal at

coordinates (‫ݔ‬, ‫)ݕ‬. The summations are taken over the entire spatial range
of the filter, typically centered around the origin.
To compute the output value at each coordinate (‫ݔ‬, ‫)ݕ‬, the filter ‫ݑ(ܭ‬, ‫ )ݒ‬is
placed on top of the input signal ‫ ݔ(ܫ‬െ ‫ݑ‬, ‫ ݕ‬െ ‫)ݒ‬, and element-wise
multiplication is performed. The results are then summed over all spatial
locations defined by the filter. This process is repeated for all coordinates
(‫ݔ‬, ‫)ݕ‬of the output signal ‫ܫ‬ᇱ (‫ݔ‬, ‫)ݕ‬.
Convolution in 2-D is often used in image processing tasks, such as filtering,

feature extraction, and image analysis. By convolving an image with
different filters, various operations can be performed, such as blurring,
sharpening, edge detection, and texture analysis. The choice of the filter
40 Chapter II
determines the specific operation and the information extracted from the
image.
Additionally, similar to 1-D convolution, 2-D convolution also has its own
convolution theorem. The 2-D convolution theorem states that the Fourier
transform of the 2-D convolution of two signals is equal to the pointwise
multiplication of their individual Fourier transforms.
‫ܫ‬ᇱ (‫ݔ‬, ‫ି ࣠ = )ݕ‬ଵ ൣ࣠[‫ݔ(ܫ‬, ‫])ݕ‬൧࣠ [‫ݔ(ܭ‬, ‫])ݕ‬. (20)
Here, ࣠ represents the Fourier transform, ࣠ ିଵ represents the inverse

Fourier transform, and ‫ܫ‬ᇱ (‫ݑ‬, ‫ )ݒ‬represents the Fourier transformed output
signal.
The convolution theorem in 2-D provides a powerful tool for analyzing

convolutions in the frequency domain, allowing for efficient filtering,
deconvolution, and other image processing operations. It is considered as a
fundamental operation in image processing, computer vision, and other
areas that deal with 2-D signals, enabling the extraction of meaningful
information, feature detection, and enhancement in various applications
involving images or spatial data.

The signal filtering and convolution section is summarized as follow.
In signal filtering, we find delight. Manipulating signals, extracting what is

right.
Low-pass, high-pass, band-pass, and band-stop. Filters of different types,

each with a unique crop.
Design parameters, we thoroughly explore. Filter order, cutoff frequencies,

and more.
Understanding these, implementation becomes clear. Enhancing signals

with fidelity, that is our sincere.
Real-world applications, we do embrace. Audio, images, and biomedical

space.
From telecommunications to diverse domains. Filtering techniques, their

usefulness sustains.
Convolution, a technique integral and strong. Extracting meaningful

information, never wrong.
With suitable filters or impulse responses, we blend. Unraveling complex

waveforms, a trend.
Theoretical foundations, we unravel.
Convolution theorem, linear time-invariant marvel.
Exploring its implications, with precision.
Signal processing's key, a necessary decision.
In summary, this chapter takes us deep. Signal filtering and convolution,

secrets we keep.
From filters' design to practical use. Understanding these concepts, we will

never lose.
Key takeaways from the signal filtering and convolution are given as
follows:
1. Signal filtering techniques, such as low-pass, high-pass, band-pass,

and band-stop filters, play a crucial role in manipulating and
extracting desired frequency components from signals.
2. Design parameters, including filter order, cutoff frequencies, and
attenuation characteristics, are essential considerations when
implementing filters, ensuring effective signal processing.
3. Practical examples and real-world applications illustrate the
versatility of signal filtering techniques across various domains,
including audio processing, image enhancement, biomedical
signal analysis, and telecommunications.
4. Convolution, a fundamental operation in signal processing,
enables the extraction of meaningful information by convolving
signals with suitable filters or impulse response functions.
5. Theoretical foundations of convolution, including the convolution
theorem and its applications in linear time-invariant systems, are
explained concisely yet rigorously.
6. By gaining a comprehensive understanding of signal filtering
techniques and convolution, readers can apply these concepts
effectively in their respective research and applications, enhancing
42 Chapter II
signal quality, eliminating noise, and extracting valuable

information.
Layman’s guide:
Signal filtering is like sorting through a signal to extract specific parts and
manipulate them. Imagine you have a bowl of soup with different
ingredients, and you want to separate out the vegetables from the broth. That
is similar to what signal filtering does with signals.
There are different types of filters that perform different tasks. Think of
them as sieves with different-sized holes. A low-pass filter allows low-
frequency components of a signal to pass through, while blocking higher
frequencies. A high-pass filter does the opposite, letting high frequencies
through and blocking low frequencies. Band-pass filters only allow a
specific range of frequencies to pass, and band-stop filters block a specific
range of frequencies.
When designing a filter, there are important considerations. The filter order
determines how complex the filter is and how well it can do its job. Cutoff
frequencies define the range of frequencies you want to allow or block.
Attenuation characteristics determine how much the filter reduces certain
frequencies.
Filters find practical use in various applications. For example, in audio

processing, filters can remove background noise to make the sound clearer.
In image enhancement, filters can reduce random noise and make images
look sharper.
Convolution is another important concept in signal processing. It involves

combining two signals together to create a new one. It is like mixing
ingredients in cooking to create a delicious dish. Convolution helps extract
meaningful information from signals and analyze them effectively.
Understanding the theory behind filters and convolution gives us a valuable

toolkit to improve signals in different fields. We can enhance signals,
remove unwanted noise, and extract valuable information from complex
waveforms.
By learning about signal filtering and convolution, we gain the ability to

manipulate and enhance signals, bridging the gap between analog and
digital worlds. It is like having a set of tools to make signals clearer, more
informative, and better suited for our needs.
Exercises of signal filtering and convolution

Problem 1:
Suppose you have recorded an audio signal that contains both the desired
speech and background noise. You want to enhance the speech signal by
removing the noise using a suitable filtering technique. Design a low-pass
filter to achieve this objective.
Solution:
To solve this problem, you can follow these steps:
1. Analyze the audio signal to determine the frequency range that

contains the speech signal and the frequency range of the
background noise.
2. Choose an appropriate cutoff frequency for the low-pass filter. The
cutoff frequency should be set below the frequency range of the
speech signal and above the frequency range of the background
noise. This ensures that the filter attenuates the noise while
preserving the speech components.
3. Select a filter design method such as Butterworth, Chebyshev, or
elliptic filter design based on your requirements. Consider factors
such as filter order, passband ripple, and stopband attenuation.
4. Design the low-pass filter with the chosen specifications, including
the cutoff frequency and filter order.
5. Apply the designed filter to the recorded audio signal to remove
the background noise. This can be done by convolving the filter's
impulse response with the audio signal using techniques like the
fast convolution algorithm.
6. Evaluate the filtered audio signal to ensure that the speech
components are enhanced and the background noise is effectively
suppressed.
MATLAB example:
% Problem 1: Audio Signal Filtering
% Load the recorded audio signal (assuming it is stored in a variable called

'audioSignal')
% Specify the sampling frequency (Fs) and the desired cutoff frequency
(Fc)
Fs = 44100; % Example sampling frequency (change accordingly)
44 Chapter II
Fc = 4000; % Example cutoff frequency (change accordingly)
% Design a Butterworth low-pass filter

filterOrder = 4; % Example filter order (change accordingly)
[b, a] = butter(filterOrder, Fc/(Fs/2), 'low');
% Apply the filter to the audio signal

filteredSignal = filtfilt(b, a, audioSignal);
% Play the original audio signal and the filtered audio signal to compare
sound(audioSignal, Fs); % Play the original audio signal
pause; % Pause before playing the filtered audio signal
sound(filteredSignal, Fs); % Play the filtered audio signal
Problem 2:
You have an image that has been corrupted by random noise, which is
affecting the image quality. Design a filter to remove the noise and enhance
the image details.
Solution:
Here is a step-by-step solution to address this problem:
1. Analyze the image and understand the characteristics of the noise.

Determine the frequency range or spatial characteristics of the
noise in the image.
2. Choose an appropriate filter type based on the noise characteristics.
For instance, if the noise is high-frequency, consider using a low-
pass filter. If the noise has a specific spatial pattern, a spatial filter
like a median filter or a Gaussian filter may be suitable.
3. Determine the filter parameters such as the filter size, cutoff
frequency, or kernel size based on the analysis of the noise and the
desired image enhancement.
4. Design and apply the chosen filter to the image. Convolve the filter
with the image using techniques like 2-D convolution to remove
the noise.
5. Evaluate the filtered image to assess the improvement in image
quality. Pay attention to factors such as noise reduction,
preservation of image details, and overall enhancement.
6. Fine-tune the filter parameters if necessary and iterate the process
to achieve the desired image quality.
Remember, the specific filter design and implementation steps may vary
depending on the nature of the noise and the image. It is important to analyze
the problem carefully and choose the appropriate filtering technique
accordingly.
MATLAB example:
% Problem 2: Image Filtering
% Load the corrupted image (assuming it is stored in a variable called

'corruptedImage')
% Apply a median filter to remove random noise

filterSize = 3; % Example filter size (change accordingly)
denoisedImage = medfilt2(corruptedImage, [filterSize, filterSize]);
% Display the original image and the denoised image to compare

figure;
subplot(1, 2, 1);
imshow(corruptedImage);
title('Original Image');
subplot(1, 2, 2);
imshow(denoisedImage);
title('Denoised Image');
Please note that the provided MATLAB codes are simplified examples and
may need adjustments based on your specific requirements, input data, and
desired outcomes. Make sure to adapt the code accordingly and incorporate
any additional preprocessing or post-processing steps as needed.
Problem 3:
Consider the following continuous input signal:
‫(݌ݔ݁ = )ݐ(ݔ‬െ‫)ݐ(ݑ)ݐ‬.
where ‫ )ݐ(ݑ‬is the unit step function, and the following continuous impulse
response function: ݄(‫(݌ݔ݁ = )ݐ‬2‫)ݐ(ݑ)ݐ‬.
Perform the continuous convolution of the input signal with the impulse
response function using MATLAB.
46 Chapter II
Solution:
To solve this problem, we can use the integral representation of the

convolution operation. The convolution of two continuous signals ‫ )ݐ(ݔ‬and
݄(‫ )ݐ‬is given by
ஶ
‫ = )ݐ(ݕ‬න ‫ ݐ(݄)߬(ݔ‬െ ߬)݀߬.
ିஶ
Substituting ‫ = )ݐ(ݔ‬exp(െ‫ )ݐ(ݑ)ݐ‬and ݄(‫ = )ݐ‬exp(2‫ )ݐ(ݑ)ݐ‬we can write

the convolution integral as
௧
‫ = )ݐ(ݕ‬න exp(െ߬)exp(2(‫ ݐ‬െ ߬))݀߬.
଴
For ‫ ݐ‬൑ 0, both ‫ )߬(ݔ‬and ݄(‫ ݐ‬െ ߬) are zero. Therefore, the result of the
integral becomes zero. For ‫ > ݐ‬0, both ‫ )߬(ݔ‬and ݄(‫ ݐ‬െ ߬) are non-zero, we
continue evaluating the integral as
௧
‫ = )ݐ(ݕ‬න exp(2‫ ݐ‬െ ߬)݀߬,
଴
= െexp[2‫ ݐ‬െ ߬]௧଴ ,
= െexp[2‫ ݐ‬െ ‫ ]ݐ‬+ exp[2‫]ݐ‬,
= െexp[‫ ]ݐ‬+ exp[2‫]ݐ‬.
Therefore, the output signal resulting from the continuous convolution of

the given input signal and impulse response function is expressed as
െexp[‫ ]ݐ‬+ exp[2‫]ݐ‬.
We can use the conv function in MATLAB along with the symbolic math
toolbox for continuous convolution. Here is the MATLAB code:
% Define the input signal

x = @(t) exp(-t).*(t>=0);
% Define the impulse response function
h = @(t) exp(2*t).*(t>=0);
% Set the range for the plot

t = -5:0.01:5;
% Compute the input signal

input = x(t);
% Compute the impulse response function

impulse = h(t);
% Compute the output signal

y = zeros(size(t));
for i = 1:length(t)
y(i) = integral(@(s) x(s).*h(t(i)-s), 0, t(i));
end
% Plot the input signal

subplot(3,1,1);
plot(t, input, 'LineWidth', 2);
xlabel('t');
ylabel('x(t)');
title('Input Signal');
grid on;
% Plot the impulse response function

subplot(3,1,2);
plot(t, impulse, 'LineWidth', 2);
xlabel('t');
ylabel('h(t)');
title('Impulse Response Function');
grid on;
% Plot the output signal

subplot(3,1,3);
plot(t, y, 'LineWidth', 2);
xlabel('t');
ylabel('y(t)');
title('Output Signal');
grid on;
% Adjust the spacing between subplots
sgtitle('Convolution Results');
Figure 2-7 presents the visual representation of the result, showcasing the
output obtained from the convolution between the input signal and the
impulse response function.
48 Chapter II
Figure 2-7: MATLAB example of the convolution.
Time and frequency domain representations

This section discusses the relationship between the time and frequency
domains of signals. It also covers different time and frequency domain
representations such as the time-frequency distribution and spectrogram.
The relationship between the time and frequency domains of signals is a
fundamental concept in signal processing. By analysing a signal in the time
domain, we can observe its behavior over time. Conversely, examining the
signal in the frequency domain allows us to identify the different frequency
components present within it. This section delves into various representations,
such as time-frequency distributions and spectrograms, which provide
insights into the signal's time and frequency characteristics.
In signal processing, we often encounter signals that vary both in amplitude

and frequency over time. Analyzing these signals simultaneously in the time
and frequency domains can provide a more comprehensive understanding
of their behavior. The time domain representation shows how the signal
changes with time, while the frequency domain representation reveals the
underlying frequency components.
Let us consider a continuous-time signal, denoted by ‫)ݐ(ݔ‬, where

‫ ݐ‬represents time. The time-domain representation of ‫ )ݐ(ݔ‬displays the
amplitude of the signal as a function of time. This can be mathematically
expressed as
‫)ݐ(ܣ=)ݐ(ݔ‬cos(2ߨ݂(‫ݐ)ݐ‬+ ߮(‫))ݐ‬, (21)
where ‫ )ݐ(ܣ‬represents the instantaneous amplitude of a signal, ݂(‫ )ݐ‬denotes

the instantaneous frequency, and ‫ )ݐ(ݔ‬is the instantaneous phase. By
examining the changes in ‫)ݐ(ܣ‬, ݂(‫)ݐ‬, and ߮(‫)ݐ‬, we can gain insights into
the signal’s time-varying characteristics.
To better understand the frequency components of a signal, we can analyze

it in the frequency domain. The Fourier Transform is a commonly used
mathematical tool for this purpose. The continuous-time Fourier Transform
of x(t) is given by
ஶ
ܺ(݂)=‫ି׬‬ஶ ‫)ݐ(ݔ‬exp(െ݆2ߨ݂‫ݐ݀ )ݐ‬, (22)
where ܺ(݂) represents the frequency-domain representation of the signal,

and the integral is taken over all time values. The Fourier Transform
provides information about the amplitudes and phases of different
frequency components present in the signal.
However, the Fourier Transform assumes that the signal is stationary,

meaning its properties do not change over time. For signals with time-
varying characteristics, alternative representations are required. Two such
representations are the time-frequency distribution and the spectrogram.
The time-frequency distribution provides a joint representation of the

signal's time and frequency information. It is a function that varies with both
time and frequency and is mathematically expressed as
ܹ(‫ݐ‬, ݂)=|‫ ߬(݃)߬(ݔ[׬‬െ ‫)ݐ‬exp(െ݆2ߨ݂‫|߬݀])ݐ‬ଶ , (23)
where ܹ(‫ݐ‬, ݂) represents the time-frequency distribution, and ݃(߬ െ ‫ )ݐ‬is a

window function that helps localize the analysis in time. The absolute
square of the integral is taken to obtain a power representation.
The spectrogram is a commonly used time-frequency representation that

uses a short-time Fourier transform (STFT). It breaks the signal into short
overlapping segments and computes the Fourier Transform for each
50 Chapter II
segment. The resulting spectrogram provides a visual representation of the

signal's time-varying frequency content.
By analyzing signals in both the time and frequency domains and employing
representations like the time-frequency distribution and spectrogram, we
can gain valuable insights into their time-varying characteristics and
frequency components. These tools are widely used in various fields,
including audio processing, image processing, and communications.

The time and frequency domain representation section is summarized as
follows:
In signal processing, we explore the connection. Between time and

frequency domains, a vital direction.
Time domain reveals a signal's behavior over time. While frequency domain
identifies frequencies, prime.
With a continuous-time signal, we find. Amplitude, frequency, and phase

are combined.
The Fourier Transform, a powerful mathematical tool. Uncovers frequency

components, making signals cool.
But for time-varying signals, alternative methods are applied. Time-

frequency distributions and spectrograms are tried.
ܹ(‫ݐ‬, ݂) showcases time and frequency details at once. Using a window

function to localize analysis, not a chance.
Spectrograms, using STFT, divide the signal in segments. To visualize time-

varying frequency content, a true testament.
By studying both domains, we grasp signals' full view. Gaining insights into
their characteristics, old and new.
These tools find use in audio, image, and communication. Unraveling the
complexities of signals, our dedicated mission.
Key takeaways from the time and frequency representations are given as
follows
x Time and frequency representations help us understand the

behavior of signals and identify the frequencies present in them.
x The Fourier transform is a mathematical tool that reveals the
frequency content of a signal, allowing us to analyze its musical
characteristics.
x Time-frequency representations provide a combined view of how
a signal changes over time and the frequencies it contains at each
moment.
x The spectrogram is a common time-frequency representation that
breaks down a signal into segments to analyze its frequency
content.
x Time and frequency representations find applications in various
fields such as music, communication, and image processing.
x They help us analyze and modify sounds, optimize signal
transmission, and understand visual content in images and videos
Layman’s guide:
Time and frequency representations help us understand how signals behave
and what frequencies are present in them. Imagine you have a song playing
on your music player. The time representation shows how the song changes
over time—when the beats drop, when the chorus comes in, and how the
volume changes. The frequency representation tells you what notes or
pitches are present in the song—whether it has high or low tones, and if
there are any specific musical instruments playing.
The Fourier transform is a mathematical tool used to analyze the frequency

content of a signal. Think of the Fourier transform as a special lens that can
reveal the different musical notes in a song. It takes the signal and breaks it
down into its individual frequencies, showing us which notes are playing
and how loud they are. This helps us understand the musical characteristics
of the signal.
Time-frequency representations provide a joint view of how a signal

changes over time and what frequencies are present at each moment.
Imagine you have a video clip of a concert. The time-frequency
representation is like watching the video in slow motion, showing you not
only the movements of the musicians but also the different musical elements
being played at each moment. It allows us to see how the sound evolves
over time and which frequencies dominate at different parts of the
performance.
52 Chapter II
The spectrogram is a common time-frequency representation that breaks

down a signal into short segments and analyzes the frequency content of
each segment. Think of the spectrogram as a series of snapshots of the
concert. It captures small portions of the performance and tells us which
musical notes are being played in each snapshot. By looking at the
spectrogram, we can see how the different musical elements change over
time and how they interact with each other.
Time and frequency representations are valuable in various fields like

music, communication, and image processing. These representations help
us understand and manipulate signals in different applications. In music,
they can be used to analyze and modify the sound of instruments or vocals.
In communication, they help in transmitting and receiving signals
efficiently. In image processing, they aid in understanding the visual content
of images or videos.
By using time and frequency representations, we gain insights into how

signals change over time, what frequencies they contain, and how they can
be manipulated or understood in different domains. These representations
are like special tools that help us unravel the secrets of sound and make
sense of the world of signals around us.
Exercises of time and frequency domain representations

Problem 1:
You have a recorded audio file of a musical performance, but it contains

background noise that is affecting the quality of the music. You want to
apply signal processing techniques to remove the noise and enhance the
musical elements.
Solution:
1. Load the audio file into MATLAB using the audioread function.
2. Apply a high-pass filter to remove low-frequency noise that might
be present in the recording. Use the designfilt function to design
the filter with a suitable cutoff frequency.
3. Plot the time-domain representation of the original audio signal
using the plot function to visualize how the music changes over
time.
4. Compute the Fourier Transform of the audio signal using the fft
function to analyze the frequency content. Plot the magnitude
spectrum to identify the dominant frequencies and musical notes

present in the music.
5. Create a time-frequency representation, such as a spectrogram,
using the spectrogram function. Adjust the parameters to obtain a
suitable balance between time and frequency resolution.
6. Apply a noise reduction algorithm, such as spectral subtraction or
Wiener filtering, to suppress the background noise while
preserving the musical elements. Use the time-frequency
representation to guide the noise reduction process.
7. Convert the filtered signal back to the time domain using the
inverse Fourier Transform.
8. Play the enhanced audio signal using the sound function and
compare it with the original recording.
MATLAB example:
% Load the audio file

[audio, Fs] = audioread('audio_file.wav');
% Apply a high-pass filter

cutoffFreq = 500; % Set the cutoff frequency for the high-pass filter
hpFilter = designfilt('highpassiir', 'FilterOrder', 8, 'PassbandFrequency',
cutoffFreq, 'SampleRate', Fs);
filteredAudio = filter(hpFilter, audio);
% Plot the time-domain representation

t = (0:length(audio)-1)/Fs; % Time axis
figure;
plot(t, audio);
xlabel('Time (s)');
title('Time-Domain Representation');
% Compute the Fourier Transform

N = length(audio);
freq = (-Fs/2 : Fs/N : Fs/2 - Fs/N); % Frequency axis
fftAudio = fftshift(fft(audio));
magnitudeSpectrum = abs(fftAudio);
figure;
plot(freq, magnitudeSpectrum);
xlabel('Frequency (Hz)');
ylabel('Magnitude');
54 Chapter II
title('Frequency-Domain Representation');
% Create a spectrogram
windowLength = 1024; % Set the window length for the spectrogram
overlap = windowLength/2; % Set the overlap ratio
spectrogram(audio, windowLength, overlap, [], Fs, 'yaxis');
title('Spectrogram');
% Apply noise reduction algorithm (example: spectral subtraction)

% ...
% Convert back to time domain

% ...
% Play the enhanced audio signal

% ...
The code above demonstrates the plotting of the time-domain

representation, frequency-domain representation, and spectrogram of the
audio signal. Please note that the code does not include the complete
implementation of the noise reduction algorithm or the conversion back to
the time domain, as those steps may vary depending on the specific
technique use.
Problem 2:
You are working on a speech recognition system and need to extract the
relevant features from a spoken sentence to classify and recognize the words
accurately. You want to use time and frequency representations to analyze
the speech signal and extract discriminative features.
Solution:
1. Load the speech signal into MATLAB using the audioread

function.
2. Preprocess the speech signal by removing any DC offset and
applying a suitable window function to mitigate artifacts caused by
sudden changes at the edges.
3. Compute the short-time Fourier transform (STFT) of the speech
signal using the spectrogram function. Choose appropriate
window length and overlap parameters.
4. Visualize the resulting spectrogram to observe the time-varying

frequency content of the speech signal. Pay attention to features
such as formants, which correspond to the resonant frequencies of
the vocal tract.
5. Extract relevant features from the spectrogram, such as Mel-
frequency cepstral coefficients (MFCCs). Use the melSpectrogram
and mfcc functions to compute the MFCCs.
6. Optionally, perform feature normalization or dimensionality
reduction techniques, such as mean normalization or principal
component analysis (PCA), to further enhance the discriminative
properties of the features.
7. Feed the extracted features into a machine learning or pattern
recognition algorithm to train a speech recognition model.
8. Test the trained model on new speech samples and evaluate its
performance in recognizing the spoken words.
Note: The provided solutions outline the general steps involved in solving
the problems. The specific MATLAB code implementation may vary
depending on the requirements, signal characteristics, and chosen signal
processing techniques.
MATLAB example:
% Load the speech signal

[speech, Fs] = audioread('speech_file.wav');
% Preprocessing
speech = speech - mean(speech); % Remove DC offset
window = hamming(0.03*Fs, 'periodic'); % Hamming window
overlap = round(0.01*Fs); % 10 ms overlap
% Compute the spectrogram

[S, F, T] = spectrogram(speech, window, overlap, [], Fs);
% Visualize the spectrogram

figure;
imagesc(T, F, 10*log10(abs(S)));
axis xy;
xlabel('Time (s)');
ylabel('Frequency (Hz)');
title('Spectrogram of Speech Signal');
56 Chapter II
% Extract Mel-frequency cepstral coefficients (MFCCs)

numCoeffs = 13; % Number of MFCC coefficients to extract
mfccs = mfcc(speech, Fs, 'Window', window, 'OverlapLength', overlap,
'NumCoeffs', numCoeffs);
% Feature normalization (optional)

mfccs = (mfccs - mean(mfccs, 2)) ./ std(mfccs, 0, 2);
% Display the MFCCs

figure;
imagesc(1:size(mfccs, 2), 1:numCoeffs, mfccs);
axis xy;
xlabel('Frame');
ylabel('MFCC Coefficient');
title('MFCCs of Speech Signal');
% Further processing and classification using machine learning algorithms

% ...
The code above demonstrates the extraction of Mel-frequency cepstral

coefficients (MFCCs) from a speech signal and visualizes the spectrogram
and MFCCs. Please note that the code does not include the complete
implementation of further processing or classification using machine
learning algorithms, as those steps may depend on the specific requirements
of your speech recognition system.
Statistical signal processing

The chapter on statistical signal processing covers important concepts and
techniques for analyzing and processing signals using statistical methods.
Thus, probability theory, random variables, and random processes, which
provide the foundation for understanding uncertainty in signal processing,
are indispensable. The chapter then delves into statistical estimation
techniques, which enable the estimation of unknown parameters from
observed signals. It also covers signal detection and hypothesis testing,
which involve making decisions based on statistical analysis to distinguish
between different signal states or hypotheses. Lastly, the chapter explores
signal classification and pattern recognition, focusing on methods to
categorize signals based on their features and patterns. Understanding these
topics is crucial for effectively processing signals in various applications.
Probability theory is like a toolbox for understanding and predicting

uncertain events. It helps us make sense of the likelihood or chance of
something happening. We use probability to analyze situations where there
are different possible outcomes, and we want to know how likely each
outcome is. Random variables are a way to describe these uncertain
outcomes with numbers. They assign values to each possible outcome and
tell us how likely each value is. Random processes, on the other hand, are
like sequences of random variables that change over time. They help us
model and understand how things change or evolve randomly. Therefore,
probability theory and random variables/processes give us the tools to
quantify and understand uncertainty in the world around us.
While this book may not delve into the fundamentals of probability theory,
random variables, and random processes in detail, it is highly recommended
for readers to refresh their understanding of these concepts before
proceeding further. Having a solid grasp of these fundamental concepts will
provide a strong foundation for comprehending the advanced topics and
techniques discussed throughout the book. It will also enable readers to
make connections and better appreciate the applications and significance of
statistical signal processing. Therefore, take a moment to recap your
knowledge and ensure you have a good understanding of probability theory,
random variables, and random processes before diving into the subsequent
chapters.
Statistical estimation techniques
Statistical estimation techniques play a crucial role in signal processing,

enabling the estimation of unknown parameters from observed signals.
These techniques involve using statistical methods to infer the values of
parameters based on available data. By leveraging probability theory and
statistical models, estimation techniques provide valuable insights into
signal properties and facilitate various signal processing tasks.
Statistical estimation techniques are employed in signal processing to

estimate unknown parameters that characterize the underlying signals or
systems. These techniques utilize statistical models and probability theory
to make educated guesses about the values of these parameters based on
observed data.
The process of statistical estimation involves the following key elements:
1. Estimators: Estimators are mathematical algorithms or formulas

used to estimate unknown parameters from the available data.
58 Chapter II
These estimators are typically functions of the observed data and

are designed to provide an optimal estimate of the true parameter
values.
2. Point Estimation: Point estimation involves finding a single
value, known as the point estimate, which represents the estimated
value of the parameter. Common point estimators include the
maximum likelihood estimator (MLE), which maximizes the
likelihood function based on the observed data, and the method of
moments estimator, which matches sample moments with
population moments.
3. Interval Estimation: Interval estimation provides a range of
values, known as a confidence interval, within which the true
parameter value is likely to lie. Confidence intervals are
constructed based on the estimated parameter value and the
variability of the estimator. They provide a measure of uncertainty
associated with the estimation process.
4. Properties of Estimators: Estimators are evaluated based on
certain desirable properties, such as unbiasedness, efficiency, and
consistency. Unbiased estimators have an expected value equal to
the true parameter value, while efficient estimators have minimum
variance among unbiased estimators. Consistent estimators
converge to the true parameter value as the sample size increases.
5. Estimation Techniques: Various estimation techniques are used
in signal processing, depending on the specific problem and
available data. These include least squares estimation, maximum a
posteriori estimation, Bayesian estimation, and minimum mean
square error estimation.
Statistical estimation techniques are widely used in signal processing

applications such as parameter estimation in signal models, system
identification, channel estimation in communication systems, and adaptive
filtering. These techniques provide valuable tools for extracting meaningful
information from observed data and improving the accuracy and
performance of signal processing algorithms.
Estimators
In statistics, estimators are mathematical algorithms or formulas used to

estimate unknown parameters based on the available data. These parameters
represent characteristics or properties of a population, such as the mean,
variance, or regression coefficients.
Let us consider an example to understand this concept better. Suppose we

are interested in estimating the population mean (ߤ) of a certain variable.
We collect a random sample of size n from the population, and we denote
the sample mean as ܺത. The sample mean is an example of an estimator.
To estimate the population mean (ߤ) using the sample mean (ܺത), we can
use the following formula:
ଵ
ߤƸ = ܺത= σ௡௜ୀଵ ܺ௜ , (24)
௡
where ݊ represents the sample size, which is the number of observations in

the sample and ܺ௜ represents each individual observation in the sample.
In this case, the estimator (ܺത) is simply the sample mean itself. It provides
an estimate of the unknown population mean (ߤ) based on the observed
data.
Let us say we collect a sample of 50 observations from a population and

calculate the sample mean to be 10. Using this estimator, we can estimate
the population mean as 10. This means that, based on our sample, we expect
the true population mean to be around 10.
Now, it is important to note that not all estimators are as straightforward as

the sample mean. In many cases, more complex mathematical formulas or
algorithms are used to estimate parameters. The choice of estimator depends
on various factors, such as the properties of the data, the underlying
assumptions, and the specific goal of the estimation.
Estimators are designed to provide the best possible estimate of the true
parameter values based on the available data. The concept of “optimal”
estimation typically involves minimizing the bias (difference between the
expected value of the estimator and the true parameter value) and/or the
variance (a measure of the estimator's variability). The sample mean serves
as an estimator for the population mean (ߤ), providing an estimate of the
average value of the variable in the population based on the observed data
in the sample. The larger the sample size, the more reliable the sample mean
becomes as an estimator for the population mean.
Point estimation
Point estimation is a statistical method used to estimate an unknown

population parameter based on a sample of data. It involves finding a single
60 Chapter II
value, often denoted as a point estimator, that represents the best guess for
the true value of the parameter.
The point estimate (ߤƸ ) is equal to the sample mean (ܺത), meaning that the
sample mean serves as both an estimator and a point estimate for the
population mean. The terms “estimator” and “point estimate” are often used
interchangeably in practice. They refer to the same concept, which is using
a statistic to estimate an unknown population parameter.
Let us consider an example to understand point estimation better. Suppose

we are interested in estimating the average height (ߤ) of a certain population
of individuals. We collect a random sample of size n from the population
and denote the sample mean as ܺത. Our goal is to use this sample mean as a
point estimator for the population mean.
It is important to note that a point estimate is a single value, and it may not
exactly match the true population parameter. There is a degree of
uncertainty associated with point estimation. The accuracy of the point
estimate depends on various factors, such as the sample size, sampling
method, and variability within the population.
Additionally, it is common to assess the precision or reliability of a point

estimate by calculating a measure of uncertainty called a confidence
interval. A confidence interval provides a range of values within which we
can be reasonably confident that the true population parameter lies.
In summary, point estimation involves using a single value, known as a

point estimator, to estimate an unknown population parameter. The point
estimator represents the best guess for the parameter based on the available
sample data. The sample mean is a simple example of a point estimator used
to estimate the population mean.
Interval estimation
Interval estimation, also known as confidence interval estimation, is a

statistical method used to estimate an unknown population parameter by
providing a range of values within which the true parameter value is likely
to lie. Confidence intervals provide a measure of uncertainty associated with
the estimation process.
To construct a confidence interval, we start with a point estimate obtained

from a sample, which represents our best guess for the true parameter value.
Then, we take into account the variability of the estimator and construct a
range of values that is likely to contain the true parameter value with a
specified level of confidence.
The formula for constructing a confidence interval depends on the specific

parameter being estimated and the distributional assumptions. However, the
general formula can be expressed as Estimate±Margin of Error.
In this formula, the “Estimate” represents the point estimate obtained from
the sample, and the “Margin of Error” represents a measure of uncertainty
that accounts for the variability of the estimator.
The margin of error is typically based on the standard error of the estimator,
which quantifies the average deviation between the estimator and the true
parameter value across different samples. The standard error is influenced
by factors such as the sample size, the variability of the data, and the
distributional assumptions.
The choice of confidence level determines the width of the confidence

interval and represents the probability that the interval contains the true
parameter value. Commonly used confidence levels are 90%, 95%, and
99%.
For example, let us consider estimating the population mean (ߤ) using the
sample mean (ܺത) with a 95% confidence interval. The formula for
௦
constructing the confidence interval is ܺത ± ‫ ݖ‬ቀ ቁ.
ξ௡
In this formula, ܺത represents the sample mean, ‫ ݖ‬is the critical value from
the standard normal distribution corresponding to the desired confidence
level (e.g., 1.96 for a 95% confidence level), ‫ ݏ‬represents the sample
standard deviation, and ݊ is the sample size.
For instance, if we have a sample of 100 individuals and calculate the

sample mean to be 170 centimeters with a sample standard deviation of 5
centimeters, the 95% confidence interval can be calculated as
ହ
170±1.96×( ). Simplifying the expression, we get 170±1.96×0.5. Thus,
ξଵ଴଴
the 95% confidence interval for the population mean would be
(169.02,170.98). This means that we are 95% confident that the true
population mean lies within the range of 169.02 to 170.98 centimeters.
In summary, interval estimation, or confidence interval estimation, provides

a range of values within which the true parameter value is likely to lie.
Confidence intervals take into account the point estimate, the variability of
62 Chapter II
the estimator (usually represented by the standard error), and the chosen
confidence level. They provide a measure of uncertainty associated with the
estimation process. The width of the confidence interval depends on the
confidence level and the variability of the data.
Signal detection and hypothesis testing
Signal detection theory and hypothesis testing are two important concepts
in statistics and decision-making. These concepts involve making decisions
based on observed data and statistical measures. Additionally, Bayesian
statistics provides a framework for decision-making that incorporates prior
beliefs and observed data.
In signal detection theory, the decision process is typically described in

terms of two possible responses: “signal present” and “signal absent.”
Signal detection theory considers factors such as sensitivity ൫݀ሖ ൯, criterion
(ܿ), and response bias (ߚ) to quantify an individual's ability to discriminate
between signal and noise and their tendency to respond in a particular way.
Bayesian statistics, on the other hand, goes beyond classical hypothesis

testing by incorporating prior beliefs and updating them based on observed
data. This is done using probability distributions.
One approach in Bayesian statistics is the Maximum A Posteriori (MAP)

decision rule. In this approach, we make decisions by selecting the option
that has the highest posterior probability, given the observed data. The
posterior probability is obtained by combining the prior probability
distribution ܲ(ߠ), which represents our initial beliefs about the unknown
parameters (ߠ), with the likelihood function (ܲ(ܺ|ߠ)), which represents the
probability of observing the data (ܺ) given the parameters.
Mathematically, the MAP decision rule can be expressed as

௉(௑|ఏ)௉(ఏ)
Decision: ߠ෠ெ஺௉ = arg maxఏ ܲ(ߠ|ܺ) = arg maxఏ . (25)
௉(௑)
Another approach in Bayesian statistics is the Maximum Likelihood (ML)

decision rule. This rule involves making decisions by selecting the option
that maximizes the likelihood function (ܲ(ܺ|ߠ)), which represents the
probability of observing the data given the parameters. The ML decision
rule does not incorporate prior beliefs and focuses solely on maximizing the
likelihood.
Mathematically, the ML decision rule can be expressed as
Decision: ߠ෠ெ஺௉ = arg maxఏ ܲ(ܺ|ߠ). (26)
Both the MAP and ML decision rules are used in Bayesian statistics to make
decisions based on observed data. They involve estimating parameters
(ߠ) and selecting the option that maximizes the posterior probability
(ܲ(ߠ|ܺ)) for MAP or the likelihood function (ܲ(ܺ|ߠ)) for ML.
Additionally, the Minimum Distance rule is another decision-making rule

used in statistics. It involves making decisions by selecting the option that
is closest to a reference point or distribution. This rule is often used in
situations where we want to minimize the discrepancy or distance between
the observed data and a reference point or distribution.
In summary, signal detection theory, hypothesis testing, and Bayesian

statistics provide different frameworks for decision-making. Signal
detection theory focuses on factors such as sensitivity ൫݀ሖ ൯, criterion (ܿ),
and response bias (ߚ). Bayesian statistics incorporates prior beliefs and uses
approaches like the MAP and ML decision rules to make decisions based
on observed data. The Minimum Distance rule is another decision-making
rule used to minimize the discrepancy between observed data and a
reference point or distribution.
Signal classification and pattern recognition
Signal classification and pattern recognition are fields within machine

learning and signal processing that involve categorizing signals or patterns
into different classes based on their features. In signal classification, the
goal is to assign predefined labels or categories to incoming signals based
on their characteristics. This is achieved by developing models or
algorithms that can accurately classify new signals into their appropriate
classes.
Mathematically, signal classification can be represented as a function that

maps input signals (ܺ) to their corresponding class labels (‫)ݕ‬
‫)ܺ(݂ = ݕ‬, (27)
where ݂(ή) represents the classification model. The classification model can
take various forms depending on the specific algorithm used. For instance,
decision trees partition the feature space based on a series of if-else
64 Chapter II
conditions, while K-nearest neighbors (KNN) classifies new signals based

on the majority class of its k nearest neighbors in the training data.
Another commonly used classification algorithm is support vector machines

(SVM). In SVM, the goal is to find an optimal hyperplane that separates the
different classes in the feature space, maximizing the margin between the
classes. Mathematically, SVM seeks to solve the optimization problem as
ଵ
݉݅݊݅݉‫ ݉ݑ‬ԡ‫ݓ‬ԡଶ
ଶ
‫ݕ ݋ݐ ݐ݆ܾܿ݁ݑݏ‬௜ (‫ݔݓ‬௜ + ܾ) ൒ 1 for all ݅
Here, ‫ ݓ‬represents the weight vector, ‫ݔ‬௜ is the feature vector of the ݅ െ ‫݄ݐ‬
training sample, ܾ is the bias term, and ‫ݕ‬௜ is the class label of the ݅ െ ‫݄ݐ‬
sample.
Deep learning models, such as Convolutional Neural Networks (CNNs),

have also gained popularity in signal classification tasks, especially in
image and audio processing. CNNs consist of multiple layers of
convolutional and pooling operations followed by fully connected layers.
Mathematically, CNNs apply convolutions and nonlinear activation
functions to extract hierarchical representations of the input signals, which
are then fed into the fully connected layers for classification.
In summary, signal classification algorithms can be represented as

mathematical functions that map input signals to their corresponding class
labels. The specific form of the classification model, such as decision trees,
SVM, KNN, or deep learning models like CNNs, determines how the
mapping is performed. These algorithms utilize mathematical techniques
such as optimization, matrix operations, activation functions, and statistical
measures to learn patterns and make accurate classifications.
To train a signal classification model, a labeled dataset is required,

consisting of input signals and their corresponding class labels. The model
is then trained to learn the underlying patterns or features that distinguish
the different classes. This training process typically involves optimization
techniques such as gradient descent, which minimizes the classification
error or maximizes the model's performance metrics.
Pattern recognition, on the other hand, is a broader field that encompasses

the identification and analysis of patterns within data. It involves extracting
meaningful features from signals or data and using those features to classify
or recognize patterns. The process in pattern recognition typically involves

several steps.
Firstly, data preprocessing is performed to prepare the data by removing

noise, normalizing values, or transforming the data into a suitable
representation for analysis. Then, feature extraction techniques are applied
to extract features from the data that capture the essential characteristics
distinguishing the patterns or classes. Feature extraction methods can
include statistical measures, signal processing techniques, or transformation
algorithms.
In some cases, feature selection is performed to identify the most relevant

or informative features for the pattern recognition task. This helps reduce
dimensionality and improve computational efficiency. Finally, a
classification algorithm is applied to assign class labels to the patterns using
the extracted features. This can involve using machine learning techniques
such as decision trees, neural networks, or statistical classifiers.
The mathematics involved in signal classification and pattern recognition

are centered around feature extraction, representation, and classification.
Feature extraction involves transforming raw signals or data into a set of
meaningful features that capture the essential characteristics of the patterns.
This can be achieved using statistical measures, signal processing
algorithms, or mathematical transforms such as Fourier transforms or
wavelet transforms.
Once the features are extracted, they are represented as feature vectors or
matrices, enabling mathematical operations and analysis. Classification
algorithms, such as decision trees, SVMs, or neural networks, are then
employed to learn the underlying patterns and relationships between the
features and their corresponding class labels. These algorithms use
mathematical optimization methods, such as gradient descent or maximum
likelihood estimation, to find the best parameters that minimize the
classification error or maximize the model's performance metrics.
The advancements in non-classical signal processing have led to innovative

approaches that challenge the assumptions of linearity, stationarity, and
Gaussianity often made in classical signal processing. Non-classical signal
processing techniques incorporate concepts from fields such as machine
learning, deep learning, compressed sensing, and sparse representation to
handle complex, non-linear, and non-stationary signals.
66 Chapter II
These approaches leverage advanced mathematical frameworks, including

matrix factorization, graph signal processing, manifold learning, and
Bayesian inference, to extract intricate patterns and structures from signals.
By embracing non-classical signal processing, researchers and practitioners
can unlock new insights, improve signal classification and pattern
recognition accuracy, and tackle challenging signal analysis problems in
diverse domains such as biomedical signal processing, environmental
monitoring, and audio and image processing.
In summary, signal classification and pattern recognition involve the

categorization of signals or patterns into different classes. Signal
classification focuses on assigning predefined labels to incoming signals,
while pattern recognition involves extracting features from data and using
them to recognize or classify patterns. The mathematics involved in these
fields includes modeling the relationship between input signals or features
and their corresponding class labels using various algorithms and
techniques.

In the realm of signals, a chapter we find. On statistical processing, concepts
designed.
Important techniques for analysis, you see. Using statistical methods, it aims
to be.
Probability theory, the foundation it lays. Random variables, in uncertain

ways.
With random processes, uncertainty unfolds. Understanding signals, as the

story unfolds.
Estimation techniques, statistical in stride. Detecting signals, hypothesis we

confide.
Classification and recognition, patterns to trace. Statistical signal

processing, a captivating chase.
Key takeaways from the statistical signal processing are given as follows:
1. Probability theory, random variables, and random processes are

essential for understanding uncertainty in signal processing. They
provide tools to quantify and analyze the likelihood or chance of

events.
2. Statistical estimation techniques are used to estimate unknown
parameters from observed signals. Estimators are mathematical
algorithms or formulas that provide optimal estimates of true
parameter values. Point estimation involves finding a single value
representing the estimated parameter, while interval estimation
provides a range of values within which the true parameter is likely
to lie.
3. Signal detection and hypothesis testing involve making decisions
based on statistical analysis to distinguish between different signal
states or hypotheses. Factors like sensitivity, criterion, and
response bias are considered in signal detection theory. Bayesian
statistics provides a framework for decision-making by incorporating
prior beliefs and observed data.
4. Signal classification and pattern recognition aim to categorize
signals based on their features and patterns. Classification models
or algorithms are developed to assign labels or categories to
incoming signals accurately. Decision trees, k-nearest neighbors,
and support vector machines are examples of classification
algorithms.
Layman's Guide:
The chapter on statistical signal processing introduces important concepts
and techniques for analyzing and processing signals using statistical
methods. It emphasizes the role of probability theory, random variables, and
random processes in understanding uncertainty in signal processing.
Probability theory is like a toolbox that helps us understand and predict

uncertain events. It allows us to analyze situations with multiple possible
outcomes and determine the likelihood of each outcome. Random variables
assign values to these outcomes and tell us how likely each value is.
Random processes are like sequences of random variables that change over
time, helping us model and understand how things change randomly.
Statistical estimation techniques are used to estimate unknown parameters

from observed signals. These techniques use statistical models and
probability theory to make educated guesses about parameter values based
on available data. Estimators are mathematical algorithms or formulas that
provide optimal estimates of the true parameter values. Point estimation
68 Chapter II
involves finding a single value, while interval estimation provides a range

of values within which the true parameter is likely to lie.
Signal detection and hypothesis testing involve making decisions based on

statistical analysis. In signal detection theory, we distinguish between
different signal states or hypotheses by considering factors like sensitivity,
criterion, and response bias. Bayesian statistics goes beyond classical
hypothesis testing by incorporating prior beliefs and observed data in
decision-making.
Signal classification and pattern recognition aim to categorize signals based

on their features and patterns. Classification models or algorithms are
developed to assign labels or categories to incoming signals accurately.
Decision trees, k-nearest neighbors, and support vector machines are
examples of classification algorithms.
It is recommended to have a solid understanding of probability theory,

random variables, and random processes before diving into the subsequent
chapters on statistical signal processing. These foundational concepts
provide a strong basis for comprehending the advanced topics and
techniques discussed in the book.
Exercises of statistical signal processing

Problem 1:
In this problem, we are dealing with a received signal that can either be a
“0” or a “1” with equal probabilities. However, the signal is corrupted by
additive white Gaussian noise (AWGN), which introduces uncertainty into
the observed signal. Our task is to design a detector that can determine
whether the received signal is a “0” or a “1” based on the observed noisy
signal.
Solution:
To solve this problem, we can use a simple threshold-based detector. The

idea is to compare the received signal with a threshold value and make a
decision based on the result of the comparison. If the received signal is
above the threshold, we will detect it as a “1”. Otherwise, if the received
signal is below the threshold, we will detect it as a “0”.
MATLAB example:
Now, let us implement the solution in MATLAB:

% Problem 1: Signal Detection
% Parameters
threshold = 0; % Threshold value
% Generate the received signal

trueSignal = randi([0, 1]); % Generate a random true signal (0 or 1)
noise = randn(); % Generate AWGN noise
receivedSignal = trueSignal + noise; % Add noise to the true signal
% Detection
if receivedSignal > threshold
detectedSignal = 1; % Signal above threshold is detected as "1"
else
detectedSignal = 0; % Signal below threshold is detected as "0"
end
% Display results
disp(['True Signal: ', num2str(trueSignal)]);
disp(['Received Signal: ', num2str(receivedSignal)]);
disp(['Detected Signal: ', num2str(detectedSignal)]);
The obtained results reveal the following findings:
True Signal: 1
Received Signal: 2.8339
Detected Signal: 1
Problem 2:
In this problem, we are given a set of observed signals corrupted by noise,

and our goal is to estimate the parameters of a signal model. Specifically,
we are dealing with a sinusoidal signal of the form as
ܺ(‫(݊݅ݏܣ = )ݐ‬2ߨ݂‫ ݐ‬+ ߮) + ݊(‫)ݐ‬,
where
x ‫ ܣ‬is the amplitude of the signal

x ݂ is the frequency of the signal
x ߮ is the phase of the signal
x ݊(‫ )ݐ‬is the additive noise corrupting the signal
70 Chapter II
Our task is to estimate the values of ‫ܣ‬, ݂, and ߮ based on the observed
signals.
Solution:
To solve this problem, we can use the technique of maximum likelihood

estimation (MLE). The idea behind MLE is to find the parameter values that
maximize the likelihood of the observed data. In the case of a sinusoidal
signal corrupted by noise, we can formulate the likelihood function as the
product of the probabilities of observing each data point given the
parameters.
Now, let us derive the maximum likelihood estimators for ‫ܣ‬, ݂, and ߮ using
a mathematical approach and then implement the solution in MATLAB:
Mathematical Solution:
1. Estimating the Amplitude (‫)ܣ‬:

x We can estimate the amplitude by taking the square root
of the average of the squared observed signal values.
ଵ
‫ܣ‬෡ =ට σே ଶ
௜ୀଵ ‫ݔ‬௜
ே
2. Estimating the Frequency (݂):

x We can estimate the frequency by finding the peak in the
power spectral density (PSD) of the observed signal. One
common method is to use the Fourier Transform.
݂መ =arg maxఏ {ܲܵ‫})݂(ܦ‬
3. Estimating the Phase (߮):

x We can estimate the phase by finding the phase shift that
maximizes the correlation between the observed signal
and a reference sinusoidal signal.
߮ො = ܽ‫{ ݔܽ݉ ݃ݎ‬σே መ

௜ୀ଴ ܺ௜ sin൫2ߨ݂ ‫ݐ‬௜ + ߮൯}
MATLAB example:
Now, let us implement the solution in MATLAB:
% Problem 2: Parameter Estimation
% Generate the true signal

t = linspace(0, 1, 1000); % Time vector
A = 1.5; % True amplitude
f = 10; % True frequency
phi = pi/4; % True phase
noise = randn(size(t)); % Additive white Gaussian noise
trueSignal = A * sin(2*pi*f*t + phi); % True signal
observedSignal = trueSignal + noise; % Observed signal
% Estimation
estimatedA = sqrt(mean(observedSignal.^2)); % Estimating the amplitude
fftSignal = abs(fft(observedSignal)); % Compute FFT
[~, index] = max(fftSignal(1:length(fftSignal)/2)); % Find peak in PSD
estimatedF = index / (length(t) * (t(2) - t(1))); % Estimating the frequency
correlation = sum(observedSignal .* sin(2*pi*estimatedF*t + phi)); %
Compute correlation
estimatedPhi = angle(correlation); % Estimating the phase
% Display results
disp(['True Amplitude: ', num2str(A)]);
disp(['Estimated Amplitude: ', num2str(estimatedA)]);
disp(['True Frequency: ', num2str(f)]);
disp(['Estimated Frequency: ', num2str(estimatedF)]);
disp(['True Phase: ', num2str(phi)]);
disp(['Estimated Phase: ', num2str(estimatedPhi)]);
% Plot signals
figure;
subplot(2, 1, 1);
plot(t, trueSignal, 'b', 'LineWidth', 2);
hold on;
plot(t, observedSignal, 'r', 'LineWidth', 1);
hold off;
xlabel('Time');
legend('True Signal', 'Observed Signal');
72 Chapter II
title('True and Observed Signals');
subplot(2, 1, 2);
plot(t, observedSignal, 'r', 'LineWidth', 1);
hold on;
plot(t, estimatedA * sin(2*pi*estimatedF*t + estimatedPhi), 'g--',
'LineWidth', 2);
hold off;
xlabel('Time');
legend('Observed Signal', 'Estimated Signal');
title('Observed and Estimated Signals');
In this MATLAB code, we first generate the true sinusoidal signal with a
given amplitude, frequency, and phase. We add AWGN to the true signal to
create the observed signal. Then, we estimate the parameters using the
derived formulas: amplitude (‫)ܣ‬, frequency, (݂), and phase (߮). Finally, we
display the true and estimated values of the parameters and plot the true
signal, observed signal, and estimated signal for visualization.
Note: The above code assumes that the noise follows a standard Gaussian
distribution with zero mean and unit variance. Adjustments may be needed
for specific noise characteristics or distributions.Top of Form
The obtained results reveal the following findings and the plot in Figure 2-
7.
True Amplitude: 1.5

Estimated Amplitude: 1.4582
True Frequency: 10
Estimated Frequency: 10.989
True Phase: 0.7854
Estimated Phase: 3.1416
Figure 2-8 illustrates the result, displaying the true signal, observed signal,
and the estimated signal obtained through parameter estimation.
Figure 2-8: MATLAB example of the parameter estimation.
Problem 3: Classification and Pattern Recognition
Description: You are given a dataset of generated signals, each belonging

to one of three classes: “A,” “B,” or “C.” Your task is to develop a
classification model using pattern recognition techniques to accurately
classify new signals into their respective classes. The dataset consists of 100
training signals and 20 test signals per class, with each signal represented as
a feature vector of length 50.
Solution:
To solve this problem, we can use a machine learning algorithm called

support vector machines (SVM) for classification. SVM is a powerful
technique commonly used in pattern recognition tasks. Here is a step-by-
step solution:
Step 1: Load the dataset
x Let X_train be a matrix of size 100x50 containing the feature

vectors of the training signals.
74 Chapter II
x Let Y_train be a vector of size 100x1 containing the labels for the
training signals (1: Class A, 2: Class B, 3: Class C).
x Let X_test be a matrix of size 20x50 containing the feature vectors
of the test signals.
x Let Y_test be a vector of size 20x1 containing the true labels for
the test signals.
Step 2: Train the SVM model
x Construct the SVM classifier using the training data.

x Let SVM_model be the trained SVM model.
Step 3: Predict the labels of test signals
x Use the trained model to predict the labels of the test signals.
x Let Y_pred be the predicted labels for the test signals.
Step 4: Evaluate the performance of the model
x Calculate the classification accuracy by comparing the predicted

labels (Y_pred) with the true labels (Y_test).
Step 5: Display the results
x Display the predicted labels and the true labels.

x Display the accuracy of the model.
MATLAB example:
% Step 1: Generate the dataset

numSamplesTrain = 100; % Number of training samples per class
numSamplesTest = 20; % Number of test samples per class
signalLength = 50; % Length of each signal
% Generate signals for Class A

classA_train = randn(numSamplesTrain, signalLength);
classA_test = randn(numSamplesTest, signalLength);
% Generate signals for Class B

classB_train = 2 + randn(numSamplesTrain, signalLength);
classB_test = 2 + randn(numSamplesTest, signalLength);
% Generate signals for Class C

classC_train = -2 + randn(numSamplesTrain, signalLength);
classC_test = -2 + randn(numSamplesTest, signalLength);
% Combine the training and test data

X_train = [classA_train; classB_train; classC_train];
X_test = [classA_test; classB_test; classC_test];
% Generate labels for the training and test data

Y_train = repelem(1:3, numSamplesTrain)';
Y_test = repelem(1:3, numSamplesTest)';
% Step 2: Train the SVM model

svmModel = fitcecoc(X_train, Y_train);
% Step 3: Predict the labels of test signals

Y_pred = predict(svmModel, X_test);
% Step 4: Evaluate the performance of the model

accuracy = sum(Y_pred == Y_test) / numel(Y_test);
% Compute the confusion matrix

numClasses = max(Y_test);
confusionMat = zeros(numClasses, numClasses);
for i = 1:numClasses
for j = 1:numClasses
confusionMat(i, j) = sum(Y_test == i & Y_pred == j);
end
end
% Compute precision
precision = zeros(numClasses, 1);
for i = 1:numClasses
precision(i) = confusionMat(i, i) / sum(confusionMat(:, i));
end
% Display the results

disp(['Accuracy: ' num2str(accuracy)]);
disp('Confusion Matrix:');
disp(confusionMat);
disp('Precision:');
disp(precision);
76 Chapter II
% Plot the confusion matrix

figure;
imagesc(confusionMat);
colorbar;
axis square;
xlabel('Predicted Class');
ylabel('True Class');
title('Confusion Matrix');
% Step 5: Test with unlabeled signals

unlabeledSignals = [1 + randn(5, signalLength); -1 + randn(5,
signalLength)];
unlabeledPredictions = predict(svmModel, unlabeledSignals);
% Define ground truth labels for the unlabeled signals

groundTruthLabels = [1 1 1 1 1 3 3 3 3 3];
% Determine correctness of unlabeled predictions

isCorrect = unlabeledPredictions == groundTruthLabels';
% Plot the comparison between predictions and ground truth

figure;
subplot(2, 1, 1);
plot(unlabeledSignals(unlabeledPredictions == 1, :)', 'b');
hold on;
plot(unlabeledSignals(unlabeledPredictions == 2, :)', 'r');
plot(unlabeledSignals(unlabeledPredictions == 3, :)', 'g');
hold off;
title('Unlabeled Signal Predictions');
xlabel('Time');
ylabel('Signal Value');
legend('Predicted Class 1', 'Predicted Class 2', 'Predicted Class 3');
subplot(2, 1, 2);
plot(unlabeledSignals(groundTruthLabels == 1, :)', 'b');
hold on;
plot(unlabeledSignals(groundTruthLabels == 2, :)', 'r');
plot(unlabeledSignals(groundTruthLabels == 3, :)', 'g');
hold off;
title('Ground Truth');
xlabel('Time');
legend('Ground Truth Class 1', 'Ground Truth Class 2', 'Ground Truth
Class 3');
% Plot correctness curve

figure;
plot(1:length(isCorrect), isCorrect, 'ko-', 'LineWidth', 1.5);
title('Unlabeled Prediction Correctness');
xlabel('Signal Index');
ylabel('Correctness');
ylim([-0.1 1.1]);
xticks(1:length(isCorrect));
yticks([0 1]);
xticklabels(1:length(isCorrect));
yticklabels({'Incorrect', 'Correct'});
figure;
bar(isCorrect);
title('Element-wise Comparison');
xlabel('Label (1, 2, and 3)');
ylabel('Comparison Result');
xticks(1:numel(groundTruthLabels));
xticklabels(groundTruthLabels);
The obtained results reveal the following findings and the relevant plots
(signals versus time, confusion matrix, element-wise comparison (correctness)
of unlabeled signals) in Figures 2-8-2-10.
Accuracy: 1
Confusion Matrix:
20 0 0
0 20 0
0 0 20
Precision:
1
1
1
78 Chapter II
The outcomes are visualized in Figures 2-9 and 2-10, depicting the
prediction of unlabeled signals alongside the ground truth and the
corresponding confusion matrix, respectively. Figure 2-11 showcases the
element-wise comparison, highlighting the correctness of the unlabeled
signals.
Figure 2-9: Signals versus time of the classification and Pattern recognition.
Figure 2-10: Confusion matrix of the classification and Pattern recognition.

80 Chapter II
Figure 2-11: Element-wise comparison (correctness) of unlabeled signals of the

classification and pattern recognition.
Note that the correctness curve indicating whether each prediction is correct
or incorrect is based on the unlabeled signals and the corresponding ground
truth labels.
CHAPTER III
NON-CLASSICAL SIGNAL PROCESSING4
In the chapter on non-classical signal processing, we delve into several

important topics that expand beyond traditional signal processing
approaches. Wavelet transforms and time-frequency analysis provide tools
for studying signals that change over time. Compressed sensing and sparse
signal processing techniques allow us to reconstruct and process signals
using fewer measurements and leveraging their sparse representations. The
application of machine learning and deep learning techniques to signal
processing enables tasks like classification and prediction. Lastly, we
explore signal processing techniques tailored for non-Euclidean data, such
as graphs and networks. This chapter introduces innovative approaches that
go beyond classical signal processing, empowering us to tackle complex
and diverse signal analysis challenges.
Wavelet transforms and time-frequency analysis
In the realm of non-classical signal processing, understanding and analyzing

signals that are not stationary, meaning their properties change over time, is
crucial. Wavelet transforms provide a valuable tool for addressing this
challenge. Unlike traditional Fourier transforms, which use fixed basis
functions like sine and cosine waves, wavelet transforms utilize functions
called wavelets that are localized in both time and frequency domains. This
localization property enables wavelet transforms to capture transient
4 Non-classical signal processing refers to advanced signal processing techniques
that go beyond the traditional methods used in classical signal processing. It involves
the utilization of innovative approaches to analyze and process signals, particularly
in challenging scenarios where traditional techniques may be limited or insufficient.
It consists of several techniques such as wavelet transforms and time-frequency
analysis, compressed sensing and sparse signal processing, machine learning and
deep learning for signals, and signal processing for non-Euclidean data.
Non-Classical Signal Processing 83
features and changes in the signal more effectively, making them well-
suited for non-stationary signals.
The continuous wavelet transform (CWT) is a mathematical operation that

breaks down a signal into different frequency components and analyzes
them with respect to time. It is defined by
‫ܽ(ܹܶܥ‬, ܾ) = ‫ ݐ((߰)ݐ(ݔ ׬‬െ ܾ)/ܽ)݀‫ݐ‬, (28)
where ‫ )ݐ(ݔ‬is the input signal, ȥ(t) is the mother wavelet function, a
represents the scale parameter that controls the width of the wavelet, and b
represents the translation parameter that shifts the wavelet along the time
axis. By applying the CWT, we obtain a time-scale representation of the
signal, revealing how its frequency content changes over different time
intervals.
A popular type of wavelet transform is the discrete wavelet transform

(DWT), which operates on discrete-time signals. The DWT decomposes a
signal into wavelet coefficients at different scales and positions through a
series of high-pass and low-pass filtering operations followed by
downsampling.
In addition to wavelet transforms, time-frequency analysis techniques such

as the short-time Fourier transform (STFT) and Gabor transform are used to
gain insights into the time-varying frequency content of a signal. The STFT
divides the signal into short overlapping segments and computes the Fourier
transform for each segment. Mathematically, the STFT is given by
ܵܶ‫ݐ(ܶܨ‬, ݂) = ‫ ݐ(ݓ )߬(ݔ ׬‬െ ܾ)exp(െ݆2ߨ݂߬) )݀߬, (29)
where ‫ )߬(ݔ‬represents the signal, ‫ )ݐ(ݓ‬is a window function that helps

localize the analysis in time, and f is the frequency variable. This reveals
how the frequency components of the signal evolve over time.
On the other hand, the Gabor transform combines elements of Fourier

analysis and windowed signal analysis. It convolves the signal with a
Gaussian window function in the time domain and takes the Fourier
transform of the resulting signal. Mathematically, the Gabor transform is
given by:
‫ݐ(ܩ‬, ݂) = ‫ ݐ(݃ )߬(ݔ ׬‬െ ߬)exp(െ݆2ߨ݂߬) )݀߬, (30)

84 Chapter III
where ‫ )߬(ݔ‬represents the signal, ݃(‫ )ݐ‬is a Gaussian window function, and
݂ is the frequency variable. The Gabor transform provides a detailed
examination of the signal's behavior in the time-frequency domain.
By applying these techniques, we can visualize and analyze the time-

frequency characteristics of non-stationary signals, unraveling their
transient behavior and frequency variations over time. These mathematical
tools provide a powerful framework for understanding and processing
signals in non-classical signal processing applications.
Let us consider a practical example to illustrate their applications.
Suppose we have a recorded audio signal of a musical piece that includes

both sustained tones and short-lived transients. By applying wavelet
transforms, we can capture the transient features and changes in the signal
more effectively than traditional Fourier transforms. This allows us to
analyze the signal with localized resolution in both time and frequency
domains.
Using the CWT, we can obtain a time-scale representation of the signal. For
instance, if we choose the Mexican hat wavelet as the mother wavelet, we
can identify the time intervals where the sustained tones and transients
occur. This information helps us understand the temporal variations in the
frequency content of the musical piece.
In addition to wavelet transforms, time-frequency analysis techniques such

as the STFT and Gabor transform are useful for examining the time-varying
frequency content of the signal. For example, by applying the STFT with a
suitable window function, we can observe how the frequency components
of the sustained tones and transients evolve over time. This provides insights
into the changing spectral characteristics of the audio signal.
Moreover, the Gabor transform allows us to analyze the signal's behavior in

the time-frequency domain in a more detailed manner. By convolving the
signal with a Gaussian window function and taking the Fourier transform,
we can visualize the varying frequency components at different time points.
This enables a comprehensive understanding of the signal's characteristics,
such as the modulation and energy distribution in both time and frequency.
By utilizing wavelet transforms, the STFT, and the Gabor transform, we can
effectively analyze and process the non-stationary audio signal. These
techniques enable us to capture the dynamic changes in the signal's
behavior, distinguish between sustained tones and transients, and unravel

the intricate time-frequency relationships embedded within the audio data.
In summary, wavelet transforms and time-frequency analysis techniques

provide powerful tools for analyzing non-stationary signals. They offer
localized resolution in both time and frequency domains, allowing us to
capture transient features, understand temporal variations in frequency
content, and gain insights into the time-frequency characteristics of the
signal. These mathematical tools, along with practical examples like
analyzing audio signals, demonstrate the importance and effectiveness of
non-classical signal processing techniques in real-world applications.

In the realm of signals, non-stationary they may be. Wavelet transforms
bring clarity, for all to see. They zoom in on intervals, both time and
frequency, Unveiling changes and patterns, with precision and decree.
Time-frequency analysis techniques, they complement. Short-time Fourier

and Gabor, to great extent.
STFT reveals how frequencies evolve in time. Gabor combines Fourier and
windows, a harmonic chime.
Transient features and changes, they capture with ease. Wavelet transforms
excel in non-stationary seas.
Continuous wavelet transform, a time-scale view. Frequency content's

journey, it unveils and ensues.
Discrete wavelet transform, on discrete-time it thrives. Decomposing

signals into coefficients, in multiple dives.
Practical examples, like audio with tones and flair. Showcasing

applications, with signals in the air.
Wavelet transforms and time-frequency analysis, profound. Unraveling

time-frequency relationships, they astound.
Understanding their power, a skill that we seek. In non-classical signal

processing, these tools we must speak.
86 Chapter III
Key takeaways from the wavelet transforms and time-frequency analysis

are given as follows:
1. Wavelet transforms are powerful tools for analyzing non-

stationary signals that change over time. Unlike traditional Fourier
transforms, wavelet transforms provide localized resolution in both
the time and frequency domains, allowing for a more detailed
examination of signal behavior.
2. Time-frequency analysis techniques such as the short-time Fourier
transform (STFT) and Gabor transform complement wavelet
transforms in understanding the time-varying frequency content of
signals. The STFT reveals how the frequency components of a
signal evolve over different time intervals, while the Gabor
transform combines Fourier analysis and windowed signal analysis
for a comprehensive understanding of signal characteristics.
3. Wavelet transforms and time-frequency analysis enable us to
capture transient features and changes in signal behavior more
effectively. These techniques are particularly useful for analyzing
non-stationary signals, such as audio signals with sustained tones
and transients.
4. The continuous wavelet transform (CWT) provides a time-scale
representation of a signal, showing how its frequency content
changes over different time intervals. The CWT uses scale and
translation parameters to control the width and position of the
wavelet, respectively.
5. The discrete wavelet transform (DWT) is a popular type of wavelet
transform that operates on discrete-time signals. It decomposes a
signal into wavelet coefficients at different scales and positions
through filtering operations and downsampling.
6. Practical examples, such as analyzing an audio signal with
sustained tones and transients, illustrate the application of wavelet
transforms and time-frequency analysis techniques. These
techniques allow for a detailed examination of the signal's
temporal and spectral characteristics.
Layman's Guide:
Imagine you have a signal, like a piece of music or an image. Wavelet
transforms and time-frequency analysis help us understand and analyze this
signal by looking at both its time and frequency characteristics.
Let us start with wavelet transforms. Unlike traditional methods, wavelet

transforms allow us to zoom in and analyze specific parts of the signal in
both time and frequency domains. It is like using a magnifying glass to
examine different details of the signal at different scales. This is particularly
useful when dealing with signals that change over time, like music with
varying rhythms or images with intricate patterns.
Now, let us talk about time-frequency analysis. This technique helps us

understand how the frequency content of a signal changes over time. Think
of it as watching a musical performance and observing how the instruments'
sounds evolve throughout the song. Time-frequency analysis techniques,
such as the short-time Fourier transform and the Gabor transform, reveal
how different frequency components of the signal appear and disappear at
different moments.
The short-time Fourier transform divides the signal into small overlapping
segments and examines the frequency content of each segment. It helps us
understand how the signal's frequency components vary over time intervals.
It is like taking snapshots of the signal's frequency content as it progresses.
The Gabor transform combines elements of Fourier analysis and windowed

signal analysis. It allows us to look at the signal's behavior in both the time
and frequency domains simultaneously. It is like using a combination of
lenses to see both the big picture and the finer details of the signal's
frequency components at different time points.
By using wavelet transforms and time-frequency analysis, we gain powerful

tools to understand and analyze signals that change over time. We can
capture the dynamic changes in the signal's behavior and unravel the
intricate relationships between time and frequency.
Thus, whether you are music lover, an image enthusiast, or just curious
about signals, wavelet transforms and time-frequency analysis open up a
whole new world of understanding. They help us appreciate the richness
and complexity of signals and enable us to extract meaningful information
from them.
88 Chapter III
Exercises of wavelet transforms and time-frequency

analysis
Problem 1: Denoising a Noisy Signal using Wavelet Thresholding
Suppose you have a signal corrupted by noise, and you want to remove the
noise to recover the original signal. One approach is to use wavelet
thresholding, which exploits the sparsity of the signal in the wavelet
domain.
Solution:
To denoise a noisy signal using wavelet thresholding, we utilize the concept

of wavelet decomposition and soft thresholding. The idea is to decompose
the signal into different frequency components using the discrete wavelet
transform (DWT). The DWT represents the signal in terms of wavelet
coefficients at different scales and positions.
In this problem, we assume that the noise is additive and follows a Gaussian
distribution. We estimate the noise standard deviation from the noisy signal
itself. Once we have the wavelet coefficients, we apply a thresholding
operation to suppress the coefficients corresponding to noise.
Soft thresholding involves comparing the absolute value of each coefficient

to a threshold value. If the absolute value is below the threshold, we set the
coefficient to zero. If the absolute value is above the threshold, we shrink
the coefficient towards zero by subtracting the threshold and keeping the
sign.
Finally, we reconstruct the denoised signal using the inverse DWT, which
combines the modified wavelet coefficients to obtain the denoised signal.
MATLAB example:
% Generate a noisy signal

fs = 1000; % Sampling frequency
t = 0:1/fs:1; % Time vector
f = 10; % Frequency of the signal
signal = sin(2*pi*f*t); % Clean signal
noise = 0.5*randn(size(t)); % Gaussian noise
noisy_signal = signal + noise; % Noisy signal
% Perform wavelet thresholding for denoising

level = 5; % Number of decomposition levels
wname = 'db4'; % Wavelet name
[C, L] = wavedec(noisy_signal, level, wname); % Wavelet decomposition
sigma = median(abs(C)) / 0.6745; % Estimate the noise standard deviation
threshold = sigma * sqrt(2 * log(length(noisy_signal))); % Threshold value
C_denoised = wthresh(C, 's', threshold); % Soft thresholding
denoised_signal = waverec(C_denoised, L, wname); % Reconstruct
denoised signal
% Plotting the signals

figure;
subplot(3,1,1);
plot(t, signal);
xlabel('Time');
title('Clean Signal');
subplot(3,1,2);
plot(t, noisy_signal);
xlabel('Time');
title('Noisy Signal');
subplot(3,1,3);
plot(t, denoised_signal);
xlabel('Time');
title('Denoised Signal');
Figure 3-1 illustrates the process of denoising a noisy signal through the
utilization of wavelet thresholding.
90 Chapter III
Figure 3-1: Denoising a noisy signal using wavelet thresholding.
Problem 2: Time-Frequency Analysis of a Speech Signal
Suppose you have a speech signal and you want to analyze its time-
frequency characteristics to identify specific phonemes or speech patterns.
Solution:
To perform time-frequency analysis of a speech signal, we use the short-

time Fourier transform (STFT). The STFT allows us to analyze how the
frequency content of the signal changes over time.
In this problem, we divide the speech signal into short overlapping segments
and compute the Fourier transform for each segment. By using a sliding
window, we obtain a series of frequency spectra corresponding to different
time intervals. This representation is commonly referred to as a spectrogram.
The spectrogram provides a visual depiction of how the energy or

magnitude of different frequencies varies over time. Darker regions in the
spectrogram indicate higher energy or stronger frequency components.
By analyzing the spectrogram, we can observe the time-varying

characteristics of the speech signal, such as pitch variations, phonetic
transitions, and presence of different speech sounds.
MATLAB example:
% Generate a speech signal

fs = 44100; % Sampling frequency
duration = 5; % Duration of the signal in seconds
t = 0:1/fs:duration; % Time vector
f1 = 200; % Frequency of the first component
f2 = 500; % Frequency of the second component
signal = cos(2*pi*f1*t) + cos(2*pi*f2*t); % Speech signal with two tones
% Perform short-time Fourier transform (STFT)

window_length = round(0.02*fs); % Window length of 20 milliseconds
overlap = round(0.01*fs); % Overlap of 10 milliseconds
spectrogram(signal, window_length, overlap, [], fs, 'yaxis');
% Set plot labels and title
xlabel('Time');
ylabel('Frequency');
title('Spectrogram of Speech Signal');
Figure 3-2 presents a visual representation of the time-frequency analysis

conducted on a speech signal.
92 Chapter III
Figure 3-2: Time-frequency analysis of a speech signal.
Compressed sensing and sparse signal processing

In the realm of non-classical signal processing, two important techniques
that have gained significant attention are compressed sensing and sparse
signal processing. These techniques provide innovative approaches for
efficiently handling signals with limited measurements or sparse
representations.
Compressed sensing is a revolutionary technique that challenges the

traditional Nyquist-Shannon sampling theorem. It allows us to reconstruct
signals accurately from far fewer measurements than required by classical
methods. By exploiting the signal's inherent sparsity or compressibility,
compressed sensing enables the recovery of the original signal with high
fidelity, even from highly undersampled measurements. This has immense
practical implications, as it reduces the need for extensive data acquisition
and storage, making it applicable to various fields such as medical imaging,
signal acquisition, and communication systems.
The concept of sparsity is fundamental to compressed sensing and sparse

signal processing. A signal is considered sparse when most of its
information is concentrated in only a few essential components, while the
rest are negligible or nearly zero. Sparsity can be quantified using the
κ΋ norm, which counts the number of non-zero elements in the signal.
Sparse signal processing focuses on processing signals that have a sparse

representation. It leverages the sparsity property to develop efficient
algorithms for signal analysis, denoising, and compression. By exploiting
sparsity, we can reduce the complexity of signal processing tasks and
enhance our ability to extract relevant information from noisy or high-
dimensional data.
One of the fundamental problems in compressed sensing and sparse signal

processing is basis pursuit. Basis pursuit is a mathematical optimization
problem that aims to find the sparsest solution to an underdetermined
system of linear equations. The problem can be formulated as minimizing
the κΌ norm of the coefficient vector subject to the constraint that the
measurement equation is satisfied. Basis pursuit is a key tool in sparse signal
recovery and serves as the basis for many compressed sensing algorithms.
The measurement process in compressed sensing is typically represented by

a compressive sensing matrix, often denoted as ઴. This matrix allows us to
acquire the signal's compressed measurements by taking inner products
between the signal and the measurement matrix. The choice of the
compressive sensing matrix is crucial and has a significant impact on the
accuracy of signal recovery. Common examples include random matrices,
partial Fourier matrices, and wavelet-based matrices.
Another important concept in compressed sensing is the restricted isometry

property (RIP). The RIP is a mathematical property of a compressive
sensing matrix that ensures stable and robust signal recovery. It quantifies
how well a matrix preserves the distances between sparse signals in the
measurement space. If a matrix satisfies the RIP, it guarantees that the
original signal can be accurately recovered from its compressed
measurements using certain algorithms.
By understanding the principles of compressed sensing and sparse signal

processing, we unlock powerful tools to reconstruct signals from limited
measurements and process signals with sparse representations. These
techniques offer innovative ways to handle data efficiently, reduce acquisition
94 Chapter III
costs, and enable more accurate and targeted analysis in various signal
processing applications.
1. Orthogonal Matching Pursuit (OMP): OMP is an iterative

algorithm used to recover sparse signals in compressed sensing. At
each iteration, it selects the index that has the highest correlation
with the residual signal. The estimate of the signal, x, is updated
by projecting the measurements, y, onto the selected indices
(denoted by the support set S) using the pseudoinverse of the
measurement matrix ઴. This can be written as
‫( ܠ‬௞ାଵ) = arg min ||‫ ܡ‬െ ઴ௌ ‫||ܠ‬΍, (31)
where ‫( ܠ‬௞ାଵ) is the updated estimate of the signal at iteration ݇ + 1.
2. Basis Pursuit Denoising (BPDN): BPDN is a variant of basis

pursuit that incorporates noise into the signal recovery problem. It
minimizes the objective function, which includes a term for the
reconstruction error and a regularization term to promote sparsity.
The BPDN problem can be formulated as
minԡ‫ܠ‬ԡଵ subject to ||‫ ܡ‬െ ઴‫||ܠ‬΍ ൑ ߝ,
where ‫ ܠ‬is the sparse signal, ‫ ܡ‬is the measurements, ઴ is the measurement
matrix, and ߝ is the noise level.
3. Approximate Message Passing (AMP): AMP is an iterative

algorithm used for sparse signal recovery in compressed sensing.
It updates the estimates of the signal iteratively by taking into
account the noise level and the sparsity level. The update equation
for AMP can be written as
‫( ܠ‬௞ାଵ) = ߟ(઴ ் ‫ ܡ‬+ ξ݊ Ȳ(‫( ܢ‬௞) )), (32)
‫( ܢ‬௞ାଵ) = ‫( ܠ‬௞ାଵ) െ Ȳ(‫( ܢ‬௞) ), (33)
where ‫( ܠ‬௞ାଵ) and ‫( ܢ‬௞ାଵ) are the updated estimates at iteration ݇ + 1, ઴ ் is

the transpose of the measurement matrix ઴, ‫ ݕ‬is the measurements, Ȳ is the
denoising function that operates on the estimates, ߟ is a scalar normalization
factor and ݊ is the number of measurements or the dimensionality of the
signal. It denotes the size or the length of the signal being processed. The
square root of ݊ in the equation scales the denoising term Ȳ(‫( ܢ‬௞) ) to ensure
proper normalization. The value of ݊ depends on the specific signal

processing problem and the context in which the equation is used.
4. Coherence: Coherence is a measure of the similarity between the

columns of the measurement matrix in compressed sensing. It is
calculated using the inner product between any two columns of the
matrix. The coherence value, ȝ, is defined as
ߤ = max{௜ஷ௝} |‫ۦ‬઴௜ , ઴௝ ۧ|, (34)
where ઴௜ and ઴௝ are the i-th and ݆-th columns of the measurement matrix
઴.
These mathematical concepts and techniques provide the foundation for

efficient and accurate signal recovery from limited measurements and
processing of sparse signals. By utilizing algorithms such as OMP, BPDN,
and AMP, and considering properties like coherence, we can effectively
leverage the principles of compressed sensing and sparse signal processing
in various applications, leading to improved data acquisition, storage
efficiency, and targeted analysis.

In signal processing, we have techniques two. Compressed sensing and
sparse processing, it is true.
They handle signals with limited measure. And sparsity, a property to

treasure.
Compressed sensing challenges traditional ways. Reconstructing signals

with fewer samples in a blaze.
Exploiting sparsity, it brings efficiency. Reducing data needs, a practical

efficiency.
Sparse processing focuses on signals sparse. Extracting info from

components that amass.
Algorithms for analysis and denoising. Extracting meaning from noisy

inputs, rejoicing.
Basis pursuit is a key problem to solve. Finding the sparsest solution to

evolve.
96 Chapter III
Measuring norms, satisfying equations. Recovering sparse signals,

unlocking revelations.
Measurement matrices play a crucial role. In compressed sensing, they

capture the soul.
Acquiring compressed measurements with precision. Random or wavelet,

choices with vision.
Restricted isometry property is a must. Preserving distances, stability a trust.

Ensuring signal recovery is robust. From compressed measurements, it is a
must.
In summary, these techniques so profound. Handle limited measures,

signals they surround.
Efficient and accurate, they process with flair. Extracting information, from
sparse signals they care.
Key takeaways from the Compressed sensing and sparse signal processing
are given as follows:
1. Compressed sensing allows accurate signal reconstruction from a

much smaller number of measurements than traditional methods.
By exploiting the sparsity or compressibility of signals, compressed
sensing reduces the need for extensive data acquisition, storage,
and transmission.
2. Sparse signal processing focuses on signals that have a sparse
representation, where most of the information is concentrated in a
few essential components while the rest are negligible or nearly
zero. This property is leveraged to develop efficient algorithms for
signal analysis, denoising, and compression.
3. Basis pursuit is a fundamental problem in compressed sensing and
sparse signal processing. It aims to find the sparsest solution to an
underdetermined system of linear equations, where the Ɛၶ norm of
the coefficients is minimized while satisfying the measurement
equation.
4. The choice of the measurement matrix in compressed sensing is
crucial. Common examples include random matrices, partial
Fourier matrices, and wavelet-based matrices. The matrix's
properties, such as coherence and restricted isometry property
(RIP), impact the accuracy and stability of signal recovery.
5. Compressed sensing and sparse signal processing have practical

implications in various fields, including medical imaging, signal
acquisition, communication systems, and data analysis. They
enable efficient data handling, reduce acquisition costs, and
enhance the extraction of relevant information from noisy or high-
dimensional data.
6. Algorithms like orthogonal matching pursuit (OMP), basis pursuit
denoising (BPDN), and approximate message passing (AMP) play
important roles in compressed sensing and sparse signal
processing. These algorithms iteratively update signal estimates
and exploit the sparsity of signals to achieve accurate reconstruction.
7. By understanding and leveraging the principles of compressed
sensing and sparse signal processing, we gain powerful tools for
reconstructing signals from limited measurements, processing
sparse representations efficiently, and extracting meaningful
information from data in various signal processing applications.
Layman's Guide:
In non-classical signal processing, there are two important techniques called
compressed sensing and sparse signal processing. These techniques offer
new ways to handle signals that have limited measurements or sparse
representations.
Compressed sensing is a groundbreaking technique that challenges the

traditional way of sampling signals. It allows us to accurately reconstruct
signals using far fewer measurements than before. By taking advantage of
the fact that signals often have a lot of empty or negligible parts, compressed
sensing can recover the original signal even from very few measurements.
This has many practical benefits, such as reducing the need for collecting
and storing large amounts of data. It finds applications in various fields like
medical imaging, signal acquisition, and communication systems.
Sparsity is a fundamental concept in compressed sensing and sparse signal

processing. A signal is considered sparse when most of its important
information is concentrated in only a few components, while the rest are
almost zero or negligible. We can measure sparsity by counting the number
of non-zero elements in the signal.
Sparse signal processing focuses on processing signals that have this sparse
property. It uses this sparsity to develop efficient algorithms for tasks like
analyzing, denoising, and compressing signals. By exploiting sparsity, we
98 Chapter III
can simplify the complexity of processing signals and extract meaningful

information from noisy or high-dimensional data more effectively.
One key problem in compressed sensing and sparse signal processing is

basis pursuit. It is a mathematical problem that aims to find the sparsest
solution to a system of linear equations. The idea is to minimize a certain
mathematical measure called the Ɛၶ norm of the coefficients while satisfying
the given measurement equation. Basis pursuit is crucial for recovering
sparse signals and forms the basis for many compressed sensing algorithms.
In compressed sensing, we use a compressive sensing matrix ઴ to measure

the signal. This matrix helps us acquire the compressed measurements by
taking inner products between the signal and the matrix. The choice of this
matrix is crucial because it greatly affects the accuracy of the signal
recovery process. Common examples of compressive sensing matrices
include random matrices, partial Fourier matrices, and wavelet-based
matrices.
Another important concept in compressed sensing is the restricted isometry

property (RIP). It is a mathematical property of the compressive sensing
matrix that ensures stable and reliable signal recovery. The RIP measures
how well a matrix preserves the distances between sparse signals in the
measurement space. If a matrix satisfies the RIP, it guarantees that the
original signal can be accurately recovered from its compressed measurements
using certain algorithms.
Overall, compressed sensing and sparse signal processing offer innovative

ways to handle signals with limited measurements or sparse representations.
They allow us to reconstruct signals accurately and efficiently, reduce data
acquisition and storage requirements, and enable more accurate analysis in
various signal processing applications.
Exercises of compressed sensing and sparse signal

processing
Problem 1:
You are given a sparse signal represented by a vector ‫ ܠ‬of length ݊.

However, the signal is corrupted by noise, and you want to recover the
original sparse signal. Design a compressed sensing algorithm using basis
pursuit to accurately reconstruct the sparse signal from the noisy
measurements.
Solution:
To solve this problem, we can use the basis pursuit algorithm. Here is a step-
by-step solution:
1. Set up the problem:

x Define the measurement matrix ઴ of size ݉ × ݊, where m is
the number of measurements and n is the length of the signal.
The measurement matrix represents the compressed
measurements.
x Generate the noisy measurements ‫ ܡ‬by taking the inner
product of ઴ and the corrupted signal ‫ܠ‬, and add noise if
necessary.
2. Formulate the optimization problem:
x The basis pursuit algorithm aims to minimize the Ɛၶ norm of

the coefficient vector subject to the constraint that the
measurement equation ઴ times the coefficient vector equals
the measurements y.
3. Solve the optimization problem:
x Use an optimization algorithm, such as linear programming or

convex optimization, to solve the basis pursuit problem. This
will give you the sparse coefficient vector z as the solution to
the optimization problem.
4. Recover the original signal:
x Reconstruct the original sparse signal by multiplying the

measurement matrix transpose ઴ ் with the obtained sparse
coefficient vector z.
MATLAB example:
% Step 1: Set up the problem

n = 100; % Length of the signal
m = 50; % Number of measurements
Phi = randn(m, n); % Measurement matrix
% Generate a sparse signal with only 10 non-zero elements

x = zeros(n, 1);
100 Chapter III
x(randperm(n, 10)) = randn(10, 1);
% Generate noisy measurements

noise = 0.1 * randn(m, 1);
y = Phi * x + noise;
% Step 2: Solve the basis pursuit problem using lasso function

lambda = 0.1; % Regularization parameter
options = statset('Display', 'off');
x_reconstructed = lasso(Phi, y, 'Lambda', lambda, 'Options', options);
% Step 3: Plot the original signal and the reconstructed signal

figure;
subplot(2, 1, 1);
stem(x, 'b', 'filled');
title('Original Sparse Signal');
xlabel('Index');
subplot(2, 1, 2);
stem(x_reconstructed, 'r', 'filled');
title('Reconstructed Sparse Signal');
xlabel('Index');
In this code, we use the lasso function from MATLAB's Statistics and
Machine Learning Toolbox to solve the basis pursuit problem. The lasso
function performs sparse regression using the L1 regularization technique.
We specify the regularization parameter lambda to control the sparsity level
of the solution. The code then plots the original sparse signal and the
reconstructed signal for comparison.
Please note that the availability of the lasso function depends on having the
Statistics and Machine Learning Toolbox installed. If you encounter any
errors related to the lasso function, please ensure that you have the required
toolbox installed.
Machine learning and deep learning for signals

In the ever-evolving field of signal processing, the integration of machine
learning and deep learning techniques has revolutionized the way we
analyze and extract information from signals. This section delves into the
exciting realm of applying machine learning and deep learning algorithms

to signal processing tasks, opening up new possibilities and enhancing our
capabilities.
Machine learning techniques, such as support vector machines (SVMs),

provide powerful tools for signal classification and prediction. SVMs utilize
labeled training data to learn patterns and relationships within signals,
allowing them to make accurate predictions on unseen data. By training an
SVM on labeled examples, we can create a model that can distinguish
between different classes of signals, enabling tasks such as signal
recognition, speech recognition, and gesture recognition.
Deep learning, a subset of machine learning, has gained significant attention

due to its exceptional performance in various domains, including signal
processing. Deep learning models, particularly neural networks, have the
ability to learn complex representations and extract intricate features
directly from raw signals. This eliminates the need for manual feature
engineering and enables end-to-end learning. Neural networks have been
successfully applied to tasks such as speech recognition, image and video
processing, and natural language understanding, among others.
By harnessing the power of machine learning and deep learning techniques,

signal processing becomes more intelligent and adaptive. These approaches
allow us to leverage the vast amounts of data available and learn from it to
gain deeper insights into signal characteristics and behaviors. The ability to
automatically extract meaningful features and recognize patterns in signals
has numerous applications across various domains, including
telecommunications, healthcare, finance, and more. Machine learning and
deep learning offer exciting avenues for advancing the field of signal
processing and unlocking new possibilities for signal analysis and
interpretation.
Let us delve deeper into the mathematics behind machine learning models
and provide examples of MATLAB code. We start from the simplest one.
1. Linear Regression: Linear regression is a basic model used for

predicting continuous outcomes. It fits a linear equation to the
data by minimizing the sum of squared differences between the
predicted and actual values as
‫ ܠ ் ܟ = ݕ‬+ ܾ, (35)
102 Chapter III
where ‫ ݕ‬is the predicted value, ‫ ܟ‬is the weight vector, ‫ ܠ‬is the
input vector, and ܾ is the bias term.
MATLAB code example for linear regression:
% Generate input data

x = linspace(0, 10, 100); % Input vector
noise = randn(1, 100); % Gaussian noise
y_true = 2 * x + 3; % True output (without noise)
y = y_true + noise; % Observed output (with noise)
% Perform linear regression

X = [x', ones(length(x), 1)]; % Design matrix
w = pinv(X) * y'; % Weight vector
y_pred = X * w; % Predicted output
% Plot the results

figure;
scatter(x, y, 'b', 'filled'); % Scatter plot of the observed data
hold on;
plot(x, y_true, 'r', 'LineWidth', 2); % True line
plot(x, y_pred, 'g', 'LineWidth', 2); % Predicted line
xlabel('x');
ylabel('y');
title('Linear Regression');
legend('Observed Data', 'True Line', 'Predicted Line');
Figure 3-3 exhibits a concrete example of a linear regression.

Figure 3-3: An example of a linear regression.
In this example, we generate synthetic data for the input vector x, the true
output y_true (without noise), and the observed output y (with added
Gaussian noise). We then perform linear regression by creating a design
matrix X that includes the input vector x and a column of ones (to account
for the bias term). The weight vector w is calculated using the pseudo-
inverse of X multiplied by the observed output y. Finally, we use the weight
vector to compute the predicted output y_pred.
The code then plots the observed data as scatter points, the true line (without
noise), and the predicted line based on the linear regression model. The x-
axis represents the input vector x, and the y-axis represents the output y.
The legend indicates the different elements in the plot.
You can run this code in MATLAB to generate the signal and visualize the
results of the linear regression.
104 Chapter III
2. Logistic Regression: Logistic regression is a classification model

used when the outcome is binary. It models the probability of an
event occurring using the logistic function as
‫ = ݌‬1 / (1 + exp(െ(‫ ܠ ் ܟ‬+ ܾ))), (36)
where ‫ ݌‬is the predicted probability, ‫ ܟ‬is the weight vector, ‫ ܠ‬is
the input vector, and b is the bias term.
MATLAB code example for logistic regression:

% Load the sigmoid function from an external file
addpath('path_to_folder'); % Replace 'path_to_folder' with the
actual path to the folder containing the sigmoid function file
% Generate input data

x = linspace(-5, 5, 100); % Input vector
p_true = sigmoid(2 * x - 1); % True probability (without noise)
y_true = rand(1, 100) < p_true; % True binary labels (0 or 1)
% Perform logistic regression

X = [x', ones(length(x), 1)]; % Design matrix
w = glmfit(X, y_true', 'binomial', 'link', 'logit'); % Weight vector
y_pred = glmval(w, X, 'logit'); % Predicted probabilities
% Plot the results

figure;
scatter(x, y_true, 'b', 'filled'); % Scatter plot of the true labels
hold on;
plot(x, y_pred, 'r', 'LineWidth', 2); % Predicted probabilities
xlabel('x');
ylabel('Probability');
title('Logistic Regression');
legend('True Labels', 'Predicted Probabilities');
In the code, replace 'path_to_folder' with the actual path to the folder that
contains the sigmoid function file. This ensures that the sigmoid function is
properly loaded for use in the code.
Please make sure that you have the sigmoid function defined in a separate
MATLAB file (e.g., sigmoid.m) and placed in the specified folder. The
sigmoid function should have the following code:
function y = sigmoid(x)
y = 1 ./ (1 + exp(-x));
end
Figure 3-4 demonstrates a practical illustration of a logistic regression.
Figure 3-4: An example of a logistic regression.
3. Decision Tree: Decision tree is a hierarchical model that makes

decisions based on a series of if-else conditions. A decision tree
partitions the input space based on a series of if-else conditions. It
recursively splits the data based on the input features, creating
branches that represent different decision paths. Each internal node
in the tree corresponds to a decision rule, and each leaf node
represents a predicted outcome.
MATLAB code example for a decision tree:

% Generate sample data
X = [1 2; 2 3; 3 4; 4 5]; % Input features
Y = [0; 0; 1; 1]; % Binary labels
106 Chapter III
% Train a decision tree classifier

tree = fitctree(X, Y);
% Plot the decision tree

view(tree, 'Mode', 'Graph');
% Predict labels for new data

newX = [1.5 2.5; 3.5 4.5]; % New input features
predictedLabels = predict(tree, newX);
% Display the predicted labels

disp(predictedLabels);
In the code example, we first generate some sample data (X) with
corresponding binary labels (Y). We then use the fitctree function to train
a decision tree classifier on the data. The fitctree function automatically
splits the data based on the input features and creates the decision tree.
We can visualize the decision tree using the view function with the 'Mode',
'Graph' option, which displays the tree as a directed graph. The resulting
graph shows the decision rules and splits at each internal node.
To make predictions for new data (newX), we use the predict function with
the trained decision tree. The predict function returns the predicted labels
based on the learned decision rules.
Finally, the predicted labels are displayed using the disp function.
Please note that the example provided uses a simple dataset for
demonstration purposes. In practice, decision trees can handle datasets with
multiple features and larger sample sizes. The MATLAB functions fitctree
and predict provide various options and parameters to customize the
decision tree's behavior and performance.
% Display the predicted labels

disp(predictedLabels);
0
0
Figure 3-5 visually showcases the predicted label of an example of a

decision tree.
Figure 3-5: An example of a decision tree.
4. Random Forest: Random forest is an ensemble model that

combines multiple decision trees. They generate predictions by
aggregating the predictions of individual trees or it is called “an
ensemble model.” Each tree in the forest is built on a random
subset of the training data and a random subset of the input
features. The predictions of individual trees are then aggregated to
make the final prediction.
The general steps to build a Random Forest model are as follows:
1. Random Sampling: Randomly select a subset of the training

data with replacement (bootstrap sampling). This creates
multiple subsets of the original data, each of which is used to
train a decision tree.
2. Random Feature Selection: At each node of the decision
tree, randomly select a subset of input features. This helps to
introduce diversity among the trees in the forest.
3. Tree Building: For each subset of data and feature subset,
build a decision tree using a specified algorithm (such as the
one used in regular decision trees).
4. Aggregation: To make predictions, aggregate the predictions
of all the trees in the forest. For classification tasks, this can
be done by majority voting, where the class with the most
108 Chapter III
votes is selected. For regression tasks, the average of the

predicted values can be taken.
Mathematically, the prediction of a Random Forest model can be

represented as
For classification: ‫݌(݁݀݋݉ = ݌‬ଵ , ‫݌‬ଶ , . . . , ‫݌‬௞ ), (37)
For regression: ‫ݕ(݊ܽ݁݉ = ݕ‬ଵ , ‫ݕ‬ଶ , . . . , ‫ݕ‬௡ ), (38)
where ݉‫ ݁݀݋‬refers to the statistical term for the mode of a set of

values, representing the value(s) that occur(s) most frequently in
the set. ݉݁ܽ݊ refers to the statistical term for the arithmetic mean
or average of a set of values. ‫ ݌‬is the predicted class label, ‫݌‬௜ is the
predicted class label of the ݅-th tree, ‫ ݕ‬is the predicted value, and
‫ݕ‬௜ is the predicted value of the ݅-th tree.
Note that the specific algorithms used to build decision trees and
aggregate predictions in Random Forests can vary. The above
formulation represents the general idea behind Random Forests.
MATLAB code example for a random forest:
% Generate Synthetic Dataset

numSamples = 10000;
X = rand(numSamples, 2); % Randomly generate feature values
Y = (X(:, 1) > 0.5) & (X(:, 2) > 0.5); % Define labels based on
feature conditions
% Train Random Forest Classifier

numTrees = 100;
model = TreeBagger(numTrees, X, Y);
% Compute Decision Boundaries

step = 0.01;
x1range = 0:step:1;
x2range = 0:step:1;
[X1, X2] = meshgrid(x1range, x2range);
XGrid = [X1(:), X2(:)];
YGrid = predict(model, XGrid);
YGrid = cellfun(@(x) str2double(x), YGrid);
% Plot Decision Boundaries

figure;
contourf(X1, X2, reshape(YGrid, size(X1)), 'LineWidth', 2);
colormap('cool');
hold on;
scatter(X(:,1), X(:,2), 30, Y, 'filled', 'MarkerEdgeColor', 'k');
hold off;
title('Random Forest Classifier Decision Boundaries');
xlabel('Feature 1');
ylabel('Feature 2');
legend('Decision Boundaries', 'Actual Class');
In this code, after training the Random Forest classifier and computing the
decision boundaries, the contourf function is used to plot the decision
boundaries as filled contour regions. The scatter function is used to plot the
actual data points, with different colors representing the two classes. The
resulting plot shows the decision boundaries learned by the Random Forest
classifier and the distribution of the data points in the feature space.
Feel free to adjust the numSamples and numTrees parameters to generate

different datasets and explore the effect of the number of trees on the
decision boundaries.
Figure 3-6 visually presents the decision boundaries of an example random

forest classifier.
110 Chapter III
Figure 3-6: An example of a random forest classifier.
5. Naive Bayes: Naive Bayes is a probabilistic classification model

based on Bayes' theorem. It assumes that the features are
conditionally independent given the class label. The model
calculates the probability of each class given a set of features and
selects the class with the highest probability as the predicted class.
Let us consider a binary classification problem with two classes,

labeled as class 0 and class 1. Given a feature vector ‫= ݔ‬
[‫ݔ‬ଵ , ‫ݔ‬ଶ , . . . , ‫ݔ‬௡ ], the Naive Bayes model calculates the probability
of each class ܲ(݈ܿܽ‫ = ݏݏ‬0|‫ )ݔ‬and ܲ(݈ܿܽ‫ = ݏݏ‬1|‫ )ݔ‬and selects the
class with the highest probability.
The Bayes' theorem can be written as
ܲ(݈ܿܽ‫))ݏݏ݈ܽܿ(ܲ)ݏݏ݈ܽܿ|ݔ(ܲ( = )ݔ|ݏݏ‬/ܲ(‫)ݔ‬. (39)
In Naive Bayes, we assume that the features are conditionally

independent given the class label. Using this assumption, the
probability P(x|class) can be factorized as
ܲ(‫ݔ(ܲ = )ݏݏ݈ܽܿ|ݔ‬ଵ |݈ܿܽ‫ݔ(ܲ)ݏݏ‬ଶ |݈ܿܽ‫ )ݏݏ‬. . . ܲ(‫ݔ‬௡ |݈ܿܽ‫)ݏݏ‬. (40)
The class probabilities ܲ(݈ܿܽ‫ )ݏݏ‬can be estimated from the training

data, and the feature probabilities ܲ(‫ݔ‬௜ |݈ܿܽ‫ )ݏݏ‬can be estimated
using various probability density estimation techniques, such as
Gaussian distribution or multinomial distribution, depending on
the type of features.
MATLAB code example for a Naive Bayes classifier:
% Generate Signal Data

numSamples = 1000;
x = linspace(0, 10, numSamples)';
y = sin(x) + randn(numSamples, 1) * 0.2;
classLabels = y > 0;
% Train Naive Bayes Classifier

nbModel = fitcnb(x, classLabels);
% Generate Test Data

xTest = linspace(0, 10, 1000)';
yTest = sin(xTest) + randn(1000, 1) * 0.2;
% Predict Class Labels

predictedLabels = predict(nbModel, xTest);
% Plot Results
figure;
scatter(x, y, 10, classLabels, 'filled');
hold on;
scatter(xTest, yTest, 10, predictedLabels, 'x');
hold off;
title('Naive Bayes Classification');
xlabel('x');
ylabel('y');
legend('Training Data', 'Test Data');
Figure 3-7 displays the results obtained from training and testing data in an
example of a Naive Bayes classifier.
112 Chapter III
Figure 3-7: An example of a Naive Bayes classifier.
6. K-Nearest Neighbors (KNN): K-nearest neighbors (KNN) is a

non-parametric classification algorithm that predicts the class of a
data point based on the majority vote of its nearest neighbors in the
feature space. The algorithm determines the class of an unseen data
point by comparing it to the labeled data points in the training set.
Here is how KNN works:
1. Given a training dataset with labeled data points and an unseen

data point to be classified:
x Each data point is represented by a set of features
(attributes) and belongs to a specific class.
2. The algorithm calculates the distance between the unseen data

point and all the labeled data points in the training set.
x The most commonly used distance metric is
Euclidean distance, but other distance metrics can
also be used.
3. It selects the K nearest neighbors (data points with the smallest

distances) to the unseen data point.
4. The algorithm determines the class of the unseen data point

based on the majority vote of the classes of its K nearest
neighbors.
x Each neighbor gets a vote, and the class with the most
votes is assigned to the unseen data point.
Let us assume we have a training dataset with ܰ labeled data

points, each having ‫ ܯ‬features. The KNN algorithm can be
summarized as follows:
Given an unseen data point ‫ݔ‬, the algorithm follows these steps:
1. Compute the Euclidean distance between x and each

labeled data point in the training set as
݀௜ =
ඥ((‫ݔ‬ଵ െ ‫ݔ‬ଵ௜ )ଶ + (‫ݔ‬ଶ െ ‫ݔ‬ଶ௜ )ଶ +. . . +(‫ݔ‬ெ െ ‫ݔ‬ெ௜ )ଶ ) , (41)
where (‫ݔ‬ଵ௜ , ‫ݔ‬ଶ௜ , . . . , ‫ݔ‬ெ௜ ) are the feature values of the ݅-th
labeled data point.
2. Select the K nearest neighbors based on the smallest

distances.
3. Determine the class of the unseen data point by majority

voting among the classes of its K nearest neighbors.
MATLAB code example for a K-nearest neighbors (KNN):
% Generate example dataset

X = rand(100, 2); % Feature matrix
Y = randi([1, 3], 100, 1); % Class labels
% Split the dataset into training and testing sets

trainRatio = 0.7; % Set the training set ratio
[trainX, testX, trainY, testY] = splitDataset(X, Y, trainRatio);
% Create a KNN classifier

knn = fitcknn(trainX, trainY, 'NumNeighbors', 3); % Specify the
number of neighbors (K) as 3
% Predict the classes of the test set
predictedY = predict(knn, testX);
114 Chapter III
% Calculate the accuracy

accuracy = sum(predictedY == testY) / numel(testY);
% Plot the accuracy

figure;
bar(accuracy);
ylim([0 1]);
xlabel('Test Set');
ylabel('Accuracy');
title('K-Nearest Neighbors Accuracy');
% Plot the training data points

figure;
gscatter(X(:, 1), X(:, 2), Y, 'rgb', 'o');
hold on;
% Plot the new data point and its predicted class

scatter(newData(1), newData(2), 'filled', 'MarkerEdgeColor', 'k',
'MarkerFaceColor', 'y');
text(newData(1), newData(2), ['Predicted class: '
num2str(predictedClass)], 'VerticalAlignment', 'bottom',
'HorizontalAlignment', 'right');
% Set plot labels and title

title('K-Nearest Neighbors Classification');
% Set legend
legend('Class 1', 'Class 2', 'Class 3', 'New Data Point');
% Set plot limits

xlim([0 1]);
ylim([0 1]);
Figure 3-8 presents an example of a K-Nearest Neighbors (KNN) with two

subplots: a) The first subplot showcases the relationship between the test set
and the corresponding accuracy of the KNN model and b) The second
subplot exhibits a scatter plot representing the distribution of feature 1
versus feature 2 in the dataset used for a KNN classifier.
a)
b)
Figure 3-8: An example of a K-nearest neighbor (KNN) classifier: a) test set versus
accuracy and b) scatter plot of feature 1 versus feature 2.
116 Chapter III
7. Support Vector Machine (SVM): Support vector machine

(SVM) is a linear model that aims to find the best hyperplane that
separates the data into different classes. The hyperplane is chosen
in a way that maximizes the margin between the classes, thus
improving the model's ability to generalize to unseen data.
The decision function of an SVM is defined as
݂(‫ ܠ ் ܟ = )ݔ‬+ ܾ, (42)
where ݂(‫ )ݔ‬represents the decision function for a given input

vector ‫ܠ‬, ‫ ܟ‬is the weight vector, and ܾ is the bias term. The SVM
aims to find the optimal hyperplane by solving the following
optimization problem:
minimize: 0.5 ||‫||ܟ‬ଶ + ‫(ݔܽ݉(ߑܥ‬0, 1 െ ‫ݕ‬௜ (‫ ܠ ் ܟ‬௜ + ܾ)))

subject to: ‫ݕ‬௜ (‫ ܠ ் ܟ‬௜ + ܾ) ൒ 1,
where ||‫ ||ܟ‬represents the ‫ܮ‬2 norm of the weight vector, ‫ ܥ‬is the
regularization parameter that balances the trade-off between
achieving a larger margin and minimizing the classification errors,
‫ݕ‬௜ represents the class label of the training sample ‫ ܠ‬௜ , and the
summation is performed over all training samples.
MATLAB code example for a support vector machine (SVM):
% Generate example signals (replace with your own signals)

numSamples = 1000;
numFeatures = 2;
numClasses = 3;
% Generate random signals for each class

class1 = randn(numSamples, numFeatures) + 1;
class2 = randn(numSamples, numFeatures) - 1;
class3 = randn(numSamples, numFeatures);
% Plot the signals for each class

figure;
scatter(class1(:, 1), class1(:, 2), 'r');
hold on;
scatter(class2(:, 1), class2(:, 2), 'g');
scatter(class3(:, 1), class3(:, 2), 'b');
title('Random Signals for Each Class');
legend('Class 1', 'Class 2', 'Class 3');
% Concatenate the signals and create labels

X = [class1; class2; class3];
Y = [ones(numSamples, 1); 2 * ones(numSamples, 1); 3 *
ones(numSamples, 1)];
% Perform feature scaling

X = zscore(X);
% Perform feature selection using PCA

coeff = pca(X);
numSelectedFeatures = 1; % Number of selected features
selectedFeatures = coeff(:, 1:numSelectedFeatures);
X = X * selectedFeatures;

rng(1); % Set the random seed for reproducibility
[trainX, testX, trainY, testY] = splitDataset(X, Y, 0.7);
% Adjust the kernel

kernel = 'rbf'; % Change the kernel type (e.g., 'linear', 'polynomial',
'rbf')
svmModel = fitcecoc(trainX, trainY, 'Coding', 'onevsall',
'Learners', templateSVM('KernelFunction', kernel));
% Perform cross-validation
cvModel = crossval(svmModel);
predictedY_all = kfoldPredict(cvModel);
% Get the predictions for the test samples

numTestSamples = numel(testY);
predictedY_test = predictedY_all(1:numTestSamples);
% Compute the element-wise comparison

comparison = (predictedY_test == testY);
118 Chapter III
% Compute the accuracy

accuracy = sum(comparison) / numTestSamples * 100;
% Test with unlabeled signal

X_unlabeled = [2 -2]; % Replace with your own unlabeled signal
X_unlabeled = (X_unlabeled - mean(X)) ./ std(X); % Apply
feature scaling
X_unlabeled = X_unlabeled * selectedFeatures; % Apply feature
selection
predictedClass_unlabeled = predict(svmModel, X_unlabeled);
% Compare the predicted label with the expected label for the
unlabeled signal
expectedLabel_unlabeled = 3; % Replace with the expected label
for the unlabeled signal
isCorrect_unlabeled = (predictedClass_unlabeled ==
expectedLabel_unlabeled);
% Calculate the accuracy for the unlabeled signal

accuracy_unlabeled = sum(isCorrect_unlabeled) /
numel(predictedClass_unlabeled) * 100;
% Plot the comparison result for the unlabeled signal

figure;
bar(isCorrect_unlabeled);
title('Element-wise Comparison (Unlabeled Signal)');
xlabel('Label');
ylabel('Comparison Result');
xticks(1:numel(predictedClass_unlabeled));
xticklabels(predictedClass_unlabeled);
% Display the accuracy for the unlabeled signal

disp(['Accuracy (Unlabeled Signal): '
num2str(accuracy_unlabeled) '%']);
Figure 3-9 illustrates an example of a support vector machine (SVM) with

two subplots: a) The first subplot presents a scatter plot depicting the
distribution of feature 1 versus feature 2 in the dataset used for SVM
classification and b) The second subplot showcases the element-wise
comparison of an unlabeled signal, providing a visual representation of the
comparison results.
a)
b)
Figure 3-9: An example of a support vector machine (SVM): a) scatter plot of feature
1 versus feature 2 and b) element-wise comparison of unlabeled signal.
120 Chapter III
Support Vector Regression (SVR): Support vector regression (SVR) is a

supervised machine learning algorithm that is used for regression tasks. It
is an extension of support vector machines (SVM) for classification.
The goal of SVR is to find a function that best approximates the mapping
between input variables (features) and the corresponding continuous target
variable. It aims to minimize the prediction error while also controlling the
complexity of the model. SVR is particularly useful when dealing with
nonlinear relationships and can handle both linear and nonlinear regression
problems.
Here are the key components and concepts of SVR:
1. Support Vectors: In SVR, the algorithm identifies a subset of

training samples called support vectors. These are the samples that
are closest to the decision boundary or have a non-zero error (also
known as slack variables). The support vectors are critical for
defining the regression model.
2. Margin: Similar to SVM, SVR aims to maximize the margin
around the regression line or hyperplane. The margin represents
the region where the model is confident that the predictions will
fall within a certain error tolerance. SVR allows for a specified
margin of tolerance around the predicted values.
3. Kernel Functions: SVR utilizes kernel functions to transform the
input features into a higher-dimensional space. This transformation
can enable the algorithm to find nonlinear relationships between
the features and the target variable. Common kernel functions used
in SVR include linear, polynomial, radial basis function (RBF),
and sigmoid.
4. Epsilon: SVR introduces a parameter called epsilon (ߝ), which
defines the width of the margin or the acceptable error tolerance.
It determines the zone within which errors are considered
negligible and do not contribute to the loss function.
5. Loss Function: SVR uses a loss function to measure the deviation
between the predicted and actual target values. The most
commonly used loss function in SVR is the epsilon-insensitive loss
function, which penalizes errors outside the epsilon tube (margin)
while ignoring errors within the tube.
6. Regularization: SVR incorporates regularization to control the
complexity of the model and prevent overfitting. The
regularization parameter (‫ )ܥ‬balances the trade-off between
minimizing the training error and minimizing the model
complexity. A larger ‫ ܥ‬value allows for a smaller margin and

potentially more support vectors, leading to a more complex
model.
The SVR algorithm aims to find the optimal regression function by solving
a convex optimization problem. This involves minimizing the loss function
while satisfying the margin constraints and incorporating regularization.
Various optimization techniques can be used, such as quadratic programming
or gradient descent.
Overall, SVR is a versatile regression algorithm that can effectively handle

nonlinear relationships and outliers in the data. It provides a flexible
approach to regression tasks and has been successfully applied in various
domains, including finance, economics, and engineering.
MATLAB code example for a support vector regression (SVR):
% Generate synthetic signals

t = linspace(0, 10, 1000)'; % Time vector
f1 = sin(2*pi*0.5*t); % Signal 1: Sine wave with frequency 0.5 Hz
f2 = 0.5*sin(2*pi*2*t); % Signal 2: Sine wave with frequency 2 Hz
f3 = 0.2*cos(2*pi*1.5*t); % Signal 3: Cosine wave with frequency 1.5 Hz
% Combine the signals

y = f1 + f2 + f3;
% Add noise to the combined signal

rng(1); % For reproducibility
noise = 0.1*randn(size(t));
y_noisy = y + noise;
% Plot the signals

figure;
subplot(2, 1, 1);
plot(t, y, 'LineWidth', 2);
xlabel('Time');
title('Combined Signal');
legend('f1', 'f2', 'f3', 'Location', 'northwest');
subplot(2, 1, 2);
plot(t, y_noisy, 'LineWidth', 1);
122 Chapter III
xlabel('Time');
% Prepare the data for SVR training

X = t; % Input features
y_target = y; % Target variable
% Split the data into training and testing sets

cv = cvpartition(size(X, 1), 'HoldOut', 0.3);
Xtrain = X(training(cv), :);
ytrain = y_target(training(cv), :);
Xtest = X(test(cv), :);
ytest = y_target(test(cv), :);
% Train the SVR model

svrModel = fitrsvm(Xtrain, ytrain, 'KernelFunction', 'gaussian',
'BoxConstraint', 1);
% Predict on the test data

ypred = predict(svrModel, Xtest);
% Evaluate the model

mse = mean((ytest - ypred).^2); % Mean Squared Error
r2 = 1 - (sum((ytest - ypred).^2) / sum((ytest - mean(ytest)).^2)); % R-
squared

disp(['Mean Squared Error: ' num2str(mse)]);
disp(['R-squared: ' num2str(r2)]);
% Plot the predicted values against the true values

figure;
plot(t(test(cv)), ytest, 'b', 'LineWidth', 2);
hold on;
plot(t(test(cv)), ypred, 'r--', 'LineWidth', 1);
xlabel('Time');
title('SVR Prediction');
legend('True Values', 'Predicted Values');
Mean Squared Error: 0.15612

R-squared: 0.76464
Figure 3-10 showcases an example of a support vector regression (SVR)

with two subplots: a) The first subplot displays the combined signal and a
noisy signal. This plot provides a visual comparison between the original
signal and its noisy version and b) The second subplot presents the SVR
prediction, demonstrating the regression results obtained by the SVR model.
a)
124 Chapter III
b)
Figure 3-10: Example of support vector regression (SVR): a) combined signal and
noisy signal and b) SVR prediction.
Gradient Boosting Machine (GBM): Gradient boosting machine (GBM)

is a popular class of machine learning algorithms that are used for both
regression and classification tasks. GBMs build an ensemble of weak
prediction models, typically decision trees, and combine their predictions to
create a strong predictive model.
Here is an explanation of the main components and steps involved in

Gradient Boosting Machines:
1. Weak Learners: GBMs use a collection of weak prediction

models, often referred to as weak learners or base learners. These
weak learners are typically decision trees, although other types of
models can also be used. Decision trees are used because they can
capture non-linear relationships and interactions between features.
2. Boosting: GBMs employ a boosting technique, which involves
training weak learners sequentially, with each subsequent learner
correcting the mistakes made by the previous ones. Boosting
focuses on instances that were incorrectly predicted by the
previous learners, assigning higher weights to these instances to

improve the model's performance.
3. Gradient Descent: The name “gradient” in Gradient Boosting
Machines refers to the use of gradient descent optimization to
iteratively minimize a loss function. In each iteration, the weak
learner is trained to fit the negative gradient of the loss function
with respect to the current model's predictions. This process is
repeated for a specified number of iterations or until a predefined
stopping criterion is met.
4. Ensemble Learning: GBMs combine the predictions of multiple
weak learners to obtain a final prediction. Each weak learner
contributes to the ensemble based on its individual strength. The
final prediction is typically calculated by aggregating the
predictions of all weak learners, often using weighted averages.
MATLAB code example for a gradient boosting machine (GBM):
% Generate synthetic signals

t = linspace(0, 10, 1000)'; % Time vector
f1 = sin(2*pi*0.5*t); % Signal 1: Sine wave with frequency 0.5 Hz
f2 = 0.5*sin(2*pi*2*t); % Signal 2: Sine wave with frequency 2 Hz
f3 = 0.2*cos(2*pi*1.5*t); % Signal 3: Cosine wave with frequency 1.5 Hz
% Combine the signals

y = f1 + f2 + f3;
% Add noise to the combined signal

noise = 0.1*randn(size(t));
y_noisy = y + noise;
% Prepare the data for GBM training

X = t; % Input features
y_target = y; % Target variable
% Split the data into training and testing sets

cv = cvpartition(size(X, 1), 'HoldOut', 0.3);
Xtrain = X(training(cv), :);
ytrain = y_target(training(cv), :);
Xtest = X(test(cv), :);
ytest = y_target(test(cv), :);
126 Chapter III
% Train the GBM model

gbmModel = fitensemble(Xtrain, ytrain, 'LSBoost', 100, 'Tree');
% 'LSBoost' specifies Least Squares Boosting as the boosting method
% 100 specifies the number of weak learners (decision trees)
% Predict on the test data

ypred = predict(gbmModel, Xtest);
% Evaluate the model

mse = mean((ytest - ypred).^2); % Mean Squared Error
r2 = 1 - (sum((ytest - ypred).^2) / sum((ytest - mean(ytest)).^2)); % R-
squared

disp(['Mean Squared Error: ' num2str(mse)]);
disp(['R-squared: ' num2str(r2)]);
% Plot the true values and predicted values

figure;
plot(Xtest, ytest, 'b', 'LineWidth', 2);
hold on;
plot(Xtest, ypred, 'r--', 'LineWidth', 2);
title('True Values vs Predicted Values');
xlabel('Time');
ylabel('Signal');
legend('True Values', 'Predicted Values');
Mean Squared Error: 0.080433
R-squared: 0.87874
In this example, we generate three synthetic signals (sine waves and a cosine
wave) and combine them to create a target signal. We then add some
Gaussian noise to the combined signal. The input features (X) are the time
vector, and the target variable (y_target) is the combined signal.
We split the data into training and testing sets using a hold-out method.
Next, we train the GBM model using the fitensemble function with the
“LSBoost” method and 100 weak learners (decision trees). We then make
predictions on the test data using the trained model.
Finally, we evaluate the model's performance by calculating the mean

squared error (MSE) and R-squared values. The results are displayed, and a
plot is generated to compare the true values and predicted values.
Figure 3-11 depicts an example of a gradient boosting machine (GBM).
Figure 3-11: An example of a gradient boosting machines (GBM).

128 Chapter III
Figure 3-12: An example of neural networks (NN) architecture.
You can modify the example by adjusting the signal frequencies, adding
more signals, changing the noise level, or exploring different boosting
methods and parameters in the GBM model.
8. Neural Networks (NN): Neural networks (NN), specifically deep

learning models, have gained immense popularity in signal
processing due to their ability to learn complex representations
directly from raw signals.
Here is an overview of neural networks:
1. Architecture: Neural networks consist of interconnected

layers of artificial neurons, also known as nodes. These nodes
are organized into three main types of layers: input layer,
hidden layers, and output layer as depicted in Figure 3-12.
The input layer is responsible for receiving the input data, such
as images, text, or numerical features. Each node in the input
layer represents a feature or input variable.
The hidden layers, as the name suggests, are the layers that
come between the input and output layers. They are
responsible for performing computations and extracting
relevant features from the input data. Each node in the hidden
layers takes the weighted sum of its inputs, which are the
outputs from the previous layer. The weights represent the
strength or importance of the connections between the nodes.
The weighted sum is then passed through an activation
function, which introduces non-linearities into the network.
The activation function determines the output or activation
level of the node based on the weighted sum. Common
activation functions include sigmoid, tanh, ReLU (Rectified
Linear Unit), and softmax.
The output layer is the final layer of the neural network. It

produces the final outputs or predictions based on the
information learned by the hidden layers. The number of
nodes in the output layer depends on the nature of the problem.
For example, in a binary classification problem, there would
be one node in the output layer representing the probability of
belonging to one class, while in a multi-class classification
problem, there would be multiple nodes representing the
probabilities of each class.
During the training process, the neural network adjusts the

weights and biases of the connections between nodes based on
the provided training data and the desired outputs. This
adjustment is done through a process called backpropagation,
which uses gradient descent optimization to minimize the
difference between the predicted outputs and the actual
outputs.
By iteratively adjusting the weights and biases, neural

networks can learn to make accurate predictions or
classifications on unseen data. They have the ability to capture
complex relationships and patterns in the data, which makes
them powerful tools for solving various machine learning
tasks.
2. Training: In the training process of a neural network, labeled

data is used to adjust the weights and biases of the neurons,
130 Chapter III
enabling the network to make accurate predictions or

classifications.
This adjustment of weights and biases is performed through a

process called backpropagation. Backpropagation is a
technique for computing the gradient of the loss function with
respect to the network's parameters, which are the weights and
biases. It calculates how much each weight and bias contribute
to the overall error of the network.
The training starts with the initialization of random weights

and biases. Then, for each training example, the input data is
fed into the network, and the predicted output is calculated.
The difference between the predicted output and the actual
output (known as the loss or error) is computed using a
suitable loss function, such as mean squared error for
regression or cross-entropy loss for classification.
Once the loss is obtained, the backpropagation algorithm

propagates this error back through the network. It calculates
the gradient of the loss function with respect to each weight
and bias by applying the chain rule of calculus. The gradients
indicate how much each weight and bias needs to be adjusted
to reduce the overall error of the network.
After calculating the gradients, an optimization algorithm,

such as stochastic gradient descent (SGD), is used to update
the weights and biases. SGD iteratively adjusts the weights
and biases in the opposite direction of the gradients, aiming to
minimize the loss function. This process is repeated for all the
training examples in the dataset, and multiple passes through
the entire dataset, known as epochs, are performed to refine
the network's parameters.
During the training process, the network learns to make more

accurate predictions by iteratively adjusting the weights and
biases based on the provided labeled data. The goal is to
minimize the difference between the predicted outputs and the
actual outputs, ultimately improving the network's performance
on unseen data.
By iteratively updating the weights and biases using

backpropagation and optimization algorithms, the neural
network gradually learns to capture the underlying patterns

and relationships in the labeled data, enabling it to generalize
and make accurate predictions on new, unlabeled data.
In summary, the training of a neural network involves using

labeled data to adjust the weights and biases of the neurons.
Backpropagation is used to compute the gradients of the loss
function with respect to the parameters, and optimization
algorithms like stochastic gradient descent are employed to
iteratively update the weights and biases. This iterative
process aims to minimize the error between predicted and
actual outputs, allowing the network to learn and improve its
performance on unseen data.
2. Deep Learning: In deep learning, neural networks with

multiple hidden layers are used to perform learning tasks.
These networks are called “deep” because they consist of
several layers stacked on top of each other, forming a deep
architecture. Each layer in the network consists of multiple
nodes or neurons. The key idea behind deep learning is that
these multiple hidden layers allow the network to learn
hierarchical representations of the input data. Each layer
learns to extract and represent different levels of abstraction
or features from the input. Lower layers capture basic features,
such as edges or simple patterns, while higher layers capture
more complex patterns or combinations of lower-level
features.
By learning hierarchical representations, deep networks can

capture and understand complex patterns and relationships in
signals. This enables them to effectively handle and analyze
high-dimensional data, such as images, speech, and text.
The ability of deep networks to automatically learn features

and hierarchies of representations makes them powerful tools
for tasks such as image classification, object detection, natural
language processing, and many others. Deep learning has
achieved remarkable success in various domains, pushing the
boundaries of what can be achieved with machine learning.
Overall, deep learning leverages the depth and hierarchical

structure of neural networks to learn and extract meaningful
132 Chapter III
representations from data, allowing for more effective and

accurate analysis of complex signals.
Figure 3-13: An example of a convolutional neural network (CNN)

architecture.
3. Convolutional Neural Network (CNN): Convolutional

neural networks (CNN) is a powerful type of deep learning
model commonly used for image and video processing tasks.
They are specifically designed to capture and extract
meaningful features from visual data.
The architecture of a CNN consists of multiple layers,

including convolutional layers, pooling layers, and fully
connected layers as illustrated in Figure 3-13. Let us explore
each of these layers in more detail as follows:
Convolutional Layers: These layers are the core building

blocks of CNNs. They apply a set of learnable filters or kernels
to the input image, performing convolution operations. Each
filter scans the input image in small regions, capturing local
patterns and features. The output of each filter is called a
feature map, which represents the presence of specific features
in different spatial locations of the input image. Convolutional
layers learn to extract low-level features (such as edges,
corners, and textures) in the early layers and high-level
features (such as objects and complex patterns) in the deeper
layers.
Pooling Layers: Pooling layers follow convolutional layers

and help reduce the spatial dimensions of the feature maps
while retaining important information. The most commonly
used pooling operation is max pooling, which selects the
maximum value from a small region of the feature map.

Pooling helps in reducing the number of parameters in the
network, making it computationally efficient, and also aids in
creating translation-invariant representations. It helps to
capture the presence of features regardless of their precise
location in the image.
Fully Connected Layers: Fully connected layers, also known

as dense layers, are traditional neural network layers where
each neuron is connected to every neuron in the previous layer.
These layers are typically placed at the end of the CNN
architecture. The purpose of fully connected layers is to
classify the features extracted by the convolutional and
pooling layers into different classes. They combine the
extracted features from the previous layers and map them to
the desired output classes using various activation functions
(e.g., softmax for classification tasks).
Overall, the architecture of CNNs allows them to

automatically learn hierarchical representations of visual data,
capturing both low-level and high-level features. This makes
them highly effective for tasks such as image classification,
object detection, image segmentation, and more.
It is important to note that while CNNs are most commonly

associated with image and video processing, they can also be
applied to other types of grid-like structured data, such as
spectrograms for audio processing or text data represented as
image-like structures.
MATLAB code example for a convolutional neural network

(CNN):
% Generate random input data

inputData = randn(32, 32, 3); % Random input data of size
32x32x3 (RGB image)
% Create a simple CNN architecture

layers = [
imageInputLayer([32 32 3])
convolution2dLayer(3, 16)
reluLayer
134 Chapter III
maxPooling2dLayer(2)
fullyConnectedLayer(10)
softmaxLayer
classificationLayer('Classes', categorical(1:10)) % Specify
the classes here
];
% Set the AverageImage property of the image input layer

layers(1).AverageImage = reshape(mean(inputData, [1, 2]), 1,
1, 3);
% Initialize the weights and biases of the convolutional layer

layers(2).Weights = randn([3 3 3 16]);
layers(2).Bias = randn([1 1 16]);
% Initialize the weights and biases of the fully connected layer

layers(5).Weights = randn([10 29*29*16]); % Adjust input
size here
layers(5).Bias = randn([10 1]);
% Create a CNN model from the layers

model = assembleNetwork(layers);
% Perform forward pass through the CNN

output = predict(model, inputData);
% Plot the results

figure;
subplot(1, 2, 1);
imshow(inputData);
title('Input Image');
subplot(1, 2, 2);
bar(output);
title('Output Probabilities');
xlabel('Class');
ylabel('Probability');
Figure 3-14 visually represents an example of a convolutional neural

network (CNN), showing an input image along with the output probabilities
for different classes generated by the CNN model.
Figure 3-14: An example of a convolutional neural network (CNN).
4. CNN Variants: Let us dive into more detail about some CNN
variants. there are also different types of CNNs based on the
dimensionality of the input data. Two commonly used variants
are 2-D CNNs and 3-D CNNs.
2-D CNNs: 2-D CNNs are primarily used for image

processing tasks, where the input data is a two-dimensional
grid of pixels. These networks are designed to operate on 2-D
input data and are well-suited for tasks such as image
classification, object detection, and image segmentation. The
convolutional layers in 2-D CNNs perform 2-D convolutions
on the input image, extracting spatial features in both the
horizontal and vertical directions.
3-D CNNs: 3-D CNNs are designed for processing volumetric

data, such as videos or medical imaging data. These networks
take into account the temporal dimension along with the
spatial dimensions. In addition to the width and height of the
input, the 3-D CNNs also consider the depth or time
136 Chapter III
dimension. This makes them suitable for tasks that involve

sequential data, such as action recognition in videos,
volumetric image analysis, and video classification. The
convolutional layers in 3-D CNNs perform 3-D convolutions,
considering both spatial and temporal features in the input
data.
Both 2-D and 3-D CNNs share similar principles with

traditional CNNs, including convolutional layers, pooling
layers, and fully connected layers. However, the main
difference lies in how the convolutional operations are applied
to the input data, taking into account the specific dimensionality
of the data.
There are some more CNN variants, which are commonly

used and have made significant contributions to the field of
computer vision. Let us take a closer look at their popularity
and impact.
VGGNet: VGGNet is a CNN architecture that gained

popularity for its simplicity and effectiveness. It consists of
multiple convolutional layers stacked on top of each other,
followed by fully connected layers. The key characteristic of
VGGNet is the use of 3 × 3 convolutional filters throughout
the network. By using smaller filters and deeper layers,
VGGNet is able to learn complex features and capture fine
details in images. The architecture of VGGNet can be
customized to have different depths, such as VGG16 (16
layers) and VGG19 (19 layers). VGGNet achieved remarkable
performance in image classification tasks, demonstrating the
power of deeper architectures.
ResNet: ResNet (short for Residual Network) is a deep CNN

architecture that introduced the concept of residual
connections. The main motivation behind ResNet was to
address the problem of vanishing gradients that occurs when
training very deep networks. Residual connections allow
information to bypass layers, enabling the network to learn
identity mappings. This helps to alleviate the vanishing
gradient problem and allows for training of deeper networks.
In ResNet, residual blocks are used, which consist of stacked
convolutional layers with shortcut connections. These shortcut
connections add the original input of a block to its output,

allowing the network to learn residual mappings. ResNet has
achieved remarkable performance in various computer vision
tasks, including image classification, object detection, and
image segmentation.
InceptionNet (or Inception): InceptionNet is a CNN

architecture known for its Inception module, which is
designed to capture multi-scale features efficiently. The
Inception module consists of parallel convolutional layers
with different filter sizes (e.g., 1 × 1, 3 × 3, 5 × 5) applied to
the same input. These parallel branches allow the network to
capture features at different scales and learn a diverse range of
information. The outputs of these branches are then
concatenated to form a composite feature representation,
which is passed on to the next layer. InceptionNet aims to
strike a balance between capturing local details with smaller
filters and global context with larger filters. This architecture
reduces the number of parameters compared to a single branch
with large filters and has shown excellent performance in
various image recognition tasks.
Apart from these variants, there are several other notable CNN
architectures, including:
AlexNet: AlexNet was one of the pioneering CNN architectures

that achieved breakthrough performance in the ImageNet
Large Scale Visual Recognition Challenge (ILSVRC) in 2012.
It consists of multiple convolutional and fully connected
layers, with the extensive use of max-pooling and the rectified
linear activation function (ReLU).
DenseNet: DenseNet is a CNN architecture that introduces

dense connections between layers. In DenseNet, each layer
receives direct inputs from all preceding layers, promoting
feature reuse and gradient flow. This architecture addresses
the vanishing gradient problem and encourages feature
propagation across the network.
MobileNet: MobileNet is a lightweight CNN architecture

designed for efficient computation on mobile and embedded
devices. It utilizes depth-wise separable convolutions to
138 Chapter III
reduce the number of parameters and computations while

maintaining good accuracy. MobileNet models are well-suited
for applications with limited computational resources.
These are just a few examples of CNN architectures, each with

its own unique design choices and benefits. Researchers and
practitioners continue to explore and develop new architectures
to tackle specific challenges and improve the performance of
deep learning models in various domains.
Figure 3-15: An example of a recurrent neural network (RNN)

architecture.
5. Recurrent Neural Network (RNN): Recurrent neural

network (RNN) is a type of deep learning model commonly
used for sequence data, such as time series or text data. Unlike
feedforward neural networks, which process inputs in a single
pass from input cell to output cell, RNNs have recurrent
connections that allow them to persist information from
previous time steps as can be seen in Figure 3-15.
The key idea behind RNNs is the concept of hidden state or

memory, which captures the information from previous time
steps and influences the predictions at the current time step.
This memory enables RNNs to capture temporal dependencies

and patterns in the data.
Mathematically, an RNN can be described as
At each time step ‫ݐ‬: ݄(‫ )ݐ(ܠ܅(݂ = )ݐ‬+ ‫ ݐ(ܐ܃‬െ 1) + ܾ)

‫ )ݐ(ܐ܄(݃ = )ݐ(ݕ‬+ ܿ)
In these equations, ‫ )ݐ(ܐ‬represents the hidden state at time step

‫ݐ‬, ‫ )ݐ(ܠ‬is the input at time step ‫ݐ‬, ‫ )ݐ(ܡ‬is the output at time step
‫ݐ‬, ݂ and ݃ are activation functions (e.g., sigmoid or tanh), ‫܅‬,
‫܃‬, V are weight matrices, and ܾ, ܿ are bias terms.
The first equation represents the recurrent connection, where

the hidden state at the current time step is computed based on
the current input, the previous hidden state, and bias terms.
The second equation computes the output at the current time
step based on the current hidden state and bias terms.
RNNs can be trained using various optimization algorithms,

such as gradient descent, and backpropagation through time
(BPTT) is commonly used to compute the gradients and
update the weights.
MATLAB code example for a recurrent neural network

(RNN):
% Generate a simulated signal

t = linspace(0, 10, 1000); % Time vector
x = sin(2*pi*0.5*t) + 0.5*sin(2*pi*2*t); % Simulated signal
with two frequency components
% Create training data for the RNN

sequenceLength = 10; % Length of input sequence
numSamples = length(x) - sequenceLength; % Number of
training samples
X = zeros(sequenceLength, numSamples); % Input sequence

matrix
Y = zeros(1, numSamples); % Output matrix
140 Chapter III
for i = 1:numSamples
X(:, i) = x(i:i+sequenceLength-1)';
Y(i) = x(i+sequenceLength);
end
% Define the RNN architecture

numHiddenUnits = 20; % Number of hidden units in the RNN
layers = [
sequenceInputLayer(sequenceLength)
fullyConnectedLayer(numHiddenUnits)
reluLayer
regressionLayer];
% Train the RNN

options = trainingOptions('adam', 'MaxEpochs', 100);
net = trainNetwork(X, Y, layers, options);
% Generate predictions using the trained RNN

X_test = X(:, end); % Input sequence for prediction
numPredictions = length(t) - sequenceLength; % Number of
predictions to make
Y_pred = zeros(1, numPredictions); % Predicted output
% Make predictions for each time step

for i = 1:numPredictions
Y_pred(i) = predict(net, X_test);
X_test = circshift(X_test, -1);
X_test(end) = Y_pred(i);
end
% Plot the original signal and predicted output

figure;
plot(t, x, 'b', 'LineWidth', 1.5);
hold on;
plot(t(sequenceLength+1:end), Y_pred, 'r--', 'LineWidth',
1.5);
xlabel('Time');
ylabel('Signal');
legend('Original Signal', 'Predicted Output');
title('RNN Predicted Signal');
grid on;
Figure 3-16 illustrates the signal prediction results obtained from an

example of a recurrent neural network (RNN).
Figure 3-16: the signal prediction results obtained from an example of a recurrent
neural network (RNN).
6. Long Short-Term Memory (LSTM): Long short-term

memory (LSTM) is a type of recurrent neural network (RNN)
architecture designed to handle long-term dependencies and
capture sequential patterns in data. Unlike traditional RNNs,
which struggle to retain information over long sequences,
LSTM introduces a memory cell and three gating mechanisms
that control the flow of information.
The key components of an LSTM are as follows:
x Memory Cell: The memory cell serves as the “memory”

of the LSTM. It can store information over long periods
and selectively read, write, and erase information.
x Forget Gate: The forget gate determines which
information to discard from the memory cell. It takes
input from the previous hidden state and the current input
142 Chapter III
and outputs a value between 0 and 1 for each memory cell

element. A value of 1 means “keep this information,”
while 0 means “forget this information.”
x Input Gate: The input gate decides which new
information to store in the memory cell. It processes the
previous hidden state and the current input and outputs a
value between 0 and 1 that represents the relevance of
new information.
x Output Gate: The output gate controls which parts of the
memory cell should be exposed as the output of the
LSTM. It considers the previous hidden state and the
current input, and selectively outputs information.
MATLAB code example for a long short-term memory

(LSTM):
% Set random seed for reproducibility

rng(0);
% Parameters
numSamples = 1000; % Number of time points in the time
series
t = 1:numSamples; % Time index
% Simulate time series data

% Example 1: Random Walk
y1 = cumsum(randn(1, numSamples));
% Example 2: Sinusoidal with Noise

frequency = 0.1; % Frequency of the sinusoidal component
amplitude = 5; % Amplitude of the sinusoidal component
noiseStdDev = 1; % Standard deviation of the Gaussian noise
y2 = amplitude * sin(2*pi*frequency*t) + noiseStdDev *
randn(1, numSamples);
% Plot the simulated time series

figure;
subplot(2, 1, 1);
plot(t, y1);
title('Simulated Time Series: Random Walk');
xlabel('Time');
ylabel('Value');
subplot(2, 1, 2);
plot(t, y2);
title('Simulated Time Series: Sinusoidal with Noise');
xlabel('Time');
ylabel('Value');
% Additional examples or modifications can be made based

on your requirements
In the given code, two examples of simulated time series data are provided.
The corresponding figures are displayed in Figure 3-17.
1. Random Walk: The y1 variable represents a random walk, where

each point is the cumulative sum of a random number drawn from
a Gaussian distribution.
2. Sinusoidal with Noise: The y2 variable represents a sinusoidal

signal with added Gaussian noise. The frequency, amplitude, and
noise standard deviation can be adjusted based on your requirements.
Figure 3-17: Two examples of simulated time series data, i.e., random
walk and sinusoidal with noise.
144 Chapter III
% Set random seed for reproducibility

rng(0);
% Parameters
numSamples = 1000; % Number of time points in the time
series
t = 1:numSamples; % Time index
% Simulate time series data

% Example 1: Random Walk
y1 = cumsum(randn(1, numSamples));
% Example 2: Sinusoidal with Noise

frequency = 0.1; % Frequency of the sinusoidal component
amplitude = 5; % Amplitude of the sinusoidal component
noiseStdDev = 1; % Standard deviation of the Gaussian noise
y2 = amplitude * sin(2*pi*frequency*t) + noiseStdDev *
randn(1, numSamples);
% LSTM model
% Prepare training data for LSTM
inputSize = 10;
outputSize = 1;
XTrain1 = [];
YTrain1 = [];
for i = 1:length(y1)-inputSize-outputSize
XTrain1 = [XTrain1; y1(i:i+inputSize-1)];
YTrain1 = [YTrain1;
y1(i+inputSize:i+inputSize+outputSize-1)];
end
XTrain2 = [];
YTrain2 = [];
for i = 1:length(y2)-inputSize-outputSize
XTrain2 = [XTrain2; y2(i:i+inputSize-1)];
YTrain2 = [YTrain2;
y2(i+inputSize:i+inputSize+outputSize-1)];
end
% Define LSTM network architecture

numHiddenUnits = 200;
layers = [ ...
sequenceInputLayer(inputSize)
lstmLayer(numHiddenUnits, 'OutputMode', 'sequence')
fullyConnectedLayer(outputSize)
regressionLayer
];
% Set training options

options = trainingOptions('adam', ...
'MaxEpochs', 100, ...
'MiniBatchSize', 32, ...
'InitialLearnRate', 0.01, ...
'Verbose', 0);
% Train the LSTM network

net1 = trainNetwork(XTrain1', YTrain1', layers, options);
net2 = trainNetwork(XTrain2', YTrain2', layers, options);
% Generate LSTM predictions for the entire time period

XTest1 = y1(end-inputSize+1:end);
YPred1 = zeros(1, numSamples);
YPred1(1:length(XTest1)) = XTest1;
for i = length(XTest1)+1:numSamples
XTest1 = YPred1(i-inputSize:i-1);
YPred1(i) = predict(net1, XTest1');
end
XTest2 = y2(end-inputSize+1:end);
YPred2 = zeros(1, numSamples);
YPred2(1:length(XTest2)) = XTest2;
for i = length(XTest2)+1:numSamples
XTest2 = YPred2(i-inputSize:i-1);
YPred2(i) = predict(net2, XTest2');
end
% Plot the results

figure;
subplot(2, 1, 1);
hold on;
146 Chapter III
plot(t, y1, 'b', 'LineWidth', 1.5);

plot(t, YPred1, 'g', 'LineWidth', 1.5);
xlabel('Time');
ylabel('Value');
legend('Actual', 'LSTM Prediction');
title('LSTM: Random Walk');
grid on;
hold off;
subplot(2, 1, 2);
hold on;
plot(t, y2, 'b', 'LineWidth', 1.5);
plot(t, YPred2, 'g', 'LineWidth', 1.5);
xlabel('Time');
ylabel('Value');
legend('Actual', 'LSTM Prediction');
title('LSTM: Sinusoidal with Noise');
grid on;
hold off;
Figure 3-18 exhibits an example of prediction using a long short-term

memory (LSTM) for the two time series data mentioned earlier. It
showcases the predicted values generated by the LSTM model.
Figure 3-18: An example of prediction using a long short-term memory (LSTM) for
the two time series data.
LSTM variants build upon this basic LSTM architecture to enhance its
capabilities and address specific requirements.
7. LSTM Variants:
Bidirectional LSTM: Bidirectional LSTM processes the

input sequence in both the forward and backward directions.
It consists of two separate LSTM layers, one processing the
input sequence in the original order and the other in reverse
order. By considering both past and future contexts
simultaneously, the bidirectional LSTM captures a more
comprehensive understanding of the input sequence. This is
especially useful when the prediction of a particular element
depends on both past and future information.
Gated Recurrent Unit (GRU): Gated Recurrent Unit (GRU)

is a simplified variant of LSTM that combines the forget and
input gates into a single update gate. This simplification
reduces the number of gating mechanisms while maintaining
148 Chapter III
performance. GRU also introduces a new gate called the reset

gate, which determines how much of the previous hidden state
to forget. The update gate controls the flow of new
information into the hidden state. GRU is computationally
efficient and has been shown to perform well on various
sequence-related tasks.
These LSTM variants offer different capabilities and cater to

specific requirements. Bidirectional LSTM enhances the
model's ability to capture context from both past and future
information, while GRU simplifies the architecture while
maintaining performance. The choice between LSTM and its
variants depends on the specific task and the nature of the data
being processed.
8. Autoencoder: An autoencoder is an unsupervised learning

algorithm used for learning efficient data representations in an
unsupervised manner. It is a neural network architecture that
aims to reconstruct its input data at its output layer. The
purpose of an autoencoder is to learn a compressed
representation of the input data, capturing its important
features while minimizing the reconstruction error.
9. As shown in Figure 3-19 the architecture of an autoencoder

consists of an encoder and a decoder. The encoder takes the
input data and maps it to a lower-dimensional representation,
often referred to as the latent space or the bottleneck layer. The
encoder typically consists of several hidden layers that
progressively reduce the dimensionality of the input data. The
decoder then takes the lower-dimensional representation and
attempts to reconstruct the original input data. The decoder
layers mirror the structure of the encoder layers but in reverse
order, gradually increasing the dimensionality of the data until
it matches the original input dimensions.
10. During the training process, the autoencoder is trained to

minimize the difference between the input data and the
reconstructed output data. This is typically done by optimizing
a loss function such as the mean squared error (MSE) between
the input and the output. The weights and biases of the
autoencoder's neural network are adjusted through
backpropagation and gradient descent algorithms.
Figure 3-19: An example of an autoencoder architecture.
Autoencoders have several applications in machine learning,

including dimensionality reduction, data denoising, and
anomaly detection. By training an autoencoder on a large
dataset, it can learn a compressed representation that captures
the most important features of the data. This compressed
representation can then be used for various downstream tasks,
such as visualization, clustering, or classification.
MATLAB code example for an autoencoder:
% Set the parameters for the synthetic dataset and autoencoder

numSamples = 1000;
inputSize = 10;
hiddenSize = 5;
% Generate random input data

inputData = rand(numSamples, inputSize);
% Generate corresponding output data (reconstructed input

data)
outputData = inputData;
% Add noise to the output data
noiseLevel = 0.1;
150 Chapter III
outputData = outputData + noiseLevel *

randn(size(outputData));
% Normalize the data

inputData = normalize(inputData);
outputData = normalize(outputData);

trainRatio = 0.8; % 80% for training, 20% for testing
trainSize = round(numSamples * trainRatio);
xTrain = inputData(1:trainSize, :);
yTrain = outputData(1:trainSize, :);
xTest = inputData(trainSize+1:end, :);
yTest = outputData(trainSize+1:end, :);
% Create the autoencoder

autoencoder = trainAutoencoder(xTrain', hiddenSize);
% Reconstruct the input data using the trained autoencoder

reconstructedData = predict(autoencoder, xTest');
% Calculate the reconstruction error

reconstructionError = mean((reconstructedData - yTest').^2);
% Display the reconstruction error

disp(['Reconstruction error: ' num2str(reconstructionError)]);
Figure 3-20 exhibits an example of data reconstruction using an

autoencoder.
Figure 3-20: An example of data reconstruction using an autoencoder

In the realm of signals, machine learning and deep learning prevail.
Analyzing data, extracting insights, and predictions they unveil.
Prepare the data, scale it right, select relevant features in sight. To unleash
the power of models and make your analysis take flight.
Support vector machines, the classifiers with finesse. Group signals into
categories, their characteristics assess. K-nearest neighbors, finding
similarity in signals' embrace. Classifying based on neighbors, their features
they embrace.
Random forest, a group of decision-makers in unity. Combining their

predictions, creating accuracy with impunity.
Deep neural networks, the masters of complex patterns and relations.

Unraveling signals' secrets, learning from data's variations.
152 Chapter III
Convolutional neural networks, focusing on signal parts. Spotting patterns,

unveiling insights, with expertise that imparts.
Recurrent neural networks, understanding sequences through and through.

Retaining past information, incorporating it anew.
Autoencoders, simplifying signals, extracting features profound. Unveiling

their essence, reducing dimensions without a sound.
These models find applications in healthcare, finance, and more. Speech

recognition, image processing, with their capabilities to explore.
By grasping these models, signals come to life with meaning. Revealing

their stories, aiding in decision-making and screening. Machine learning
and deep learning empower our understanding. In the world of signals, their
potential is truly outstanding.
Key takeaways from the machine learning and deep learning for signals are
given as follows:
1. Preprocess and feature selection: Before training any model,

preprocess the signals by scaling and selecting relevant features
using techniques like principal component analysis (PCA) to
improve performance.
2. Support Vector Machines (SVM): SVM is a powerful algorithm
for signal classification. It works well with labeled data and can
handle multiple classes through techniques like one-vs-all or one-
vs-one coding.
3. K-Nearest Neighbors (KNN): KNN is a simple yet effective
algorithm for classification. It classifies signals based on the labels
of their nearest neighbors. It is non-parametric and can handle
multi-class problems.
4. Random Forest: Random Forest is an ensemble learning method
that combines multiple decision trees to make predictions. It works
well for both classification and regression tasks. It handles high-
dimensional data and can provide feature importance rankings.
5. Deep Neural Networks (DNN): DNNs are powerful models for
signal analysis. They consist of multiple layers of interconnected
nodes (neurons). They can automatically learn complex patterns
and relationships in the data. Techniques like Convolutional Neural
Networks (CNNs) are commonly used for signal classification tasks.
6. Convolutional Neural Networks (CNN): CNNs are specifically
designed for processing grid-like data, such as images or signals.
They utilize convolutional layers to extract local patterns and

hierarchically learn complex representations. CNNs have been
successful in various signal processing applications.
7. Recurrent Neural Networks (RNN): RNNs are useful for
sequential data, where the order of input matters. They have
memory to capture dependencies in temporal signals. Long short-
term memory (LSTM) and gated recurrent unit (GRU) are popular
RNN variants.
8. Autoencoders: Autoencoders are unsupervised learning models
used for signal denoising, compression, and feature extraction.
They consist of an encoder and a decoder network. By training on
reconstructed signals, they learn to capture important signal
characteristics.
Layman's Guide:
Machine learning and deep learning models are powerful tools for analyzing
different types of data.
Data preparation is important before using any model to ensure the data is
scaled and relevant features are selected.
Support vector machines (SVM) classify signals based on their characteristics,

assigning them to different categories.
K-nearest neighbors (KNN) classifies signals based on their similarity to

other signals.
Random Forest is a group of decision-makers that work together to make

predictions based on different factors.
Deep neural networks (DNN) learn from data and understand complex
patterns and relationships in signals.
Convolutional neural networks (CNN) focus on specific parts of signals to

find important patterns.
Recurrent neural networks (RNN) understand sequences of signals and can

remember important information from previous signals.
Autoencoders simplify signals and extract important features from them.
These models are used in various fields such as healthcare, finance, and
speech recognition to analyze signals and make predictions.
154 Chapter III
Understanding these models helps in making sense of the signals around us

and utilizing them effectively for different applications.
Exercises of machine learning and deep learning

for signals
Problem 1: Classification of Signal Data
Solution: To solve the classification problem using machine learning and

deep learning techniques for signal data, we can follow the following steps:
1. Dataset Preparation: Collect a dataset of labeled signal data

suitable for classification. This dataset should consist of a set of
input signals along with their corresponding class labels.
2. Feature Extraction: Extract relevant features from the signal data
that capture important characteristics for classification. This could
involve techniques such as time-domain or frequency-domain
analysis, wavelet transforms, or other signal processing methods.
3. Dataset Split: Split the dataset into training, validation, and test
sets. The training set will be used to train the classification model,
the validation set will be used for hyperparameter tuning and
model selection, and the test set will be used for evaluating the
final model's performance.
4. Model Selection: Choose an appropriate machine learning or deep
learning model for signal classification. This could include
traditional machine learning algorithms such as support vector
machines (SVM), k-nearest neighbors (KNN), or decision trees.
Alternatively, deep learning models such as convolutional neural
networks (CNN) or recurrent neural networks (RNN) can be used.
5. Model Training: Train the selected model using the training
dataset. This involves feeding the input signals and their
corresponding labels to the model and adjusting its internal
parameters to minimize the classification error.
6. Model Evaluation: Evaluate the trained model using the
validation dataset. Calculate metrics such as accuracy, precision,
recall, or F1 score to assess the model's performance. Adjust the
model's hyperparameters if necessary to improve performance.
7. Model Testing: Once the model has been trained and evaluated,
use the test dataset to assess its performance on unseen data.
Calculate the same evaluation metrics to determine the model's
accuracy on new signal samples.
8. Fine-tuning and Optimization: Depending on the model's

performance, fine-tune and optimize the model by adjusting
hyperparameters, trying different architectures, or applying
regularization techniques to improve classification accuracy.
9. Deployment and Prediction: Once satisfied with the model's
performance, deploy it to classify new, unseen signal data. Use the
trained model to predict the class labels of new signal samples.
10. Iteration and Improvement: Continuously monitor the model's
performance, gather feedback, and iterate on the steps above to
improve classification accuracy and address any challenges or
limitations observed during the process.
Remember, the specific implementation details and choice of algorithms

may vary depending on the nature of the signal data and the specific
classification task at hand.
MATLAB example:
In this example, the code classifies handwritten digits. It trains a

convolutional neural network (CNN) using the training data (XTrain) and
labels (YTrain) for handwritten digits. Once the network is trained, it uses
the trained network (net) to classify the test data (XTest). The predicted
labels are then compared to the true labels (YTest) to calculate the accuracy
of the classification. The accuracy represents the percentage of correctly
classified digits in the test set.
Please note that this is a simplified example, and you can customize it based
on your specific requirements and dataset.
clear all
%
% Train a convolutional neural network on some synthetic images
% of handwritten digits. Then run the trained network on a test
% set, and calculate the accuracy.
[XTrain, YTrain] = digitTrain4DArrayData;
layers = [ ...
imageInputLayer([28 28 1])
convolution2dLayer(5,20)
reluLayer
maxPooling2dLayer(2,'Stride',2)
156 Chapter III
softmaxLayer
classificationLayer];
options = trainingOptions('sgdm', 'Plots', 'training-progress');
net = trainNetwork(XTrain, YTrain, layers, options);
[XTest, YTest] = digitTest4DArrayData;
YPred = classify(net, XTest);

accuracy = sum(YTest == YPred)/numel(YTest)
% Display the accuracy

fprintf('Accuracy: %.2f%%\n', accuracy * 100);
% Calculate confusion matrix

confMat = confusionmat(YTest, YPred);
% Calculate precision and recall

precision = diag(confMat) ./ sum(confMat, 2);
recall = diag(confMat) ./ sum(confMat, 1)';
% Compute F1-score
f1 = 2 * (precision .* recall) ./ (precision + recall);
% Display the precision

fprintf('Precision: %.2f%%\n', mean(precision) * 100);
% Display the recall

fprintf('Recall: %.2f%%\n', mean(recall) * 100);
% Display the F1-score

fprintf('F1-Score: %.2f%%\n', mean(f1) * 100);
accuracy =
0.9802
Accuracy: 98.02%
Precision: 98.02%
Recall: 98.03%
F1-Score: 98.02%
Figure 3-21 illustrates the accuracy and the loss curves of a Convolutional
Neural Network (CNN) classifier for handwritten digits classification.
Figure 3-21: The accuracy and the loss curves of a Convolutional Neural Network
(CNN) classifier for handwritten digits classification.
Signal processing for non-Euclidean data

This section explores signal processing techniques tailored specifically for
non-Euclidean data. Graphs and networks are powerful representations for
modeling complex relationships and interactions among entities, such as
social networks, biological networks, or sensor networks. However,
analyzing signals on graphs poses unique challenges that require specialized
algorithms.
Signal processing has traditionally focused on analyzing signals in

Euclidean spaces, such as one-dimensional time series or two-dimensional
images. However, as the need arises to analyze data that does not adhere to
the Euclidean framework, such as graphs and networks, signal processing
techniques have evolved to accommodate these non-Euclidean structures.
Graphs and networks provide powerful representations for modeling

complex relationships and interactions among entities, such as social
networks, biological networks, or sensor networks. However, analyzing
signals on graphs poses unique challenges that require specialized algorithms.
Signal processing for non-Euclidean data involves techniques such as graph
signal processing and network analysis. Graph signal processing extends
traditional signal processing concepts and tools to graphs, enabling the
158 Chapter III
study of signal properties and characteristics defined on graph nodes or

edges. This opens up avenues for tasks such as graph signal denoising,
graph-based filtering, and graph-based learning algorithms.
In graph signal processing, we aim to analyze signals defined on graph

nodes or edges. The graph structure influences the behavior of these signals.
For example, graph signal denoising seeks to remove noise from signals
defined on graph nodes while considering the connectivity and structure of
the graph. Graph-based filtering involves designing filters that exploit the
graph structure to process signals efficiently. Moreover, graph-based
learning algorithms leverage the relationships in the graph to enhance
prediction and classification tasks.
Network analysis focuses on understanding the structure and dynamics of

networks by examining signals that propagate through them. Diffusion
models, for instance, describe how signals or information spread across a
network. Centrality measures help identify the most influential or important
nodes in a network, while community detection techniques reveal groups of
tightly connected nodes. By studying signals on networks, we gain insights
into network connectivity, information flow, and interactions between
network components.
Signal processing for non-Euclidean data is increasingly relevant in various

domains, including social sciences, biology, transportation, and communication
networks. These techniques enable us to extract meaningful information
from complex network structures and uncover hidden patterns and
relationships that traditional Euclidean signal processing may overlook.
By expanding the signal processing toolbox to include methods for non-

Euclidean data, we enhance our understanding of complex systems and
improve our ability to analyze and interpret signals in diverse domains. This
opens up exciting opportunities for extracting knowledge from networked
data and addressing real-world challenges in interconnected systems.
Here is a list of techniques that can be used to expand the signal processing
toolbox for non-Euclidean data:
1. Graph Signal Processing: Extends traditional signal processing

concepts and tools to graphs, allowing the analysis of signals
defined on graph nodes or edges. It includes techniques such as
graph signal denoising, graph-based filtering, and graph-based
learning algorithms.
2. Network Analysis: Focuses on understanding the structure and

dynamics of networks by studying the signals that propagate
through them. Techniques in network analysis include diffusion
models, centrality measures, and community detection.
3. Graph Signal Denoising: Aims to remove noise from signals
defined on graph nodes while considering the connectivity and
structure of the graph. It is particularly useful for denoising signals
in networked data.
4. Graph-Based Filtering: Involves designing filters that exploit the
graph structure to process signals efficiently. These filters take into
account the relationships between nodes in the graph to enhance
signal processing tasks.
5. Graph-Based Learning Algorithms: Leverage the relationships in
the graph to improve prediction and classification tasks. By
incorporating the graph structure, these algorithms can capture the
dependencies and interactions between network components.
6. Diffusion Models: Describe how signals or information spread
across a network. Diffusion models can help understand the flow
of information or influence in networked systems.
7. Centrality Measures: Identify the most influential or important
nodes in a network. Centrality measures, such as degree centrality,
betweenness centrality, and eigenvector centrality, provide
insights into the significance of nodes within a network.
8. Community Detection: Reveals groups of tightly connected nodes
in a network. Community detection algorithms identify clusters or
communities within a network, helping to uncover hidden
structures or functional modules.
By incorporating these techniques into the signal processing toolbox, we

can enhance our understanding of complex systems, analyze and interpret
signals in diverse domains, and extract valuable knowledge from networked
data. These advancements open up exciting opportunities for addressing
real-world challenges in interconnected systems.
Here are a few examples of MATLAB code snippet demonstrating graph

signal processing on simple networks.
160 Chapter III
Example 1: An application of graph Laplacian filtering
In this example, we define an adjacency matrix A representing a simple

network with four nodes. We generate a random signal on these nodes and
apply graph Laplacian filtering by multiplying the signal with the adjacency
matrix. Finally, we visualize the original and filtered signals using a stem
plot.
% Define the adjacency matrix of a network

A = [0 1 1 0; 1 0 1 1; 1 1 0 1; 0 1 1 0];
% Generate a random signal on the network nodes

signal = rand(4, 1);
% Apply graph Laplacian filtering to the signal

filtered_signal = A * signal;
% Display the original and filtered signals

figure;
subplot(2, 1, 1);
stem(signal);
title('Original Signal');
xlabel('Node');
subplot(2, 1, 2);
stem(filtered_signal);
title('Filtered Signal');
xlabel('Node');
% Visualize the network

figure;
G = graph(A);
plot(G, 'Layout', 'force');
title('Network Visualization');
Figure 3-22 visually presents the network visualization of Example 1,

showcasing the application of graph Laplacian filtering. On the other hand,
Figure 3-23 displays both the original signal and the filtered signal obtained
through graph Laplacian filtering, demonstrating the impact of this filtering
technique on the signal processing outcome.
Figure 3-22: Network visualization of Example 1.
Figure 3-23: Network visualization of Example 1.

162 Chapter III
Example 2: A graph signal denoising and graph-based filtering on a graph

with 5 nodes
This example demonstrates graph signal denoising and graph-based filtering

on a graph with 5 nodes. The graphSignalDenoising function applies graph
Laplacian denoising to the signal, while the graphBasedFiltering function
designs a graph-based filter to process the signal. The resulting original,
noisy, denoised, and filtered signals are displayed using the plot function
on the graph structure as shown in Figure 3-24.
Note that you may need to install the MATLAB Graph and Digraph toolbox
(graph and digraph functions) to run this code.
% Define the main script
% Create a graph with 5 nodes

G = graph([1 1 1 2 3 3 4 5], [2 3 4 3 4 5 5 1]);
% Define a signal on the graph nodes

signal = [0.8; 0.5; -0.2; 1.0; -0.7];
% Perform graph signal denoising

noisy_signal = signal + 0.1 * randn(size(signal));
denoised_signal = graphSignalDenoising(G, noisy_signal);
% Perform graph-based filtering

filtered_signal = graphBasedFiltering(G, signal);
% Display the graph and signals

plotGraphAndSignals(G, signal, noisy_signal, denoised_signal,
filtered_signal);
% Graph signal denoising function

function denoised_signal = graphSignalDenoising(G, noisy_signal)
% Define graph Laplacian matrix
L = laplacian(G);
% Denoise the signal using graph Laplacian

denoised_signal = (eye(size(L)) + 0.5 * L) \ noisy_signal;
end
% Graph-based filtering function

function filtered_signal = graphBasedFiltering(G, signal)
% Define graph adjacency matrix
A = adjacency(G);
% Design graph-based filter

H = 0.8 * eye(size(A)) - 0.2 * A;
% Filter the signal using graph-based filtering

filtered_signal = H * signal;
end
% Function to plot the graph and signals

function plotGraphAndSignals(G, signal, noisy_signal, denoised_signal,
filtered_signal)
% Display the graph
figure;
subplot(1, 2, 1);
plot(G, 'MarkerSize', 10);
title('Graph');
% Display the signals

subplot(1, 2, 2);
stem(1:numel(signal), signal, 'filled', 'MarkerSize', 6, 'LineWidth', 1.5);
hold on;
stem(1:numel(noisy_signal), noisy_signal, 'filled', 'MarkerSize', 6,
'LineWidth', 1.5);
stem(1:numel(denoised_signal), denoised_signal, 'filled', 'MarkerSize', 6,
'LineWidth', 1.5);
stem(1:numel(filtered_signal), filtered_signal, 'filled', 'MarkerSize', 6,
'LineWidth', 1.5);
hold off;
legend('Original', 'Noisy', 'Denoised', 'Filtered', 'Location', 'best');
title('Signals');
xlabel('Node');
end
164 Chapter III
Figure 3-24: The 5-node graph structure and the original, noisy, denoised, and
filtered signals of Example 2.
Example 3: Analysis of a random signal on the network nodes
In this example, we create a graph G directly from the adjacency matrix A.

We then generate a random signal on the network nodes using the rand
function. This example demonstrates the application of signal processing
techniques to non-Euclidean data, where the network structure is analyzed
and the signal values on the nodes are visualized.
To visualize the network, we plot it using the ‘force’ layout. Additionally,

we plot the signal values on the nodes using a stem plot as shown in Figure
3-25.
The colormap function sets the color map to ‘parula’, and the node_colors
variable assigns normalized signal values to each node. We then update the
NodeCData property of the plot object (p) to reflect the node colors.
Additionally, we adjust the marker size and remove the node labels for
better visualization. By coloring the nodes based on signal values, we
visually highlight the ability of non-Euclidean signal processing techniques
to extract meaningful information from complex network structures and
uncover hidden patterns and relationships that traditional Euclidean signal

processing may overlook.
% Define the adjacency matrix of a network

A = [0 1 1 0 0; 1 0 1 0 0; 1 1 0 1 0; 0 0 1 0 1; 0 0 0 1 0];
% Create a graph from the adjacency matrix

G = graph(A);
% Generate a random signal on the network nodes

signal = rand(size(A, 1), 1);
% Plot the network with colored nodes based on signal values

figure;
subplot(1, 2, 1);
p = plot(G, 'Layout', 'force');
title('Network');
% Color the nodes based on signal values

colormap(parula);
node_colors = signal/max(signal); % Normalize signal values between 0
and 1
p.NodeCData = node_colors;
p.MarkerSize = 8;
p.NodeLabel = [];
% Add colorbar to indicate signal values

colorbar;
% Plot the signal values on the nodes

subplot(1, 2, 2);
stem(1:numel(signal), signal, 'filled');
title('Signal Values');
xlabel('Node');
166 Chapter III
Figure 3-25: An example of network analysis and the visualization of the signal
values on the nodes.
Example 4: A modularity-based approach for community detection
This example uses the modularity-based approach for community detection.

It computes the modularity matrix, performs eigenvalue decomposition, and
assigns nodes to communities based on the sign of the leading eigenvector
elements. The resulting community assignments are then visualized on the
network plot as shown in Figure 3-26.
We also compute and display the modularity score of the network.

Modularity measures the quality of the community structure in a network,
where higher modularity values indicate better community structure.
The modularity score of -2.9167 means that the community structure of the
network is not well-defined or does not exhibit strong modular patterns.
Modularity scores typically range from -1 to 1, where values close to 1
indicate a strong community structure and values close to -1 indicate a lack
of community structure.
In this case, the negative modularity score suggests that the network's
partition into communities is not significantly different from a random
network. It indicates that the network's edges are not strongly clustered
within communities and do not exhibit clear modular patterns.
It's worth noting that the modularity score can be influenced by various
factors, such as the specific algorithm used for community detection, the
network topology, and the quality of the input data. In this example, the
modularity score suggests a lack of well-defined communities in the
network.
Please note that this simplified approach may not produce the same results
as more sophisticated community detection algorithms. However, it
provides a basic implementation for demonstration purposes.
The main script executes the community detection and plot the network with
community colors. The modularity_communities() function performs the
community detection using the modularity-based approach.
% Define the adjacency matrix of the network

A = [0 1 1 0 0; 1 0 1 1 0; 1 1 0 1 0; 0 1 1 0 1; 0 0 0 1 0];
% Perform community detection

community = modularity_communities(A);
% Generate a colormap for communities

unique_communities = unique(community);
num_communities = numel(unique_communities);
colormap_values = linspace(1, num_communities, num_communities);
colormap_community = zeros(size(community));
for i = 1:num_communities
colormap_community(community == unique_communities(i)) =
colormap_values(i);
end
% Plot the network with community colors

G = graph(A);
figure;
scatter(1:size(A, 1), 1:size(A, 1), 100, colormap_community, 'filled');
colormap(jet(num_communities));
title('Community Detection in a Network');
xlabel('Node');
ylabel('Node');
168 Chapter III
% Compute the modularity score

modularity_score = compute_modularity(A, community);
disp(['Modularity Score: ', num2str(modularity_score)]);
function modularity_score = compute_modularity(A, community)

% Compute the modularity matrix
B = A - sum(A, 'all') * sum(A, 'all')' / (2 * sum(A, 'all'));
% Compute the modularity score

modularity_score = sum(sum(B(community == community'))) / (2 *
sum(A, 'all'));
end
Modularity Score: -2.9167
Figure 3-26: An example of a modularity-based approach for community detection.

Signal processing for non-Euclidean data. Unveils patterns and insights, oh
how it does captivate.
In domains like social sciences and more. It delves into network structures,
right at the core.
From biology to transportation and communication. Non-Euclidean signals

find their application.
They extract knowledge from complex networks wide. Revealing hidden

relationships we cannot hide.
Community detection, a technique of great worth. Uncovers tightly

connected nodes, a network's true birth.
Clusters and modules, they come to light. Unveiling structures that were
once out of sight.
With graph-based filtering, signals find their way. Exploiting connectivity,

efficiency on display.
Denoising and filtering, removing noise and more. Enhancing predictions

like never before.
Centrality measures identify the most influential nodes. Those that control
the network's ebbs and flows.
And graph-based learning, leveraging connections so strong. Enhancing

predictions and classifications, it throngs.
Non-Euclidean signal processing, a realm so vast. Expanding our toolbox,

enabling us to surpass.
Complex systems' understanding, it surely does enhance. Unveiling

knowledge in interconnected systems, a joyful dance.
Key takeaways from the Signal processing for non-Euclidean data are given
as follows:
1. Non-Euclidean signal processing expands traditional signal

processing concepts and tools to analyze signals defined on graphs
or networks.
2. Graph signal processing allows us to study properties and
characteristics of signals defined on graph nodes or edges.
3. Techniques like graph signal denoising, graph-based filtering, and
graph-based learning algorithms are employed in non-Euclidean
signal processing.
170 Chapter III
4. Network analysis focuses on understanding the structure and

dynamics of networks by examining signals that propagate through
them.
5. Diffusion models describe how signals or information spread
across a network, while centrality measures identify influential
nodes.
6. Community detection algorithms reveal groups of tightly
connected nodes, helping uncover hidden structures or functional
modules.
7. Non-Euclidean signal processing is increasingly relevant in
domains such as social sciences, biology, transportation, and
communication networks.
8. It enables the extraction of meaningful information from complex
network structures, revealing hidden patterns and relationships.
9. Non-Euclidean signal processing techniques complement traditional
Euclidean signal processing methods and address real-world
challenges in interconnected systems.
10. By incorporating these techniques, researchers and practitioners
can deepen their understanding of complex systems and extract
knowledge from networked data.
Layman's Guide:
Signal processing for non-Euclidean data involves analyzing signals that
exist in networks or graphs instead of traditional one-dimensional or two-
dimensional spaces.
Graph signal processing is a technique that extends traditional signal

processing to work with graphs or networks. It allows us to study and
analyze signals that are defined on the nodes or connections of these
networks.
Some specific techniques used in non-Euclidean signal processing include

denoising, filtering, and learning algorithms that are tailored for graphs or
networks. These techniques help us remove noise from signals, design
efficient filters, and improve prediction and classification tasks using the
relationships within the graph.
Network analysis is an important aspect of non-Euclidean signal processing.

It focuses on understanding the structure and dynamics of networks by
studying how signals propagate through them. This analysis helps us gain
insights into network connectivity, information flow, and the interactions

between different components of the network.
By expanding our signal processing toolbox to include methods for non-

Euclidean data, we enhance our understanding of complex systems and
improve our ability to analyze and interpret signals in diverse domains. This
opens up exciting opportunities for extracting meaningful information from
networked data and addressing real-world challenges in interconnected
systems.
Non-Euclidean signal processing is increasingly relevant in various

domains, including social sciences, biology, transportation, and
communication networks. It allows us to uncover hidden patterns and
relationships that traditional Euclidean signal processing may overlook,
enabling us to extract valuable knowledge from complex network
structures.
Exercises of signal processing for non-Euclidean data

Problem 1:
You have been given a set of temperature readings recorded at different time
intervals throughout a day. However, the readings contain some random
noise, making it difficult to identify any underlying patterns. Your task is to
apply a low-pass filter to the temperature data to reduce the noise and extract
the underlying trend.
Solution:
Applying a low-pass filter to temperature data involves reducing high-

frequency noise while preserving the underlying trend or slow variations.
The filter attenuates rapid fluctuations and emphasizes the low-frequency
components associated with the desired trend. By convolving the
temperature data with a filter kernel, the high-frequency noise is suppressed,
resulting in a smoother, clearer representation of the underlying pattern.
This filtered signal allows for easier analysis and interpretation of the long-
term variations in temperature.
172 Chapter III
MATLAB example:
% Problem: Apply a low-pass filter to temperature data
% Generate synthetic temperature data with noise

t = 0:0.1:10; % Time vector
temperature = 25 + sin(2*pi*t/24) + 2*randn(size(t)); % True temperature
with added noise
% Apply low-pass filter to smooth the temperature data

filtered_temperature = your_filter_function(temperature);
% Plot the original and filtered temperature data

figure;
subplot(2, 1, 1);
plot(t, temperature);
title('Original Temperature Data');
xlabel('Time');
ylabel('Temperature');
subplot(2, 1, 2);
plot(t, filtered_temperature);
title('Filtered Temperature Data');
xlabel('Time');
ylabel('Temperature');
Figure 3-27 shows an example of applying a low-pass filter to smooth the

temperature data.
Figure 3-27: An example of applying a low-pass filter to smooth the temperature

data.
Problem 2:
You have a synthetic signal composed of two sinusoidal components with

different frequencies. However, the signal is contaminated with noise,
making it challenging to analyze. Your task is to perform signal filtering on
the given non-Euclidean data to reduce the noise and obtain a clearer
representation of the underlying signal.
Solution:
To solve this problem, you can follow these steps:
1. Generate the synthetic signal: Create a time vector t spanning the

desired time interval. Specify the frequencies (f1 and f2) and
amplitude of the sinusoidal components. Combine the two
sinusoids to create the synthetic signal using the given formula.
2. Apply signal filtering: Implement the your_filter_function to
apply the desired filtering technique to the generated signal. The
174 Chapter III
specific filtering method will depend on your requirements and the

nature of the noise you want to reduce.
3. Obtain the filtered signal: Pass the synthetic signal through the
filter function to obtain the filtered signal, which will have reduced
noise and emphasize the underlying signal components.
4. Visualize the results: Plot both the original and filtered signals
using the plot function. This allows for a visual comparison of the
two signals and provides insight into the effectiveness of the
applied signal filtering.
By following these steps, you can successfully apply signal filtering to the
synthetic non-Euclidean data, reducing the noise and obtaining a clearer
representation of the underlying signal. The filtering process enhances the
analysis and interpretation of the signal by emphasizing the desired
components and attenuating unwanted noise.
The filtering techniques can vary depending on your specific requirements,

and the choice of an appropriate filter should be based on the characteristics
of the noise and the desired signal components. The final filtered signal will
exhibit reduced noise, allowing for better analysis and understanding of the
underlying signal behavior.
MATLAB example:
% Perform signal filtering on non-Euclidean data
% Generate the synthetic signal

t = 0:0.01:10; % Time vector
f1 = 1; % Frequency of the first component
f2 = 2; % Frequency of the second component
amplitude = 1; % Amplitude of the signal
signal = amplitude * sin(2*pi*f1*t) + amplitude * sin(2*pi*f2*t);
% Apply signal filtering

filtered_signal = your_filter_function(signal);
% Plot the original and filtered signals

figure;
subplot(2, 1, 1);
plot(t, signal);
xlabel('Time');
subplot(2, 1, 2);
plot(t, filtered_signal);
title('Filtered Signal');
xlabel('Time');
function filtered_signal = your_filter_function(signal)

window_size = 5; % Size of the moving average window
% Pad the signal to handle edge cases

half_window = floor(window_size/2);
padded_signal = [signal(1:half_window), signal, signal(end-
half_window+1:end)];
% Apply the moving average filter

filtered_signal = movmean(padded_signal, window_size);
% Remove the padding from the filtered signal

filtered_signal = filtered_signal(half_window+1 : end-half_window);
end
Figure 3-28 shows an application of signal filtering to the synthetic non-

Euclidean data, reducing the noise and obtaining a clearer representation of
the underlying signal.
176 Chapter III
Figure 3-28: An example of applying a signal filtering to the synthetic non-

Euclidean data, reducing the noise.
CHAPTER IV
APPLICATIONS OF SIGNAL PROCESSING
This chapter explores the diverse applications of signal processing

techniques across various domains. Signal processing involves manipulating
and analyzing signals to extract meaningful information and improve
system performance. Here is a summary of the different sections covered in
this chapter:
1. Audio and Speech Processing: This section focuses on signal

processing techniques specifically designed for audio and speech
applications. It covers important topics such as audio coding,
which aims to compress audio signals for efficient storage and
transmission, speech recognition for converting spoken language
into text, and speaker identification for recognizing individuals
based on their unique voice characteristics.
2. Image and Video Processing: The section on image and video
processing discusses signal processing techniques used in the
realm of visual data. It covers image and video compression
techniques that reduce the size of images and videos without
significant loss of quality. Additionally, it explores object
recognition and tracking, enabling computers to identify and track
objects of interest in images and videos.
3. Biomedical Signal Processing: Biomedical signal processing
focuses on applying signal processing techniques to medical and
healthcare-related applications. This section covers electrocardiogram
(ECG) analysis, which involves processing and interpreting the
electrical activity of the heart, magnetic resonance imaging (MRI)
for non-invasive medical imaging, and the emerging field of brain-
computer interfaces, allowing direct communication between the
brain and external devices.
4. Communications and Networking: Signal processing plays a
crucial role in communication and networking systems. This
section delves into signal processing techniques used in channel
coding, where redundancy is added to transmitted signals for error
detection and correction. It also covers modulation techniques used
178 Chapter IV
to encode information onto carrier signals and equalization

methods to compensate for channel impairments.
5. Sensor and Data Fusion: Sensor and data fusion involves
combining information from multiple sources to improve decision-
making and system performance. This section explores signal
processing techniques used in data integration, feature extraction,
and classification. These techniques enable the extraction of
meaningful information from sensor data and facilitate intelligent
decision-making processes.
Audio and speech processing
Audio and speech processing is a specialized area within signal processing

that focuses on techniques specifically designed for analyzing, encoding,
decoding, and manipulating audio signals, as well as processing spoken
language. Here are some key aspects covered in this field:
1. Audio Coding: Audio coding, also known as audio compression

or audio encoding, involves the efficient representation of audio
signals to reduce the amount of data required for storage or
transmission. Various audio coding algorithms, such as MP3,
AAC, and FLAC, utilize signal processing techniques to compress
audio signals while maintaining perceptual quality. These
techniques exploit the characteristics of human auditory perception
to discard or reduce redundant information in the audio signal,
resulting in smaller file sizes without significant loss of perceived
audio quality.
2. Speech Recognition: Speech recognition is the task of converting
spoken language into written or textual form. Signal processing
techniques are used to preprocess the audio input, extract relevant
features, and apply machine learning algorithms to recognize and
interpret the speech patterns. Techniques such as feature
extraction, hidden Markov models (HMMs), deep neural networks
(DNNs), and recurrent neural networks (RNNs) are commonly
employed in speech recognition systems. Speech recognition has
applications in voice-controlled systems, transcription services,
and automatic speech-to-text conversion.
3. Speaker Identification: Speaker identification is the process of
recognizing individuals based on their unique voice characteristics.
Signal processing techniques are used to extract speaker-specific
features from speech signals, such as pitch, formants, and spectral
features. Machine learning algorithms, such as Gaussian mixture
Applications of Signal Processing 179
models (GMMs) and support vector machines (SVMs), are then

used to train models that can identify or verify speakers based on
these features. Speaker identification has applications in voice
authentication systems, forensic investigations, and personalized
services.
Audio and speech processing techniques play a crucial role in various

applications, including audio streaming, telecommunication systems, voice
assistants, language processing, and more. These techniques enable efficient
storage and transmission of audio signals, facilitate the conversion of
spoken language into text, and enable the recognition of individuals based
on their unique voice characteristics. Ongoing research in audio and speech
processing continues to advance the field, leading to improved algorithms,
better accuracy, and enhanced applications in diverse domains.
Image and video processing
Image and video processing is a specialized area within signal processing

that focuses on techniques for analyzing, manipulating, and enhancing
visual data, including images and videos. Here are some key aspects
covered in this field:
1. Image and Video Compression: Image and video compression

techniques are essential for reducing the size of images and videos
while maintaining an acceptable level of quality. Signal processing
techniques are utilized to remove redundancy and exploit the
perceptual limitations of human vision. Popular image compression
algorithms include JPEG (Joint Photographic Experts Group) and
PNG (Portable Network Graphics), which employ techniques like
discrete cosine transform (DCT) and quantization. Video
compression algorithms, such as H.264/AVC (Advanced Video
Coding) and HEVC (High-Efficiency Video Coding), build upon
image compression techniques and incorporate additional temporal
and spatial redundancy reduction techniques specific to video data.
2. Object Recognition and Tracking: Object recognition and
tracking involve identifying and tracking specific objects or
regions of interest within images and videos. Signal processing
techniques play a vital role in these tasks, including feature
extraction, pattern recognition, and machine learning algorithms.
Object recognition algorithms analyze visual features, such as
edges, textures, and colors, to classify and identify objects. Object
tracking algorithms use motion estimation and tracking techniques
180 Chapter IV
to follow objects' movement over time in videos. These techniques

are used in various applications, including video surveillance,
autonomous vehicles, augmented reality, and robotics.
3. Image and Video Enhancement: Signal processing techniques
are also applied to enhance the quality and visual appearance of
images and videos. Image enhancement techniques can improve
contrast, sharpness, color balance, and reduce noise or artifacts.
Video enhancement techniques address challenges specific to
video data, such as motion blur, video stabilization, and temporal
noise reduction. These techniques help to improve visual quality,
enhance details, and optimize the visual experience for various
applications, including broadcasting, multimedia, and medical
imaging.
Image and video processing techniques are essential in various fields,

including digital photography, entertainment, medical imaging, remote
sensing, and computer vision. These techniques enable efficient storage and
transmission of visual data, facilitate object recognition and tracking, and
enhance the visual quality of images and videos. Ongoing research in image
and video processing continues to advance the field, leading to improved
algorithms, better performance, and a wide range of applications in both
industry and academia.
Biomedical signal processing
Biomedical signal processing involves the application of signal processing

techniques to medical and healthcare-related data, specifically signals
acquired from the human body. Here are some key aspects covered in this
field:
1. Electrocardiogram (ECG) Analysis: Electrocardiogram (ECG)

analysis focuses on processing and interpreting the electrical
activity of the heart. Signal processing techniques are used to
extract meaningful information from the ECG signal, such as heart
rate, rhythm analysis, and detection of abnormal cardiac
conditions. Techniques like filtering, feature extraction, waveform
analysis, and pattern recognition are employed to analyze and
interpret the ECG signal. ECG analysis plays a crucial role in
diagnosing cardiac diseases, monitoring patient health, and
assessing the effectiveness of medical treatments.
2. Magnetic Resonance Imaging (MRI): Magnetic Resonance
Imaging (MRI) is a non-invasive medical imaging technique that
generates detailed images of internal body structures. In

biomedical signal processing, techniques are applied to MRI data
to enhance image quality, reduce noise, and improve image
reconstruction. Signal processing methods are used for image
denoising, image registration, image segmentation, and image
fusion. These techniques enable accurate diagnosis, visualization,
and analysis of anatomical and functional information in medical
imaging.
3. Brain-Computer Interfaces (BCIs): Brain-Computer Interfaces
(BCIs) are systems that enable direct communication between the
brain and external devices. Biomedical signal processing plays a
vital role in BCIs by processing and interpreting brain signals, such
as electroencephalogram (EEG) or functional magnetic resonance
imaging (fMRI) data. Signal processing algorithms are used to
extract relevant features from brain signals, perform classification,
and translate brain activity into control commands for external
devices. BCIs have promising applications in assistive
technologies, neurorehabilitation, and neuroprosthetics.
4. Breast Cancer Detection/Classification: Biomedical signal
processing techniques are used in breast cancer detection and
classification. For instance, mammograms, which are X-ray
images of the breast, can be analyzed using signal processing
techniques to detect abnormal patterns or masses indicative of
breast cancer. Signal processing methods, such as image
enhancement, feature extraction, and machine learning algorithms,
are employed to aid in the early detection and classification of
breast cancer. These techniques assist in improving the accuracy
of diagnosis and guiding appropriate treatment strategies.
5. Stroke Detection/Classification: Biomedical signal processing
plays a role in stroke detection and classification. For example, in
ischemic stroke, signal processing techniques can be applied to
analyze electroencephalogram (EEG) signals or blood flow data to
identify patterns indicative of stroke. Signal processing algorithms
can help detect abnormal brain activity or changes in blood flow,
enabling timely diagnosis and intervention. This facilitates the
classification of stroke types, such as ischemic or hemorrhagic,
aiding in effective treatment planning.
6. Wireless Capsule Endoscopy for Intestinal Imaging: Wireless
capsule endoscopy is a non-invasive medical imaging technique
used to visualize the gastrointestinal tract. It involves a pill-sized
capsule with a built-in camera that captures images as it passes
182 Chapter IV
through the digestive system. Biomedical signal processing

techniques are applied to the captured images to enhance image
quality, remove artifacts, and facilitate the interpretation of
intestinal structures and abnormalities. Image processing
algorithms, such as image segmentation and feature extraction, can
assist in detecting and classifying gastrointestinal conditions like
polyps, ulcers, or tumors. These techniques contribute to more
accurate diagnosis and monitoring of intestinal health.
Biomedical signal processing techniques contribute to various medical and

healthcare applications, including disease diagnosis, monitoring patient
health, treatment planning, and medical research. These techniques enable
the analysis and interpretation of physiological signals, provide insights into
the functioning of the human body, and support medical professionals in
making informed decisions. Ongoing research in biomedical signal
processing continues to advance the field, leading to improved algorithms,
enhanced diagnostic accuracy, and innovative applications in healthcare.
Communications and networking
Communications and networking systems heavily rely on signal processing

techniques for efficient and reliable transmission of information. Here are
some key aspects covered in this field:
1. Channel Coding: Channel coding is a technique used to enhance

the reliability of communication systems by adding redundancy to
the transmitted signals. Signal processing techniques, such as error
detection and correction codes, are applied to protect the
transmitted data from noise and channel impairments. These codes,
such as Reed-Solomon codes, convolutional codes, and turbo
codes, introduce redundancy that enables the receiver to detect and
correct errors, ensuring the integrity of the transmitted information.
Channel coding is essential in wireless communication systems,
satellite communication, and digital communication over noisy
channels.
2. Modulation Techniques: Modulation involves encoding
information onto a carrier signal to enable efficient transmission
over a communication channel. Signal processing techniques are
used to modulate the carrier signal by varying its amplitude,
frequency, or phase. Modulation techniques include amplitude
modulation (AM), frequency modulation (FM), and phase modulation
(PM). These techniques allow for the efficient transmission of
data, voice, and video signals over various communication

systems, such as radio, television, mobile networks, and satellite
communication.
3. Equalization Methods: Equalization is the process of
compensating for channel distortions and impairments introduced
during signal transmission. Signal processing techniques, such as
equalization algorithms, are employed to mitigate the effects of
channel distortion, including multipath fading, intersymbol
interference (ISI), and frequency-selective fading. Equalization
methods, such as linear equalizers, decision feedback equalizers
(DFE), and adaptive equalizers, aim to restore the transmitted
signal's original quality and mitigate the effects of channel
impairments, ensuring reliable and accurate communication.
4. Adaptive Design: Adaptive signal processing techniques play a
crucial role in communications and networking systems. Adaptive
designs enable systems to dynamically adjust their parameters
based on the changing environment or system conditions. Adaptive
filtering algorithms, such as the least mean squares (LMS)
algorithm or the recursive least squares (RLS) algorithm, are used
to adaptively estimate and track channel characteristics, equalize
distorted signals, and mitigate interference. Adaptive designs
allow communication systems to adapt and optimize their
performance in real-time, improving signal quality, data rates, and
system robustness.
5. Joint Communication and Sensing: Joint communication and
sensing refer to the integration of communication and sensing
functionalities in a unified system. Signal processing techniques
are applied to jointly optimize the communication and sensing
tasks, enabling efficient utilization of resources and improved
performance. For example, in cognitive radio systems, signal
processing techniques are used to sense the spectrum availability
and adaptively allocate communication resources to maximize
throughput while avoiding interference. Joint communication and
sensing approaches are also employed in applications such as radar
communication systems, wireless sensor networks, and distributed
sensing platforms.
6. Multiple Input Multiple Output (MIMO) Systems: MIMO
systems utilize multiple antennas at both the transmitter and
receiver to improve communication performance. Signal processing
techniques are applied to exploit the spatial diversity and multipath
propagation characteristics in MIMO systems. Techniques such as
184 Chapter IV
space-time coding, beamforming, and spatial multiplexing are

employed to enhance data rates, increase spectral efficiency, and
improve link reliability. MIMO systems are widely used in modern
wireless communication standards, such as Wi-Fi, 4G LTE, and
5G, to achieve higher data rates and improved system capacity.
7. Massive Antenna Arrays: In massive MIMO, the base station is
equipped with a significantly larger number of antennas compared
to traditional MIMO systems. While traditional MIMO systems
may have 2-4 antennas, massive MIMO can have tens or even
hundreds of antennas. This large number of antennas enables
improved spatial multiplexing, signal diversity, and interference
management.
8. Simultaneous Multi-User Communication: Massive MIMO
allows the base station to communicate with multiple users
simultaneously using the same time-frequency resources. Each
user is served with a dedicated beamformed signal from different
antenna elements, allowing for increased capacity and spectral
efficiency. The large antenna array enables spatial multiplexing,
where the base station can transmit different data streams to
different users in the same time-frequency resources.
9. Beamforming and Spatial Processing: Massive MIMO utilizes
advanced beamforming and spatial processing techniques to
enhance signal quality and improve system performance.
Beamforming involves steering the transmit beams towards each
user using precise antenna weightings, improving signal strength
and minimizing interference. Spatial processing algorithms, such
as channel estimation, precoding, and spatial multiplexing,
optimize the transmission and reception of signals to mitigate
interference and improve overall system capacity.
10. Interference Suppression and Energy Efficiency: Massive
MIMO systems have inherent interference suppression
capabilities. The large antenna array helps mitigate interference
from other users, allowing for better signal quality and improved
system performance. Moreover, massive MIMO systems can
achieve higher energy efficiency compared to traditional systems
by leveraging spatial processing techniques to focus transmit
energy towards the desired users and minimize wasteful radiation.
11. Localization: Localization refers to the process of determining the
position or location of an object or device in a given environment.
Signal processing techniques are commonly used for localization
in different contexts, such as:
1. Global Positioning System (GPS): GPS is a widely used

technology for outdoor localization. It relies on signal
processing algorithms to analyze the signals received
from multiple GPS satellites to determine the receiver's
position accurately.
2. Indoor Localization: In indoor environments where GPS
signals may be unavailable or inaccurate, alternative
techniques like Wi-Fi-based localization, Bluetooth
beacons, or Ultra-Wideband (UWB) signals can be used.
Signal processing algorithms are applied to analyze
received signal strengths, time-of-arrival, angle-of-
arrival, or other parameters to estimate the location of a
device within an indoor space.
3. Acoustic Localization: In scenarios where audio signals
are utilized, such as in underwater environments or room
acoustics, signal processing techniques can be applied to
estimate the location of sound sources. This involves
analyzing the time differences of arrival (TDOA) or phase
differences of audio signals received by multiple
microphones.
12. Tracking: Tracking involves continuously monitoring and
estimating the movement or trajectory of an object over time.
Signal processing techniques are applied to track objects using
various sensor data, including:
1. Radar Tracking: Radar systems emit radio waves and
analyze the reflected signals to track the motion of
objects, such as aircraft or vehicles. Signal processing
algorithms are used to estimate the target's range,
velocity, and position based on the radar echoes.
2. Visual Tracking: In computer vision applications, visual
tracking is used to follow objects in a video sequence.
Signal processing algorithms analyze the visual features
of objects and track their motion across frames by
estimating parameters such as position, velocity, or
appearance models.
3. Sensor Fusion: Tracking can also involve combining
information from multiple sensors, such as radar, LiDAR,
and cameras, to improve tracking accuracy. Signal
processing techniques are used to fuse the sensor data,
align coordinate systems, and estimate the object's
position and motion in a unified manner.
186 Chapter IV
Signal processing techniques in communications and networking systems

enable efficient and reliable transmission of information. These techniques
enhance the robustness of communication systems by incorporating error
detection and correction codes, optimize the utilization of available
bandwidth through modulation schemes, and compensate for channel
distortions using equalization methods. Ongoing research and
advancements in signal processing continue to improve the performance,
capacity, and reliability of communication networks, contributing to
advancements in wireless communication, internet technologies, and
beyond.
Sensor and data fusion
Sensor and data fusion is a crucial aspect of signal processing that aims to
leverage information from multiple sensors or data sources to enhance
decision-making and system performance. Here is a more detailed
explanation of this concept:
1. Data Integration: Sensor and data fusion involves integrating data

from various sensors or sources into a unified representation. This
integration can be challenging due to differences in data formats,
sampling rates, or measurement units. Signal processing
techniques are employed to synchronize, align, and normalize the
data, ensuring compatibility and coherence across the different
sources.
2. Feature Extraction: Once the data is integrated, signal processing
techniques are applied to extract relevant features or characteristics
from the combined data. These features capture important patterns,
trends, or properties of the underlying phenomenon being
monitored. Feature extraction may involve statistical analysis,
time-frequency analysis, wavelet transforms, or other signal
processing methods to identify meaningful information embedded
in the data.
3. Classification: Sensor and data fusion techniques also encompass
classification algorithms that use the extracted features to make
intelligent decisions or predictions. Classification methods, such
as support vector machines, neural networks, or decision trees, can
be employed to classify the fused data into different classes or
categories. For example, in an environmental monitoring system,
fused data from various sensors can be classified to identify
specific events or conditions, such as pollution levels or abnormal
behaviors.
4. Intelligent Decision-Making: The ultimate goal of sensor and

data fusion is to enable intelligent decision-making based on the
combined and processed information. By integrating data from
multiple sensors and extracting meaningful features, signal
processing facilitates a more comprehensive understanding of the
system under observation. This, in turn, supports intelligent
decision-making processes, such as anomaly detection, event
recognition, target tracking, or situation assessment.
5. Internet of Things (IoT): IoT refers to the network of
interconnected devices embedded with sensors, actuators, and
communication capabilities, allowing them to collect and
exchange data. Signal processing techniques are employed in IoT
systems for various purposes, including:
1. Data Preprocessing: IoT devices generate a massive
volume of data, often with noise, missing values, or
inconsistencies. Signal processing techniques are used to
preprocess the raw sensor data, removing noise, handling
missing values, and ensuring data quality before further
analysis.
2. Data Compression: Due to the limited bandwidth and
storage capacity of IoT devices, signal processing
techniques are used for data compression. By reducing
the size of data while preserving important information,
efficient data transmission and storage can be achieved.
3. Signal Filtering and Enhancement: Signal processing
algorithms are applied to filter out unwanted noise and
enhance the quality of sensor signals. This improves the
accuracy and reliability of the collected data, leading to
more effective analysis and decision-making.
6. IoT/AI Embedded Smart Systems: IoT systems integrated with
artificial intelligence (AI) capabilities are known as IoT/AI
embedded smart systems. Signal processing techniques are vital in
these systems to facilitate intelligent data analysis and decision-
making. Key aspects include:
1. Data Analytics: Signal processing techniques are
employed to analyze the collected sensor data and extract
meaningful insights. This may involve feature extraction,
pattern recognition, anomaly detection, or predictive
modeling using machine learning algorithms.
2. Adaptive Learning and Optimization: Signal processing
plays a role in adaptive learning and optimization within
188 Chapter IV
IoT/AI systems. By continuously analyzing and

processing incoming data, these systems can adapt their
behavior, optimize resource usage, and improve overall
performance.
3. Context Awareness: Signal processing techniques enable
IoT/AI embedded smart systems to be context-aware. By
analyzing sensor data, environmental conditions, and user
context, these systems can provide personalized services,
make intelligent decisions, and automate tasks based on
the specific context.
4. Real-Time Decision-Making: Signal processing
algorithms are applied to process sensor data in real-time,
enabling quick and accurate decision-making within
IoT/AI embedded smart systems. This is crucial for
applications such as smart homes, healthcare monitoring,
industrial automation, and transportation systems.
Sensor and data fusion find applications in various domains, including

surveillance systems, autonomous vehicles, environmental monitoring,
aerospace, and robotics. By combining information from multiple sources
and applying signal processing techniques, it becomes possible to extract
valuable insights, enhance system performance, improve reliability, and
enable more informed decision-making processes.
Rhyme Summary:
In this chapter's pages, we explore. Signal processing's applications galore.
Audio and speech, we process with care. Transforming sounds into words,
so fair.
Images and videos come to life. Through processing, reducing their strife.
Biomedical signals, a vital quest. Improving healthcare, we do our best.
Communications and networking, a vital link. Signal processing makes

connections sync.
Sensor and data fusion, a grand feat. Combining information, making

systems complete.
Across domains, signal processing shines. Extracting insights, enhancing

designs.
From audio to networking, fusion to view. Signal processing's power, we

pursue.
Layman's Guide:
In summary, this chapter highlights the broad range of signal processing
applications across different domains. From audio and speech processing to
image and video processing, biomedical signal processing, communications
and networking, and sensor and data fusion, signal processing techniques
are vital for extracting valuable insights, enhancing system performance,
and enabling advanced applications in each respective domain.
Exercises of applications of signal processing

Problem 1:
You are tasked with developing a signal processing system for a smart city
application. The system needs to address several key applications, including
environmental monitoring, energy management, intelligent traffic control,
and anomaly detection. Design a solution that incorporates signal processing
techniques to extract valuable insights, enhance system performance, and
enable advanced functionalities for each of these applications.
Solution:
To address the requirements of the smart city application, a comprehensive

signal processing system can be developed as follows:
1. Environmental Monitoring:
x Implement sensors to measure temperature and humidity
in different locations of the city.
x Use signal processing techniques such as smoothing and
normalization to process the sensor data.
x Analyze the processed data to monitor environmental
conditions and identify any anomalies or patterns.
2. Energy Management:
x Install smart energy meters to measure energy
consumption in residential and commercial buildings.
x Retrieve energy consumption data from the smart meters.
x Apply signal processing techniques such as peak
detection and average calculation to analyze the energy
consumption patterns.
190 Chapter IV
x Develop algorithms to optimize energy usage and identify

energy-saving opportunities.
3. Intelligent Traffic Control:
x Deploy traffic sensors and cameras at strategic locations
to monitor vehicle count and traffic speed.
x Retrieve data from the sensors and cameras for analysis.
x Utilize signal processing techniques to smooth and
process the collected traffic data.
x Develop algorithms for intelligent traffic control,
including traffic flow optimization and adaptive signal
control.
4. Anomaly Detection:
x Integrate data from multiple sources, including
environmental sensors, energy meters, and traffic sensors.
x Apply data fusion techniques to combine and analyze the
data.
x Utilize advanced signal processing algorithms for
anomaly detection, such as statistical modeling, pattern
recognition, and machine learning.
x Develop real-time monitoring and alerting systems to
identify and respond to anomalies in the city's operations.
5. Noise Reduction and Signal Enhancement:
x Implement noise reduction techniques to enhance the
quality of sensor data.
x Use denoising algorithms to reduce unwanted noise and
interference.
x Apply signal enhancement techniques such as filtering
and amplification to improve the accuracy and reliability
of the data.
6. Data Fusion and Integration:
x Develop algorithms for integrating and fusing data from
multiple sensors and sources.
x Apply data fusion techniques such as Kalman filtering,
Bayesian inference, or sensor fusion algorithms to
combine and refine data.
x Ensure seamless integration of data from different
domains to enable holistic analysis and decision-making.
7. Predictive Analytics:
x Utilize signal processing techniques to analyze historical
data and identify patterns or trends.
x Develop predictive models using machine learning

algorithms to forecast environmental conditions, energy
consumption, traffic flow, and other relevant parameters.
x Use predictive analytics to optimize resource allocation,
plan for future demands, and make data-driven predictions
for smart city operations.
8. Real-time Monitoring and Control:
x Implement real-time monitoring systems to continuously
collect and process sensor data.
x Utilize signal processing algorithms for real-time
analysis, anomaly detection, and event triggering.
x Develop control algorithms to dynamically adjust
systems based on the analyzed data, such as adjusting
traffic signal timings based on traffic flow patterns.
9. Data Visualization and User Interface:
x Design user-friendly interfaces and visualization tools to
present processed data in a meaningful and intuitive
manner.
x Develop interactive dashboards, graphs, and maps to
display real-time and historical data.
x Enable users to monitor environmental conditions, energy
consumption, traffic status, and anomalies through visual
representations.
10. Scalability and Adaptability:
x Design the signal processing system to be scalable and
adaptable to accommodate future expansion and
integration of additional sensors and applications.
x Ensure the system can handle large volumes of data and
adapt to changing requirements and technologies.
By considering these additional aspects in the solution, the smart city can
achieve comprehensive signal processing capabilities, enabling effective
monitoring, control, and optimization of various applications. The solution
will facilitate data-driven decision-making, resource efficiency, and
improved quality of life for the residents.
Problem 2:
How can signal processing techniques be applied to enhance the learning

environment and optimize resource management in a smart campus and
classroom setting? Develop a comprehensive solution that addresses
192 Chapter IV
attendance monitoring, lighting control, noise detection, lecture capture,

gesture recognition, emotion analysis, energy management, and personalized
learning.
Solution:
To apply signal processing techniques for smart campus and classroom We

can develop the following solutions:
1. Attendance Monitoring:
x Utilize signal processing techniques to process data from
attendance tracking systems, such as RFID or biometric
sensors.
x Develop algorithms to accurately detect and identify
students' presence in the classroom.
x Apply signal processing techniques to analyze attendance
data, identify patterns, and generate reports for
administrative purposes.
2. Smart Lighting Control:
x Implement sensors to monitor lighting conditions in
classrooms and campus areas.
x Use signal processing techniques to adjust lighting levels
based on occupancy, natural lighting, and user preferences.
x Develop algorithms for energy-efficient lighting control,
ensuring optimal lighting conditions while minimizing
energy consumption.
3. Noise and Disturbance Detection:
x Deploy microphones or sound sensors to monitor noise
levels in classrooms, study areas, and common spaces.
x Apply signal processing algorithms to detect and classify
different types of noises, such as conversation, loud
noises, or disturbances.
x Implement real-time alert systems to notify administrators or
teachers about noise violations or potential disruptions.
4. Smart Lecture Capture:
x Utilize signal processing techniques to enhance the
quality of audio and video recordings during lectures.
x Apply noise reduction and speech enhancement algorithms
to improve the clarity of recorded lectures.
x Implement automatic speech recognition (ASR) algorithms

to enable transcription and indexing of recorded lectures
for easy retrieval.
5. Gesture Recognition and Interaction:
x Implement cameras or depth sensors to capture hand
gestures and body movements in classrooms or
interactive spaces.
x Utilize signal processing techniques, such as computer
vision and machine learning, to recognize and interpret
gestures.
x Develop interactive applications that enable gesture-
based control of presentation slides, multimedia content,
or smart devices.
6. Emotion and Sentiment Analysis:
x Apply signal processing techniques to analyze facial
expressions, voice intonation, or physiological signals to
infer emotions and sentiments of students.
x Develop algorithms for real-time emotion detection,
providing valuable insights into students' engagement and
well-being.
x Use sentiment analysis to gauge student satisfaction and
identify areas for improvement in campus services or
teaching methods.
7. Smart Energy Management:
x Implement energy monitoring systems to track energy
consumption in classrooms, laboratories, and campus
buildings.
x Utilize signal processing algorithms for energy data
analysis, identifying energy-saving opportunities and
optimizing energy usage.
x Integrate with smart grid technologies to enable demand
response and load management strategies for efficient
energy utilization.
8. Personalized Learning:
x Utilize signal processing techniques to analyze students'
learning patterns, performance data, and feedback.
x Develop adaptive learning systems that tailor educational
content and activities based on individual student needs
and preferences.
194 Chapter IV
x Apply machine learning algorithms to personalize

recommendations and provide real-time feedback to
enhance learning outcomes.
By applying signal processing techniques to smart campus and classroom

environments, educational institutions can benefit from enhanced
efficiency, improved learning experiences, and optimized resource
management. These applications enable data-driven decision-making,
promote student engagement, and create an environment conducive to
personalized and interactive learning.
Problem 3:
In the context of futuristic applications like the metaverse, how can signal
processing techniques be applied to enhance the immersive experience,
optimize data processing, and enable seamless interactions within virtual
environments? Develop a comprehensive solution that addresses audio
processing, video processing, user interactions, data integration, and real-
time communication within the metaverse.
Solution:
To apply signal processing techniques for futuristic applications like the

metaverse, we can develop a comprehensive solution that focuses on
several key aspects:
1. Immersive Audio Processing:

x Utilize advanced signal processing algorithms to create
realistic spatial audio effects within the metaverse.
x Implement techniques such as sound localization,
reverberation, and audio scene analysis to enhance the
immersive audio experience.
x Apply adaptive filtering and noise reduction algorithms
to improve the clarity of audio signals and reduce
background noise.
2. Enhanced Video Processing:
x Develop video processing techniques to enhance visual
quality and realism within virtual environments.
x Apply image and video compression algorithms to
optimize bandwidth utilization without compromising
visual fidelity.
x Implement real-time video processing techniques, such as

object tracking and recognition, to enable dynamic
interactions and augmented reality overlays.
3. User Interaction and Gestural Control:
x Utilize signal processing techniques, such as gesture
recognition and hand tracking, to enable natural and
intuitive user interactions within the metaverse.
x Develop algorithms for real-time analysis of body
movements and gestures to control avatars, objects, and
virtual interfaces.
x Implement haptic feedback systems to provide realistic
tactile sensations, further enhancing the sense of immersion.
4. Data Integration and Fusion:
x Develop algorithms for integrating and fusing data from
various sources within the metaverse, such as sensor data,
user inputs, and environmental data.
x Apply signal processing techniques to analyze and
interpret the combined data streams, enabling real-time
adaptation and dynamic environment rendering.
x Implement data fusion algorithms to ensure seamless
integration of diverse data types, enhancing the overall
metaverse experience.
5. Real-Time Communication and Collaboration:
x Utilize signal processing techniques to enable real-time
audio and video communication between users within the
metaverse.
x Implement echo cancellation, noise suppression, and
bandwidth optimization algorithms to ensure high-quality
and low-latency communication.
x Develop collaborative features, such as shared whiteboards
or virtual meeting spaces, to facilitate effective
collaboration and interaction among users.
By applying these signal processing techniques within the metaverse, we

can create a highly immersive and interactive virtual environment. The
solution aims to optimize the audiovisual experience, enable natural user
interactions, integrate diverse data sources, and support seamless real-time
communication. This will contribute to the development of a futuristic
metaverse where users can engage, collaborate, and explore virtual worlds
with unprecedented realism and interactivity.
196 Chapter IV
Problem 4:
You are working on a medical imaging project that aims to detect breast
cancer using ultra-wideband (UWB) signals. Your task is to design a
MATLAB script to generate a UWB signal and visualize the resulting image
of breast tissue. The script should also implement a thresholding technique
to detect the presence of cancer in the image.
Requirements:
1. Generate a UWB signal with the following specifications:

x Sample rate: 1 GHz
x Pulse duration: 1 ns
x Center frequency: 3 GHz
x Bandwidth: 4 GHz
2. Simulate the presence of cancer in the UWB signal by introducing
an abnormality. The cancer signal should start at a specific time,
have a predefined duration, and a specified amplitude.
3. Add noise to the UWB signal to simulate real-world conditions.
The Signal-to-Noise Ratio (SNR) should be adjustable.
4. Implement a thresholding technique to segment the cancer region
in the image. Choose an appropriate threshold value.
5. Visualize the UWB signal with cancer in the time domain.
6. Visualize the UWB signal spectrum with cancer in the frequency
domain using a logarithmic (dB) scale.
7. Display the detection result indicating whether cancer is detected
or not based on the thresholding technique.
Your task is to write a MATLAB script that fulfills the requirements

mentioned above. Make sure to include comments to explain the different
sections of your code.
Solution:
UWB signals are characterized by their extremely short duration and wide
bandwidth. They are commonly used in applications such as radar,
communications, and medical imaging. In medical imaging, UWB signals
can be employed for breast cancer detection due to their ability to provide
high-resolution images and better tissue differentiation.
Generating a UWB signal involves designing a pulse with a short duration

and a wide bandwidth. Here is a brief explanation of the steps involved in
generating a UWB signal:
1. Define the parameters:

x Sample rate (‫ܨ‬௦ ): It represents the number of samples per
second and determines the resolution of the generated
signal.
x Pulse duration (ܶ): It determines the time span of the
UWB pulse. UWB pulses are typically on the order of
picoseconds or nanoseconds.
x Center frequency (‫ܨ‬௖ ): It represents the central frequency
of the UWB pulse.
x Bandwidth (‫)ܤ‬: It determines the range of frequencies
covered by the pulse and influences the signal resolution.
2. Create the time vector:
x The time vector is generated using the linspace function
in MATLAB. It creates a vector of equally spaced time
values within the desired pulse duration.
x For example: t = linspace(0, T, T * Fs); creates a time
vector ranging from 0 to T seconds with T * Fs samples.
3. Generate the UWB pulse:
x A common approach for generating a UWB pulse is to
use a Gaussian function.
x The normpdf function in MATLAB can be used to create
a Gaussian pulse with a mean (center) at T/2 and a
standard deviation determining the pulse width.
x For example: pulse = normpdf(t, T/2, T/8); generates a
Gaussian pulse centered at T/2 with a standard deviation
of T/8.
4. Modulate the pulse to the desired center frequency:
x To modulate the pulse to the desired center frequency, a
carrier signal is used.
x By multiplying the pulse with a cosine function at the
desired center frequency, the pulse is shifted to the
frequency domain.
x For example: modulated_pulse = pulse .* cos(2 * pi *
Fc * t); modulates the pulse with a cosine function at the
frequency Fc.
5. Visualize the UWB signal:
x The resulting modulated pulse represents the generated
UWB signal.
198 Chapter IV
x To visualize the UWB signal, you can use the plot

function in MATLAB to plot the time-domain
representation of the modulated pulse.
x For example: plot(t, modulated_pulse); plots the
modulated pulse as a function of time.
6. Process the UWB signal for imaging:
x After generating the UWB signal, further signal
processing techniques are applied to extract information
and generate an image.
x Various techniques can be used, such as time-domain or
frequency-domain analysis, beamforming, or image
reconstruction algorithms.
x These techniques exploit the unique properties of UWB
signals to obtain high-resolution images and distinguish
different tissues within the breast.
It is important to note that the specific implementation and signal processing

techniques for UWB imaging can vary depending on the specific
application, imaging goals, and available hardware. Advanced algorithms
and methods are often employed to enhance image quality and extract
relevant information from the UWB signals.
MATLAB example:
% Parameters
Fs = 1e9; % Sample rate (1 GHz)
T = 1e-9; % Pulse duration (1 ns)
Fc = 3e9; % Center frequency (3 GHz)
B = 4e9; % Bandwidth (4 GHz)
threshold = 0.5; % Threshold for cancer detection
% Create the time vector

t = linspace(0, 10*T, 10*T * Fs);
% Generate the UWB pulse

pulse = normpdf(t, 5*T, T/8);
% Modulate the pulse to the desired center frequency

modulated_pulse = pulse .* cos(2 * pi * Fc * t);
% Simulate cancer by introducing an abnormality

cancer_start_time = 5*T; % Time when the cancer starts
cancer_duration = T; % Duration of the cancer signal

cancer_amplitude = 2; % Amplitude of the cancer signal
cancer_signal = zeros(size(t));
cancer_indices = t >= cancer_start_time & t <= (cancer_start_time +
cancer_duration);
cancer_signal(cancer_indices) = cancer_amplitude;
% Add the cancer signal to the UWB signal

signal_with_cancer = modulated_pulse + cancer_signal;
% Add noise to simulate real-world conditions

SNR = 10; % Signal-to-Noise Ratio in dB
noise = randn(size(t));
noise_power = norm(modulated_pulse)^2 / (10^(SNR/10));
noisy_signal = signal_with_cancer + sqrt(noise_power) * noise;
% Cancer detection using thresholding

cancer_detected = max(noisy_signal) > threshold;
% Plot the UWB signal with cancer and the detection result in the same
figure
figure;
subplot(2, 1, 1);
plot(t, signal_with_cancer, 'b');
hold on;
plot(t, noisy_signal, 'r--');
title('UWB Signal with Cancer (Time)');
xlabel('Time');
legend('Signal with Cancer', 'Noisy Signal');
% Plot the UWB signal with cancer in the frequency domain (dB scale)
subplot(2, 1, 2);
f = linspace(-Fs/2, Fs/2, length(t));
signal_with_cancer_freq = fftshift(abs(fft(signal_with_cancer)));
plot(f, 20*log10(signal_with_cancer_freq), 'b');
hold on;
plot([min(f), max(f)], [20*log10(threshold), 20*log10(threshold)], 'k--');
title('UWB Signal Spectrum with Cancer (Frequency)');
xlabel('Frequency');
200 Chapter IV
ylabel('Magnitude (dB)');
xlim([-Fs/2, Fs/2]);
ylim([min(20*log10(signal_with_cancer_freq)),
max(20*log10(signal_with_cancer_freq))]);
% Display the detection result

if cancer_detected
detection_result = 'Cancer detected!';
else
detection_result = 'No cancer detected.';
end
text(-0.8*Fs/2, 20*log10(threshold)+10, detection_result, 'Color', 'red',
'FontWeight', 'bold');
% Adjust plot spacing and labels

subplot(2, 1, 1);
legend('Signal with Cancer', 'Noisy Signal');
subplot(2, 1, 2);
ylabel('Magnitude (dB)');
legend('Signal Spectrum with Cancer', 'Threshold');
% Adjust y-axis scale to dB for the frequency domain plot

subplot(2, 1, 2);
set(gca, 'YScale', 'linear');
This is an example MATLAB code that generates a UWB signal and

demonstrates a simple approach for cancer detection using thresholding. A
simulated cancer signal is introduced by creating an abnormality that
appears as a separate signal within the UWB signal. You can specify the
start time, duration, and amplitude of the cancer signal to control its
characteristics. The cancer signal is then added to the modulated UWB pulse
to create a UWB signal with simulated cancer.
The code generates a UWB signal by creating a Gaussian pulse, modulating

it to the desired center frequency, and adding noise to simulate real-world
conditions. It then performs cancer detection by comparing the maximum
value of the noisy signal with a predefined threshold.
Please note that this is a simplified example for demonstration purposes. In

real-world scenarios, more sophisticated signal processing techniques and
classification algorithms are typically employed for accurate cancer detection.
These can include feature extraction, machine learning algorithms, and

statistical analysis. The specific techniques used depend on the available
data, imaging system, and the complexity of the cancer detection task.
Note that this simulation is for illustrative purposes only and does not
capture the full complexity of real cancer signals. In practice, cancer
detection requires more advanced techniques and accurate models based on
clinical data and imaging systems.
Figure 3-29 showcases an example of a medical imaging project that utilizes

UWB signals for breast cancer detection. The figure provides a visual
representation of the signals in both the time and frequency domains,
demonstrating the key aspects of the detection process.
Figure 3-29: An example of a medical imaging project using ultra-wideband (UWB)

signals for breast cancer detection.
Problem 5:
Design a MATLAB script to generate 2-D images representing breast tissue
with and without cancer using UWB signals. The goal is to visualize the
differences between normal tissue and cancerous tissue. Write a script that
creates a 2-D grid and generates two images: one depicting breast tissue
202 Chapter IV
with a cancerous region and another representing normal breast tissue

without any cancer. Apply appropriate colormaps to differentiate between
the two tissue conditions. Finally, plot the images side by side to compare
the cases of with and without cancer.
Solution:
The design is based on the following principles and concepts:
x UWB Signals: UWB signals are characterized by their broad

frequency bandwidth and short duration pulses. They are used in
medical imaging due to their ability to penetrate tissues and
provide high-resolution images.
x Breast Tissue Image: The code generates a 2-D grid representing
the breast tissue. Each pixel in the grid corresponds to a location in
the tissue, forming a coordinate system.
x Cancerous Region: The code simulates the presence of cancerous
tissue by defining a region within the breast tissue image. This
region represents the cancerous area, characterized by its size,
shape, and position.
x Amplitude Representation: The amplitude of the image pixels
represents the intensity or energy of the UWB signal reflected or
transmitted through the breast tissue. The code assigns higher
amplitude values to the cancerous region compared to the normal
tissue.
x Colormap Visualization: Colormaps are used to visually represent
the amplitude values in the image. Different colors are assigned to
different amplitude levels, aiding in the visual distinction between
normal tissue and cancerous tissue.
By generating and visualizing these 2-D images, the code allows for a visual
understanding of the differences between normal tissue and cancerous
tissue. It provides a means to analyze and compare the amplitude
distribution in different tissue regions, which can be beneficial for cancer
detection and classification purposes.
MATLAB example:
% Parameters
Nx = 200; % Number of pixels in x-direction
Ny = 200; % Number of pixels in y-direction
threshold = 0.5; % Threshold for cancer detection
% Create a 2-D grid

[X, Y] = meshgrid(linspace(-1, 1, Nx), linspace(-1, 1, Ny));
% Generate an image of breast tissue with cancer

cancer_radius = 0.3; % Radius of the cancer region
cancer_center = [0.2, -0.2]; % Center coordinates of the cancer region
cancer_amplitude = 1; % Amplitude of the cancer region
image_with_cancer = zeros(Ny, Nx);

cancer_indices = sqrt((X - cancer_center(1)).^2 + (Y - cancer_center(2)).^2)
<= cancer_radius;
image_with_cancer(cancer_indices) = cancer_amplitude;
% Generate an image of normal breast tissue without cancer

image_without_cancer = zeros(Ny, Nx);
% Plot the image with cancer and the image without cancer
figure;
subplot(1, 2, 1);
imagesc(X(1,:), Y(:,1), image_with_cancer);
title('Image with Cancer');
xlabel('X');
ylabel('Y');
colormap(gca, [1 1 1; 1 0 0]); % White for normal tissue, red for cancerous
tissue
colorbar;
axis equal;
axis tight;
subplot(1, 2, 2);
imagesc(X(1,:), Y(:,1), image_without_cancer);
title('Image without Cancer');
xlabel('X');
ylabel('Y');
colormap(gca, [1 1 1; 0 0 1]); % White for normal tissue, blue for healthy
tissue
colorbar;
axis equal;
axis tight;
% Adjust plot spacing

204 Chapter IV
Figure 3-30 illustrates another example of a medical imaging project that

utilizes ultra-wideband (UWB) signals for breast cancer detection. In this
case, the project focuses on generating 2-D images to visualize the
differences between normal breast tissue and cancerous tissue. The images
are created by simulating the presence of cancerous regions within the
breast tissue using UWB signal properties. Through appropriate
visualization techniques such as colormaps, the images provide a clear
visual representation of the distinctions between normal and cancerous
tissue areas. This approach facilitates the analysis and understanding of
breast cancer detection using UWB signals.
Figure 3-30: An example of a medical imaging project using 2-D images and UWB
signals for breast cancer detection.
CHAPTER V
FUTURE DIRECTIONS IN SIGNAL PROCESSING
In this chapter, the focus is on the future directions of signal processing,

highlighting emerging techniques, challenges, opportunities, and
concluding remarks. Here is a summary of the key sections covered in this
chapter:
1. Emerging Signal Processing Techniques and Applications: The

chapter begins by discussing the emergence of new signal
processing techniques and applications. It explores cutting-edge
areas such as quantum signal processing, which leverages
principles from quantum mechanics to enhance signal processing
capabilities. Additionally, it explores signal processing for
blockchain, demonstrating how signal processing techniques can
be applied to address challenges and enhance the security of
blockchain technology.
x Quantum Signal Processing: Quantum signal processing is

an exciting field that combines concepts from quantum
mechanics and signal processing to improve signal processing
capabilities. Here are a few examples of how quantum signal
processing techniques can be applied:
¾ Quantum Fourier Transform: The quantum
Fourier transform (QFT) is a quantum algorithm that
can efficiently perform the Fourier transform of a
quantum state. This allows for efficient analysis of
signals in the quantum domain and can provide
advantages over classical signal processing methods.
¾ Quantum Sensing and Metrology: Quantum
sensors can leverage quantum properties to achieve
high precision and sensitivity in measuring physical
quantities such as time, frequency, and magnetic
fields. These sensors can be used for applications
such as signal detection, imaging, and environmental
monitoring.
206 Chapter V
¾ Quantum Machine Learning: Quantum machine

learning algorithms are being explored to process and
analyze signals efficiently. These algorithms utilize
quantum systems to perform tasks such as
classification, clustering, and regression, opening up
new possibilities for signal processing in machine
learning applications.
x Signal Processing for Blockchain: Blockchain technology
has revolutionized various industries by providing a
decentralized and secure platform for transactions and data
management. Signal processing techniques can be applied to
address challenges and enhance the security of blockchain
technology. Here are some examples:
¾ Anomaly Detection: Signal processing algorithms
can analyze the transaction data within a blockchain
network to identify anomalies or suspicious
activities. By detecting abnormal patterns, it
becomes possible to enhance the security and
integrity of the blockchain network.
¾ Data Validation and Consensus: Signal processing
techniques can be employed to validate the data
stored in the blockchain. By analyzing the
consistency, reliability, and authenticity of the data,
signal processing algorithms can contribute to
ensuring the integrity and trustworthiness of the
blockchain ledger.
¾ Privacy Preservation: Signal processing techniques
can be used to develop privacy-preserving mechanisms
within blockchain networks. For instance, techniques
such as homomorphic encryption and secure
multiparty computation can enable computations on
encrypted data, thereby preserving the privacy of
sensitive information stored on the blockchain.
These examples demonstrate how emerging signal processing

techniques can be applied in areas like quantum signal processing
and signal processing for blockchain technology. These
advancements have the potential to revolutionize various fields by
improving signal processing capabilities, enhancing security, and
enabling novel applications. Ongoing research in these areas holds
Future Directions in Signal Processing 207
promise for future developments and breakthroughs in signal

processing.
2. Challenges and Opportunities in Signal Processing Research:

The section on challenges and opportunities in signal processing
research dives into the current landscape of the field. It explores
the challenges faced by researchers in signal processing, including
the development of more efficient algorithms capable of handling
increasingly complex data. It also emphasizes the importance of
addressing privacy and security concerns to ensure the ethical and
secure use of signal processing techniques. Moreover, it presents
opportunities for advancements in machine learning, artificial
intelligence, and data-driven approaches that can further enhance
signal processing capabilities.
Challenges in Signal Processing Research:
1. Complexity of Data: As the volume and complexity of

data continue to grow rapidly, one of the primary
challenges in signal processing is developing efficient
algorithms that can handle and process large-scale, high-
dimensional data effectively. Signal processing researchers
need to explore innovative techniques to extract valuable
information from complex datasets while managing
computational complexity and maintaining real-time
processing capabilities.
2. Privacy and Security: With the increasing prevalence of
data sharing and interconnected systems, privacy and
security concerns have become critical in signal
processing. Researchers face the challenge of developing
robust signal processing methods that ensure data
privacy, protect against cyber-attacks, and address ethical
concerns related to data collection, storage, and
processing.
3. Real-Time Processing: Many signal processing
applications require real-time or near real-time processing
to make timely decisions or respond to dynamic
environments. Developing algorithms and architectures
that can handle high-speed data streams and provide real-
time analysis poses a significant challenge for researchers.
4. Interpretability and Explainability: Signal processing
techniques often involve complex models and algorithms.
208 Chapter V
Ensuring the interpretability and explainability of these

models is crucial, especially in critical applications such
as healthcare or finance, where decisions need to be
transparent and understandable. Researchers need to
address this challenge by developing techniques to
explain the outcomes and provide insights into the signal
processing results.
Opportunities in Signal Processing Research:
1. Advancements in Machine Learning and Artificial

Intelligence: Signal processing research can leverage
advancements in machine learning and artificial
intelligence (AI) to enhance its capabilities. Techniques
such as deep learning, reinforcement learning, and neural
networks can be applied to signal processing tasks,
enabling automatic feature extraction, pattern
recognition, and decision-making.
2. Data-Driven Approaches: The availability of large-
scale datasets and advancements in data acquisition
techniques present opportunities for signal processing
researchers to develop data-driven approaches. By
leveraging big data and applying statistical and machine
learning techniques, researchers can uncover hidden
patterns, extract meaningful information, and improve the
accuracy and efficiency of signal processing algorithms.
3. Interdisciplinary Collaborations: Signal processing
research can benefit from interdisciplinary collaborations.
Collaborations with experts from domains such as
medicine, finance, communications, and environmental
sciences can lead to innovative applications and the
development of tailored signal processing techniques to
address specific challenges in these fields.
4. Edge Computing and Internet of Things (IoT): The
proliferation of edge devices and IoT networks generates
vast amounts of data that require efficient signal
processing solutions. Researchers have the opportunity to
develop signal processing algorithms that are specifically
designed for resource-constrained edge devices, enabling
real-time analysis and decision-making at the network's
edge.
Future Directions in Signal Processing 209
By addressing these challenges and capitalizing on the

opportunities, signal processing research can advance and pave the
way for new applications in various domains, improve data
analysis capabilities, and contribute to scientific and technological
advancements.
3. Concluding Remarks: The chapter concludes with final remarks,
underlining the significance of signal processing in various
domains and its potential impact on future developments. It
emphasizes the importance of continuous exploration, innovation,
and collaboration in signal processing research to unlock new
possibilities and address emerging challenges. The chapter also
highlights the role of signal processing in shaping the future of
technology and its potential to revolutionize industries across the
board.
Rhyme Summary:
In this chapter's quest. Signal processing's future we invest.
Emerging techniques take flight. Quantum and blockchain shining bright.
Challenges and opportunities, we see. Efficient algorithms and security's

plea.
Research and development pave the way. Transforming technology, day by

day.
Signal processing is power, it is clear. Advancing fields far and near.
Layman's Guide:
In summary, this chapter delves into the future directions of signal
processing, exploring emerging techniques such as quantum signal
processing and signal processing for blockchain. It addresses the challenges
and opportunities that lie ahead, including the need for more efficient
algorithms and the importance of privacy and security considerations. The
chapter concludes by emphasizing the importance of ongoing research and
development in signal processing, highlighting its potential to drive
transformative advancements in technology and impact diverse fields.
CHAPTER VI
APPENDICES
Mathematical and computational tools for signal

processing
The appendices section of this chapter focuses on the mathematical and
computational tools that are essential for signal processing. These tools
provide the foundation for understanding and implementing signal
processing techniques.
Signal processing involves working with signals, which are essentially

patterns of information. To process these signals effectively, we need
mathematical and computational tools that help us analyze, manipulate, and
make sense of the data contained within the signals.
Mathematical tools in signal processing include concepts and techniques

from areas like algebra, calculus, and probability theory. These tools help
us describe and model signals mathematically, enabling us to perform
various operations on them. For example, we can use mathematical tools to
extract important features from signals, remove unwanted noise, or
compress signals to reduce their size while maintaining important
information.
Computational tools are software programs or algorithms that implement

signal processing techniques on computers. These tools allow us to apply
complex mathematical operations to signals efficiently. They provide us
with the ability to perform tasks like filtering, transforming, and analyzing
signals using powerful algorithms. Computational tools can be implemented
using programming languages like MATLAB, Python, or specialized
software packages designed for signal processing tasks.
By using mathematical and computational tools, signal processing

engineers and researchers can explore and develop new techniques to
extract useful information from signals. These tools enable us to uncover
Appendices 211
hidden patterns, enhance the quality of signals, and make informed

decisions based on the analyzed data.
Imagine you have a collection of different sounds recorded from various

sources, such as musical instruments, voices, and environmental noises.
These sounds are like patterns of information, and signal processing is all
about understanding and manipulating these patterns to make sense of the
data they contain.
To effectively process these sounds, we need mathematical tools that help

us describe and model them. Just like algebra helps us solve equations,
calculus helps us understand rates of change, and probability theory helps
us analyze uncertainty, these mathematical concepts are used in signal
processing to describe and manipulate the signals mathematically.
For example, let us say you want to extract the melody from a recorded
piece of music. By using mathematical tools, you can analyze the pattern of
frequencies in the sound and identify the important features that make up
the melody. You can also remove any unwanted background noise from the
recording, making the melody clearer and more enjoyable to listen to.
Additionally, computational tools play a crucial role in signal processing.

They are like software programs or algorithms that implement the
mathematical techniques on computers. These tools allow you to apply
complex mathematical operations to signals efficiently. They provide you
with the ability to perform tasks like filtering out unwanted frequencies,
transforming the signals into different representations, and analyzing the
signals using powerful algorithms.
Think of these computational tools as the “magic” that takes the

mathematical concepts and turns them into practical actions. They can be
implemented using programming languages like MATLAB or Python, or
specialized software packages designed specifically for signal processing
tasks.
By utilizing these mathematical and computational tools, signal processing

engineers and researchers can explore new techniques to extract useful
information from signals. They can uncover hidden patterns, enhance the
quality of signals, and make informed decisions based on the analyzed data.
Here are some additional mathematical concepts and operations used in

signal processing, along with their corresponding math equations and
MATLAB code examples:
212 Chapter VI
1. Fourier Transform: The Fourier Transform is a mathematical

operation that decomposes a signal into its frequency components. It
allows us to analyze the frequency content of a signal.
ஶ
x Math Equation: ‫ି׬ = )߱(ܨ‬ஶ ݂(‫ି ݁)ݐ‬௝ఠ௧ ݀‫ݐ‬
x MATLAB Code:
% Compute Fourier Transform

F = fft(f);
% Plot the magnitude spectrum
plot(abs(F)); xlabel('Frequency'); ylabel('Magnitude');
2. Discrete Fourier Transform (DFT): The DFT is a discrete version of

the Fourier Transform, commonly used to analyze digital signals. It
converts a discrete-time signal from the time domain to the frequency
domain.
x Math Equation: X[k] = σ[‫݁ ]݊[ݔ‬െ݆ଶగ௞௡/ே ]

x MATLAB Code:
% Compute DFT
X = fft(x);
plot(abs(X)); xlabel('Frequency'); ylabel('Magnitude');
3. Fast Fourier Transform (FFT): The FFT is an efficient algorithm for

computing the DFT. It speeds up the calculation of the DFT by
exploiting symmetry properties and reducing the number of
computations.
x Math Equation: Same as DFT, but computed using a more

efficient algorithm.
x MATLAB Code: Same as DFT
% Compute FFT
X = fft(x);
plot(abs(X)); xlabel('Frequency'); ylabel('Magnitude');
4. Z-transform: The Z-transform is a mathematical transform used for

analyzing discrete-time systems. It provides a way to represent discrete
signals and systems in the complex plane.
Appendices 213
x Math Equation: ܺ(‫ = )ݖ‬σ[‫ି( ݖ ]݊[ݔ‬௡) ]

x MATLAB Code:
% Compute Z-transform
X = ztrans(x, n, z);
% Plot the Z-transform
plot(X); xlabel('z'); ylabel('X(z)');
5. Convolution: Convolution is an operation used to combine two signals

to produce a third signal. It is widely used in signal processing for tasks
such as filtering and convolutional neural networks.
x Math Equation: (݂ ‫ ݐ(݃ כ )߬(݂[ ׬ = )ݐ()݃ כ‬െ ߬)] ݀߬

x MATLAB Code:
% Perform convolution y = conv(f, g);

% Plot the convolved signal
plot(y); xlabel('Time'); ylabel('Amplitude');
6. Auto-correlation: Auto-correlation is a measure of the similarity

between a signal and a delayed version of itself. It is used to detect
repeating patterns and estimate signal properties.
x Math Equation: ܴ௫௫ [݇] = σ[‫ ݊[ݔ כ ]݊[ݔ‬െ ݇]]

x MATLAB Code:
% Compute auto-correlation Rxx = xcorr(x);

% Plot the auto-correlation function
plot(Rxx); xlabel('Time Lag'); ylabel('Auto-correlation');
7. Cross-correlation: Cross-correlation measures the similarity between

two signals as a function of the time lag. It is used for tasks such as
aligning signals and detecting similarities between different signals.
x Math Equation: ܴ௫௬ [݇] = σ[‫ ݊[ݕ כ ]݊[ݔ‬െ ݇]]

x MATLAB Code:
% Compute cross-correlation
Rxy = xcorr(x, y);
% Plot the cross-correlation function
plot(Rxy); xlabel('Time Lag'); ylabel('Cross-correlation');
214 Chapter VI
8. Wavelet Transform: The wavelet transform is a mathematical

transform that analyzes signals at different scales. It is used for tasks
such as signal denoising, compression, and feature extraction.
x Math Equation: The discrete wavelet transform (DWT) is

computed using a discrete set of scales and translations. It
decomposes a discrete-time signal ‫ )݊(ݔ‬into a set of coefficients at
different scales and positions.
The DWT can be expressed as

ஶ
ܹ(ܽ, ܾ) = ෍ ‫ ݊((߰ )݊(ݔ‬െ ܾ)/2௔ ).

௡ୀିஶ
In this equation, ܹ(ܽ, ܾ) represents the wavelet coefficients at

scale ܽ and position ܾ, ‫ )݊(ݔ‬is the input signal, and ߰(ή) is the
wavelet function. The scale parameter ܽ controls the size of the
wavelet, and the translation parameter ܾ determines the position of
the wavelet within the signal.
The DWT can be computed iteratively by applying a series of low-

pass and high-pass filters and downsampling operations to the
input signal. The resulting coefficients capture different frequency
bands of the signal at varying resolutions.
By analyzing the wavelet coefficients, one can gain insights into

the frequency content of the signal at different scales. This allows
for tasks such as denoising by removing coefficients associated
with noise, compression by discarding less significant coefficients,
and feature extraction by identifying important coefficients related
to specific signal characteristics.
MATLAB Code:
% Compute Wavelet Transform

[C, L] = wavedec(x, wavelet, level);
% Plot the wavelet coefficients
plot(C); xlabel('Scale'); ylabel('Coefficient');
9. Statistical Analysis: Signal processing often involves statistical

analysis to estimate signal parameters, model noise, and make
inferences about signal properties.
Appendices 215
x Math Equation: Various statistical techniques such as parameter

estimation, hypothesis testing, and Bayesian inference.
x MATLAB Code: Dependent on the specific statistical analysis
being performed.
Examples: The mean and variance of a noisy signal
Specifically, we estimate the mean and variance of a noisy signal.
Math Equation: The estimation of mean (ߤ) and variance (ߪ ଶ ) can be done
using the following sample mean and sample variance formulas:
Sample mean (ߤ) = (1/N) σ ‫]݊[ݔ‬ and sample variance (ߪ ଶ ) =

(1/N)σ(‫ ]݊[ݔ‬െ ߤƸ)ଶ .
MATLAB Code:
% Generate a noisy signal

N = 100; % Number of samples
signal = 5 * sin(2*pi*(1:N)/10); % Original signal
noise = randn(1, N); % Gaussian noise
noisy_signal = signal + noise; % Noisy signal
% Estimate mean and variance

mean_estimate = mean(noisy_signal);
variance_estimate = var(noisy_signal);

fprintf('True mean: %.2f\n', mean(signal));
fprintf('Estimated mean: %.2f\n', mean_estimate);
fprintf('True variance: %.2f\n', var(signal));
fprintf('Estimated variance: %.2f\n', variance_estimate);
% Plot the original signal and the noisy signal

t = 1:N;
subplot(2,1,1);
plot(t, signal);
xlabel('Time');
subplot(2,1,2);
plot(t, noisy_signal);
216 Chapter VI
xlabel('Time');
Examples: Hypothesis testing using statistical analysis in signal processing
Here is an example that demonstrates hypothesis testing using statistical

analysis in signal processing. Specifically, we perform a hypothesis test to
determine if a received signal is more likely to belong to one of two possible
transmitted signals.
Math Equation: Hypothesis testing involves formulating null and alternative

hypotheses, selecting a significance level, and using statistical tests to make
inferences about the hypotheses. In this example, we consider two
transmitted signals A and B, and the null hypothesis (H0) is that the received
signal belongs to signal A, while the alternative hypothesis (H1) is that the
received signal belongs to signal B.
MATLAB Code:
% Generate transmitted signals A and B

N = 100; % Number of samples
signal_A = sin(2*pi*(1:N)/10); % Signal A
signal_B = cos(2*pi*(1:N)/10); % Signal B
% Generate received signal with added noise

received_signal = signal_A + randn(1, N); % Received signal with noise
% Perform hypothesis test

alpha = 0.05; % Significance level
[h, p] = ttest2(received_signal, signal_A, 'Alpha', alpha);

if h == 1
fprintf('Reject the null hypothesis. The received signal is more likely to
belong to signal B.\n');
else
fprintf('Fail to reject the null hypothesis. The received signal is more
likely to belong to signal A.\n');
end
fprintf('p-value: %.4f\n', p);
Appendices 217
% Plot the transmitted signals and the received signal

t = 1:N;
subplot(3,1,1);
plot(t, signal_A);
title('Signal A');
xlabel('Time');
subplot(3,1,2);
plot(t, signal_B);
title('Signal B');
xlabel('Time');
subplot(3,1,3);
plot(t, received_signal);
title('Received Signal');
xlabel('Time');
In this example, we generate two transmitted signals A and B, represented

by sinusoidal waves. We then generate a received signal by adding noise to
signal A. We perform a two-sample t-test (ttest2 function in MATLAB) to
compare the received signal with signal A, with a specified significance
level (alpha). The test returns a hypothesis decision (h) and a p-value (p).
The results are displayed, indicating whether we reject or fail to reject the
null hypothesis based on the significance level. Finally, we plot the
transmitted signals and the received signal for visual comparison.
Note that this is just one example of hypothesis testing in signal processing,
and there are various statistical tests and approaches that can be used
depending on the specific hypothesis and requirements.
10. Linear Algebra: Linear algebra plays a crucial role in signal

processing, particularly in areas such as signal representation,
transformation, and solving linear equations.
x Math Equation: Linear equations, matrix operations, eigenvalue

decomposition, etc.
x MATLAB Code: Dependent on the specific linear algebra

operation being performed.
Here are two examples that demonstrate the application of linear algebra in
signal processing:
218 Chapter VI
Examples: Eigenvalue decomposition:
Math Equation: Eigenvalue decomposition is a crucial operation in signal

processing that allows us to decompose a matrix into its eigenvalues and
eigenvectors. It is used in various applications, such as modal analysis,
system stability analysis, and dimensionality reduction.
MATLAB Code:
% Perform eigenvalue decomposition

A = [3 1; 1 2]; % Matrix A
[V, D] = eig(A); % Perform eigenvalue decomposition
% Display the eigenvalues and eigenvectors

disp('Eigenvalues:');
disp(diag(D));
disp('Eigenvectors:');
disp(V);
In this example, we have a matrix A. We use the eig function in MATLAB

to perform eigenvalue decomposition, which returns the eigenvectors in
matrix V and the eigenvalues in matrix D. The eigenvalues represent the
scaling factors, and the eigenvectors represent the directions associated with
those scaling factors.
The resulting eigenvalues and eigenvectors are then displayed.
This example demonstrates how eigenvalue decomposition can be applied

in signal processing to analyze the properties of a matrix and extract
valuable information about its behavior.
Please note that matrix operations in linear algebra can be complex and
depend on the specific problem at hand. The provided example is a
simplified illustration, and more advanced applications may involve larger
matrices and more intricate calculations.
Examples: Singular value decomposition (SVD),
Here is an example involving Singular Value Decomposition (SVD),

another important matrix operation in signal processing
Appendices 219
Math Equation: SVD is a powerful technique that decomposes a matrix into

three components: U, Ȉ, and V. It is commonly used for data compression,
noise reduction, and matrix approximation.
Given a matrix A, SVD can be represented by the equation A = UȈV^T,

where U is an orthogonal matrix, Ȉ is a diagonal matrix of singular values,
and V is another orthogonal matrix.
MATLAB Code:
% Perform Singular Value Decomposition (SVD)

A = [4 11 14; 8 7 -2]; % Matrix A
[U, S, V] = svd(A); % Perform SVD
% Display the singular values and matrices U, S, V

disp('Singular Values:');
disp(diag(S));
disp('Matrix U:');
disp(U);
disp('Matrix V:');
disp(V);
In this example, we have a matrix A. We use the svd function in MATLAB

to perform Singular Value Decomposition, which returns the matrices U, S,
and V. The diagonal elements of matrix S represent the singular values of
A.
The resulting singular values and matrices U, S, V are then displayed.
SVD is widely used in various signal processing applications, such as image

compression, noise reduction, and data analysis. It allows for dimensionality
reduction and the extraction of dominant features from the original matrix.
Please note that SVD can be applied to matrices of any size, and the
resulting U, S, and V matrices may have different dimensions depending on
the original matrix.
220 Chapter VI
In summary, the appendices section of this chapter introduces the

mathematical and computational tools used in signal processing. These tools
provide the necessary foundation and software implementations to
effectively analyze and process signals. By leveraging these tools, signal
processing practitioners can gain insights from signals and perform various
operations to extract meaningful information from the data they contain.

Attaphongse Taparugssanagorn - Classical Signal Processing and Non-Classical Signal Processing - The Rhythm of Signals-Cambridge Scholars Publishing (2023)

Uploaded by

Copyright:

Available Formats

You might also like

Attaphongse Taparugssanagorn - Classical Signal Processing and Non-Classical Signal Processing - The Rhythm of Signals-Cambridge Scholars Publishing (2023)

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Attaphongse Taparugssanagorn - Classical Signal Processing and Non-Classical Signal Processing - The Rhythm of Signals-Cambridge Scholars Publishing (2023)

Uploaded by

Copyright:

Available Formats

Classical Signal

The Rhythm of Signals

The Rhythm of Signals

This book first published 2023

Cambridge Scholars Publishing

Lady Stephenson Library, Newcastle upon Tyne, NE6 2PA, UK

British Library Cataloguing in Publication Data

Copyright © 2023 by Attaphongse Taparugssanagorn

ISBN (10): 1-5275-2864-2

1. Background and motivation: This chapter introduces the concept

1. Basic signal concepts: This section covers the fundamental

band-stop filters. It also discusses convolution, which is an

Chapter III ................................................................................................ 82

1. Wavelet transforms and time-frequency analysis: This section

Chapter IV .............................................................................................. 177

1. Audio and speech processing: This section covers signal

Chapter V ............................................................................................... 205

1. Emerging signal processing techniques and applications: This

Chapter VI .............................................................................................. 210

Background and motivation

In “Classical Signal Processing and Non-Classical Signal Processing: The

Motivated by the desire to provide a comprehensive overview of classical

Ideal for students, researchers, and industry professionals, “Classical Signal

understanding of classical and non-classical signal processing techniques.

Overall, “Classical Signal Processing and Non-Classical Signal Processing:

Overview of signal processing

Signal processing is a field of study that focuses on the analysis, synthesis,

In order to carry out signal processing effectively, it is crucial to have a deep

By utilizing these various signal processing techniques, we can extract

Signal processing plays a crucial role in many fields, such as

There are various signal processing techniques available, ranging from

Rhyme summary and key takeaways:

The overview of signal processing is critical. As it highlights the different

In communication, signals are encoded and decoded to transmit messages

Signal processing is also used in entertainment to make music and movies

Making signals more exciting and the science behind it compelling.

Key takeaways from the introduction chapter are given as follows:

1. Signals are diverse and have significance in various fields such as

4. Newer techniques such as wavelets and machine learning are

Overall, the introduction chapter establishes the importance of signal

Signal processing is used in different areas. In communication, signals are

The book “Classical Signal Processing and Non-Classical Signal

CLASSICAL SIGNAL PROCESSING1

Classical signal processing is a branch of electrical engineering and applied

Classical signal processing consists of several fundamental techniques,

The applications of classical signal processing are widespread, and it has

1 Classical Signal Processing refers to the traditional methods of analyzing and

manipulating signals that are based on mathematical and engineering principles. It

signal processing techniques are used for feedback control, system

Basic signal concepts

Fourier analysis and signal spectra

Fourier analysis is a mathematical technique used to represent a signal in

In summary, understanding the fundamental concepts of signals, such as

The Fourier transform, denoted by F(Ȧ), is used to represent a signal in the

where ݂ሺ‫ݐ‬ሻ is the original signal, Ȧ is the angular frequency, and ݆ =

Amplitude, frequency, and phase are three fundamental concepts of signals.

Overall, this example illustrates the connection between a sinusoidal signal,

PSD is a measure of the power of a signal at each frequency and is defined