PROJECT SEMINAR REPORT

On

CONTACTLESS HEART RATE DETECTION

Submitted in partial fulfilment for the Award of Degree of

BACHELOR OF ENGINEERING

IN

INFORMATION TECHNOLOGY

BY

Sai Jayanth Kumar (160119737167)

Sai Krishna (160119737168)

Sai Srinivas (160119737169)

Under the guidance of

Ms K. SWATHI

DEPARTMENT OF INFORMATION TECHNOLOGY

CHAITANYA BHARATHI INSTITUTE OF TECHNOLOGY (A)

(Affiliated to Osmania University: Accredited by NBA (AICE) and NAAV (UGC), ISO

Certified 9001:2015), Kokapet (V), GANDIPET (M), HYDERABAD - 500075

Website: www.cbit.ac.in

2022-2023

1
 CERTIFICATE

This is to certify that the project seminar report entitled “Contactless Heart Rate Detection”
submitted to Chaitanya Bharathi Institute of Technology, in partial fulfilment of the
requirements for the award of degree of B.E. (Information Technology) during the academic
year 2022-23 is a record of original work carried out by Sai Jayanth(160119737167),Sai
Krishna(160119737168) and Sai Srinivas(160119737169) during the period of study in the
Department of Information Technology, CBIT, Hyderabad.

Project Guide Head of the Department

Ms K Swathi Dr. K. Radhika

Assistant Professor Professor,IT Dept

CBIT, Hyderabad CBIT, Hyderabad

2
 ACKNOWLEDGEMENT

We would like to express our heartfelt gratitude to Ms. K Swathi, our project guide, for his
invaluable guidance and constant support, along with her capable instruction and persistent
encouragement. We are grateful to our Head of Department, Dr .K. Radhika, for her steady
support and for the provision of every resource required for the completion of this project.
We would like to take this opportunity to thank our Principal, Dr. P .RAVINDER REDDY, as
well as the management of the institute, for having designed an excellent learning
atmosphere. Our thanks to all members of the staff and our lab assistants for providing me
with the help required to carry out the groundwork of this project

3
 ABSTRACT

CONTACTLESS HEART RATE DETECTION

Heart Rate (HR) is one of the most important Physiological parameters and a vital indicator
of people’s physiological state and is therefore important to monitor. Monitoring of HR often
involves high costs and complex application of sensors and sensor systems. Research
progressing during the last decade focuses more on non-contact based systems which are
simple, low-cost, and comfortable to use. Still, most of the non-contact based systems are fit
for lab environments in offline situations but need to progress considerably before they can
be applied in real time applications. This method presents a real time HR monitoring method
using a webcam of a laptop computer. The heart rate is obtained through facial skin colour
variation caused by blood circulation. Three different signal processing methods such as Fast
Fourier Transform (FFT), Independent Component Analysis (ICA) and Principal Component
Analysis (PCA) have been applied on the colour channels in video recordings and the blood
volume pulse (BVP) is extracted from the facial regions. HR is subsequently quantified and
compared to corresponding reference measurements.

4
 TABLE OF CONTENTS

Title Page No

Abstract 4

List of Figures 6

1. INTRODUCTION

1.1. Overview 7

1.2. Applications 7

1.3. Problem Definition 7

1.4 Aim of Project 8

2. LITERATURE SURVEY

2.1. Efficient Real-Time Camera Based Estimation 9

2.2. rPPG-Based Heart Rate Estimation 14

2.3. Heart Rate Estimation from Facial Photoplethysmography 18

3. SYSTEM REQUIREMENT SPECIFICATION

3.1. Hardware Requirements 21

3.2. Software Requirements 21

4. METHODOLOGY/PROPOSED SYSTEM

4.1. System Architecture 22

4.2. Methods 24

5. CONCLUSION 26

5
 LIST OF FIGURES

Name of Figure Page Numbers

1. FIGURE 1. RPPG setup 11

2. FIGURE 2. Inter Beat Levels(IBL) 12

3. FIGURE 3.Flowchart of proposed architecture 15

4. FIGURE 4.Temporal Mask Branch 17

5. FIGURE 5. Example of ROI of Participant face 19

6. FIGURE 6. Flowchart Of Proposed Algorithm 22

6
1. INTRODUCTION

1.1.Overview

Remote measurement of human vital signs has a promising potential for clinical and
non-clinical scenarios. It provides a comfortable way for determining subjects’ physiological
parameters for diagnosis and regular checking. Quite recently, considerable attention has
been paid to remote Photoplethysmography (rPPG), which is a camera-based contactless
acquisition method of the human cardiac pulse. A remarkable feature of this emerging
method is the ability to estimate the vital signs without any physical contact with the subject,
which eliminates any inconvenience during the assessment process. This striking feature
opens the door for employing rPPG technology in non-clinical applications, such as home
health monitoring, driver monitoring, fitness-cardio training, video surveillance, and face
anti-spoofing. Traditional contact methods for measuring HR comprise Electrocardiography
(ECG) and Pulse Oximetry.Lately, there have been numerous studies investigating contactless
HR measuring methods; including methods based on Doppler effect, thermal imaging, and
piezoelectric measurements . These techniques have been examined in literature, but with
limited applications due to size, cost and complexity. Most recent studies as well as current
work focus on camera based HR monitoring methods. These methods are widely considered
to be very efficient techniques for measuring the HR remotely, and many researchers have
proven its feasibility.

1.2. Applications

❖ In the fitness industry accurate heart rate measurement helps in controlling the
training load.
❖ In the healthcare industry, it can help in detection of a possible cardiac arrest.
❖ remote medical diagnosis and evaluations in telemedicine.

1.3. Problem Definition

To discuss rPPG measurement using signal processing methods as well as its recent
furtherance in the deep learning environment. To harness the insight into the challenges on
rPPG, and we anticipate some suggestions on the future direction.

7
1.4 Aim of Project

The main aim is to detect the heart rate of a person using live feed data of facial videos using
signal processing methods.As,this is a non-invasive method it reduces the discomfort while
calculating heart rate.

8
2. LITERATURE REVIEW

2.1. Paper 1:Efficient Real-Time Camera Based Estimation of Heart Rate and Its Variability

A method known as photo-plethysmography can be used to measure human vital signs such
as heart rate, blood oxygen saturation, and associated physiological measurements (PPG).
This method includes optically observing variations in blood volume-related light absorption
in tissues. Usually, a touch sensor fastened to the skin's surface is used. By monitoring
variations in skin reflectance as seen by a camera, remote photoplethysmography (rPPG)
detects the blood volume pulse. We provide a unique system in this research for obtaining
face-based heart rate (HR) and heart rate variability (HRV) measurements.The process of
rPPG essentially involves two steps: detecting and tracking the skin colour changes of the
subject, and analysing this signal to compute measures like heart rate, heart rate variability
and respiration rate. Recent advances in computer video, signal processing, and machine
learning have improved the performances of rPPG techniques significantly. Current
state-of-the-art methods are able to leverage image processing by deep neural networks to
robustly select skin pixels within an image and perform HR estimation. However, this
reliance upon heavy machine learning (ML) processes has two primary drawbacks: (i) it
necessitates rPPG specific training of the ML model, thereby requiring collection of large
training sets; (ii) complex models can require significant computation time on CPUs and thus
can potentially add a bottleneck in the pipeline and limit real-time utility. Since rPPG analysis
is originally a signal processing task, the use of an end-to-end trainable system with no
domain knowledge leaves room for improvement in efficiency (e.g., we know that pulse
signal is embedded in average skin colour changes , but the ML system has to learn this). We
introduce a simplified and efficient rPPG pipeline that performs the full rPPG analysis in
real-time. This method achieves state-of-the-art results without needing any rPPG related
training. This is achieved via extracting regions of interest robustly by 3D face modelling,
and explicitly reducing the influence of head movement to filter the signal.

9
METHODOLOGY

Skin pixel selection

Fitting an active appearance model (AAM) and face detection are two of the pipeline's initial
steps. Then, the head orientation and face landmarks are determined using this AAM. A
region of interest (RoI) that only contains skin-colored pixels on the face is created using the
landmarks. This enables us to continuously follow the pixels in this RoI throughout the entire
video. Our field of interest is the top part of the face, excluding the eyes. The pitch, roll, and
yaw angles of the head each frame is measured and tracked using the head orientation.

Signal extraction

The running window for tracking the colour signals and head orientation angles is 8.53
seconds. This window's duration translates to 256 frames at 30 frames per second or 512
frames at 60 frames per second. To account for changes in the input's frame rate, all signals
are resampled using linear interpolation. Depending on which is closer to the frame rate of
the original video, they are resampled to either 30 or 60 fps. The POS approach is then used
to combine the three colour signals from the R, G, and B channels into a single rPPG signal.
The R, G, and B signals are projected on a plane orthogonal to an empirically determined
normalised skin tone vector in the POS approach to remove intensity variations. A 1-D signal
with one dimension is created from the resulting 2-D signal. The dimensions of the incoming
signal are weighted by an alpha the ratio of the standard deviations of a parameter every
signal By doing this, the amount of pulsing component in the resulting rPPG signal is
maximised.

10
 Figure-1:- The light scattering involved in the rPPG method.

Signal filtering

Rhythmic motion noise suppression A copy of the extracted rPPG signal as well as the
head-orientation signals are converted to frequency domain using Fast Fourier Transform.
The three resulting head-orientation spectra (one each of pitch, roll, and yaw) are combined
into one via averaging. This is then subtracted from the raw rPPG spectrum after amplitude
normalisation. This way, the frequency components having a high value in the
head-orientation spectrum are attenuated in the rPPG spectrum. Subsequently, the frequencies
outside of the human heart rate range [(0.7-4 Hz )/(42-200 bpm)] are removed from the
spectra. Wide and narrow band filtering The highest frequency component inside the
resulting spectrum is then used to determine the passband range of a narrow-bandpass filter
with a bandwidth of 0.47 Hz. Such a filter can either be realised via inverse FFT or a high
order FIR filter (e.g. ~50th order Butterworth). The selected filter is then applied to the
original extracted rPPG signal to produce noise-free BVP.

Post processing

To prevent minor shifts in the locations of the crest of each beat over multiple overlapping
running windows,

11
Figure 2: Even when the heart rate is almost constant, the underlying interbeat intervals can
have many fluctuations. This is detected by rising squared successive differences , a measure
of heart rate variability.

The signals from each window overlap with earlier signals. First, the filtered rPPG signal is
normalised by subtracting its mean and dividing it by its standard deviation. During
resampling of the signal, the number of samples to shift is determined based on the source
and resampled frame rates. The signal is then shifted back in time accordingly and added to
the previous/already overlapped signals. Older values are divided by the times they have been
added, to ensure all temporal locations lie in the same amplitude range. Over time, a cleaner
rPPG signal is obtained from this.

Output calculation

Once a clean rPPG signal is obtained, we can perform peak detection on it to locate the
individual beats in time in the signal. From the located beats, heart rate and heart rate
variability can be calculated. To do this, we first extract the inter-beat-intervals (IBIs) from
the signal, which are the time intervals between consecutive beats.

12
Heart rate calculation

Heart rate is calculated by averaging all IBIs over a time window, and computing the inverse
of it. That is, HRw = 1/IBIw, where IBIw is the mean of all inter-beat intervals that fall
within the time window w. The choice of this time window can be based on the user’s
requirement (e.g. instantaneous HR, long-term HR).

Heart rate variability calculation

Multiple metrics can be computed to express the measure of heart rate variability in different
units. In this work, we focus on one of the most popular time-domain metrics for
summarising HRV called the root mean square of successive differences (RMSSD),
expressed in units of time. As the name suggests, this is computed by calculating the root
mean square of time difference between adjacent IBIs:

In addition, we also compute two frequency-domain metrics of HRV, simply known as

Low-frequency (LF) and High-frequency (HF) band (as well as a ratio of them), that are
commonly used in rPPG HRV literature . The LF and HF components are calculated using
Welch’s power spectral density estimation. Since Welch’s method expects evenly sampled
data, the IBIs are interpolated at a frequency of 2.5Hz and zero padded to the nearest power
of two. The power of each band is calculated as total power in a region of the periodogram:
the LF band from, and the HF band from . Details about these metrics can be found . Both
metrics are converted to normalised units by dividing them by the sum of LF and HF.

13
2.2 Paper : rPPG-Based Heart Rate Estimation using Spatial-Temporal Attention Network.

Heart rate (HR) is an important indicator of physiological conditions, which can be measured
with contact devices, such as electrocardiogram (ECG) and Photoplethysmography (PPG).
The ECG extracts HR by sensing human cardiac current through electrodes. But, it is not
robust enough for movement and the application scenarios are limited . The PPG is a simple
and low-cost method for obtaining the HR, which uses optical techniques to detect blood
volume changes in capillaries . Some convenient PPG detection methods use a portable and
wearable device (e.g smart watch or bracelet) to acquire the PPG signal . However, these
PPG methods require contact with the human body, which may cause skin irritation or
discomfort to some patients who need to monitor their heart rate for a long period of time.
Human-computer interaction has some limitations (e.g., the user needs to wear a
measurement device for the computer to receive the heart rate information).Most of the rPPG
techniques can be divided into several stages. First, the region of interest (ROI) is
automatically extracted using face detection and tracking algorithms. Some methods further
subdivide the ROI by extracting, for example, the forehead or the cheeks, since the
physiological information in these areas tends to be stronger, and the interference from eye
blinks can be removed . Then, the rPPG signal is extracted in the ROI using different
algorithms. The roughly extracted rPPG signal may contain noise, and it would be necessary
to apply some signal processing methods to refine the signal. Finally, discrete fourier
transform (DFT) or peak detection algorithm can be used to compute the HR. To summarise,
most existing models learn the mapping relationship between facial video information and the
groundtruth HR. However, there are still three major problems. First, the slow change of
video facial information causes redundant information. Existing networks rarely consider
reducing unnecessary redundant spatial information. Second, the head movement and
environmental illumination variation bring noise, which affects the model’ s accuracy
towards rPPG signals extraction. Third, the above methods need to go through multiple
stages, leading to tuning and implementation constraints.

Methodology

Preprocessing Techniques

Non-end-to-end deep learning can typically leverage prior knowledge to assist the networks
in converging more quickly and accurately. By preprocessing the data, the networks can focus

14
more easily on the information we need. In this section, we utilise the plane orthogonal to
skin (POS) algorithm with motion representation normalisation technique to process the
input. The procedure of preprocessing is similar to McDuff et al.By using a five band camera
to augment input information. We use multiple ways to process the inputs and conjoin them
to achieve the same purpose. First of all, to make the output signal independent of the colour
and intensity of the light source, we normalise the input image sequence over a time interval :
xnt = xt µ (xt) , where t is the index of the input image sequence. xt and xnt are the original
video frame and the normalised video frame, respectively. µ (xt) is the average value of the
frames in the vicinity of the current image over a time interval. Next, the image is projected
from the RGB space to the plane perpendicular to h using the projection matrix, xpt =Px*T
nt = Gnt−Bnt−2*Rnt+Gnt+Bnt where xpt is the projected video frame and xnt is the temporal
normalised image. Rnt, Gnt, Bnt denote the three channel maps of the temporal normalised
image, respectively. By projecting the RGB channels, we bring the image into the plane most
likely to contain the pulse signal. Besides the colour information that can be used, it is also
possible to add frame-to-frame variation information to the input, which would allow the
model to learn and filter out the noisy information due to the motion. A common method for
extracting inter-frame differences is the optical flow technique. But this technique requires
constant target brightness, which clearly contradicts the theory of rPPG and is therefore not
applicable to our task. Hence, we use the motion representation proposed by Chen et al. to
represent the difference between frames: xmt = xt+1 − xt xt+1 + xt .

Figure3: Flowchart of proposed network architecture,hyper-parameter N=3,Tframes

represents the time window .

15
Network Architecture

The backbone of our network architecture is the residual attention network. We develop our
network by stacking the attention module , the temporal fusion module and the residual block
on top of this backbone. Besides, we add a convolutional block attention module (CBAM) to
the residual block to enhance the selection of the channel and spatial features. The residual
block identified by the grey square . 3 consists of three convolutional layers with a shortcut
from the input to the output.

Temporal Fusion Module (TFM): The rPPG signal contains a time dimension. Therefore, the
time-domain information becomes essential in our task. Usually, LSTM or ConvLSTM can
be used to learn the time-domain correlations. In this paper, we utilise the simpler and more
efficient 3D convolution supplemented by a mask branch to exploit the temporal
information.The proposed TFM is composed of the aggregation branch and the temporal
mask branch. Although the temporal information is highly important, there are still many
redundant frames in the video frames.

Loss Function:

The PPG signal is used as the ground truth. We optimize the accuracy of HR and the signal
quality of rPPG by two loss functions, respectively. For a clean rPPG signal, the frequency
corresponding to its heart rate should be much higher than the noise frequency. So we use the
Signal-to-Noise Ratio (SNR) loss function to optimize the accuracy of HR. The SNR can be
calculated by power spectral density (PSD). The PSD of the rPPG signal extracted by our
proposed network is calculated first by the following equation: PSD( ˆf, X) = TX−1 n=0 f
(xn) · cos 2π ˆf n fs !2 + TX−1 n=0 f (xn) · sin 2π ˆf n fs !2 , where f (xn) is the output of the
n-th facial image. ˆf represent the given frequency. In the experiments, ˆf is considered to be
0.67 to 4 Hz, which is the normal human heart rate range. fs is the input video’s frame rate,
which is 25 to 30 fps in the experiments. X = (x1, x2, ..., xT ) is the input of facial images. It
is worth noting that it is not only involved in the calculation of the loss function, but is also
an important step in our calculation of heart rate. After the PSD is calculated, the SNR can be
calculated by comparing the sum of the power spectral intensities near the ground truth HR
value with the sum of other power intensities. Specially, the SNR is calculated as follow:
SNR(fT, X) = 10 · log10( X fˆ∈F+ PSD( ˆf, X)/ X fˆ∈F\F+ PSD( ˆf, X)), where fT is the
frequency of the ground truth HR. F is the collection of all HR values, which also takes

16
values from 0.67 to 4 Hz. F + = (fT − ∆, fT + ∆) is the interval around the true HR frequency,
and ∆ represents the region around the ground truth HR. F\F+ denotes all other frequencies
other than the true HR’s. In our experiments, we set ∆ to 0 and optimize the PSD directly by
cross-entropy loss: LossSNR (fT , X) = − log exp(PSD(fT , X)) P fˆ exp(PSD( ˆf, X))!.

Figure-4

While the rPPG has the convenience to measure HR remotely from facial videos, head
movements, illumination variations and environment conditions, may have great impact on
rPPGbased approaches. In this paper, we propose an effective spatial-temporal attention
network which can effectively address the above problems by adopting an attention module
and temporal fusion module. To address the difficulty of ROI selection, we design an
attention module to automatically locate regions with strong physiological amplitude. In
addition, we propose a temporal fusion module, which can fully utilise the temporal
information, suppress the impact of noise, and improve the computational efficiency. In
cooperation with different loss functions, the network can take full advantage of spatial
context and temporal context information. Moreover, we use the POS algorithm and motion
representation normalisation technique to process the input and incorporate the neural
network to fully exploit the advantages of different algorithms. Through ablation experiments
on the PURE and MMSE-HR dataset, we validate the effectiveness of the proposed method.
Finally, we compare the experimental results on the public PURE, UBFC-rPPG and the
MMSE-HR dataset with the existing methods. The results show the superiority of our
method. In future research, we will mainly focus on investigating the effectiveness of the
proposed approach for the other rPPG-based physiological status measurement tasks, such as
the breath rate and blood pressure measurement from videos.

17
2.3 Paper 3:Heart Rate Estimation from Facial Photoplethysmography during Dynamic
Illuminance Changes

Camera-based remote photoplethysmography (rPPG) has the advantage to noninvasively

monitor cardiovascular activity in a daily environment. As the widespread use of
camera-equipped devices to the daily living environment such as smart TV, dash cam, and
security cam, rPPG has more chances to be applied in practical use. Previous studies have
evaluated the feasibility of rPPG in various practical situations. Zhang et al. proposed a rPPG
system to monitor heart rate during car driving . Poh at al. proposed a method to extract rPPG
using a webcam built-in a laptop . Applying rPPG to practical use has some limitations such
as illuminance changes. In daily situations, the environmental illuminance is continuously
changed and the dynamic illuminance changes can cause the artefacts on a signal extracted
using camera-based rPPG. While previous works have studied the illuminance variation in
rPPG, there were few limitations to be applied to practical situations. Jeanne at al. proposed a
camera-based heart rate monitoring system in a dynamic light environment .In this study, we
propose an artefact reduction method in rPPG, which is caused by the variation of the
illuminance. During watching a video in a dark room, for example, watching a TV at night
without illuminance, there is a high correlation between the brightness changes of a video and
the illuminance variation on the skin of the viewer’s face. We assumed that the colour values
from facial video contain both cardiac pulses and the variation of the illuminance. The
proposed method subtracts the artefacts from the raw facial rPPG signal by applying
multi-order curve fitting between the illuminance information from the facial rPPG signal and
the brightness information from a video.

Methodology

Experiment setup

Ten healthy participants (8 males, 2 females) were recruited in the experiment. The
participants were asked to sit 2m in front of the 42-inches monitor (LG 42LC3DQ ) in a dark
room. Then, the subjects were asked to watch movie clips for 5 minutes. The camera (Canon
6D) with fixed focal length lens (Canon EF 50mm F1.4 USM ) was installed 1m in front of

18
the participants and recorded the facial video. The videos were recorded at the frequency of
59.94Hz with HD resolution (1280x720 pixels). We disabled the embedded automatic white
balance function in the camera and focused the lens on the participant’s cheek manually. The
reference photoplethysmography (PPG) was synchronously acquired from the index fingertip
using a FDA-approved PPG measurement system (BIOPAC MP150) to measure the reference
heart rate.

Extraction of a raw trace of rPPG and movie brightness

We extracted the signals for the raw trace rPPG and movie brightness variation using
MATLAB R2014b. The overview of the method we proposed. We set a region of interest
(ROI) in a facial video. Then, we extracted the raw trace rPPG signal from the ROI using the
method used in our previous work .We conducted the correlation analysis between the raw
trace of rPPG signal from a facial video and the RGB channels of a movie. To extract the
brightness variation signal, we chose the green channel of the movie, which showed the
highest correlation with the raw trace of rPPG. The brightness variation signal was extracted
from a movie using the equation as follows: 𝐼(𝑡) = ∑ ∑ 𝐶𝑣(𝑥,𝑦,𝑡) ℎ 𝑦=1 𝑤 𝑥=1 𝑤×ℎ where 𝐼(𝑡)
is illuminance at time t, and 𝐶𝑣(𝑥, 𝑦,𝑡) is the green channel value of a pixel. 𝑥, 𝑦 are spatial
coordinates of image frame of the movie and 𝑤, ℎ are width and height of the frame.

Figure-5 The image shows the ROI to signal extraction flow.

Movie clip Two movie clips whose dynamics of illuminance variation are low and high were
used in the experiment. We defined the dynamics of the illuminance variation as the ratio
between high frequency band and cardiac pulse frequency band power as follows: 𝑟 = ∑ 𝑆̂(𝑓)
30 𝑓=2 ∑ 𝑆̂(𝑓) 2 𝑓=1 where 𝑆̂(𝑓) is the power spectral density of the signal. We selected two
movie clips in which r were 0.44 (low-dynamic movie) and 1.83 (high-dynamic movie),
respectively. The frames per second (fps) of the movies was the same as the recorded facial
videos.

19
Artifact subtraction

We assumed that the raw trace rPPG signal from the facial video contains both cardiac pulse
and illuminance change. Our method cancelled the illuminance change from the raw trace
rPPG signal using the extracted brightness variation signal from the movie. We used the least
square curve fitting method to precisely estimate the brightness variation signal in the raw
trace rPPG signal. The curve showed non-linearity and we tried several fitting models to
minimise the fitting error, and the 9th order polynomial model was used because the
goodness-of-fit of it was the best. The estimated brightness signal was subtracted from the
raw trace rPPG signal for artefact cancellation. The window size of fitting was determined as
2 seconds empirical.

We have presented a new method for cancelling the light artifacts during dynamic
illuminance changes in measuring camera-based remote PPG.We applied the curve fitting
model with the 9th order polynomial in this paper, but in some cases the lower order
polynomial model showed better performance. Also, some other parameters can affect the
performance, such as the window size of curve fitting. Future work will investigate the
influences of these parameters.

20
3.SYSTEM REQUIREMENTS SPECIFICATIONS

3.1. Hardware Requirements

● Processor: intel®Core(TM) i5-8250U CPU @ 1.60GHz 1.80GHz

● RAM: 8.00 GB
● Hard-disk: 32 GB

3.2. Software Requirements

● Operating system : windows 10 or higher

● Programming language : python
● Python libraries (tkinter,etc..)

21
4. METHODOLOGY/PROPOSED SYSTEM

4.1.System Architecture

Figure-6:Flowchart of proposed algorithm using adaptive ROI’s.

DETRENDING

Due to the noises caused by changing of the environmental parameters, in this step, we
remove the linear trends from the raw signal. The effect of the detrending process. A
stationary time series oscillates about the horizontal line. If a series does not have a trend or
we remove the trend successfully, the series is said to be trend stationary.

22
NORMALISATION

We are only interested in the periodicity of the signal. Therefore, the raw signal is normalised
by dividing the raw signal.Normalization is the process of organising the data in the
database.Normalisation is used to minimise the redundancy from a relation or set of relations.

SMOOTHING
In this step, the normalised RGB signals are smoothed using a sliding average filter. This step
leaves out noise, and it clarifies the signal. Therefore, it increases the robustness and
flexibility of the analyses.

Smoothing may be distinguished from the related and partially overlapping concept of curve
fitting in the following ways:

● curve fitting often involves the use of an explicit function form for the result,
whereas the immediate results from smoothing are the "smoothed" values with no
later use made of a functional form if there is one;
● The aim of smoothing is to give a general idea of relatively slow changes of value
with little attention paid to the close matching of data values, while curve fitting
concentrates on achieving as close a match as possible.
● Smoothing methods often have an associated tuning parameter which is used to
control the extent of smoothing. Curve fitting will adjust any number of
parameters of the function to obtain the 'best' fit.

BLIND SIGNAL SEPARATION

The RGB signals contain information about the HR in mixed components. Therefore, ICA is
used to recover the source signals from these mixed signals. In our system, the FastICA
method is used to analyse the RGB signals to reveal the original source signals. Moreover,
FastICA is an effective technique that can be utilised to eliminate noise artifacts. The
extracted raw signal is composed of three single series, which were obtained from red, green,
and blue channels. The goal of BSS is to recover the one-dimensional plethysmographic
signal from the three raw signals. BSS is implemented to FIGURE 9. ICA components in the
time domain. FIGURE 10. Filtered RGB signals in the time domain. compute the optimal
combination of the three raw signals. We use the BSS algorithm [56] to get three independent

23
source components from the raw signals. The three ICA components are compared, and the
component with the highest periodicity is chosen for the next step. BSS is in general highly
undetermined. Many applications involve a single-channel source separation problem.
Among different realisations of a mixing system, it is crucial to deal with single-channel
source separation because a wide range of applications involve only a single recording
channel but mix or convolve with various sources or interferences. Fig. 1.3 demonstrates a
scenario of monaural source separation using a single microphone with three sources. We aim
to suppress the ambient noises, including birds and aeroplanes, and identify the human voices
for listening or understanding. Therefore, single-channel source separation can be generally
treated as a venue to speech enhancement or noise reduction in a way that we want to
enhance or purify the speech signal in the presence of surrounding noises.

FILTERING

The goal of this step is to increase the quality of the acquired plethysmographic signal by
removing undesirable noise. Driving conditions include: non-stationary subject and varying
illumination, these factors degrade SNR ratio. In this step, the raw signal is applied to a
band-pass-filter with ideal behaviour to eliminate high and low frequency noise. The filter
removes components which exist outside the 0.7 Hz to 4 Hz frequency band. This band was
commonly used in previous studies, and it corresponds to heart rate measurements between
42 and 240 bpm.

4.2.Method

REMOTE PHOTOPLETHYSMOGRAPHY (rPPG)

ALGORITHMIC BASICS OF rPPG In previous studies, many approaches have been

adopted in rPPG algorithms. These are:

i.Blind Source Separation (BSS)-based method: This approach can be divided into
ICA-based methods , and PCA-based methods. In this approach, the BVP signal is recovered
by splitting the mixed RGB signals into independent source signals.

ii.Chrominance-based method (CHROME): In this method, a linear addition of the

chrominance-signals is performed, and a standard skin tone is used to white balance the
images.

24
iii. BVP signature based method:This method is robust to motion artifacts. It separates
colour variations, due to heart rate, from the noise produced by motion artifacts. This
separation is done by utilising the blood volume pulse signature in different optical
wavelengths . iv. Spatial Subspace Rotation (2SR):This is a novel method introduced in . In
this technique, the BVP signal is recovered by measuring spatial subspace timerotation of the
skin pixels. The main difference between each of these rPPG approaches is how red, green
and blue signals are combined to extract the BVP signal. Our rPPG system is based on Blind
Source Separation (BSS) method.We use the rPPG mathematical model defined in to
understand the basic principles of rPPG algorithms. This mathematical model enables us to
define and analyse different problems and limitations regarding rPPG in detail.

25
5.CONCLUSION:

The heart rate detection using this method helps the doctors and patients a step closure to
monitoring the health condition during major contagious diseases like covid.The system also
help in analysing the person’s heart rate with the video data.As a result this project and the
proposed methodology minimise the effort involving in the usage of manual devices involves
in attaching different sensors on patients.

26
 References

Premkumar, S.; Hemanth, D.J. Intelligent Remote Photoplethysmography-Based Methods for

Heart Rate Estimation from Face Videos: A Survey. Informatics 2022, 9, 57. https://
doi.org/10.3390/informatics9030057

Lee, H.; Lee, J.; Kwon, Y.; Kwon, J.; Park, S.; Sohn, R.; Park, C. Multitask Siamese Network
for Remote Photoplethysmography and Respiration Estimation. Sensors 2022, 22, 5101.
https://doi.org/10.3390/ s22145101

Haugg, F.; Elgendi, M.; Menon, C. Effectiveness of Remote PPG Construction Methods: A
Preliminary Analysis. Bioengineering 2022, 9, 485. https://doi.org/
10.3390/bioengineering9100485

Lee, H.; Cho, A.; Whang, M. Fusion Method to Estimate Heart Rate from Facial Videos
Based on RPPG and RBCG. Sensors 2021, 21, 6764. https://doi.org/10.3390/ s21206764

27