2021 International Conference on Decision Aid Sciences and Application (DASA)
Detected pulses from the motion of the head to
estimate heartbeat rate
Nguyen Thi Ngoc Trinh
Faculty of Electrical and Electronics Engineering
Ho Chi Minh City University of Technology, VNU – HCM
Vietnam National University Ho Chi Minh City
Ho Chi Minh, Vietnam
trinh.nguyen.020923@hcmut.edu.vn
Nguyen Thanh Binh
Faculty of Computer Science and Engineering
Ho Chi Minh City University of Technology, VNU - HCM
Vietnam National University Ho Chi Minh City
Ho Chi Minh City, Vietnam
ntbinh@hcmut.edu.vn
Abstract—The beating of the heart sends blood throughout
the body to feed the cells and puts pressure on the blood vessel
walls. Therefore, monitoring this process is extremely important
in the diagnosis and treatment of the disease. In the past, people
did this by counting pulses of artery or wearing of heart monitor
on the body. In both of these cases, physical contact with the
body is required to get the heart rate. Today, with the
advancement of science and technology, we can overcome these
limitations based on computer vision techniques. This paper
proposes one such method - measuring heartbeat rate through
frequent pulses from the motion of blood vessel on the head. We
extract them from a regular computer's camera in real-time.
Our method identifies and records the oscillation of the feature
points. Then, using the algorithm principal component analysis
decays their trajectories into a group of component motions.
Thus, we can determine the best component corresponding to
heartbeats based on its frequency. In order to do that, we built
an electrocardiogram and analyzed it to identify the peaks of the
trajectory, which corresponded to the heartbeats. Based on that
data, we calculated the subject's heartbeat rate. When evaluated
on 10 subjects, our approach reports heartbeat rate roughly the
same as a conventional sphygmomanometer. This result shows
that the proposed method is correct, robust, and economically
suitable. However, in order to achieve high technical efficiency
without changing the economy, this method needs to be further
improved in the future.
Keywords— Electrocardiogram, Heartbeat rate, Head motion
detection, Trajectory, Feature point tracker, Principle component
analysis, Discrete cosine transforms, Photoplethysmograph.
I. INTRODUCTION
Heartbeat rate is obviously a vital sign of the human body's
activity and its measurement is of great importance in many
applications, such as fitness assessment, training programs,
and especially in medical diagnosis. For example, in the
medical diagnosis of an operation, heartbeat rate, along with
other parameters, is used as a crucial sign that helps to assess
the condition of the cardiovascular system. In this example, it
can be used also for ensuring the safety of the process. If
heartbeat rate goes beyond the normal range, it is no longer
safe to continue surgery without further intervention. Not only
that, but heartbeat rate also has a particularly important
meaning in health monitoring, especially for premature
neonates and the elderly. Because they have delicate skin and
are vulnerable to the traditional sensors, a non-invasive
heartbeat rate extraction method is more suitable.
Furthermore, as the population ages, continuous or at least
frequent monitoring outside the clinic can provide doctors
XXX-X-XXXX-XXXX-X/XX/$XX.00 ©20XX IEEE
978-1-6654-1634-4/21/$31.00 ©2021 IEEE 892
Do Tran Hung
Faculty of Electrical and Electronics Engineering
Ho Chi Minh City University of Technology, VNU – HCM
Vietnam National University Ho Chi Minh City
Ho Chi Minh, Vietnam
hung.do.tran2302@hcmut.edu.vn
Dang Thanh Tin
Faculty of Electrical and Electronics Engineering
Ho Chi Minh City University of Technology, VNU – HCM
Vietnam National University Ho Chi Minh City
Ho Chi Minh City, Vietnam
dttin@hcmut.edu.vn
with not only timely samples but also long-term trends and
statistical analyses for a more comprehensive assessment of
the treatment process. The non-invasive measurement method
is highly applicable because it is non-invasive and non-
obtrusive.
These contactless methods are largely based on computer
vision techniques or algorithms using artificial intelligence.
Among them, one type can be mentioned is
photoplethysmography (PPG). Its basic principle is to use red
or infrared light to shine on the body, where there are dense
vascular areas such as the face, hands, etc. Then, the system
will pick up variations in reflected light and analyzed for heart
rate parameters. Overall, it meets the requirements of a non-
contact heart rate measurement system. However, it requires
the use of expensive specialized light sources to improve
accuracy. At the same time, PPG is very sensitive to
movement. To overcome the above disadvantages, another
method is proposed based on the hypothesis of the periodicity
of the blood circulation system. Each time blood is pumped
into the arteries to the organs, a subtle change is caused. It
could be a change in skin color or the location of blood
vessels. These changes are usually invisible to the naked eye,
but they can be viewed with techniques such as Eulerian video
magnification. The resulting periodic changes are then used to
measure cardiovascular indices.
In this paper, we use microscopic head vascular
oscillations related to the cardiac cycle as above. However, in
order to extract heart rate information at a low cost without
compromising accuracy, we use the camera of an ordinary
computer and calculate the measurement in real - time.. Our
algorithm detects and tracks the pulse from this movement.
From a biomechanical point of view, due to the spine as a
pillar, the vertical axis of the head is less likely to have large
amplitude movements that affect the oscillations related to the
heart rate. Meanwhile, the horizontal axis tends to capture
most of the swaying of the dynamic balance. Therefore, the
vertical direction is the best axis to measure the pulses needed
for heart rate analysis. However, besides blood circulation,
respiration is another source of involuntary head movements.
Thus, after monitoring the head oscillations, we proceed to
filter out the high frequencies due to the respiratory activity
and the subject's intentional movement. These pulses are then
passed through several other algorithms to increase accuracy.
In it, we used the Discrete Cosine Transform (DCT). About
DCT, we will detail it in the following section. After obtaining
an electrocardiogram (ECG), the heart rate is calculated.
 2021 International Conference on Decision Aid Sciences and Application (DASA)

Although it has not yet reached absolute accuracy, the
results achieved are very positive and there is a lot of potential
for development in the coming time. The method that we give
above in addition to the technical advantages mentioned
above, the cost of the system is also relatively cheaper than
other types available on the market.
The remainder of this paper is organized as follows:
Comparisons and comments of other related works are given
in the next section. Section 3 explains the used hardware
structure of the proposed system. Section 4 presents some of
the main algorithms of our method. Experimental results are
reported in Section 5. Finally, the conclusion of this paper is
presented in Section 6.
• Using an additional moving average filter before the
Butterworth filter that is employed in Balakrishnan
[2].
• Replacing the FFT of Balakrishnan's system with
DCT.
• Using webcam in laptop instead of high quality
camera.
To build our system, in addition, we also refer to a few
related pieces of research such as rapid object detection using
a boosted cascade of simple features by Viola [3], pyramidal
implementation of the Lucas Kanade feature tracker by
Bouguet [4],…

II. RELATED WORKS III. METHODOLOGY

Aforementioned the system we proposed in this paper
develops a visual-based contactless algorithm for heartbeat
rate measurement using the assumption that periodic blood
circulation by the heart to the head generates subtle periodic
motion on the face. To build this system, we refer to many
recent works:
Poh's research [1] used the periodic changes in skin color
of the face. This signal is separated into its color channels and
each channel is monitored independently. For each channel, a
trajectory is found. Then, all the trajectories are processed by
Independent Component Analysis (ICA) algorithm. The
output of the ICA presents independent sources that have
caused a change in skin color on the face. Poh's system
assumes that the most periodic output of the ICA should be
produced by the most circulating source present in the face,
which is the circulatory system of the blood. The system is
relatively efficient, but it is sensitive to skin tone and noise. It
means that if the skin is not detected properly or if the recorded
face video is noisy, the system will not provide accurate
results.
Based on the same assumption as that of Poh [1],
Balakrishnan [2] came up with a new idea to overcome the
sensitivity to noise and skin detection of the system. However,
instead of using signals from periodic changes in skin color,
Balakrishnan's system uses pulses from head movements.
Similar to Poh, this system also uses a simple camera to record
the subject's face. After detecting the face, they extracted the Fig. 1. The block diagram of our proposed system.
vertical component of the head movement by tracking the
feature points and generating several trajectories for each Our proposed system is presented in a simple way through
feature point. These trajectories are then processed by a the block diagram as shown above (Figure 1). As can be seen
Butterworth filter to remove extraneous frequencies. In from this image, first, we use the webcam of the Asus
contrast to Poh, they next used Principle Component Analysis GL552VX laptop with a resolution of 1280 x 720 pixels to get
(PCA) instead of ICA to analyze the filtered trajectories into a continuous data which is the subject's face in real time. The
dataset of source signals. To find the periodicity of the PCA's survey subjects are required to keep their head does not move.
outputs, they apply the Fast Fourier Transform (FFT) to the
trajectories and use a percentage of the signal's total spectral
power calculated as the frequency of peak power and its first
harmonic. This system gives reasonable results when the face
is frontal and does not move. Furthermore, as mentioned
above, this system is based on using the frequency with peak However, it is not possible to directly use the
power as the first harmonic of the estimated heartbeat rate. regions detected by Viola's detector in our system, as it
However, this assumption is not always true, especially when contains eye and mouth areas that are not suitable for the
facial expressions are changing. purposes of our system. Because the areas mentioned above
are the most easily changed areas of the face, causing
Building on Balakrishnan's work, we established a more
movements not related to heart rate such as changing faces,
progressive system by:
blinking, breathing,... Therefore, the results can be very
misleading.

893
 2021 International Conference on Decision Aid Sciences and Application (DASA)
So, from the detected face, our system localizes regions of
interest and from there, the feature points are extracted and
tracked by an algorithm like Lucas Kanade of Bouguet [4].
The most stable parts of the face, most resistant to changes in
facial expressions, are the forehead and cheek areas around the
nose. To keep these regions, we first kept 60% of the width
and 80% of the height (obtained experimentally in proportion
to the human face) of the region detected by Viola's face
detector. Then in this refined area, we remove the eye area by
removing all pixels that are within 25% to 45% of the height
(obtained experimentally in proportion to the human face) of
the refined area (Figure 2).
Fig. 2. The green box is returned by Viola’s face detector and the red boxes
are those that are of interest to the proposed system.
After that, our system applies a moving average and a
bandpass filter to the vertical component of the trajectory of
each feature point to remove irrelevant frequencies and
involuntary head motion. We found that the head movements
associated with cardiac activity were very small and mixed
with a variety of other involuntary head movements. Based on
the structure of the human body, the head-neck and trunk
system can be thought of as a sequence of stacked inverted
pendulums. This construction allows for unrestricted head
movement in most axes. There are several sources of
involuntary head movement that complicate the isolation of
movements attributable to the pulsatile activity. The first is the
pendulum oscillatory motion that keeps the head in dynamic
equilibrium. Thus, the vertical direction is the best axis to
measure the movement of the upright caused by the pulsing
head which is suitable for the purposes of our research because
the horizontal axis tends to capture most of the dynamic
equilibrium swaying. The second source of involuntary head
movement is respiratory bobbing. That's why we filter out
low-frequency motion. The actual acceleration of the
involuntary vertical head movement measured about 0.098
m/s2. Meanwhile, the typical duration of the left ventricular
ejection time of a heart cycle is approximately 1/3 seconds.
Using these numbers, we can calculate a rough estimate of the
displacement of the head to be about:
Although this calculation ignores the complex structure of
the head system, it provides an indication of how small the
motion is.
In the next step, the filtered trajectories are fed to PCA to
find the strongest independent components. The PCA method
allows us to isolate the motion corresponding to the pulse and
project it onto a 1D signal allowing us to extract individual
beat boundaries from the peaks of the trajectory. To select the
correct source for analysis and calculate the duration of each
beat, we check the frequency spectrum and choose the source
894
with the clearest main frequency. The average pulse rate is
determined using this frequency. For more detailed analysis
and beat duration calculation, we perform peak detection in
the time domain. More details about this PCA algorithm will
be presented in the next section.
Of the components found by PCA, the most periodic
component belonged to heart rate. To find this most periodic
one, our proposed system applies DCT to all components
obtained by the PCA. As mentioned above, instead of using
the FFT algorithm to calculate the percentage of the signal's
total spectral power thanks to the frequency of the peak power
and its first harmonic, we improve by using the DCT
algorithm. After performing this substitution, our system
gives more stable and accurate results than Balakrishnan's
system.
The processed signal can be viewed as an
electrocardiogram, from which we can calculate heart rate.
IV. ALGORITHMS
A. Trajectory Generation and Smoothing
As mentioned in the previous section, we have selected
the appropriate area of the face that is related to the heart rate.
Our aim in this step is to extract the oscillation pulses in that
region which are recorded from the feature points as input
data for our system. To do this, we use the algorithm of Lucas
Kanade of Bouguet [4]. It helps us to capture the necessary
data in each frame. However, not all frequencies of the
trajectories are required or useful for pulse detection. The
resting heart rate of a normal adult is between 45 and 120
beats/min in other words, between 0.75 and 2 Hz. Based on
that, we conducted a test of the data in different frequency
ranges. Finally, we have found that the frequency range from
0.75 to 2 Hz gives a much more accurate result. However,
with the small frequency range, the data we have is not much
and affects the periodicity of the heart rate. Therefore, after
considering harmonics higher than 2 Hz, we decided to filter
the signal from 0.75 to 5 Hz because in this range the result
is more stable with relative accuracy. This is done using an
8th order Butterworth filter.
B. Principle component analysis (PCA)
The primary source signal of interest is the head
movement caused by the cardiovascular pulse. The
characteristic point trajectories are combinations of this as
well as other movements caused by sources such as
respiration, vestibular activity, and changes in facial
expression. While we only need information related to the
heart rate, we decompose this superimposed motion into sub-
signals to isolate pulse. For the above, we treat each facial
point in each frame as a separate data point and use PCA to
select for altered positions that are cardiovascular-specific
components. Next, we choose a dimension to project the
position time series to obtain the pulse signal. Analytically,
given N features, we represent the N-dimensional position of
the head at frame t as mt = [y1(t), y2(t), · · ·, yN(t)]. The mean
and the covariance matrix of those positions are:
1
� = |T| × ∑Ti=1 mi
m (2)
 2021 International Conference on Decision Aid Sciences and Application (DASA)

1
Σm = × ∑Ti=1(mt − m
� )T+1 (3) • Si with the largest Qi is the heartbeat rate signal, and
T
the heartbeat rate can be obtained as:
PCA algorithm finds the principal axes of the
transformation of the position as the eigenvalues of the FFT(IDCT (min{Khi})) × 60 bpm (7)
covariance matrix: V. EXPERIMENTAL RESULTS

Σm Φm = Φm Λ m (4) The proposed approach has been implemented in Python

3.0 and Matlab R2020a. The electrocardiogram signal which
where Λm represents the diagonal matrix of the is obtained after processing by our system is shown below.
eigenvalues λ1, λ2, · ·, λN corresponding to the eigenvalues in
the columns of Φm, φ1, φ2, · · ·, φN. Each eigenvector
represents the N-dimensional direction and magnitude of
motion for the feature points. The eigenvectors are different
for different objects. We obtain the 1-D position signal si(t)
by projecting the position time series onto φ𝑖𝑖 :
Fig. 3. ECG signal obtained from our proposed system.
𝑚𝑚1 To be able to compare and verify the accuracy of our
𝑚𝑚2 system, we recorded the actual heart rate of our test subjects
𝑠𝑠𝑖𝑖 (𝑡𝑡) = � ⋮ � φ𝑖𝑖 (5)
using the Omron HEM - 7121 blood pressure monitor. The
𝑚𝑚 𝑇𝑇 obtained results are shown in the following table:

In the real-time video, there are periods where the head

moves abnormally. Such movement adds variance to the
position vectors, thereby affecting PCA decomposition. To Heartbeat rate (bpm)
solve this problem, we discard a percentage α of the mt with No.
Real value Measured value Error
the largest L2-norms before performing PCA. However, all
1 84 87 4%
mt must still be used in the projection step (5) to produce a
complete signal. We set α at a value of 20% for our 2 79 74 6%
experiments. 3 84 84 0%
C. Discrete cosine transforms (DCT) 4 84 89 6%
After separating the sources that cause head movement 5 79 78 1%
using PCA, in this step we look for the signal generated by
6 74 78 5%
the heartbeat which is the most periodic. As mentioned
above, to quantify the periodicity of the signal, our system 7 89 88 1%
utilized DCT instead of Balakrishnan's system (2013) which 8 81 82 1%
used FFT. Because we consider the signal that we receive to
be discrete. Also, the DCT is a Fourier-related transformation 9 81 83 2%
similar to the discrete Fourier transform (DFT). However, 10 82 84 2%
DCT usually deals with Fourier series of a periodic and
symmetric extended series while DFT deals with Fourier From the above data table, it can be concluded that our
series coefficients of a periodic expanding series. We found system gives high accuracy results. For the heartbeat rate, the
that this symmetry contributes to a more substantial average error of this system is about 3% (±1%). This is
improvement of our system. acceptable because the human heart rate is an indicator that
After applying DCT, we keep only those DCT changes over time. Although the time difference between
components that carry the most significant power of the when we take a measurement with our system and when it is
signal. We do it as follows: measured with a sphygmomanometer is short, it still affects
• Calculate the DCT of the ith trajectory and obtain SCi. the results obtained. At the same time, this error is within the
• Determine {Kj}i which is the set of indexes for {Si(t)} allowable range for medical devices.
such that Kj is the index of the M first highest power However, in terms of medical emergencies, more
components into SCi which consists 50% of the power accuracy is required. This is an area for improvement as our
of Si. Where jϵ[1 ⋯ Mi ] ( Mi is some components that system has not yet collected large enough data and has not
carry 50% of the total power of Si). yet classified the groups of objects by age, gender,…
• Determine {Khl}i which is the set of the first 5 smallest VI. CONCLUSIONS
indexes into {Kj}i for each Si such that 2×Khl be found
on SCi. Where l = 1:5. The method of estimating the heart by pulses detected
• Calculate the cycle of the signal can be obtained by: from head movement is a new method. Although it has not
been tested by world medicine, it has brought a very positive
norm[SC(Khl ),SC(2×Khl )]
Qi = (6) result in many applications such as remote patient
norm[SCi ] monitoring. At the same time, it also opens a new direction
for non-contact heart rate measurement methods using

895
 2021 International Conference on Decision Aid Sciences and Application (DASA)
computer vision techniques. This non-contact measurement
method, not only makes the patient feel more comfortable and
convenient, but it can also provide long-term trends and
statistical analyzes for a more comprehensive assessment of
the treatment process. The system that we propose in addition
to that advantages, gives a quick result with high accuracy
and low cost. Nowadays, regular monitoring of heart rate in
the usual way is very time-consuming and laborious.
Sometimes, when these values are calculated, it is no longer
valid because of the loss of urgency. With the algorithm we
propose, this limitation has been overcome. Although not
exactly the same as the actual value, our system strikes a good
balance between accuracy, speed and economy.
The errors of our system, as mentioned above, are
remediable. In the future, heart rate will be more accurate if
our system is integrated with age recognition, gender
recognition,... In addition to correcting errors, we can also
develop our system to measure a person's heart rate when
there are head movements such as turning the neck, raising
the head,... when there are symptoms strong feelings on the
face such as smiling, talking, ...
ACKNOWLEDGMENT
We acknowledge the support of time and facilities from
Ho Chi Minh University of Technology (HCMUT), VNU-
HCM for this study.
896
REFERENCES
[1] M. Poh, D. McDuff and R. Picard, “Non-contact, automated cardiac
pulse measurements using video imaging and blind source separation,”
Optic Express, Optics Express, Vol. 18, Issue 10, pp.10762-10774,
2010.
[2] G. Balakrishnan, F. Durand, and J. Guttag “Detecting Pulse from Head
Motions in Video,” Computer Vision and Pattern Recognition,
Proceedings CVPR 13, 2013 IEEE Computer Society Conference on,
2013.
[3] P. Viola and M. Jones, “Rapid object detection using a boosted cascade
of simple features,” Computer Vision and Pattern Recognition,
Proceedings CVPR 01, 2001 IEEE Computer Society Conference on,
pp. 511-518, 2001.
[4] J. Bouguet, “Pyramidal implementation of the Lucas Kanade feature
tracker,” Intel Corporation, Microprocessor Research Labs, Technical
Report, 2000.
[5] N. Bernacchia et Al., “Noncontact measurement of heart and
respiration rates based on Kinect TM,” IEEE International Symposium
on Medical Measurements and Applications, 2014.
[6] Irani, Ramin, Nasrollahi, Kamal, Moeslund, B. Thomas, “.: Improved
Pulse Detection from Head Motions Using DCT,” Proceedings of the
9th International Conference on Computer Vision Theory and
Applications, Vol. 3, pp. 118-124, 2014.
[7] J. Shi, C. Tomasi, “Good features to track, Computer Vision and
Pattern Recognition,” Proceedings CVPR 94, 1994 IEEE Computer
Society Conference on, pp.593-600, 1994.
[8] C. Takano and Y. Ohta, “Heart rate measurement based on a time-lapse
image,” Medical Engineering and Physics, Vol. 29, Issue 8, pp. 853-
857, 2007.