Professional Documents
Culture Documents
Visvesvaraya Technological University Belagavi, Karnataka-590 014
Visvesvaraya Technological University Belagavi, Karnataka-590 014
Visvesvaraya Technological University Belagavi, Karnataka-590 014
Submitted by:
Arjun Halli 1MS17ML009
Chaitra M R 1MS17ML011
Jawad Shareif N 1MS17ML017
2020-2021
RAMAIAH INSTITUTE OF TECHNOLOGY, BANGALORE-
560054 DEPARTMENT OF MEDICAL ELECTRONICS
CERTIFICATE
Certified that the project work entitled “Study and Investigation of Body Sensor Localization for
health Monitoring Applications” carried out by Mr. Arjun Halli (USN: 1MS17ML009), Ms.
Chaitra M R (USN: 1MS17ML011), Mr. Jawad Shareif N (USN:
1MS17ML017) are the bonafide students of Ramaiah Institute of Technology. It is certified that all
corrections/suggestions indicated for Internal Assessment have been incorporated in the Report.
The project report has been approved as it satisfies the academic requirements in respect of Mini
Project work prescribed for the said Degree.
External Viva:
Examiner Signature with date:
1.
2.
DECLARATION
We, the students of Final semester of Medical Electronics, Ramaiah Institute of Technology,
Bangalore 560054 declare that the work entitled "Study and Investigation of Body Sensor
Localization for health Monitoring Applications" has been successfully completed under the
guidance of Mrs. Ravikala Pai, Associate professor, Medical Electronics Department,
Dr.N.Sriraam, Professor, Medical Electronics Department, and Mrs. Uma Arun, Assistant professor,
Medical Electronics Department, Ramaiah Institute of Technology. Further the matter embodied in
the project report has not been submitted previously by anybody.
Place:
Date:
Team members:
1. Arjun Halli 1MS17ML009
2. Chaitra M R 1MS17ML011
3. Jawad Shareif N 1MS17ML017
ACKNOWLEDGMENT
We express our sincere thanks to Dr. N.V.R. Naidu, Principal, RIT for providing us with the
necessary technical and administrative support.
We express our sincere gratitude and heartfelt appreciation to our project supervisor, Mrs. Ravikala
Pai, Associate Professor, Department of Medical Electronics, and co supervisors Dr.N.Sriraam,
Professor, Department of Medical Electronics and Mrs. Uma Arun, Assistant Professor, Department
of Medical Electronics, for providing us with continuous support and guidance to proceed in the right
direction whenever we were stuck in any stage of the project phase. We also like to thank them for
their blessings and constant motivation.
We would like to thank the Department of Medical Electronics, RIT, for providing us with good
facilities to complete our project.
Finally, we express our eternal gratitude to our parents and friends for their motivation and support
throughout.
TABLE OF CONTENT:
Abstract……………………………………………………………………………..………7
1. CHAPTER 1 INTRODUCTION……………………………………………..…...8
1.1 Introduction to Accelerometer….…………………………………………........9
1.2 Introduction to ECG and heart rate….…….……………………………………9
1.3 NEED FOR STUDY………………….…………………………………...........10
1.4 PROBLEM STATEMENT………………….…………………………………11
1.5 PROJECT OBJECTIVES………………….…………………………………….11
2. CHAPTER 2-LITERATURE SURVEY………………….………………………..12
2.1LITERATURE REVIEW………………….…………………………….............12
2.2 LITERATURE TABLE…………………………………………………………16
3. CHAPTER3 METHODOLOGY………………….……………………………..19
3.1 FLOW CHART OF PROPOSED WORK ANDCOMPONENTS USED……….19
3.1.1 COMPONENTS USED………………….…………………………….............19
3.1.1 Accelerometer ADXL345………………….……………………………........20
3.1.2 ECG SENSOR MAX30003…………………………………………….........21
3.1.3ESP32………………….…………………………………………..................22
3.2 WORKFLOW………………….………………………...................................23
3.3 PROCEDURE…………………………………………….................................23
3.4 Signal acquisition system………………….…………………..........................24
3.4.1SUBJECTS DETAILS…………………………………………………..........24
3.5 Data acquisition………………….…………………………………………......24
3.5.1 Wearable belts and Hardware setup…………………………………………...25
3.5.2 SLAVE PART………………………………………………............................26
3.5.3 MASTER PART……….…………………………………………...................27
3.5.4. Data obtained at master node………………….……………………………...28
3.6 SIGNAL PROCESSING……………………………………………….............29
3.6.1 Raw data plot………………….………………………………………….......29
3.6.2 Processing using MATLAB………………………………………….............31
3.6.2.1Steps of processing….…………………………………………...................31
3.6.2.2Bandpass filter for processing……………………………………………...31
3.6.2.3 Single axes processing………………….…………………………….........32
3.6.3 Feature extraction……………………………………………………….........32
3.6.4 Power Spectral Density (PSD)………………….…………………….............32
3.6.5 ACTIVITY CLASSIFICATION………………….…………………………34
3.7 Processing using PYTHON Human activity recognition using convolutional
neural network…………………………………………………………………34
3.7.1 Steps to be followed for signal processing of accelerometer data in python as
follow………………………………………………………………………….35
3.7.1.1Software tools required to perform the above steps………………………….35
3.7.2Preprocessing data for classification………………….………………………..35
3.7.3Visualization of data………………….…………………………......................36
3.8 Convolution Neural Network (CNN)…………………………………………….37
4. CHAPTER 4- RESULTS………………….……………………………………………39
4.1 RESULTS……………………………………………………………………….39
4.1.1Performance of model is measured in terms of accuracy and loss…………….40
4.1.1.1 The Training and test validation plot.……………………………………...40
4.1.1.1.a Model accuracy………………….………………….................................40
4.1.1.1.b Model loss………………….………………….........................................41
5. CHAPTER 5- CONCLUSIONS AND FUTURE REFERENCES………………….43
5.1 CONCLUSIONS………………….…………………........................................43
5.2 FUTURE SCOPE………………….…………………………………………....43
5.3REFERENCES………………….………………………………………….........44
LIST OF FIGURES
1. FIG 1.1 Capacitive Accelerometer.
2. FIG 1.2. NORMAL ECG WAVE
3. FIG 3.1 Flowchart of the design proposed.
4. FIG 3.2 ADXL345.
5. FIG 3.3 Layout of ADXL345.
6. FIG 3.4 MAX30003 sensor
7. FIG 3.5 Layout of MAX30003.
8. FIG 3.6 ESP32 Microcontroller.
9. FIG 3.7 Layout of ESP32.
10. FIG 3.8 Methodology of project.
11. FIG 3.9 Wearable belts and their position on the body.
12. FIG 3.10 Hardware setup.
13. FIG 3.11 Slave node.
14. FIG 3.12 Master node.
15. FIG 3.13 Data obtained at Master node through Bluetooth.
16. FIG 3.14 Data obtained from master node saved in csv file
17. FIG 3.15 2D Plot of all six-axis data obtained from hardware by performing specific activity.
18. FIG 3.16 Signal processing and classification flowchart.
19. FIG 3.17 Bandpass filter of walking signal.
20. FIG 3.18 PSD of Walking.
21. FIG 3.19 PSD of Walking upstairs.
22. FIG 3.20 SVM model outcomes.
23. FIG 3.21 Obtained dataset.
24. FIG 3.22 Modified dataset for analysis.
25. FIG 3.23 Visualization of walking data.
26. FIG 3.24 Visualization of sitting data.
27. FIG 3.25 CNN simple network.
28. FIG 3.26 CNN parameters with code is shown.
29. FIG 4.1 Confusion matrix of all activities.
30. FIG 4.2 Model accuracy with number of epochs.
31. FIG 4.3 Model loss with number of epochs.
LIST OF TABLES
1. Table 2.1 Literature table.
2. Table 3.1 Subjects background details.
3. Table 3.2 Comparison table of features for single Chest Y axis.
4. Table 3.4 Comparison of Power spectral density of walking activities chest Y axis.
5. Table 4.1 The precision table for activities.
6. Table 4.2 Testing and Validation with accuracy and loss values with respect to number
of epochs.
ABSTRACT
Our project aims at the development of an activity monitoring device that can detect the activity along
with cardiac monitoring for continuous ECG acquisition. Two accelerometers (three axis) ADXL345
made from Analog devices has been used to detect the activity. One of them is placed on the left
thigh, and another one was worn on the chest using the specially designed belt and fabric. The
accelerometer placed on the chest will detect the positions of the person, data generated is processed
using Microcontroller ESP32 having Bluetooth & Wi Fi capability. ESP32 controller also connected
to the ECG sensor module MAX30003 from Maxima Electronics, which continuously monitors heart
activity. Another ADXL345 which is placed on the thigh detects human activity like walking,
jogging, climbing steps, etc. Motion activity detected by ADXL345 with ESP32 which is placed on
the thigh acts as a master node, one placed on the chest acts as a Slave. Data are collected from
subjects for six different activities. Collected data were processed and analyzed using MATLAB, an
algorithm was developed based on the threshold of each activity and identifies the activity displaying
the name of the activity along with the display of heart rate.
CHAPTER 1
INTRODUCTION
1. INTRODUCTION
Human activity detection using wearable sensors is an emerging technology. And also a broad field
of study concerned with identifying the specific movement or action of a person based on sensor
data, movements are often typical activities performed indoors, such as walking, sleeping, standing,
and sitting. Human activity detection can be used for monitoring several metrics such as the daily
energy expenditure, body temperature, skin moisture, heart rate, and other human vital signs. The
proliferation of these wearable devices results in higher diversity of their usage. In addition, being
non-intrusive in daily activities requires these devices to be adaptable to individual users’ habits.
Continuous monitoring of the human body is becoming pervasive with the advent of wearable
devices capable of processing and storing large of amounts of an individual’s health data
Activity recognition is also helpful for the sports person along with monitoring other vital parameters
of the human body like Heart rate, ECG values, breathing patterns,
etc.
The effectiveness of health monitoring devices is heavily contingent on the correctness of sensor
installation/placement. As such, more work is needed to overcome several limiting challenges for
optimal validity of data. Ignoring these issues will result in faulty interfaces, responsible for unfair
physical activity gaming or false physical activity recording.
Wearable belts are designed in order to hold the sensors intake in order to accurately measure the
raw signals from the subjects
In order to collect the raw data from the subject there are two ways one with visual data that are
acquired from the cameras and other with using Micro Electro-Mechanical System (MEMS) sensors
which are like accelerometer, gyroscope, and magnetometer sensors. MEMS sensors are user
friendly, easily carryable that helps the subjects to easily perform the activity and it is low cost, so
with the recent technological advances in acceleration sensors, microprocessors, and wireless
communication areas, especially wearable systems are becoming interesting ways for helping
professionals in the health area.
Our project builds on the work by taking an end-to-end view of our proposed on-body sensor
localization technique. Our technique identifies the human activity devices using acceleration data
and also displays the heart rate using ECG sensor data.
The sensor data is recorded, processed, and analyzed for recognizing activity, helps in monitoring the
activities of the patients in hospital, or home remotely, also helpful for monitoring elderly people for
better health and also in some applications in healthcare, smart environments, and sports.
Accelerometer is one of the most widely used types of motion sensors, which is used for a variety of
applications such as device orientation detection, game controlling, shock protection, and activity
discovery. Given that accelerometers are relatively cheap and consume low energy, nowadays they
are embedded in most of the high-end portable systems.
An accelerometer is an electromechanical sensor used to measure acceleration forces. Such
forces may be static, like the continuous force of gravity or, as is the case with many mobile
devices, dynamic to sense movement or vibrations. Acceleration is the measurement of the
change in velocity, or speed divided by time.
They measure in meters per second squared (m/s2) or in G- forces (g). A single G-
force for us here on planet Earth is equivalent to 9.8 m/s2, but this does vary slightly with
elevation (and will be a different value on different planets due to variations in
gravitational pull). Accelerometers are useful for sensing vibrations in systems or for
orientation applications.
Capacitive accelerometers diagram shown in the FIG 1.1. The capacitive accelerometer measures a
change across a bridge however, instead of measuring a change in resistance, they measure a change
in capacitance. The sensing element consists of two parallel plate capacitors acting in a differential
mode.
An electrocardiogram, abbreviated as ECG, measures the electrical activity of the heartbeat. With
each beat, an electrical impulse (or “wave”) travels through the heart.
This wave causes the muscle to squeeze and pump blood from the heart. A normal heartbeat on
ECG will show the timing of the upper and lower chambers contractions. The normal ecg graph is
shown in the FIG 1.2 that indicates electrical activities of the heart and the right and left atria or
upper chambers make the first wave called a “P wave" following a flat line when the electrical
impulse goes to the bottom chambers. The right and left bottom chambers or ventricles make the
next wave called a “QRS complex." The final wave or “T wave” represents electrical recovery or
return to a resting state for the ventricles.
ECG sensor records the pathway of electrical impulses through the heart muscle, and can be recorded
on resting and ambulatory subjects, or during exercise to provide information on the heart's response
to physical exertion.
Heart rate is the speed of the heartbeat measured by the number of contractions of the heart per
minute. The heart rate can vary according to the body’s physical needs including the need to absorb
oxygen and excrete carbon dioxide, but is also modulated by a myriad of factors including but not
limited to genetics, physical fitness, stress or psychological status, diet, drugs, hormonal status,
environment, and disease/illness as well as the interaction between and among these factors. It is
usually equal or close to the pulse measured at any peripheral point.
In some research it mentioned that the Heart rate and ECG values continuously varies according to
the activities and to find relation between the activity and heart rate the the accelerometer and Heart
rate are considered for the project.
Wearable technology has the potential to provide enormous benefits to healthcare providers. By using
wearable devices to monitor patient data over a long period of time, medical professionals can get a
better view of the issues that are affecting the patient.
The developed sensor provides the information of body movements and direction values of physical
activities like sitting, standing, walking, etc. The activities are detected by values obtained from
sensing devices and processing those data with required algorithms for better outcomes, and also
making the sensing device as wearable so as to continually monitor.
Also wearable systems will provide promising applications in the medical system to enhance people’s
quality of life, to facilitate independent living, and to even save the life of people bearing the risk of
sudden attack.
Human activity recognition plays an important role in designing a health monitoring application. But
the idea is that once the subject's activity is recognized and known, an intelligent computer system
can then offer assistance. It is a challenging problem because there is no clear analytical way to
relate the sensor data to specific actions in a general way.
In order to get raw information which helps accuratly to classify human activity, a sensor that can
measure the changes in the accelerometer has to be chosen. Once an accelerometer is selected then
the main problem is about placement of the sensor in the correct location on the body so that it can
measure minute changes in the acceleration caused by bodily movement.
Once the sensor location is chosen a wearable belt has to be designed in order to place the sensors
that can stay intact on the subject's body. Then the signal processing of the raw accelerometer data is
to be done very carefully with proper requirement of parameters so to reduce noise in the signal and
then development of an algorithm to differentiate the activity is required which is major work for
classifying data for recognizing the activity with help of acquired data.
CHAPTER 2
LITERATURE SURVEY
For monitoring ECG and accelerometer the device is developed and it has display and it is also
connected through Bluetooth and in display along with activity there is medication menu which has
checklist it includes Palpitations, Weakness, Shortness of Breath, Chest Pain, and Fatigue, and many
other option for monitoring and for emergency situations for doctors and also pre detection of
tachycardia, the hardware components consists of The system hardware consists of off-the-shelf
components: an ECG monitor, a two-axis accelerometer and two iPaq units. The iPaq units provide
local processing and storage, but are mainly intended to be used as a link between the user and a
back-end system with far greater resources. The iPaq allows the wearer to see sensor signals as they
are collected and to annotate the record with specific activities, symptoms and medications using: a
checklist based Wellness Diary. The accelerometer data collection with ECG is considered two types
according to activity performed one as the lower activity like sitting, standing, sleeping, etc. and
higher activity like running, climbing stairs, etc. and the accelerometer classified according to
activity by applying RMS which gives good differentiation between periods of high and low activity
and can distinguish stationary activities from similar activities in moving vehicles also algorithms
on the back end server were able to accurately identify the R-wave complexes in most of the
ambulatory data, which is the basis for the arrhythmia and ectopic beat analysis programs.
This paper mainly focuses on the localization of the sensor in order to get the accurate raw data .A
technique to discover the position of sensors on the human body. Automatic on-body device
localization ensures correctness and accuracy of measurements in health and medical monitoring
systems. In addition, it provides opportunities to improve the performance and usability of
ubiquitous devices. Our technique uses accelerometers to capture motion data to estimate the
location of the device on the user’s body, using mixed supervised and unsupervised time series
analysis methods. We have evaluated our technique with extensive experiments on 25 subjects. On
average, our technique achieves 89% accuracy in estimating the location of devices on the body.
Wearable non-intrusive embed: The operation of motion and energy expenditure monitoring
systems heavily relies
on the placement and orientation of motion sensors, which usually include accelerometers and gyros.
The functionality of pedometers is an example of the relation between the accuracy and the device
placement. The step counting accuracy changes if the pedometer is attached to anywhere other than
the waist, because the user’s movements will be projected differently on the accelerometer. In an
observation, we attached 6 pedometers to different regions on a subject body and asked him to take
220 steps ded systems are used to remotely and ubiquitously monitor subjects conditions such as
basal body temperature, skin moisture, and ultraviolet exposure. Correct functionality of these
monitoring devices highly depends on the correct placement of them on the subject’s body. For
example, the following table summarizes the normal temperature range variation for normal adults
according to the location of the thermometer on the body.
Deep learning techniques have further improved human activity recognition performance on several
benchmark datasets. This paper presents a novel framework to classify and analyze human activities.
A new convolutional neural network strategy is applied to a single user movement recognition using
a smartphone. Three parallel CNNs are used for local feature extraction, and later they are fused in
the classification task stage. The whole CNN scheme is based on a feature fusion of a fine-CNN, a
medium-CNN, and a coarse-CNN. A tri-axial accelerometer and a tri-axial gyroscope sensor
embedded in a smartphone are used to record the acceleration and angle signals. Six human activities
successfully classified are walking, walking-upstairs, walking-downstairs, sitting, standing and
laying. Performance evaluation is presented for the proposed CNN.
4. Activity Recognition Algorithm for Daily Living Activities CNN framework and
Machine Learning Algorithm is presented to classify single:
MEMS-based accelerometers measure the acceleration directly.They have integrated, the velocity
and position can be derived from the measured acceleration.But the measured acceleration is
usually interfered with errors such as noise, offset and drift, which are enlarged under
integration. So simple and effective signal processing algorithms are desired to solve the problem,
improve the result and guarantee the accuracy of the measurement system. The aim of this paper is to
design and evaluate signal processing algorithms for the reduction of the drift of MEMS-based
accelerometer algorithms that are designed and used to improve the signal quality. These are low-
pass filter, threshold method with delay and a method for velocity trend estimation. The low-pass
filter is used to reduce the noise; the threshold method with delay is a method based on the threshold
method but improved for the continuous movements; the method for velocity trend estimation is
designed according to the features of the velocity signals and it is a method which can estimate the
velocity trend.
Threshold method with delay: Threshold method with delay is based on the simple threshold
method in which the value is set to zero if the value is smaller than threshold. However, the
drawback of the threshold method is that there is often not a smooth transition
at the threshold setting point and the signal becomes choppy, ex. suddenly becomes zero. Thus, the
threshold method with delay is designed to improve the signal under threshold such as continuous
movement signal, the algorithm can be implemented by the following steps:
● First is the comparison of input signals with threshold parameters. If the signal is larger than
threshold at time point i –1 and smaller than threshold from time point i , the time calculation
is set to zero and output signal is the input signal without processing. If at the next time point
the signal is still smaller than threshold, the time calculation adds.
● Then compare the calculated time with delay parameter, if the input signal value is smaller
than threshold and calculated time value is larger than delay parameter, the signal is set to
zero; otherwise, the output is just input without processing.
● Go back to point 1 for next value vid time point i +1.
Human activity recognition is an area of growing interest facilitated by the current revolution in
body-worn sensors. Activity recognition allows applications to construct activity profiles for each
subject which could be used effectively for healthcare and safety applications. Automated human
activity recognition systems face several challenges such as number of sensors, sensor precision, gait
style differences, and others. This work proposes a machine learning system to automatically
recognise human activities based on a single body-worn accelerometer. The in-house collected
dataset contains 3D acceleration of 50 subjects performing 10 different activities. The dataset was
produced to ensure robustness and prevent subject-biased results. The feature vector is derived from
simple statistical features. The proposed method benefits from RGB-to-YIQ colour space transform
as kernel to transform the feature vector into more discriminable features. The classification
technique is based on an adaptive boosting ensemble classifier. The system shows consistent
classification performance up to 95% accuracy among the 50 subjects.
The characterized effect of the number of convolutional layers employed to automatically learn
feature representations shows that increasing the number of convolutional layers tends to increase
the performance dataset, improving by 1% when a new layer is added. Performance changes are not
significant in the case of the Skoda dataset, showing a plateau. Results for the dataset show that
performance may be further improved if the number of convolution operations are increased. And
the accuracy of the model is about 69%.
7. A Public Domain Dataset for Human Activity Recognition Using Smartphones:
In this paper, human activity recognition data collected by a group of 30 volunteers with ages
ranging from 19 to 48 years were selected for this task. Each person was instructed to follow a
protocol of activities while wearing a waist-mounted Samsung Galaxy S II smartphone. Activities
are standing, sitting, laying down, walking, walking downstairs and upstairs. The collected signal
data median & third order low pass Butterworth filters are applied. This rate is sufficient for
capturing human body motion since 99% of its energy is contained below 15Hz for the output data
the feature mapping is done by mean, correlation, signal magnitude area (SMA) and autoregression
coefficients, A total of 561 features were extracted to describe each activity window. In order to ease
the performance assessment, the dataset has been also randomly partitioned into two independent
sets, where 70% of the data were selected for training and the remaining 30% for testing.
The data is classified by multiclass SVM (MC-SVM) & They show an overall accuracy of 96% for
the test data composed of 2947 patterns. Similar work on HAR using special purpose sensors have
shown comparable performance (90%-96%), such as in [3] where a system developed by collecting
data from 6 volunteers for the classification of 12 ADL using a waist-mounted triaxial accelerometer
provided an accuracy of 90.8%.
Six people participated in this study. These participants were required to abstain from consuming
stimulating beverages, such as coffee and alcohol, during the day before the experiment. The
participants recruited were college students. The experiment was conducted in a light and quiet
environment. External noise was excluded to ensure the participants remained calm. A TD1A system
(K&Y Labs) was employed because of its small size and reasonable cost. This wireless system
comprised one ECG and tri-axis acceleration sensor. The acceleration specification was 0.73 G/cm,
the size was 50*30*10 mm, and the weight was 11 g. The sampling frequency was 500 Hz. Using a
belt, an amplifier was fixed to each participant in the same relative position between the abdomen
and chest for each experimental measurement. The ECG measurements were performed using
electrodes placed on the participants’ left wrist and right ear bone. The analysis software used for
coding was MATLAB. The purpose of this experiment was to examine various body motion factors.
The three factors investigated were the motion rate, motion angle, and direction. The participants
were instructed to move in both directions (anterior-posterior and left-to-right) for varying durations
(2, 3, 5, and 7 s) and at differing motion angles (10, 15, and 30). Accelerometer data were
transmitted wirelessly, saved on a host computer in text format, and then analyzed offline using an
analysis program. The analysis process comprised several steps. The first step was raw data
extraction, and the second step was preprocessing. By removing the DC component derived from
the mode
value of each axis, the motion index series was the sum of the square values for each axis. The final
step involved extracting features from the motion index series. Both the mean and maximum values
of the motion index series were used. The corresponding distribution percentage of each axis at the
maximum point was also evaluated. The motion index was defined as the square sum of the three
axes’ acceleration signals. Therefore, each axis contained major motion components. Specific
posture change patterns had corresponding axis acceleration variations.
3. Coarse Activity
Fine Carlos hr Three parallel CNNs are used for Recognition
Convolutio Aviles Cruz, Local feature extraction, and latter Algorithm for
n Deep Andres they are fused in the Classification Daily Living
Learning Ferreyra task stage. Activities
Strategy for Ramirez,
Human · Whole CNN scheme is based on CNN
Arturo
Activity feature fusion of fine, medium, and framework and
Lopez
Recognitio n coarse CNN. Machine
Learning
· A tri-axial accelerometer and tri
CHAPTER 3
METHODOLOGY
The essential idea behind the on-body localization is to analyze the acceleration data when the user is
performing a specific activity and also to display the heart rate of the person while performing that
particular activity. The FIG 3.1 shows the flow chart of the proposed work. There are two
accelerometers set up interfaced with Microcontroller, one of the set up placed in Thigh to record the
Movements and other in chest along with the ECG sensor to record the movement along with the
ECG. Signal processing of data acquired by the Microcontroller is done and it is classified to
recognize the various activity.
.
ADXL345 accelerometer MEMS sensor provides 3 axis data i.e. X, Y, and Z of 3-dimensional
orientation values with 2, 4, 8, and 16g selectable sensitivity, FIG 3.2 shows the image of ADXL345
sensor and some of its features are mentioned below.
● Ultralow power: as low as 23 µA in measurement mode and 0.1 µA in standby mode at VS =
2.5 V (typical).
● Power consumption scales automatically with bandwidth User-selectable resolution Fixed 10-
bit resolution Full resolution, where resolution increases with g range, up to 13-bit resolution
at ±16 g (maintaining 4 mg/LSB scale factor in all g ranges).
● Embedded memory management system with FIFO technology minimizes host processor
load Single tap/double tap detection Activity/inactivity monitoring Free-fall detection.
● SPI (3- and 4-wire) and I2C digital interfaces Flexible interrupt modes mappable to either
interrupt pin Measurement ranges selectable via serial command Bandwidth selectable via
serial command.
The circuit layout of ADXL345 is shown in the FIG 3.3 the image is taken from the Adafruit
website.
FIG3.4 shows the image of MAX30003 AFE, It is a single lead ECG low power sensor by Proto
central which uses SPI communication for transferring ECG data with microcontroller. It provides
heart rate, ECG data, R to R between ECG waves using Pan-Tompkins algorithm, here are some
features of MAX30003.
● MAX30003 IC on-board
● Single-lead ECG monitoring
● R-R peak detection for heart rate computation
● High DC Offset range
● Heart Rate computation using Pan-Tompkins algorithm
● On-board level translator for 5V-tolerant operation
● On-board low-noise 1.8V and 3.3V voltage regulator
In the FIG 3.5 the layout of MAX3003 sensor is shown and its taken from the Protocentral website.
3.1.4 ESP32:
ESP32 is a series of low-cost, low-power system on chip microcontrollers with integrated Wi-Fi and
dual-mode Bluetooth. The ESP32 series employs either a Tensilica Xtensa LX6 microprocessor in
both dual-core and single-core variations or a single-core RISC-V microprocessor and includes built-
in antenna switches, RF balun, power amplifier, low-noise receive amplifier, filters, and power-
management modules. ESP32 is created and developed by Espressif Systems, and the FIR 3.6 is a
ESP32 microcontroller image is shown and in FIG 3.7 is the development circuit layout of ESP32 is
shown which are taken from the ESP ressif website.
Processors - CPU: Xtensa dual-core (or single-core) 32-bit LX6 microprocessor, operating at 160
or 240 MHz and performing at up to 600 DMIPS, Ultra low power (ULP) co-processor. And
Memory: 520 KiB SRAM, 448 KiB ROM, with Wireless connectivity - Wi-Fi: 802.11 b/g/n also
Bluetooth: v4.2 BR/EDR and BLE.
3.2 WORKFLOW:
FIG 3.8 workflow of the project Consisting of Two Hardware setup with accelerometer with
Acquisition modules one on Thigh and other on Chest for acquiring body movements. Data collected
is transferred through Bluetooth present along with Microcontroller to the PC for storage, later for
analysis and classification to identify the various human activity using MAT Lab & Python
For any work the information or data is most important, for human activity recognition there is
plenty of data is required this data is obtained by the sensors.
In order to collect the data we developed two compressed boards, first one which consist of an
ADXL345 accelerometer sensor, MAX3003 ECG sensor along with a microcontroller and power
circuit. Similarly a second board was built with just an ADXL345 sensor, microcontroller and a
power circuit.
Once the boards were ready we designed two wearable belts which could fit both the boards. We
designed the belts in such a way that the subject can wear it comfortably inside the clothing and
perform all their daily activities.
There are two belts, one on the chest and the other on the left thigh which is shown in the FIG 3.9
which helps holding hardware components.
The one belt on the chest is the slave part which has ADXL345 and MAX30003. It collects
accelerometer and heart rate data and sends it to the master part through ESPNOW protocol.
ESP-NOW is yet another protocol developed by Espressif, which enables multiple devices to
communicate with one another without using Wi-Fi. The protocol is similar to the low-power
2.4GHz wireless connectivity that is often deployed in wireless mouses. So, the pairing between
devices is needed prior to their communication. After the pairing is done, the connection is secure
and peer-to-peer, with no handshake being required.
Features of ESP-NOW:
● Encrypted and unencrypted unicast communication;
● Mixed encrypted and unencrypted peer devices;
● Up to 250-byte payload can be carried;
● Sending callback function that can be set to inform the application layer of. transmission
success or failure.
And we are using one way communication between two ESP32.
The one wearable goes on the thigh with a pouch which has a strap with gets attached to the
adjustable belt around the waistline, another one is to be worn around the chest area, which has three
way support that helps the sensors to be in right position all the time in spite of what activity the
subject is up to, and it holds the master part components i.e. ADXL345 and microcontroller with
power circuit, microcontroller collects data from slave and accelerometer then it stores it in sd card
also transferred through bluetooth for further requirements.
Features of HC05(BLE):
● Bluetooth protocol: Bluetooth Specification v2. ...
● Frequency: 2.4GHz ISM band.
● Modulation: GFSK(Gaussian Frequency Shift Keying)
● Emission power: ≤4dBm, Class 2.
● Sensitivity: ≤-84dBm at 0.1% BER.
● Speed: Asynchronous: 2.1Mbps(Max) / 160 kbps, Synchronous: 1Mbps/1Mbps.
● Security: Authentication and encryption.
The complete hardware setup is shown in the FIG 3.10 which has both slave and master nodes.
The FIG 3.12 is the master node hardware set up indicated with components mentioned.
FIG 3.14 gives the excel data in CSV file. Datasets are collected at PC which is transferred from
the Bluetooth of then the master node and the 1st column is heart rate data followed by X1, Y1, Z1,
X2, Y2 and Z2 values of the two accelerometer representing Master and Slave nodes respectively.
FIG 3.14 Data obtained from master node saved in csv file.
The accelerometer data has been recorded using two accelerometer sensors placed on chest and
thigh, each sensor has data of X, Y, and Z axis combining two sensors of chest and thigh makes
data of (X1 Y1 Z1) axis chest data and (X2 Y2 Z2) axis of thigh data and the 2D plot of all 6-
axis data from two accelerometer sensors with different activity are shown in the FIG 3.15 with
X-axis as Amplitude in g, and Y-axis is time in second in all 6 graph plots, and Series1 is X1,
Series2 is Y1, Series3 is Z1, Series4 is X2, Series5 is Y2, and Series6 is Z2 in all activities plot
graph.
FIG 3.15 2D Plot of all six axis data obtained from hardware by performing specific activity.
We have processed the raw accelerometer dataset in two different software for increase in the
accuracy of the classification.
The two software are
1. MATLAB
2. Human activity recognition using convolutional neural network
And the FIG 3.16 shown here will give visual information about the signal processing and
classification method.
3.6.2.2 Preprocessing:
The pre-process the data to remove background noise and only look at events, the sampling
frequency is 20Hz, each activity sitting, standing, walking, upstairs, downstairs, sleeping has
different sensor values. Above plots are not periodic and look random and have a noise of low
amplitude, that is it looks constant and cut-off any frequency below 0.5Hz and above 80hz
because this range contains useful data. Using a Butterworth Bandpass filter is very suitable
for this processing action to remove the impact of gravitational force and the FIG 3.17 is the
bandpass filtering of walking signal is shown. After processing and filtering of raw
accelerometer data then processed signal is used for Features extraction, Time domain features
mean, covariance are extracted for each activities.
To acquire the data in single axis, program is written in MATLAB to for acquiring six axis data with
two sets of accelerometer setup, requires program to process with individual filtering to remove
noise and obtain better signal. All the 6 axes are plotted separately.
For MATLAB accelerometer signal processing we are specifically looking at the single axis
Bandpass filtering of the activity for all six activities causes all the axes to overlap and the features
extracted are distorted, hence processing single axis data is useful for feature extraction.
MEAN and COVARIANCE: The features considered for analyzing data are Mean and Covariance in
time domain which are quite differentiates values as compared with sitting and standing activities
with other activities which is shown below table with less variation in sleeping, walking, walking
upstairs and down stairs activities.
FIG 3.18 shows the PSD of walking and FIG 3.19 shows the PSD of climbing upstairs, and the
considering power spectral density of the signal for walking vs walking upstairs Chest Y axis This
psd describes the present power in the signal or data as a function of frequency,
Here in the plot X-axis indicates the frequency range and Y-axis indicates power and expressed in
watts per hertz. The noticeable change in the PSD plot of walking vs walking upstairs these features
easily help in activity detection.
Using these frequency domain features we can successfully train the machine learning classifier for
the classification of the activities
Table 3.4 Comparison of Power spectral density of walking activities chest Y axis.
ACTIVITIES PSD VALUE
Walking 0.0059
Sleeping 0.16
It's really very difficult to visualize all this data but for a machine, especially a deep learning neural
network, it's really very easy to memorize all these data patterns and then they can classify it very
accurately. All activities have larger values hence we balance the data by considering 4000 values.
The user value or code is assigned such that N number of trainings does not overlap and it should not
be difficult for the classifier to predict the data and decision making. The data should be disputed
into training and the testing sample, hence ⅔ of data is considered for training and ⅓ of range of date
considered for the testing purpose out of whole data. And FIG 3.23 and FIG 3.24 shown below has
the graph plot of the walking and sitting data respectively.
The figure FIG 3.26 shows the parameters used for applying CNN classifier and their bit of codes are
mentioned. Sequential is the simple way to develop a model in TensorFlow and Keras. It allows to
build a model layer by layer. The ‘add()’ function is to add layers to the
model. The layers are Conv2D layers that are the convolution layers that are fed with input data or
signal, which are 2-dimensional matrices. 16 in the first and 32 in the second layer are the number of
nodes in each layer. This can be adjusted to be lower or higher, depending on the size of the data set,
16 and 32 fits well for the developed model. Kernel size or filter matrix for the convolution used is 2,
so a kernel size of 2 then it will be a 2x2 filter matrix. The activation function used for our layer is
the ReLU, or Rectified Linear Activation. This activation function has proved that it works well for
neural networks. Using Dropout layer, it will be beneficial to use large networks with less risk of
overfitting. The model with more layers and nodes should be using dropout to reduce the complexity
of working and increase efficiency of the model. Large weights in a neural network is an example of
a more complex network model that has overfit the training data. Large amount of training will be
possible if the dropout layer is used hence the model is designed with dropout range 0-0.5. In
inclusion between the Conv2D layers and the dense layer there is a Flatten layer serving as the
connection between the convolution and dense layers. It allows to change dimension of data from
vector of matrix into format that dense layer can interpret. Dense is a standard layer that is used
in models developed for neural networks with function 64. Design and flattening the output of
convolutional layers to build a single feature vector connected to the final classification model.
Softmax is implemented through a neural network layer just before the output layer. The Softmax
layer must have the same number of nodes as the output layer. The activation softmax used to
normalize network output between zero and one so the output can be represented as probabilities,
then model will make its prediction
CHAPTER 4
RESULTS
4.1 RESULTS:
To validate the performance of the proposed model, the model was trained with ⅔rd range dataset of
each sample activity that is recorded from the accelerometer sensor developed model in time series
data. The model successfully classified the activities with the accuracy of 91%. The precision of the
model is good for predicting Sleeping, sitting, standing, and walking. The model has problems for
clearly identifying upstairs and downstairs activities. And the confusion matrix of the classification
is shown in figure FIG 4.1.
Walking 0.88
Upstairs 0.61
Standing 1.00
Sitting 1.00
Sleeping 1.00
Downstairs 0.89
The model has some less precision for clearly identifying upstairs and downstairs activities. And all
other activities are predicted with good precision.
As the number of Epochs increases the accuracy of model increases, below graph has accuracy at
Y-axis and the number of epochs at X-axis and the two values are plotted in the graph the blue
color line is accuracy of train values with respect to number of epoch values and orange line is
accuracy of validation with respect to number of epoch values and FIG 4.2 indicates the model of
accuracy.
Thus the designed model successfully classified human activity recognition with very less error.
below graph has accuracy at Y-axis and the number of epochs at X-axis and the two values are
plotted in the graph the blue colour line is accuracy of train values with respect to number of epoch
values and orange line is accuracy of validation with respect to number of epoch values and FIG 4.3
shows the model loss graph.
The model was trained by keeping the hidden layers and learning constant and varying number
of epochs, the observation table is detailed here.
Table 4.2 Testing and Validation with accuracy and loss values with respect to number of
epochs.
Number of Testing Validation
epochs
From the observation it is cleared that as the number of epochs increases the performance of the
model and accuracy for classification of all activities also increases
CHAPTER 5
CONCLUSIONS
AND FUTURE REFERENCES
5.1 CONCLUSIONS:
The better human activity recognition system requires good data acquisition and proper positioning of
sensor on the body, so to achieve that we have designed our own data acquisition system with the
help of sensors and designed the wearable belts that fits the heardwares and makes comfortable to
subject to wear it and provide the specified position for the sensor on the body. The obtained data has
the Heart rate with it and we can analyse the variation of heart rate with respect to activity and also
the patients or subjects who are going under the normal physical activity can be monitored with help
of a device which is bluetooth supportive. And for obtaining data preprocessing is done and CNN
classification is used for classifying the data according to the activities.
The use of CNNs as a feature learning method for Human Activity Recognition is trending to become
common because conventional machine learning methods that require specific time domain and
frequency domain features, CNNs can extract features automatically to provide an illustration of
using a CNN feature extractor, rather than complex analysis. While architectures with different layers
of convolutional layers and kernel sizes for the convolution are tested the learning rate was kept to the
value of 0.001
further investigation of CNN for development as feature learning method in order to develop pre-
trained model
Pre-trained models can be model and applied directly to initialize weights or to extract features from
data directly in simple terms
5.3 REFERENCES:
1. A.S Abdull Sukor , A. Zakaria , N. Abdul Rahim, Wearable Wellness Monitoring Using ECG
and Accelerometer, 18-21 Oct. 2005 Hewlett-Packard Cambridge Research Laboratory, IEEE
Xplore 12 December 2005, DOI: 10.1109/ISWC.2005.59.
2. Alireza Vahdatpour, Navid Amini, Majid Sarrafzadeh, On-body Device Localization for
Health and Medical Monitoring Applications, Pervasive Mob Comput. 2011 Dec, 746–760.
doi: 10.1016/j.pmcj.2011.09.002.
3. Carlos Aviles Cruz, Andres Ferreyra Ramirez, Arturo Lopez, Coarse Fine Convolution Deep
Learning Strategy for Human Activity Recognition, 31 March 2019, Sensors (Basel). 2019
Apr; 19(7): 1556. Published online 2019 Mar 31. doi: 10.3390/s19071556.
4. Nhan duc Nguyen, Duong Trong Bui,Phyu Huu Truong,and Gu-Min Jeon, Activity
Recognition Algorithm for Daily Living Activities CNN framework and Machine Learning
Algorithm is presented to classify single user activities based on local feature extraction,
March 2020 IJARCCE 9(3):1-8, DOI:10.17148/IJARCCE.2020.9301.
6. Francisco Javier Ordóñez and Daniel Roggen, Deep Convolutional and LSTM Recurrent
Neural Networks for Multimodal Wearable Activity Recognition, 18 January 2016, Sensors
2016, 16(1), 115; https://doi.org/10.3390/s16010115.
7. Federico Cruciani, Chen Sun†, Shuai Zhang, Chris Nugent, Chunping Li, Shaoxu Song,
Cheng Cheng, Ian Cleland and Paul McCullagh, A Public Domain Dataset for Human Activity
Recognition in Free-living Conditions, 19-23 Aug. 2019, 2019 IEEE SmartWorld, Ubiquitous
Intelligence & Computing, Advanced & Trusted Computing, Scalable Computing &
Communications, Cloud & Big Data Computing, Internet of People and Smart City Innovation
(SmartWorld/SCALCOM/UIC/ATC/CBDCom/IOP/SCI), DOI:
10.1109/SmartWorld-UIC-ATC-SCALCOM-IOP-SCI.2019.00071.
8. Kang Ming chang, Sih hui chan, January 2013, Lecture Notes in Electrical Engineering
234:147-15. Intelligent Technologies and Engineering Systems (pp.147-151),
DOI:10.1007/978-1-4614-6747-2_18.