Date of publication xxxx 00, 0000, date of current version xxxx 00, 0000.

Digital Object Identifier 10.1109/ACCESS.2021.DOI

HIAuth: A Hierarchical Implicit

Authentication System for IoT Wearables
Using Multiple Biometrics
SUDIP VHADURI1 , (Member, IEEE), SAYANTON V. DIBBO2 , WILLIAM CHEUNG3 .
1
Department of Computer and Information Technology, Purdue University, West Lafayette, IN 47907 USA (e-mail: svhaduri@purdue.edu)
2
Department of Computer Science, Dartmouth College, Hanover, NH 03755 USA
3
Department of Computer and Information Science, Fordham University, NYC, NY 10458 USA
Corresponding author: Sudip Vhaduri (e-mail: vhaduri@gmail.com).

ABSTRACT Day-by-day wearable devices, such as smartwatches, are getting popular with their wide
range of services, including allowing financial transactions, unlocking cars, tracking health and fitness,
among many others. Most often, these services are managed based on users’ personal data. Unfortunately,
due to various limitations, most of the market-wearables either do not have any user authentication or have a
knowledge-based authentication, such as passwords, PINs, or pattern locks, which are not only burdensome
for users in the age of the internet of things (IoT) when the users are already flooded with so many passwords.
Therefore, there is a need for a burden-free implicit authentication mechanism for wearables that can utilize
different less-informative soft-biometric data easily obtainable from the market wearables. In this work, we
present a hierarchical implicit authentication (HIAuth) system that utilizes the heart rate, gait, and breathing
audio signals based on their availability to authenticate a user. From our detailed analysis, we find that binary
support vector machine (SVM) classifiers with radial basis function (RBF) kernels can achieve an average
accuracy of 0.98±0.04 (non-sedentary) and 0.94±0.03 (sedentary), F1 score of 0.98±0.03 (non-sedentary)
and 0.94±0.04 (sedentary), and genuine rejection rate of about 0.97±0.07 (non-sedentary) and 0.93±0.04
(sedentary), which shows the feasibility and promise of this work.

INDEX TERMS Wearable User Authentication, Implicit Authentication, Security, Multi-Biometric Model

I. INTRODUCTION the knowledge-based password, PIN, and pattern locks [13].

E first introduce the motivation of this work, followed
W by the existing research, and our contributions.
A. MOTIVATION
While the Internet of Things (IoT) enables us to remotely
collect information or control a multitude of physical objects
(ranging from smart cars to smart homes) utilizing the ex-
isting network infrastructure, recent advancements in smart-
phones and wearables with enhanced sensing and computa-
tional capabilities have made it possible to accomplish var-
ious IoT supported tasks, e.g., unlock cars, monitor homes,
monitor health and fitness, and perform financial transactions
with the help of smartphone- and wearable-applications [1]–
[10].
Though the smart wearables are getting popular with their
wide range of IoT supported services [11], [12], smart wear-
ables either have no authentication systems or often have
Many times, users disable these knowledge-based authentica-
tions to avoid the hassle of remembering passwords, numeric
or voice PINs, and unlocking patterns [14]. Unfortunately,
this recall burden is getting even worse during the age of IoT,
where a person has access to multiple devices, with each hav-
ing its own authentication. According to Lastpass, a password
security firm, an average worker needs to remember 25 to 85
passwords [15]. This causes an extra burden to the user and
motivates users to create “lazy”, easily hackable passwords.
According to McAfee, around 5% of all passwords are either
“Password” or “123456”, and over 79.9% of all passwords
are weak [16]. Thereby, an implicit biometric authentication
can greatly alleviate this memory and creativity burden.
Knowledge-based authentications are not only creating
recall and creativity burdens, but they are also erroneous.
Researchers have found that due to “fat finger” effects,
numeric PINs, voice PINs, draw PINs, and pattern locks

VOLUME 4, 2016 1

suffer from error rates of 7.5%, 14.0%, 20.7%, and 9.3%,
respectively [17]. An implicit biometric authentication nei-
ther suffers from the same issues nor requires passwords
retention. Due to all these benefits of implicit biometric au-
thentications over knowledge-based authentications, implicit
biometric authentications will be a good choice [17].
B. RELATED WORK
We first present different wearables limitations that restrict
us from adopting traditional biometric-authentications for
wearables, followed by advantages of multi-modal biometric
authentication approaches over single biometric authentica-
tion approaches, and finally, existing works on wearable
authentication.
1) Wearable Constraints
Due to smaller size and limited computing capabilities,
most of the authentications using traditional biometrics,
such as image [18]–[21], video [22]–[24], fingerprints [25],
voice [26]–[28], breathing patterns [29], [30], and keystroke
dynamics [31]–[33], are not adoptable in wearables [34],
[35]. However, with the advancement of technology, smart-
watch producers have started integrating new sensors into
wearables to collect various types of opportunistic data; for
example, Apple has started integrating electrocardiogram
(ECG) sensors into their smartwatches from series 6. While
these opportunistic data can authenticate users, these new
sensor data often suffer from low accuracy compared to their
implementations in dedicated clinical devices. For example,
researchers have found that Apple Watch and its ECG sensor
are only 19.6% accurate while detecting atrial fibrillation
of people under 55 years of old [36]. Additionally, users
need to touch the crown or screen of the smartwatches
with their second hand to construct an electric circuit to
obtain ECG readings while wearing the smartwatches. This
inconvenience limits the use of ECG as a wearable-biometric
that can be collected seamlessly and used for 24/7 non-stop
implicitly (i.e., with no active user interaction) to authenticate
the users. Finally, wearables have limited computing capa-
bility compared to other dedicated authentication devices.
Therefore, powerful models using deep neural networks that
can utilize fine-grained sensor data are still not applicable
to almost all market wearables. However, shallow machine
learning models could be implemented on wearables to avoid
cases where raw wearable data needs to be sent to the
server to authenticate users and thereby overcome the risks
of middleman attacks [37].
2) Multi-modal Biometric Authentication
Most of the single biometrics have underlying weaknesses
that make them less effective. For example, gait-based au-
thentication is a popular single biometric authentication
system, which can achieve 92% accuracy [38]. However,
this approach struggles during sedentary periods. To over-
come similar limitations of a single biometric approach,
researchers have started relying on combined multi-biometric
2 VOLUME 4, 2016
Vhaduri et al.: HIAuth
approaches, which can achieve around 98% genuine accep-
tance rates [39], [40], while stabilizing the performance loss
due to various factors, such as suffer from intra-class vari-
ations, inter-class similarities, spoof attacks, and presence
of noise [41], that adversely affect individual biometrics
and the performance of a single-biometric authentication
system. For instance, background noise can affect speech-
based authentication, the presence of dirt on fingers can affect
fingerprint-based authentication, and poor lighting can affect
facial recognition-based authentication. Still, a combined
approach can balance the performance loss due to one af-
fected biometric [42]. However, most of these existing multi-
modal authentications are not adoptable to wearable due
to various limitations, as discussed in the previous section
(Section I-B1). Therefore, we need to develop an authenti-
cation system that will work on multiple biometrics readily
available in market wearables to validate a user continuously
without requiring any active user action.
3) Wearable Authentication
Wearables, such as smartwatches, are continuously used
compared to other IoT objects, such as smartphones. There-
fore, IoT wearables need a non-stop authentication to se-
cure various sensitive data collected continuously by those
wearables compared to one-time authentication designed for
most gadgets. However, asking users for continuously ex-
plicit authentications is not convenient. Thereby, researchers
have been relying on various behavioral biometrics, such
as gait [43]–[45], activity types [34], [46], gesture [47],
keystroke dynamics [31], that can be obtained from wear-
ables continuously and can be utilized to uniquely iden-
tify a valid user without requiring any active user action.
Most of these authentication systems are limited to appli-
cation scenarios, e.g., gait-based behavioral authentication
approaches [44], [45] only work during walking. While other
projects have addressed some of the limitations of gait-based
approaches by considering different types of gestures [47] or
activities [34], [46], all of these models are based on move-
ment and fail to work properly when the subject is seden-
tary [35], [48]. Compared to behavioral biometrics, physio-
logical biometrics, such as heart rate obtained from photo-
plethysmogram (PPG) sensor [49] and bio-impedance [50]
are always available irrespective of a user’s physical state
(sedentary or non-sedentary) but require very fine-grained
samples, and sensor readings are easily affected by noise,
motion, etc. While some researchers develop a prototypical
authentication scheme combining one-time explicit authenti-
cation, such as fingerprint, followed by a continuous implicit
authentication using heart rate [51], [52], this will not be
applicable to most of the market wearables since they do
not have a fingerprint scanner and also heart rate alone is not
a powerful biometric to authenticate a user [14]. Therefore,
there is a need for a complete implicit authentication that uti-
lizes multiple biometrics available in most market wearables.
Vhaduri et al.: HIAuth
C. CONTRIBUTIONS
Main contribution of this paper is a hierarchical implicit au-
thentication (HIAuth) system for market-wearables utilizing
multiple coarse-grained soft-biometrics based on their avail-
ability (Section II-E). Though minute-level coarse-grained
heart rate samples are less informative, they are available
almost all the time when the wearable watch is worn. Addi-
tional biometrics, such as gait, can improve the authentication
system performance. However, unlike the heart rate data, gait
is available only when a user moves. On the other hand,
breathing audio data can have better availability. Still, audio
data usually suffer from various issues, such as a user’s
physical and mental state, presence of other sounds in the
environment. Thereby, a hierarchical multi-biometric-based
implicit authentication system, such as the HIAuth system,
that utilizes the availability of different biometrics can be
extremely powerful and useful.
From our detailed analysis, we find that the HIAuth
system can authenticate a user with an average accuracy
of 0.98 ± 0.04, F1 score of 0.98 ± 0.03, and a genuine
rejection rate of about 0.97 ± 0.07 during the non-sedentary
period using the three biometrics, i.e., heart rate, gait, and
breathing biometrics together and the binary SVM classifier
with RBF kernel (Section III-D). However, during sedentary
periods the HIAuth system can authenticate a user with an
average accuracy of 0.94 ± 0.03, F1 score of 0.94 ± 0.04,
and a genuine rejection rate of about 0.93 ± 0.04 using heart
rate and breathing data, and the binary SVM classifier with
RBF kernel. Compared to binary classification models, unary
classifier-based models achieve lower performance with sim-
ilar trends as binary models. But, the unary models have an
inherent advantage – we need only one class of data (i.e.,
valid user’s data) to train the models. Most importantly, all
our models achieve very low error rates (Section III-E).
In this manuscript, we present the HIAuth system that
has been built on our previous works [53], [54]. The main
contributions of this manuscript, compared to our previous
works, are summarized below:
• In this work, we compute seven types of features,
including GFCC, BFCC, LPC, and RPLP for every
breathing instance, compared to only MFCC used in our
previous work (Section II-C). Additionally, we intro-
duce the magnitude of acceleration features to generate
another layer of gait features.
• When selecting features, we follow a two-step process
(i.e., “correlation” method-based selection followed by
either “principal component analysis” (PCA) or “vari-
ance” method-based selection), compared to the “Se-
lectKBes” method-based feature selection used in our
previous work (Section II-D).
• Compared to our previous work, in this work, we use
polynomial kernels with a varying degree in addition
to the RBF kernels while developing unary classifiers
to better understand the intrinsic distribution of the
data obtained from a valid user (Tables 1 and 2 in
VOLUME 4, 2016
Section III-D).
• In this work, we have presented comparison among
different classifiers and models graphically using spider
plots (Figures 4 and 5 in Section III-D). Additionally,
we have introduced a novel aggregated metric, i.e.,
area within a spider polygon (AWSP) to give a holistic
measure of the overall performance while comparing
classifiers and models at different levels of the HIAuth
system.
• We have further compared the classifiers/models based
on relative gain/loss compared to the best classi-
fiers/models across the three critical performance mea-
sures, i.e., genuine rejection rate, genuine acceptance
rate, and area within a spider polygon (Tables 3 and 4
in Section III-D).
• In the case of non-sedentary periods, we achieve around
6% increase in average accuracy and F1 score (i.e., 0.98
versus 0.93) using binary models with heart rate, gait,
and breathing data altogether, compared to our previous
work [54]. Similarly, we achieve around 13% increase
in average accuracy and F1 score (i.e., 0.81 versus
0.72) using unary models, compared to our previous
work [53].
• In the case of sedentary periods, we achieve around 8%
increase in average accuracy and F1 score (i.e., 0.78
versus 0.72) using unary models with heart rate and
breathing data, compared to our previous work [53].
In the case of binary models, we observe similar per-
formance to our previous work during sedentary peri-
ods [53]. However, in this work, we witness a lower
standard deviation of around 0.03, compared to 0.07 that
we found in our previous work, which means our recent
models perform more consistently than the previous
models.
Organization: We first present data processing steps, in-
cluding different types of augmentation approaches to create
real-world effects and increase data volume (Section II-B).
After that we present feature engineering steps, including
feature computation (Section II-C) and two-step feature se-
lection approaches (Section II-D). Next, we present the data
availability-based hierarchical implicit authentication (HI-
Auth) system which utilizes multiple biometrics to validate
a user both in sedentary and non-sedentary periods (Sec-
tion II-E). Finally, in Section III-D, we present a detailed
evaluation of the HIAuth system, followed by conclusions
(Section IV) and discussion about our limitations which we
aim to address in the future (Section V).
II. APPROACH
In this section, we present the hierarchical continuous im-
plicit authentication (HIAuth) system. However, before pre-
senting the HIAuth system, we first introduce the datasets,
pre-processing steps, and feature engineering.
3

A. DATASETS
We use the following three types of data to develop the
HIAuth system. Data collection procedures were approved
by the Institutional Review Board (IRB).
• Heart Rate: We collected heart rate data at a rate of
one sample per minute from 10 subjects using the Fitbit
Charge HR device (described in [53], [54]) similar to
our previous work [14], [35], [55]–[66].
• Gait Data: The gyroscope and accelerometer readings
are gathered at a rate of one sample in 50 milliseconds
using the LG G Watch from 10 subjects [53], [54].
• Breathing Sounds: We recorded breathing sounds at
44.1 kHz sampling frequency from 10 subjects with six
distinct inhalation breathing events per subject using the
Evistr digital voice recorder [53], [54].
B. DATA PRE-PROCESSING
We first clean the raw data. Next, we segment the continuous
stream of biometrics to compute and select the most influen-
tial features.
1) Data Segmentation
We segment the heart rate and gait samples into 10-sample
windows with 50% overlap sliding window. From each sub-
ject, we obtain six clean inhalation sounds, each of which
we augment in 102 ways mentioned in the next section
(Section II-B2) to capture real-world effects and increase
data volume. Thereby, we obtain 612 original and augmented
breathing (i.e., inhalation) events from each subject. We
obtain the same 612 windows, i.e., instances from the heart
rate and gait data. This way, we obtain the same 612 instances
from each biometric to compute features and develop various
models (discussed in the Method: HIAuth System section,
i.e., Section II-E) of our HIAuth system.
2) Breathing Sound Augmentation
Breathing could be altered due to changes in environments,
physical state, or mood. To simulate these variations, we
augment the original audio breathing events in three ways.
• Pitch shift: We consider 15 pitch shifts ranging from -
3.5 to 3.5 with 0.5 increments.
• Speed change: We consider seven speed changes rang-
ing from .25x to 2x times the speed of an original
clip with an increment of .25x, skipping 1x since that
represents the original clip, which is already included in
the previous step, i.e., pitch shift with value 0.
• Noise superposition: We randomly select five vacuum
and five washing machine sound clips from the Envi-
ronmental Sound Classification (ESC-50) database [67],
as background noises to modify each original inhalation
sound with eight different signal-to-noise ratio levels
ranging from 10−4 to 104 , incremented by magnitudes
of 10 while skipping 1.
As a result, each original breathing clip is modified 102
times.
4 VOLUME 4, 2016
Vhaduri et al.: HIAuth
C. FEATURE COMPUTATION
We compute the following sets of candidate features.
• Heart rate features: From each window, we compute 21
statistical features: mean, median, standard deviation,
variance, coefficient of variance, range, coefficient of
range, first quartile or 25th percentile, third quartile or
75th percentile, maximum, interquartile range, coeffi-
cient of interquartile, mean absolute deviation, median
absolute deviation, energy, power, root mean square,
root sum of squares, signal to noise ratio, skewness, and
kurtosis, described in [14].
• Gait features: We compute the same above mentioned
21 features from each window p of x-, y-, z-axis, and
magnitude of acceleration f (= x2 + y 2 + z 2 ) based
on the readings obtained from both gyroscope (captures
angular acceleration) and accelerometer (captures linear
acceleration).
• Breathing features: From each inhalation breathing
event (original and augmented), we compute a wide
range of cepstral coefficients, including 40 Mel-
frequency cepstral coefficients (MFCCs), 13 Gamma-
tone frequency cepstral coefficients (GFCCs), 13 Bark
frequency cepstral coefficients (BFCCs), 13 Linear pre-
dictive coding (LPC), 13 Revised perceptual linear pre-
diction coefficients (RPLP), 12 chromas, and six tonal
features. Additionally, we compute five statistical fea-
tures from each chroma feature and tonal feature. We
use the Librosa and Spafe libraries to compute these
audio features [68], [69].
Thereby, we obtain 21 heart rate, and 168 gait features (21
from each of the six axes and two magnitude of acceleration
values) features from a single window of heart rate and gait
data, respectively, and 182 features from every inhalation
breathing event.
D. FEATURE SELECTION
To select the most influential features for binary models, we
follow a two-step approach. First, we filter features through a
correlation test. If a pair of features has a correlation value
above 0.95, we exclude one feature from the pair. In the
second step, we use Principal Component Analysis (PCA)
to reduce the feature count. In the case of feature selection
for unary models, we follow a different two-step approach.
Similar to the binary models, the first step is a correlation
check with the same 0.95 thresholds. In the second step, we
follow a variance-based approach. We select features that
have the lowest variance among the samples of one class
of data (i.e., target/valid user’s data) used to develop unary
models. Both for binary and unary models, we try various
feature counts to find an optimal choice. In Tables 1 and 2,
we present the optimal feature counts that we find from our
experiment.
E. METHOD: HIAuth SYSTEM
In Figure 1, we present the proposed hierarchical implicit
authentication (HIAuth) system for wearable-users utilizing
Vhaduri et al.: HIAuth
person-dependent multiple biometrics. Based on the avail-
ability of different biometrics and the confidence of an ac-
ceptance decision, we define the HIAuth system as below:
1) Level#1 Heart rate data-driven model (L#1 HR
model): The system initially tries to authenticate a
user using the heart rate data obtained from the photo-
plethysmogram (PPG) sensor, which is always avail-
able irrespective of a user’s physical state and easy
to obtain. However, coarse-grained (one sample per
minute) heart rate data may not be precise enough to
identify the user. Additionally, factors such as motion
artifacts or stress/emotion can affect the heart rate
data. Therefore, if the system cannot authenticate the
user with enough confidence, it checks whether the
user is moving from the on-device accelerometer and
gyroscope data, and based on that, the HIAuth system
either proceed to the Level#2 (if the user is moving) or
Level#3a (if the user is not moving) the authentication
module to consider additional biometrics based on their
availability. Otherwise, the system allows the user to
access the device.
2) Level#2 Heart rate and gait data-driven model (L#2
HRG model): At this level, the system tries to au-
thenticate the user based on a combination of gait and
heart rate biometrics. The user can access the device
if the system can authenticate the user with enough
confidence. If the system cannot authenticate the user
with enough confidence, it checks the authentication
module in the next level (i.e., Level#3b) that combines
breathing sounds with heart rate and gait biometrics.
Otherwise, the system allows the user to access the
device.
FIGURE 1: Overview of the HIAuth system
VOLUME 4, 2016
3) Level#3a Heart rate and breathing data-driven model
(L#3a HRB model): In the sedentary states, although
gait is not available, audio recordings from wearables
are still available. Therefore, breathing audio record-
ings and heart rate can be a good biometric combina-
tion to identify users during sedentary states. Still, if
the system cannot authenticate the user with enough
confidence using both heart rate and breathing data, the
user will be asked for explicit authentication, such as
password and PIN.
4) Level#3b Heart rate, gait, and breathing data-driven
model (L#3b HRGB model): At this level, the system
will try to authenticate the user based on heart rate,
gait, and breathing data. If the system cannot authen-
ticate the user with enough confidence, the user will be
asked for explicit authentication, such as password and
PIN similar to failure at Level#3a.
In the HIAuth system, we do not use the HRB seden-
tary model or the HRGB non-sedentary model as a default
choice. Instead, we use different models based on sensor data
availability and confidence of a model while authenticating a
user. This mechanism is important in the age of IoT when
people use different types of wearables, and the users may
want to turn off certain sensors for privacy or other issues.
Therefore, a hierarchical impact authentication system, such
as the HIAuth with multiple levels of user modeling will be
helpful to support the users in complex cases where different
sensors are either not active or not available/present in a
wearable.
We consider various classifiers to develop different models
in our HIAuth system. These include random forest (RF),
k-nearest neighbor (k-NN), naive bayes (NB), and support
vector machine (SVM) with binary and unary schemes. The
SVM classifiers come with two popular kernels, i.e., radial
basis function (RBF) kernel and polynomial (Poly.) kernel.
III. USER AUTHENTICATION
Now we present how the training-testing set split and our
modeling schemes, followed by the performance measures
we use to compare the learners and hyper-parameter opti-
mization.
A. TRAINING-TESTING SET
In our binary modeling, we have two classes: a valid user
(class-0) and impostors (class-1). To avoid overfitting, con-
sider at least 10 times more feature windows, i.e., number
of instances than the number of features. Since we obtain
M = 6 inhalation breathing audio events from one of N =
10 users, we follow a leave-one-sample-out strategy during
training-testing. In this approach, for each subject, we train-
test M = 6 separate models, where each model is trained
from M − 1 = 5 original and their 102 augmented breathing
events (i.e., in total 510 instances) and tested on remaining
original and its 102 augmented breathing events (i.e., in total
102 instances) to keep train and test sets mutually exclusive.
For other models that use non-breathing data, such as heart
5

rate and gait, we also pick 510 instances out of 612 instances
to train models and use the rest of the 102 instances to
test the trained models. Thereby, for each level of modeling
(discussed in the Method: HIAuth System section, i.e., Sec-
tion II-E), we train-test N × M = 10 × 6 = 60 models. For
class balancing, we consider the same M − 1 = 5 original
and their augmented breathing events (in total 510 instances)
from each class. Since the imposter class (class-1) consists of
N − 1 = 9 person, we uniformly pick 510/9 ≈ 56 instances
from each imposter. Similarly, in the test set, we consider
102 instances from the valid/target user and 102/9 ≈ 11
windows from each imposter. In the case of unary models, we
also follow the leave-one-sample-out strategy. But, compared
to the binary, unary models are developed with only a valid
user’s data with an outlier rate threshold (ν) to split the user’s
data into valid and outlier groups.
B. PERFORMANCE MEASURES
To evaluate the performance of different modeling ap-
proaches, we consider the following measures:
Accuracy (ACC), which is the fraction of predictions that
are correct, i.e.,
TP + TN
ACC = (1)
TP + FN + FP + TN
Root Mean Square Error (RMSE), which is the square
root of the sum of squares of the deviation from the prediction
to the actual value. It is equivalent to the square root of the
rate of misclassification, i.e.,
r
FP + FN
RM SE = (2)
TP + FN + FP + TN
Genuine Rejection Rate (GRR), which is the fraction of
invalid users rejected by an authentication system, or the
inverse of False Acceptance Rate (FAR), i.e.:
TN
GRR = = 1 − F AR (3)
FP + TN
Genuine Acceptance Rate (GAR), which is the fraction
of valid users accepted by an authentication system, or the
inverse of False Rejection Rate (FRR), i.e. :
TP
GAR = = 1 − F RR (4)
TP + FN
F1 Score, which is the measure of performance of an
authentication system based on both precision (positive pre-
dictive value) and recall (true positive rate) measures, i.e.:
 −1
TP TP
F1 Score = 2 + (5)
TP + FN TP + FP
Area Under the Curve - Receiver Operating Characteris-
tic (AUC-ROC), which is the graphical relationship between
GAR and FAR with the change of score thresholds. Termi-
nologies used in Equations 1 – 5 have their usual meaning
in machine learning when identifying a subject. Therefore, a
desirable authentication system should have lower negative
6 VOLUME 4, 2016
Vhaduri et al.: HIAuth
measures (i.e., RMSE, FAR, and FRR) but higher positive
measures (i.e., ACC, GRR, GAR, F1 Score, and AUC-ROC)
of performance. We also use Equal Error Rate (EER), which
is defined as the point when FRR and FAR are equal, i.e.,
a trade-off between the two error measures. Additionally,
we use an aggregated measure, i.e., Area Within a Spider
Polygon (AWSP), which combines multiple performance
measures, such as accuracy, genuine rejection rate, genuine
acceptance rate, F1 score, and AUC-ROC into a single met-
ric. The AWSP is defined as the shape polygon whose axes
represent different performance measures and values ranging
from zero to one. In the case of binary and unary models,
we obtain pentagon (Figure 4) and quadrilateral (since unary
models do not have AUC-ROC) (Figure 5), respectively.
C. HYPER-PARAMETER OPTIMIZATION
During each leave-one-sample-out validation, we perform a
grid search to find an optimal set of parameter values from
a range of values. We obtain similar values from different
leave-one-sample-out runs. In Tables 1 and 2 we present the
optimal configurations that are observed across most of the
runs.
D. HIAuth SYSTEM EVALUATION
In Tables 1 and 2, we present the performance of various
models, with optimal classifiers and their optimal set of
parameter values, developed at different levels of the hierar-
chical continuous implicit authentication (HIAuth) system.
As described previously in Section II-E, the heart rate data-
driven model (i.e., L#1 HR model) is the first level of the
HIAuth system. Therefore, in Table 1, we first introduce
the performances of the best L#1 HR model. Then, we
observe that the binary L#1 HR model (developed using
SVM classifier with RBF kernel) can achieve an average
ACC and AUC-ROC of 0.67 ± 0.12. Compared to binary,
unary HR model (developed using SVM classifier with Poly.
kernel) achieves a lower performance, i.e., an average ACC
of 0.55 ± 0.05 and an average AUC-ROC of 0.55 ± 0.07.
This happens since unary models are trained from only one-
class data (valid/target user), and part of the valid user’s data
is used as outliers while training.
As discussed in Section II-E, if the L#1 HR model is not
confident enough to authenticate a user or if it fails to authen-
ticate a user, the HIAuth system moves to the next level to
consider additional biometrics based on their availability. In
Table 1, we observe that addition of gait (when the user is
moving) with heart rate at Level#2 the performance of our
HIAuth system improves. In the table, we observe that the
best binary L#2 HRG model achieves a 37% increase both in
ACC and AUC-ROC, compared to the best binary L#1 HR
model. Similarly, the F1 score, GRR, and GAR are increased
by 44%, 26%, and 52%, respectively, while comparing the
binary L#2 HRG model with the binary L#1 HR model.
Though gait data is only available while a user is moving,
its addition to less accurate heart rate data can significantly
improve authentication performance. Contrary to binary, the
Vhaduri et al.: HIAuth

TABLE 1: The best L#1, L#2, and L#3a models with average and standard deviation (in parenthesis) values of different performance measures

Models Classifier (optimal feature ACC RMSE GRR GAR F1 AUC- AWSP
parameter values) count score ROC
BINARY
L#1 HR SVM (RBF kernel, 5 0.67 0.04 0.74 0.60 0.63 0.67 0.43
γ = 0.05, C = 5) (0.12) (0.01) (0.16) (0.18) (0.15) (0.12)
L#2 HRG SVM (RBF kernel, 10 0.92 0.02 0.93 0.91 0.91 0.92 0.84
γ = 0.05, C = 5) (0.06) (0.01) (0.06) (0.08) (0.06) (0.06)
L#3a HRB SVM (RBF kernel, 10 0.94 0.02 0.93 0.95 0.94 0.94 0.89
γ = 0.08, C = 4) (0.03) (0.01) (0.04) (0.04) (0.04) (0.03)
UNARY
L#1 HR SVM (Poly. kernel, 10 0.55 0.05 0.54 0.57 0.55 N/A 0.31
d = 2, ν = 0.5) (0.05) (0.09) (0.09) (0.07) (0.07)
L#2 HRG SVM (RBF kernel, 20 0.64 0.04 0.61 0.67 0.45 N/A 0.41
γ = 0.05, ν = 0.5) (0.08) (0.00) (0.14) (0.13) (0.09)
L#3a HRB SVM (RBF kernel, 20 0.78 0.03 0.86 0.70 0.76 N/A 0.60
γ = 0.05, ν = 0.25) (0.06) (0.00) (0.11) (0.05) (0.05)
TABLE 2: Different L#3b HRGB models with average and standard deviation (in parenthesis) values of different performance measures

Classifiers (optimal parameter values) feature ACC RMSE GRR GAR F1 AUC- AWSP
count score ROC
BINARY
RF (no of estimators = 50) 20 0.84 0.03 0.77 0.91 0.86 0.84 0.71
(0.11) (0.01) (0.24) (0.08) (0.08) (0.11)
k-NN (k = 5, minkowski distance) 20 0.97 0.01 0.96 0.99 0.98 0.98 0.95
(0.03) (0.01) (0.05) (0.02) (0.03) (0.03)
NB 20 0.59 0.04 0.56 0.62 0.57 0.59 0.34
(0.09) (0.01) (0.30) (0.25) (0.15) (0.09)
SVM (RBF kernel,γ = 0.08, C = 4) 20 0.98 0.01 0.97 0.99 0.98 0.98 0.96
(0.04) (0.01) (0.07) (0.02) (0.03) (0.04)
SVM (Poly. kernel, d = 3, C = 16) 20 0.96 0.01 0.95 0.97 0.96 0.96 0.92
(0.06) (0.01) (0.12) (0.02) (0.04) (0.06)
UNARY
SVM (RBF kernel, γ = 0.05, ν = 0.1) 40 0.81 0.03 0.79 0.82 0.81 N/A 0.65
(0.09) (0.01) (0.13) (0.06) (0.08)
SVM (Poly. kernel, d = 1, ν = 0.75) 40 0.48 0.05 0.42 0.54 0.51 N/A 0.24
(0.11) (0.01) (0.17) (0.08) (0.10)
SVM (Poly. kernel, d = 2, ν = 0.75) 40 0.47 0.05 0.05 0.89 0.62 N/A 0.23
(0.12) (0.01) (0.22) (0.11) (0.10)

unary L#2 HRG model shows a similar performance when
compared with the unary L#1 HR model, i.e., 16%, 13%, and
18% increase in ACC, GRR, and GAR, respectively, but an
18% drop in F1 score. This indicates the need to consider gait
data from both classes instead of relying only on valid user’s
gait and consider part of it as outliers.
If the HIAuth system cannot verify a user at level#1 and
the user is not moving, the authentication system goes to
Level#3a, where the system combines the user’s breathing
data with heart rate to attempt to verify the user utilizing
the L#3a HRB models. In Table 1, we observe that the best
L#3a HRB models (i.e., both binary and unary) perform
better than the best L#1 HR models and L#2 HRG models.
This demonstrates the influence of breathing data to uniquely
VOLUME 4, 2016
identify a user compared to the other two biometrics.
Finally, if the HIAuth system cannot verify a user at
level#2, the authentication system goes to Level#3b, where
the system combines the user’s breathing data with heart
rate and gait data to attempt to verify the user utilizing
the L#3b HRGB models. In Table 2, we present different
classifiers with their optimal parameter values while devel-
oping different L#3b HRGB models. We find that SVM with
RBF kernel performs the best both for binary and unary
classifiers. Furthermore, while comparing the performance
measures of the best L#3b models (Table 2) with the best
models of other levels in the HIAuth system (Table 1), we
observe that the best L#3a HRGB models (i.e., both binary
and unary) perform better than the best L#1 HR, L#2 HRG,
7
 Vhaduri et al.: HIAuth

TABLE 3: Performance gain (written inside parenthesis) of different L#3b

and L#3a HRB models, which demonstrates the importance HRGB model classifiers with respect to the best SVM (RBF) classifiers across
three performance measures
of the addition of more biometrics based on their availability
while identifying a user. Classifiers GRR GAR AWSP
In Figures 2 and 3, we use boxplots to present a more BINARY
in-detailed analysis of different performance measures in RF 0.77 0.91 0.71
addition to the average values of the best L#3b HRGB models (-20.6%) (-8.1%) (-26.0%)
presented in Table 2. In general, we observe that median k-NN 0.96 0.99 0.95
values are higher than the average values, i.e., average values (-1.0%) (0.0%) (-1.0%)
have been affected by outliers. For example, in the case of NB 0.56 0.62 0.34
the binary model, we obtain 18% and 11% higher GRR and (-42.3%) (-37.4%) (-64.6%)
GAR, respectively, while comparing median with the average SVM (RBF) 0.97 0.99 0.96
values (Figure 2). Similarly, in Figure 3, we find around 4% (0.0%) (0.0%) (0.0%)
higher GRR while comparing the median value with the av- SVM (Poly.) 0.95 0.97 0.92
erage value of the best L#3b HRGB unary model. Compared (-2.1%) (-2.0%) (-4.2%)
to the binary model, the unary model has narrow interquartile UNARY
ranges, which show the consistency of performance measures SVM (RBF) 0.79 0.82 0.65
of the unary model. (0.0%) (-8.1%) (0.0%)
SVM (Linear) 0.42 0.54 0.24
(-47.2%) (-39.7%) (-63.9%)
SVM (Poly.) 0.05 0.89 0.23
(-93.50%) (0.0%) (-64.91%)

TABLE 4: Performance gain (written inside parenthesis) of different HIAuth

models with respect to the best L#3b HRGB models across three performance
measures

Models GRR GAR AWSP

BINARY
L#1 HR 0.74 0.60 0.43
(-24.0%) (-39.7%) (-35.8%)
L#2 HRG 0.93 0.91 0.84
FIGURE 2: Boxplot of performance measures of the best binary L#3b HRGB (-4.4%) (-8.5%) (-6.7%)
model (SVM RBF) with cross markers (×) representing the average values
L#3a HRB 0.93 0.95 0.89
(-3.7%) (-3.8%) (-3.8%)
L#3b HRGB 0.97 0.99 0.96
(0.0%) (0.0%) (0.0%)
UNARY
L#1 HR 0.54 0.57 0.31
(-37.2%) (-30.6%) (-31.6%)
L#2 HRG 0.61 0.67 0.41
(-29.1%) (-18.3%) (-36.9%)
L#3a HRB 0.86 0.70 0.60
(0.0%) (-14.6%) (-7.7%)
L#3b HRGB 0.79 0.82 0.65
(-8.1%) (0.0%) (0.0%)

FIGURE 3: Boxplot of performance measures of the best unary L#3b HRGB

model (SVM RBF) with cross markers (×) representing the average values
Next, in Figures 4 and 5, we use spider plots to compare
the performance of different classifiers (with their optimal
parameters), used to develop models at different levels of
the HIAuth system, in an aggregated form. We use the Area
Within the Spider Polygon (AWSP) measure (numeric values
are presented in the last column in Tables 1 and 2) to compare
the aggregated performance of different classifiers of each
model.
8 VOLUME 4, 2016
While we use the spider plots to compare different clas-
sifiers graphically, we further investigate each classifier’s
performance gain/loss with respect to the best classifier based
on three major measures, i.e., GRR, GAR, and AWSP. While
the AWSP can be thought of as an optimal measure (since it
combines all other measures), GRR and GAR are considered
as the most critical security and usability measures, respec-
tively, of an authentication system. Therefore, we investigate
these three measures further. In Table 3, for each perfor-
mance measure, we first compare how much an individual
Vhaduri et al.: HIAuth
FIGURE 4: Spider plot of performance measures of four binary models (with five classifiers) used at different levels of the HIAuth system
FIGURE 5: Spider plot of performance measures of four unary models (with three classifiers) used at different levels of the HIAuth system
classifier can gain or loss with respect to the best value of
that performance measure, while modeling Level#3b HRGB
(i.e., models trained with heart rate, gait, and breathing data).
In the table, average values are first written down, and then
performance gain values are written down inside parentheses.
Negative gain means performance loss with respect to the
best classifier performance. In the table, we observe that
in the case of binary classifiers, the SVM classifier with
RBF kernel achieves the highest AWSP of 0.96 with no
loss on security (GRR) and usability (GAR). Thereby, the
SVM classifier with RBF kernel is an obvious choice for
the binary L#3b HRGB model. However, when comparing
among the unary classifiers, the SVM with RBF kernel losses
8.1% GAR compared to the highest GAR score of 0.89
obtained from the SVM classifier with Poly. kernel, which
is a quadratic polynomial kernel, i.e, degree, d = 2. Though
the SVM classifier with Poly. kernel has the highest usability
score (GAR), it has a drastic loss in security score (i.e., 93.5%
loss in GRR score) while comparing with the SVM classifier
with RBF kernel. Therefore, the SVM classifier with RBF
kernel is a better option for the unary L#3b HGRB modeling.
Similarly, in Table 4, we investigate how the availability
of data can contribute to the performance gain or loss of
a model with respect to the best value while developing
models at different levels of the HIAuth system. In the case
of binary models, the L#3b HRGB (model that combines all
three biometrics – heart rate, gait, and breathing data) has the
best security (GRR), usability (GAR), and overall (AWSP)
performance measures. While binary model trained with only
VOLUME 4, 2016
heart rate data, i.e., L#1 HR model, faces significantly high
loss across all three performance measures, either L#2 HRG
(models trained with heart rate and gait data) or L#3a HRB
(models trained with heart rate and breathing data) faces
relatively smaller loss compared to the L#1 HR model with
respect to the L#3b HRGB model. This demonstrates the
power of additional biometrics while developing the HIAuth
authentication system. When looking at the unary model, we
see a similar trend as we have witnessed for the binary mod-
els. When the user is not moving, the addition of breathing
data with heart rate data (i.e., moving from L#1 HR to L#3a
HRB) we observe performance gain. Similarly, when the user
is moving, the addition of new biometrics with heart rate
contributes to performance gain, i.e., performance improves
moving from L#1 HR to L#2 HRG, and then to L#3b HRGB.
This demonstrates the importance of additional biometrics in
a multi-biometric authentication system.
Finally, in Figures 6 and 7, we present the Probability Dis-
tribution Function (PDF) and Cumulative Distribution Func-
tion (CDF) of performance (in terms of ACC and Fs core)
of the best models. In Figure 6, we find that around 98%
of the performance values (both ACC and F1 scores) fall
in the range of 0.90 – 1, which shows that the best binary
L#3b HRGB model performs consistently well for most of
the cases. Similarly, in Figure 7, we observe that in the case
11
of unary models, a 20 (i.e., ≈ 55%) of the values fall in the
range of 0.6 – 0.8, which is also a reasonable performance
for the best unary L#3b HRGB model [14].
9

FIGURE 6: PDF and CDF of the best binary L#3b HRGB model performances
FIGURE 7: PDF and CDF of the best unary L#3b HRGB model performances
E. ERROR RATE ANALYSIS
In this section, we present an analysis, i.e., the trade-
off between security (in terms of FAR) and usability/user-
friendliness (in terms of FRR) measures of the HIAuth
system. While the False Acceptance Rate (FAR) of an au-
thentication system measures the fraction of cases where the
system accepts imposters as a valid user and allows access
to the wearable, the False Rejection Rate (FRR) indicates the
fraction of cases where the authentication system rejects a
valid user as an imposter. Thereby, an ideal system should
have a lower FAR and FRR. In Figure 8, we present our
analysis of error rates (FAR and FRR) with varying con-
fidence thresholds for the best binary L#3b HRGB models
FIGURE 8: Change of error rates (FAR and FRR) with varying confidence
thresholds using the best binary L#3b HRGB model
10
Vhaduri et al.: HIAuth
trained using the SVM classifier with RBF kernel. In the
figure, we observe while the FRR (usability) starts to drop
drastically with the increase of confidence threshold after
0.5, FAR (security) remains steady. However, choosing a
relatively high threshold, such as 0.8 or 0.9, will lead to a
situation where the system will ignore most of the decisions
made by the classifier because most of the decisions made by
the classifier will have a confidence value lower than the high
threshold and then, the system will go for recurrent attempts
to find a decision with a confidence value higher than the
threshold to either accept or reject the user. Thereby, this
will incur additional delays before making a real decision
(either accept or reject), which in the long run may lead to
situations where users opt-out of using the authentication
system. Additionally, since we cannot find a threshold where
the two lines intersect (i.e., FAR = FRR), we do not have an
Equal Error Rate (EER) in this graph. But, both error rates
are very low throughout the entire range of the confidence
threshold. Therefore, we can use a moderate confidence value
of around 0.5 – 0.7 to avoid optimize both the security and
usability of the system.
IV. CONCLUSIONS
In this paper, we perform feasibility testing of a hierarchical
implicit user authentication (HIAuth) system based on the
availability of three separate biometrics, i.e., heart rate, gait,
and breathing sounds easily obtainable in most of the market
wearables. While this work demonstrates the feasibility and
importance of a sensor/data availability-based hierarchical
multi-modal continuous user-authentication, it shows the
promise to adopt in other types of market wearables with
different types of sensors. During non-sedentary periods,
the HIAuth system can authenticate a user with an average
accuracy and AUC-ROC of 0.98 ± 0.04 with low error rates
using the L#3b HRGB model, i.e., model at level#3b that
relies on the availability of all three biometrics – heart rate,
gait and breathing data. Similarly, during sedentary periods,
the HIAuth system can authenticate a user with an average
accuracy and AUC-ROC of 0.94 ± 0.03 with low error rates
using the L#3a HRB model, i.e., model at level#3a that
relies on the availability of two biometrics – heart rate and
breathing data. These findings show the promise to develop
a continuous implicit-authentication system for the market
wearables utilizing their limited sensing and computational
capability to secure our valuable information and create a
safe gateway to unlock cars, access online accounts, etc.
V. DISCUSSION
This work has some limitations, which we plan to address
in the future. First, we have a limited number of original
breathing (inhalation) sounds from each subject. However,
we consider a total of 102 augmentations using pitch shift,
speed change, and noise superposition with different signal-
to-noise ratio levels to create the real-world effects caused
by the change of a user’s physical and mental states as well
as the change in environments (Section II-B2). These 102
VOLUME 4, 2016
Vhaduri et al.: HIAuth
augmentations also contribute to an increased data volume.
Additionally, to ensure mutual exclusion, we first split the
train-test sets based on original breathing instances and then,
including the augmented versions, to their respective sets.
Thereby, the findings from this work show the feasibility
of such a hierarchical implicit authentication system for
IoT wearables. Second, in this feasibility work, we use a
limited set of 10 subjects. However, we perform all our
model evaluations using the leave-one-sample-out validation
approach with mutual exclusion between train-test sets and
class balancing to obtain a better insight into our analy-
sis and findings. Thereby, results find from our analysis
show a promise further to investigate this with a large-
scale extended-period study dataset. Third, we use three
relatively independent biometric data with feature selection
to optimize implementation; thereby, our results potentially
show a baseline performance, which can be improved using
three dependent/correlated biometrics from the same subject.
Finally, once the wearables are improved with advanced
computation capabilities for deep neural networks, we aim
to develop more advanced models to deploy in wearables.
REFERENCES
[1] S. Seneviratne, Y. Hu, T. Nguyen et al., “A survey of wearable devices and
challenges,” IEEE Communications Surveys & Tutorials, vol. 19, no. 4,
pp. 2573–2620, 2017.
[2] S. V. Dibbo, Y. Kim, and S. Vhaduri, “Applicability of Generic Cough
Models to Detect Respiratory-Coughs with Noisy Backgrounds,” in IEEE
International Conference on Wearable and Implantable Body Sensor Net-
works (BSN), 2021.
[3] S. Vhaduri and C. Poellabauer, “Impact of different pre-sleep phone use
patterns on sleep quality,” in IEEE International Conference on Wearable
and Implantable Body Sensor Networks (BSN), 2018.
[4] S. Vhaduri, A. Munch, and C. Poellabauer, “Assessing health trends of
college students using smartphones,” in IEEE Healthcare Innovation Point-
of-Care Technologies Conference (HI-POCT), 2016.
[5] S. Vhaduri and C. Poellabauer, “Design and Implementation of a Remotely
Configurable and Manageable Well-being Study,” in EAI SWIT-Health,
2015.
[6] S. Vhaduri, “Nocturnal cough and snore detection using smartphones in
presence of multiple background-noises,” in ACM SIGCAS Conference
on Computing and Sustainable Societies (COMPASS), 2020.
[7] S. Vhaduri and C. Poellabauer, “Human factors in the design of longitu-
dinal smartphone-based wellness surveys,” in IEEE International Confer-
ence on Healthcare Informatics (ICHI), 2016.
[8] S. Vhaduri, T. Van Kessel, B. Ko, D. Wood, S. Wang, and T. Brun-
schwiler, “Nocturnal cough and snore detection in noisy environments
using smartphone-microphones,” in IEEE International Conference on
Healthcare Informatics (ICHI), 2019.
[9] S. Vhaduri and C. Poellabauer, “Design factors of longitudinal
smartphone-based health surveys,” Journal of Healthcare Informatics Re-
search, vol. 1, no. 1, pp. 52–91, 2017.
[10] M. T. Al Amin, S. Barua, S. Vhaduri, and A. Rahman, “Load aware
broadcast in mobile ad hoc networks,” in IEEE International Conference
on Communications (ICC), 2009.
[11] “Forecasted value of the global wearable devices market,” Accessed:
February 2018. [Online]. Available: https://goo.gl/C682Rv
[12] J. Unar, W. C. Seng, and A. Abbasi, “A review of biometric technology
along with trends and prospects,” Pattern recognition, vol. 47, no. 8, pp.
2673–2688, 2014.
[13] M. Guerar, L. Verderame, A. Merlo, F. Palmieri, M. Migliardi, and L. Val-
lerini, “Circlepin: A novel authentication mechanism for smartwatches to
prevent unauthorized access to iot devices,” ACM Transactions on Cyber-
Physical Systems, vol. 4, no. 3, pp. 1–19, 2020.
[14] S. Vhaduri and C. Poellabauer, “Multi-Modal Biometric-Based Implicit
VOLUME 4, 2016
Authentication of Wearable Device Users,” IEEE Transactions on Infor-
mation Forensics and Security, vol. 14, no. 12, pp. 3116–3125, 2019.
[15] “New lastpass research finds password habits remain key obstacle
to business’ security,” Accessed: June 2020. [Online]. Available:
https://rb.gy/edavus
[16] “Mcafee research finds troubling use of insecure cloud passwords,”
Accessed: June 2020. [Online]. Available: https://rb.gy/7fnde8
[17] T. Nguyen and N. Memon, “Smartwatches locking methods: A compara-
tive study,” in Symposium on Usable Privacy and Security, 2017.
[18] A. Dantcheva, C. Velardo, A. D’angelo, and J.-L. Dugelay, “Bag of soft
biometrics for person identification,” Multimedia Tools and Applications,
vol. 51, no. 2, pp. 739–777, 2011.
[19] C.-l. Tisse, L. Martin, L. Torres, M. Robert et al., “Person identification
technique using human iris recognition,” in Vision Interface, 2002, pp.
294–299.
[20] N. O’Hare and A. F. Smeaton, “Context-aware person identification in
personal photo collections,” IEEE Transactions on Multimedia, vol. 11,
no. 2, pp. 220–228, 2009.
[21] M.-F. Balcan, A. Blum, P. P. Choi, J. D. Lafferty, B. Pantano, M. R.
Rwebangira, and X. Zhu, “Person identification in webcam images: An
application of semi-supervised learning,” 2005.
[22] E. Corvee, F. Bremond, M. Thonnat et al., “Person re-identification using
spatial covariance regions of human body parts,” in Advanced Video
and Signal Based Surveillance (AVSS), 2010 Seventh IEEE International
Conference on. IEEE, 2010, pp. 435–440.
[23] C. BenAbdelkader, R. Cutler, and L. Davis, “Stride and cadence as a
biometric in automatic person identification and verification,” in Au-
tomatic Face and Gesture Recognition, 2002. Proceedings. Fifth IEEE
International Conference on. IEEE, 2002, pp. 372–377.
[24] ——, “Person identification using automatic height and stride estimation,”
in Pattern Recognition, 2002. Proceedings. 16th International Conference
on, vol. 4. IEEE, 2002, pp. 377–380.
[25] E. Camlikaya, A. Kholmatov, and B. Yanikoglu, “Multi-biometric tem-
plates using fingerprint and voice,” in Biometric Technology for Human
Identification V, vol. 6944. International Society for Optics and Photon-
ics, 2008, p. 69440I.
[26] M. D. Bugdol and A. W. Mitas, “Multimodal biometric system combining
ecg and sound signals,” Pattern Recognition Letters, vol. 38, pp. 107–112,
2014.
[27] R. Brunelli and D. Falavigna, “Person identification using multiple cues,”
IEEE transactions on pattern analysis and machine intelligence, vol. 17,
no. 10, pp. 955–966, 1995.
[28] T. J. Hazen, E. Weinstein, and A. Park, “Towards robust person recognition
on handheld devices using face and speaker identification technologies,” in
Proceedings of the 5th international conference on Multimodal interfaces.
ACM, 2003, pp. 289–292.
[29] J. Chauhan, Y. Hu, S. Seneviratne et al., “Breathprint: Breathing acoustics-
based user authentication,” in ACM Mobile Systems, Applications, and
Services, 2017.
[30] S. V. Dibbo, W. Cheung, and S. Vhaduri, “On-Phone CNN Model-based
Implicit Authentication to Secure IoT Wearables,” in EAI International
Conference on Safety and Security in Internet of Things (SaSeIoT), 2021.
[31] A. Acar, H. Aksu, A. S. Uluagac et al., “Waca: Wearable-assisted continu-
ous authentication,” arXiv preprint arXiv:1802.10417, 2018.
[32] H. Crawford and E. Ahmadzadeh, “Authentication on the go: Assessing
the effect of movement on mobile device keystroke dynamics,” in Thir-
teenth Symposium on Usable Privacy and Security (SOUPS). USENIX
{Association}, 2017, pp. 163–173.
[33] K. Revett, “A bioinformatics based approach to user authentication via
keystroke dynamics,” International Journal of Control, Automation and
Systems, vol. 7, no. 1, pp. 7–15, 2009.
[34] A. Bianchi and I. Oakley, “Wearable authentication: Trends and opportu-
nities,” it-Information Technology, vol. 58, no. 5, pp. 255–262, 2016.
[35] S. Vhaduri and C. Poellabauer, “Wearable device user authentication using
physiological and behavioral metrics,” in IEEE International Symposium
on Personal, Indoor, and Mobile Radio Communications (PIMRC), 2017.
[36] “Apple watch ekg not as accurate for younger people, physician says,”
Accessed: January 2020. [Online]. Available: shorturl.at/cnwS7
[37] V. Kapoor, R. Singh, R. Reddy, and P. Churi, “Privacy issues in wearable
technology: An intrinsic review,” Available at SSRN 3566918, 2020.
[38] F. Sun, C. Mao, X. Fan, and Y. Li, “Accelerometer-based speed-adaptive
gait authentication method for wearable iot devices,” IEEE Internet of
Things Journal, vol. 6, no. 1, pp. 820–830, 2018.
11

[39] A. K. Baughman, C. J. Dawson, B. M. Graham, and D. J. Kamalsky,
“System and method for virtual world biometric analytics through the
use of a multimodal biometric analytic wallet,” Jun. 9 2020, uS Patent
10,679,749.
[40] D. Kumari, U. Sharma et al., “A novel approach for secure multimodal
biometric system using multiple biometric traits,” IJRAR-International
Journal of Research and Analytical Reviews (IJRAR), vol. 7, no. 1, pp.
62–67, 2020.
[41] M. Ghayoumi, “A review of multimodal biometric systems: Fusion meth-
ods and their applications,” in IEEE/ACIS Computer and Information
Science (ICIS), 2015.
[42] Y. Liang, S. Samtani, B. Guo, and Z. Yu, “Behavioral biometrics for con-
tinuous authentication in the internet of things era: An artificial intelligence
perspective,” IEEE Internet of Things Journal, 2020.
[43] N. Al-Naffakh, N. Clarke, F. Li et al., “Unobtrusive gait recognition using
smartwatches,” BIOSIG, 2017.
[44] G. Cola, M. Avvenuti, F. Musso et al., “Gait-based authentication using a
wrist-worn device,” in ACM Mobile and Ubiquitous Systems: Computing,
Networking and Services, 2016.
[45] A. H. Johnston and G. M. Weiss, “Smartwatch-based biometric gait recog-
nition,” in IEEE Biometrics Theory, Applications and Systems (BTAS),
2015.
[46] Y. Zeng, A. Pande, J. Zhu et al., “Wearia: Wearable device implicit au-
thentication based on activity information,” in IEEE A World of Wireless,
Mobile and Multimedia Networks (WoWMoM), 2017.
[47] S. Davidson, D. Smith, C. Yang et al., “Smartwatch user identification
as a means of authentication,” Department of Computer Science and
Engineering Std, 2016.
[48] Y. Li, H. Hu, and G. Zhou, “Using data augmentation in continuous
authentication on smartphones,” IEEE Internet of Things Journal, vol. 6,
no. 1, pp. 628–640, 2018.
[49] N. Karimian, M. Tehranipoor, and D. Forte, “Non-fiducial ppg-based
authentication for healthcare application,” in IEEE Biomedical & Health
Informatics (BHI), 2017.
[50] C. Cornelius, J. Sorber, R. A. Peterson et al., “Who wears me?
bioimpedance as a passive biometric.” in HealthSec, 2012.
[51] D. Ekiz, Y. S. Can, Y. C. Dardagan, and C. Ersoy, “Can a smartband be
used for continuous implicit authentication in real life,” IEEE Access,
vol. 8, pp. 59 402–59 411, 2020.
[52] A. Muratyan, W. Cheung, S. V. Dibbo, and S. Vhaduri, “Opportunistic
Multi-Modal User Authentication for Health-Tracking IoT Wearables,” in
EAI International Conference on Safety and Security in Internet of Things
(SaSeIoT), 2021.
[53] W. Cheung and S. Vhaduri, “Context-Dependent Implicit Authentication
for Wearable Device Users,” in IEEE International Symposium on Per-
sonal, Indoor, and Mobile Radio Communications (PIMRC), 2020.
[54] ——, “Continuous Authentication of Wearable Device Users from Heart
Rate, Gait, and Breathing Data,” in IEEE RAS & EMBS International Con-
ference on Biomedical Robotics and Biomechatronics (BioRob), 2020.
[55] S. Vhaduri and C. Poellabauer, “Cooperative discovery of personal places
from location traces,” in International Conference on Computer Commu-
nication and Networks (ICCCN), 2016.
[56] S. Vhaduri, C. Poellabauer, A. Striegel, O. Lizardo, and D. Hachen,
“Discovering places of interest using sensor data from smartphones and
wearables,” in IEEE Ubiquitous Intelligence & Computing (UIC), 2017.
[57] S. Vhaduri and C. Poellabauer, “Towards reliable wearable-user identifi-
cation,” in 2017 IEEE International Conference on Healthcare Informatics
(ICHI), 2017.
[58] C.-Y. Chen, S. Vhaduri, and C. Poellabauer, “Estimating sleep duration
from temporal factors, daily activities, and smartphone use,” in IEEE
Computer Society Computers, Software, and Applications Conference
(COMPSAC), 2020.
[59] S. Vhaduri and C. Poellabauer, “Biometric-based wearable user authenti-
cation during sedentary and non-sedentary periods,” International Work-
shop on Security and Privacy for the Internet-of-Things (IoTSec), 2018.
[60] Y. Kim, S. Vhaduri et al., “Understanding College Students’ Phone Call
Behaviors Towards a Sustainable Mobile Health and Wellbeing Solution,”
in International Conference on Systems Engineering, 2020.
[61] S. Vhaduri and C. Poellabauer, “Opportunistic discovery of personal
places using multi-source sensor data,” IEEE Transactions on Big Data,
vol. 7, no. 2, pp. 383–396, 2021.
[62] S. V. Dibbo, Y. Kim et al., “Visualizing College Students’ Geo-Temporal
Context-Varying Significant Phone Call Patterns,” in IEEE International
Conference on Healthcare Informatics (ICHI), 2021.
12
Vhaduri et al.: HIAuth
[63] S. Vhaduri and C. Poellabauer, “Opportunistic discovery of personal
places using smartphone and fitness tracker data,” in IEEE International
Conference on Healthcare Informatics (ICHI), 2018.
[64] S. Vhaduri, S. V. Dibbo, C.-Y. Chen, and C. Poellabauer, “Predicting
Next Call Duration: A Future Direction to Promote Mental Health in the
Age of Lockdown,” in IEEE Computer Society Computers, Software, and
Applications Conference (COMPSAC), 2021.
[65] S. Vhaduri and C. Poellabauer, “Hierarchical cooperative discovery of
personal places from location traces,” IEEE Transactions on Mobile Com-
puting, vol. 17, no. 8, pp. 1865–1878, 2018.
[66] S. Vhaduri, S. V. Dibbo, and Y. Kim, “Deriving College Students’ Phone
Call Patterns to Improve Student Life,” IEEE Access, vol. 9, pp. 96 453–
96 465, 2021.
[67] “Esc-50: Dataset for environmental sound classification,” Accessed:
November 2019. [Online]. Available: https://bit.ly/2uT9Ddc
[68] B. McFee and S. Balke, “Librosa: Python tools for music and audio
analysis.” [Online]. Available: https://github.com/librosa
[69] A. Malek, “Spafe: Simplified python audio-features extraction.” [Online].
Available: https://github.com/SuperKogito/spafe
SUDIP VHADURI is an assistant professor in
the department of computer and information tech-
nology at Purdue University. Before that he has
been working as an assistant professor in the de-
partment of computer and information science at
Fordham University. He received the B.Sc. de-
gree in computer science and engineering from
Bangladesh University of Engineering and Tech-
nology, Bangladesh, the M.Sc. degree in com-
puter science from University of Memphis, and the
Ph.D. degree in computer science and engineering at the University of Notre
Dame. His research interests include mobile and wearable computing, user
authentication, mobile health, artificial intelligence, and machine learning.
He is a member of the IEEE and the IEEE Computer Society.
SAYANTON VHADURI DIBBO received the
B.Sc. degree in Computer Science and Engineer-
ing from University of Dhaka, and the M.Sc.
degree in Computer Science from the University
of California, Riverside. He is currently pursuing
the Ph.D. degree in Computer Science from the
Dartmouth College. His research interests include
but not limited to machine learning, mobile health,
and deep learning.
WILLIAM CHEUNG received a Bachelor of Sci-
ence in Applied Mathematics and Economics at
Stony Brook University. He has received a Master
of Science in Data Science at Fordham Univer-
sity. His research interests include IoT wearable
security, point of interest tracking, and machine
learning.
VOLUME 4, 2016