2018 17th IEEE International Conference On Trust, Security And Privacy In Computing And Communications/ 12th
IEEE International Conference On Big Data Science And Engineering
Continuous Authentication System Using Online Activities
Hidehito Gomi, Shuji Yamaguchi, Kota Tsubouchi and Naomi Sasaya
Yahoo Japan Corporation
Email: {hgomi, shyamagu, ktsubouc, nasasaya}@yahoo-corp.jp
Abstract—This paper poses the question, “Is it possible to
identify users with just a set of Web activity logs?” The answer
is yes: we can provide a continuous authentication system that
does not require explicit actions by the users while monitoring
their interactions regarding what they do for services as well
as how they use their devices. We propose an activity-based
authentication (ABA) system for active authentication that con-
tinuously verifies the identity of a user accessing multiple online
services by means of their activity histories. ABA involves a
machine-learning technique for authentication with a bagging-
data-summarization approach, as it is difficult to identify users
by using small logs. We assessed the performance and effect
of various activity features extracted from the activity logs of
1,000 users of commercially deployed Web sites. Our findings
provide valuable insights to guide the development of an
authentication system utilizing the online activities of users.
Index Terms—Continuous Authentication, Activity-based Au-
thentication, Behavioral Features
1. Introduction
Continuous authentication on computers and mobile
devices is being extensively investigated. Behavioral bio-
metric information such as key strokes, touch gestures,
and mouse movements are continuously collected through
sensor devices and are used to analyze a user’s behavioral
characteristics for authentication [1], [2], [3]. Because mo-
bile and Internet-of-Things (IoT) devices are widespread
nowadays, they can gather information pertaining to user
contexts (such as geolocation) more accurately than before
without explicit operations regardless of whether users are
online or offline [4]. This environment changes the nature
of authentication.
Behavioral information includes users’ activity histories
for online services. For example, queries provided by users
when using a search service accurately represent their in-
terests because they are not assumed to be made public,
unlike e-mails and social networking services [5]. Some
search engine providers infer a user’s intentions from the
semantics of the query and present an appropriate adver-
tisement. In another example, a user’s history of online
shopping reflects the behavioral features of his/her daily
life. Many online shopping sites now offer recommendations
for users on the basis of their preferences identified using
2324-9013/18/31.00 ©2018 IEEE 522
DOI 10.1109/TrustCom/BigDataSE.2018.00080
Authorized licensed use limited to: Bodleian Libraries of the University of Oxford. Downloaded on December 16,2022 at 18:34:43 UTC from IEEE Xplore. Restrictions apply.
the tendencies obtained from the analysis of their shopping
histories. Although the service providers in the above cases
do not identify a specific person, the analyzed interests and
intentions, which correspond to a user’s private information,
may potentially authenticate him/her with a high degree of
accuracy.
Activity-based authentication (ABA) has advantages
over knowledge-based authentication in that activity histo-
ries are not easily copied or stolen and special credentials are
not necessary [6]. In addition, they can be collected and used
for authentication through existing interfaces when a user
accesses the corresponding services without special hard-
ware or actions specific to authentication. They represent the
characteristics and interests of users that cannot be obtained
from behavioral biometrics, but are implicitly reflected by
their activities, such as transactions and interactions with
Web-based systems in their daily lives.
From the viewpoint of security, only a few attempts at
ABA have been made so far in the research community.
Previous studies on ABA investigated a few types of behav-
iors on a single device for a small number of subjects [7],
[8]. In these studies, they installed a specific application
on mobile devices for collecting specific user operations,
such as touching a screen and typing on a keyboard, so
the types of user actions were limited. Additionally, their
data did not directly link with user preferences and interests
because the data originated from the operational behavior
of the users. New light needs to be shed on higher level
activities from a user’s multiple devices and applications,
since more activities are now fused with context information
that continuously originates from mobile and IoT devices.
The purpose of this paper is to investigate the possibility
of ABA to extract users’ identity by using their activities
from multiple devices on a Web-based system from a se-
curity perspective. We focus on active authentication by
utilizing user histories generated by accessing large-scale
and Web-based services. Our core idea is to continuously
capture users’ activities from their multiple devices in a
way that reflects their identities enough to individually
authenticate them. This study is the first attempt in the
world to explore ABA on a real commercial Web-based
system that has already been deployed on a large scale to
the best of our knowledge. Since a single activity of a user
is generally transient, extracting a user’s persistent features
from a limited number of activities has been a challenge for
ABA.
Our contributions in this work are i) proposing an ABA
 system for profiling individual user activity histories to
authenticate the user as utilizing a specific type of data unit,
ii) conducting test experiments on ABA using the activity
history dataset on large-scale and commercial Web sites, and
iii) exploring the system’s feasibility by calculating several
performance measures regarding the accuracy, false accep-
tance rate (FAR), and false rejection rate (FRR) towards its
adoption as one of multiple modalities for authentication.
The rest of this paper is organized as follows: In Sec-
tion 2, we present related work on authentication. In Sec-
tion 3, we describe our ABA system for authenticating a user
on the basis of online activities for Web-based systems. The
experiments we conducted to evaluate our proposed system
and the data we used are presented in Section 4. In Section 5,
we discuss the experimental results. In Section 6, we discuss
the limitations and issues. We conclude in Section 7 with a
brief summary and mention of future work.
2. Related Work
In general, current biometric user authentication systems
can be categorized as physiological or behavioral [9].
Physiological Biometric Authentication. Systems
for authenticating users by utilizing biometrics have
been extensively investigated. Biometric authentication
has features such as passwords that are advantageous
over knowledge-based authentication in that biometric
information is difficult to copy or steal. There has been
extensive research on authentication using physiological
biometrics such as fingerprints [10], irises [11], and
faces [12].
Behavioral Authentication. Behavioral traits are used to
authenticate users by continuously collecting actions and
movements on their devices [6]. Research on such traits
includes gait [13], keystroke [14], and screen touch [15],
[16].
This approach has been expanded to include
implicit authentication [4], [17], [18] and continuous
authentication [2]. Alzubaidi et al. [19] proposed a system
for authenticating smartphone users by using behavioral
biometrics. Krombholz et al. [20] proposed integrating
pressure-sensitive touchscreen interactions into knowledge-
based authentication schemes.
Context-Aware Authentication. Hayashi et al. [21]
developed a context-aware authentication system, Karapanos
et al. [22] proposed a sound-proof authentication system that
leverages ambient sound, Wójtowicz et al. [23] proposed
a context-based biometric authentication model for mobile
devices, Preuveneers et al. [24] proposed a non-intrusive
authentication scheme by evaluating fingerprints generated
from information dynamically collected from a user’s
context, Truong et al. [25] proposed a zero-interaction
authentication solution that fuses different sensor modalities
such as audio, WiFi, Bluetooth, and GPS for avoiding
replay attacks, Holz et al. [26] proposed a biometric
523
Authorized licensed use limited to: Bodleian Libraries of the University of Oxford. Downloaded on December 16,2022 at 18:34:43 UTC from IEEE Xplore. Restrictions apply.
touch-sensing approach that associates the user’s identity
with every single touch, Brainard et al. [27] introduced the
concept of “somebody you know” for authentication (in
addition to “something you have”, “something you are”,
and “something you know”), and Peng et al. [28] proposed
a continuous authentication system that extracts behavioral
and voice features on wearable glasses. These studies
relied on how users interact with their devices by sensing
their contextual information or by monitoring movements
on their devices [29]. In contrast, we rely on what kind
of interactions the users do when accessing various Web
services with their device.
Activity-based Authentication. Fridman et al. [1]
developed an active authentication scheme on mobile
devices that fuses factors of stylometry, application usage,
Web browsing, and GPS location. Dandapat et al. [30]
developed a dynamic authentication system that mines a
user’s daily activities to extract passwords. Although these
studies are related to ours in that they attempted to extract
users’ characteristics and features from their behavioral
traits, they were conducted on a small scale and are thus
somewhat limited. Mare et al. [29] proposed an approach
for authenticating users by continuously monitoring their
hand movements via a bracelet worn on the wrist. Gomi
et al. [31] proposed an authentication method using the
logs of users’ activity histories on a large scale. These
studies are related to ours, but we go further by evaluating
authentication performance using measures, such as FFR
and FAR that have traditionally been used for biometric
authentication.
User Recognition. Our study is also related to user
recognition using behavioral traits on Web-based systems.
Wang et al. [32] proposed an approach for user recognition
across multiple devices that anonymously recognizes the
same user without identifying that user. In contrast, we
investigated user authentication using activity histories on
Web-based systems. Interestingly, these approaches for
authentication and recognition are related, although the
purposes are different.
Privacy and Anonymity. Authentication and identifying
users by sensing their behaviors may involve privacy issues
because personal activities and contextual information
surrounding a user may be collected and associated with
that user. Garvais et al. [5] discussed privacy issues
pertaining to Web search queries that reveal users’ personal
lives and the possibility of protecting users’ privacy by
obfuscating search queries because they represent personal
interests and intentions. Jessica et al. [33] argued that Web-
browsing histories can be used to clarify the linkability of
a user’s identity. Although we focus on authentication from
the viewpoint of security, we need to concern ourselves
with the possibility of privacy invasion that may be caused
by operations related to authentication.
Identity Management. Identity management standardiza-
 tion activities related to authentication have been conducted
over the past decade. The technologies on identity federation
and personal attribute exchange are widely used in many
commercial Web sites. These activities can be integrated
into a Web-based system as one approach for multi-factor
or multi-modal authentication. Fast Identity Online (FIDO)1
is developing a technical specification for authentication
using public key cryptography on a user’s client device. The
proposed system can be deployed as a FIDO authenticator
for Web-based systems.

3. Analysis for Activity-Based Authentication Figure 1. Process of creating activity bags.

We investigated users’ activities related to online ser-

vices such as Web shopping and search. A system providing 3.2. Machine Learning for Identifying a User
these services manages and analyzes log data generated from
users’ activities when they interact with the services. In this We explore mining a user’s features by utilizing activity
system, an activity of a user includes the context, namely, bags to identify one user among many in a practical fashion
where the user is located and the intention of the user for that limits computation. With this approach, we evaluate the
accessing services. feasibility and practicality of our ABA system.
We argue that identifying and authenticating users re- The process of the machine learning technique we use
quires a set of the user’s own activity logs because limited for ABA is shown in Fig. 2. Extracting features from online
pieces of actions do not represent identities very well and activities using the bag approach is specific to our study,
because other users may also take the same action. We also although the overall process is common in machine learning
argue that a set of activity logs for longer periods should by following the above procedure.
represent a user’s features and tendencies; however, the logs
may contain more redundant or obsolete activities that cause
identification to be more difficult and error-prone.
This consideration led us to develop a learning frame-
work that mitigates the above trade-off and derives a rea-
sonably up-to-date set of user activities that can distinguish
users by analyzing their features from the viewpoints of
security and practicality.

3.1. Collecting Activity Logs into Bags

We introduce an activity bag that contains a set of
activity logs of a particular user. An activity bag for a user
has two parameters: a bag period (BP) and a bag capacity
(BC). The BP is the duration (in minutes) before a specific
time during which a bag contains the corresponding actions
a user takes, and the BC is the maximum number of a user’s
actions managed in a single bag. Whenever a user takes an
action, a new bag is created for him/her.
Figure 1 shows an example of the process of creating and
managing activity bags for a user (Alice) when BP = 10
and BC = 3. Alice accesses a Web portal site at times
TA , TB , TC , and TD . At time TA (8:50), bag 1 is created
to store her access logs indicating her actions. When she
takes an action at TB , bag 2 is created to store the logs for
her actions at TA and TB , which took place within BP . In
the same manner, bag 3, which is created at TC , manages
her action logs for events C, B, and A, as these take place
during the last BP before TC . Bag 4, which is created at
TD , manages the corresponding logs for D, C, and B, while
the action for A is excluded because there are already BC
actions (D, C, and B) during the last BP before TD .
1. https://fidoalliance.org/
Figure 2. Process of machine learning for ABA.
In the training phase, a user’s action invokes the creation
of an activity bag that is associated with that user’s identifier
if he/she has already been authenticated by the system.
Activity bags containing the history logs of legitimate and
authenticated users are generated whenever they take actions
of online activities for Web-based systems. In this case, we
manage the activity bags associated with the corresponding
users. With the data set of an activity bag and the associated
user identifier, the system generates a model for each bag for
identifying the user by extracting the features that represent
that user’s behavioral characteristics. The models can be
used to determine whether an action taken by a user without
authentication corresponds to that by a legitimate user whom
the system manages.

524

Authorized licensed use limited to: Bodleian Libraries of the University of Oxford. Downloaded on December 16,2022 at 18:34:43 UTC from IEEE Xplore. Restrictions apply.
 In the identification phase, the proposed ABA system
attempts to identify a particular user on the basis of the
authentication model that has already been generated in the
training phase. When a user takes an action for accessing
a Web site, our ABA system extracts that user’s features
from the information obtained as a result of his/her access.
Our system determines whether the extracted features match
the authentication model and calculates the confidence with
which the model corresponding to the user predicts the
user’s action. The system then determines whether the user
is legitimate on the basis of a threshold for the calculated
confidence.
Although the above procedure follows a general
machine-learning approach, the bag approach used in this
study is novel because it enables ABA for large-scale Web-
based systems from a practical viewpoint.
4. Experiments
In this section, we describe the framework of the exper-
iments we conducted to evaluate the proposed ABA system.
We also describe the data and features that we used in the
experiments.
4.1. Experimental Framework
In our experiments, we used the activity histories of
1,000 users whose frequencies of actions on Yahoo! JAPAN
Web sites 2 that have already been deployed were at least
one and ranked at an average level among all the users
that were identified during the 27 days between February
1st and 27th, 2017. The sites provide more than 100 Web-
based services such as shopping, search, auction, news, and
weather reports. The activity histories were collected in
a unified manner regardless of which devices (e.g., PCs,
smartphones) and respective software (e.g., browsers, mobile
applications) were used for access. With the histories, we
created authentication models for each user by machine
learning and conducted test experiments with the models on
users’ activity logs on February 28th, which was excluded
from the data set for learning but was used for evaluation.
The collected logs included a total of 862,994 actions.
The logs for evaluation had 32,461 actions, each of which
triggers the generation of an activity bag. In this test, we
evaluated whether each bag could identify its corresponding
user from among the subject users.
For machine learning, we used logistic regression with
L1 regularization and a one-versus-the-rest classifier to solve
multi-class classification problems. We trained models by
using random sampling along with balancing because there
is class imbalance. To handle the randomness in the sam-
pling, we used a numerical result as an average value from
those obtained after ten trials. We used LIBLINEAR [34]
v1.94 for multi-label classification. We optimized the cost
parameter for logistic regression by ten-fold cross validation.
Note that our main aim was not to propose a new learning
2. https://www.yahoo.co.jp
525
Authorized licensed use limited to: Bodleian Libraries of the University of Oxford. Downloaded on December 16,2022 at 18:34:43 UTC from IEEE Xplore. Restrictions apply.
method but to investigate how the proposed system with
different parameters affects ABA performance.
4.2. Activities and Features
Table 1 lists six activity elements contained in an activity
bag: timestamp at access (TS), IP address (IP), user agent
(UA), search query string (Query), URL for user access
(URL), and location name from IP (Loc). The far right
row indicates the total number of each element in the
experiment we conducted. This number is the summation of
the appearance of an individual element without repeatedly
adding the same one. TS does not have any number because
its value changes as time passes. UA depends on the combi-
nation of the device, browser, and their versions. Note that
URL indicates a user’s specific actions for services, since
each one is identified in the same organized domain, and
action methods such as POST and GET stand for how the
user interacts with the Web-based applications. The URL
includes the meta data about the application that a user
accesses (e.g., news and auction) and the information about
a user’s interests (e.g., the identifier of shopping stores and
areas for weather reports).
From the activity elements, we extracted the five features
listed in Table 2. Actions, UAs/URLs/Locs, Web ratio, and
Duration were normalized to range from 0 to 1 in the
experiments. Elements are an element-flag vector whose size
is the summation of the total numbers of IP, UA, Query,
URL, and Loc, that is, about 30,000. Each element-flag has
a ‘1’ or ‘0’ value, where ‘1’ denotes that the target bag
has a corresponding feature and ‘0’ if not. We binarize the
flags because all elements have a discretized value such
as ‘198.1.1.1’ and ‘https://example.com’. We used these
features as standard criteria for evaluating authentication
accuracy, although a feature generated by combining these
five features might deliver a higher performance.
Figure 3 shows the histograms of features in the activity
bags listed in Table 2. These reveal the characteristics of
our data in a bag that we used for the experiments.
Figure 3a shows the relationship between the number of
actions in a bag and the frequency. The majority of bags had
several actions. The four intervals from 0 to 20 comprised
more than 90% of the total actions.
Figure 3b shows a histogram of the duration between the
period of time when the first and last actions in a bag were
taken. The majority of actions took less than one minute
under the assumption that BP was set to 10 min (600 sec).
More than 50% of durations are included in the top four
short intervals from 0 to 100 sec.
Figure 3c shows a histogram of the number of UAs in
a bag. As shown, about 89% of bags had only one device
during the BP.
Figure 3d shows a histogram of the number of URLs in
a bag. The majority of users accessed a few services during
the BP. About 81% of bags had less than five URLs.
Figure 3e shows a histogram of the number of Locs in
a bag. Almost all users accessed from the same location
within the BP. About 97% of bags had only one Loc.
 TABLE 1. ACTIVITY ELEMENTS

Element Description Example Total number

TS Timestamp at access 1496740589 -
IP IP address 183.79.135.206 17,629
UA User agent Mozilla /5.0 (iPhone; CPU ... Safari/601.1 2,064
Query Search query string “Internet dog” 3,781
URL URL of user access https://www.example.com 5,915
Loc Location name from IP San Francisco, CA, USA 780

(a) Number of actions (b) Duration

(c) UA (d) URL

(e) Location (f) Web ratio

Figure 3. Histograms of data in activity bags.

526

Authorized licensed use limited to: Bodleian Libraries of the University of Oxford. Downloaded on December 16,2022 at 18:34:43 UTC from IEEE Xplore. Restrictions apply.
 TABLE 2. F EATURES FOR BAG Table 3 shows 16 (24 ) cases of whether each activity element
Feature Description is used for learning or not, whereas other elements are used
Actions No. of actions in bag by default. The context features (TS, IP, UA, and Loc) reflect
UAs/URLs/Locs No. of types of UA, URL, and Loc in bag a user’s situation when the user takes actions on specific
Web ratio Web access ratio of actions in bag services, and the default features (Query and URL) reflect
Duration Duration of bag a user’s intentions when using the services, such as their
Elements Flag of each element (‘0’ or ‘1’)
names and specific requests.
The AAR in which they were all effective was 0.85. This
Figure 3f shows a histogram of the Web access ratio in result indicates that the combination of features, especially
a bag. The class interval whose upper limit is 1.0 represents IP and UA, improves performance because the AARs of all
that the majority of users accessed the Web sites using the results including IP and UA (in the 1st, 2nd, 9th, and
Web browsers. The class interval whose lower limit is 0.0 10th lines) was more than about 0.82.
represents accesses from mobile applications, whereas the
interval between 0.9 and 1.0 indicates accesses from Web 5.2. Effect of Action Number on Accuracy
browsers. Other intervals show accesses from both Web
browsers and mobile applications. About 90% of bags were Figure 4 shows the relationship between the number of
accessed from Web browsers. actions in a bag and the AAR when BC = 100. More
actions in a bag, which implies more frequent accesses,
lead to higher performance by using the bag approach. The
5. Experimental Results and Evaluation effectiveness of this approach is demonstrated by the fact
that another experiment without using the bag approach
In this section, we present our analysis of authentica- showed an AAR of 0.82.
tion measures for ABA and then evaluate the calculated
performance on the basis of the results obtained from the
experiments with different parameters.

5.1. Features and Accuracy

In our experiments, we derived the authentication accu-

racy of ABA as follows. We assumed that our proposed
authentication system manages N numbers of legitimate
users and that authentication model mj had been generated
from the activity histories of user uj (1 ≤ j ≤ N ). We
(k)
also let c(ai , mj ) (1 ≤ i, j ≤ N ) denote the confidence
(k)
with which mj for uj predicts action ai of ui , where
it is the k th action among K actions taken by all users
during the specific time duration. Then, we define δ (k) as the
indicator of whether ui is accurately identified by verifying
whether the features obtained from ui match with those of
the authentication models for each user uj (1 ≤ j ≤ N ), as
follows:
⎧
(k) (k)
⎨ 1 if c(ai , mi ) ≥ c(ai , mj )
(k)
δ = (1 ≤ ∀ i = j ≤ N ),
⎩
0 otherwise. Figure 4. Number of actions in bag vs. accuracy rate.
(k)
The above equation indicates that mi predicts ai with
the highest confidence among all those with which any other
5.3. Effect of Log Utilization on Accuracy
authentication model mj (1 ≤ j ≤ N ) for any j except i
(k)
predicts ai . Using these notations, we define AAR as the We conducted an experiment to evaluate how the number
authentication accuracy rate of ABA in the proposed system: of activity logs used affects ABA performance, as we had
K predicted that the fewer the logs representing a user, the

AAR = δ (k) /K. (1) more difficult identifying the user would be. We defined
the log-utilization ratio (LUR) as the ratio of user activities
k=1
for testing to the entire data set and prepared a set of user
On the basis of the above definition of AAR, we in- activities by randomly selecting from the entire data set
vestigated how four activity elements representing a user’s for various LURs. Figure 5 shows the relationship between
context (TS, IP, UA, and Loc) affect the AAR of ABA. the AAR and the LUR. The AAR slightly increased along

527

Authorized licensed use limited to: Bodleian Libraries of the University of Oxford. Downloaded on December 16,2022 at 18:34:43 UTC from IEEE Xplore. Restrictions apply.
 TABLE 3. F EATURES AND AAR (1: USED , 0: UNUSED )

Feature Case
TS 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0
IP 1 1 1 1 0 0 0 0 1 1 1 1 0 0 0 0
UA 1 1 0 0 1 1 0 0 1 1 0 0 1 1 0 0
Loc 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0
AAR 0.843 0.824 0.708 0.669 0.731 0.523 0.422 0.168 0.847 0.83 0.698 0.677 0.72 0.504 0.406 0.151

with increased LUR. This demonstrates that more activities

resulted in better accuracy. The accuracy rate decreased for ⎧
⎨ 1 if c(a(k) (k)
i , mi ) ≥ c(aI , mI )
LURs, as we predicted. Nevertheless, the graph shows about
δr(k) = (1 ≤ ∀i ≤ N, 1 ≤ ∃I = N α/100 ≤ N ),
a 0.6 AAR for an LUR of 10%. We argue that the rate is ⎩
still high if we bear in mind that identifying a unique user 0 otherwise.
among 1,000 was originally challenging. Therefore, we can Using the above relation, we define F RR as the FRR
reasonably conclude that the combination of a user’s activity of ABA in the proposed system:
histories can be used for authentication as a behavioral
modality. Kr

F RR = δr(k) /Kr , (2)
k=1

where Kr is the number of authentication attempts triggered

by user actions.
Similarly, we define F AR as the FAR of ABA in the
(k)
proposed system by using δa , which denotes the indicator
that a user is inaccurately identified for the k th action among
Ka actions taken by all users during a specific duration:
⎧
(k) (k)
⎪
⎨ 1 if c(ai , mj ) ≥ c(aI , mJ )
⎪
δa(k) = (1 ≤ ∀i = j ≤ N, 1 ≤ ∃I = i ≤ N,
⎪
⎪ 1 ≤ ∃J = N α/100 ≤ N ),
⎩
0 otherwise.
Using the above relation, we define F AR as the FAR
of ABA in the proposed system:
Ka

F AR = δa(k) /Ka , (3)
k=1

where Ka is the number of authentication attempts triggered

Figure 5. Log utilization ratio vs. accuracy rate.
5.4. Calculation of FRR and FAR
We calculated the performance measures of FRR and
FAR for the previous experimental results to evaluate ABA
performance and feasibility from a security perspective, as
these measures have traditionally been used for evaluating
biometric authentication systems. To do so, we define the
threshold as the rate of users that are estimated to be
accurately legitimate, whereas for ARR, we regard only a
single user for ARR who has the highest confidence for
each model as accurate, as described in Section 4.2. That
is, if a threshold is assumed to be α, the confidence of
an authenticated user is ranked as in the top I among N
confidences, where I = N α/100. In the same manner
(k)
as the notation in Eq. (1), we define δr as the indicator
specifying that user ui is accurately identified as follows:
by user actions.
Figure 6 shows the relationship between a pair of FAR
and FRR (y-axis) and the threshold (x-axis) for the 16
combinations of features, which are listed in Table 3. The
solid and broken lines in the graphs represent the FRR
and FAR, respectively. Each graph indicates whether four
features (TS, IP, UA, and Loc) are assumed used, while the
other features are used for every testing result by default.
The “checked” in the check-box for a feature means that it
was used in this experiment. For example, the graph on the
top left uses only the default features. Each graph shows the
equal error rate (EER), which is defined as the value of an
intersection of the FAR and FRR lines. Note that a small
EER indicates a high-performance authentication system.
The FARs in all the graphs have almost the same straight
line that linearly increases as the threshold increases because
the authenticated users were randomly selected using the
ranking of the confidence according to the definition of FAR
in Eq. (3). For the FRRs, there is a common trend that the

528

Authorized licensed use limited to: Bodleian Libraries of the University of Oxford. Downloaded on December 16,2022 at 18:34:43 UTC from IEEE Xplore. Restrictions apply.
 Figure 6. FAR and FRR vs. threshold for feature combinations.
curved lines gradually decline, as biometric authentication
systems traditionally indicate.
In these graphs, the EER varies from 0.03 to 0.12. These
values are significant for an authentication system regardless
of the combination of the four features. Graphs (1) and (9)
reveal the worst EER, 0.12. This indicates that TS does not
greatly affect ABA performance because one included TS,
whereas the other does not.
Five graphs reveal the highest EER value, 0.03. These
commonly include a UA. Therefore, UA is the most impor-
tant to obtain the highest for ABA systems, whereas UA
and IP are important features for AAR, as mentioned in
Section 4.2.
If we compare the four graphs (2), (6), (10), and (14)
and the ones (4), (8), (12), and (16), we can determine how
UA affects EER. Each comparison between the four graphs
(1), (5), (9), and (13) and the ones (3), (7), (11), and (15)
also reveals that UA improves EER.
529
Authorized licensed use limited to: Bodleian Libraries of the University of Oxford. Downloaded on December 16,2022 at 18:34:43 UTC from IEEE Xplore. Restrictions apply.
6. Discussion
In this section, we discuss several issues and limitations
of our system.
6.1. Machine-Learning Methodology
We used logistic regression with L1 norm regularization,
which enabled us to acquire a white box model with which
to validate the prepared features. However, we can apply any
machine-learning technique for multi-class classification.
Practically speaking, we are not concerned with whether
the model is interpretable but with performance such as
calculation time and classification accuracy.
Figure 7 shows individual users’ top ten significant
features that have high weight for identifying the users.
The different colors in the table signify the type of feature.
We randomly selected 30 users and colored their significant
features. As shown in the figure, the significant features
differ between users: e.g., user 1 (u1) is identified by IP
 and UA, and users 1 and 18 are similar in that most IPs
are important for identifying them. Overall, IPs are mostly
shown in Fig. 7, which provides insight into what is acquired
from the result listed in Table 3.
The continued ratio of IP, UA, URL, and Loc calculated
using the values listed in Table 3 was about ‘17:2:6:1’, while
that obtained from Fig. 7 was about ‘4:2:2:1’. UA and Loc
greatly affected authentication accuracy despite their smaller
values of the above ratio.
The most effective feature for authentication accuracy
differed between users. For example, the most effective
feature for user 1 was IP, for users 3 and user 4 was UA,
and for user 21 was URL. We assume that the difference
stemmed from the variations of users’ activities, where each
action was correlated with each other or several actions
occurred sequentially. Such relationship between the actions
in users’ activities will be studied in our future work.
6.2. Toward Adoption of ABA at Scale
Let us consider adapting ABA into an authentication
system as one modality. We have evaluated the proposed
system authenticating a unique user among 1,000 users, as
reported in Section 4. We will further discuss the authentica-
tion accuracy and the computation time of our ABA system
in terms of its scalability to support a larger number of users.
6.2.1. Dataset Maintenance. Because our system continues
to collect a large number of datasets for Web site users, it
is difficult to maintain separate models for each user when
the number of users is large (e.g., more than a million).
6.2.2. Accuracy. When the number of users increases to
10,000 or more, we anticipate that user identification will
become much more difficult and the accuracy may become
worse than in previous tests because many users will use
the same service at the same time and consequently have
similar features.
As the number of users increases in this case, we will
collect more types of features (e.g., IPs), which will help
uniquely identify a particular user with high confidence.
Thus, we expect that the deterioration in ABA accuracy
regarding the increase in users will be limited because of
the above tradeoff between the uniqueness and commonness
of increasing features.
Accuracy may deteriorate if the increase in the number
of common features is more significant than that in the
number of unique features. In this case, however, we expect
to control deterioration by introducing a two-step authenti-
cation that adopts ABA and a grouping-key technique. This
approach divides users into a specific number of groups
and assigns a group-specific key to a user that belongs to
that group. For example, if a hundred million users are
divided into one hundred groups, each group will consist
of a million users. Although this approach does not identify
a single user uniquely, ABA will be effective by decreasing
the order of target users that need to be authenticated.
530
Authorized licensed use limited to: Bodleian Libraries of the University of Oxford. Downloaded on December 16,2022 at 18:34:43 UTC from IEEE Xplore. Restrictions apply.
Again, this approach will contribute to developing a scalable
authentication system that adopts ABA.
The authentication accuracy of ABA was about 85%.
Apart from the discussion of whether the accuracy is good
or bad, we can sufficiently identify an abnormal access by
evaluating the degree of dissociation from ordinary accuracy.
For example, if today’s access has a 70% accuracy for a
user who ordinarily has a 40% accuracy, we can doubt that
the access should be unauthorized despite its high accuracy
score.
6.2.3. Computation Time. The computation time for gen-
erating authentication models of the authentication systems
increases as the number of users in the systems increases.
This ABA issue is expected to be solved by operating related
ABA tasks in parallel. In ABA, complete parallel computing
is possible because a set of ABA computations for each user,
which involves importing activity histories and generating
an authentication model for identification, is independent
of that for other users except the immediately preceding
user. In addition, generating a model for a particular user in
ABA requires his/her positive examples as well as the same
number of his negative ones, even if users and the quantity
of the corresponding data increase. Randomly extracting
negative examples prior to model generation will not cause
a drastic overhead in its computation as the number of users
increases.
Although the procedure for extracting the top α%
rankers of all users, such as sorting the scores of many
users (e.g., in an order of one hundred million), takes a
lot of time, the computation time for identifying a user
will not dramatically increase if we use only the data
corresponding to the users that are ranked in the top α%. We
can easily avoid computation explosion by using engineering
techniques when our ABA system is enabled.
6.3. Bag Approach
We took the bag approach that uses a special type of
unit to manage users’ activity data for generating ABA
models on a large-scale Web-based system by means of
machine learning. From the experimental results reported
in Section 5.3, we found that this bag approach is effective.
However, the bag essentially contains the latest activities
that are limited by the length of the BP for a specific dura-
tion using the authentication models, which were generated
for about one month during the experiment. Thus, we are
interested in finding out how the result will change if we
conduct learning for longer periods.
6.4. Cold-Start Problem
If the training dataset for a new user is difficult to obtain,
or if the collected dataset is still insufficient or limited,
our system cannot draw appropriate inferences for the user;
therefore, user authentication may be unsuccessful. This
issue is unavoidable because our system uses a data-driven
approach.
 Figure 7. Top 10 significant features used in learning model.
6.5. Static Data
We conducted an experiment and observed a positive
performance in terms of authentication accuracy, especially
for IP and UA, as reported in Section 4. However, it is
possible that such features are likely to be static, making
identification of a single user easy. We will further investi-
gate this as future work.
6.6. Privacy and Activities on the Web
We conducted experiments on ABA, that is, for authen-
tication, from the viewpoint of security, where the proposed
system identifies a particular user. Our system discovers a
user’s personal interests and intentions that were extracted
by analyzing the user’s online activities in his/her daily life
and consequently identifies a legitimate user. Thus, we need
to be concerned with users’ privacy, since the identification
process tracks their activities and obtains their private infor-
mation, which they are not aware is being tracked. Although
our experiments were conducted in accordance with the
privacy policies of the commercial sites, we need to make
sure users understand the benefits of ABA and the privacy
aspects of how to deal with extracted personal information in
order to deploy our ABA system for large-scale Web-based
systems. We will explore the tradeoff between the privacy
involved with being tracked and the security necessary for
strong authentication as future work.
7. Conclusion and Future Work
We proposed an ABA system and conducted experi-
ments by using a bag approach involving activity histories
for large-scale commercial Web sites. Additionally, we also
analyzed performance measures such as authentication ac-
curacy, false rejection rate, and false acceptance rate.
The results show that our ABA system has an AAR of
more than 0.8 if a reasonable number of activities are stored
and an EER of about 0.03 if the features extracted from a
user’s activities include IP and UA. We conclude that these
are significant results to advance the deployment of ABA
for large-scale Web-based systems.
531
Authorized licensed use limited to: Bodleian Libraries of the University of Oxford. Downloaded on December 16,2022 at 18:34:43 UTC from IEEE Xplore. Restrictions apply.
Future work includes developing a framework for be-
havioral authentication by introducing ABA as one of the
modalities for authentication and examining challenges re-
garding privacy related to ABA.
References
[1] L. Fridman, S. Weber, R. Greenstadt, and M. Kam, “Active Authen-
tication on Mobile Devices via Stylometry, Application Usage, Web
Browsing, and GPS Location,” the IEEE Systems Journal, pp. 1–9,
2016.
[2] H. Saevanee, N. Clarke, S. Furnell, and V. Biscione, “Continuous
User Authentication Using Multi-modal Biometrics,” Computers &
Security, vol. 53, pp. 234–246, 2015.
[3] E. Vasiete, Y. Chen, I. Char, T. Yrh, V. Patel, L. Davis, and R. Chel-
lappa, “Toward a Non-Intrusive, Physio-Behavioral Biometric for
Smartphones,” in Proceedings of the 16th International Conference
on Human-Computer Interaction with Mobile Devices and Services
(MobileHCI’14), 2014, pp. 501–506.
[4] E. Shi, Y. Niu, M. Jakobsson, and R. Chow, “Implicit Authentica-
tion Through Learning User Behavior,” in Proceedings of the 13th
International Conference on Information Security, 2011, pp. 99–113.
[5] A. Gervais, R. Shokr, A. Sing, S. Capn, and V. Lenders, “Quantifying
web-search privacy arthur,” in Proceedings of the ACM Conference on
Computer and Communications Security (CCS’14), 2014, pp. 966–
977.
[6] K. Bailey, J. Okolica, and G. Peterson, “User Identification and Au-
thentication Using Multi-modal Behavioral Biometrics,” Computers
& Security, vol. 43, pp. 77–89, June 2014.
[7] M. Abramson and S. Gore, “Associative patterns of web browsing
behavior,” in AAAI Fall Symposium Series, 2013.
[8] M. Abramson and D. W. Aha, “User authentication from web brows-
ing behavior,” in Proceedings of the 26th International Florida Ar-
tificial Intelligence Research Society Conference (FLAIRS’13), 2013,
pp. 268–273.
[9] W. Meng, D. S. Wong, S. Furnell, and J. Zhou, “Surveying the
Development of Biometric User Authentication on Mobile Phones,”
IEEE Communications Surveys and Tutorials, vol. 17, no. 3, pp.
1268–1293, 2015.
[10] Y. Javed, M. Shehab, and E. Bello-Ogunu, “Investigating User Com-
prehension and Risk Perception of Apple’s Touch ID Technology,”
in Proceedings of the 12th International Conference on Availability,
Reliability and Security (ARES’17), 2017, pp. 1–6.
[11] P. Loya and A. R. Pais, “SIRIS - Secure IRIS Authentication System,”
in Proceedings of the Fifth International Conference on Security of
Information and Networks, no. 7204857117, 2012, pp. 148–152.
 [12] A. Ouamane, M. Belahcene, A. Benakcha, S. Bourennane, and [29] S. Mare, A. Molina-Markham, C. Cornelius, R. Peterson, and D. Kotz,
A. Taleb-Ahmed, “Robust Multimodal 2D and 3D Face Authenti- “ZEBRA: Zero-Effort Bilateral Recurring Authentication,” in Pro-
cation Using Local Feature Fusion,” Signal, Image and Video Pro- ceedings of IEEE Symposium on Security and Privacy (S&P’14),
cessing, vol. 10, no. 1, pp. 129–137, 2016. 2014, pp. 705–720.
[13] W. Xu, Y. Shen, Y. Zhang, N. Bergmann, and W. Hu, “Gait-Watch: A [30] S. K. Dandapat, S. Pradhan, B. Mitra, R. R. Choudhury, and N. Gan-
Context-aware Authentication System for Smart Watch Based on Gait guly, “ActivPass: Your Daily Activity is Your Password,” in Proceed-
Recognition,” in Proceedings of the Second International Conference ings of the ACM Conference on Human Factors in Computing Systems
on Internet-of-Things Design and Implementation, 2017, pp. 59–70. (CHI’15), vol. 1, 2015, pp. 2325–2334.
[14] D. Buschek, A. De Luca, and F. Alt, “Improving Accuracy, Applica- [31] H. Gomi, S. Yamaguchi, K. Tsubouchi, and N. Sasaya, “Towards
bility and Usability of Keystroke Biometrics on Mobile Touchscreen Authentication Using Multi-modal Online Activities,” in Proceedings
Devices,” in Proceedings of the 33rd Annual ACM Conference on of the International Joint Conference on Pervasive and Ubiquitous
Human Factors in Computing Systems (CHI’15), 2015, pp. 1393– Computing (UbiComp’17), 2017, pp. 37–40.
1402.
[32] X. Wang, T. Yu, M. Zeng, and P. Tague, “XRec: Behavior-Based
[15] T. Feng, J. Yang, Z. Yan, E. Tapia, and W. Shi, “TIPS: Context-aware User Recognition Across Mobile Devices,” Proceedings of the ACM
Implicit User Identification Using Touch Screen in Uncontrolled on Interactive, Mobile, Wearable and Ubiquitous Technologies, vol. 1,
Environments,” in Proceedings of the 15th Workshop on Mobile no. 3, pp. 1–26, 2017.
Computing Systems and Applications, 2014, pp. 9:1—-9:6.
[33] J. Su, A. Shukla, S. Goel, and A. Narayanan, “De-anonymizing Web
[16] S. Budulan, E. Burceanu, T. Rebedea, and C. Chiru, “Continuous Browsing Data with Social Networks,” in Proceedings of the 26th
User Authentication Using Machine Learning on Touch Dynamics,” International Conference on World Wide Web (WWW’17), 2017, pp.
Neural Information Processing, pp. 591–598, November 2015. 1261–1269.
[17] M. Jakobsson, E. Shi, P. Golle, and R. Chow, “Implicit Authentication [34] R.-E. Fan, K.-W. Chang, C.-J. Hsieh, X.-R. Wang, and C.-J. Lin, “Li-
for Mobile Devices,” in Proceedings of the 4th USENIX Conference blinear: A library for large linear classification,” Journal of Machine
on Hot Topics in Security (HotSec’09), 2009, p. 9. Learning Research, vol. 9, pp. 1871–1874, 2008.
[18] H. Khan, U. Hengartner, and D. Vogel, “Usability and Security Per-
ceptions of Implicit Authentication: Convenient, Secure, Sometimes
Annoying,” in Proceedings of the 11th Symposium On Usable Privacy
and Security (SOUPS’15), 2015, pp. 225–239.
[19] A. Alzubaidi and J. Kalita, “Authentication of Smartphone Users Us-
ing Behavioral Biometrics,” IEEE Communications Surveys Tutorials,
vol. 18, no. 3, pp. 1998–2026, 2016.
[20] K. Krombholz, T. Hupperich, and T. Holz, “Use the Force: Evaluating
Force-Sensitive Authentication for Mobile Devices,” in Proceedings
of the Symposium On Usable Privacy and Security (SOUPS’16),
2016, pp. 207–219.
[21] E. Hayashi, J. Hong, S. Das, S. Amini, and I. Oakley, “CASA:
Context-Aware Scalable Authentication,” in Proceedings of the Sym-
posium On Usable Privacy and Security (SOUPS’13), 2013, pp. 1–10.
[22] N. Karapanos, C. Marforio, C. Soriente, and S. Capkun, “Sound-
Proof: Usable Two-Factor Authentication Based on Ambient Sound,”
in Proceedings of 24th USENIX Security Symposium (USENIX Secu-
rity’15), 2015, pp. 483–498.
[23] A. Wójtowicz and K. Joachimiak, “Model for Adaptable Context-
based Biometric Authentication for Mobile Devices,” Personal and
Ubiquitous Computing, vol. 20, no. 2, pp. 195–207, 2016.
[24] D. Preuveneers and W. Joosen, “SmartAuth: Dynamic Context Fin-
gerprinting for Continuous User Authentication,” in Proceedings of
the 30th Annual ACM Symposium on Applied Computing, 2015, pp.
2185–2191.
[25] H. T. T. Truong, X. Gao, B. Shrestha, N. Saxena, N. Asokan,
and P. Nurmi, “Comparing and Fusing Different Sensor Modalities
for Relay Attack Resistance in Zero-Interaction Authentication,” in
2014 IEEE International Conference on Pervasive Computing and
Communications (PerCom’14), 2014, pp. 163–171.
[26] C. Holz and M. Knaust, “Biometric Touch Sensing: Seamlessly Aug-
menting Each Touch with Continuous Authentication,” in Proceedings
of the 28th Annual ACM Symposium on User Interface Software &
Technology (UIST’15), 2015, pp. 303–312.
[27] J. Brainard, A. Juels, R. L. Rivest, M. Szydlo, and M. Yung, “Fourth-
Factor Authentication: Somebody You Know,” in Proceedings of
the ACM Conference on Computer and Communications Security
(CCS’06), 2006, pp. 168–178.
[28] G. Peng, G. Zhou, D. T. Nguyen, X. Qi, Q. Yang, and S. Wang,
“Continuous Authentication With Touch Behavioral Biometrics and
Voice on Wearable Glasses,” IEEE Transactions on Human-Machine
Systems, vol. 47, no. 3, pp. 404–416, 2017.

532

Authorized licensed use limited to: Bodleian Libraries of the University of Oxford. Downloaded on December 16,2022 at 18:34:43 UTC from IEEE Xplore. Restrictions apply.