Journal Pre-proof

Developing a mobile epilepsy management application integrated with an

electronic health record for effective seizure management

Sooyoung Yoo, Kahyun Lim, Hyunyoung Baek, Sae-Kyun Jang,

Ga-young Hwang, Hunmin Kim, Hee Hwang

PII: S1386-5056(19)30745-2
DOI: https://doi.org/10.1016/j.ijmedinf.2019.104051
Reference: IJB 104051

To appear in: International Journal of Medical Informatics

Received Date: 9 July 2019

Revised Date: 25 November 2019
Accepted Date: 3 December 2019

Please cite this article as: Yoo S, Lim K, Baek H, Jang S-Kyun, Ga-young H, Kim H, Hwang H,
Developing a mobile epilepsy management application integrated with an electronic health
record for effective seizure management, International Journal of Medical Informatics (2019),
doi: https://doi.org/10.1016/j.ijmedinf.2019.104051

This is a PDF file of an article that has undergone enhancements after acceptance, such as
the addition of a cover page and metadata, and formatting for readability, but it is not yet the
definitive version of record. This version will undergo additional copyediting, typesetting and
review before it is published in its final form, but we are providing this version to give early
visibility of the article. Please note that, during the production process, errors may be
discovered which could affect the content, and all legal disclaimers that apply to the journal
pertain.

© 2019 Published by Elsevier.

Developing a mobile epilepsy management application integrated with

an electronic health record for effective seizure management

Sooyoung Yoo, PhDa*, Kahyun Lim, MSa*, Hyunyoung Baek, RN, MPHa, Sae-Kyun

Jang, MPHb, Ga-young, Hwang, MSb, Kim, MDc, Hee Hwang, MDc

of
a
Office of eHealth Research and Businesses, Seoul National University Bundang Hospital,

South Korea,

ro
b
Healthconnect Co., Ltd. South Korea,

c
-p
Department of Pediatrics, Seoul National University Bundang Hospital, South Korea
re
lP

*These authors contributed equally to this work.

Corresponding and co-corresponding authors:

na

Hee Hwang, neuroandy@snubh.org

Hunmin Kim, hunminkim@hanmail.org

ur

Address for correspondence:

Jo

Department of Pediatrics, Seoul National University Bundang Hospital, 82 Gumi-ro 173

beon-gil, Bundang-gu, Seongnam 13620, Korea. Tel: +82-031-787-7284, E-mail:
neuroandy@snubh.org

Department of Pediatrics, Seoul National University Bundang Hospital, 82 Gumi-ro 173

beon-gil, Bundang-gu, Seongnam 13620, Korea. Tel: +82-031-787-7297, E-mail:
hunminkim@hanmail.net
Highlights

 The study developed and evaluated an mHealth service for managing epilepsy

compatible with the hospital’s EHR system.

 Functions such as seizure Diary, Medication Reminder, Appointments, Outpatient

Survey, and My Epilepsy were developed to aid patients’ epilepsy management.

of
 The usability test showed that participants were highly satisfied with the service

and there were differences between participants in scenario-based task success

ro
rate.


-p
The degree of perceived importance of the functions differed by participants’

characteristics.
re
lP

Keywords: Epilepsy, Seizure, Medication, Mobile application, Electronic medical record

na

Abstract1
ur

Introduction: Epilepsy is a chronic neurological disorder characterized by recurrent

spontaneous seizures. Over 70% epilepsy patients can live normally if their seizures can be
Jo

controlled. For this, many factors should be tracked and managed, but doing so is hard

because of individual differences. There are mobile applications to help track these factors;

1
TF: task force; GUI: graphic user interface; UI: user interface; EHR: electronic health record; UX: user
experience; FHIR: Fast Healthcare Interoperability Resources; SUS: System Usability Scale
however, no application covers crucial factors comprehensively, and they are complicated to

use. Therefore, this study aimed to develop a mobile epilepsy management application

covering crucial factors comprehensively in a user-friendly way. We evaluated the pilot

version with a usability and satisfaction survey and an interview.

Methods: We established a task force comprising professionals from various fields who

participated in all processes of this research. Existing service analysis and professional

interviews were conducted to draw a function list. User interface and graphic user interface

of
were designed under the supervision of the task force. After developing the application’s pilot

version, usability and satisfaction of the application were evaluated with eight patients and

ro
caregivers through scenario-based usability test, satisfaction survey, and interview.

-p
Results: All existing mobile epilepsy management applications provide seizure and

medication diary functions. We decided to provide six main functions: seizure diary,
re
medication reminder, appointments, outpatient survey, education materials, and personal
lP

dashboard (My epilepsy). We also integrated the application with the hospital’s electronic

health record system. To simplify usability, frequently used and relatively important functions
na

are located in the main page as “seizure recording” and “medication diary.” Additionally,

when designing graphics, art therapy was used to enhance psychological stability. For
ur

evaluation, eight participants were recruited. In scenario-based tasks, among 10 tasks, all

participants completed six tasks. However, only 37.5% participants recorded seizures in
Jo

detail. System Usability Scale score was 84.5 points, indicating system satisfaction.

Conclusion: This study confirmed that patients’ satisfaction of this application were high.

Additionally, it helped them record their seizures accurately, which is very useful for seizure

trend analysis, discovering seizure trigger factors, and ensuring efficient management of

epilepsy. Through integration with the electronic health record, patient medical information
could be utilized to guide physicians’ decision-making for future medical treatment and could

contribute greatly to the overall management of epilepsy.

Keywords: Epilepsy, Seizure, Medication, Mobile application, Electronic medical record

Introduction

Epilepsy is a chronic neurological disorder characterized by recurrent spontaneous seizures.

of
The worldwide prevalence of epilepsy is estimated to be 0.4–1%[1]. Due to abnormal brain

activity, certain seizures cause convulsion or loss of consciousness and result in injuries. In

ro
addition, such damage can result in traumatic brain injury, broken bones, concussions, and

breathing problems [2]. -p

re
Over 70% epilepsy patients can live their lives without major problems if they have the

appropriate means to control seizures [3]. Medication is the most common treatment among
lP

numerous interventions [4]. However, it is challenging to determine the appropriate treatment

to control seizures for each patient, as the disorder is highly complex, and each patient has
na

different physiological status. To determine which medication regime is appropriate,

physicians initially suggest some medications for trial. If the first trial is effective without
ur

serious side effects, patients continue the medications. If not, physicians suggest other

medications as an alternative until they find the active ones [5]. To reduce time and ensure
Jo

efficiency, information concerning seizure status, medication side effects, dose history, and

seizure trigger factors should be tracked continuously [6]. Therefore, patients or caregivers

note these in their own way, such as on paper or on their mobile phones, to show physicians.

This is very inconvenient for the patients to record and difficult for physicians to read and

infer associations [7].

To address these issues, many mobile applications including EpApp [8], EpiWatch [9], and

Seizure Sync [10] have been developed. These provide seizure diary and medication

reminders in common. Additionally, depending on applications, various functions are

included such as educational materials, emotion diary, and automatic seizure detection. There

are many factors to be addressed for managing epilepsy, but there is no application covering

all these factors comprehensively. Moreover, there is no application integrated with the

hospital’s electronic health record (EHR) system, which is crucial for usability. This not only

of
makes it hard to communicate with the physician but also degrades usability. Puneet et al.

recently demonstrated that although the number of mobile epilepsy management applications

ro
is increasing, their functional coverage is still limited [11]. Another review of mobile epilepsy

self-management apps also noted this and suggested the need to develop mobile epilepsy
-p
management apps that provide broader domains of functionality to support self-management
re
[12]. Usability is particularly important in the design and development of health applications

to ensure their adoption and sustained use, as users tend to quickly abandon applications
lP

when facing inconvenience (e.g., problems in access and complex processes) [13]. Therefore,

a usability test is a useful evaluation method to detect fatal usability problems before
na

releasing the application [14].

Therefore, the primary objective of this study was to develop a mobile epilepsy management
ur

application that comprehensively covers crucial factors for managing epilepsy. As a

secondary goal, considering usability, we tried to enhance the application’s convenience

Jo

through a simple user interface (UI), graphic design, and integration with the hospital’s EHR

system. To draw a list of functions, interviews with professionals from various fields and

existing service analysis were conducted. Moreover, integration with the hospital EHR

system was processed to reduce user effort in typing data such as prescribed medicine and lab
results and improving communication with physicians. In addition, UI was designed as

simple as possible. Art therapy was adopted in graphic design for helping users feel

psychologically relieved. After developing the pilot version of the application, a scenario-

based usability test, satisfaction survey, and interview were conducted.

1. Methods

1.1 Study site and task force team

of
This study was conducted at Seoul National University Bundang Hospital (SNUBH), a public

tertiary general hospital located in the Seoul metropolitan area. The EHR system of the

ro
hospital was accredited by the Healthcare Information and Management System Society’s

Electronic Medical Record Adoption Model Stage 7, which indicates that the hospital has an
-p
optimal information technology infrastructure that can develop and evaluate mobile
re
applications linked to the EHR system.
lP

To design the mobile epilepsy management application, we established a multidisciplinary

task force (TF) composed of experts from various fields including neurology, nursing,

medical informatics, user experience (UX), and software development. The TF held a
na

meeting every two weeks from January 1 to December 31, 2018 and were involved in every

process from service planning to development and evaluation.

ur

1.2 Functions of mobile epilepsy management application

Jo

We investigated the existing mobile epilepsy management applications to derive a list of

functions to be adopted. Six applications were analyzed: EpiWatch [9], Alert App [15],

SeizAlarm [16], Epilepsy Tool Kit [17], EpiDiary [18], and EpApp [8]. Other than these

applications, there are other useful services such as My Seizure Diary [19] and Seizure

Tracker [20]. The selected applications were in the top six positions of the total number of
considered downloads in App Store, a mobile application distribution platform. The

application functions found were categorized and divided into seven main features: seizure

diary, automatic seizure detection, emergency call, medication (reminder), education,

image/video upload, and emotion management. With the exception of automatic seizure

detection and emergency call, all features were adopted for the application. The automatic

seizure detection function was excluded due to its low accuracy and emergency call was ruled

out due to the local regulatory issues with service linkage and provision. Educational

of
materials were produced reflecting a common opinion of physicians that most patients and

caregivers find it difficult to understand the various types of seizures. New features for

ro
checking lab test results, physical growth, and self-surveying their psychological status were

designed and provided through the integration with the hospital’s EHR system. The key
-p
functions derived by the TF for patients are as follows.
re
 Seizure diary: recording and monitoring the seizure type, trigger/aggravation factors,

seizure occurrence time, duration of the seizure, upload image/video, and memo
lP

 Medication diary: checking prescribed medications, recording dose and adverse drug
na

effects, and setting a medication reminder

 Education: 3D-animation video explaining different seizure types, physicians’

ur

answers to frequently asked questions

Jo

 Emotion management: recording daily emotion changes using emoticons

 Test results: Checking medical test results such as electrocardiogram and blood test

 Survey: providing self-survey tools such as the Children’s Depression Inventory [21],

Beck Anxiety Inventory [22], Attention Deficit Hyperactivity Disorder rating scale

[23], physical development assessment [24], quality of life survey [25], and
 outpatient examination survey

Based on the derived function list, we designed the information architecture as depicted in

Figure 1. Along with the patient mobile application, we developed a monitoring web

application for physicians linked to the EHR system.

Log In

Basic Health
Survey *Only first login user

of
Survey &
Main My Epilepsy Test results Education Setting

ro
Message Dashboard Education
Survey Materials My Info.

Connect with
Seizure Diary

Medication
Diary

Report Calendar
-p
Test Results health apps.

Notification
Setting
re
My Epilepsy Seizure Diary Privacy Policy

Survey &
Test Results Medication Version Info.
lP

Education Emotion Log Out

Sleep

Appointment
na

Figure 1. Information architecture of Brain4U for patients; Info: Information

ur

1.3 System architecture

Jo

The system architecture of Brain4U is detailed in Figure 2. As the data within the Brain4U

contains features of personal medical information, we located the system within the

demilitarized zone of the hospital. Fast Healthcare Interoperability Resources (FHIR)

standard was used for interoperability. Data in Brain4U were mapped to analogous FHIR

resources [26]. The data elements included FHIR resources such as Encounter (associated
with a particular encounter), Appointment (information concerning a planned appointment),

Medication Order (all kinds of medication orders for the patient), Observation (i.e. vital signs,

laboratory data, and imaging test results), Patient (demographic information of patient),

Practitioner (detailed description of the physician), Medication (identification and definition

of a medication - i.e. manufacturing site or injection method).

ExtraNet DMZ IntraNet

Push Alarm
-Address: android.googleapis.com
-Port: 443, 5228, 5229, 5230

of
Epilepsy Management System EHR
AP P AP I
Monitoring Web
Sleep
EHR I/F

ro
FCM I/F

HealthKit -Usage of Mobile Service Batch P rocess Retrieve Patient Inf o.

-Survey, Seizure diary -Blood test result
Smartphone WAS(Tomcat 8.x)
-Steps, Sleep -Brain MRI result
(Android, iOS)
-Address: any MariaDB -EEG test result
-P ort: 446(https) OS(Win2012)
-p
-P atient Info.
-Medication
re
Physician’s Web
Steps, Sleep data
-Retrieve patient test results
(https, json) Physicians
-Retrieve patient info from
lP

the mobile app

Figure 2. System architecture of Brain4U linked to the Electronic Health Record (EHR)

system; DMZ: demilitarized zone; MRI: Magnetic Resonance Imaging; EEG:

na

electrocardiogram; FHIR: Fast Healthcare Interoperability Resources; WAS: Web Application

Server; FCM: Firebase Cloud Messaging; API: Application Programming Interface; Info:
ur

Information
Jo

1.4 Usability test

Eight participants were recruited for the usability test. The number of participants was based

on a guideline that 6–8 people is an appropriate sample size for measuring UX in a usability

test [27]. Outpatients diagnosed with epilepsy for over 3 years and their respective caregivers

were included as potential participants for this study. Patients or their parents who were
unavailable or unable to use a smart phone and refused to provide informed consent were

excluded. The usability test was conducted in a quiet meeting room with the minimum

objects essential for the test such as desk, cameras, and camera holders. It was conducted

from November 29 to December 29, 2018.

We designed 10 scenario-based tasks using main functions as described in Table 1. Three

measures were evaluated: task success rate, time for completion, and satisfaction. To

calculate the task success rate, we defined a scoring policy. When a participant completed the

of
task immediately, we gave him 1 point for the task. If not, the participant was assigned 0

point. After finishing tests, the average score was calculated for each task to find out which

ro
task had defects in usability. Efficiency refers to the time spent by an individual for

-p
completing a task. Satisfaction evaluates user satisfaction, and we used the System Usability

Scale (SUS) [28] as it is a reliable tool for measuring the level of satisfaction.
re
We educated participants in advance regarding the “Think Aloud” method, which involves
lP

voicing every emerging idea. In the test, one participant, one moderator, and one assistant

moderator attended. When the participant entered, first, the outline of the application and the
na

usability test were explained, and we acquired a consent form representing the participant’s

consent to record and collect data. Before starting the usability test, a short survey was
ur

conducted asking demographic information and pre-expectations for the application. Finally,

the scenario-based usability test was started. The entire process was performed under the
Jo

guidance of a moderator, while the observer recorded what he/she observed. The moderators

took the role of instructing the whole process to the participants and were instructed not to

intervene while the participants were working on tasks. They could offer advice once the

participant completely gave up on the task. The entire process was recorded in three ways: a

smartphone screen recording, participant’s facial expression recording, and physical motion
recording. After completing all the tasks, a short interview inquiring their broad feelings and

thoughts and a SUS were conducted.

Table 1. Scenario-based tasks for evaluating usability of Brain4U mobile application

No. Task Contents

1 Log in Log in with your hospital account.

2 Basic Survey Complete the basic survey.

3 Main Move to the main page.

of
4 Seizure Diary Record seizure based on the scenario.
Seizure Diary

ro
5 Record seizure in detail based on the scenario.
(detail)
6 Medication Record medication and adverse events based on the scenario.

7 Appointments Check your appointments.

Outpatient
-p
8 Complete the outpatient survey.
re
Survey
Physical
9 Record changes in your physical development.
Development
lP

10 My Epilepsy Check the overall trend of your disorder.

na

We summarized the results and compiled a list of improvement requirements, such as adding

a scroll bar on the right of vertically long menus, changing words that might confuse users,
ur

and rearranging information. The list was sent to the development team, and changes were

reflected in the released version of the application. This study was approved by the
Jo

Institutional Review Board at SNUBH (IRB No. B-1804/460-306) and every participant

voluntarily decided to take part in this research, and written consent was obtained.
 2. Results

2.1 Mobile epilepsy management application

The mobile based EHR-integrated epilepsy management application was launched on

February 9, 2019. Figure 3 shows the main screenshots of the launched version. Art therapy

was used to design the user interface (UI) for relieving stress and improving the

psychological stability of users. Art therapy is a therapeutic method to positively influence

users’ emotion through design. We aimed to relieve stress and stabilize the mental status of

of
users with friendly illustrations including calm pastel toned bright and soft colors. Graphic

ro
design varies by season. Two functions, “Seizure Record” and “Medication,” are located on

the main screen considering the frequency of their use and relative importance. Patients’
-p
medical data such as prescription and test results are automatically linked with the mobile

application.
re
lP
na
ur
Jo

(a) (b) (c) (d)

Figure 3. Brain4U mobile application for patients

*(a) main page, (b) seizure record, (c) my epilepsy, (d) 3D animation-based seizure type

educational contents

Besides the patient mobile application, we developed a monitoring web application for
physicians as shown in Figure 4. The application enabled physicians to check their patients’

daily disease management status such as medication adherence, frequency of seizures,

accompanying diseases, and emotional changes.

of
ro
-p
re
lP
na
ur

Figure 4. Physicians’ monitoring page integrated with hospital Electronic Health Record
Jo

system; ADHD: Attention Deficit Hyperactivity Disorder; OXC: oxcarbazepine; VPA:

valproic acid; LTG: lamotrigine

2.2 End-user usability test result

3.2.1 Demographics of participants

Of the eight participants, five were caregivers (mean age = 42.6 years) and three were

patients (mean age = 20.7 years). Five participants were female and three were male. Four

participants answered that they were familiar with the mobile application and the remaining

participants reported being unfamiliar with it.

of
3.2 Patients’ and caregivers’ perceptions about epilepsy management

Figure 5 (Left) indicates what participants think is important for managing epilepsy and what

ro
they actually care about in their daily lives. According to the results, participants considered
-p
“sleep” as the most important factor for managing epilepsy. Medication records, seizure

records, and stress control were implemented. Only few participants answered that they did
re
not regulate the intensity of sunlight exposure in practice though many of them consider
lP

sunlight an important seizure trigger. Most participants, in reality, handled sleep time,

medication, and stress carefully; however, seizure recording and regulating sunlight exposure

were rarely practiced. Moreover, no participant engaged in exercise as a means to manage

na

epilepsy. Figure 5 (Right) refers to the intensity of interest. All participants reported that they

were interested in managing epilepsy.

ur
Jo
 of
ro
-p
Figure 5. (Left) Response rate of perceived important factors and what participants actually

did for managing epilepsy; (right) response rate of the degree of interest in managing epilepsy
re
and the status of recording seizures and medications
lP

3.3 Patients’ and caregivers’ expectations of the epilepsy management service

Participants were then asked to complete questionnaires about their prior expectations of the
na

application. For all questions, the average scores were found to be greater than 4.5 points out

of 5 as shown in Table 2.
ur

Table 2. Pre-expectation survey result

Jo

Score
Questions
(Mean)
I think this application is very essential for managing epilepsy. 4.8
I think this application is improving my understanding of epilepsy. 4.5
I think this application is improving the efficacy of epilepsy
4.8
management.
I think I can manage seizure and medication better with this application. 4.8
 I think this application is very helpful to communicate with physicians. 4.9
I think I will consistently use this application. 4.9

3.4 Task success rate

of
ro
-p
re
lP
na

Figure 6. Task success rate; info: information

In terms of the success rate, most participants completed the tasks well without significant
ur

difficulties as shown in Figure 6. However, in recording the seizure in detail (task 5), only

37.5% completed it on their first try. Among all participants, 25% had difficulties in checking
Jo

appointments (task 7) and changing physical growth information (task 9). They could not find

the right menu for achieving tasks. Additionally, 37.5% participants could not access the My

Epilepsy page (task 10) at once, because it was difficult to figure out the meaning of the My

Epilepsy button in the main screen. In terms of the completion time for each task, the basic

survey (task 2) took the longest time as shown in Figure 7. However, this was not due to a
usability issue, because it took considerable time to answer all questions related to their status

in task 2. None of the participants experienced problems in accessing the function (basic

survey). However, participants appeared very confused when they tried to record the seizure

in detail (task 5). They were unsure of where to navigate in order to record the seizure in

detail. In the outpatient survey (task 8), except for one participant who spent some time to

find “outpatient survey” in the survey list, all other participants finished without any

hindrance.

of
ro
-p
re
lP
na
ur
Jo

Figure 7. Completion time for each task; info: information

3.5 Satisfaction survey

The SUS and few more questions were used to investigate participants’ satisfaction and

willingness to use the application after its launch [28]. The SUS result indicates scores of

84.5±12.336, which fall within the top 10%. Figure 8 shows participants’ responses to the
usefulness and necessity of the application and willingness to use it. My Epilepsy and the

outpatient survey function received the highest scores, while the basic survey received the

lowest score. Only patients gave high score to the medication function, and caregivers

(parents) were more interested in the seizure diary than patients.

How useful and necessary do you think How frequently are you going to
the function is in managing epilepsy? use each function in reality?

of
ro
-p
re
lP

Figure 8. Usefulness and expected frequency of each function’s use by participant

characteristics
na

3.6 Interview
ur

As a final stage, an in-depth interview was conducted for identifying why they had any
Jo

trouble completing tasks and their overall feelings regarding Brain4U. Table 3 depicts the

issues raised by the participants collected through usability tests and interviews. Issues were

found in Basic survey, Seizure record, and Medication record functions, and these functions

were transferred to the development team to improve service usability.

Table 3. Issues and implications of Brain4U design of user interface

Task User Interface Issues Implication

of
Some participants were All possible diagnoses
confused because they were listed but
could not find their participants could not find

ro
diagnosis within the it because there was no
Basic
selection box. scroll bar available, which
Survey
prevented them from
-p viewing the lists below.
Additionally, every select
box should be modified.
re
The “No” and “Yes” We judged that the word
buttons actually meant used on the button may
“cancel” and “save” confuse users, so we
lP

respectively. However, decided to change the

because it was located word “No” to “Cancel”
directly below the and “Yes” to “Save” for
na

Seizure selection box asking, clarity.

Record “Was there an
aggravation factor?”,
participants tended to
ur

not use the selection

box but click the “Yes”
button when they had
Jo

aggravation factors.
Participants were We concluded that this
confused as to which could confuse users, so
Medication button status – colored we decided to change the
Record or not – was a mark to button to a toggle shape
indicate taken. button, because it is more
intuitive.
Aside from this, we compiled an overview of participants’ positive and negative feedback on

Brain4U, as described in Table 4. Every participant indicated that the EHR-integrated service

would be very useful. Three applications—patient mobile application, EHR system, and

physician monitoring web—are connected and share useful information without any

additional efforts from the end-users. Participants reported that it makes using the mobile

application and managing the disorder very simple and convenient. In addition, based on

users’ opinion that information about the prescribed drug should be available, we added a

of
function that allowed users to check this information by clicking on the prescription drug list.

Table 4. Overview of the positive and negative opinions of Brain4U

ro
Advantages of use Disadvantages of use
● It is very useful; we can
-p
confirm medical test results with
the application.
● It seems like we are more
re
closely connected with the
physicians and it comforts me.
lP

Mentioned 5 times or ● Medication reminders are set

more automatically based on the
prescription data saved in the EHR
na

system. This is very convenient.

● It was not difficult to use
compared to my expectations.
ur

● I liked the design of this

application.
● (All patient participants) Self- ● Can I get more detailed
Jo

survey functions that manage my information, such as

behavioral status would be very
Mentioned 2 to 4 useful.
times ● (All patient participants) An
outpatient survey will be very
useful, because usually I go to the
hospital with my parents, and even
expected side effects and the
appropriate dosage, about
drugs I am taking?
●I think the patient account
should be shared with
caregivers. I (parent) want to
 if I have things to ask, I can’t ask
because my parents would worry
about me.
● Seizure animations will be
very beneficial in helping me
understand my disorder.
Mentioned once
supervise whether my child
inputs information properly.
● For some patients with
serious conditions,
managing these factors is
not very practical.

of
3. Discussion

ro
Finding proper medication and monitoring seizure trigger factors is crucial in managing

epilepsy. Therefore, patients write down or type details on their mobile phone to record them.
-p
However, this is very troublesome for patients, and difficult for physicians to read and infer
re
associations as well, making this method difficult to use practically in the clinical field. To

support this, numerous mobile epilepsy management applications have been released. We
lP

analyzed six epilepsy management applications; all applications provide seizure diary

function and most of them provide medication reminder function. Other functions such as
na

automatic seizure detection, education materials, photo/video upload, emergency call, and

emotion management function are adopted depending on applications. Therefore, we decided

ur

to provide all these functions except automatic seizure detection. Automatic seizure detection

function was excluded because of issues in its accuracy.

Jo

Educational materials and 3D animation videos explaining seizure types were also developed,

reflecting the opinions of physicians, with the understanding that identifying different types

of seizures is very challenging for most patients. Additionally, frequently asked questions and

physician answers were added to the menu. To develop a practically helpful service,
accessibility and convenience of use are of utmost importance. Similarly, recent studies

developing healthcare applications tend to place importance on the usability test for deriving

unrecognizable issues and needs of the end-users. Therefore, unlike other applications, we

integrated the application with the hospital EHR system that strongly contributes to the

convenience of use by drawing patient data such as lab results and prescription automatically

from the hospital EHR system. Moreover, FHIR standard was adopted for exchanging EHR.

Simultaneously, to overcome concerns that the application may become too complicated, we

of
particularly focused on the UI. As a result of these efforts, participants completed tasks

without major difficulties in the usability test, and the satisfaction scores indicated high

ro
satisfaction.

-p
In addition, we compiled the advantages and disadvantages of use from the interview. Every

participant stated that the EHR-integrated application would be very useful. Further, we
re
realized that perceived important functions differ depending on the user characteristics.

Parents normally place more emphasis on seizure occurrence, while patients tend to be
lP

concerned more with regular medicine dose. However, bedridden patients who have

difficulties in controlling seizures with medications were less willing to use this application
na

because seizures occur very often among this group, and so their caregivers may not forget

medication, and it is impossible to record every seizure’s occurrence. This response was
ur

completely unexpected because we anticipated that the seriousness of the disorder would

correspond to the demand for the application.

Jo

In addition, although we could not quantify data in this paper, we found few issues

unexpectedly through the usability test. Adolescent epilepsy patients tend not to communicate

with their parents, especially regarding their disorder. This was a common distress for

adolescents and their parents. Parents were pleased because they can monitor their children
without conflicts. Normally, caregivers (parents) of adolescent patients visit the hospital

instead of patients themselves, or they visit together. Adolescent patients confessed feeling

uncomfortable about asking questions to the physicians when they were with their caregivers.

Similarly, they appreciated the outpatient survey because they could ask or notice something

about their disease privately and directly to their physicians. Unfortunately, we could not

address all detected issues in this research. We expect to handle them in the future for a more

considerate service.

of
We would like to suggest a few guidelines for future studies. First, when collecting

information about epilepsy from mobile users, mHealth epilepsy data standard should be

ro
considered. Common Data Elements are currently available for mHealth in epilepsy devices

-p
[29]. Integration of our data with other international datasets for research purposes will

contribute to the development of epilepsy management protocols and improvement of

re
interoperability, and patients will have great flexibility when managing their disorder. Second,

in the aspects of utilizing data on seizure occurrence data, the gathered data can be used to
lP

evaluate temporal patterns of seizure events with statistical models such as zero‐ inflated

negative binomial mixed‐ effects model or incidence rate rations. Through this, the
na

relationship of seizures with its triggers factors can be identified more accurately [30].

Moreover, the general pattern analysis is possible, which provides valuable insight to the
ur

physicians [31, 32]. Lastly, when physicians or researchers refer to seizure data that patients

input, they should consider that patient reports are not accurate in terms of frequency and
Jo

seizure type. Hence, research to complement this needs to be processed [33].

The test results were positive, but this should only be speculated upon while considering its

limitations. First, we conducted the usability test only with a single hospital, so it is difficult

to generalize the study results. Therefore, it is necessary to initiate research later with more
participants to draw insights for representing a wider range of users. Second, we could not

verify the clinical effectiveness of the application with control and intervention groups.

Nevertheless, we want to highlight that this study tried to overcome the limitations of existing

epilepsy management mobile applications by improving UX, the wide coverage of trigger

factors, practical educational materials, and the integration with the hospital EHR system.

4. Conclusions

of
In this study, we designed, developed, and evaluated the usability of the EHR-integrated

mobile application Brain4U. This investigation covered a wide range of factors that are

ro
important in the management of epilepsy. In addition, based on the concern that this

-p
application could be complicated to use, we tried to design the UI and flow to be as simple as

possible. Our efforts paid off upon observing the usability test results, but there are still many
re
limitations to be overcome in future research. We expect our application to be a foundational

stone for improving not only epilepsy disorder management but also the quality of life of
lP

patients and their caregivers.

5. Acknowledgments
na

This work was supported by the Information Technology Research & Development program
ur

of Seoul National University Bundang Hospital and SK Telecom.

Jo
 6. Summary Points

What was already known

Controlling seizures to manage epilepsy is important, but it is not easy due to the diversity

of
and complexity of seizures.
There are various epilepsy management mHealth services but none that provide functions
targeting the management of seizure trigger factors comprehensively or that are compatible

ro
with physicians’ EHR systems.
What this study added

with the hospital’s EHR system.

-p
The study developed and evaluated an mHealth service for managing epilepsy compatible

Functions such as seizure Diary, Medication Reminder, Appointments, Outpatient Survey,

re
and My Epilepsy were developed to aid patients’ epilepsy management.

The usability test showed that participants were highly satisfied with the service and there
lP

were differences between participants in scenario-based task success rate.

The degree of perceived importance of the functions differed by participants’
characteristics.
na
ur
Jo
Legends

Figure 1. Information architecture of Brain4U for patients

Figure 2. System architecture of Brain4U linked to the Electronic Health Record system

of
Figure 3. Brain4U mobile application for patients

Figure 4. Physicians’ monitoring page integrated with hospital Electronic Health Record

ro
system

Figure 5. (Left) Response rate of perceived important factors and what participants actually
-p
did for managing epilepsy; (right) response rate of the degree of interest in managing epilepsy
and the status of recording seizures and medications
re
Figure 7. Completion time for each task

Figure 8. Usefulness and expected frequency of each function’s use by participant

lP

characteristics

Table 1. Scenario-based tasks for evaluating usability of Brain4U mobile application

na

Table 2. Pre-expectation survey result

Table 3. Issues and implications of Brain4U design of user interface

ur

Table 4. Overview of the positive and negative opinions of Brain4U

Jo
References

[1] WHO. (2019). Epilepsy. Retrieved from https://www.who.int/news-room/fact-

sheets/detail/epilepsy

[2] S.M. Bowman, M.E. Aitken, G.B. Sharp, Disparities in injury death location for people

with epilepsy/seizures, Epilepsy Behav, 17 (2010) 369-372.

[3] H.B. Alexander, D.K. Broshek, M. Quigg, Quality of life in adults with epilepsy is

associated with anticonvulsant polypharmacy independent of seizure status, Epilepsy Behav,

of
78 (2018) 96-99.

[4] G. Mignot, Drug trials in epilepsy, BMJ, 313 (1996) 1158.

ro
[5] WebMD. (unknown). Epilepsy Treatments: Find the Right Medication. Retrieved from

-p
https://www.webmd.com/epilepsy/guide/choosing-best-epilepsy-medication-for-you#1

[6] J.S. Mbwana, Z.M. Grinspan, R. Bailey, M. Berl, J. Buchhalter, A. Bumbut, Z. Danner, T.
re
Glauser, A. Glotstein, H. Goodkin, B. Jacobs, L. Jones, B. Kroner, G. Lapham, T.

Loddenkemper, D.M. Maraganore, D. Nordli, W.D. Gaillard, Using EHRs to advance

lP

epilepsy care, Neurol Clin Pract, 9 (2019) 83-88.

[7] R.S. Fisher, D.E. Blum, B. DiVentura, J. Vannest, J.D. Hixson, R. Moss, S.T. Herman,
na

B.E. Fureman, J.A. French, Seizure diaries for clinical research and practice: limitations and

future prospects, Epilepsy Behav, 24 (2012) 304-310.

ur

[8] F.A. Le Marne, S. Butler, E. Beavis, D. Gill, A.M.E. Bye, EpApp: Development and
Jo

evaluation of a smartphone/tablet app for adolescents with epilepsy, J Clin Neurosci, 50

(2018) 214-220.

[9] The Johns Hopkins University (unkown). EpiWatch. Retrieved from

https://apps.apple.com/us/app/epiwatch/id1047757228EpiWatch.EpiWatch.

[10] Neutun Labs inc (2017). Seizure Sync Epilepsy Log. Retrieved from
https://apps.apple.com/us/app/seizure-sync-epilepsy-log/id1084926036 Seizure Sync.Seizure

Sync.

[11] P.S. Pandher, K.K. Bhullar, Smartphone applications for seizure management, Health

Informatics J, 22 (2016) 209-220.

[12] C. Escoffery, R. McGee, J. Bidwell, C. Sims, E.K. Thropp, C. Frazier, E.D. Mynatt, A

review of mobile apps for epilepsy self-management, Epilepsy Behav, 81 (2018) 62-69.

[13] S.N. Haron, M.Y. Hamida, A. Talib, Towards Healthcare Service Quality: An

of
Understanding of the Usability Concept in Healthcare Design, Procedia - Social and

Behavioral Sciences, 42 (2012) 63-73.

ro
[14] I. Maramba, A. Chatterjee, C. Newman, Methods of usability testing in the development

of eHealth applications: A scoping review, Int J Med Inform, 126 (2019) 95-104.
-p
[15] Empatica. (2019, August 26). Alert for Embrace watch. Retrieved from
re
https://play.google.com/store/apps/details?id=com.empatica.embrace.alert&hl=en.Alert App,

Empatica Inc.,.Alert App, Empatica Inc.

lP

[16] SeizAlarm, LLC. (2015). SeizAlarm. Retrieved from

https://apps.apple.com/us/app/seizalarm-seizure-detection/id978475280.SeizAlarm.
na

[17] MCM Net Limited. (unknown). Epilepsy Tool Kit. Retrieved from

http://myhealthapps.net/app/details/73/epilepsy-tool-kit.Epilepsy Tool Kit.Epilepsy Tool Kit.

ur

[18] Irody Inc.. (unknown). EpiDiary. Retrieved from

https://apps.apple.com/us/app/epidiary/id362460642.EpiDiary.EpiDiary.
Jo

[19] E. Foundation. (unknown)., My Seizure Diary. Retrieved from

https://diary.epilepsy.com/.

[20] S.T. LLC. (unknown). , Seizure Tracker. Retrieved from

https://www.seizuretracker.com/index.php.

[21] M. Kovacs, The Children's Depression, Inventory (CDI), Psychopharmacol Bull, 21

(1985) 995-998.

[22] A.T. Beck, N. Epstein, G. Brown, R.A. Steer, An inventory for measuring clinical

anxiety: psychometric properties, J Consult Clin Psychol, 56 (1988) 893-897.

[23] S. Zhang, D.E. Faries, M. Vowles, D. Michelson, ADHD Rating Scale IV: psychometric

properties from a multinational study as a clinician-administered instrument, Int J Methods

Psychiatr Res, 14 (2005) 186-201.

[24] B. Echenne, V. Humbertclaude, R. Cheminal, [Treatment of epilepsy in children],

of
Pediatrie, 48 (1993) 883-887.

[25] S.W. Goodwin, A.I. Lambrinos, M.A. Ferro, M. Sabaz, K.N. Speechley, Development

ro
and assessment of a shortened Quality of Life in Childhood Epilepsy Questionnaire

(QOLCE-55), Epilepsia, 56 (2015) 864-872.

-p
[26] J.C. Mandel, D.A. Kreda, K.D. Mandl, I.S. Kohane, R.B. Ramoni, SMART on FHIR: a
re
standards-based, interoperable apps platform for electronic health records, J Am Med Inform

Assoc, 23 (2016) 899-908.

lP

[27] T. Tullis, W. Albert, Measuring the User Experience: Collecting, Analyzing, and

Presenting Usability Metrics (Interactive Technologies), 2nd edition, Morgan

na

Kaufumann2013.

[28] System Usability Scale (SUS), U.S. Department of Health & Human Services
ur

[29] D.M. Goldenholz, R. Moss, D.A. Jost, N.E. Crone, G. Krauss, R. Picard, C. Caborni, J.E.
Jo

Cavazos, J. Hixson, T. Loddenkemper, T.D. Salazar, L. Lubbers, L.C. Harte-Hargrove, V.

Whittemore, J. Duun-Henriksen, E. Dolan, N. Kasturia, M. Oberemk, M.J. Cook, M.

Lehmkuhle, M.R. Sperling, P.O. Shafer, Common data elements for epilepsy mobile health

systems, Epilepsia, 59 (2018) 1020-1026.

[30] V. Ferastraoaru, D.M.G.S. Chiang, R. Moss, W.H. Theodore, S.R. Haut, Characteristics
of large patient-reported outcomes: Where can one million seizures get us?, Epilepsia Open, 4

(2018) 364-373.

[31] M. Daniel Goldenholz, PhD, M. Kshitiz Rakesh, P. Kush Kapur, M. Gaínza-Lein, R.

Hodgeman, R. Moss, M. William H Theodore, M. and Tobias Loddenkemper, Different as

Night and Day: Patterns of Isolated Seizures, Clusters, and Status Epilepticus, Epilepsia, 59

(2018) e73–e77.

[32] L.D. Schneider, R.E. Moss, D.M. Goldenholz, Daylight saving time transitions are not

of
associated with increased seizure incidence, Epilepsia, 60 (2019).

[33] P. Karoly, D. Goldenholz, M. Cook, Are the days of counting seizures numbered?,

ro
Current Opinion in Neurology, 31 (2018) 162–168.

-p
re
lP
na
ur
Jo