Continuous, Objective Behavioral Measurement

Using Smartphones:
An Introduction to the EARS System

Nick Allen, PhD

Ann Swindells Professor of Clinical
Psychology, University of Oregon
Director, Center for Digital Mental
Health, University of Oregon
Co-Founder, Ksana Health Inc.
Disclosures
I am the cofounder of, and have an equity interest in, Ksana Health Inc.,
which provides commercial services based on some of the research
methods discussed in this presentation.
Real-world Behavioral
Measurement:

• uses sensors
mobile and
wearable devices

• for richer,
continuous,
objective
behavioral data
3
 MOBILE
CURRENT
MEASUREMENT SENSING

Snapshot / “moment in time” Always on & continuous

Subjective, usually self-reported Objective

Effortful, requires user input Passive and automatic

Single input Multimodal sensors

Reactive Proactive and timely

4
Smartphones are rich with sensors that
can continuously measure behavioral processes

Physical activity Social activity Sleep Apps and phone use Communicative
language

5
Why phones and not wearables?
Downsides of wearables
Compliance burden
Logistics Burden (i.e., Hardware delivery and
retrieval)
Fewer sensors related to communication and
phone usage

Upsides of wearables
◦ More accurate tracking of sleep and formal
exercise
◦ Measurement of physiological variables
◦ Black box proprietary algorithms
The Mobile Sensing Tradeoff
Good Bad

Ecological validity You don’t have control of your primary

research tool!
Low participant burden
Dependent on Tech and Telecommunications
Low Hawthorne effects companies for your research infrastructure
Highly scalable Missing data
Continuous behavioral measurement allowing Noisy “in the wild” data
investigation of fine-grained temporal patterns
Why not open
source?
There are open-source options for
those who want them (see Wikipedia
page)

It costs money to provide and

maintain these systems!
◦ Customer support
◦ Back-end (cloud services)
◦ Bug fixes
◦ Adapting to changes in phone
operating systems and handsets
◦ Data processing support for
converting raw data to features
Ethics
EARS is only used in IRB-regulated studies (40+ have
passed it)
We have a “SOC-2” certificate where an external
standards organization has certified our data security
and privacy standards, including HIPAA and GDPR.
We do not keep any standard PII in our system
However, the most privacy sensitive signals are GPS and
keyboard, because although they are de-identified, it
may be possible to reverse engineer identify (i.e., not
fully anonymized)
Risk Management should be achieved with standardized
procedures - not based on passive sensing. (NB We have
the capacity for risk alerts)
 Overview of the EARS System

Private & Confidential. © 2021 by Ksana Health Inc. All rights reserved. Ksana, Vira, and Vira Pro names and logos are trademarks of Ksana Health. 10
 EARS Research Platform
●Mobile sensing and Ecological
Momentary Assessment
●Our approach emphasizes:
○Phone only
■ No special equipment

○“In the wild”

■ No special instructions

○Continuous data collection

■ Naturalistic patterns of behavior
 Overview of the EARS System

DATA FEATURES
DATA CLOUD
(AWS)
RESEARCH PARTICIPANTS
• Keyboard • Language patterns
Surveys & Momentary Objective collection of
(Mood, Cognition)
real world-evidence on • GPS
Assessments • Sleep
patterns of behavior and • Accelerometer
behavioral data • Physical activity
• Phone and App
usage • Mobility
Receives questions and • Social interaction
• Questions sent to
momentary assessments the participant

INVESTIGATOR
Collects real-world data related to passive
behaviors and dashboard with participant
RESEARCHER DASHBOARD
status
The EARS tool –
Mobile Sensing Features Extracted
Phone and app use data
Language and Cognitive Style (keyboard)
Sleep
Physical activity
Geographic location and movement
Music and other media choice
Circadian patterning
 Benefits for Researchers

•Self-service & configurable

•Onboarding support and expert guidance

Data science services included
Broad objective data monitoring array

•Flexible application in clinics, labs, and beyond

Remote study management tools
Objective measurement that appeals to grant reviewers
14
Product Screens

Uses smartphone
hardware most
participants already
have

15
Product Screens

Each study is
configurable by
the researcher

16
Product Screens

Configurable EMAs,
questions, and timing

17
Product Screens

• Near Real time

reporting of data
uploads by data
streams
• Supports large
cohorts
• Enables large,
reliable data pool

18
Product Screens

Includes onboarding and

recruitment materials
like videos, support
documentation, and IRB
templates

19
Where is EARS available?
Device and Operating System Compatibility
 EARS Videos

Private & Confidential. © 2021 by Ksana Health Inc. All rights reserved. Ksana, Vira, and Vira Pro names and logos are trademarks of Ksana Health.
22
 Example Research Using EARS

Private & Confidential. © 2021 by Ksana Health Inc. All rights reserved. Ksana, Vira, and Vira Pro names and logos are trademarks of Ksana Health.
23
Changing
Patterns of
Social
Communication
Adolescent
EARS
64,307
Keyboard
Smartphone Messages
(N=50)
Language, 53,132
Internalizing Social
Messages
Symptoms and (N=50)
Min. 5 Days
Daily Mood. Concurrent
Text + EMA
(N=29) Final Sample
21,365
Messages LIWC Text
246 Daily
Analysis
Observations
(N=29)

Elizabeth McNeilly et al.,

2023; Clinical Psychological Science
First-Person Pronouns
(I, me, my, mine)

Key Takeaway
Lower daily well-being
and higher depression
symptoms were
associated with higher
first-person pronouns
In general, how have you been feeling over the last day?
Private & Confidential. © 2021 by Ksana Health Inc. All rights reserved. Ksana, Vira, and Vira Pro names and logos are trademarks of Ksana Health.
 Present Focus Words
(present tense verbs)

Well-being Trait

Key Takeaway
Both trait and daily well-
being were associated
with higher present focus
words
Private & Confidential. © 2021 by Ksana Health Inc. All rights reserved. Ksana, Vira, and Vira Pro names and logos are trademarks of Ksana Health.
In general, how have you been feeling over the last day?
 Smartphone Communication and Mood Dynamics
A

Daily Mood Ratings

Daily Key Inputs (Sentiment)

Baseline Day 90
Note. Adolescents (N = 83); Messages = 354,278; Daily sentiment predicts next day mood
(controlling for previous day mood), B = .03, SE = .01, p = .04; Li et al., In Press.

B Emoji C Laughter D Disagreement

Note. Latent Dirichlet Allocation Models (B) Emojis predicted next day mood (p<.01), (C) Laughter
associated with depression symptoms (rho = -.30, p<.01); (D) Disagreement associated with dysphoria
28
symptoms (rho = .28, p<.01); Li et al., In Press, Journal of Psychopathology and Clinical Science.
Predicting Emergence of MDD

Weekly Depression Ratings

Weekly A-Life Ratings
4
Daily Mood
3
Daily Key Inputs
2

BL 6M
-60 -30 0 30 60
Weekly Mood Ratings

Note. N = 90 (44% remMDD); (A) Weekly mood aggregates associates with A-

LIFE Ratings (β = -.11, p < .001) and (B) Personal pronoun frequency (e.g., I) from
~1.6 million messages associates with A-LIFE ratings for subthreshold MDD ≥4 (β 29

= .05, p = .027); Funkhouser et al., in press.

Predicting Emergence of MDD
Predicting MDD

Weekly A-Life Ratings

Sentiment

Daily Mood
“I” words
Daily Key Inputs

Absolutist

BL 6M 1.0 1.5
Odds Ratio (95% CI)

Note. N = 90 (44% remMDD); (A) Weekly mood aggregates associates with A-

LIFE Ratings (β = -.11, p < .001) and (B) Personal pronoun frequency (e.g., I) from
~1.6 million messages associates with A-LIFE ratings for subthreshold MDD ≥4 (β 30

= .05, p = .027); Funkhouser et al., in press.

Geographic mobility
Geolocation Predicts Suicidal Events

Suicide Attempt ED Visit

Geolocation Features

Baseline 6-Months
0.8
Suicidal Events
0.6

Probability (%)
Home Stay
0.4

Entropy
0.2

Avg. Distance
0.1 1.0 10 0.0
-4 -2 0 2
Odds Ratio Home Stay

Note. N = 180 psychiatric adolescents enriched for suicidal thoughts and

behaviors (63.5%); Suicide Events = Attempts, Psychiatric Hospitalizations,
Emergency Department visits; Auerbach et al., in preparation.
32
 Next Steps with EARS

Private & Confidential. © 2021 by Ksana Health Inc. All rights reserved. Ksana, Vira, and Vira Pro names and logos are trademarks of Ksana Health.
33
 Next Steps with Mobile Sensing – Clinical Applications

Private & Confidential. © 2021 by Ksana Health Inc. All rights reserved. Ksana, Vira, and Vira Pro names and logos are trademarks of Ksana Health.
34
Private & Confidential. © 2021 by Ksana Health Inc. All rights reserved. Ksana, Vira, and Vira Pro names and logos are trademarks of Ksana Health.
New Developments

iOS App usage measurement On the Roadmap

Implemented! Contextually triggered EMA & EMI

Widgets
Live Activities EMA notifications
Wearable integration through OS health kits
Suicidal Youth Receiving
Intensive Outpatient Treatment
Vira Patient The Vira
Smartphone Practitioner
App Platform

Behavioral
Geolocation
patterns

Notifications
Accelerometer and nudges

Keyboard
logger

Weekly
surveys

Note. 3-Year RCT (enrollment begins October 2023); ID: NCT05920252 38

Thank You!