BUILDING A MOOD SENSOR FROM

SMARTPHONE USAGE PATTERNS

MOODSCOPE
CLAIMS
Feasibility of inferring mood from
smartphone usage in energy efficient and
privacy preserving system.
Effect of mood on smartphone usage is
personal.
Lightweight and easy to deploy realization
of moodscope.
API for developers.
INFORMATION LOGGED
SMS
Email
Phone call
Application Usage
Web browsing
Location
The Science of Mood
Mood
Emotion
Less Intense
Lasts Longer
Reaction to cumulative
events
More internal

Intense
Spontaneous
Reaction caused by a
specific event
Visible to others
MEASURING MOOD
Circumplex Mood Model

Discrete Categories

PANAS
CIRCUMPLEX MOOD MODEL
RELATED WORK

Recognizing emotions from voice and video

Alternate physiological signals

Leveraging signals from smartphones
USER STUDY & DATA COLLECTION

Pre-Study Focus Group

Field Study

Post-Study Focus Group
PARTICIPANTS
0
5
10
15
20
25
30
College and
graduate students
S/W Engineers Web Editor Salesman Teacher Others
Occupation
Males
66%
Females
34%
Beijing
78%
US
22%
Location
PRE STUDY FOCUS GROUP
Does application usage changes with mood?
Do you communicate with people differently?
Would they like to share their mood?
Would they like their phone to adapt to their mood?

FIELD STUDY
Mood Journaling
Application
Captures the users self
reported mood.
Input taken 4 times a day at 3
hr. interval
Smartphone Interaction
Logger
Captures users
smartphone interaction.
For creating feature table.

MOOD JOURNALING APP.
POST STUDY FOCUS GROUP

Is application easy to use?
Is 4 mood input/day too frequent?
How quickly they respond to input reminder?
Characteristics of Mood Entries
Response Rate

Daily Mood Persistence

Mood Distribution


RESPONSE RATE

0
10
20
30
40
50
60
70
80
1 2 3 4 or more
P
E
R
C
E
N
T
A
G
E
(
%
)

NUMBER OF RESPONSES
DAILY MOOD PERSISTENCE

Average standard deviation- 0.41
Mood is fairly constant and slow to change.

Max. standard deviation- 1.73
Large mood swings do occur some days.
MOOD DISTRIBUTION
DESIGN OF MOODSCOPE MODELS

Daily Averages as Output

Usage Records as Input

Internal Model

Feature Table Characteristics
DAILY AVERAGES AS OUTPUT
USAGE RECORD AS INPUT
Social Interaction Records
Phone call, SMS, E-mails
Routine Activity Records
Browser History, App Usage, Location History
Application Categorization
Categories like built-in, games, office, social.
INTERNAL MODEL
Using mood averages as labels and usage
record as feature table, Multi linear
regression and SFS were applied to develop:
Personalized Mood Model
All-User Mood Model
Hybrid Mood Model
PERSONALIZED MOOD MODEL
Pleasure Activeness
Min. MSE 0.002 0.001
Avg. MSE 0.075 0.085
Max. MSE 0.176 0.215
Std. Deviation 0.050 0.062
Multi Linear Regression on each user data.

Requires Training.
ALL USER MOOD MODEL
One-Size-Fits-All Mood model.

For Pleasure Dimension Personalized All User
Min. MSE 0.002 0.069
Avg. MSE 0.075 0.296
Max. MSE 0.176 0.79
Std. Deviation 0.050 0.175
HYBRID MODEL

Combines personalized and all user
model approach.
Accuracy depends on prioritization
given to personalized data relative to
data from general population.

FEATURE TABLE CHARACTERISTICS

Discriminative Features
Number of occurrences in reduced table
Number of statistically significant features
Number of most Discriminative Features

Positively and Negatively Correlated Features
MOOD INFERENCE ENGINE

PERFORMANCE
API
APPLICATIONS

Video & Music recommender systems.
Social Sharing.
Mood logger and browser.
ADVANTAGES
No extra hardware sensor.
Light weight and power efficient
3.5mWH/day ~ 20 mins. of standby time.
Application independent.
Easy deployment.

LIMITATIONS
Homogenous, small scale user
population contribution to data set.
Frequency of mood input.
Not every factor can be captured.
Lack of privacy.

Questions we could not answer
Why not ?

What happens when no Wifi is available?

Is training of data a continuous process?

Difference between other and cannot categorize
app categories.
38