IUI 2016 • Student Consortium March 7–10, 2016, Sonoma, CA, USA

Assessing Empathy through

mixed-reality

Cassandra Oduola Abstract

Texas A & M University This research seeks to produce a new way of assessing
College Station, TX empathy in individuals. The current widely used diagnostic
tools are questionnaires. These questionnaires are easy to
âĂIJpassâĂİ if the individual simply lies and chooses the
answers that would be most beneficial to them.
Furthermore, it is shown, assessing empathy is harder in a
clinical setting because it is not the natural world, a person
may purposely inhibit their behavior to seem more
âĂIJnormalâĂİ. Finding methods that would assess affect
while interacting with a computer could yield higher
accuracy in diagnosis

ACM Classification Keywords

HCI [Mixed-Reality]: Virtual Reality

Permission to make digital or hard copies of part or all of this work for personal or
classroom use is granted without fee provided that copies are not made or distributed
for profit or commercial advantage and that copies bear this notice and the full citation
on the first page. Copyrights for third-party components of this work must be honored.
For all other uses, contact the owner/author(s). Copyright is held by the
author/owner(s).
IUI’16 Companion, March 7–10, 2016, Sonoma, CA, USA.
ACM 978-1-4503-4140-0/16/03.
http://dx.doi.org/10.1145/2876456.2876466
Introduction
Empathy influences human interaction daily. Severe deficits
in empathy can be seen in borderline personality disorder
(PD), narcissistic personality disorder, antisocial personality
disorder, psychopathy and Autism. Though someone may
not be diagnosed with a PD, they can still have empathy
levels lower than the norm. These individuals will be more
prone to anger outburst, insensitive comments and other
socially unacceptable actions. Simon Baron Cohen has
created an empathy bell curve with scores ranging from
zero to six. Zero-empathy suggesting psychopathy or

142
IUI 2016 • Student Consortium
Aspergers and six being those highly altruistic with the
general population falling between four to five. CohenâĂŹs
research suggest the majority of men fall on a four level of
empathy (human interactions focused on topics of problem
solving), and most women score level five (human
interactions fueled by desire to emotionally connect). This
research seeks to use a mixed reality environment to
categorize users on an empathy bell curve. This research
also seeks to bridge the gap in misdiagnosis of individuals
with AS, children with psychopathy, and adults with other
disorders such as ADHD that may exhibit similar tendencies
yet lack comorbidity. Mental health professionals are
reluctant to diagnose a person with a PD before 21 years of
age. Autism and Psychopathy are both genetic and can be
seen in childhood. The diagnosis of Autism in a young child
is very helpful but the diagnosis of Psychopathy can be hard
to accept. A famous case is Adam Lanza, the Sandy Hooke
shooter who manifested symptoms of low empathy but was
misdiagnosed as Asperger’s. An empathy measuring game
will be developed using the Unity game engine SDK for the
Hololens. The goal is to make this empathy measuring
game available for under $200.00 to mental health
professionals, schools, employers and any other
establishment in need of empathy measuring. The objective
is to help diagnosticians, therapists, and psychiatrists in
establishing proper diagnosis of these complex disorders.
We also seek to help employers hire the right individuals. A
person with a level two empathy prone to anger outburst
would not be suited for a job working with young children.
These defects may not show up during an interview
because of the ease of âĂIJfakingâĂİ it. Through identifying
patterns in mixed reality interactions, the test will be
designed to reveal the user’s natural tendencies.
March 7–10, 2016, Sonoma, CA, USA
Needs
Currently virtual reality is used for rehabilitation of sexual
predators as well as forensic psychiatry. These uses have
had positive results. Psychopaths are known to be very
rational, though VR is a useful tool, it is still virtual. With
mixed reality the low empathetic individual would have the
freedom to manipulate their environment at will. This allows
for better differentiation of individuals as each person will
manipulate their environments in a way unique to
themselves. Studies have shown low empathetic individuals
often feel comfortable expressing themselves through social
media outlets. This may show up in blatant trolling or
fabricated profiles which exaggerate a personâĂŹs
success. This may also show up in fake profiles, where the
person pretends to be someone they are not simply to fool
others. For these reasons it is often hard to identify low
empathetic individuals on social media. Though some may
fall cleanly in identifying traits such as high amounts of
sexual content and profanity, not all will be so obvious.
Identifying low empathetic individuals through profanity or
sexual content also does not yield accurate diagnosis as a
young person seeking attention online may exhibit the same
behavior without being on a spectrum.
Related Research
Simon Baron-Cohen has done extensive research in the
development of an empathizing-systematizing test. While
this work is highly relevant to this research question, it is
potentially limited due to its specific focus on Autism. Sam
Vankin is a self-professed malignant narcissist and author of
"Malignant Self Love: Narcissism Revisited". While the
information on narcissism is helpful to those in contact with
such a person, it does not aid diagnosis. Currently, official
diagnosis is performed through questionnaires, which are
subject to exploitation. The Hare Psychopathy checklist is a
psychological assessment used to assess psychopathy in
143
IUI 2016 • Student Consortium
individuals, but some concerns exist as to its susceptibility
to exploitation . Researchers have used VR for forensic
psychology. Our methods seek to be useful not just in
extreme cases.
System Design
Practitioner UI
The mental health professional will load the selected
environment. There will be tests geared towards children,
adolescents, young adults and adults 35+. These tests will
present as different scenarios, children will have the
opportunity to play with others and animals allowing callous
or unemotional traits to manifest. Adolescents may be given
more complex scenarios, a person may be seen crying or
they may witness bullying. Events that are highly annoying
may occur in order to elicit a negative reaction. Young
adults will be given the opportunity to interact with others as
well including people they know. They will be able to do
whatever they want to a close relative or friend. Adults
would be given similar scenarios. All levels would have
different instances to show altruistic behavior some obvious
while others being more subtle. At the end of the interaction
the physician will receive an empathy score of the individual.
Hololens
The user will use the Hololens to be immersed in a mixed
reality. The Hololens will display the holographic images
March 7–10, 2016, Sonoma, CA, USA
allowing the user to interact and manipulate them.
Algorithms will be developed utilizing the speech, gesture
and eye recognition capabilities of the Hololens. Through
machine learning we will analyze a combination of features
that suggest lower levels of empathy and vice versa.
User
The user will not have access to their score. The goal of the
user interaction is to feel as natural as possible. If the user
feels they are being watched, they may alter their behavior.
To prevent this, the user will be told they are to simply
interact with their environment without constraint.
Ethics
Due to the damaging stigma of being labeled a psychopath,
all individuals will remain anonymous. Individuals will be
represented by numbers with no follow up contact
information unless stated otherwise. Analysis will not be
started until user studies are complete. This ensures after
an individual has take a test their scores will not be
immediately available to researchers but rather stored in a
database for future use. User studies will not begin until the
Institutional review board has approved our methods of
study.
Goals
We seek to answer the following questions:
1. How do low empathetic individuals interact in a mixed
2. reality environment? Can empathy be measured
using a mixed reality environment?
Uncanny Valley
BehaVR currently aides forensic psychologists in identifying
sexual preferences among sexual predators for use in
144
IUI 2016 • Student Consortium
Figure 1: Empathizing bell curve
rehabilitation and identification. While wearing a VR
headset, subjects are shown various virtual characters.
While subjects are viewing these characters, physical
arousal is assessed by scientists.
The results were successful but scientists found virtual
characters had to be tailored to fit the desires and sexual
history of offenders. The notion of uncanny valley did not
hinder subjects from being aroused.
In applying my research hypothetically to this specific case,
it would be obvious that a sexual offender has lower then
normal levels of empathy. This research would go past
simply viewing, the subject would also be able to interact
with virtual characters.
Methods
Through assessment of 200 to 300 users we will score
individuals based on Simon Baron CohenâĂŹs Empathy
bell curve. We will then verify this data with a licensed
Psychiatrist.
References and Citations
Acknowledgements
I would like to thank the Texas A & M Department of
Computer Science and Engineering as well as the Sketch
Recognition lab.
March 7–10, 2016, Sonoma, CA, USA
References
Massil Benbouriche, Kevin Nolet, Dominique Trottier, and
Patrice Renaud. 2014. Virtual reality applications in
forensic psychiatry. In Proceedings of the 2014 Virtual
Reality International Conference. ACM, 7.
Carlos Busso, Zhigang Deng, Serdar Yildirim, Murtaza
Bulut, Chul Min Lee, Abe Kazemzadeh, Sungbok Lee,
Ulrich Neumann, and Shrikanth Narayanan. 2004.
Analysis of emotion recognition using facial expressions,
speech and multimodal information. In Proceedings of the
6th international conference on Multimodal interfaces.
ACM, 205–211.
R. Cowie, E. Douglas-Cowie, N. Tsapatsoulis, G. Votsis, S.
Kollias, W. Fellenz, and J.G. Taylor. 2001. Emotion
recognition in human-computer interaction. Signal
Processing Magazine, IEEE 18, 1 (Jan 2001), 32–80.
DOI:http://dx.doi.org/10.1109/79.911197
David Diamond. 2003. Love Machine Building computers
that care. WIRED-SAN FRANCISCO- 11, 12 (2003),
230–235.
William James. 1884. II.âĂŤWhat is an emotion? Mind 34
(1884), 188–205.
Christian Kleine-Cosack. 2008. Recognition and
simulation of emotions. PDF). Archived from the original
on May 28 (2008).
Jia Luo. 2012. Affective computing and intelligent
interaction. Vol. 137. Springer.
Theodore Millon and Roger O Davis. 1996. Disorders of
personality: DSM-IV and beyond . John Wiley & Sons.
Rosalind Wright Picard. 1995. Affective computing.
(1995).
Rosalind W Picard. 2000. Affective computing. MIT press.
Eric Siegel. 2013. Predictive analytics: the power to
predict who will click, buy, lie, or die. John Wiley & Sons.
145