See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/337915394

Intergenerational Shared Action Games for Promoting Empathy Between

Japanese Youth and Elders

Conference Paper · December 2019

DOI: 10.1109/ACII.2019.8925483

CITATIONS READS

13 430

4 authors, including:

Katie Seaborn Nina Lee

Tokyo Institute of Technology National Cheng Kung University
90 PUBLICATIONS 2,744 CITATIONS 5 PUBLICATIONS 33 CITATIONS

SEE PROFILE SEE PROFILE

Marla Narazani
Technical University of Munich
9 PUBLICATIONS 38 CITATIONS

SEE PROFILE

All content following this page was uploaded by Katie Seaborn on 13 December 2019.

The user has requested enhancement of the downloaded file.

8th International Conference on Affective Computing and Intelligent Interaction, 2019

Intergenerational Shared Action Games

for Promoting Empathy Between
Japanese Youth and Elders

KATIE SEABORN
RIKEN Center for Advanced Intelligence Project (AIP)
The University of Tokyo, RCAST

NINA LEE
National Cheng Kung University

MARLA NARAZANI
Technical University of Munich

ATSUSHI HIYAMA
RIKEN Center for Advanced Intelligence Project (AIP)
The University of Tokyo, RCAST

CITATION: Seaborn, K., Lee, N., Narazani, M. & Hiyama, A. (2019). Intergenerational shared action games for
promoting empathy between Japanese youth and elders. Proceedings of the 2019 8th International Conference on
Affective Computing and Intelligent Interaction (ACII 2019), D2-20. Cambridge, UK: IEEE.
http://dx.doi.org/10.1109/ACII.2019.8925483

The final publication is available via IEEE at http://dx.doi.org/10.1109/ACII.2019.8925483.

©2019 IEEE
 2019 8th International Conference on Affective Computing and Intelligent Interaction (ACII)

Intergenerational shared action games for promoting

empathy between Japanese youth and elders
Katie Seaborn Nina “Ikea” Lee Marla Narazani
Advanced Intelligence Project (AIP) Department of Industrial Design Department of Informatics
RIKEN National Cheng Kung University Technical University of Munich
Tokyo, Japan Tainan, Taiwan Munich, Germany
0000-0002-7812-9096 p38051054@mail.ncku.edu.tw marla.narazani@tum.de

Atsushi Hiyama
Advanced Intelligence Project (AIP),
Research Center for Advanced Science
and Technology (RCAST)
RIKEN, The University of Tokyo
Tokyo, Japan
hiyama@star.rcast.u-tokyo.ac.jp

Fig. 1. The intergenerational shared action game system. On the left, the first and second versions of the finger prototype. On the right, the foot version.

Abstract—Japan is a super-aging society, one marked by a
steadily silvering population and shrinking birthrates. Social
effects range from a decline in intergenerational homes to new
tensions between the generations as elders delay retirement or
return to work. Towards improving relations between the
generations, two versions of an inclusive, intergenerational
game employing “shared action,” or time-movement synchrony,
were developed: a finger-based version inspired by medical
exercise technology from gerontology and a smart shoe-based
version as a familiar form factor. A series of three exploratory
studies were conducted with young Japanese adults and elders.
Findings suggest that the foot version was most suitable overall
and reveal initial, complex effects of age and gender on empathy
measures that can inform affective computing initiatives within
and beyond the cultural milieu of modern Japan.
Keywords— inclusive design, intergenerational games, shared
action, synchrony, body movement, empathy, Japan
I. INTRODUCTION
Japan is a super-aging society, characterized by falling
birthrates and a swiftly silvering population [1]. This has
resulted in cascading social and structural effects within and
beyond the issue of supporting and managing the aging
population [2], [3]. Well-being initiatives focusing on elders’
physical, mental, and social needs—participation in society,
community engagement, and social relationships—have been
proposed as a way forward [2]. In particular, connecting older
and younger generations may have the potential to tackle the
well-being of elders, support and diversify opportunities for
youth, especially women [4], and relieve the systemic burden
of the super-aging society [5]. Yet there is an increasing gap
between elders and youth. The notion of “filial piety,” or
respect for and care of elders, has sharply declined in the last
20+ years [6]. As more elders return to work post-retirement,
intergenerational tensions are occurring due to the perception
that opportunities are being taken away from youth [7]. A way
forward that tackles these tensions while connecting the
generations would be ideal for intergenerational well-being.
New pressures are leading to the use of technology to
support and enhance the well-being of older and younger
generations, albeit not intergenerationally per se [3]. Post-
retirement work apps (e.g., [8]) are showing that Japanese
elders can and are using modern technology to engage with
society [9]. Games are also being considered as a mechanism
for behavior change [10] and community engagement [11],
such as Pokémon Go for youth experiencing social withdrawal
(hikikomori) [12] and adults with psychological distress [13].
Thus, technology-enabled play could meet intergenerational
well-being needs and facilitate connections across ages.
A primary concern is how best to design for elders [14]
and intergenerationally [15]. We chose to explore a form of
synchrony based on same-time, same-action movement, or
“shared action,” as a design strategy, which has been linked to
increases in prosocial measures between pairs of near-peer
strangers [16], [17], but has not been applied in engineering
practice for a complex activity with multi-generational pairs.
Our goal was thus to explore shared action as a practical
mechanism for empathy-based intergenerational engagement
in a game that uses physical action and movement.

978-1-7281-3888-6/19/$31.00 ©2019 IEEE

 2019 8th International Conference on Affective Computing and Intelligent Interaction (ACII)
Towards this end, a shared action game was developed and
evaluated in a year-long, multiphase exploratory study with
elders and youth. Two interfaces were explored: a finger-
based physical prototype resembling a hand exercise tool
actuated by a smartphone, combining two ideas familiar to
elders; and a smart shoe that looks and feels like a normal shoe
for a novel yet familiar experience. The main contributions
are: (i) the proof-of-concept system; (ii) design knowledge;
and (iii) initial results for intergenerational engagement and
empathy. To the best of our knowledge, this work is the first
to explore shared action as a design strategy in the context of
a physical game for intergenerational empathy.
II. BACKGROUND
A. Intergenerational Technology and Games
While much work has explored designing for and
engaging elders with technology (see e.g., [18]–[22]), less has
considered the intersection of age group or generation as a
factor of design. Also, most has focused on more common
forms of interaction and technologies, particularly common
information communication technologies (ICTs) such as
smartphones or tablets and the Internet and its various
offerings (the Web, chat boards, virtual environments, social
media, etc.). Being mainstream, such technologies are most
readily available to elders, and as many are in the midst of
ageing themselves, they are most familiar to and therefore
may have a higher rate of acceptance by elders. However, such
ICTs tend to provide or allow for physically stationary or
sedentary experiences. While elders may be online engaging
digitally with others, there may not be subsequent effects on
“real life” social engagement [23].
Some work at the intersection of games, play and
technology has shown an alternative [24]. Khoo et al. [25]
created Age Invaders, a family-focused, multi-generation,
inclusive, age-customized mixed reality game for social
interaction and physical action. Elders found the novel mixed
reality format of the game to be inviting and physically
engaging. Seaborn, Pennefather and Fels [26] created a two-
player mixed reality game for elders who use electronic
wheelchairs or scooters that required the elder and a younger
participant to use a wheelchair during the game. This led to a
“role reversal” with the elder taking the lead and helping the
younger, as well as increased understanding and empathy
between the generations. Commercial game systems that
provide diverse user interfaces and interaction possibilities,
such as the Nintendo Wii [27], have also been shown to
mediate positive relations between elder and younger family
members. Taken together, this work inspired us to explore
physical, wearable, and movement-based interaction styles
that are used in in-person contexts.
B. Empathy and Affective Computing
Empathy is defined as an involuntary emotional response
to another person’s experience such that the responder’s
feelings can be said to have merged with the other person’s
[28]. Within affective computing, empathy has been explored
as a detectable emotion that a system can then respond to, e.g.,
[29], an emotion to design for, e.g., [30], and an emotion to be
designed into systems, particularly robots or other computer-
based agents, e.g. [31]. Other work has explored how to create
empathic experiences through body sharing; for instance,
sharing, merging, or swapping visual first-person perspectives
[32], touch [33], heartbeat [34], and breathing [35]. We seek
to add to this literature on designing for empathy using
physical interaction, body movement, and body augmentation
with technology, but without direct body or perception
modification. Instead, we have focused on how to design
technology and interactive experiences by harnessing known
psychological phenomena, in our case a form of synchrony, to
elicit effects on empathy.
C. Synchrony and Shared Action
Synchrony is a social phenomenon in which individuals,
knowingly or otherwise, coordinate their actions in social
interaction [36]. The powerful effects of synchrony and
related phenomena, including joint action [37], interpersonal
coordination [38], and coordinated action [39] are well-
known, with particular effects on cooperation, empathy,
altruism, and other prosocial states and behaviors. Valdesolo
and DeStano [17] found that a simple, repetitive finger-
tapping task generated feelings of similarity, liking, and
compassion as well as provoked empathetic behaviors.
Studies on body rocking from interpersonal coordination [40]
and coordinated action [41] perspectives have showed effects
on cooperation ability and rapport. Such work crafts a
persuasive argument for our ability to use synchrony as a
design strategy: that is, to design for synchrony to bring about
prosocial affective states and behavior. Yet this work, while
striking, has been limited to experimental lab studies with
limited participant pools (for instance, young recruits) and
impractical, boring tasks. We attempted to address these
limitations in this work through a fun game-based format with
intergenerational pairs in a community setting.
In human-computer interaction, some work has explored
synchrony in playful exercise experiences. Ahn et al. [42]
created an exercise game where two people control the speed
and direction of a boat when their physical actions on a
treadmill are synchronized. Similarly, Park and colleagues
[43] created an exercise game based on repetitive aerobics that
used an “interpersonal synchrony,” defined as occurring when
physical movement rhythms and/or physiological responses
are synchronized, albeit not necessarily at exactly the same
time or in the same way, or indeed at the volition of the
players. Alavesa et al. [44] created a similar game involving
two people controlling one avatar through an air cycling
activity while their physical bodies were tied together. While
not exploring a direct impact on empathy, these show how
synchrony can be used as a design strategy and provide an
engineering basis for our work.
This work seeks to build on these efforts by applying the
experimental, lab-based results of shared action synchrony in
design for a real impact on intergenerational empathy. Based
on this literature, we have applied in-phase [39] coordinated
action—same time, same movement—what we call “shared
action,” as a design strategy for generating feelings of
empathy between intergenerational pairs. Next, we describe
how we did so for our shared action game.
III. SYSTEM DESIGN
The shared action game (Fig. 1) is comprised of four main
parts: (i) the finger interface, (ii) the foot interface, (iii) the
display, and (iv) the game controller.
A. Game Controller
The game controller provided the basic gameplay as well
as networked (using Node.js with Socket.io) the various
devices and game displays (created with HTML5) together for
a seamless realtime experience. It also allowed admin control
 2019 8th International Conference on Affective Computing and Intelligent Interaction (ACII)
over the experimental conditions (interface version and shared
action mode). The game was modelled on the ideas generated
in the idea jam (Phase I)—nature-based, physical, puzzle
game—as well as the beat-based synchrony task in one of the
empathy-building studies [17]. The objective of the game was
to co-create flowers by matching movements (finger taps or
foot gestures) to a modulated rhythmic beat.
B. Finger Version
Daily self-directed exercise is common among Japanese
elders. Many elders are starting to use finger-strengthening
equipment to enhance physical ability and prevent injury from
such exercises. We created a prototype similar to these that
uses rubber bands for the fingers and provide physical
feedback. One downside of current devices is the lack of
visual feedback, which requires users to practice with a
therapist. We realized that smartphones, which are being
increasingly used by elders, could detect finger taps as well as
provide visual feedback. The first version of the prototype
(Fig. 1, left) integrated these requirements, but when Study 1
(S1) and Study 2 (S2) results indicated that the visual feedback
was distracting from the synchronization effect, we reduced
these in the final version (Fig. 1, center). Other improvements
were applied: (i) transparent acrylic boards to see each other’s
fingers and the screen more easily; (ii) larger rubber bands for
stronger support; (iii) multiple height levels for placing the
rubber bands to support diversity in body size; (iv) an easier
way to install the smartphone; and (v) a higher base for users
to more comfortably place their wrist. Tapping gestures were
captured with a HTML5-powered app.
C. Foot Version
A shoe as input method (Fig. 1, right) was explored as a
form factor familiar to elderly people. We used a proprietary
smart shoe named Orphe 1 . It is equipped with inertial
measurement units (IMU) that provide accurate realtime
sensor data, such as acceleration, velocity, gravity, and
orientation. The shoes also have built-in LEDs that can
simulate different light patterns depending on the detected
foot gesture. By using the Orphe framework, we were able to
detect four different foot gestures: flat step, heel step, toe step,
and kick. However, due to technical limitations related to the
accuracy of the flat step and kick movements, we extended the
existing framework to include two new gestures: double heel
and double toe steps. We also developed a macOS app that
communicates with the shoes of both players via Bluetooth
using the Open Sound Control (OSC) protocol. The app tracks
foot movements, detects the four different gestures for both
feet of both players, and forwards these to the game controller.
D. Display
In S1 and S2, a Microsoft Surface tablet was used to
present the shared game screen for both versions. In S3, the
tablet was used only for the finger version; for increased
visibility, a floor projector was used for the foot version.
IV. METHODS
A year-long, multipart, exploratory mixed methods study
founded on a human factors design and evaluation approach
[45] was conducted in four phases: (i) three idea jams with
elders, youngers, and experts in gerontology (n=7) for needs
analysis and requirements gathering to inform the design of
the system and prototype technology selection, leading to the
1
https://orphe.shoes
selection of finger and foot input methods and the generation
of over thirty age- and culture-relative ideas; (ii) an initial
study with youngers (S1); (iii) a usability study with elders
(S2); (iv) and the full intergenerational study with pairs of
youngers and elders (Study 3, or S3). All studies used a 2x2
factorial design with interface version (finger or foot) as the
within-subjects factor and shared action mode (shared or not
shared) as the between-subjects factor. All studies were
approved by the university ethics committee.
A. Participants and Location
S1 involved six pairs of youngers (n = 12, women, or W =
2, men, or M = 10, Med Age = 23) and was conducted in-lab.
S2 involved twelve pairs of elders (n = 24, W = 15, M = 9,
Med Age = 67) and was also conducted in-lab. S3 had twenty
intergenerational pairs (n = 39) of elders (n = 20, W = 4, M =
16, Med Age = 71) and youngers (n = 19, W = 7, M = 13, Med
Age = 23.5) participating in non-lab rooms at a university
gerontology center. Due to a sudden cancellation, one of the
research assistants acted as a younger participant in one pair.
Participants in S1 knew each other, but participants in S2 and
S3 did not necessarily know each other. Elderly participants
were recruited through a job and volunteer service, while
youngers were recruited through a university mailing list. All
were given ￥1200 (~$10 USD) per hour plus transportation.
B. Materials and Measures
A series of inventories related to synchronized action for
empathy and game experience were embedded in pre- and
post-play questionnaires, with most translated from English to
Japanese. All participants completed a pre-questionnaire on
demographics, such as age and gender, which were used as
independent variables (IVs); in S3, the Japanese version [46]
of the Interpersonal Reactivity Index (IRI) was used to capture
dispositional empathy as a possible explanatory IV [47]. A
post-questionnaire about the game experience and the
interface used (finger or foot) included the Enjoyment and
Social Presence modules from the Game Experience
Questionnaire (GEQ) [48] as dependent variables (DVs)
related to synchronized cooperative game experiences.
Similarity (two translations, 1 and 2) and liking, from the
original research [17], and rapport [49] as a subjective social
measure of empathy were also captured as DVs. Open-ended
questions included a comparison question about the interface
(finger and foot), likes and dislikes about the game experience,
and any other thoughts, impressions, or ideas. These as well
as video and audio were captured and/or transcribed for
qualitative analysis and illustration of attitudes, perceptions,
patterns of behavior, and to help explain quantitative findings.
C. Procedure
All studies followed the same general procedure. Each pair
participated in one 1.25-hour session in which they played the
shared action game twice with both interface versions (with
the order counterbalanced across pairs to account for learning
effects) but only one shared action mode: shared action (SA),
where the game encouraged same-time, same-action input, or
non-shared action (non-SA), where the game encouraged out-
of-phase input timing and provided different sets of
instructions to each player. After each game, they completed
the post-play questionnaire, and after experiencing both
versions, they were asked to compare the two. They were then
debriefed about the goals of the study and compensated.
 2019 8th International Conference on Affective Computing and Intelligent Interaction (ACII)
Fig. 2. Difficulty (left) and social presence (right) for youngers in S1.
D. Data Analysis
Correlations, t-tests, mixed analyses of variance, and
regressions were used to explore quantitative data. Chi-square
analyses were used to check whether groups were even for
each factor; gender was found to be uneven (F = 10, M = 23)
but gender composition of pairs was not (same = 21, different
= 12). As such, non-parametric statistical tests were used
where possible. In S3, data from seven participants was
removed due to missing data, no-shows, or evaluating only
one prototype. For the qualitative data, word frequency
analyses were used to guide the thematic analysis, for which
all themes between two raters achieved κ > .79.
V. RESULTS
A. Study 1: Youngers
In the first study2, pairs of youngers evaluated the game
and prototypes. Comparing difficulty for the finger version
(Med = 2, IQR = 1, Mean = 2.1, STD = 0.4) and foot version
(Med = 4, IQR = 1.25, Mean = 4.0, STD = 0.8), a significant
difference was observed, t(6) = 5.35, p < 0.01, where the foot
version was rated more difficult than the finger version (Fig.
2, left). According to qualitative comments, this may have
been due to the smart shoe’s gesture detection problems and
the use of headphones (which prevented dialogue among
players when difficulties arose).
Regardless of SA mode, participants experienced greater
social presence in the foot version (Med = 3, IQR = 1.25,
Mean = 2.6, STD = 1.2) compared to the finger version (Med
= 2, IQR = 2, Mean = 2, STD = 1.4), via the Friedman test,
χ2(1) = 4, p = .046. However, only the finger version showed
an effect of SA on social presence via the Wilcoxon signed-
rank test, Z = -2.43, p = .015 (Fig. 2, right). For youngers, it
seems the finger prototype facilitated a stronger SA effect.
B. Study 2: Elders
In the second study with pairs of elders, comparing
difficulty for the finger version (Med = 1, IQR = 3, Mean =
1.8, STD = 1.1) and foot version (Med = 2, IQR = 4, Mean =
2.1, STD = 1.1) yielded no significant difference between
interfaces (Fig. 3, left), unlike in S1. There was also no
significant difference between the finger version (Med = 4,
IQR = 2, Mean = 3.9, STD = 1.0) and foot version (Med = 4,
IQR = 2, Mean = 3.8, STD = 0.7) for game enjoyment, and
scores were high for both interfaces.
For social presence, a significant difference was found
between the finger version (Med = 3, IQR = 2, Mean = 2.6,
STD = 0.9) and foot version (Med = 3, IQR = 3, Mean = 3.0,
STD = 1.0), via the Wilcoxon signed-rank test, Z = -2.24, p =
.025, indicating that the foot version had higher social
2
Originally presented in Japanese as a technical report without peer
review at the annual 2018 Virtual Reality Society of Japan conference [50].
Fig. 3. Difficulty (left) and social presence (right) for elders in S2.
presence (Fig. 3, right), as in S1. However, neither showed a
SA effect, unlike S1 for the finger version. Elders’ comments
suggested several reasons why. The finger version forced
concentration on the game, was tiring to use, and blocked the
line of sight. The foot version’s detection inaccuracy remained
an issue despite lower difficulty ratings than S1, and foot
visibility was reduced due using chairs. Since the SA effect is
theorized to rely on visual perception of similarity [51], we
concluded that visibility of both prototypes needed to be
improved, and so the finger version and environmental setup
was redesigned (as in III. B.) for maximum visibility.
C. Study 3: Intergenerational
For dispositional empathy, scores did not differ based on
gender, nationality, or age when considering the full scale.
However, scores differed by gender for two of its subscales:
empathic concern, U = 167, p = .042, and perspective-taking,
U = 189, p = .003, with women rating higher self-scores than
men. They were thus included as covariates when gender was
considered in the analysis.
Unlike in S1 or S2, there were no significant differences
found for difficulty, enjoyment, or social presence based on
interface version or SA mode. A mixed ANCOVA for rapport
found a significant main effect of interface on SA mode, F(1,
24) = 4.85, p = .038 (Fig. 4, left). The estimated marginal
means suggest that rapport was higher for the foot version in
SA mode; the opposite trend was seen for the non-SA mode,
with higher rapport for the finger version. There were
significant main effects of enjoyment, F(1, 24) = 11.91, p =
.002, similarity 2, F(1, 24) = 7.54, p = .011, and the interaction
of SA mode and pair gender composition, F(1, 24) = 6.00, p =
.022. Rapport and similarity 2 will be interpreted later as
regression models [52].
The interaction of SA mode and gender was also found in
a multivariate model including social presence, F(2, 16) =
7.48, p = .005, Wilk's Λ = .518, partial η2 = .48 (Fig. 4, right).
The estimated marginal means suggest that, for both factors,
pair gender composition interacted with SA mode. In SA
mode, rapport and social presence were higher when pairs had
the same gender; the effect was not seen in non-SA mode.
Several univariate effects were found for rapport and
social presence. For rapport, significant main effects were
found with similarity 2, F(1, 24) = 7.54, p = .011, and
enjoyment, F(1, 24) = 11.91, p = .002. For social presence,
enjoyment, F(1, 21) = 9.57, p = .006, liking, F(1, 21) = 10.19,
p = .004, and similarity 2, F(1, 21) = 6.55, p = .018. These will
be interpreted in the regression analysis next.
 2019 8th International Conference on Affective Computing and Intelligent Interaction (ACII)

Fig. 4. Rapport (left) and rapport and social presence (right) in S3.

Fig. 5. Path analysis linking pair and game factors with empathy variables
A simple model based on [17] of SA mode, synchrony for intergenerational pairs in S3. Dashed lines indicate theoretical predictors
factors—liking and similarity, with two versions translated to that are not significantly correlated with the DV. * p < .05. ** p < .01.
Japanese: feeling similar (1) and similarity of character (2)—
and empathy variables was not significant. Thus, we explored, of similarity 2 led to higher rapport. The addition of higher
based on the above analyses of variance, a new explanatory reports of liking also led to higher social presence.
model. Table 1 shows regressions found between pair
(individual and pair demographic and synchrony The word frequency analysis of 87 comments from the
characteristics), game (experiential) factors, and empathy post-play questionnaires resulted in four clusters: social focus
variables (rapport and social presence). A path analysis was (36), comprised of references to partner (24), the other person
then constructed to depict the relationships between these (6), and myself (6); learning (35), comprised of problems with
regression models and their factors (Fig. 5). understanding (24) or the instructions (5), lack of guidance
(3), or desire for more training (2); hardware (9), comprised
TABLE I. SIGNIFICANT REGRESSIONS (N = 33) IN S3. of sensor (5) and detection (4) problems; and challenge (24),
comprised of difficulty (19) and hard (5). For the interfaces,
DV Predictor t Β F df p adj. R2
15 preferred the foot, 11 preferred the finger, and 8 liked both
Enjoyment Overall Model 5.39 1,31 .027 .121 equally, with the rest not holding a strong opinion any way.
Difficulty -2.32 -.39 .027 The positive feelings (20) theme was characterized by
Similarity 1 Overall Model 4.92 2,30 .014 .197
having fun, especially with the shoes; a sense of cooperation
and synchrony, especially for the SA mode; and a sense of
Gender 2.74 .49 .010 accomplishment, especially at seeing the flower at the end of
Age Difference 2.61 .46 .014
each level. The empathic feelings (24) theme had comments
from elders about their younger partner (5) and vice versa
Similarity 2 Overall Model 6.09 3,29 .002 .323 (19). Elders felt attentive and empathetic towards youngers in
Age 2.77 .41 .010
the shoe version regardless of SA mode, and not during the
non-SA mode regardless of interface. Youngers’ comments
Pair Gender 3.07 .47 .005 focused on the exercise benefits for elders or concern about
Gender 2.69 .42 .012
elders due to a perception that the game or interface, especially
the finger version, was more difficult for elders.
Liking Overall Model 11.61 2,30 .000 .399
VI. DISCUSSION
Similarity 1 3.57 .51 .001
Shared action as a design strategy, regardless of interface
Gender 2.12 .30 .042 version, appears to have been influenced by multiple factors
Rapport Overall Model 17.00 2,30 .000 .500
including and beyond the interface version and game format,
such as individual differences and pair characteristics. We
Enjoyment 3.87 .50 .001 discuss and synthesize the findings from all studies here.
Similarity 2 3.20 .42 .003 For intergenerational pairs, age and gender had indirect
Social
Overall Model 14.10 3,29 .000 .551 and intersecting effects that influenced the synchrony factors
Presence of similarity and liking more than SA mode. Regardless of
Enjoyment 3.69 .46 .001 interface, shared action was most successful at shaping
Liking 3.28 .39 .002
subjective empathy when the gender makeup of the pair was
the same, e.g., two women or two men. This is not surprising
Similarity 2 3.35 .41 .003 given the strict yet unexamined gender divisions in Japanese
society. Indeed, the qualitative analysis did not unearth any
The difficulty of the game affected enjoyment, with higher insights about gender; even when youngers expressed concern
difficulty leading to lower enjoyment. Within the pair, higher about their elderly partner, this was based on an age-based
scores for similarity 1 were explained by higher age perception of difficulty, with no comment on gender.
differences, especially for women. For similarity 2, having the However, the path analysis for S3 (Fig. 5) indicates that age
same gender led to higher scores, especially for youngers and interacted with gender. Elderly women had higher
women. Also, elders, especially women, tended to report impressions of similarity and liking regardless of age, but
feeling more similar to their partner than youngers. Higher youngers responded better in same-gender pairs. Still, these
reports of similarity 1 led to higher reports of liking, especially novel findings may not just be due to the cultural context of
for women. Enjoying the game experience and higher ratings Japan, but also because previous research may not have
 2019 8th International Conference on Affective Computing and Intelligent Interaction (ACII)
included diverse ages or many women; indeed, the participant
makeup of the studies that found a shared action effect were
with youngers who were mostly male (S1) or with same-
gender pairs (S3), seemingly matching the participant makeup
of the original research [17]. Future work on designing
intergenerational shared action games will need to explore
how to increase feelings of similarity in youngers towards
elders and within non-gender-equal pairs.
Across all studies, findings indicate that form factor and
interaction style need to be carefully considered in light of
game difficulty differences, especially based on age. In
general, the foot version had higher empathy factor scores,
even though it could not be designed to elicit a shared action
effect. The finger version was less stable, showing a shared
action effect with youngers, but not elders; further, in
intergenerational pairs, it garnered lower scores in the shared
action mode compared to the non-shared action mode. The
difficulty of the finger version may be explained in part by
elders’ finger festination—tapping too early and therefore out
of sync—which is tied to age [53]. Other findings and the
thematic analysis provide further explanation. Regardless of
age and SA mode, participants enjoyed and felt high social
presence, if not synchrony, when using the smart shoes. As
one younger (P5b) explained, “The visual information about
my partner's movements made it easier to feel empathetic.”
Elders’ experience with the finger version may have impacted
both in the pair: elders being distracted by the difficulty, which
youngers noticed and were therefore distracted by—but,
ironically, this may be considered a form of empathizing.
The results, especially in light of the qualitative findings,
suggest that there may be a designable tradeoff between
challenge and empathetic response in a shared action game
experience. Although the game and interfaces were not
designed to be difficult or hard to use, and indeed much effort
was put towards preventing this, many elders did experience
difficulties—and not only did youngers notice, elders noticed
youngers noticing, as one (P19a) wrote: “My partner looked
worried and concerned about me.” Many game and experience
designers aim for a flow model [54] where difficulty is
modulated with respect to users’ skill to achieve optimal
experience (not too difficult and not too boring). However,
these findings suggest another reading of the flow model with
respect to designing for building empathy between the
generations: modulating difficulty for elders to encourage an
empathetic response in youngers. Although this was
unintentional here, previous research on intergenerational
games suggests the same would be true in reverse, for elders
to youngers, such as in the example of the role reversal
observed in [26]. A design challenge in future work would be
to create an experience, playful or otherwise, that would have
both generations perceiving each other as experiencing
difficulty, thereby simultaneously eliciting an empathetic
response in elders and youngers for each other.
A. Limitations
Some results need to be interpreted with caution. Gender
had uneven groups, which could have affected the multiple
factor ANOVAs. Additionally, the n was low for the number
of factors analyzed, particularly for the multiple regressions
with more than two predictors, and should be doubled in future
analyses. A larger pool of participants and even numbers of
men and women can confirm these initial results. Finally,
participants in the last study did not necessarily know each
other; since some research suggests that familiarity may affect
feelings of similarity [55], future work could compare pairs of
strangers and pairs already familiar with each other.
VII. CONCLUSIONS
Eliciting an effect on subjective empathy-related factors
by manipulating the design of two versions of a shared action
game was not straightforward. Findings from the three studies
show that the shared action effect was influenced by
individual and pair factors. Still, we can tentatively conclude
from these results that the foot version of the game was most
suitable for intergenerational play, given the context (notably,
maximized visibility), difficulty level, and training, especially
for elders. Future work may refine the detection algorithms,
simplify the game for elders, consider matched-gender pairs,
and explore the effect on intergenerational attitudes, notably
changing social structures and elders’ return to work.
ACKNOWLEDGMENT
Thanks to Naohira Eda, Seito Matsubara, and Ken Sakuma
for research assistant support. Thanks to Second Life Factory
(Akira Nakaya) for recruiting. Thanks to the Kashiwa Elders’
English Club, notably Koudo Ohno, for translation support
and assistance. Thanks to the Inami-Hiyama Lab for general
assistance, notably Koki Katsumata, Tomoya Sasaki, Kenta
Toshima, “Maark” Kao-Hua Liu, Atsushi Izumihara, Sohei
Wakisaka, and Yasuko Otaki for admin, translation, technical,
and research support. Thanks to Peter Pennefather for
reviewing drafts of this paper. This work was supported by the
Japanese Society for the Promotion of Science (JSPS) and
Japanese Science and Technology Agency (JST).
REFERENCES
[1] Statistics Bureau, “Population estimates by age (5-year age group) and
sex,” Ministry of Internal Affairs and Communications, Tokyo, Japan,
Jul. 2016.
[2] T. Obi, D. Ishmatova, and N. Iwasaki, “Promoting ICT innovations for
the ageing population in Japan,” Int. J. Med. Inf., vol. 82, no. 4, pp.
e47–e62, 2013.
[3] N. Tamiya et al., “Population ageing and wellbeing: Lessons from
Japan’s long-term care insurance policy,” The Lancet, vol. 378, no.
9797, pp. 1183–1192, 2011.
[4] H. Takeda, “Reforming families in Japan: Family policy in the era of
structural reform,” Home Fam. Jpn. Contin. Transform., pp. 46–64,
2011.
[5] M. Kaplan, A. Kusano, I. Tsuji, and S. Hisamichi, Intergenerational
Programs: Support for Children, Youth, and Elders in Japan. SUNY
Press, 1998.
[6] W. Koyano, “Intergenerational relationships of Japanese seniors:
Changing patterns,” Glob. Aging Chall. Fam., pp. 272–283, 2003.
[7] A. Kondo, “Effects of increased elderly employment on other workers’
employment and elderly’s earnings in Japan,” IZA J. Labor Policy, vol.
5, no. 1, p. 2, Jan. 2016.
[8] N. Lee, K. Seaborn, A. Hiyama, M. Inami, and M. Hirose, “Evaluating
a smartphone-based social participation app for the elderly,” in Proc.
HCI Int. 2018, Las Vegas, NV, 2018, vol. 26, pp. 505–517.
[9] Ministry of Internal Affairs and Communication, “Preliminary results
of Communications Usage Trend Survey 2010,” 2010.
[10] E. A. Boyle et al., “An update to the systematic literature review of
empirical evidence of the impacts and outcomes of computer games
and serious games,” Comput. Educ., vol. 94, pp. 178–192, 2016.
[11] M. Y. Pang, J. J. Eng, and W. C. Miller, “Determinants of satisfaction
with community reintegration in older adults with chronic stroke: Role
of balance self-efficacy,” Phys. Ther., vol. 87, no. 3, pp. 282–291,
2007.
[12] M. Tateno, N. Skokauskas, T. A. Kato, A. R. Teo, and A. P. S.
Guerrero, “New game software (Pokémon Go) may help youth with
severe social withdrawal, hikikomori,” Psychiatry Res., vol. 246, pp.
848–849, Dec. 2016.
 2019 8th International Conference on Affective Computing and Intelligent Interaction (ACII)
[13] K. Watanabe et al., “Pokémon GO and psychological distress, physical
complaints, and work performance among adult workers: A
retrospective cohort study,” Sci. Rep., vol. 7, no. 1, p. 10758, Sep. 2017.
[14] T. Nishida, J. B. Pick, and A. Sarkar, “Japan’s prefectural digital
divide: A multivariate and spatial analysis,” Telecommun. Policy, vol.
38, no. 11, pp. 992–1010, 2014.
[15] E. T. Khoo, T. Merritt, and A. D. Cheok, “Designing physical and
social intergenerational family entertainment,” Interact. Comput., vol.
21, no. 1–2, pp. 76–87, 2008.
[16] S. Järvelä, J. M. Kivikangas, J. Kätsyri, and N. Ravaja, “Physiological
linkage of dyadic gaming experience,” Simul. Gaming, vol. 45, no. 1,
pp. 1–17, 2014.
[17] P. Valdesolo and D. DeSteno, “Synchrony and the social tuning of
compassion,” Emotion, vol. 11, no. 2, pp. 262–266, 2011.
[18] S. E. Lindley, R. Harper, and A. Sellen, “Designing for elders:
Exploring the complexity of relationships in later life,” in Proc. 22nd
British HCI Group Annu. Conf. People and Computers, 2008, pp. 77–
86.
[19] J. Zhou, P.-L. P. Rau, and G. Salvendy, “Use and design of handheld
computers for older adults: A review and appraisal,” Int. J. Hum.-
Comput. Interact., vol. 28, no. 12, pp. 799–826, 2012.
[20] T. T. Cota, L. Ishitani, and N. Vieira, “Mobile game design for the
elderly: A study with focus on the motivation to play,” Comput. Hum.
Behav., vol. 51, pp. 96–105, 2015.
[21] M. S. Al-Razgan, H. S. Al-Khalifa, M. D. Al-Shahrani, and H. H.
AlAjmi, “Touch-based mobile phone interface guidelines and design
recommendations for elderly people: A survey of the literature,” in
Proc. 19th Int. Conf. Neural Information Processing, Doha, Qatar,
2012, pp. 568–574.
[22] S. Liu and S. Joines, “Developing a framework of guiding interface
design for older adults,” in Proc. Human Factors and Ergonomics
Society Annu. Meeting, 2012, vol. 56, pp. 1967–1971.
[23] R. Hill, L. R. Betts, and S. E. Gardner, “Older adults’ experiences and
perceptions of digital technology: (Dis)empowerment, wellbeing, and
inclusion,” Comput. Hum. Behav., vol. 48, pp. 415–423, 2015.
[24] L. Costa and A. Veloso, “Being (grand) players: review of digital
games and their potential to enhance intergenerational interactions,” J.
Intergenerational Relatsh., vol. 14, no. 1, pp. 43–59, 2016.
[25] E. T. Khoo, A. D. Cheok, T. H. D. Nguyen, and Z. Pan, “Age Invaders:
Social and physical inter-generational mixed reality family
entertainment,” Virtual Real., vol. 12, no. 1, pp. 3–16, Mar. 2008.
[26] K. Seaborn, P. S. Pennefather, and D. Fels, “Eudaimonia and hedonia
in the design and evaluation of a cooperative game for psychosocial
well-being,” Human–Computer Interact., 2019.
[27] A. Voida and S. Greenberg, “Console gaming across generations:
Exploring intergenerational interactions in collocated console
gaming,” Univers. Access Inf. Soc., vol. 11, no. 1, pp. 45–56, Mar.
2012.
[28] J. E. Escalas and B. B. Stern, “Sympathy and empathy: Emotional
responses to advertising dramas,” J. Consum. Res., vol. 29, no. 4, pp.
566–578, 2003.
[29] T. Chaspari et al., “Exploring sparse representation measures of
physiological synchrony for romantic couples,” in Proc. 2017 7th Int.
Conf. Affective Computing and Intelligent Interaction (ACII), San
Antonio, TX, 2017, pp. 267–272.
[30] A. Toet et al., “Reach out and touch somebody’s virtual hand:
Affectively connected through mediated touch,” in Proc. 2013
Humaine Assoc. Conf. Affective Computing and Intelligent Interaction,
Geneva, Switzerland, 2013, pp. 786–791.
[31] L. D. Riek, T. Rabinowitch, B. Chakrabarti, and P. Robinson,
“Empathizing with robots: Fellow feeling along the anthropomorphic
spectrum,” in Proc. 2009 3rd Int. Conf. Affective Computing and
Intelligent Interaction, Amsterdam, Netherlands, 2009, pp. 1–6.
[32] Y. Lee, K. Masai, K. Kunze, M. Sugimoto, and M. Billinghurst, “A
Remote Collaboration System with Empathy Glasses,” in Proc. 2016
IEEE Int. Symp. Mixed and Augmented Reality (ISMAR-Adjunct),
2016, pp. 342–343.
[33] S. Singhal, C. Neustaedter, Y. L. Ooi, A. N. Antle, and B. Matkin,
“Flex-N-Feel: The design and evaluation of emotive gloves for couples
to support touch over distance,” in Proc. 2017 ACM Conf. Computer
Supported Cooperative Work and Social Computing, 2017, pp. 98–
110.
View publication stats
[34] L. Heiss, “Enabled apparel: the role of digitally enhanced apparel in
promoting remote empathic connection,” AI Soc., vol. 22, no. 1, pp.
15–24, 2007.
[35] J. Kim, Y.-W. Park, and T.-J. Nam, “BreathingFrame: An inflatable
frame for remote breath signal sharing,” in Proc. 9th Int. Conf.
Tangible, Embedded, and Embodied Interaction, 2015, pp. 109–112.
[36] E. Delaherche, M. Chetouani, A. Mahdhaoui, C. Saint-Georges, S.
Viaux, and D. Cohen, “Interpersonal synchrony: A survey of
evaluation methods across disciplines,” IEEE Trans. Affect. Comput.,
vol. 3, no. 3, pp. 349–365, 2012.
[37] N. Sebanz, H. Bekkering, and G. Knoblich, “Joint action: Bodies and
minds moving together,” Trends Cogn. Sci., vol. 10, no. 2, pp. 70–76,
2006.
[38] F. J. Bernieri and R. Rosenthal, “Interpersonal coordination: Behavior
matching and interactional synchrony,” in Studies in Emotion & Social
Interaction: Fundamentals of Nonverbal Behavior, R. S. Feldman and
B. Rimé, Eds. New York, NY: Cambridge University Press, 1991, pp.
401–432.
[39] L. K. Miles, L. K. Nind, and C. N. Macrae, “The rhythm of rapport:
Interpersonal synchrony and social perception,” J. Exp. Soc. Psychol.,
vol. 45, no. 3, pp. 585–589, May 2009.
[40] M. J. Richardson, K. L. Marsh, R. W. Isenhower, J. R. Goodman, and
R. C. Schmidt, “Rocking together: Dynamics of intentional and
unintentional interpersonal coordination,” Hum. Mov. Sci., vol. 26, no.
6, pp. 867–891, 2007.
[41] P. Valdesolo, J. Ouyang, and D. DeSteno, “The rhythm of joint action:
Synchrony promotes cooperative ability,” J. Exp. Soc. Psychol., vol.
46, no. 4, pp. 693–695, Jul. 2010.
[42] M. Ahn et al., “Swan boat: Pervasive social game to enhance treadmill
running,” in Proc. 17th ACM Int. Conf. Multimedia, Beijing, China,
2009, pp. 997–998.
[43] T. Park et al., “ExerSync: Facilitating interpersonal synchrony in social
exergames,” in Proc. 2013 Conf. Computer Supported Cooperative
Work, San Antonio, Texas, 2013, pp. 409–422.
[44] P. Alavesa, J. Schmidt, A. Fedosov, R. Byrne, and F. “Floyd” Mueller,
“Air Tandem: A collaborative bodily game exploring interpersonal
synchronization,” in Proc. 2015 Annu. Symp. Computer-Human
Interaction in Play, London, United Kingdom, 2015, pp. 433–438.
[45] W. E. Woodson, B. Tillman, and P. Tillman, Human Factors Design
Handbook: Information and Guidelines for the Design of Systems,
Facilities, Equipment, and Products for Human Use. New York, NY:
McGraw-Hill, 1992.
[46] T. Himichi et al., “Development of a Japanese version of the
Interpersonal Reactivity Index,” Shinrigaku Kenkyu, vol. 88, no. 1, pp.
61–71, 2017.
[47] M. H. Davis, “A multidimensional approach to individual differences
in empathy,” JSAS Cat. Sel. Doc. Psychol., vol. 10, p. 85, 1980.
[48] W. A. IJsselsteijn, Y. A. W. De Kort, and K. Poels, “The game
experience questionnaire.” Technische Universiteit Eindhoven, 2013.
[49] N. M. Puccinelli and L. Tickle-Degnen, “Knowing too much about
others: Moderators of the relationship between eavesdropping and
rapport in social interaction,” J. Nonverbal Behav., vol. 28, no. 4, pp.
223–243, 2004.
[50] K. Seaborn, N. Lee, M. Narazani, C. Chien-Hsu, A. Hiyama, and M.
Inami, “Design and evaluation of a shared action game for empathy,”
in Proc. 23rd Annu. Conf. Virtual Reality Society in Japan, Sendai,
Tohoku, Japan, 2018.
[51] K. L. Marsh, M. J. Richardson, and R. C. Schmidt, “Social connection
through joint action and interpersonal coordination,” Top. Cogn. Sci.,
vol. 1, no. 2, pp. 320–339, Apr. 2009.
[52] A. Field, Discovering Statistics Using SPSS. Thousand Oaks, CA:
Sage, 2009.
[53] H. Nagasaki, H. Itoh, K. Hashizume, T. Furuna, H. Maruyama, and T.
Kinugasa, “Walking patterns and finger rhythm of older adults,”
Percept. Mot. Skills, vol. 82, no. 2, pp. 435–447, 1996.
[54] M. Csikszentmihalyi, Flow: The Psychology of Optimal Experience.
New York, New York, USA: Harper Perennial, 1991.
[55] Y. Motomura, A. Takeshita, Y. Egashira, T. Nishimura, Y. Kim, and
S. Watanuki, “Interaction between valence of empathy and familiarity:
Is it difficult to empathize with the positive events of a stranger?,” J.
Physiol. Anthropol., vol. 34, no. 1, p. 13, 2015.