Sandgreen Et Al (2020) Meta-Analysis Digital Interventions

④
Journal of Autism and Developmental Disorders

MATO8
https://doi.org/10.1007/s10803-020-04778-9 Karina Cinel 2023
ORIGINAL PAPER
Digital Interventions for Autism Spectrum Disorder: A Meta-analysis

Helena Sandgreen1,2,3 · Line Hofmann Frederiksen1,2,3,4 · Niels Bilenberg1,3
Accepted: 29 October 2020

© Springer Science+Business Media, LLC, part of Springer Nature 2020
Abstract
This study aimed to review digital interventions in the treatment of autism spectrum disorder (ASD). A systematic review
and meta-analysis was conducted. Nineteen studies were included. The interventions aimed to improve social skills (n = 11),
developmental skills (n = 2) and 6 other different targets. Technology used were computer programs (n = 14), tablet apps
(n = 3), a robot (n = 1) and an interactive DVD (n = 1). The meta-analysis resulted in an overall effect size (Cohen’s d) of
0.32 [0.12–0.51], indicating a small effect. Heterogeneity between studies was high (I2 = 100%), limiting the generalization
of results. Therefore, we recommend larger RCT studies, and guidelines for the development of trials evaluating digital
interventions for ASD, for making comparison of future studies possible.
Registration can be found online at Prospero: https://www.crd.york.ac.uk/prospero/, registration no. CRD42020146542.
Keywords Autism · Digital · Technology · Computer · Meta-analysis · Systematic review
Autism spectrum disorder (ASD) is characterized by difficul- smartphones, wearable technologies, virtual reality, robotics
-
ties in social communication and unusually restricted, repeti- and tablets (Odom et al. 2015) have been used. The interven-
tive behavior and interests. The most effective interventions tions attempt to teach or train e.g. communication (Ram-
are developmental and behavioral approaches (Sandbank doss et al. 2011), social and emotional skills (Ramdoss et al.
et al. 2020). However, many families are unable to access 2012) and academic skills (Root et al. 2017). The use of
interventions due to time consumption and substantial costs digital interventions may be beneficial for autistic individu-
(Peters-Scheffer et al. 2012; Rogge and Janssen 2019). als, since such interventions are consistent, predictable, and
An increasing amount of research has investigated the use without social interaction, which is preferred by autistic indi-
of digital interventions for support and treatment of autis- viduals (Golan and Baron-Cohen 2006). Moreover, digital
tic individuals under terms such as digital health (Hollis interventions on commonly used devices such as comput-
et al. 2017), computer-based (Ramdoss et al. 2011, 2012), ers and tablets have the potential to be much cheaper than
computer-assisted (Root et al. 2017), innovative technol- individual directed live therapy with a clinician (Ramdoss
ogy-based (Grynszpan et al. 2014) and technology-aided et al. 2012).
interventions (Odom et al. 2015). Devices like computers, A meta-analysis investigating research about digital inter-
ventions for ASD was conducted by Grynszpan et al. (2014)
including articles published until 2012. Since there has been
* Helena Sandgreen a rapid growth in the development and evaluation of digital
helena.sandgreen@gmail.com interventions for mental health problems (Hollis et al. 2017),
1
an updated meta-analysis on the subject is of relevance. This
Department of Child and Adolescent Mental Health Odense,
Research Unit (University Function), Mental Health Services
meta-analysis is largely inspired by Grynszpan et al. (2014),
in the Region of Southern Denmark, J. B. Winsløws Vej 16, but is not an exact continuation.
entrance 230, 5000 Odense C, Denmark The aim of this study was to conduct a systematic review
2
Psychiatric Research Academy, Mental Health Services and meta-analysis of digital interventions for ASD, by deter-
in the Region of Southern Denmark, Odense, Denmark mining (a) the digital devices used, (b) the skills targeted,
3
Department of Clinical Research, University of Southern (c) the effect size of the interventions and (d) quantity and
Denmark, Odense, Denmark quality of research supporting this.
4
Department of Internal Medicine, Aabenraa Hospital,
Aabenraa, Denmark
13
Vol.:(0123456789)
PICO of the study was: journals. Exclusion criteria were (a) Interventions based
only on conversation with a therapist on a device. (b) Stud-
Population Individuals with autism spectrum disorder in all age ies that assessed improvements in training material without
groups
using external outcome measures. (c) Studies that relied on
Intervention Interactive digital interventions single or multiple single-case designs.
Comparison Control group of individuals with autism spectrum Data extraction was conducted including reference and
disorder, not receiving a targeted digital intervention characteristics about study design, participants, interven-
Outcome Quantitative measure of skill tions and outcomes. During data extraction additional exclu-
sion criteria proved necessary, due to incomparable study
designs. Additional exclusion criteria were (a) Studies in
which the digital intervention was not the main part of the
Methods intervention. (b) Studies comparing two interventions. These
were excluded since a potential effect would be concealed,
Study Design due to that the effect size measured (Cohen’s d) is based on
differences on postoutcomes from the intervention group
This is a systematic review and meta-analysis regarding tri- and control group.
als of digital interventions in the treatment of ASD. The
protocol of the study can be found at https://www.crd.york. Quality Assessment
ac.uk/prospero/, registration no. CRD42020146542. This
systematic review was conducted in accordance with the The methodological quality assessment was examined using
PRISMA standard (Moher et al. 2009). the Revised Cochrane risk-of-bias tool for randomized tri-
als (RoB 2) (Sterne et al. 2019). Studies were assessed in
Search Procedure 5 domains: randomization process, deviations from the
intended interventions (effect of assignment to interven-
A systematic literature search was conducted in the online tion), missing outcome data, measurement of the outcome,
databases Ovid MEDLINE, Ovid Embase and PsycINFO and selection of the reported result. Each study was given an
on the 15th of June 2019. Keywords used for searching overall Rob 2 rating of low risk, some concerns or high risk.
were ’Autism’, ’digital’ and ’trials’ and variations of these.
MeSH terms, free text, and MeSH trees were used in the Data Synthesis and Statistical Analysis
search. Full search string can be found at https://www.crd.
york.ac.uk/PROSPEROFILES/146542_STRATEGY_20190 Stata 16 was used for the statistical analyses. Standardized
816.pdf. Furthermore, a hand search was conducted of the effect sizes were computed for all studies, due to the diver-
reference lists of included studies. sity between outcome measures. The effect sizes were meas-
ured with Cohen’s d (Cohen 1977). This term is calculated
from means and standard deviations from post outcomes of
Study Selection and Data Extraction
the intervention group and the control group. Magnitudes
of Cohen’s d were fixed at 0.2, 0.5, and 0.8, respectively as
Title/abstract screening followed by full-text screening was
commonly accepted for small, medium and large (Zakzanis
performed independently by the first and second authors.
2001). Cohen’s d was calculated from the primary outcome.
Disagreements were resolved through consensus.
Cohen’s d was calculated for each outcome and averaged if
Inclusion criteria were (a) Studies about interactive digi-
studies had more than one primary outcome. Cohen’s d was
tal interventions, defined as interventions with two-way
calculated from all outcomes measured and averaged if stud-
communication between the program and the user, e.g.
ies did not choose one or more primary outcomes. Results
tablet-based interventions, computer programs, apps for
were requested by e-mailing the authors, if effect size could
smartphones, wearable devices or virtual reality. (b) Par-
not be found or calculated.
ticipants of any age groups were diagnosed with autism
Post outcomes were calculated as a sum of baseline out-
according to DSM-IV (American Psychiatric Association
comes and change in outcomes, for studies reporting these
2000), DSM-5 (American Psychiatric Association 2013)
(Hatfield et al. 2017; Whitehouse et al. 2017). Standard devi-
or ICD-10 (WHO 1992). (c) Interventions must aim for
ation was assumed to be the standard deviation from base-
improvement and measure skills in autistic individuals. (d)
line measures. Cohen’s d could not be calculated in three
Controlled trials or randomized controlled trials (RCT) with
studies due to lack of data (Faja et al. 2008, 2012; Tanaka
an intervention group and a control group, both with autism.
et al. 2010). However, these were extracted from the prior
(e) Full-text articles in English published in peer-reviewed
meta-analysis (Grynszpan et al. 2014).
13
Fridenson-Hayo et al. (2017) reported separate outcomes Results

from two identical trials from different countries. Out-
comes from the two trials were combined. It was decided Figure 1 shows a PRISMA flow diagram of the study selec-
that the difference of nationalities would not be of notewor- tion process. Twenty-three studies fulfilled the selection cri-
thy impact. Hopkins et al. (2011) reported separate results teria. Six publications were discarded due to unavailability
from participants with the Kaufman Brief Intelligence of data, despite efforts to contact authors (Humm et al. 2014;
Test (KBIT) score greater and less than 70 (Kaufman and Milne et al. 2018; Mitchell et al. 2007; Silver and Oakes
Kaufman 1990). This was treated as two individual studies 2001; Strickland et al. 2013; Voss et al. 2019). Golan and
because of the potential of study populations to perform dif- Baron-Cohen (2006) reported two experiments, where only
ferently. De Vries et al. (2014) examined two different digital Experiment 1 fulfilled selection criteria. Two publications
interventions with two different targets, why these were also each reported two separate interventions (De Vries et al.
treated as two individual studies. 2014; Hopkins et al. 2011) adding two more studies to the
A random-effects meta-analysis was conducted includ- sample. The resulting 19 studies included 815 participants
ing all studies, and a random-effects meta-analysis was con- with ASD.
ducted including only studies with either low risk of bias or
with some concerns. Heterogeneity was assessed by comput-
Study Characteristics
ing I 2 . Thresholds were set as low, moderate, and high to I 2
values of 25%, 50%, and 75% (Higgins et al. 2003).
Overall characteristics of studies are shown in Table 1, and
A random-effects meta-regression of Cohen’s d was per-
the characteristics of each study are shown in Table 2. The
formed, testing for association to the duration of the study
overall distribution of digital devices is shown in Fig. 2. The
(in total hours) and age (in age groups, tested overall and
overall distribution of intervention targets is shown in Fig. 3.
between subgroups: < 5 years, 5–10 years, > 10–15 years
(reference), > 15 years). Further variables tested were if the
intervention target was social skills (including emotion rec- Assessment of Efficacy
ognition, face recognition and social communication) (yes/
no) and if the intervention was assisted by a researcher/ The meta-analysis revealed an overall effect size (Cohen’s
health care professional, but not applied to the control group d) of 0.32 [0.12–0.51], p < 0.001, corresponding to a small
(yes/no). IQ was not considered as a moderator because only effect, significantly different from zero. Assessment of het-
one study reported a mean IQ < 85. All moderators were erogeneity revealed an I 2 of 100%, corresponding to a very
tested separately. high degree of heterogeneity. A forest plot is shown in Fig. 4.
Cohen’s d was calculated using only the post-intervention After excluding studies with a high risk of bias, overall
outcome score and standard deviation. Therefore, the poten- effect size (Cohen’s d) was 0.39 [0.16–0.62], p < 0.001, I 2
tial impact of differences at baseline (scores measured before =100%, also corresponding to a small effect, significantly
the intervention period) between the intervention group and different from zero, with a very high heterogeneity.
the control group was not considered. To check for signifi- The meta regression analysis revealed no significant asso-
cant differences in baseline measures, a two-sample t-test ciations between Cohen’s d and any of the variables tested.
was performed for all baseline measures when available. These were duration of intervention (β = − 0.03, p = 0.052),
When not available, test results of baseline measure com- age group, (β = 1.93, p = 0.59) age group between sub-
parisons were noted if reported. groups (< 5 years: β = − 0.35, p = 0.24; 5–10 years: β = 0
A noteworthy possible bias of meta-analytical procedures (baseline); > 10–15 years: β = − 0.25, p = 0.36; > 15 years:
is the tendency for authors, not to publish studies yielding β = − 0.27, p = 0.34), assistance by a researcher/health care
no significant results. To check for publication bias, we con- professional (β = 0.41, p = 0.15) or if the intervention target
ducted a Funnel plot that provided a graphical representation was social skills (β = 0.08, p = 0.71).
of the relationship between the standard error of included
trials and associated effect sizes. A presence of asymmetry Baseline Differences
was considered potentially indicative of publication bias.
Egger’s test was performed as a supplementary test. Baseline measures are main/primary outcomes before the inter-
vention period, e.g. measures of communication skills. There
was no information about baseline measures in two studies
(Kumazaki et al. 2017; Tanaka et al. 2010). In Parsons et al.
13
Records identified through database

searching (Embase, Ovid MEDLINE,
Identification
PsycINFO)
(n = 1073)
Records after duplicates removed

(n = 851)
Screening
Records screened Records excluded

(n = 851) (n = 751) because of irrelevance.
Articles excluded during full-text screening

Full-text articles (n=52) because of wrong patient population,
assessed for eligibility
Eligibility
not digital intervention, only abstract/protocol,

(n = 100) wrong comparator, duplicate not found initially,
Intervention on parents, no external
Articles from measures.
hand search Articles excluded during data-extraction
(n=1) (n=32) because of overlapping data, control
intervention promotes target, digital
Studies included in intervention not main part of intervention, not
qualitative and autism related skill, not interactive
Included
quantitative synthesis intervention, measures made from partly

(meta-analysis) trained material, diagnostics insufficient, skill
(n =17a) cannot be used outside of context, unable to
get data despite effort to contact author.
Fig. 1 Study selection flow diagram showing the Preferred Reported Items for Systematic reviews and Meta-analyses (PRISMA). Two publica-
tions each reported two separate interventions, resulting in 19 studies included for analysis
Table 1 Study characteristics Variable No. of studies reporting Mean (SD) [range] or %
of included studies about digital the variable
interventions for individuals (total = 19)
with ASD
Age n = 19 Mean = 10.6 (3.4) [3.3–30.7] years
IQ n = 12 Mean = 99.8 (18.1) [54.9–117.2]
Sample size n = 19 Mean = 44.4 (24.1) [10–94] participants
Gender n = 15 19.5% females
Duration n = 18 Mean = 12.3 (7.2), [2.5–27.0] h
Publication year n = 19 [2006–2019]
Risk of bias (methodological bias n = 19 10.5% (2 studies) low risk
assessed for each study) 68.4% (13 studies) some concerns
21.1% (4 studies) high risk
13
Table 2 Details of the included studies used to assess the efficacy of digital interventions for individuals with ASD
References (year) Mean age Mean IQ (SD) Met diagnostic Intervention Control condition Training duration Primary out- Cohen’s d (SD) Risk of Bias
in years criteria (assess- condition (sample (sample size) comes
(SD) ment tools)a size)
Conaughton et al. 9.7 (1.3) N/A Childhood Asper- Anxiety treat- Waitlist control Children: 0.6 h pr. No. of diagnoses 0.88 (1.12) Some concerns. No
(2017) ger Syndrome ment with (n = 21) week × 12 weeks (evaluated protocol found.
Test (CAST) Cognitive Parents: 0.4 h pr. using Anxiety Missing data not
Based Therapy week × 12 weeks Disorders Inter- considered in the
program, view Schedule analysis
called BRAVE for DSM-IV)
ONLINE. Par- Clinician Sever-
ticipants receive ity Rating
weekly, online (CSR)
contact with The Children’s

a therapist in Global Assess-
response to ses- ment Scale
sion activitiesb (CGAS)
(n = 21) Child Behavior
Checklist—the
internalizing
scale (CBCL-
int)
Spence Chil-
dren’s Anxiety
Scale-Child
(SCAS-C)
Spence Chil-
dren’s Anxiety
Scale-Parent
(SCAS-P)
13
Table 2 (continued)
13
Faja et al. (2008) 19.7 (5.8) 103.5 (11.9) Autism Diagnos- Face recognition No intervention 2.3 h pr. Benton Facial − 0.58 (0.61) High risk. Not
tic Interview training with a (n = 5) week × 3 weeks Recognition randomized. No
Revised computer pro- Test-Long protocol found.
(ADI–R), gram (n = 5) Form No information
Autism Diag- Facial memory about the blind-
nostic Observa- task of the ing of outcome
tion Schedule Children’s assessors
(ADOS) Memory Scale
-module 4, and (CMS) or
DSM-IV the Wechsler
Memory Scale
(WMS-III)
Test of sensitiv-
ity to second-
order relations
in facesc
Test of holistic
processing of
facesc
Faja et al. (2012) 22.0 (5.0) 117.3 (16.9) Autism Diagnos- Face recognition Mock train- 1.5 h x pr. Benton Facial 0.10 (0.31) Some concerns. No
tic Interview training with a ing with a week × 3.2 weeks Recognition protocol found.
Revised (ADI- computer pro- computerized Test-Long No information
R), Autism gram (n = 9) training pro- Form about blinding
Diagnostic gram involving Facial memory of assessors,
Observation houses (n = 9) task of the randomization
Schedule Wechsler process and con-
(ADOS) and Memory Scale cealing of alloca-
DSM-IV (WMS-III) tion sequence
Test of sensitiv-
ity to second-
order relations
in facesc
Test of holistic
processing of
facesc
Fletcher-Watson 4.1 (1.0) N/A Autism Diagnos- Social com- Waitlist control 0.8 h pr. Brief observa- 0.08 (0.02) Some concerns.
et al. (2016) tic Observa- munication (n = 25) week × 10.29 weeks tion of social The primary
tion Schedule skill training communica- outcome in the
(ADOS)— with tablet app tion change article is not the
social commu- FindMe (n = 24) (BOSCC) same as in the
nication protocol
Table 2 (continued)
Fridenson-Hayo 7.3 (1.1) Within normal Autism Diagnos- Emotion Recog- Waitlist control 2 h pr. week × 8 weeks Emotion recogni- 0.40 (0.27) Some concerns.
et al. (2017) range tic Observation nition training (n = 40) tion task score No information
Schedule 2nd with Emoti- including Face about the han-
ed. (ADOS-2) play, a Serious task, Voice dling of missing
Game played Task, Body data. No protocol
on a computer task and Inte- found
(n = 34) grative task
Golan et al. 5.9 (1.0) 98.8 (9.4) N/A Emotion recogni- No intervention 1.8 h pr. Situation-Expres- 1.37 Some concerns. No
(2010) tion training (n = 18) week × 4 weeks sion Matching protocol found
with ’The tasks—Level 3c and no informa-

Transporters’ tion about the
interactive randomization
DVD (n = 20) process, alloca-
tion sequence
concealing and
handling of miss-
ing data
Golan and 30.7 (10.8) 111.4 (11.5) N/A Emotion Recog- No intervention 1.8 h pr. The Cambridge 0.19 (0.29) High risk. No
Baron-Cohen nition training (n = 22) week × 10 weeks Mindreading protocol found.
(2006)—Exper- with Mind (CAM) face No informa-
iment 1 Reading com- task tion about the
puter program The Cambridge randomization
(n = 19) Mindreading process and allo-
(CAM) voice cation sequence
task concealing. Per-
The Cambridge protocol analysis
Mindreading made
(CAM) no.
of concepts
recognized
Reading the
Mind in the
Eyes task
Reading the
Mind in the
Voice task
Reading the
Mind in Films
task
13
Table 2 (continued)
13
Hatfield et al. 14.9 (1.2) N/A N/A Support of transi- No intervention Logged in 5 times AIR Self-Deter- 0.10 (0.06) Some concerns.
(2017) tion from high (n = 45) over one year mination Scale Randomization
school with the (AIR) process insuf-
BOOST-A™, ficient. Outcome
an online pro- assessors were
gram (n = 49) aware of inter-
vention condition
Hopkins et al. 10.0 (2.6) 92.4 (22.7) Childhood Emotion and Mock-training 0.6 h pr. Emotion recogni- 1.07 (0.77) Some concerns. No
(2011)— Autism Rating facial recogni- (n = 11) week × 6 weeks tion in pictures protocol found.
KBIT > 70 Scale (CARS) tion training and drawingsc No information
with the Fac- Benton Facial about randomi-
eSay computer Recognition zation process
program Test -Short and allocation
(n = 13) Form sequence con-
Benton Facial cealing
Recognition
Test -Long
Form
Social Skills
rating system
(SSRS)
Social Skills
Observation
(SSO)
Hopkins et al. 10.5 (3.3) 54.9 (18.8) Childhood Emotion and Mock-training 0.6 h pr. Emotion recogni- 0.67 (0.20) Some concerns. No
(2011)— Autism Rating facial recogni- (14) week × 6 weeks tion in pictures protocol found.
KBIT < 70 Scale (CARS) tion training and drawingsc No information
with the Fac- Benton Facial about randomi-
eSay computer Recognition zation process
program Test -Short and allocation
(n = 11) Form sequence con-
Benton Facial cealing
Recognition
Test -Long
Form
Social Skills
rating system
(SSRS)
Social Skills
Observation
(SSO)
Table 2 (continued)
Jouen et al. 7.0 (1.5) 97.8 (21.5) Autism Diagnos- Training in No intervention Computer game: 0.7 h Autism Diagno- 0.41 High risk. Not
(2017) tic Interview communica- (n = 10) pr. week × 26 weeks sis Observa- randomized
Revised (ADI- tion and social One-to-one sessions: tion Schedule (allocation based
R) interaction with 0.2 h pr. (ADOS) on parent’s moti-
a computer pro- week × 26 weeks vation)
gram, GOLIAH
(Gaming Open
Library for
Intervention
in Autism at
Home) and
structured one-
to-one sessions
focused on
imitation and
joint attention
activities with
a therapistb
(n = 14)
Kumazaki et al. 21.9 (2.6) 86.7 (12.3) Diagnostic Job interview Interview guid- 2.5 h pr. Non-verbal 1.01 (0.06) Some concerns.
(2019) Interview for training pro- ance by teach- week × 1 week communica- No information
Social and gram using an ers (n = 13) tion, includ- on the handling
Communica- android robot. ing posture, of missing
tion Disorders Teacher assisted gaze, voice outcomes. No
(DISCO) through half of volume, nod- protocol found
the program.b ding and facial
+ Interview guid- expressionsc
ance by teach- Confidence rat-
ers (n = 13) ing scalec
Lopata et al. 8.8 (1.5) 105.7 (13.3) Autism Diagnos- Emotion Recog- Five-week sum- 3.5 h pr. Cambridge mind 0.16 (0.64) Low risk
(2016) tic Interview- nition training mer treatment week × 3 weeks reading face-
Revised with Mind (SummerMAX) voice battery
(ADI-R) Reading com- without the for children
puter program Mind Read- (CAM-C)
as a part of a ing program Emotion recogni-
five-week sum- (n = 18) tion and display
mer treatment survey (ERDS)
(SummerMAX)
(n = 18)
13
Table 2 (continued)
13
Parsons et al. 5.2 (1.6) N/A N/A Training in devel- Waitlist control. 1.3 h pr. Mullen Scales of 0.11 (0.28) Some concerns.
(2019) opmental skills Received week × 13 weeks Early Learning Not blinded out-
with TOBY tablet, but not (MSEL) come assessors.
app on a tablet the TOBY app Communication One outcome
(n = 30) (n = 29) and Symbolic in the article
Behavior was not in the
Scales (CSBS) protocol
Spaniol et al. 8.0 (1.1) N/A Diagnostic crite- Attention training Mock training 1.5 h pr. Childhood 0.45 (0.45) Some concerns.
(2018) ria reviewed for strengthen- with standard week × 8.7 weeks Autism Rating No informa-
ing of academic computer Scale (CARS) tion about the
performance games. One-to- Ravens Colored randomization
with Computer- one supervision Progressive process and allo-
ized Progres- was provided Matrices cation sequence
sive Attentional for each child (Raven CPM) concealing.
Training during mock- Academic perfor- Missing data not
(CPAT), a com- training (n = 7) mance in math considered. No
puter program. Academic protocol found
One-to-one performance in
supervision reading
was provided Academic
for each child performance in
during training copying
(n = 8)
Tanaka et al. 10.9 (3.8) 94.7 (22.8) N/A Face recognition Waitlist control 1.1 h pr. Let’s Face It! -0.11 (0.19) High risk. Per-
(2010) training with (n = 37) week × 19.1 weeks Skills Batteryc protocol analysis
the Let’s Face that assesses made. No proto-
It! Computer processing of col. No informa-
games (n = 42) facial stimuli tion about the
and visual com- randomization
parators (cars process and allo-
and houses) cation sequence
concealing
De Vries et al. 10.6 (1.4) 109.5 (21.1) N/A Flexibility Mock training 3.1 h pr. Number-gnome -0.18 (0.39) Some concerns. No
(2014)—Cogni- training with with Braingame week × 6 weeks switch-taskc a protocol found
tive flexibility Braingame Brian, a flexibility task
training Brian, a computer game measured by
computer game (n = 29) reaction time
(n = 26) and %errors
Table 2 (continued)
De Vries et al. 10.5 (1.4) 112.3 (21.2) N/A Working memory Mock training 3.1 h pr. N-back task 0.29 (0.19) Some concerns. No
(2014)—Work- training with with Braingame week × 6 weeks (1-back and protocol found
ing memory Braingame Brian, a 2-back)c
training Brian, a computer game a working
computer game (n = 29) memory task
(n = 31) measured by
reaction time
and %errors
Whitehouse et al. 3.3 (0.6) N/A Autism Diagnos- Developmental No intervention 1.0 h pr. The Autism -0.03 Low risk
(2017) tic Observa- skills training (n = 36) week × 26 weeks Treatment

tion Schedule with the TOBY Evaluation
– generic app on a tablet Checklist
(ADOS-G) (n = 39) (ATEC)
Module 1 or 2
Based on trial designs with a control group of participants with ASD

N/A not available, IQ intelligence quotient, SD Standard deviation, KBIT Kaufman Brief Intelligence Test (Kaufman and Kaufman 1990)
a
All participants had a formal diagnosis of ASD. The screening tools reported was applied at the entry of the trial
b
These interventions were assisted by a researcher/health care professional, not applied to the control group
c
Measurement instruments developed as a part of the study
13
Digital devices Publication Bias
18 Inspection of the funnel plot (Fig. 5) suggested publica-

16 tion bias, as the study distribution around the pooled mean
14 effect size was asymmetrical. Egger’s test for funnel-plot
14
asymmetry was not statistically significant (p = 0.99).
12
10
8 Discussion
6
4 3 Trials of digital interventions for ASD were mostly com-
2 1 1 puter-based interventions and targets were mainly social
0 skills. The meta-analysis revealed that the overall effect size
Computer Tablet app Robot Interactive of the interventions was positive, however small. We found
program DVD a very high heterogeneity across the study data, indicating
publication bias. Risk of Bias assessments of each study
Fig. 2 Overall distribution of digital devices used in interventions revealed that four studies were being at high risk of bias, 13
had some concerns and two were at low risk of bias.
A prior meta-analysis on the same subject (Grynsz-
pan et al. 2014) found that interventions were mostly
Intervention targets computer-based and targets were mainly social skills and
literacy skills. However, all studies in Grynszpan et al.
18 (2014) examining literacy skills included control groups
16 receiving a target promoting intervention. Therefore, the
14 studies focusing on literacy skills were not included in
12 11 our meta-analysis. A systematic review focusing digi-
10 tal interventions for only adolescents with ASD (Odom
8 et al. 2015) identified additional intervention targets e.g.
6
6 independence and academic skills, and additional digital
4 devices as smartphones, personal digital assistants and
2 Bluetooth® technology. These studies were either based
2
on single-case designs and/or based on video-modeling,
0
hence they were excluded from the present analysis. One
Social skills Developmental Other
skills study (Strickland et al. 2013) examining virtual reality
intervention was excluded from the present review due to
missing outcome data. This suggests that more interven-
Fig. 3 Overall distribution of intervention targets. Social skills
include emotional recognition (n = 6), face recognition (n = 3) and tion targets and technologies are being tested than the ones
social communication (n = 2). Developmental skills include e.g. vis- analyzed by our review.
ual motor, imitation, language, and social skills. ‘Other’ include anxi- We found the effect size, Cohen’s d for digital inter-
ety treatment, job interview skills, working memory, cognitive flex-
ventions to be 0.32 indicating a positive, however small
ibility, attention training and support for transition from high school
respectively effect. The prior meta-analysis by Grynszpan et al. (2014)
found an overall effect size, Cohen’s d of 0.47, indicating
a medium effect. However, there are some differences in
(2019) the baseline was significantly higher for the interven- the selection criteria between this prior and the present
tion group compared to the control group in 1 of 6 measures, in meta-analysis. For instance, they included studies with a
Hopkins et al.—KBIT > 70 (2011) in 1 of 5 measures and in De combination of digital and non-digital interventions and
Vries et al. Working memory training (2014) in 2 of 4 measures. studies comparing two interventions.
In Faja et al. (2012) baseline was significantly higher This meta-analysis revealed a large heterogeneity, lim-
for the control group compared to the intervention group iting generalization of conclusions. This could be due to
in 1 of 5 measures and in Lopata et al. (2016) in 1 of 4 the very different intervention targets, procedures, and
measures. There were no significant differences between outcomes of the studies included. Research in this field is
the intervention group and the control group for all base- in general very different at this point of time, as the field
line measures in the remaining 12 studies. of digital interventions are very young. For instance, there
13
Fig. 4 Forest plot showing

mean effect size (Cohen’s d)
and confidence intervals for all
studies included
was no tendency for studies to assess the same outcomes, It remains unclear whether the quality of the studies has an
even when promoting the same target. Also, six studies impact on the effect, since there were too few studies with
had each different intervention targets. Some studies were low risk of bias for the conduction of a specific analysis. The
excluded because the control condition promoted the inter- funnel plot suggested publication bias. However, Egger’s test
vention target, e.g. a computer program was compared to for funnel-plot asymmetry was not statistically significant,
traditional teaching. Some studies were excluded because which may be due to the outlying study (Golan et al. 2010)
the digital interventions were only a smaller part of larger with a large Cohen’s d and very small standard error.
interventions. This suggest that the research field in digital Limitations to this study includes the possibility that
interventions for ASD is still under establishing. publications may have slipped through the searching. New
Risk of bias assessment measured with Cochrane’s Risk relevant articles may have been published since the search
of Bias tool revealed that only two studies were at low risk was conducted. Additionally, four relevant publications were
of bias, where 13 had some concerns and four had high risk. excluded, since it was not possible to get data, despite efforts
to contact authors. Basing our study on the standardized
effect size, Cohen’s d, is a limitation.
Partly, because it is a measure calculated from any main
outcome, regardless of how meaningful and valuable the out-
come is. Therefore, the utility may not be reflected. Partly,
because Cohen’s d is exclusively based on post-intervention
outcomes and standard deviation, meaning that potential
baseline differences will skew the effect size. To consider
this problem, we calculated baseline differences between
the intervention group and the control group for all main
outcomes available. However, we found no indications for
systematic bias in a noteworthy degree on the meta-analytic
level; still bias may anyhow occur on a study level.
This meta-analysis suggests that digital interventions for
ASD have not proven conclusive efficacy but tends to gain a
Fig. 5 Funnel plot of the meta-analysis
small effect. It should be considered that these interventions
13
in general are of very short duration (average duration was American Psychiatric Association. (2013). Diagnostic and statistical
12.3 h overall) compared to traditional therapy. Therefore, manual of mental disorders (5th ed.).
Cohen, J. (1977). Statistical power analysis for the behavioral sciences.
long term interventions may increase efficacy. If so, there New York: Academic Press.
may be considerable economic advantages by implementing Conaughton, R. J., Donovan, C. L., & March, S. (2017). Efficacy
digital interventions in clinical practice, even if digital inter- of an internet-based CBT program for children with comorbid
ventions show to be of equal efficacy as traditional therapy. High Functioning Autism Spectrum Disorder and anxiety: A
randomised controlled trial. Journal of Affective Disorders, 218,
260–268. https://doi.org/10.1016/j.jad.2017.04.032
De Vries, M., Prins, P. J. M., Schmand, B. A., & Geurts, H. M. (2014).
Conclusion Working memory and cognitive flexibility-training for children
with an autism spectrum disorder: A randomized controlled trial.
J Child Psychol Psychiatry. https://doi.org/10.1111/jcpp.12324
The overall effect size of the interventions was positive, Faja, S., Aylward, E., Bernier, R., & Dawson, G. (2008). Becoming a
however small. Most of the included studies raised some face expert: A computerized face-training program for high-func-
concerns about the risk of bias. There was a pronounced tioning individuals with autism spectrum disorders. Developmen-
tal Neuropsychology, 33(1), 1–24. https://doi.org/10.1080/87565
heterogeneity between effect sizes and additionally a risk of 640701729573
publication bias, reducing generalization of results. There- Faja, S., Webb, S. J., Jones, E., Merkle, K., Kamara, D., Bavaro, J.,
fore, the efficacy is still not proven conclusive. Research in et al. (2012). The effects of face expertise training on the behav-
digital interventions for ASD is still too heterogeneous to ioral performance and brain activity of adults with high function-
ing autism spectrum disorders. Journal of Autism and Develop-
allow full comparison. Therefore, guidelines for the developmental Disorders, 42(2), 278–293. https://doi.org/10.1007/s1080
ment of trials are wanted. 3-011-1243-8
For the possibility of comparing future studies, we sug- Fletcher-Watson, S., Petrou, A., Scott-Barrett, J., Dicks, P., Graham, C.,
gest research designs with the control groups either not O’Hare, A., et al. (2016). A trial of an iPadTM intervention target-
ing social communication skills in children with autism. Autism,
receiving an intervention or receiving mock training. If treat-
20(7), 771–782. https://doi.org/10.1177/1362361315605624
ment as usual is given to the control group, the intervention Fridenson-Hayo, S., Berggren, S., Lassalle, A., Tal, S., Pigat, D., Meir-
group should receive the same treatment as usual in addition Goren, N., et al. (2017). ‘Emotiplay’: A serious game for learning
to the evaluated intervention. We recommend giving prior- about emotions in children with autism: Results of a cross-cultural
evaluation. European Child & Adolescent Psychiatry, 26(8), 979–
ity to larger RCT studies evaluating digital interventions of
992. https://doi.org/10.1007/s00787-017-0968-0
ASD in future research. Especially, there is a lack of RCT Golan, O., Ashwin, E., Granader, Y., McClintock, S., Day, K., Leggett,
studies evaluating interventions based on smartphones and V., & Baron-Cohen, S. (2010). Enhancing emotion recognition
virtual reality, furthermore on long-term interventions. We in children with autism spectrum conditions: An intervention
using animated vehicles with real emotional faces. Journal of
recommend more focus on interventions of longer duration, Autism and Developmental Disorders, 40(3), 269–279. https://
since there may be an increased effect. doi.org/10.1007/s10803-009-0862-9
Golan, O., & Baron-Cohen, S. (2006). Systemizing empathy: Teach-
Acknowledgements We would like to thank Maria Tækker, Povl ing adults with Asperger syndrome or high-functioning autism
Munk-Jørgensen, Pia Veldt Larsen, Céline Croucher Vasarhelyi, Psy- to recognize complex emotions using interactive multimedia.
chiatric Research Academy, Ole Jacob Storebø and Thomas Bernhard Development and Psychopathology, 18(2), 591–617. https://doi.
for their help and support. org/10.1017/S0954579406060305
Grynszpan, O., Weiss, P. L., Perez-Diaz, F., & Gal, E. (2014). Inno-
Author Contributions This study was designed and prepared by HS vative technology-based interventions for autism spectrum dis-
and NB. The search and selection of studies was conducted by HS and orders: A meta-analysis. Autism, 18(4), 346–361. https ://doi.
LHF. The dataextraction, Risk of Bias assessment and analysis was org/10.1177/1362361313476767
conducted by HS. The manuscript was written by all authors. Hatfield, M., Falkmer, M., Falkmer, T., & Ciccarelli, M. (2017). Effec-
tiveness of the BOOST-ATM online transition planning program
for adolescents on the autism spectrum: A quasi-randomized con-
Funding This research received no specific grant from any funding
trolled trial. Child and Adolescent Psychiatry and Mental Health,
agency in the public, commercial, or not-for-profit sectors.
11(1), 1–12. https://doi.org/10.1186/s13034-017-0191-2
Higgins, J. P. T., Thompson, S. G., Deeks, J. J., & Altman, D. G.
Compliance with Ethical Standards (2003). Measuring inconsistency in meta-analyses. British
Medical Journal, 327(7414), 557–560. https://doi.org/10.1136/
Conflict of interest The authors declare that they have no conflict of bmj.327.7414.557
interest. Hollis, C., Falconer, C. J., Martin, J. L., Whittington, C., Stockton, S.,
Glazebrook, C., & Davies, E. B. (2017). Annual Research Review:
Digital health interventions for children and young people with
mental health problems: A systematic and meta-review. Journal
References of Child Psychology and Psychiatry, 58(4), 474–503. https://doi.
org/10.1111/jcpp.12663
American Psychiatric Association. (2000). Diagnostic and statistical Hopkins, I. M., Gower, M. W., Perez, T. A., Smith, D. S., Amthor,
manual of mental disorders (4th ed., text rev.). F. R., Casey Wimsatt, F., & Biasini, F. J. (2011). Avatar assis-
tant: Improving social skills in students with an ASD through a
13
computer-based intervention. Journal of Autism and Developmen- to teach communication skills to children with autism spectrum
tal Disorders, 41(11), 1543–1555. https://doi.org/10.1007/s1080 disorders: a systematic review. Journal of Behavioral Education,
3-011-1179-z 20(1), 55–76. https://doi.org/10.1007/s10864-010-9112-7
Humm, L. B., Olsen, D., Be, M., Fleming, M., & Smith, M. (2014). Ramdoss, S., Machalicek, W., Rispoli, M., Mulloy, A., Lang, R., &
Simulated job interview improves skills for adults with serious O’Reilly, M. (2012). Computer-based interventions to improve
mental illnesses. In Annual Review of Cybertherapy and Tele- social and emotional skills in individuals with autism spec-
medicine 2014: Positive Change: Connecting the Virtual and the trum disorders: A systematic review. Developmental Neurore-
Real (Vol. 99, pp. 50–54). IOS Press. https://doi.org/https://doi. habilitation, 15(2), 119–135. https ://doi.org/10.3109/17518
org/10.3233/978-1-61499-401-5-50 423.2011.651655
Jouen, A. L., Narzisi, A., Xavier, J., Tilmont, E., Bodeau, N., Bono, Rogge, N., & Janssen, J. (2019). The economic costs of autism spec-
V., et al. (2017). GOLIAH (Gaming open library for intervention trum disorder: A literature review. Journal of Autism and Devel-
in autism at home): A 6-month single blind matched controlled opmental Disorders., 49, 2873–2900. https ://doi.org/10.1007/
exploratory study. Child and Adolescent Psychiatry and Mental s10803-019-04014-z
Health, 11(1), 1–14. https://doi.org/10.1186/s13034-017-0154-7 Root, J. R., Stevenson, B. S., Davis, L. L., Geddes-Hall, J., & Test,
Kaufman, A. S., & Kaufman, N. L. (1990). Kaufman brief intelligence D. W. (2017). Establishing computer-assisted instruction to teach
test. Circle Pines, MN: American Guidance Service. academics to students with autism as an evidence-based practice.
Kumazaki, H., Muramatsu, T., Yoshikawa, Y., Corbett, B. A., Mat- Journal of Autism and Developmental Disorders, 47(2), 275–284.
sumoto, Y., Higashida, H., et al. (2019). Job interview training https://doi.org/10.1007/s10803-016-2947-6
targeting nonverbal communication using an android robot for Sandbank, M., Bottema-Beutel, K., Crowley, S., Cassidy, M., Dunham,
individuals with autism spectrum disorder. Autism. https://doi. K., Feldman, J. I., et al. (2020). Project AIM: Autism intervention
org/10.1177/1362361319827134 meta-analysis for studies of young children. Psychological Bul-
Kumazaki, H., Warren, Z., Corbett, B. A., Yoshikawa, Y., Matsumoto, letin, 146(1), 1–29. https://doi.org/10.1037/bul0000215
Y., Higashida, H., et al. (2017). Android robot-mediated mock Silver, M., & Oakes, P. (2001). Evaluation of a new computer interven-
job interview sessions for young adults with autism spectrum dis- tion to teach people with autism or Asperger syndrome to recog-
order: A pilot study. Frontiers in Psychiatry, 8, 1–8. https://doi. nize and predict emotions in others. Autism, 5(3), 299–316. https
org/10.3389/fpsyt.2017.00169 ://doi.org/10.1177/1362361301005003007
Lai, M.-C., Lombardo, M. V., & Baron-Cohen, S. (2014). Autism. Spaniol, M. M., Shalev, L., Kossyvaki, L., & Mevorach, C. (2018).
The Lancet, 383(9920), 896–910. https://doi.org/10.1016/S0140 Attention training in autism as a potential approach to improving
-6736(13)61539-1 academic performance: A school-based pilot study. Journal of
Lopata, C., Thomeer, M. L., Rodgers, J. D., Donnelly, J. P., & McDon- Autism and Developmental Disorders, 48(2), 592–610. https://
ald, C. A. (2016). RCT of mind reading as a component of a doi.org/10.1007/s10803-017-3371-2
psychosocial treatment for high-functioning children with ASD. Sterne, J., Savović, J., Page, M., Elbers, R., Blencowe, N., Boutron, I.,
Research in Autism Spectrum Disorders, 21, 25–36. https://doi. et al. (2019). RoB 2: A revised tool for assessing risk of bias in
org/10.1016/j.rasd.2015.09.003 randomised trials. BMJ, 366, I4898.
Milne, M., Raghavendra, P., Leibbrandt, R., & Powers, D. M. W. Strickland, D. C., Coles, C. D., & Southern, L. B. (2013). JobTIPS:
(2018). Personalisation and automation in a virtual conversa- A transition to employment program for individuals with autism
tion skills tutor for children with autism. Journal on Multimodal spectrum disorders. Journal of Autism and Developmental
User Interfaces, 12(3), 257–269. https://doi.org/10.1007/s1219 Disorders, 43(10), 2472–2483. https ://doi.org/10.1007/s1080
3-018-0272-4 3-013-1800-4
Mitchell, P., Parsons, S., & Leonard, A. (2007). Using virtual envi- Tanaka, J. W., Wolf, J. M., Klaiman, C., Koenig, K., Cockburn, J.,
ronments for teaching social understanding to 6 adolescents with Herlihy, L., et al. (2010). Using computerized games to teach face
autistic spectrum disorders. Journal of Autism and Developmen- recognition skills to children with autism spectrum disorder: The
tal Disorders, 37(3), 589–600. https ://doi.org/10.1007/s1080 Let’s Face It! program. Journal of Child Psychology and Psychia-
3-006-0189-8 try and Allied Disciplines, 51(8), 944–952. https://doi.org/10.111
Moher, D., Liberati, A., Tetzlaff, J., & Altman, D. G. (2009). Preferred 1/j.1469-7610.2010.02258.x
reporting items for systematic reviews and meta-analyses: The Voss, C., Schwartz, J., Daniels, J., Kline, A., Haber, N., Washing-
PRISMA statement. PLoS Medicine, 6(7), e1000097. https://doi. ton, P., et al. (2019). Effect of wearable digital intervention for
org/10.1371/journal.pmed.1000097 improving socialization in children with autism spectrum disorder.
Odom, S. L., Thompson, J. L., Hedges, S., Boyd, B. A., Dykstra, J. R., JAMA Pediatrics, 173(5), 446. https://doi.org/10.1001/jamapediat
Duda, M. A., et al. (2015). Technology-aided interventions and rics.2019.0285
instruction for adolescents with autism spectrum disorder. Journal Whitehouse, A. J. O., Granich, J., Alvares, G., Busacca, M., Cooper,
of Autism and Developmental Disorders, 45(12), 3805–3819. https M. N., Dass, A., et al. (2017). A randomised controlled trial of an
://doi.org/10.1007/s10803-014-2320-6 iPad-based application to complement early behavioural interven-
Parsons, D., Cordier, R., Lee, H., Falkmer, T., & Vaz, S. (2019). A ran- tion in Autism Spectrum Disorder. Journal of Child Psychology
domised controlled trial of an information communication tech- and Psychiatry and Allied Disciplines, 58(9), 1042–1052. https
nology delivered intervention for children with autism spectrum ://doi.org/10.1111/jcpp.12752
disorder living in regional Australia. Journal of Autism and Devel- WHO. (1992). The ICD-10 classification of mental and behavioural
opmental Disorders, 49(2), 569–581. https ://doi.org/10.1007/ disorders: Clinical descriptions and diagnostic guidelines.
s10803-018-3734-3 Zakzanis, K. K. (2001). Statistics to tell the truth, the whole truth, and
Peters-Scheffer, N., Didden, R., Korzilius, H., & Matson, J. (2012). nothing but the truth: formulae, illustrative numerical examples,
Cost comparison of early intensive behavioral intervention and and heuristic interpretation of effect size analyses for neuropsy-
treatment as usual for children with autism spectrum disorder in chological researchers. Archives of Clinical Neuropsychology,
the Netherlands. Research in Developmental Disabilities. https:// 16(7), 653–667.
doi.org/10.1016/j.ridd.2012.04.006
Ramdoss, S., Lang, R., Mulloy, A., Franco, J., O’Reilly, M., Didden, Publisher’s Note Springer Nature remains neutral with regard to
R., & Lancioni, G. (2011). Use of computer-based interventions jurisdictional claims in published maps and institutional affiliations.
13
3-011-1179-z 20(1), 55–76. https://doi.org/10.1007/s10864-010-9112-7
org/10.3233/978-1-61499-401-5-50 423.2011.651655
org/10.3389/fpsyt.2017.00169 ://doi.org/10.1177/1362361301005003007
3-018-0272-4 3-013-1800-4
doi.org/10.1016/j.ridd.2012.04.006
13
3-011-1179-z 20(1), 55–76. https://doi.org/10.1007/s10864-010-9112-7
org/10.3233/978-1-61499-401-5-50 423.2011.651655
org/10.3389/fpsyt.2017.00169 ://doi.org/10.1177/1362361301005003007
3-018-0272-4 3-013-1800-4
doi.org/10.1016/j.ridd.2012.04.006
13

Sandgreen Et Al (2020) Meta-Analysis Digital Interventions

Uploaded by

Copyright:

Available Formats

You might also like

Sandgreen Et Al (2020) Meta-Analysis Digital Interventions

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Sandgreen Et Al (2020) Meta-Analysis Digital Interventions

Uploaded by

Copyright:

Available Formats

④

Journal of Autism and Developmental Disorders

Digital Interventions for Autism Spectrum Disorder: A Meta-analysis

Accepted: 29 October 2020

Keywords Autism · Digital · Technology · Computer · Meta-analysis · Systematic review

Fridenson-Hayo et al. (2017) reported separate outcomes Results

Records identified through database

Records after duplicates removed

Records screened Records excluded

Articles excluded during full-text screening

not digital intervention, only abstract/protocol,

quantitative synthesis intervention, measures made from partly

contact with The Children’s

with ’The tasks—Level 3c and no informa-

(2017) tic Observa- skills training (n = 36) week × 26 weeks Treatment

Based on trial designs with a control group of participants with ASD

Digital devices Publication Bias

18 Inspection of the funnel plot (Fig. 5) suggested publica-

Fig. 4 Forest plot showing

You might also like