Journal of Autism and Developmental Disorders (2020) 50:4230–4242

https://doi.org/10.1007/s10803-020-04488-2

ORIGINAL PAPER

Wisconsin Card Sorting Test‑Learning Potential: Usefulness

for Assessing Children with Autism Spectrum Disorder
M. Mar Gómez‑Pérez2 · Sara Mata1,3 · Francisca Serrano1,2 · M. Dolores Calero1

Published online: 3 April 2020

© Springer Science+Business Media, LLC, part of Springer Nature 2020

Abstract
This study analyzes the Wisconsin Card Sorting Test-Learning Potential (WCST-LP) in children with Autism Spectrum
Disorder (ASD) versus children with typical development (TD). Its main aim was to assess: the test’s construct validity; the
effect of IQ on its pretest and LP scores; and whether the WCST-LP held any relationship to cognitive/EF and social abili-
ties. Participants were 105 children (43 with ASD/62 with TD). Results showed evidence of construct validity in an ASD
population (improvements from pretest to posttest), that full IQ influenced pretest performance but did not affect LP, and
that a relationship between LP and verbal and social abilities existed only in children with ASD. Conclusions indicate the
appropriateness of the WCST-LP in ASD prognosis assessment.

Keywords  Autism spectrum disorders · Wisconsin card sorting test · Response training · Learning potential · Prognosis

Introduction EFs are important for adequate psychological adjustment

Executive functions (EFs) refer to a number of higher-order,
interrelated neurocognitive processes that are involved in the
organization of action and thought. They are considered nec-
essary to deal with new, complex situations involving objec-
tives or goals for which we have no established, automated
behavior (Blijd-Hoogewys et al. 2014; Diamond 2013; Hill
2004a), and for successful achievement in changing environ-
ments (Blijd-Hoogewys et al. 2014; Diamond 2013). Several
processes are considered as part of EFs (such as cognitive
flexibility, inhibitory control, working memory, planning,
initiation of behavior, and self-regulation), the main being
working memory, cognitive flexibility, and inhibitory control
(Hill 2004a; Robinson et al. 2009).
* Sara Mata
saramata@ugr.es
1
Centro de Investigación Mente, Cerebro y Comportamiento
(CIMCYC), Granada, Spain
2
Departamento de Psicología Evolutiva y de la Educación,
Facultad de Psicología, Universidad de Granada, Granada,
Spain
3
Departamento de Personalidad, Evaluación y Tratamiento
Psicológico, Facultad de Psicología, Universidad de Granada,
Campus Cartuja S/N, 18071 Granada, Spain
and proper cognitive, social and psychological develop-
ment. Specifically, according to some authors, EF is directly
related to social competence in childhood and adolescence
(Diamond 2013; Landry et al. 2009; Maddio and Greco
2010); therefore, deficits in EFs hinder people’s adaptation,
affecting their problem-solving processes, decision-making,
inhibition of inappropriate behavior that may offend others,
emotional intelligence, and the generalization of learning
(Diamond 2013; Montgomery et al. 2013).
Many psychological disorders have associated deficits in
EFs, as is the case in Autism Spectrum Disorder (ASD). The
DSM-5 (American Psychiatric Association 2013) classifies
ASD among neurodevelopmental disorders, as characterized
by persistent deficits in communication and social interac-
tion skills, non-verbal communication, social participation,
and restricted, repetitive patterns of behaviour, interests,
or activities. The ASD diagnosis involves specifications
about the presence or absence of either intellectual deficit
or severe language deficit (pragmatic language impairment,
expressive-receptive language disorder) or both, because
these deficits can be an index of symptom severity, accord-
ing to the DSM-5.
Research approaches have differed in their consideration
of the specific ASD deficits (De Vries et al. 2015; Mont-
gomery et al. 2013). For example, the Executive Functions
Theory proposes that a general cognitive disturbance in

13
Vol:.(1234567890)
Journal of Autism and Developmental Disorders (2020) 50:4230–4242 4231
EFs—cognitive flexibility, inhibitory control, self-regu-
lation, planning and working memory—is responsible for
problems in interaction, social adaptation, and everyday
behavioral characteristics in ASD (Blijd-Hoogewys et al.
2014; Happé et al. 2006; Hill 2004a, b; Leung et al. 2015;
Pugliese et al. 2016).
There is some controversy, however, as to which EFs are
damaged and if there are concrete factors that either mediate
or cause the difficulties, such as age, IQ, task’s characteris-
tics or ASD symptomatology (Hill 2004b; Van Eylen et al.
2015). Currently, heterogeneous findings have signalled dif-
ferent affected EFs: some research points out deficits in some
of the main EFs but not in the others (Brunsdon et al. 2015;
Robinson et al. 2009), and some research has even found
similar results when testing EFs in populations with typical
development (TD) and populations with ASD (De Vries and
Geurts 2012). The variations seem to be due to the different
assessment methods used—open vs. structured tasks, self-
reports vs. third-party assessment—and differences in the
definition of EFs (Gómez-Pérez et al. 2016; Robinson et al.
2009; Van Eylen et al. 2015).
Nevertheless, many research studies have demonstrated
executive dysfunction in children with ASD, namely, in cog-
nitive flexibility (Mostert-Kerckhoffs et al. 2015; Pooragha
et al. 2013), inhibitory control (Montgomery et al. 2013;
Pooragha et al. 2013) and planning (Geurts et al. 2004;
Brunsdon et al. 2015). Such studies have shown that diffi-
culties appear more consistently in cognitive flexibility and
inhibition tasks, such as the Wisconsin Card Sorting Test
(WCST), Planning Drawing Task and Trail Making Test
(Brunsdon et al. 2015; Van Eylen et al. 2015).
Several neuropsychological tools for assessing EFs in
children with ASD are available for the purpose of study-
ing how EFs are altered in these children. Among them,
the Wisconsin Card Sorting Test (WCST) (Heaton 1981)
is most often used for assessing EFs in the framework of
ASD, and is known as “the gold standard EF task” (Ozonoff
et al. 2005; p. 532). This test assesses EFs such as strategic
planning, organized searching, and environmental feedback
to shift cognitive sets. Moreover, it is one of the tests most
often used for testing EFs in school age, especially from
age six to eight (Romine et al. 2004). The WCST is a clas-
sification task based on set-shifting skills. It consists of 128
cards that have to be sorted according to a certain matching
criterion (color, shape, and number), based on the experi-
menter’s feedback. The matching criterion changes after 10
consecutive correct matches, but without explicit instruc-
tion to the participant. Researchers and practitioners have
extensively used the WCST in the ASD population. Since
the year 2000, two review articles and four meta-analyses
about WCST in ASD can be found (Hill 2004a, b; Landry
and Al-Taie 2016; Leung and Zakzanis 2014; Romine et al.
2004; Westwood et al. 2016). These reviews/meta-analyses
study the utility of the WCST in an ASD population. In
general, they analyze which WCST scores were used most
often in the studies reviewed, and which presented greater
discriminant validity between persons with ASD and per-
sons with TD. The results from these studies show that the
perseverations score (cognitive flexibility) followed by the
number of complete categories (global indicator of concep-
tual problem-solving skills) were the WCST scores most
often used in the research on EFs, in children, youth and
adults with ASD, and were the scores where persons with
ASD showed the greatest difficulties.
Adapted from the classical WCST, the Wisconsin Card
Sorting Test-Learning Potential (WCST-LP) was devel-
oped by Wiedl and Wienöbst (1999) and uses 64 cards in a
pretest-training-posttest design. The main goal is to test the
individual’s ability to personally benefit from a short inter-
vention with regard to the tasks included in the assessment
procedure; in other words, it tests the individual’s capacity
for learning, or Learning Potential (LP). The WCST-LP is
based on an assessment methodology known as LP Assess-
ment or as Dynamic Assessment methodology. LP meth-
odological approaches emerged in the second half of the
twentieth century as a complement to the information pro-
vided by conventional intelligence tests, especially in indi-
viduals with intellectual disability. LP assessment follows
a pretest-training-posttest format, where a training phase is
included in the middle of two standard tests (also known as
mediation). While conventional assessment (intelligence and
aptitude tests) emphasizes the individual’s final task per-
formance as a product, LP assessment focuses on cognitive
and metacognitive processes involved in learning, and the
change involved in reaching the solution to the task, in other
words, cognitive modifiability. Consequently, the informa-
tion provided by traditional assessment and LP assessment is
different in nature, the former is static (measuring the result
of knowledge/skills that have already been acquired) and the
second is dynamic (measuring the capacity for change or
learning) (Calero 2012; Haywood and Lidz 2007).
The theoretical basis of LP methodology rests on the
work of Vygostky (1978) and the idea of Zone of Proximal
Development (ZPD) and Feuerstein et al. (1979) and the
Theory of Structural Cognitive Modifiability and the con-
cept of “mediated learning experiences”. Both theoretical
approaches maintain that certain populations with difficul-
ties cannot manifest their actual abilities in the context of
traditional assessment. LP methodology has been broadly
validated in populations with severe conditions, like schiz-
ophrenia, dementia, learning disabilities, and Down syn-
drome, showing that it is possible to design and/or adapt
any intervention as a function of the specific learning strate-
gies of persons, by considering the LP index (for a review,
see Calero 2012; Haywood and Lidz 2007). Nevertheless,
only five studies to date have addressed the LP assessment
13

4232
methodology in the ASD population, and only one of them
focuses on WCST-LP testing (Aljunied and Frederickson
2011a, b; Bonete et al. 2010; Calero et al. 2015; Donaldson
and Olswang 2007).
Donaldson and Olswang (2007) investigated the capac-
ity of preschoolers with ASD to make requests (request
information from an adult, request an action or ask for an
object), using two conditions of structured play. The first
condition assessed the ability to request information in a tra-
ditional assessment situation (assessor is a nonparticipating
observer), compared to a second LP assessment condition,
where the assessor manipulated conditions of the context,
the choice of materials/activities and the linguistic vari-
ables involved in the game. The results of this study showed
that including the LP assessment along with the traditional
assessment made it easier to identify ASD children who had
greater difficulties in requesting information, and to identify
the type of assistance that was most adequate for improving
their performance. On the other hand, Bonete et al. (2010)
used LP methodology to study the ability to learn how to
solve matrices, along with an assessment of interpersonal
skills, in 10 children with ASD and 10 children with TD
(ages 11 to 16). Results showed that the children with ASD
presented similar LP to that of children with TD, and that LP
did not correlate to deficits in interpersonal skills, leading
these authors to conclude that social deficits were not related
to a limited capacity for learning. Aljunied and Frederick-
son (2011a) investigated relationships between nonverbal
intelligence, central coherence, and LP—assessed with an
analogies task in a pretest-training-posttest format—in 52
children (ages 9 and 10) with ASD. They found that all chil-
dren showed LP (they showed gains from pretest to posttest,
after receiving a training phase), although these improve-
ments were associated with the children’s scores in nonver-
bal intelligence and central coherence. In fact, even chil-
dren with a non-verbal IQ lower than 85 and weak central
coherence (field-independence) showed some improvement
on the LP assessment test. In a second study, Aljunied and
Frederickson (2011b) investigated the diagnostic utility of
intelligence measures (total IQ), LP measures and Theory of
Mind measures in 52 children with ASD from three different
educational settings: conventional/unsupported, with mini-
mal educational support, with moderate support. The results
showed that 82.7% of participants were correctly discrimi-
nated and grouped according to their educational setting,
by considering 3 factors: full IQ, performance on Theory
of Mind tasks, and LP score. Finally, Calero et al. (2015)
was the only study to use the WCST-LP in an ASD popu-
lation (45 children from 7 to 13 years old). They showed
that children with ASD obtained significant gains from pre-
test to posttest in the WCST-LP measures of perseverative
answers, perseverative errors and number of complete cat-
egories; although the study identified different gain profiles.
13
Journal of Autism and Developmental Disorders (2020) 50:4230–4242
These results offered evidence of the construct validity of the
WCST-LP in an ASD population, because participants made
significant pretest-to-posttest improvements (the WCST-LP
uses EF measures to evaluate learning capacity as a main
construct). The measure of LP in correct answers signifi-
cantly correlated with emotion recognition measures and
interpersonal conflict solution skills.
All the previous studies have tried to determine the use-
fulness of LP methodology in children with ASD, as well as
the relationship of LP to relevant factors such as intelligence,
Theory of Mind and social abilities.
Intelligence may influence the WCST performance of
children with ASD, as occurs with other types of measures,
but there have not been conclusive results to date. Thus,
Ozonoff (1995) found normative WCST performance in
children with ASD with an IQ over 70. For their part, Liss
et al. (2001) found no differences in WCST between children
with ASD and children with specific language impairments,
once verbal IQ was controlled. In fact, several studies have
shown that once verbal-IQ or full-IQ effect is controlled,
many of the differences between participants with and
without ASD also disappear, specifically, in the percent of
perseverative errors, the number of perseverative errors and
the number of non-perseverative errors (Hill 2004b; Liss
et al. 2001; Rumsey 1985). These studies have concluded
that flexibility deficits, as measured by WCST in children
with ASD, could be related with IQ; this measure should
therefore be controlled in this area of research. By contrast,
some other studies like Filley et al. (1999) have pointed out
that the diagnostic validity of the WCST may be independ-
ent of intelligence in children with ASD. However, all these
studies have analyzed scores from the traditional version of
the WCST, without having information about how IQ might
influence LP scores from the WCST-LP.
Based on the foregoing, the main aim of this study was
to analyze the utility of a WCST adaptation designed for
testing LP (WCST-LP) in EFs tasks, for comparing chil-
dren with ASD to children with typical development (TD).
It attempted to ascertain the relationship among test per-
formance, intelligence, EF (switching) and social abilities.
More specific goals were:
• To obtain evidence that supports the construct validity
of WCST-LP in children with ASD, because the WCST-
LP uses EF measures to evaluate learning potential as a
main construct. Hypothesis 1 (H.1): Children with ASD
will benefit from the training in the WCST-LP obtain-
ing significantly higher results in the posttest than in the
pretest, thus demonstrating LP.
• To determine the role of a static measure of intelligence
(full IQ) in the diagnostic validity of the pretest and gain
scores from the WCST-LP, by observing its sensitiv-
ity for discriminating between children with ASD and
Journal of Autism and Developmental Disorders (2020) 50:4230–4242 4233
children with TD, once full IQ has been controlled as a
covariable. Hypothesis 2.1 (H.2.1): Full IQ will have a
significant effect on level of performance in the WCST-
LP pretest scores (performance scores similar to those
obtained in a standard application of the WCST), for
children with ASD and children with TD. Hypothesis
2.2 (H.2.2): Full IQ will not have a significant effect on
WCST-LP gain scores (pretest-to-posttest improvement
after receiving training), for either children with ASD or
children with TD.
• To explore the relationship between WCST-LP measures
(pretest and gain scores) and the indexes of IQ, EF and
social abilities in children with ASD vs. children with
TD. Hypotheses were not established; the purpose of the
study was exploratory, due to lack of prior consistent
evidence.
Method
Participants
Participants were 105 children from ages 7 to 13  years
(MAge = 9.88, SD = 1.60), divided into two groups: 43 chil-
dren diagnosed with ASD (38 male and 5 female), without
associated intellectual disability or language impairment (or
Asperger Syndrome when the diagnosis was prior to DSM-
5), MAge = 10.07 (SD = 1.65) and MIQ = 94.16 (SD = 14.80);
and 62 children (33 males and 29 females) with TD,
MAge = 9.74 (SD = 1.56) and MIQ = 91.56 (SD = 12.20).
The general inclusion criteria were age between 7 and
13 years, and full IQ between 70 and 130, as tested by the
WISC-IV. For the ASD group, a diagnosis of ASD with-
out accompanying intellectual or language impairment was
established as a specific inclusion criterion for the ASD
group (or Asperger syndrome if the diagnosis preceded the
DSM-5). This subgroup of children with ASD was selected
because the training included in the WCST-LP consists
of instruction, assistance and feedback provided through
oral language. All children with ASD had a previous ASD
diagnosis provided by mental health professionals, based
on assessments with the Autism Diagnostic Observation
Schedule (ADOS; Lord et al. 2002) and the Autism Diag-
nostic Interview-Revised (ADI-R; Le Couteur et al. 2003),
as well as an overall assessment of their skills. In addition,
they were receiving treatment in autism centres or in chil-
dren’s mental health services at hospitals. The previous ASD
diagnosis was verified through available clinical reports for
all children, along with any additional comorbid disorders
(ADHD, behaviour problems, sensory problems) or use of
medication. Based on these reports submitted by the profes-
sionals who made the diagnosis, the presence of additional
comorbid psychological disorders or use of medication were
determined as exclusion criteria for both groups (ASD/TD).
None of the selected children showed any sensory or physi-
cal disease.
For ethical reasons, the groups were selected from autism
centers and schools; we respected the percentages of male
and female students in each of the groups proposed by the
professionals responsible for the students. Consequently,
there are initial differences in the number of female partici-
pants in the group of children with ASD, compared to the
group of children with TD.
Instruments
Measure of Intelligence
Wechsler Intelligence Scale for Children-IV (WISC-IV;
Wechsler 2005). The scale contains 15 subtests, 10 core
subtests and 5 complementary subtests, which assess the
intellectual capacity of children from the ages 6 to 16. The
scale provides four composite scores: verbal comprehension
index, perceptual reasoning index, working memory index
and processing speed index. The scale also provides a total
IQ score that reflects the child’s general cognitive capacity.
The assessment lasted approximately one hour and fifty min-
utes. Reliability data on the core indices fall between 0.86
and 0.95, with an average stability coefficient of 0.85. Analy-
ses of content and internal structure validity have shown
satisfactory results, with a factorial structure corresponding
to the 4 indices.
Measure of EF and LP
Wisconsin Card Sorting Test-Learning Potential (WCST-LP)
(Wiedl et al. 2001). This reduced version of the WCST (64
cards) is applied in three phases (pretest-training-posttest),
thus measuring LP with respect to the various EFs evalu-
ated, namely, cognitive flexibility, inhibition, self-regulation
and planning. The assessment starts with the standard static
test (pretest), with the reduced version of the WCST (64
cards). Participants must sort the cards according to a cer-
tain matching criterion (color, shape, number), based on the
experimenter’s feedback. The matching criterion changed
after 10 consecutive correct matches, but without explicit
instruction to the participant. The WCST provides nine main
measurements: (1) correct answers (CA), that is, number
of right matches; (2) total errors (E), number of incorrect
matches; (3) perseverative answers (PA), repeated responses
using the wrong matching-criterion—they could be correct
and incorrect answers; (4) perseverative errors (PE), incor-
rect responses following the perseveration principle; (5) non-
perseverative errors (NPE), incorrect responses not follow-
ing the perseveration principle; (6) conceptual level answers
(CLA), answers showing comprehension of the matching
13

4234
criterion—these are counted as groups of 3 or more correct
responses; (7) number of complete categories (NCC), as the
number of complete sequences (10 correct matches) follow-
ing the right matching criteria; (8) tries to complete the first
category (TC1stC), the number of tries needed to complete
one category for the first time; (9) failure to maintain accu-
racy (FMA), that is, 5 consecutive correct responses fol-
lowed by one error.
Several studies have pointed out that the different meas-
urements are associated with different EFs. That is, CA and
E test task-related functions; PA and PE will relate to cogni-
tive flexibility—the best predictor of prefrontal dysfunction;
CLA relates to abstract thinking; NCC constitutes a global
indicator of conceptual problem-solving skills; TC1stC
would serve as an indicator of initial conceptualization
prior to criterion-change; finally, FMA would test attention
(Leung and Zakzanis 2014; Tsuchiya et al. 2005).
After the pretest (standard WCST procedure with 64
cards), a training phase followed. The experimenter told
the participant that he/she would redo the task with help.
Before starting the second block of cards, the experimenter
explained to the participant that there were three card-match-
ing-rules (according to color, shape, and number, respec-
tively). Each time that the participant placed a card, he/she
was told if his/her answer was right or wrong. After 10 con-
secutive correct responses, the experimenter informed the
participant about the change in matching criteria. Finally,
once the training was done, the posttest assessment was
administered without any help. The pretest assessment pro-
vided the same nine measurements as the traditional version.
However, the WCST-LP added the possibility to calculate
gains and LP after the posttest assessment. Gain scores could
be obtained for any of the traditional measurements using
the formula posttest-pretest = simple gain score. The WCST-
LP has shown evidence of construct validity in patients with
schizophrenia, showing gains from the pretest to the posttest,
that is, after the intervention; there is also evidence of con-
current validity with the Stroop test and the Auditory Verbal
Learning Test (Heubrock 1992); and, finally, evidence of
predictive validity regarding the effectiveness of intervention
programs for job skills and cognitive recovery in patients
with schizophrenia (Wiedl et al. 1999,2004).
Measure of EF (Switching)
Stroop: Color and Word test (Golden 2006). This test is a
measure of switching ability and involves inhibitory con-
trol, assessing, and attentional control. It is applicable in
ages 7 to 80, and consists of three tasks with a duration of
45 s each. The tasks are word reading (Stroop-W), in which
the participant must read the written names of colors; color
naming (Stroop-C), in which he/she must name the color of
the typeface; and color-word (Stroop-CW), in which he/she
13
Journal of Autism and Developmental Disorders (2020) 50:4230–4242
must name the color of the typeface, ignoring any conflict
with the word meaning, which may be a different color. An
interference index is also obtained (Stroop-I), which assesses
interference in the participant and his/her attention control.
Test–retest reliability is 0.89 for Stroop-W, 0.84 for Stroop-
C, 0.73 for Stroop-CW and 0.70 for Stroop-I (Golden 2006).
Measures of Interpersonal Conflict Resolution Skill
Social Comprehension Index–Factor proposed by Golstein
et al. (2008) for high-functioning ASD children and adults,
corresponding to social cognition in these populations. It is
composed of the WISC-IV supplemental subtests of Picture
Completion and Word Reasoning.
Ekman 60 Faces test (Young et  al. 2002). This is a
computerized task assessing emotion recognition in facial
expressions, as a key component of interpersonal conflict
solving skill. The computer screen shows photographs of
faces expressing the six basic emotions (anger, disgust, fear,
happiness, sadness, and surprise). The images (first pub-
lished by Ekman and Friesen 1976) are presented at random
for 5 s each, and the participant is required to indicate his/
her impression of the emotion by pressing the corresponding
key on the keyboard. During the task, a small sign placed
over the keyboard indicates what key should be pressed
for each of the emotions. Separate reliability values have
been obtained for the emotion types (anger: 0.68; disgust:
0.92; fear: 0.88; sadness: 0.65; and surprise: 0.33) with
adequate results (p < 0.001); the value for happiness was
the only exception (0.18), where scores showed a ceiling
effect (Young et al. 2002). Regarding validity of this instru-
ment (Young et al. 2002) compared to the original (Ekman
and Friesen 1976), a strong correlation was found with the
recognition rate presented by Ekman and Friesen (r = 0.81,
t = 10.35, p < 0.001). In Spain, this test has been adapted and
standardized, with similar results to those of the original
sample; Cronbach alpha for the entire sample is 0.93 (hap-
piness: 0.82; anger: 0.69; disgust: 0.73; fear: 0.72; sadness:
0.68; and surprise: 0.73) (see Molinero et al. 2015).
Assessment of Interpersonal Conflict Solving [Evaluación
de Solución de Conflictos Interpersonales (ESCI) (García-
Martín and Calero 2018)]. The test assesses interpersonal
conflict solving skills, namely, emotion recognition in other
people, skill in identifying situational agreement during a
conflict, and skills for generating a viable solution which
will benefit everyone. The test consists of 17 sequences of
sketches that represent an interpersonal conflict, shown on a
computer monitor. The first four sketches show a single per-
son, while the remaining sketches show two or more charac-
ters in a conflict situation. The participant is required to give
written answers to the following questions: (1) How does
the main character in the drawing feel? (2) Why does he/
she feel this way? (3) What could he/she do to remedy this
Journal of Autism and Developmental Disorders (2020) 50:4230–4242 4235
situation? The assessment provides a total score (ESCI-T)
and a score for each construct: emotion (ESCI-E), situational
agreement (ESCI-SA) and solutions (ESCI-S). The instru-
ment was constructed according to the 7-phase model of
interpersonal problem solving by Pelechano (1996), and has
been validated in Spain (García-Martín and Calero 2018).
The Cronbach index obtained was α = 0.90 for the whole
test, with indexes of α = 0.69 for emotion, α = 0.91 for situ-
ational agreement, and α = 0.81 for solutions. Exploratory
factor validity analysis revealed three main factors (emotion,
situational agreement, solutions) and confirmatory factor
validity analysis revealed the same three-factor structure,
as well as a single second-order factor named interpersonal
skills. The test also showed evidence of concurrent validity,
the emotion factor correlating significantly with the Ekman
60 Faces test, the situational agreement factor with Figure
Completion and Picture Arrangement from the Wechsler
Adult Intelligence Scale (WAIS-III; Wechsler 1997), and
the solutions factor with the rational solution sub-scale from
the Social Problem-Solving Inventory-Revised (D’Zurilla
et al. 2002) (see García-Martín and Calero 2018).
Procedure
Firstly, permission from the Ethics Committee of Univer-
sity of Granada was obtained. Secondly, participants were
selected by requesting voluntary participation from ASD
Associations in several Spanish cities. Neither material nor
money incentives were given for participation. After agree-
ing to participate, the psychologists of the associations were
asked to pre-select the participants, following the established
inclusion/exclusion criteria. Thirdly, once the participants
with ASD were identified, a group of children with TD,
matched to the ASD group in age and IQ, was selected from
different schools. Finally, information about the research
goals, procedures and implications was offered in group
meetings with the parents. These meetings were organized
at the ASD centres and the schools. Informed parental con-
sent was obtained for 107 children, but two of them were
excluded because they had additional comorbid disorders
(ADHD). None of the final 105 participants withdrew from
the study.
Participants were individually assessed in three sessions
of approximately one hour each; testing with all the instru-
ments (Spanish versions) was counterbalanced, to avoid a
possible effect of testing order.
Design and Data Analysis
A retrospective ex post facto design with two groups was
used. First, we verified that the sample fulfilled assumptions
of normality and homogeneity using Kolmogorov–Smirnov
and Levene statistics.
• To test H.1, a repeated-measures ANOVA was carried
out with the Greenhouse–Geisser correction.
• To test H.2.1 and H.2.2, multivariate analyses of covari-
ance (MANCOVA) were used, including full IQ as a
covariate to control its possible effect on WCST-LP pre-
test and gain (LP) scores.
• To test objective 3, the Pearson correlation coefficient
was used.
Results
Preliminary analysis included analyses of variance
(ANOVA), showing that the two groups were matched by
age F(1,104) = 1.07, p = 0.303 and full IQ F(1,104) = 0.97,
p = 0.328, but not by sex χ2 = 14.32, p = 0.0001.
The main objective of this study was to analyze the utility
of the set of LP measurements provided by the WCST-LP in
children with ASD, aiming to provide a source of evidence
of both construct and diagnostic validity (controlling for full
IQ), and also to test the relationships between WCST-LP test
performance, cognitive measures (WISC-IV indexes), EF
(switching), and social abilities.
Regarding the first specific objective, that is, to obtain
evidence that supports the construct validity of the WCST-
LP for testing LP in children with ASD, hypothesis (H.1)
stated that children with ASD would benefit from the train-
ing that forms part of the WCST-LP, obtaining significantly
higher posttest results than pretest results, thus demon-
strating LP (see Table 1). In this respect, results from the
repeated measures ANOVA showed significant improve-
ments from pretest to posttest in all WCST-LP measures.
Specifically, CA (F(1,42) = 89.40, p = 0.0001, η2partial = 0.68,
s.p. = 1.00), PA (F(1,42) = 23.69, p = 0.0001, η2partial = 0.36,
s.p. = 0.99), PE (F(1,42) = 29.45, p = 0.0001, η2partial = 0.41,
s.p. = 1.00), NPE (F(1,42) = 12.56, p = 0.001, η2partial = 0.23,
s.p. = 0.93), CLA (F(1,42) = 71.84, p = 0.0001,
η 2 partial  = 0.63, s.p. = 1.00), NCC (F(1,42) = 46.50,
p = 0.0001, η 2 partial  = 0.53, s.p. = 1.00), TC1stC
(F(1,42) = 11.08, p = 0.002, η2partial = 0.21, s.p. = 0.91), and
FMA (F(1,42) = 6.16, p = 0.017, η2partial = 0.13, s.p. = 0.68)
in the ASD group (see Table 1).
The second aim was to determine whether the full IQ
score significantly influenced the diagnostic validity of
the WCST-LP (pretest and gain scores) by comparing its
sensitivity for discriminating between children with ASD
and children with TD, once full IQ had been controlled as
a covariable. Hypothesis (H.2.1) stated that full IQ would
have a significant effect on performance level in WCST-LP
pretest scores (performance scores similar to those obtained
13

4236

13
Table 1  Descriptive statistic, range, minimum and maximum of WCST-LP in ASD and TD groups and multivariate analyses del repeated measures ANOVA with one within-subjects factor-
assessment moment (from pre-treatment to post-treatment) in ASD group
Group Variable Test range Pre-test Post-test Gain score Multivariate analyses
M SD Min Max M SD Min Max M SD Min Max F(1,42) p η2p s.p

Group ASD (n = 43) WCST-LP

   CA 0–64 34.35 12.69 13.00 57.00 50.81 9.48 13.00 63.00 16.47 11.42 − 3.00 44.00 89.40 .0001 .68 1.00
   PA 0–63 18.79 16.62 2.00 62.00 7.93 8.02 2.00 52.00 − 10.86 14.63 − 56.00 13.00 23.69 .0001 .36 .99
   PE 0–63 15.71 12.36 2.00 46.00 6.83 6.03 1.00 39.00 − 8.88 10.73 − 40.00 8.00 29.45 .0001 .41 1.00
   NPE 0–64 12.33 12.64 .00 42.00 5.91 6.97 .00 42.00 − 6.43 11.89 − 38.00 12.00 12.56 .001 .23 .93
   CLA 0–61 28.81 16.32 .00 55.00 46.14 11.84 .00 60.00 19.33 14.95 − 4.00 56.00 71.84 .0001 .63 1.00
   NCC 0–6 1.81 1.47 .00 5.00 3.33 1.46 .00 6.00 1.51 1.45 − 2.00 5.00 46.50 .0001 .53 1.00
   TC1stC 10–64 26.88 23.02 10.00 65.00 16.12 13.13 10.00 65.00 − 10.77 21.21 − 55.00 24.00 11.08 .002 .21 .91
   FMA 0–10 0.56 0.73 .00 3.00 1.07 1.06 .00 3.00 0.51 1.35 − 2.00 3.00 6.16 .017 .13 .68
Group TD (n = 62) WCST-LP
   CA 0–64 31.77 9.90 13.00 52.00 49.82 8.15 22.00 61.00 18.05 12.64 − 14.00 43.00
   PA 0–63 17.13 13.28 4.00 62.00 6.53 4.28 1.00 24.00 − 10.60 13.41 − 59.00 7.00
   PE 0–63 14.77 9.61 4.00 46.00 6.19 3.70 1.00 19.00 − 8.58 9.91 − 43.00 6.00
   NPE 0–64 17.44 10.00 1.00 42.00 7.94 5.48 1.00 29.00 − 9.50 11.66 − 37.00 14.00
   CLA 0–61 23.47 12.35 0.00 47.00 45.73 10.43 10.00 59.00 22.26 15.54 − 19.00 50.00
   NCC 0–6 1.69 1.08 0.00 4.00 3.15 1.27 1.00 5.00 1.45 1.60 − 2.00 5.00
   TC1stC 10–64 25.69 21.01 10.00 65.00 15.84 10.80 10.00 53.00 − 9.85 21.94 − 55.00 33.00
   FMA 0–10 0.44 0.67 0.00 3.00 1.26 1.09 .00 4.00 0.82 1.33 − 2.00 4.00

WCST-LP Wisconsin Card Sorting Test-Learning Potential; CA Correct answers, PA Perseverative answers, PE perseverative errors, NPE Non-perseverative errors, CLA Conceptual level
answers, NCC Number of complete categories, TC1stC Tries to complete the first category; FMA Failure to maintain the attitude
Inverse variable in bold
Journal of Autism and Developmental Disorders (2020) 50:4230–4242
Journal of Autism and Developmental Disorders (2020) 50:4230–4242 4237

on a standard application of the WCST), for both ASD and
TD children. Accordingly, the results of the MANCOVA did
not show significant between-group differences in WCST-LP
pretest scores Ʌ = 0.87, F(8,95) = 1.73, p = 0.101, η2 = 0.13,
s.p. = 0.72 when controlling for full IQ. Univariate analy-
ses only showed significant differences between the two
groups in the NPE pretest score (see Table 2), favoring the
ASD group (see Table 2). Results also indicated a signifi-
cant effect of full IQ on pretest scores in CA, PA, PE, CLA,
TC1stC, FMA (see Table 2) confirming our hypothesis.
Regarding the direction of full IQ, b-value results indicate
a positive relationship in CA (b = 0.210), CLA (b = 0.265)
and FMA (b = 0.012); and a negative relationship in meas-
urements of PA (b = − 0.296), PE (b = − 0.222) and TC1stC
(b = − 0.330) for the pretest.
The second hypothesis of this objective (H.2.2) con-
sidered that full IQ would not have a significant effect on
WCST-LP gain scores (pretest-to-posttest improvement
after receiving training), for either children with ASD or
children with TD. MANCOVA results did not show sig-
nificant between-group differences in the gain scores when
the full IQ effect was controlled (see Table 2). Regarding
the covariable, results indicate that full IQ did not have a
significant effect on results obtained in gain scores, except
for the TC1stC measure (see Table 2), thereby confirming
the hypothesis for the most part. For the TC1stC measure,
b-value results indicate that intelligence had a positive direc-
tion (b = 0.368).
The third aim was to explore the relations between
WCST-LP measurements (pretest and gain scores) and the
indexes of IQ, EF and social abilities in children with ASD
vs. children with TD. In this case, results for the ASD group
showed statistically significant, there were positive Pearson
correlations between the WCST-LP measures in CA, CLA,
NCC and FMA, with VC, PR, WM, PS, SC, Stroop PC
and ESCI-T; and negative correlations between WCST-LP
measures of PA, PE and TC1stC, with VC, PR, WM, PS,
SC, Stroop PC and ESCI-T (see Table 3). Gain measures
(estimated LP), however, showed a lower number of signifi-
cant correlations, limited to a few WISC-IV indexes: VC
index correlated significantly with gain scores in PA, PE and
TC1stC; WM index correlated with gain scores in NCC and

Table 2  Mean difference (MANCOVA-Full-IQ covariate) between Group ASD and TD in the WCST-L’s measures in pretest assessment and
gain score as a function of Group (ASD and TD)
Variable Group TD Group ASD Mean difference Covariate
(n = 62) (n = 43)
M* SD M* SD F(1.103) p η2partial s.p F(1.103) p η2partial s.p

WCST-LP pretest
 CA 32.00 9.90 34.03 12.69 0.89 .349 .01 .15 6.88 .010** .06 .74
 PA 16.81 13.28 19.24 16.62 0.73 .395 .01 .14 7.89 .006** .07 .80
 PE 14.53 9.61 16.02 12.36 0.54 .464 .01 .11 8.61 .004** .08 .83
 NPE 17.46 10.00 12.29 12.65 5.35 .023* .05 .63 0.10 .757 .001 .06
 CLA 23.75 12.35 26.41 16.32 0.94 .334 .01 .16 6.81 .010** .06 .73
 NCC 1.71 1.08 1.79 1.47 0.10 .750 .001 .06 3.01 .086 .03 .41
 TC1stC 25.34 21.01 27.39 23.02 0.23 .634 .002 .08 4.31 .040* .04 .54
 FMA 0.45 0.67 0.54 0.73 0.45 .505 .004 .10 6.05 .016* .06 .68
WCST-LP gain score
 CA 17.95 12.64 16.61 11.42 0.30 .583 .003 .09 1.13 .291 .01 .18
 PA − 10.41 13.42 − 11.12 14.63 0.07 .798 .001 .06 2.83 .096 .03 .38
 PE − 8.44 9.91 − 9.09 10.73 0.10 .749 .001 .06 3.23 .075 .03 .43
 NPE − 9.53 11.66 − 6.38 11.90 1.80 .183 .017 .26 0.14 .714 .001 .07
 CLA 22.15 15.54 19.48 14.95 0.76 .385 .01 .14 0.83 .363 .008 .15
 NCC 1.44 1.60 1.52 1.45 0.07 .798 .001 .06 0.41 .521 .004 .10
 TC1stC − 9.46 21.94 − 11.33 21.21 0.20 .659 .002 .07 5.51 .021* .05 .64
 FMA 0.82 1.33 0.51 1.35 1.38 .243 .01 .21 0.02 .888 .001 .05

WCST-LP Wisconsin card sorting test-learning potential, CA Correct answers, PA Perseverative answers, PE Perseverative errors, NPE Non-
perseverative errors, CLA Conceptual level answers, NCC Number of complete categories, TC1stC Tries to complete the first category, FMA
Failure to maintain the attitude
Inverse variable in bold
M*: Means are estimated as a function of a full IQ covariate value = 92.6286
p < .05*. p < .01**

13

4238

13
Table 3  Bivariated correlations in pretest and gain scores WCST-LP’s measures with indexes of WISC-IV, Stroop-PC, Stroop-I, Ekman total and ESCI-Total (EF measures and social compe-
tency) as a function of Group (ASD and TD)
WCST-LP Group ASD (n = 43) Group TD (n = 62)
VC PR WM PS Stroop-PC Stroo-I* SC Ekman ESCI-T VC RP WM PS Stroop-PC Stroop-I* SC Ekman ESCI-T

Pretest
 CA .42** .31* .37* .48** .32* .0001 .48** .14 .31* − .035 .71 − .03 .08 .07 − .02 .08 .22 .16
 PA − .47** − .33* − .28 − .35* − .19 − .04 − .54** .10 − .13 .004 − .12 − .07 − .03 .05 − .03 − .05 − .09 − .10
 PE − .46** − .34* − .28 − .36* − .22 − .06 − .54** .07 − .13 .001 − .11 − .10 − .08 .02 − .04 − .06 − .10 − .12
 NPE .03 .01 − .03 − .17 − .01 .11 .09 − .29 − .31* − .03 .04 .12 − .01 − .09 .05 − .02 − .13 − .05
 CLA .43** .30 .39** .48** .30* − .02 .49** .18 .35* − .02 .03 − .04 .04 .07 − .01 .01 .15 .07
 NCC .37* .21 .34* .32* .18 − .11 .37* .22 .26 − .10 .01 − .15 .04 .05 − .02 .08 .06 .03
 TC1stC − .46** − .26 − .38* − .38* − .14 .27 − .56** − .21 − .34* .11 − .09 .10 − .04 .10 .13 − .05 .02 − .05
 FMA .32* .19 .26 .40** .34* .16 .42* − .06 .20 .12 .13 .09 − .002 .05 .15 − .02 .13 − .01
Gain score
 CA − .21 − .18 − .21 − .29 − .24 .02 .43** − .21 .05 .04 − .002 .09 − .17 − .19 − .10 − .002 − .23 − .11
 PA .30* .19 .15 .27 .14 .09 .43** − .19 − .16 − .01 .08 .05 .06 − .01 .08 .05 .11 .08
 PE .31* .20 .16 .29 .19 .12 − .19 .05 − .12 − .01 .06 .06 .11 .03 .09 .05 .12 .09
 NPE .07 .01 .05 .06 .03 − .14 − .21 .17 .14 − .03 − .05 − .14 .08 .19 .03 − .04 .15 .04
 CLA .17 − .16 − .20 . − 27 − .19 .04 − .13 .23 − .03 .02 .05 .06 − .12 − .20 − .11 .07 − .17 − .04
 NCC − .21 − .08 − .31* .01 − .09 .06 .47** .06 .02 .08 − .01 .18 − .18 − .21 − .13 − .01 .01 .14
 TC1stC .51** .28 .48** .35* .24 − .19 − .13 − .01 .28 − .09 .12 − .13 .16 − .01 − .11 .07 .01 .02
 FMA .01 − .06 .05 − .29 − .17 .02 .48** .14 − .01 − .02 .14 − .01 .13 .002 .01 .07 − .17 − .17

WCST-LP Wisconsin card sorting test-learning potential, CA Correct answers, PA Perseverative answers, PE Perseverative errors, NPE Non-perseverative errors, CLA Conceptual level answers,
NCC Number of complete categories, TC1stC Tries to complete the first category, FMA Failure to maintain the attitude
VC: Verbal Comprehension index WISC-IV; PR: Perceptive Reasoning index WISC-IV; WM: Working Memory index WISC-IV; PS: Processing Speed index WISC-IV; Stroop-PC: Stroop Test
word-color measure; Stroop-I: Stroop Test interference index; SC: Social Comprehension index WISC-IV; Ekman: Total score Ekman test; ESCI-T: Evaluación de Solución de Conflictos Inter-
personales [Assessment of Interpersonal Conflict Resolution] Total score
Inverse variable in bold
p < .05*. p < .01**
Journal of Autism and Developmental Disorders (2020) 50:4230–4242
Journal of Autism and Developmental Disorders (2020) 50:4230–4242 4239
TC1stC; PS index correlated with gain scores in TC1stC;
and SC index correlated with gain scores in CA, PA, NCC
and FMA. All correlations were in a positive direction (see
Table 3). In the TD group, there were no significant corre-
lations between WCST-LP pretest and gain scores with the
other measures (see Table 3).
Discussion
The main objective of this study was to analyze the use-
fulness of the WCST-LP set of measurements in children
with ASD. We sought to find some source of evidence of
both construct validity (improved scores after a training
phase) and diagnostic validity (discriminating between
children with ASD and children with TD in pretest and gain
scores, when controlling for full IQ), and to test the relation-
ships between WCST-LP performance, cognitive measures
(WISC-IV indexes), EF (switching), and social abilities.
The findings represent a step forward in the field of
neuropsychological assessment, providing evidence of the
validity of a WCST adaptation that measures capacity for
learning in EFs-related tasks in the ASD population. Until
now, only one published study had reported data showing the
possible diagnostic and prognostic utility of these measures
(Calero et al. 2015). Now, we have taken a further step by
showing the differential influence of intelligence on pretest
and gain scores, and by exploring the relations between the
measures supplied by this version of the WCST, and impor-
tant performance variables of children with ASD, such as
their verbal and social comprehension skills. The funda-
mental importance of these skills is that they inform us of
the individual’s competence for daily life and for relating
to his or her environment. This study can thus contribute to
the development of neuropsychological measures of LP in
children with ASD, having shown its relationship to verbal
and social skills.
Overall, the findings support the idea that children with
ASD can benefit from brief training included in a classic EF
assessment task (WCST-LP); thus, they demonstrate LP, like
children with TD. The test therefore offers some evidence
of construct validity in an ASD population. These results
agree with the few previous studies that investigate LP
and children with ASD (Aljunied and Frederickson 2011a,
2011b; Bonete et al. 2010; Calero et al. 2015; Donaldson
and Olswang 2007) and with previous research on LP in
populations with severe developmental conditions (Calero
2012; Haywood and Lidz 2007). Furthermore, this study
shows how LP extends to all types of WCST-LP responses,
and hence, to the different EF-related domains assessed by
these (cognitive flexibility, inhibition, self-regulation, plan-
ning), extending the available information regarding use of
the WCST-LP in general, and with an ASD population spe-
cifically. These results add supplemental evidence to other
previous studies of LP in patients with schizophrenia, which
have traditionally reduced LP measures to Correct responses
or Perseverative answers (Bonete 2012).
The second goal was to determine the influence of intelli-
gence in the diagnostic validity of the WCST-LP, by observ-
ing whether its pretest and gain scores were sensitive for
discriminating between children with ASD and children
with TD. The study has shown that when intelligence is
controlled, children with ASD do not differ from children
with TD, except in the NPE measure. These results high-
light the influence of intelligence on performance in the
pretest level of the different WCST-LP scores. This result
was to be expected, given that the data are showing the
influence of a static cognition measure (full IQ) on other
measures that are also static (WCST-LP pretest scores); in
other words, measures that assess performance ability and
demonstrate acquired skills at the moment of an EF task
assessment (Calero 2012; Haywood and Lidz 2007). In this
regard, these findings agree with those of Rumsey (1985),
Ozonoff (1995), and Liss et al. (2001), indicating that level
of intelligence plays a role in one’s performance on tradi-
tional versions of the WCST, and that it should be controlled
in diagnostic assessments of children with possible ASD.
However, the present study represents an advance in the
existing knowledge, by showing how dynamic measures of
LP assessment in EF tasks are independent of one’s former
IQ (Calero 2012; Haywood and Lidz 2007). In this way,
this version of the WCST is shown to contribute qualita-
tively different information from the traditional version of
the test, and allows us to assess cognitive modifiability, or
the capacity for change (learning) in significant tasks for the
ASD population. The fact that LP scores are not influenced
by intelligence leads us to interpret them as a prognostic
indicator that can identify the capacity for learning in ASD
children prior to planning their intervention, regardless of
previous factors like IQ.
The third exploratory objective allowed us to determine
that pretest measures of the WCST-LP were related to intel-
ligence, flexibility and social abilities in children with ASD
(positively, in that more right answers accompanied higher
IQ; and negatively, in that more perseverations accompanied
lower IQ); this relationship was not found in the group of
children with TD. However, the most significant gains in
WCST-LP scores (CA, PE) correlated to performance on
measures of verbal comprehension and social skills only in
the children with ASD, while there were no correlations in
the TD children. This may indicate their prognostic value
for these competencies, beyond that of intelligence and/or
performance measures. These children’s LP is related to
their verbal and social competencies, not to their IQ. Simi-
lar findings had been found by Wiedl and Wienöbst (1999)
13

4240
in their study of adults with a schizophrenia diagnosis, by
Calero and Navarro (2007) in older adults with dementia,
and by Calero et al. (2015), where significant relationships
were found between LP scores in CA on the WCST-LP, and
emotion recognition measures and conflict resolution skills.
In the studies cited above, this sensitivity to verbal and social
comprehension was accounted for by the fact that the inter-
vention supplied within the LP assessment is based on ver-
bal interaction, and requires collaborative work between the
assessor and the subject; it is act of learning within a social
context, modulated by verbal skills. On the other hand, these
results do not agree with those of Bonete et al. (2010), who
did not find significant correlations between LP score and
deficits in interpersonal skills, probably due to their sample
size of 10 participants with ASD, a number that limits the
validity of their results.
Another interesting finding is the absence of significant
correlations between any of the WCST-LP measures and the
other measurements in children with TD. For this reason, the
WCST-LP appears to be a sensitive task for revealing EF
deficits that relate to the social and non-social behavior of
children with ASD in their everyday routine; these deficits
cannot be observed in children with TD (Happé, et al. 2006;
Hill 2004a; Leung et al. 2015; Pugliese et al. 2016).
Overall, these findings must be interpreted with consid-
eration of certain limitations affecting the study methodol-
ogy. Firstly, information about prior intervention received
by the participants with ASD could not be controlled: these
children had a prior diagnosis and therefore were receiving
regular therapy sessions, but that information could not be
accessed or controlled in this study. Secondly, although our
sample of children with ASD might be large enough, and
certainly has surpassed the sample size of previous stud-
ies (Bonete et al. 2010; Donaldson and Olswang 2007), the
present sample size cannot guarantee ecological validity of
these findings. The small size of the ASD group has made
it difficult to compare these children’s performance accord-
ing to their IQ levels, and there were also discrepancies
between the IQ indexes. Finally, the small number of girls
in the sample precluded any consideration of possible gender
differences.
Therefore, generally speaking, this study shows that the
WCST-LP may be a useful assessment tool in children with
ASD both for making a diagnosis (pretest performance) and
for establishing a prognosis (based on gain scores), because
it offers construct-based evidence of the validity of the
WCST-LP in children with ASD. The availability of a meas-
ure for effectively assessing learning capacity profiles could
help in understanding the ASD prognosis. The WCST-LP
can be complementary to traditional assessment measures,
offering dynamic measures that identify the individual’s
cognitive modifiability. In other words, a measure with
special emphasis in LP can be included for consideration
13
Journal of Autism and Developmental Disorders (2020) 50:4230–4242
in individualized intervention designs; it informs about the
individual’s ability to profit from training, which may be a
possible predictive index of change after the intervention.
Further testing of the utility of this test in an ASD popula-
tion is still required, because only two studies have pursued
this goal to date. The stability of these results must be veri-
fied, on one hand, to understand the utility of LP for deter-
mining inter-individual differences in children with ASD
(investigating different LP profiles/levels), and on the other
hand, to analyze its predictive power of intervention success.
Finally, in light of these findings, it would be interesting
to compare test results in groups of children with ASD who
have different intelligence levels, to try to clarify this test’s
discriminant validity for ASD children with different IQ lev-
els (Aljunied and Frederickson 2011a, b; Ozonoff 1995; Liss
et al. 2001; Rumsey 1985).
Conclusions
This study has provided a source of construct-based evi-
dence for the validity of the WCST-LP set of measurements,
and for its greater prognostic sensitivity in children with
ASD, based on their capacity for learning when full IQ is
controlled. The results obtained here have indicated the per-
tinence of including the WCST-LP set of measures in ASD
prognosis assessment. Moreover, the WCST-LP is a measure
of LP, which provides information about possible satisfac-
tory teaching styles for each person.
Finally, this study has confirmed the relation between LP
and verbal and social comprehension, again highlighting
the need for comprehensive assessment and intervention in
children with ASD.
Acknowledgment  We want to thank to all children for their participa-
tion, their parents as well as educators & psychologist working with
them in the different ASD Associations who collaborated in this study.
Funding  The research reported in this paper is funded by the Junta de
Andalucía (HUM-129 & HUM-820), and the Research project (Excel-
lence call, 2012) [P12-SEJ-560]
Compliance with Ethical Standards 
Conflict of interests  The authors declare that they have no conflict of
interest.
References
Aljunied, M., & Frederickson, N. (2011a). Does central coherence
relate to the cognitive performance of children with autism
Journal of Autism and Developmental Disorders (2020) 50:4230–4242 4241
in dynamic assessments? Autism, 17, 172–183. https​: //doi.
org/10.1177/13623​61311​40996​0.
Aljunied, M., & Frederickson, N. (2011b). Cognitive indicators of
different levels of special educational support needs in autism.
Research in Autism Spectrum Disorders, 5(1), 368–376.
American Psychiatric Association. (2013). Diagnostic and statisti-
cal manual of mental disorders (5th ed.). American Psychiatric
Association: Washington, DC.
Blijd-Hoogewys, E. M., Bezemer, M. L., & Van Geert, P. L. (2014).
Executive functioning in children with ASD: An analysis of the
BRIEF. Journal of Autism Development Disorder, 44, 3089–
3100. https​://doi.org/10.1007/s1080​3-014-2176-9.
Bonete, S. (2012). Evaluación del potencial de aprendizaje en psico-
sis y trastornos del desarrollo [Assessment of learning potential
in psychosis and developmental disorders.]. In M. D. Calero
(Ed.), Evaluación del potencial de aprendizaje: Fundamentos y
aplicaciones [Assessment of learning potential: Fundamentals
and applications] (pp. 163–206). Madrid: EOS.
Bonete, S., Vives, M. C., Fernández-Parra, A., Calero, M. D., &
García-Martín, M. B. (2010). Potencial de aprendizaje y habi-
lidades sociales en escolares con el Trastorno de Asperger
[Learning potential and social skills in schoolchildren with
Asperger’s Disorder]. Behavioral Psychology, 18, 473–490.
Brunsdon, V. E. A., Colvert, E., Ames, C., Garnett, T., Gillan, N.,
Hallett, V., … Happé, F. (2015). Exploring the cognitive fea-
tures in children with autism spectrum disorder, their co-twins,
and typically developing children within a population-based
sample. The Journal of Child Psychology and Psychiatry, 56,
893–902. https​://doi.org/10.1111/jcpp.12362​
Calero, M. D. (2012). Evaluación del potencial de aprendizaje:
fundamentos y aplicaciones [Learning potential assessment:
Fundamentals and applications]. Madrid: EOS.
Calero, M. D., Mata, S., Bonete, S., Molinero, C., & Gómez-Pérez,
M. M. (2015). Relations between learning potential, cognitive
and interpersonal skills in Asperger children. Learning and
Individual Differences, 44, 53–60. https​: //doi.org/10.1016/j.
lindi​f.2015.07.004.
Calero, M. D., & Navarro, E. (2007). Cognitive plasticity as a mod-
ulating variable on the effects of memory training in elderly
persons. Archives of Clinical Neuropsychology, 22(1), 63–72.
https​://doi.org/10.1016/j.acn.2006.06.020.
De Vries, M., & Geurts, H. M. (2012). Cognitive flexibility in
ASD: Task switching with emotional faces. Journal of Autism
and Developmental Disorders, 42, 2558–2568. https​: //doi.
org/10.1007/s1080​3-012-1512-1.
De Vries, M., Prins, P. J., Schmand, B. A., & Geurts, H. M. (2015).
Working memory and cognitive flexibility-training for children
with an Autism Spectrum Disorder: A randomized controlled
trial. Journal of Child Psychology and Psychiatry, 56(5),
566–576.
Diamond, A. (2013). Executive functions. Annual Review of Psychol-
ogy, 64, 135–168. https​://doi.org/10.1146/annur​ev-psych​-11301​
1-14375​0.
Donaldson, A. L., & Olswang, L. B. (2007). Investigating requests
for information in children with Autism Spectrum Disorders:
Static versus dynamic assessment. Advances in Speech-Language
Pathology, 9, 297–311. https:​ //doi.org/10.1080/144170​ 40701​ 4137​
20.
D’Zurilla, T. J., Nezu, A. M., & Maydeu-Olivares, A. (2002). Social
problem-solving Inventory—revised (SPSI-R). North Tonawanda,
NY: Multi-Health Systems.
Ekman, P., & Friesen, W. V. (1976). Pictures of facial affect. Palo Alto,
CA: Consulting Psychologists Press.
Feuerstein, R., Rand, Y., & Hoffman, M. B. (1979). Dynamic assess-
ment of retarders performers. The learning assessment device:
Theory, instruments and techniques. Baltimore: University Park
Press.
Filley, C. M., Young, D. A., Reardon, M. S., & Wilkening, G. N.
(1999). Frontal lobe lesions and executive dysfunction in children.
Neuropsychiatry, Neuropsychology, and Behavioral Neurology,
12(3), 156–160.
García-Martín, M. B., & Calero, M. D. (2018) ESCI: Solución de Con-
flictos Interpersonales. Cuestionario y Programa de Entrenami-
ento [ESCI: Interpersonal Conflict Resolution. Questionnaire and
Intervention Program]. Bogotá: El Manual Moderno.
Goldstein, G., Allen, D. N., Minshew, N. J., Williams, D. L., Volkmar,
F., Klin, A., et al. (2008). The structure of intelligence in children
and adults with high functioning autism. Neuropsychology, 22(3),
301.
Gómez-Pérez, M. M., Calero, M. D., Mata, S., & Molinero, C. (2016).
Discrepancies between direct and indirect measures of interper-
sonal and neurocognitive skills in autism spectrum disorder chil-
dren. Journal of Clinical and Experimental Neuropsychology, 38,
875–886. https​://doi.org/10.1080/13803​395.2016.11701​06.
Geurts, H. M., Verté, S., Oosterlaan, J., Roeyers, H., & Sergeant, J. A.
(2004). How specific are executive functioning deficits in atten-
tion deficit hyperactivity disorder and autism? Journal of Child
Psychology and Psychiatry, 45, 836–854. https​://doi.org/10.111
1/j.1469-7610.2004.00276​.x.
Golden, C. J. (2006). Stroop: Test de Colores y Palabras [Stroop: Color
and Word test]. Madrid: TEA ediciones.
Happé, F., Booth, R., Charlton, R., & Hughes, C. (2006). Executive
function deficits in autism spectrum disorders and attention defi-
cit/ hyperactivity disorder: Examining profiles across domains and
ages. Brain and Cognition, 61, 25–39. https​://doi.org/10.1016/j.
bandc​.2006.03.004.
Heaton, R. K. (1981). The Wisconsin Card Sorting Test manual.
Odessa, FL: Psychological Assessment Resources.
Haywood, H. C., & Lidz, C. S. (2007). Dynamic assessment in prac-
tice: Clinical and educational applications. Cambridge: Univer-
sity Press.
Heubrock, D. (1992). The Auditory Verbal Learning Test (AVLT)
in clinical and experimental neuropsychology: Administration,
evaluation and research findings. Zeitschrift fuer Differentielle und
Diagnostische Psychologie, 3, 161–174.
Hill, E. L. (2004a). Executive dysfunction in autism. Trends in Cogni-
tive Science, 8, 26–32. https​://doi.org/10.1016/j.tics.2003.11.003.
Hill, E. L. (2004b). Evaluating the theory of executive dysfunction
in autism. Developmental Review, 24, 189–233. https​://doi.
org/10.1016/j.dr.2004.01.001.
Landry, O., & Al-Taie, S. (2016). A meta-analysis of the Wisconsin
Card Sort Task in Autism. Journal of Autism Development Disor-
der, 46, 1220–1235. https​://doi.org/10.1007/s1080​3-015-2659-3.
Landry, S. H., Smith, K. E., & Swank, P. R. (2009). New directions
in evaluating social problem solving in childhood: Early precur-
sors and links to adolescent social competence. New Directions
for Child and Adolescent Development, 123, 51–68. https​://doi.
org/10.1002/cd.235.
Le Couteur, A. N. N. E., Lord, C., & Rutter, M. (2003). The autism
diagnostic interview-revised (ADI-R) (pp. 659–685). Los Angeles,
CA: Western Psychological Services.
Leung, R. C., & Zakzanis, K. K. (2014). Brief report: Cognitive flex-
ibility in autism spectrum disorders: A quantitative review. Jour-
nal of Autism Development Disorder, 44, 2628–2645. https​://doi.
org/10.1007/s1080​3-014-2136-4.
Leung, R. C., Vogan, V. M., Powell, T. L., Anagnostou, E., & Taylor,
M. J. (2015). The role of executive functions in social impair-
ment in Autism Spectrum Disorder. Child Neuropsychology, 22,
336–344. https​://doi.org/10.1080/09297​049.2015.10050​66.
Liss, M., Harel, B., Fein, D., Allen, D., Dunn, M., Feinstein, C.,
… Rapin, I. (2001). Predictors and correlates of adaptive
13

4242
functioning in children with developmental disorders. Journal
of  Autism  Developmental Disorder, 32, 219-230. https​://doi.
org/10.1023/A:10107​07417​274
Lord, C., Rutter, M., DiLavore, P. C., & Risi, S. (2002). Escala de
Observación de Diagnóstico de Autismo [Autism Diagnostic
Observation Schedule, ADOS]. Madrid: TEA Ediciones.
Maddio, S. L., & Greco, C. (2010). Cognitive flexibility to solve inter-
personal problems. Does this ability differ among peers in chil-
dren from elementary public urban and urban-marginal schools?
Interamerican Journal of Psychology, 44, 98–109.
Molinero, C., Bonete, S., Gómez-Pérez, M. M., & Calero, M. D.
(2015). Normative study of Ekman’s 60-face test for Spanish
adolescents. Behavioral Psychology, 23, 361–371.
Montgomery, J. M., Stoesz, B. M., & McCrimmon, A. W. (2013).
Emotional intelligence, theory of mind, and executive functions
as predictors of social outcomes in young adults with Asperger
Syndrome. Focus on Autism and Other Developmental Disabili-
ties, 28, 4–13. https​://doi.org/10.1177/10883​57612​46152​5.
Mostert-Kerckhoffs, M. A. L., Staal, W. G., Houben, R. H., & De-
Jonge, M. V. (2015). Stop and change: Inhibition and flex-
ibility skills are related to repetitive behavior in children and
young adults with autism spectrum disorders. Journal of Autism
and Developmental Disorders, 45, 3148–3158. https ​ : //doi.
org/10.1007/2Fs10​803-015-2473-y.
Ozonoff, S. (1995). Reliability and validity of the Wisconsin Card
Sorting Test in studies of autism. Neuropsychology, 9, 491–500.
Ozonoff, S., Goodlin-Jones, B. L., & Solomon, M. (2005). Evidence
based assessment of Autism Spectrum Disorder in children and
adolescents. Journal of Clinical Child and Adolescent Psychol-
ogy, 34, 523–540. https:​ //doi.org/10.1207/s15374​ 424jc​ cp340​ 3_8.
Pelechano, V. (Ed.). (1996). Habilidades interpersonales: Teoría
mínima y programas de intervención [Interpersonal skills:
Minimum theory and intervention programs] (Vol. 1). Valencia:
Promolibro.
Pooragha, F., Kafi, S. M., & Sotodeh, S. O. (2013). Comparing
response inhibition and flexibility for two components of execu-
tive functioning in children with autism spectrum disorder and
normal children. Iranian Journal of Pediatrics, 23, 309–314.
Pugliese, C. E., Anthony, L. G., Strang, J. F., Dudley, K., Wallace,
G. L., Naiman, D. Q., et al. (2016). Longitudinal examination
of adaptive behavior in autism spectrum disorders: Influence of
executive function. Journal of Autism and Developmental Disor-
ders, 46, 467–477. https​://doi.org/10.1007/s1080​3-015-2584-5.
Robinson, S., Goddard, L., Dritschel, B., Wisley, M., & Howlin, P.
(2009). Executive functions in children with Autism Spec-
trum Disorders. Brain and Cognition, 71, 362–368. https​://doi.
org/10.1016/j.bandc​.2009.06.007.
Romine, C. B., Lee, D., Wolfe, M. E., Homack, S., George, C., & Ric-
cio, C. A. (2004). Wisconsin Card Sorting Test with children: A
meta-analytic study of sensitivity and specificity. Archives of Clin-
ical Neuropsychology, 19, 1027–1041. https​://doi.org/10.1016/j.
acn.2003.12.009.
13
Journal of Autism and Developmental Disorders (2020) 50:4230–4242
Rumsey, J. M. (1985). Conceptual problem-solving in highly verbal,
nonretarded autistic men. Journal of Autism and Developmental
Disorders, 15, 23–36.
Tsuchiya, E., Oki, J., Yahara, N., & Fujieda, K. (2005). Computer-
ized version of the Wisconsin Card Sorting Test in children with
high-functioning autistic disorder or attention-deficit/hyperac-
tivity disorder. Brain & Development, 27, 233–236. https​://doi.
org/10.1016/j.brain​dev.2004.06.008.
Van Eylen, L., Boets, B., Steyaert, J., Wagemans, J., & Noens, I. (2015).
Executive functioning in autism spectrum disorders: influence of
task and sample characteristics and relation to symptom severity.
European Child & Adolescent Psychiatry, 24, 1399–1417. https​
://doi.org/10.1007/s0078​7-015-0689-1.
Vygostky, L. S. (1978). El desarrollo de los procesos psicológicos
superiores. [Development of higher psychological processes.]
Barcelona: Grijalbo.
Wechsler, D. (1997). WAIS-III: Escala de inteligencia Wechsler para
adultos-III [Wechsler Intelligence Scale for Adults-III]. Madrid:
Manual Moderno.
Wechsler, D. (2005). Escala de inteligencia de Wechsler para niños-IV
(WISC-IV) [Wechsler Intelligence Scale for Children-IV (WISC-
IV)]. Madrid: TEA ediciones.
Westwood, H., Stahl, D., Mandy, W., & Tchanturia, K. (2016). The set-
shifting profiles of anorexia nervosa and autism spectrum disorder
using the Wisconsin Card Sorting Test: A systematic review and
meta-analysis. Psychological Medicine, 46, 1809–1827. https​://
doi.org/10.1017/S0033​29171​60005​81.
Wiedl, K. H., Schottke, H., & Calero, M. D. (2001). Dynamic assess-
ment of cognitive rehabilitation potential in schizophrenic per-
sons and in elderly persons with and without dementia. European
Journal of Psychological Assessment, 17, 112–119. https​://doi.
org/10.1027//1015-5759.17.2.112.
Wiedl, K. H., Schöttke, H., Green, M. F., & Nuechterlein, K. H. (2004).
Dynamic testing in schizophrenia: Does training change the con-
struct validity of a test? Schizophrenia Bulletin, 30, 703–711. https​
://doi.org/10.1093/oxfor​djour​nals.schbu​l.a0071​24.
Wiedl, K. H., & Wienöbst, J. (1999). Inter-individual differences in
cognitive remediation research with schizophrenic patient’s indi-
cators of rehabilitation potential? International Journal of Reha-
bilitation Research, 22, 1–5.
Wiedl, K. H., Wiënobst, J., & Schöttke, H. (1999). Estimation reha-
bilitation potential in schizophrenic subjects. In F. Böker, H. D.
Brenner, & R. Genner (Eds.), Rehabilitation of cognitive disorders
in schizophrenic patients (pp. 221–242). Bern: Huber, P.
Young, A. W., Perrett, D., Calder, A., Sprengelmeyr, R., & Ekman,
P. (2002). Facial expressions of emotions: Stimuli and test. San
Antonio, TX: Harcourt Assessment.
Publisher’s Note Springer Nature remains neutral with regard to
jurisdictional claims in published maps and institutional affiliations.
Journal of Autism & Developmental Disorders is a copyright of Springer, 2020. All Rights
Reserved.