See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/222008635

Reading, listening, and viewing comprehension in English as a foreign

language: One or more constructs?

Article  in  Intelligence · November 2010

DOI: 10.1016/j.intell.2010.09.003

CITATIONS READS

22 5,101

3 authors, including:

Ulrich Schroeders Oliver Wilhelm

Universität Kassel Ulm University
97 PUBLICATIONS   1,545 CITATIONS    239 PUBLICATIONS   10,800 CITATIONS   

SEE PROFILE SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Special Issue "New Methods and Assessment Approaches in Intelligence Research" View project

Anchoring Vignettes in Personality Research View project

All content following this page was uploaded by Oliver Wilhelm on 01 November 2017.

The user has requested enhancement of the downloaded file.

This article appeared in a journal published by Elsevier. The attached
copy is furnished to the author for internal non-commercial research
and education use, including for instruction at the authors institution
and sharing with colleagues.
Other uses, including reproduction and distribution, or selling or
licensing copies, or posting to personal, institutional or third party
websites are prohibited.
In most cases authors are permitted to post their version of the
article (e.g. in Word or Tex form) to their personal website or
institutional repository. Authors requiring further information
regarding Elsevier’s archiving and manuscript policies are
encouraged to visit:
http://www.elsevier.com/copyright
 Author's personal copy

Intelligence 38 (2010) 562–573

Contents lists available at ScienceDirect

Intelligence

Reading, listening, and viewing comprehension in English as a foreign

language: One or more constructs?
Ulrich Schroeders ⁎, Oliver Wilhelm, Nina Bucholtz
Institute for Educational Progress, Humboldt-Universität zu Berlin, Germany

a r t i c l e i n f o a b s t r a c t

Article history: Receptive foreign language proficiency is usually measured with reading and listening
Received 31 March 2010 comprehension tasks. A novel approach to assess such proficiencies – viewing comprehension –
Received in revised form 3 September 2010 is based on the presentation of short instructional videos followed by one or more comprehension
Accepted 6 September 2010
questions concerning the preceding video stimulus. In order to evaluate a newly developed
viewing comprehension test 485 German high school students completed reading, listening, and
Keywords: viewing comprehension tests, all measuring the receptive proficiency in English as a foreign
Reading language. Fluid and crystallized intelligence were measured as predictors of performance. Relative
Listening
to traditional comprehension tasks, the viewing comprehension task has similar psychometric
Comprehension
qualities. The three comprehension tests are very highly but not perfectly correlated with each
Language assessment
Intelligence other. Relations with fluid and crystallized intelligence show systematic differences between the
three comprehension tasks. The high overlap between foreign language comprehension measures
and between crystallized intelligence and language comprehension ability can be taken as support
for a uni-dimensional interpretation. Implications for the assessment of language proficiency are
discussed.
© 2010 Elsevier Inc. All rights reserved.

1. Introduction factor accounting for a broad variety of language ability facets

For decades the question whether language ability is
represented through a single factor or through a couple of
traits has been discussed intensively (cp. Bachman, 1990;
Carroll, 1993; Purpura, 1999). Lado (1961) and Carroll (1961,
1968) proposed models that discriminate between four basic
skills (reading, listening, writing, and speaking) and three
components of language knowledge (structure, vocabulary,
and phonology/graphology) to map native and secondary
language proficiency. In contrast to the multi-constructs
perspective, Oller (1976, 1979) proposed a model with only
one general factor. This unified assumption was based on the
concept of a pragmatic expectancy grammar that enables
guessing in situations with reduced context. The single factor
theory was rebutted in its extreme form of a single general
⁎ Corresponding author. Institute of Psychology, University of Duisburg-
Essen, Berliner Platz 6-8, 45127 Essen, Germany.
E-mail address: ulrich.schroeders@uni-due.de (U. Schroeders).
(Carroll, 1983), but survived in an alleviated form of the four
language skill factors which postulates four first order factors
(reading, listening, writing, and speaking) below an over-
arching second order factor.
In this study about the dimensionality of language skills we
focus on receptive skills because of their fundamental role in
second language acquisition. In order to be engaged in the –
other things being equal – more challenging productive
activities such as writing and speaking it is indispensable to
comprehend text and speech. Besides the distinction between
receptive (reading and listening) and productive (writing and
speaking) skills, the four basic language skills can also be
classified into oral (listening and speaking) vs. written-text
skills (reading and writing; see Table 1). It is important to note
that reading and listening abilities both can be assessed through
comprehension tasks, but they do rely on different input
modalities.
On a more fine grained level it can be asked, whether or
not the mode of comprehension tasks is essential in receptive

0160-2896/$ – see front matter © 2010 Elsevier Inc. All rights reserved.
doi:10.1016/j.intell.2010.09.003
 Author's personal copy
U. Schroeders et al. / Intelligence 38 (2010) 562–573
Table 1
Taxonomy of global foreign language skills.
Language Skill
Receptive
Input Oral Listening, (viewing)
Written Reading
foreign language ability testing. From a modality unspecific or
single skill perspective (Spolsky, 1973; Oller, 1979, 1983) it has
been proposed that a single comprehension factor is sufficient
in order to explain individual differences in comprehension
ability whereas from a modality specific or multiple skill
perspective (Lado, 1961; Carroll, 1961, 1968) it has also been
suggested that there are several distinguishable latent
comprehension abilities. The modality unspecific view is
mainly motivated by the concept of a unique mental
representation that underlies reading and listening. As a
result both reading and listening comprehension (henceforth,
abbreviated to RC and LC) tasks supposedly rely on the same
mental representation and imply the same cognitive pro-
cesses. The fact that normally reading and listening ability in
first-language acquisition develop simultaneously is often
cited as evidence for the common representation approach. In
order to support the modality specific view, many different
arguments on how reading and listening comprehension
differ have been proposed. First, speech is obviously more
comprehensive than text and listening comprehension
typically demands the integration of both linguistic and
non-linguistic knowledge (Buck, 2001). For example, it is
easier to understand a quarrel between a mother and her
child if the speech reflects their emotions or if accentuation
highlights critical information. Second, spoken text varies in
accordance to the characteristics of the speaker's voice and
speech. Prosodic features such as the speech rate (Carver,
1973), the timbre of voice, and dialects (Adank, Evand, Stuart-
Smith, & Scott, 2009) may significantly contribute to required
processing effort (for an overview see Samuels, 1987).
Therefore, the difficulty of a listening item depends on its
specific instantiation whereas the text of a reading item is
static and more or less fixed. Third, spoken text enforces
linear information encoding at an essentially compulsory
pace. On the contrary, reading allows for re-reading a text
passage several times looking for specific information. The
opportunity to flexibly allocate processing resources to
elements of information that are relevant for an individual
understanding allows the application of more challenging
items. Consistently, it has been shown that significantly more
information of a complex text can be recalled in a reading
than in a listening condition (Green, 1981; Lund, 1991).
Fourth, listening and reading may use different encoding
systems to store information in long-term memory. Through
this point the discussion about the influence of modality on
reading and listening comprehension ability becomes vividly
connected to research on multimedia learning. One of the
basic assumptions in cognitive theory of multimedia (CTML)
is that information is processed via two autonomous routes,
one for visual and one for auditory input (Paivio, 1986;
Mayer, 2005). Some researchers even assume two different
representations, a descriptive representation for written text
563
and mathematical equations and a depictive representation for
pictures or physical models (Schnotz, 2005; Schnotz &
Bannert, 2003).
However, research concerning multimedia learning is
Productive
often limited to the first moment of ability distributions.
Speaking Apparently, the difficulty of the task depends on the
Writing
complexity and type of information provided and the
modality that is involved. Whether or not the modality of a
task affects the bivariate relations between measures is a
different research question. For example LC stimuli might be
processed on a different processing route than RC stimuli—
nevertheless individual differences in a LC and a RC measure
might be perfectly correlated with each other. Such a result
would emerge if the different processing routes do not affect
the relative standing of any two individuals completing a LC
and a RC task.
We want to pick up the controversy about the dimen-
sionality of the comprehension abilities by focusing on
individual differences and by implementing a relatively new
type of comprehension task. Measuring comprehension
ability in a foreign language often requires the participant
to either read a text that is sometimes enriched with tables
and figures or to listen to a spoken audio sequence. We want
to introduce another type of comprehension task, labeled
viewing comprehension (VC).1 In a VC task the stimulus is a
short video, for example, about how the city of Boston
developed economically in the 20th century. Subsequent to
the video presentation, multiple-choice questions are
prompted asking for detailed information or conclusions
that can be drawn out of the previously presented video. In
order to respond correctly, first, auditory and visual informa-
tion has to be extracted from the video. After gathering pieces
of information the next process in the chain is relating the
pieces to each other and connecting them with already stored
knowledge in a mental model of the video sequence. In
principle, this sequence of intertwined processing steps is
similar to the processing steps in completing RC and LC tasks.
Besides such similarities in information processing, there are
obvious differences between VC and the traditional compre-
hension tasks. For instance, VC is not restricted to one
modality (seeing or listening) but rather combines both
modalities. The primary objective of this paper is the
structural relationship of the constructs RC, LC, and VC.
1.1. Research questions
The first research question is whether the new VC task is a
proper indicator for measuring proficiency in English as a
foreign language at all. Including videos might add interesting,
but unnecessary information to the test material. Even though
there is an increased amount of information this may lead to
poorer performance in a comprehension task (Mayer, Heiser, &
Lonn, 2001). This paradox is also known as the seductive-
details effect. Different explanations have been proposed to
1
Alternatively the terms “audio-visual comprehension”, “video compre-
hension”, “multimedia comprehension” or “watching comprehension” are
conceivable. We tried to coin a new succinct term in reference to the already
existing labels of reading and listening comprehension that both describe
activities rather than sensory input (visual and auditory) or stimulus
material (text and audio). It is our understanding that the term “viewing
comprehension” encompasses also the processing of auditory material.
 Author's personal copy

564 U. Schroeders et al. / Intelligence 38 (2010) 562–573

account for the phenomenon: a) distraction, b) cognitive
overload (Sanchez & Wiley, 2006), or c) activation of improper
schemata (Harp & Mayer, 1998). Therefore, a potential gain in
ecological validity – that is given if the material and setting of
the testing approximates real-life (Shadish, Cook, & Campbell,
2002) – might be accompanied by a loss of psychometric
quality. In order to evaluate the psychometric quality we
establish measurement models for all comprehension tasks –
reading, listening, and viewing – and compare them in terms of
reliability of the measures and model fit.
The second research question addresses the nature of the
supposedly new construct VC: is VC substantially different
from RC and LC or do all three comprehension measures
assess the same general comprehension ability? In a series of
confirmatory measurement models we want to test whether
the constructs are distinguishable or not. Therefore, we
translate the two competing theories mentioned above –
the modality unspecific and the modality specific theory –
into testable measurement models.
From the modality unspecific perspective a parsimonious
single-factor model can be derived (see Fig. 1, panel A). The
modality specific theory is represented through a model with
three factors, one factor for each comprehension domain (see
Fig. 1, panel B). If modality affects the relative position of an
individual in a comprehension tasks, then the correlation
between LC and VC should be more pronounced than the
correlations with RC because both rely (mainly) on auditory
input. Furthermore, the correlation between RC and LC should
be lowest because they rely on distinct modalities. This model
is a relaxed version of a two-factor model in which – according
to the modality involved – a RC factor and a combined LC–VC
factor account for the bivariate relations. This two-factor
model can be derived from the three factor model by
constraining the correlation between LC and VC to unity and
the remaining correlations (RC–LC and RC–VC) to equality.
However, it is essential to bear in mind that the discussion of
similarity between reading, listening, and viewing is not
limited to a controversy about sensory input. Sensory input is
one prominent source of potential differences between the
different types of comprehension tasks, but not the only one.
Differences may also lie in the form of representation (e.g.,
descriptive vs. depictive; Schnotz & Bannert, 2003). For this
reason, we decided to set up a model with three correlated
comprehension factors.

(A) Modality-unspecific
Compre-
single-factor model hension

R01 R02 … R20 R21 L01 L02 … L20 L21 V01 V02 … V30 V31

(B) Modality-specific model

with three correlated
comprehension factors
Listening Reading Viewing

R01 R02 … R20 R21 L01 L02 … L20 L21 V01 V02 … V30 V31

(C) Single-factor model with

Compre-
one nested method factor hension

R01 R02 … R20 R21 L01 L02 … L20 L21 V01 V02 … V30 V31

Method
factor

Fig. 1. Competing measurement models. Note. The last panel shows just an example of all possible nested factor models. Actually, seven different models belong to
this category: three with only one nested factor, three with two nested factors, and one with a nested factor for each comprehension domain.
 Author's personal copy
U. Schroeders et al. / Intelligence 38 (2010) 562–573
An alternative modeling approach treated the different
comprehension tasks as different methods, all assessing one
general comprehension ability factor. By specifying addition-
ally one or more nested factors that tap such method specific
variance, we want to test if the individual differences in
comprehension tasks can be explained better by taking into
account such task specific components or measurement
artifacts (see Fig. 1, panel C). Comparing model fit of both
models and scrutinizing the magnitude of the correlations
between latent variables will determine which theory is
supported by empirical evidence on the level of individual
differences.
The third research question is whether or not evidence
exists for the specificity of VC by investigating established
ability constructs as covariates. A convenient way to show
that a supposedly new construct actually differs from similar
constructs is to demonstrate differential relationships to
covariates or to establish incremental validity in the predic-
tion of relevant outcomes (Sechrest, 1963; Wilhelm, 2009). In
order to provide evidence for the similarity or dissimilarity of
the constructs we compare the correlations of the different
comprehension types with fluid and crystallized intelligence.
Differential correlations would provide valuable information
about the nature of the differences in comprehension tasks.
Using confirmatory factor analysis is essential for under-
standing individual differences in these constructs. Horn and
Noll (1997) stated that this approach provides structural
evidence. They also pointed out that other construct aspects –
for instance, how abilities develop or how abilities are tied to
the neural substrate – requires other forms of validity
evidence such as developmental or neurocognitive data.
2. Methods
2.1. Participants
This study was embedded in the process of establishing
national performance scales in English as a foreign language
in Germany. The RC and LC scales are based on the national
educational standards in English as the first foreign language
in the 5th or 6th year of foreign language education. The Data
Processing Center of the International Association for the
Evaluation of Educational Achievement selected the schools
participating in the present study and also proctored the tests
in schools according to guidelines provided by the authors. In
order to increase feasibility and to cut costs only schools from
one northern state of Germany were recruited. Participation
was mandatory for the students. N = 485 German high school
students of intermediate-track Realschule (n = 241) and
academic-track Gymnasium (n = 244) participated in this
study. Nearly two thirds of them attended 9th grade,
(n = 361) and the remaining third, 10th grade (n = 124).
Fifty-four percent of the participants were female (n = 262).
Mean age for the overall sample was about 16 years
(M = 15.9, SD = 0.72, range 13.8–18.4).
2.2. Measures
Participants completed measures of RC, LC, and VC as well
as tests for fluid and crystallized intelligence. After completing
all ability tests participants completed a socio-demographic
565
background questionnaire. Instructions for all comprehension
measures were give in English.
2.2.1. Reading and listening comprehension measures
Items for RC and LC were drawn from a larger database of
field-tested items assessing proficiency in English as a foreign
language (Rupp, Vock, Harsch, & Köller, 2008). In order to
compile multiple-choice items it was necessary to change the
response format for some of the items. The RC measure
consisted of 12 short texts covering various topics, for
example Amazon parrots or the Hurricane Katrina. The screen
was split in two halves: on the left side participants saw the
text, on the right side the questions. Test takers were given the
opportunity to go back and forth within the RC part to review
and change previous answers while the text was constantly
visible. Although there is good empirical evidence that the
majority of examinees will change only a few responses
(Revuelta, Ximénez, & Olea, 2003), we added this function-
ality to keep the differences in response format across the
three comprehension tasks as small as possible. The 7 audio
stimuli for the LC items were mostly short dialogs, for
instance, excerpts of a radio interview. Participants were
able to read all questions concerning a specific item for a short
period of time before presenting the audio stimulus and while
listening to the audio input. RC and LC both had a testlet
structure with 29 and 24 items, respectively.
2.2.2. Viewing comprehension measure
The newly developed VC task was based on short film
sequences of the television production “USA — The sound
of...” (information available at SWR broadcasting) lasting
between 1.5 and 7.5 min. The 5 video stimuli provided
historical, geographical and societal information about dif-
ferent regions of the USA, for example, about the conse-
quences of the discovery of oil in Louisiana. Similar to a
typical television report, a narrator or an interview partner
provided critical information on the topic and the visual
material only supported the understanding. For example, the
narrator's text “In 1901 oil was discovered in Louisiana. One
year later there were already 76 oil companies digging in the
swamp for the black gold” was accompanied by old photos
picturing a drilling rig framed with workers of an oil
company. The essential information has to be processed via
the auditory channel. Unlike the procedures in the LC task
participants did not see the questions in the VC task while
watching the videos once. VC had a testlet structure with 39
items. Items of all comprehension tasks had the same
response format, that is, multiple-choice format with four
response alternatives and only one correct solution. The items
included all information that was necessary to answer the
questions, thus, minimizing the effect of prior knowledge. All
three comprehension tests were designed to assess different
comprehension subskills such as finding detailed information
and deriving conclusions from different pieces of information.
2.2.3. Intelligence measures
In addition to the three comprehension tasks participants
worked on measures assessing fluid and crystallized intelli-
gence (Wilhelm, Schroeders, & Schipolowski, 2009). A
common way to capture fluid intelligence (gf) is to assess
 Author's personal copy
566 U. Schroeders et al. / Intelligence 38 (2010) 562–573
only its figural content that is considered prototypical for
reasoning (Wilhelm, 2005).
More specifically, participants had to detect regularities by
which geometric figures change their shape, position, shading
et cetera in a sequence of squares. The task of participants was
to complete two positions of the sequence by choosing
amongst three alternatives for. Fig. 2 shows a sample item of
the test. The actual content of the gf measure was non-verbal;
instructions were in German.
The measure for crystallized intelligence (gc) consisted of
64 multiple-choice declarative knowledge items covering
three broad domains—natural sciences, humanities, and civics
(see Fig. 2). The questions range from biology (What is the
function of red blood cells?) to politics (Who or what is
elected every 5 years at the “European Election”?) and do not
overlap with the comprehension measure regarding the
content. This approach of operationalizing crystallized intel-
ligence by means of declarative knowledge tests (Ackerman,
2000; Rolfhus & Ackerman, 1999) varies from the predom-
inantly used vocabulary measures (Carroll, 1993). Arguably,
declarative knowledge measures represent a more adequate
and prototypical form of assessment of crystallized intelli-
gence (Beauducel, 2003). In contrast to the comprehension
measures, the gc measure was administrated in German
language.
2.2.4. Other measures
Socio-economical, biographical and computer-related
background information were collected with questionnaires
at the end of the testing session. Participants also worked on a
completion test. Results for the C-Tests, the computer-related
background information, and socio-economic status will be
reported elsewhere.
Fig. 2. Sample items for fluid and crystallized intelligence.
2.3. Procedure
2.3.1. Data collection
Participants were randomly assigned to one of two groups in
a within-subject-design. For group 1 the sequence of tests was: a)
completion test (25 min), b) listening comprehension (25 min),
c) reading comprehension (25 min), d) viewing comprehension
(40 min), e) fluid intelligence (14 min), f) crystallized intelli-
gence (20 min), and g) questionnaires (20 min). For group 2 only
the order of the LC and the VC task was interchanged.
Specifications of time represent effective test time without
instructions. Total test time added up to 179 min. All testing was
computer-based except the completion tests.
2.3.2. Data analysis
Analyses were computed with Mplus 5.21 (Muthén &
Muthén, 2009). All measurement and structural models were
based on the weighted least squares mean and variance
adjusted estimator (WLSMV, B. O. Muthén, 1993) because
simulation studies showed that for dichotomous data the
WLSMV estimator is superior to the maximum likelihood
estimator (ML) both in terms of model rejection rates and
appropriate estimation of factor loadings (Beauducel &
Herzberg, 2006). WLSMV is a robust weighted least squares
estimator using a diagonal weight matrix (Muthén & Muthén,
2007) that is based on the asymptotic variances and
covariances of tetrachoric (or polychoric) correlations. In a
simulation study of Flora and Curran (2004) estimation with
robust WLS resulted in accurate test statistics, parameter
estimates and standard errors, even if the latent responses are
not normally distributed and sample size is low.
Data were complete for RC (N = 485). LC and VC data for
one case were missing (N = 484). N = 483 participants
 Author's personal copy

U. Schroeders et al. / Intelligence 38 (2010) 562–573 567

completed the crystallized intelligence task and N = 478 the
fluid intelligence task, respectively, resulting in a minimal
covariance coverage of 98.4%. Because missingness was not
an issue data were analyzed with pairwise present analysis,
that is, by exploiting the information of all individuals who
worked on a specific item pair (Muthén & Muthén, 2007).
3. Results
3.1. Item selection
Item selection was necessary for the comprehension tasks
because new or modified parts of the item database were not
sufficiently field-tested. Items with one of the following
characteristics were removed (numbers of excluded items are
denoted in parentheses for each task in the order RC/LC/VC):
a) extreme proportion of correct responses pi N0.95 or below
guessing probability (1/3/6), b) non-significant loadings in
one factorial measurement models computed separately for
each task (0/3/1), and c) considerable improvement in model
fit under item exclusion in the single-factor model (2/2/1).
This item selection procedure results in 21 items for both the
RC and LC task and 31 items for the VC task. The sample was
range-restricted in that only educational institutions at the
upper end of the ability spectrum were included. The range
restriction is partly responsible for the high rates of correct
responses. The RC and LC items were originally constructed
for paper-pencil-testing. Whether the change of test medium
affected the psychometric properties of items will be
addressed in another paper based on independent data
using multi-group confirmatory factor analysis (Schroeders
& Wilhelm, submitted for publication). For RC that was
assessed with the same items used in this study we showed
strict measurement invariance. The LC measure that was
comprised of different items from a common item database
was strongly invariant.
3.2. Measurement models
Descriptive statistics for the comprehension tasks and
indices of model fit are listed in Table 2. Because in WLSMV
estimation the degrees of freedom (df) are estimated rather
than computed (as in the case of ML estimation) neither χ2
nor df can be used as in models with ML estimation. However,
the comparative fit index (CFI), the root mean square error of
approximation (RMSEA), and the weighted root mean square
residual (WRMR) are suitable statistics to evaluate goodness
of fit. For categorical data and the present sample size, Yu
(2002) recommends the following cutoff values indicating
good model fit, CFI ≥0.96, RMSEA ≤0.05, and WRMR ≤0.95.
After item selection measurement models of all compre-
hension tasks yielded excellent fit with regard to all goodness
of fit statistics. The measurement models for both intelligence
scales show acceptable fit without exclusion of items.
Reliabilities by means of McDonald's omega (1999) – for
the VC factor ω = 0.87, the RC factor ω = 0.85, the LC factor
ω = 0.81, the gf factor ω = 0.82, and the gc factor ω = 0.79 –
can all be considered high or at least sufficient. The misgiving
that the VC task is more prone to testlet effects because in
comparison with RC and LC more questions are asked per
stimulus is unfounded: The model fit is good for all measures
even without explicitly modeling these interdependencies.
Different models that take the testlet structures into account
have been proposed (for a comprehensive overview see
Rijmen, 2009). The testlet model (Bradlow, Wainer, & Wang,
1999) is obtained by constraining the loadings on the testlet
factor to be proportional to the loadings on the comprehen-
sion factor within each testlet. We checked all analyses with
testlet models and compared them to the non-testlet models
in order to evaluate the divergence. The differences are small
and bare no effect on the interpretation of results. In answer
to the first research question, we therefore conclude that the
psychometric quality of the VC, the LC, and the RC is decent
and of comparable magnitude.

Table 2
Measurement models of all three comprehension tasks, fluid and crystallized intelligence.

na ∅ pb pmin c pmax d χ2e df e p CFI f RMSEA g WRMR h ωi

RC 21 0.74 0.45 0.93 120.5 104 0.13 0.974 0.013 0.884 0.85
LC 21 0.69 0.39 0.93 121.2 113 0.28 0.983 0.012 0.857 0.81
VC 31 0.55 0.27 0.91 227.6 204 0.12 0.978 0.015 0.898 0.87
Gf 16 0.56 0.24 0.95 107.5 73 0.01 0.933 0.031 0.952 0.82
Gc j 16 0.63 0.54 0.81 101.5 78 0.04 0.977 0.025 0.736 0.79

Note. RC, reading comprehension (N = 485). LC, listening comprehension (N = 484). VC, viewing comprehension (N = 484). Gf, fluid intelligence (N = 478). Gc,
crystallized intelligence (N = 483).
a
Number of items after item selection.
b
Mean item difficulty.
c
Lowest item difficulty.
d
Highest item difficulty.
e
Because all models are based on the weighted least squares mean and variance adjusted estimator (WLSMV) the degrees of freedom cannot be interpreted as
the number of unspecified model parameters.
f
Comparative fit index.
g
Root mean square error of approximation.
h
Weighted root mean square residual.
i
Reliability of the scale (McDonald, 1999).
j
For gc item parcels were considered containing four items of a content domain (e.g., chemistry). To ensure comparability of the characteristics, scores were
divided by four. Note, that test time for VC was nearly twice as much as for the other comprehension tasks (45 vs. 25 min).
 Author's personal copy

568 U. Schroeders et al. / Intelligence 38 (2010) 562–573

The second research question examines the relation of the the remaining correlations to equality) does not deteriorate
supposedly new construct VC in contrast to more traditional model fit significantly (Δχ2 (2, N = 485) = 5.6, p = 0.06).2
comprehension measures. Is it essential to differentiate these However, this result is to interpret with caution because
constructs or can one single comprehension factor account fixing a correlation to the upper boundary is problematic
for the variance of all indicators? To get a first impression of (Stoel, Garre, Dolan, & van den Wittenboer, 2006).
the relationship between the comprehension skills and the Additional models, for example, models that specify
intelligence scale, Table 3 provides the correlations between nested factors in addition to a single common factor – also
the means (below the diagonal) and factors (above the termed hierarchical models – have inconclusive loading
diagonal). patterns for one or more nested factors. Therefore, with
In order to answer this question we translated the respect to the second research question we conclude that a
competing perspectives into testable models and compared model with correlated comprehension factors describes the
them in terms of model fit. The single skill perspective that data best.
treats all three comprehension tasks as different instantia- In the third research question we evaluate the validity and
tions of a single construct is represented through a single- specificity of the different comprehension constructs by
factor model. The fit of this modality unspecific model is very introducing criteria into a structural model. The main interest
good (χ2 (326, N = 485) = 352.2, p = 0.15, CFI = 0.982, here is to investigate the relations of the correlated
RMSEA = 0.013, WRMR = 0.927). The modality specific per- comprehension factors with fluid and crystallized intelligence
spective treats the three comprehension tasks as separable as criteria. Any difference in correlation could be interpreted
but correlated constructs. Therefore, a model with three as differential validity between the factors and substantiate
correlated content factors (i.e., RC, LC, and VC) represents the differences in comprehension constructs.
modality specific perspective. This model is equivalent to a Fig. 3 depicts the measurement model for the compre-
higher-order model of comprehension ability in which the hension measures in combination with gf and gc. The two
first order comprehension factors are regressed on a second latent factors for intelligence share roughly 25% of their
order comprehension factor. The fit of this model is also variance (ρ(gf, gc) = 0.52). Overall the three comprehension
excellent (χ2 (326, N = 485) = 345.7, p = 0.22, CFI = 0.986, factors correlated moderately with fluid and highly with gc.
RMSEA = 0.011, WRMR = 0.918). If all factor correlations in For gf the comprehension tasks show a somewhat differential
model 2 are constrained to unity, model 1 results. Because correlative pattern. More precisely, gf is more highly
model 2 is nested in model 1, model fit of both models can be correlated with RC than with VC (ρ(RC, gf) = 0.55 vs. ρ(VC,
compared with a likelihood-ratio-test (Bollen, 1989). Such a gf) = 0.45; Δχ2 (1, N = 485) = 5.2, p = 0.02). VC in turn might
comparison shows, that model 2 offers a significantly better be correlated more highly with gf than LC (ρ(VC, gf) = 0.45 vs.
model fit (Δχ2 (3, N = 485) = 29.4, p b 0.01). Hence, the ρ(LC, gf) = 0.36; Δχ2 (1, N = 485) = 3.2, p = 0.08). The data
correlations between the first order factors are smaller than support the conclusion that reasoning is significantly more
unity. Moreover, constraining the three correlations to strongly related with RC than with LC and VC (Δχ2 (1,
equality also leads to a decline in model fit (Δχ2 (2, N = 485) = 14.7, p b 0.01). The correlations of gc with RC and
N = 485) = 7.5, p = 0.02). More specifically, the correlation VC, respectively, are equal (ρ(RC, gc) = 0.83 vs. ρ(VC, gc) =
between RC and VC is significantly higher (ρ(RC, VC) = 0.94) 0.85; Δχ2 (1, N = 485) = 0.5, p = 0.47). However, the corre-
than the correlations between RC and LC (ρ(RC, LC) = 0.86; lation with LC turned out to be significantly lower (ρ(LC, gc)=
Δχ2 (1, N = 485) = 5.5, p = 0.02) and LC and VC, respectively 0.72 vs. ρ(RC, gc)= 0.83; Δχ2 (1, N = 485)= 6.0, p = 0.01; ρ(LC,
(ρ(VC, LC) = 0.85; Δχ2 (1, N = 485) = 4.6, p = 0.03). Fixing gc) = 0.72 vs. ρ(VC, gc) = 0.85; Δχ2 (1, N = 485) = 9.5,
this correlation between RC and VC to unity (and constraining p b 0.01).
The model incorporating fluid and crystallized intelligence
can also be specified as a regression model, in which both fluid
Table 3 and crystallized intelligence are used as predictors for the
Descriptive statistics and correlation matrix of the means (below diagonal) three comprehension factors (Fig. 4). In this model correla-
and factors (above diagonal) for the comprehension tasks, fluid and
tions of the residuals of the comprehension factors are
crystallized intelligence.
estimated. Furthermore, the correlation between fluid and
RC LC VC Gf Gc crystallized intelligence is replaced by a regression path from
N 485 485 484 478 483 fluid to crystallized intelligence. This model is theoretically
Mean 0.74 0.69 0.55 0.55 2.54 a derived from Cattell's investment theory where gc is concep-
SD 0.16 0.16 0.16 0.18 0.46 tualized as “a product of environmentally varying, experien-
λ 0.81 0.69 0.85 0.42 0.75 tially determined investments of gf” (Cattell, 1963, p. 4).
SE(λ) 0.02 0.03 0.02 0.04 0.02
Statistically, both models are equivalent (MacCallum, Wege-
RC 1.00 0.86 0.94 0.55 0.83
LC 0.55 1.00 0.85 0.36 0.72 ner, Uchino, & Fabrigar, 1993). The conceptual difference of
VC 0.68 0.56 1.00 0.45 0.85 the regression model to the one with correlated group factors
Gf 0.38 0.22 0.32 1.00 0.52 is that the weights of the intelligence factors are computed
Gc 0.60 0.49 0.65 0.37 1.00
considering the collinearity of the predictors. Comparing the
Note. RC, reading comprehension. LC, listening comprehension. VC, viewing regression weights of the predictors shows that crystallized
comprehension. Gf, fluid intelligence. Gc, crystallized intelligence. λ, factor
loadings on the first unrotated factor in an explanatory factor analysis with
2
maximum likelihood estimator. SE(λ), standard error of factor loadings. Please note that for these comparisons of correlations it is necessary to
a
For gc item parcels were considered containing four items per content fix factor variances to one instead of scaling the factors by fixing a loading to
domain. one.
 Author's personal copy

U. Schroeders et al. / Intelligence 38 (2010) 562–573 569

Gf01 Gf02 … Gf15 Gf16 Gc01 Gc02 … Gc15 Gc16

Fluid Crystallized
intelligence .52 intelligence

.55 .36 c .45 c

.83 a .72 .85 a

.86 b .85 b
Reading Listening Viewing

.94

R01 R02 … R20 R21 L01 L02 … L20 L21 V01 V02 … V30 V31

N = 485, χ² = 411.8, df = 375, p = .09, CFI = .977, RMSEA = .014, WRMR = .942

Fig. 3. Three correlated content factors with fluid and crystallized intelligence as covariates. Note. Only the correlations marked with the same superscripts can be
set to equality without leading to deterioration in model fit. Indicators for gc were domain-specific item parcels; all other indicators were dichotomous variables.

Fluid Crystallized
intelligence .52 intelligence

a a
.17 -.02 .01

.74 .73 .85

Reading Listening Viewing

.86 .85

.94

.70 .65

.84

N = 485, χ² = 411.8, df = 375,p = .09, CFI = .977, RMSEA = .014, WRMR = .942

Fig. 4. Three correlated content factors with fluid and crystallized intelligence as predictors in a regression model. Note. Indicators are omitted. The correlations
between the residuals are symbolized with a dashed line. a Regression weight is not significantly different from zero.
 Author's personal copy
570 U. Schroeders et al. / Intelligence 38 (2010) 562–573
intelligence is the decisive predictor for all three compre-
hension tasks (βgc(RC) = 0.74; βgc(LC) = 0.73; βgc(VC) =
0.85). Obviously, this is also due to the fact that the influence
of gf is primarily mediated through gc. The indirect effect of
fluid intelligence mediated through crystallized intelligence
(βgf–gc(RC) = 0.39; βgf–gc(LC) = 0.38; βgf–gc(VC) = 0.44) is
much higher than the small to negligible direct effects of gf
on comprehension abilities (βgf(RC)= 0.17; βgf(LC) = −0.02;
βgf(VC)= 0.01). This finding is in line with Cattell's investment
theory that complex, acquired skills such as the measured
comprehension skills are indicators of crystallized intelligence
and that crystallized intelligence depends appreciably on the
level of fluid intelligence (Cattell, 1987).
4. Discussion
We begin this discussion by highlighting the research
questions and the results we found. The first research
question dealt with the psychometric quality of the VC
measure relative to established measures of LC and RC.
Compared to the traditional comprehension indicators, the
VC task yielded comparable psychometric features and was
well accepted by participants. Because it comes closer to
language comprehension in real-life where understanding is
enhanced through visual material, it might even offer a higher
authenticity. Authenticity might be considered one of the six
major components of test usefulness (Bachman & Palmer,
1996). Items usually used in language assessment to assess LC
map rare situations (e.g., listening to a podcast) whereas a
movie or a real-life conversation provide visual stimulus that
allows for compensating erroneous speech. Also VC relates to
the pupils' environment. The most likely way for German
high school students to get in contact with the English
language beyond lessons is to watch films and videos.
Although the VC measure is a psychometrically adequate,
well accepted, and supposedly more authentic instrument, it
has only been used sporadically in the context of measuring
individual differences in language proficiency. Several rea-
sons may account for this circumstance, for instance, the
anticipated higher costs for acquiring test devices or
constructing measurement instruments (for cost–benefit
considerations, see Farcot & Latour, 2009). However, with
the proceeding transition to computer-based assessment
(Scheuermann & Björnsson, 2009) it is more likely that
established constructs such as language comprehension will
be assessed in new ways.
With the second research question we investigated
aspects of validity of the construct VC. As reported, all
comprehension tasks correlated very highly with each other
depicting a very high overlap between the constructs. This
can also be seen from the factor loadings of the comprehen-
sion and intelligence skills on the first unrotated factor in an
explanatory factor analysis (see Table 3, R2 = 0.60). Although
the one-dimensional model had to be rejected, the magnitude
of the correlations comes close to unity. Taking into account
the fact that no substantial nested method factors could be
introduced to the single-factor model the specificity of the
different comprehension tasks is rather low. On the one hand,
it is possible that the language skills of the participants who
had 5 years of foreign language education at the time of
testing had not yet differentiated (Garrett, 1946). On the
other hand, the restriction of school type with the associated
range restriction in foreign language skills might cause an
underestimation of the relations between manifest variables.
It also seems likely that different learning experiences
affected the three comprehension skills differentially. For
example, a student who watches foreign language films may
have developed a more subtle ability to discriminate auditory
stimuli than a student who likes reading the original crime
stories of Agatha Christie. This raises the question about what
residuals of the first order comprehension factors express.
They may reflect individual differences in specific language
components such as syntactic proficiencies or receptive
vocabulary (Joshi, 2005) which are conceptualized as inde-
pendent from the actual skills (Carroll, 1961).
VC as a construct might reflect on the debate about the
cause of the differences between RC and LC. Comparing the
correlations between the different types of comprehension
we found the highest correlation between RC and VC.
Put differently, the correlations enclosing LC (i.e., LC–RC and
LC–VC) are lower than the relation between RC and VC.
Reviewing the demands of the specific comprehension tasks
from the perspective of CTML it would have been reasonable
to assume the highest correlation between LC and VC,
because in both tasks the relevant information was primarily
given in aural, not in visual form. Even though both LC and VC
have the same flow and processing of information according
to CTML the relation between these two comprehension tasks
was not higher than the correlation between LC and RC.
Therefore, neither modality per se nor information processing
seems to account for differences between RC and LC on the
level of covariances. It is reasonable to suggest that different
modalities affect the difficulty of a task. However, we cannot
identify this effect from the covariance structure.
The Partnership for Accessible Reading Assessment (PARA)
has stimulated considerable research to make large-scale
assessment of reading proficiency more accessible for
students with reading disabilities. In addition to the treat-
ment of the actual disorder, the use of assistive technology
has been suggested in order to tackle that issue. For example,
Thurlow, Moen, Lekwa, and Scullin (2010) studied the use of
a so-called reading pen that recognizes written text and
produces synthesized speech out of it in the assessment of RC.
However, they found no beneficial effects through the
implementation of the tool. In this context, the result that
RC and VC are essential perfectly correlated gains new
significance. Moreover, we replicated this finding in another
study using a similar VC measure in the field of sciences that
was administered in the first language (Schroeders, Bucholtz,
Formazin, & Wilhelm, submitted for publication). Taken
together, the findings of the two studies are encouraging
that VC can be used as a substitute for RC when comprehen-
sion ability of students with reading disabilities should be
measured. Moreover, it seems possible that a great gap
between the performances in a RC vs. VC task indicates a
specific learning disability.
Compared to the high correlation between RC and VC, the
significant lower correlations with LC may be caused by several
reasons. One explanation might be that the differences between
the comprehension skills could be attributed to differences in
test compilation rather than differences in construct. Or, the
more formal language used in documentaries and school books
 Author's personal copy
U. Schroeders et al. / Intelligence 38 (2010) 562–573
might hold different demands in comparison to the more
informal and casual language used in conversations. In order to
test these hypotheses it would be necessary to unconfound
items from test in a subsequent study. Controlling for these
differences is likely to result in equal and even higher
correlations between the comprehension factors.
In the third research question we questioned the specific-
ity of the different comprehension skills by investigating their
relationship to established measures of cognitive ability.
Seemingly, fluid and crystallized intelligence contributed
differently to the prediction of RC, LC, and VC as denoted by
the diverging amounts of explained variance: 52% of the LC
factor, 70% of RC factor, and 73% of the VC factor. Although
fluid intelligence is a major contributor of crystallized
intelligence and comprehension ability in foreign language,
it does not account for all the variance. Other important
determinants such as motivation, education by parents and
quality of school instructions, and socio-economic status are
not taken into account in the present study. Despite the
collinearity between fluid and crystallized intelligence, the
regression model helps in gauging the gf specific influence on
the different comprehension tasks. The significantly higher
regression weight of gf on RC – in comparison with LC and VC –
indicates that RC is more reliant on reasoning processes. This
could originate from different cognitive demands in compre-
hension skills (cp. Reves & Levine, 1988): Textual reasoning
(i.e., creating a cognitive model and drawing inferences) could
be more prevalent in RC whereas searching for details (i.e.,
detection and comparison of pieces of information, for a
discussion concerning the dimensionality of RC see Rost,
1985) may be more prominent in the LC and VC tasks.
However, the discussion whether reading and listening have
different subskills has a long tradition and is still unsettled
(Alderson, 2000; Song, 2008).
The high correlations between RC, LC, and VC indicate that
all three tasks represent different instantiations of a higher-
order ability to comprehend independent of the sensory input
while the tasks are unique in some respects (cp. Buck, 2001).
This comprehension ability in English as a foreign language is
also highly correlated with a measure of crystallized
intelligence. This finding is consistent with previous research
on the influence of gc on reading comprehension for the
corresponding age group (Benson, 2007) and underlines the
importance of gc in higher-level cognition (Hambrick, 2005).
The high correlations could also be interpreted as a cause of
beneficial interactions (or mutualism) between cognitive
processes according to the dynamic model of general
intelligence proposed by Maas et al. (2006). In the present
study gc was assessed with a declarative knowledge test (see
Ackerman, 2000; Rolfhus & Ackerman, 1999) rather than the
often used vocabulary measures and was administered in
German language. Regardless of the ostensible heterogeneity
of the demands of the gc task and the comprehension tasks
they correlated highly. This finding is in line with the
conceptualization that gc subsumes primarily verbal or
language-based knowledge that is acquired through cognitive
investment during education and general life experiences.
Therefore, gc should be more sensitive to effects of schooling
than other cognitive abilities. In Carroll's Three-Stratum
Theory of Intelligence (1993) foreign language proficiency
in general and comprehension skills in particular are listed in
571
the first stratum as narrow ability below gc (cp. Carroll,
1993). In the Cattell–Horn–Carroll theory of cognitive
abilities (Alfonso, Flanagan, & Radwan, 2005; McGrew,
2005, 2009) that integrates the three-level structure of
Carroll's theory with the ability factors of Cattell–Horn-
tradition (Cattell, 1987; Horn, 1968, 1991) the classification is
equivocal: Reading comprehension and verbal language
comprehension are sorted beneath a broad ability labeled
reading and writing whereas foreign language aptitude/
proficiency and listening ability are still part of gc. This
notion of a separate Stratum II or broad factor traces back to
conceptualizations of the Cattell–Horn model (McGrew,
2009). Such an allocation and distinction seems inconclusive
and is questioned through the findings of the present study.
Gc is actually as highly correlated with comprehension skills
as the comprehension skills are correlated with each other.
Therefore, these results advocate the perspective that within
the Carroll (1993) model comprehension ability should be
understood as a Stratum 1 factor below a more general gc
factor. This is not to say that comprehension measures per se
should be placed below a gc factor. Reducing the relevance of
prior knowledge and increasing the reasoning demand in
comprehension measures should shift such tasks towards gf.
The correlative results may also reflect on the recent
controversy about the role of g in large-scale student assess-
ment. In short, some researchers argued that the performance
measures included in large-scale-assessments and intelli-
gence tests essentially capture the same construct, that is,
intelligence (Rindermann, 2006, 2007a; Frey & Detterman,
2004; Gottfredson, 2003), whereas other researchers showed
their distinctiveness by means of confirmatory factor analysis
(Brunner, 2008; Baumert, Lüdtke, Trautwein, & Brunner,
2009). In our opinion, the dispute may profit from a broader
definition and assessment of intelligence. In educational
research the influence of such a broadly defined construct
of intelligence is often underestimated or neglected. In order
to overcome this shortcoming, we would like to recommend
two things for future research: First, intelligence is often
simply modeled as the common variance of heterogeneous
indicators of cognitive ability and the statistical abstraction is
then treated as synonymous with fluid or general intelli-
gence. However, this psychometric approach to g reveals little
about the psychological construct behind g (Thorndike,
1994). Therefore, it seems indicated to assess fluid intelli-
gence with an array of specific indicators across the three
content domains—verbal, numerical, and figural (Wilhelm,
2005). Second, it seems necessary to extend the assessment
of intelligence and include additional facets of intelligence
such as gc. In contemporary theories of the structure of
intelligence there is a broad consensus that both a decontex-
tualized reasoning part and a domain-specific knowledge
part are essential in order to capture the most general parts of
intelligence (Ackerman, 1996; Ackerman & Beier, 2005;
McGrew, 2005, 2009). This two-component approach should
also affect modern conceptions of student achievement more
profoundly. Moreover, the existence of these two compo-
nents is recognized by supporters of both perspectives (e.g.,
Baumert et al., 2009; Rindermann, 2007b). In the current
study we measured both components—decontextualized
fluid intelligence and crystallized intelligence. We conclude
that both are major contributors to comprehension ability
 Author's personal copy
572 U. Schroeders et al. / Intelligence 38 (2010) 562–573
and account roughly for half to three quarters of the factors'
variance. One reason why we might experience these
empirical results as an uncomfortable truth is that our
understanding of crystallized intelligence is much poorer
than our understanding of fluid abilities.
Acknowledgements
We thank Patrick C. Kyllonen and three anonymous
reviewers for their comments on an earlier draft of this article
and Frank Rijmen for detailed information about the specifica-
tion of the testlet model. Special thanks to the SWR for
providing video material of their broadcasting Schulfernsehen.
References
Ackerman, P. L. (1996). A theory of adult intellectual development: Process,
personality, interests, and knowledge. Intelligence, 22, 227−257.
Ackerman, P. L. (2000). Domain-specific knowledge as the “dark matter” of
adult intelligence: Gf/gc, personality and interest correlates. Journal of
Gerontology: Psychological Sciences, 55B, 69−84.
Ackerman, P. L., & Beier, M. E. (2005). Knowledge and intelligence. In O.
Wilhelm, & R. W. Engle (Eds.), Handbook of understanding and measuring
intelligence (pp. 125−139). London: Sage.
Adank, P., Evand, B. G., Stuart-Smith, J., & Scott, S. K. (2009). Comprehension of
familiar and unfamiliar native accents under adverse listening conditions.
Journal of Experimental Psychology: Human Perception and Performance, 35,
520−529.
Alderson, J. C. (2000). Assessing reading. Cambridge, UK: Cambridge University
Press.
Alfonso, V. C., Flanagan, D. P., & Radwan, S. (2005). The impact of the Cattell–
Horn–Carroll theory on test development and interpretation of cognitive
and academic abilities. In D. Flanagan, & P. L. Harrison (Eds.), Contemporary
intellectual assessment: Theories, tests, and issues (pp. 185−202). New York:
Guilford.
Bachman, L. F. (1990). Fundamental considerations in language testing. Oxford:
Oxford University Press.
Bachman, L. F., & Palmer, A. S. (1996). Language testing in practice. Oxford:
Oxford University Press.
Baumert, J., Lüdtke, O., Trautwein, U., & Brunner, M. (2009). Large-scale
student assessment studies measure the results of processes of
knowledge acquisition: Evidence in support of the distinction between
intelligence and student achievement. Educational Research Review, 4,
165−176.
Beauducel, A. (2003). Fluid and crystallized intelligence: Theory and measure-
ment. In R. Fernández-Ballesteros (Ed.), Encyclopedia of Psychological
Assessment. pp, 1. (pp. 416−419) London: Sage.
Beauducel, A., & Herzberg, P. Y. (2006). On the performance of maximum
likelihood versus means and variance adjusted weighted least square
estimation in confirmatory factor analysis. Structural Equation Modeling,
13, 186−203.
Benson, N. (2007). Cattell Horn Carroll cognitive abilities and reading
achievement. Journal of Psychoeducational Assessment, 26, 27−41.
Bollen, K. A. (1989). Structural equations with latent variables. Oxford, England:
John Wiley & Sons.
Bradlow, E. T., Wainer, H., & Wang, X. (1999). A Bayesian random effects
model for testlets. Psychometrika, 64, 153−168.
Brunner, M. (2008). No g in education? Learning and Individual Differences,
18, 152−165.
Buck, G. (2001). Assessing listening. Cambridge: Cambridge University Press.
Carroll, J. B. (1961). Fundamental considerations in testing for English
language proficiency of foreign students. In Testing Center for Applied
Linguistics, Washington, DC. Reprinted in H. B. Allen, & R. N. Campbell
(Eds.), Teaching English as a Second Language: A Book of Readings
(1972). McGraw Hill: New York.
Carroll, J. B. (1968). The psychology of language testing. In A. Davies (Ed.),
Language testing symposium: A psycholinguistic approach (pp. 46−69).
London: Oxford University Press.
Carroll, J. B. (1983). Psychometric theory and language testing. In J. W. Oller Jr.
(Ed.), Issues in language testing research (pp. 80−107). Rowley, MA:
Newbury House.
Carroll, J. B. (1993). Human cognitive abilities. A Survey of factor-analytic
studies. New York: Cambridge University Press.
Carver, R. P. (1973). Effects of increasing the rate of speech presentation
upon comprehension. Journal of Educational Psychology, 65, 118−126.
Cattell, R. B. (1963). Theory of fluid and crystallized intelligence: A critical
experiment. Journal of Educational Psychology, 54, 1−22.
Cattell, R. B. (1987). Intelligence: Its structure, growth, and action. Amsterdam:
Elsevier.
Farcot, M., & Latour, T. (2009). Transitioning to computer-based assess-
ments: A question of costs. In F. Scheuermann & J. Björnsson (Eds.), The
transition to computer-based assessment (pp. 108–116). JRC Scientific
and Technical Reports.
Flora, D. B., & Curran, P. J. (2004). An empirical evaluation of alternative
methods of estimation for confirmatory factor analysis with ordinal data.
Psychological Methods, 9, 466−491.
Frey, M. C., & Detterman, D. K. (2004). Scholastic assessment or g? The
relationship between the scholastic assessment test and general
cognitive ability. Psychological Science, 15, 373−378.
Gottfredson, L. S. (2003). G, jobs and life. In H. Nyborg (Ed.), The scientific
study of general intelligence: Tribute to Arthur R. Jensen (pp. 293−342).
Amsterdam: Pergamon Press, Netherlands.
Garrett, H. E. (1946). A developmental theory of intelligence. American
Psychologist, 1, 372−378.
Green, R. (1981). Remembering ideas form text: The effect of modality of
presentation. British Journal of Educational Psychology, 51, 83−89.
Hambrick, D. Z. (2005). The role of domain knowledge in higher-level
cognition. In O. Wilhelm, & R. W. Engle (Eds.), Handbook of understanding
and measuring intelligence (pp. 361−372). London: Sage.
Harp, S. F., & Mayer, R. E. (1998). How seductive details do their damage: A
theory of cognitive interest in science learning. Journal of Educational
Psychology, 90, 414−434.
Horn, J. L. (1968). Organization of abilities and the development of
intelligence. Psychological Review, 75, 242−259.
Horn, J. L. (1991). Measurement of intellectual capabilities: A review of
theory. In K. S. McGrew, J. K. Werder, & R. W. Woodcock (Eds.), WJ-R
technical manual (pp. 197−232). Chicago: Riverside.
Horn, J. L., & Noll, J. (1997). Human cognitive capabilities: Gf-Gc theory. In D. P.
Flanagan, J. L. Genshaft, & P. L. Harrison (Eds.), Contemporary intellectual
assessment. Theories, tests, and issues (pp. 53−91). New York: The
Guilford Press.
Joshi, R. M. (2005). Vocabulary: A critical component of comprehension.
Reading and Writing Quarterly: Overcoming Learning Difficulties, 21,
209−219.
Lado, R. (1961). Language testing: The construction and use of foreign language
tests. London: Longman.
Lund, R. J. (1991). A comparison of second language listening and reading
comprehension. Modern Language Journal, 75, 196−204.
Maas, H. L. J. V. D., Dolan, C. V., Grasman, R. P. P. P., Wicherts, J. M., Huizenga,
H. M., & Raijmakers, M. E. J. (2006). A dynamical model of general
intelligence: The positive manifold of intelligence by mutualism.
Psychological Review, 113, 842−861.
MacCallum, R. C., Wegener, D. T., Uchino, B. N., & Fabrigar, L. R. (1993). The
problem of equivalent models in applications of covariance structure
analysis. Psychological Bulletin, 114, 185−199.
Mayer, R. E. (2005). Cognitive theory of multimedia learning. In R. E. Mayer
(Ed.), The Cambridge handbook of multimedia learning (pp. 31−48). New
York: Cambridge University Press.
Mayer, R. E., Heiser, J., & Lonn, S. (2001). Cognitive constraints on multimedia
learning: When presenting more material results in less understanding.
Journal of Educational Psychology, 93, 187−198.
McDonald, R. P. (1999). Test theory: A unified treatment. Mahwah, NJ:
Erlbaum.
McGrew, K. S. (2005). The Cattell–Horn–Carroll theory of cognitive abilities:
Past, present, and future. In D. P. Flanagan, & P. L. Harrisson (Eds.),
Contemporary Intellectual Assessment: Theories, tests, and issues
(pp. 136−181). New York: The Guilford Press.
McGrew, K. S. (2009). CHC theory and the human cognitive abilities project:
Standing on the shoulders of the giants of psychometric intelligence
research. Intelligence, 37, 1−10.
Muthén, B. O. (1993). Goodness of fit with categorical and other non-normal
variables. In K. A. Bollen, & J. S. Long (Eds.), Testing structural equation
models (pp. 205−243). Newbury Park, CA: Sage.
Muthén, L. K., & Muthén, B. O. (2007). Mplus user's guide. Los Angeles:
Muthén & Muthén.
Muthén, L. K., & Muthén, B. O. (2009). Mplus. Los Angeles: Muthén & Muthén.
Oller, J. W. (1976). Evidence of a general language proficiency factor: An
expectancy grammar. Die Neueren Sprachen, 76, 165−174.
Oller, J. W. (1979). Language tests at school. London: Longman.
Oller, J. W. (1983). A consensus for the eighties? In J. W. Oller (Ed.), Issues in
language testing research (pp. 351−356). Rowley, MA: Newbury House.
Paivio, A. (1986). Mental representations: A dual coding approach. New York:
Oxford University Press.
Purpura, J. E. (1999). Learner strategy use and performance on language tests: A
structural equation modeling approach. Cambridge: Cambridge University.
 Author's personal copy
U. Schroeders et al. / Intelligence 38 (2010) 562–573
Reves, T., & Levine, A. (1988). The FL receptive skills: Same or different?
System, 16, 327−336.
Revuelta, J., Ximénez, M. C., & Olea, J. (2003). Psychometric and psychological
effects of item selection and review on computerized testing. Educational
and Psychological Measurement, 63, 791−808.
Rijmen, F. (2009). Three multidimensional models for testlet-based tests:
Formal relations and an empirical comparison. Educational Testing
Service Research Report No: RR-09-37. Princeton, NJ: ETS.
Rindermann, H. (2006). What do international student assessment studies
measure? School performance, student abilities, cognitive abilities,
knowledge or general intelligence? Psychologische Rundschau, 57, 69−86.
Rindermann, H. (2007a). Intelligenz, kognitive Fähigkeiten, Humankapital
und Rationalität auf verschiedene Ebenen. [Intelligence, cognitive
abilities, human resources, and rationality on different levels.]. Psycho-
logische Rundschau, 59, 137−145.
Rindermann, H. (2007b). The big G-factor of national cognitive ability
(author's response on open peer commentary). European Journal of
Personality, 21, 767−787.
Rolfhus, E., & Ackerman, P. (1999). Assessing individual differences in
knowledge: knowledge, intelligence and related traits. Journal of
Educational Psychology, 91, 511−526.
Rost, D. H. (1985). Dimensionen des Leseverständnisses [Dimensions of reading
comprehension]. Braunschweig: Pedersen.
Rupp, A. A., Vock, M., Harsch, C., & Köller, O. (2008). Developing standards-
based assessment tasks for English as a first foreign language. Context,
processes, and outcomes in Germany. Münster: Waxmann.
Samuels, S. J. (1987). Factors that influence listening and reading compre-
hension. In R. Horowitz, & S. J. Samuels (Eds.), Comprehending oral and
written language (pp. 295−325). San Diego: Academic Press.
Sanchez, C. A., & Wiley, J. (2006). An examination of the seductive details
effect in terms of working memory capacity. Memory & Cognition, 34,
344−355.
Scheuermann, F., & Björnsson, J. (Eds.) (2009). The transition to computer-
based assessment. JRC Scientific and Technical Reports. Retrieved March,
1, 2010, from http://crell.jrc.it/RP/reporttransition.pdf
Schnotz, W. (2005). An integrate model of text and picture comprehension.
In R. E. Mayer (Ed.), The Cambridge handbook of multimedia learning
(pp. 49−69). New York: Cambridge University Press.
Schnotz, W., & Bannert, M. (2003). Construction and interference in learning
from multiple representations. Learning and Instruction, 13, 141−156.
View publication stats
573
Schroeders, U., Bucholtz, N., Formazin, M., & Wilhelm, O. Modality specificity
of individual differences in comprehension measures in sciences.
Manuscript submitted for publication.
Schroeders, U., & Wilhelm, O. Equivalence of reading and listening
comprehension across test media. Manuscript submitted for publication.
Sechrest, L. (1963). Incremental validity: A recommendation. Educational and
Psychological Measurement, 23, 153−158.
Shadish, W., Cook, T., & Campbell, D. (2002). Experimental and quasi-experimental
designs for generalized causal inference. Boston: Houghton Mifflin.
Song, M. -Y. (2008). Do divisible subskills exist in second language (L2)
comprehension? A structural equation modeling approach. Language
Testing, 25, 435−464.
Spolsky, B. (1973). What does it mean to know a language: Or how do you
get someone to perform his competence? In J. W. Oller, & J. C. Richards
(Eds.), Focus on the learner: Pragmatic perspectives for the language
teacher (pp. 164−176). Rowley, MA: Newbury House.
Stoel, R. D., Garre, F. G., Dolan, C., & van den Wittenboer, G. (2006). On the
likelihood ratio test in structural equation modeling when parameters
are subject to boundary constraints. Psychological Methods, 11, 439−455.
SWR broadcasting. “USA — The Sound of...”. Retrieved March, 1, 2010, from
http://www.planet-schule.de/wissenspool/usa-the-sound-of/inhalt.
html
Thorndike, R. L. (1994). g. Intelligence, 19, 145−155.
Thurlow, M. L., Moen, R. E., Lekwa, A. J., & Scullin, S. B. (2010). Examination of
a reading pen as a partial auditory accommodation for reading assessment.
Minneapolis, MN: University of Minnesota, Partnership for Accessible
Reading Assessment.
Wilhelm, O. (2005). Measuring reasoning ability. In O. Wilhelm, & R. W.
Engle (Eds.), Handbook of understanding and measuring intelligence
(pp. 373−392). London: Sage.
Wilhelm, O. (2009). Issues in computerized ability measurement: Getting
out of the Jingle and Jangle Jungle. In F. Scheuermann, & J. Björnsson
(Eds.), The transition to computer-based assessment (pp. 145–150). JRC
Scientific and Technical Reports.
Wilhelm, O., Schroeders, U., & Schipolowski, S. (2009). BEFKI. Berliner Test
zur Erfassung fluider und kristalliner Intelligenz [Berlin test of fluid and
crystallized intelligence]. Unpublished manuscript.
Yu, C. Y. (2002). Evaluating cutoff criteria of model fit indices for latent
variable models with binary and continuous outcomes. Doctoral
dissertation, University of California, Los Angeles.