2009 - Vandewaetere - Measuring Students Attitude Towards CALL

This article was downloaded by: [National Cheng Kung University]
On: 23 April 2013, At: 19:45

Publisher: Routledge
Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered
office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK
Computer Assisted Language Learning

Publication details, including instructions for authors and
subscription information:
http://www.tandfonline.com/loi/ncal20
Introducing psychometrical validation

of questionnaires in CALL research:
the case of measuring attitude towards
CALL
a b b c
M. Vandewaetere & P. Desmet
a
Faculty of Psychology, University of Leuven – Campus Kortrijk,
Belgium
b
ITEC, Interdisciplinary Research on Technology, Education &
Communication, University of Leuven – Campus Kortrijk, Belgium
c
Faculty of Arts, University of Leuven – Campus Kortrijk, Belgium
Version of record first published: 04 Sep 2009.
To cite this article: M. Vandewaetere & P. Desmet (2009): Introducing psychometrical validation of
questionnaires in CALL research: the case of measuring attitude towards CALL, Computer Assisted
Language Learning, 22:4, 349-380
To link to this article: http://dx.doi.org/10.1080/09588220903186547
PLEASE SCROLL DOWN FOR ARTICLE
Full terms and conditions of use: http://www.tandfonline.com/page/terms-and-

conditions
This article may be used for research, teaching, and private study purposes. Any
substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing,
systematic supply, or distribution in any form to anyone is expressly forbidden.
The publisher does not give any warranty express or implied or make any representation
that the contents will be complete or accurate or up to date. The accuracy of any
instructions, formulae, and drug doses should be independently verified with primary
sources. The publisher shall not be liable for any loss, actions, claims, proceedings,
demand, or costs or damages whatsoever or howsoever caused arising directly or
indirectly in connection with or arising out of the use of this material.
Computer Assisted Language Learning
Vol. 22, No. 4, October 2009, 349–380
Introducing psychometrical validation of questionnaires in CALL

research: the case of measuring attitude towards CALL
M. Vandewaeterea,b* and P. Desmetb,c
a
Faculty of Psychology, University of Leuven – Campus Kortrijk, Belgium; bITEC,
Interdisciplinary Research on Technology, Education & Communication, University of Leuven –
Campus Kortrijk, Belgium; cFaculty of Arts, University of Leuven – Campus Kortrijk, Belgium
Downloaded by [National Cheng Kung University] at 19:45 23 April 2013
(Received 20 October 2008; final version received 20 May 2009)
The great majority of questionnaires measuring non-observable constructs such

as attitude towards CALL are often developed from a specific point of view and
are seldom followed by psychometrical validation. Psychometrical properties of
the questionnaire, such as construct validity and reliability, then remain
unanswered too often, laying a heavy burden upon the generalizability of the
obtained results. In light of recent calls for more theory construction in CALL it
would be very fruitful if researchers start to work with validated operationaliza-
tions of the variables they work with in developing CALL theories. The research
presented in this paper aims at introducing a methodological approach to develop
an empirically-based and psychometrically-sound instrument to measure the
attitude towards computer-assisted learning (A-CAL), attitude towards foreign
language learning (A-FLL) and, more specifically, attitude towards computer-
assisted language learning (A-CALL).
Keywords: survey research; questionnaire; attitude; CALL; confirmatory factor
analysis; structural equation modeling
Introduction
More and more CALL applications are already implemented in many classrooms or
language labs. This evolution increases the concern that the effectiveness of CALL is
studied and corroborated. Recently, some ‘state of the art’ studies were published
that give a critical overview of research and practice in CALL in recent years, such as
the work of Beatty (2003), Chambers and Davies (2001), Felix (2005) or Levy and
Stockwell (2006). In her paper ‘The unreasonable effectiveness of CALL: What have
we learned in two decades of research?’ Felix (2005) presents a thorough discussion
on strengths and weaknesses in CALL research. A strength is that learners’ overall
perceptions of CALL are positive if they are provided with stable technologies and
receive good support in order to deal with the technology (Felix, 2005). Moreover,
the use of technologies may also positively affect learners’ attitudes and
participation. Summarizing, we can state that there is growing evidence for the
effectiveness of CALL, and more specifically on the domains spelling, reading and
*Corresponding author. Email: Mieke.Vandewaetere@kuleuven-kortrijk.be
ISSN 0958-8221 print/ISSN 1744-3210 online

Ó 2009 Taylor & Francis
DOI: 10.1080/09588220903186547
http://www.informaworld.com
350 M. Vandewaetere and P. Desmet
writing (Felix, 2005). However, research on the effectiveness of CALL faces some
problems which have been summarized by Felix (2008): misleading titles; poor
description of the research design; failure to investigate previous research; poor
choice of variables to be investigated; and overambitious reporting of results. Besides
the research related problems there are also concerns about technical difficulties
interfering with the learning process: older students not feeling comfortable with
computers, younger students not possessing the necessary metaskills for coping with
the new learning environments, training needs in computer literacy for both students
and teachers (Felix, 2004). All these factors might heavily influence CALL
effectiveness, without being accounted for.
Notwithstanding the above-mentioned critical considerations, interest in
CALL research as such still increases and focuses on several fields. Chapelle
(2003) distinguishes three types of research in CALL, with respectively a focus on

software, on the learning task or task pedagogy and on the learners. Most of
research in CALL focuses on software, indicating the most successful strategies
and possibilities for software design. Other research comprises studies examining
the learning task and how to structure learning tasks in order to produce ideal
language learning conditions for learners. Only a few studies focus on the learners
and their interaction with the task and the presented software, and an even
smaller part of research takes into account the individual differences between
learners using an electronic learning environment. Individual differences such as
personal attitudes are a major factor to affect individual usage of information
technology, and the understanding of users’ (learners and/or teachers) attitudes
facilitates the creation of appropriate CALL-applications for teaching and learn-
ing (Desmet, 2007; Liaw, Huang, & Chen, 2007). A major strength of CALL
research putting the emphasis on the learners and their attitudes towards CALL
is that learners can be ensured against failure and a more adaptive way of
CALL becomes possible.
What is clear to all researchers is that positive attitudes towards language
learning can raise learners’ motivation and help language learning (Merisuo-Storm,
2007), although some studies have yielded mixed results (Sagarra & Zapata, 2008).
In line with this, it was found that, as individuals’ attitudes on e-learning and
computer-based learning become more positive, they will have greater behavioral
intention to use it (Liaw et al., 2007). In a study of Ayres (2002) it was concluded that
learners appreciate and value the learning that they do using computers, resulting in
high face validity for CALL. Students who see CALL as an important part of the
course also have high motivation and perceive CALL work as relevant to their
needs. However, Ayres (2002) also clearly states that while there is a definite
correlation between motivation and the perceived need of CALL, it is not possible to
say if the relationship is causal, nor in which direction. Also, a meta-analysis from
Masgoret and Gardner (2003) revealed a positive relationship between five classes of
variables such as attitudes towards the learning situation, ‘integrative-ness’ (i.e.,
openness to, identify, at least in part, with another language community), motivation
and instrumental orientation (i.e., practical reasons for learning a foreign language)
and the achievement in a second language, with motivation as most strongly related.
It is thus clear that constructs such as motivation and attitude are very related to the
degree in which CALL-applications are successful. On the other hand, the (causal)
direction of the relationships is not always clear, and often relationships have been
established in very specific CALL-situations, from a specific point of view. Most
Computer Assisted Language Learning 351
studies that have investigated learners’ attitudes and perceptions towards CALL
applications, have focused on the short-term use of synchronous and asynchronous
tools developed specifically for a particular study. This has resulted in different
methods that are handled in CALL research. For a comprehensive overview of these
studies, we refer the reader to the study of Felix (2004).
This approach lead Laurillard (2001) to expound that, more and more, we need
to drop the naı̈ve belief in the surplus value of new technologies in learning and the
naı̈ve conclusion that ICT has a positive impact on motivation, simply because there
is a large discrepancy between the specific questions asked in evaluation studies in
new technology, and the often predictable conclusions they come to. Stating that
positive perceptions of CALL correlate with higher learning outcomes in CALL is
nihil novo sub solem, especially since most research reporting such correlation has
made use of a very specific tool- or task-oriented procedure involving a

non-validated set of items to measure constructs such as attitude or perceptions
towards CALL. When it comes to relating several constructs such as attitude
towards technology, attitude towards foreign language learning and attitude towards
CALL in specific, we formulate two suggestions: the first is to use theories that
represent interrelations and effects between the constructs, from which testable
hypotheses should be derived. Therefore, a strong line of theory development in
CALL is needed. The second suggestion is that we need to incorporate clearly
formulated, generic, and validated operational definitions of the constructs that are
represented in the theories. In line with this, we need a framework to construct
operational definitions.
In his overview of CALL theories, Hubbard (2008) demonstrated an exceedingly
small amount of research on theory development in CALL research. Following this,
Hubbard makes the pertinent conclusion that a conceptual framework for CALL
may help promote theory development in CALL research. Next to that conceptual
framework, we state that there is also a strong need for a methodological framework
offering possibilities for theory confirmation or falsification. This paper presents a
methodological framework for the psychometrical validation of non-observable,
latent constructs in CALL research. As a consequence, using reliable and validated
constructs, may lead to more fine-grained CALL theories, from which testable
hypotheses can be derived.
Defining attitude
Although the construct of attitude towards computers has gained recognition as a
critical factor in the use and acceptance of information technologies, there is no
single clearly defined definition of computer-related attitude (Liaw, 2002). Several
researchers tried to define and validate the construct of attitude, and most of them
subscribe to the viewpoint of the tripartite model, suggesting that attitudes can be
decomposed into three major components: cognitive, affective and behavioral (Liaw,
2002; Smith, 1971; Wenden, 1991). The cognitive component involves beliefs or
perceptions about the objects or situations related to the attitude. The affective
component expresses the feelings that arise about the cognitive element and the
appraisal (good or bad) of these feelings. Finally, the evaluation of the affect is
translated into a behavioral component that gives utterance to the attitude and
certain attitudes tend to prompt learners to adopt particular learning beha-
viors. More general foreign language attitude questionnaires also adopt this
three-component structure, although this structure has never been validated or

confirmed by statistical analyses. The BALLI-questionnaire (Beliefs About
Language Learning Inventory; Horwitz, 1988) was based on the five (not-computer
related) major areas Horwitz (1987) identified: foreign language aptitude, the
difficulty of language learning, the nature of language learning, effective learning and
communication strategies and motivation. Gardner’s AMTB (Attitude/Motivation
Test Battery; Gardner, 1985) consists of 19 subtests, focusing on factors such as
anxiety, parental encouragement, desire to learn, motivation and teacher influence.
Cotterall (1995) used exploratory factor analysis to identify a six-dimensional
construct underlying learning readiness for autonomy:
(1) the role of the teacher;

(2) the role of feedback;

(3) learner independence;
(4) learner confidence in their ability to study;
(5) experience of language learning; and
(6) approach to studying.
This was however limited to the construct of self-directed language learning.

A study that not only examined attitude towards language learning in general but
was also related to attitude towards computer use in language learning was done by
Ayres (2002). Ayres (2002) demonstrated a clear correlation between considering
CALL as a substantial part of a language course on the one hand and having a high
motivation and the perception of CALL as relevant to students’ needs on the other
hand. But, as Ayres cautions, the direction and causality of the relationship is not
clear. Therefore, it remains unclear for instructional designers on which variables an
intervention should be created. Does the focus of instruction need to be placed on
motivation of learners and building a sound perception of CALL relevancy, or is the
first requirement to substantially embedding CALL in a language course? A
shortcoming of the Ayres’ study, however, is that attitude was only measured in
terms of perception towards a specific application of CALL. This again
demonstrates the difficulty to compare CALL research that includes constructs
such as attitude and motivation.
The study presented here attempts to give a first move towards an operational
and generic definition of attitude towards CALL, CAL and FLL (Foreign Language
Learning), by integrating the three-component theory of attitude.
Conceptual development
Attitude towards CALL (Computer-Assisted Language Learning)
Several researchers already have measured one or more of the three components of
attitude. Items that have been used in other research were extracted and adjusted for
CALL context. The cognitive component consists of items related to intelligence
(Lightbrown & Spada, 1999) and foreign language aptitude (Horwitz, 1988; Nikitina
& Furuoka, 2007) (for instance: ‘I think I would do well when learning a foreign
language with computer assistance’ beliefs or preconceptions about CALL compared
to traditional language learning as a component of metacognitive knowledge (Bernat
& Gvozdenko, 2005; Lightbrown & Spada, 1999), for instance: ‘Computer-based
tests can never be as good as paper-and-pencil tests’). Three subsets of items form the
affective/evaluative component: integrative/instrumental orientation and motivation
(Gardner, 2001; Horwitz, 1988), for instance: ‘I (would) like learning a new language
by computer’; teacher influence (Cotterall, 1995), for instance: ‘Teachers’ proficiency
of using computers in language learning largely defines my attitude towards
computer use in language learning’; specific beliefs about CALL (Bax, 2003) and
trust in CALL, for instance: ‘I have faith in computer-based language tests’. The
third component, behavior/personality, comprises the subset inhibition/exhibition
(Sil International, 1999), for instance: ‘In a face-to-face learning situation
(classroom), I often experience anxiety when speaking in the foreign language’.
Attitude towards CAL (computer-assisted learning)

The item set of the A-CAL questionnaire was again constructed following the
tripartite model of attitude, representing a cognitive component (e.g., ‘I have an
innate computer knack’), an affective/evaluative component (e.g., ‘Being able to
work with a computer enlarges my job opportunities’) and a behavioral/personality
component (e.g., ‘I love learning new things on a computer’). No further subsets
within the three components have been created since we wanted to avoid item
overlap between items belonging to the A-CALL questionnaire and items belonging
to the A-CAL questionnaire.
Attitude towards FLL (foreign language learning)

Again, attitude towards foreign language learning was broken down into three
components. The cognitive component comprises items related to intelligence and
foreign language aptitude (e.g., ‘I think I would do well when learning a foreign
language with computer assistance’) and beliefs (e.g., ‘The most important part in
learning a language is vocabulary’) or preconceptions about FLL (e.g., ‘Elderly
people have more difficulties in learning a language’). The second component, the
affective/evaluative part, comprises integrative/instrumental orientation and motiva-
tion (e.g., ‘The international status of a language is a strong motivator for learning a
language’); and teacher influence (e.g., ‘Language teachers’ proficiency influences my
attitude towards the language’). Finally, the behavioral/personality component
consists of items related to exhibition/inhibition (e.g., ‘I discover some degree of
initial resistance when having to start talking in a foreign language’); items related to
tolerance of ambiguity or the innate ability to be deal with ambiguity in an open way
(e.g., ‘I don’t mind switching to another language’); and items related to the
construct of learning effort (e.g., ‘I’m losing the pleasure of language learning when
the learning effort becomes too big’).
Aim of this study

This research aims at introducing a general framework to validate questionnaires
measuring latent or unobservable constructs in research on CALL. Whereas most
research on the construction of questionnaires leaves off with the step of exploratory
factor analysis, this research takes an additional step into account: confirmatory
factor analysis, used in order to validate both the developed questionnaires and the
associated subscales. Although CALL is used in different learning contexts (e.g.,
remedial learning, distance learning, blended learning, etc.) and comes in many
applications (e.g., learning platform, computer-mediated communication, course-
ware, etc.), we want to present a preliminary attempt to an operational definition of
attitude towards CALL, CAL and FLL, making use by the presented framework.
The operational definition of attitude towards CALL, CAL and FLL is disconnected
from specific learning contexts or experiments and is instead more focusing on the
learners’ attitudes towards several faces of CALL (Gan, 2004; Sengupta, 2001).
Having defined the construct of attitude in the introduction, in a next part of this
research we focus on the measurement of attitude towards CALL, CAL and FLL. In
order to let the reader interpret the results of the analysis we present a short
introduction to some frequently used statistics in research on questionnaire
validation. For each construct a questionnaire was developed and the results, as
obtained by a four-step validation procedure, are described separately.
Method
Participants
Participants were recruited via e-mail. Four populations were contacted:
(1) bachelors students

(2) university employees
(3) users of the Franel electronic language learning environment (Desmet &
Héroguel, 2005) and
(4) people with other employment or study backgrounds.
There were no predetermined exclusion criteria, so all 240 participants that

completed the questionnaire were eligible for inclusion, resulting in a heterogeneous
sample. The mean age of all participants is 37.00 years (sd ¼ 16.25), with range from
18 to 74 years old. The proportion of females in the sample is .60 (n ¼ 144). About
47% are lower-educated (no education after the age of 18 years) and most of the
participants (77%) followed foreign language education for more than five hours a
week (below the age of 18 years). The majority of participants (71%) use a computer
mainly for their work or at least equally between work and private use.
Of all participants, 35% of them are students and 50% are employed half- or
full-time. Most of the working participants (half- or full-time) are employed in
education and research (53%).
Besides their mother language (Dutch for 96% of all participants), 99% have at
least a basic notion of a second language, mainly French (79%). For a third language
(mainly English, 68%), 92% have minimal knowledge. A majority of participants, 76%,
also have notions of a fourth language (this is German for 52% of all participants).
More detailed descriptive statistics are presented in Table 1.
Regarding the experience with CALL, about 24% of the participants had no
experience with any method of computer-assisted language learning. Table 2 gives an
overview of several CALL-methods and the experience participants have with them.
Measures
All participants completed a set of items measuring attitudes in the context of
computer-assisted (language) learning and general language learning. In the final
Table 1. Frequencies (N), percentages (%) of foreign language knowledge (at least notions).
Mother language Second language Third language

language N % language N % language N %
Dutch 227 95.8 none 3 1.3 none 18 7.6
French 8 3.4 French 188 79.3 French 31 13.0
other 2 0.8 English 33 14.0 English 162 68.1
German 3 1.3 German 7 2.9
Spanish 2 0.8 Spanish 14 5.9
Dutch 9 3.8 Dutch 1 0.4
Italian 5 2.1
Table 2. Frequencies (N), percentages (%) of usage of CALL-applications.
N (%)
1. Assignment from teacher to student (via electronic learning 80 (33.5)
environment, weblog, . . .) when learning a foreign language
2. Interaction between teacher and student (via forum, chat, . . .) 44 (18.4)
when learning a foreign language
3. Mutual interaction between students (via forum, chat, . . .) when 38 (15.9)
learning a foreign language
4. Interaction between students and native speakers (via chat, 33 (13.8)
weblog, forum, . . .) when learning a foreign language
5. Doing language exercises via computer 159 (66.5)
6. Doing language tests via computer 86 (36.0)
7. None of the above 57 (23.9)
questionnaire, a set of 24 items was offered to measure attitude towards computer-

assisted language learning (A-CALL); 64 items were presented to measure attitude
towards foreign language learning (A-FLL); and 15 items were used for measuring
attitude towards general computer-assisted learning (A-CAL). The questionnaire
was completed with general questions related to job or study level and further
demographic variables.
Participants were instructed to describe the degree to which they agree with each
statement. Each item was answered on a seven-point Likert scale, ranging from 1
(‘totally disagree’), over 4 (‘neutral’) to 7 (‘totally agree’).
Items were developed in line with the three-component structure of attitude that
was previously proposed in studies from Liaw (2002) and Smith (1971). This
tripartite model of attitude was also found in more general foreign language attitude
questionnaires such as Horwitz’s (1988) Beliefs About Language Learning Inventory
(BALLI) and the Attitude/Motivation Test Battery (AMTB) by Gardner (1985). It
was taken into account that all items were related to the specific attitude towards
CALL, CAL and FLL and that they were applicable for subjects with and without
previous CALL-experience. To select items for the final test version, we used the
following criteria: items must be uni-directionally formulated in a simple and clear
language and must be considered unambiguous and relevant to the constructs at
stake. A pilot version of the first draft was offered to 10 subjects that rated the items
on their clarity and simplicity. The results of this procedure have led to the removal
of eight items that were considered as not sufficiently clear.
Validation procedure
For the validation of each questionnaire, a four-step strategy was used to investigate
the construct validity of the questionnaire. Confirmatory factor analyses were
performed by using AMOS 16.0 (Arbuckle, 1995–2007; Byrne, 2001). This software
program implements the structural equation modeling (SEM) approach or analysis
of covariance structures, and, as a specific case of SEM, confirmatory factor analysis.
For the other analyses, we used SPSS 16.0.
(1) First, the descriptive statistics, the internal consistency (Pearson intercorrela-
tions) and reliability (Cronbach’s a) were computed per subscale on the entire
sample.
(2) Second, exploratory factor analysis (EFA) was conducted on the entire
sample. Using principal axis factoring with orthogonal varimax rotation we

tried to identify and corroborate the three underlying components of
attitude. Following Cattell (1978), the scree test with the criterion of an
eigenvalue 4 1 was used to determine the number of factors. The mini-
mum consideration level was set to j.30j, so items with lower factor loadings
did not contribute significantly to the factor solution and hence were
removed.
(3) In a third step, the sample was randomly split into a calibration sample
(n1 ¼ 120) and a validation sample (n2 ¼ 120). The calibration sample data
were used to validate and confirm the factor structure by means of
confirmatory factor analysis (CFA). The model as obtained by the EFA
was tested in order to preserve a final model that met the earlier posited
goodness-of-fit criteria. The second subsample, the validation sample, was
used to cross-validate the solution obtained with the calibration sample. The
estimation method used is maximum likelihood, which yields efficient,
sufficient and consistent parameter estimates (Myung, 2003). Following the
recommendations of Bollen and Long (1993) and Byrne (2001), several fit
indices were used to assess the model of fit. These indices were w2, w2 divided
by the degrees of freedom (CMIN/DF), the root mean square error of
approximation (RMSEA), goodness-of-fit index (GFI), adjusted goodness-
of-fit index (AGFI), comparative fit index (CFI) and non-normed fit index
(NNFI) or Tucker-Lewis coefficient (TLI). The fit indices are discussed
shortly in the subsequent paragraph.
Again, we removed (non significantly contributing) items from the final
solution in order to attain a good model fit.
(4) In a fourth and last step, we again investigated the reliability and validity of
the remaining items in the three questionnaires. This was done by again
computing the descriptive statistics, internal consistency and reliability per
subscale on the entire sample. Also correlations between the subscales and
correlations between subscale and total scale have been measured to indicate
logical consistency.
Fit indices
In step three of the validation procedure, we test how well the constructed item set
represents the underlying construct of attitude. To indicate the model fit, several fit
indices were used. We briefly discuss these indices.
(1) w2 is the most frequently used goodness-of-fit index. A statistically significant

w2 indicates that a significant amount of observed covariance between items
remains unexplained by the model, while a non-significant w2 implies a good
fit to the data. One of the shortcomings of this index is that the value for w2
decreases when parameters are added to the model and that a good model fit
may result either from a correctly specified model or from a highly over
parameterized model (Shermelleh-Engel, Moosbrugger, & Müller, 2003).
(2) CMIN/DF or the relative chi-square is the ratio of the minimum discrepancy
to degrees of freedom. Values should be close to 1 for correct models.
Byrne (2001) states that a CMIN/DF ratio of more than 2 represents an
inadequate fit.
(3) The RMSEA (root mean square error approximation) (Browne and Cuddeck,
1993) is a fit measure based on population error of approximation. It is

unreasonable to assume that the model will hold exactly in the population.
Therefore, the RMSEA takes into account the error of approximation in the
population. This index is sensitive to the number of parameters in the model.
According to Browne and Cuddeck (1993), RMSEA values 0.05 can be
considered as a good fit, values between 0.05 and 0.08 as an adequate fit and
values between 0.08 and 0.10 as a mediocre fit. When sample size is small, the
RMSEA tends to over reject true population models.
(4) The GFI (goodness of fit index) and AGFI (adjusted goodness of fit index)
(Jöreskog & Sörbom, 1984) test how much better the model fits as compared
to no model at all (null model), i.e., when all parameters are fixed to zero. The
AGFI adjusts for a bias resulting from model complexity and rewards less
complex models with fewer parameters. The indices have a range between 0
and 1, with values closer to 1 indicating a better fit. A GFI larger than 0.90
and an AGFI larger than 0.85 indicate a good fit of the model. Both indices
can be influenced by sample size (Hu & Bentler, 1999).
(5) The CFI (comparative fit index) is an incremental fit index (Bentler, 1990). It
represents the proportionate improvement in model fit by comparing the
target model with a null model. This index avoids the underestimation of fit
often noted in small samples (Shermelleh-Engel et al., 2003). The CFI ranges
between 0 and 1, with values larger than 0.90 indicating an adequate fit.
(6) The NNFI (non normed fit index) or TLI (Tucker-Lewis index) takes the
degrees of freedom of the specified model as well as the degrees of freedom of
the independence model into consideration. The value for this index ranges in
general from 0 to 1, but as this index is not normed, values can sometimes
leave this range, with higher NNFI values indicating better fit (Tucker and
Lewis, 1973). An advantage of the NNFI is that it is one of the fit indices less
affected by sample size (Bentler, 1990).
Results
Data preparation
Since maximum likelihood estimation assumes multivariate normality of the data,
this was tested (DeCarlo, 1997) for all items. In the item set that represents the
A-CALL,five observations with a high Mahalanobis D2-value were examined. Only
one observation was considered as outlier and was removed from further analyses
related to the A-CALL. The D’Agostino-Pearson K2 tests and Jarque-Bera LM tests
showed multivariate normality for all variables in the analyses, except for the items
‘CALL is a valuable extension of the classical learning methods’, ‘I have faith in
computer-based language tests’ and ‘I have faith in computer-based language
exercises’, having a rather positive skewed distribution.
For the item sets that represent the A-CAL and the A-FLL, one observation
could be labeled as outlier, having an extremely high Mahalanobis D2-value. This
observation was removed from subsequent analysis related to A-CAL and A-FLL.
The D’Agostino-Pearson K2 tests and Jarque-Bera LM tests demonstrated
multivariate normality for only a minor part of the selected variables. Variables in
this item set showed a high degree of kurtosis (degree to which the data are peaked or
flat relative to a normal distribution) and skewness (degree to which the data are
asymmetric or skewed to the left or right, relative to a normal distribution). Most
items have been answered affirmatively and mean scores of the items have a tendency
towards right (i.e., totally agree with statement). In view of the Likert-scale property
of the questionnaires presented here and because of the fact that the assumption of
multivariate normality is violated in our data, an alternative estimation method is
often recommended. Instead of using the maximum likelihood (ML) estimation
method that is highly influenced by non-normal data, it is suggested that the
asymptotically distribution free (ADF) method or weighted least squares (WLS)
method is used to estimate model parameters (Browne, 1984). However, simulation
studies suggest that ML estimation in general tends to be more stable, but also shows
higher accuracy in terms of model fit compared to the other estimators, such as GLS
or WLS (Olsson, Foss, Troye, & Howel, 2000). Therefore, ML estimation was used
to analyze the data.
A-CALL questionnaire
STEP 1: internal consistency and reliability of draft questionnaire
The descriptive statistics, the internal consistency and reliability (Cronbach’s a)
were computed for each subscale (the three components of attitude). Internal
consistency values for the subscales are .659 (cognition), .840 (evaluative-affective)
and .536 (behavioral), indicating that only the subscale ‘evaluative-affective’ is
internally consistent. A closer look to the item structure and interrelations
between the items is needed in order to explore fully the factor structure of the
questionnaire.
STEP 2: exploratory factor analysis

The results of the EFA revealed a five-factor solution. However, all of the items but
one that loaded on the third factor also had a loading on the first factor. Therefore,
this third factor was put together with factor one. This solution accounts for 54.25%
of the variance. Inspection of the factor loadings showed that all items load
substantially on at least one of the four factors.
Factor I (put together with factor III) was labeled ‘surplus value of CALL’. Ten
items capture the beliefs of subjects in the CALL approach (e.g., ‘Computer-assisted
language learning gives more flexibility to language learning’). Factor II was labeled
‘effectiveness of CALL vs. non-CALL’. Four items define the comparison in
effectiveness between CALL and traditional language learning (e.g., ‘Computer-
assisted language learning is less adequate as the traditional language learning’).
Factor IV was labeled ‘teacher influence’ and three items summarize the influence of
the teacher on the attitude and motivation towards CALL (e.g., ‘Teacher’s
proficiency of using computers in language learning largely defines my attitude
towards computer use in language learning’). Factor V consists of three items and
was labeled ‘degree of exhibition to CALL’ and captures the extent of inhibition/
exhibition that subjects experience with CALL (e.g., ‘I feel less inhibited when
communicating in the foreign language via computer (chat) than in a face-to-face
situation’).
STEP 3: confirmatory factor analysis

The four-factor structure was tested with CFA using the calibration sample
(n1 ¼ 120) resulting in an RMSEA ¼ 0.106, indicating a rather bad fit. Four items
(‘Computer-based language tests are easier than paper-and-pencil tests’, ‘CALL
requires a certain degree of computer proficiency’, ‘CALL leads to a heavier
workload’ and ‘I prefer the classical way of learning languages’) had low factor
loadings or loadings on more than one factor on the factor structure as revealed by
EFA. These items were removed from further analyses, leaving 20 items in the item
set. Appendix 1 contains the English translation of the final version of the
questionnaire. Factor I (effectiveness of CALL vs. non CALL) has four items (2, 3, 4
and 5, all reversed), factor II (surplus value of CALL) has 10 items (1, 6, 7, 8, 9, 10,
11, 12, 16 and 17), factor III (teacher influence) has three items (13, 14 and 15) and
factor IV (degree of exhibition to CALL) consists of four items (18, 19 and 20, with
items 19 and 20 reversed).
The modification indices of the tested model suggested correlated residuals
between items 6 (‘Computer-assisted language learning is a valuable extension of the
classical learning methods’) and 9 (‘Computer-assisted language learning can stand
alone’), items 6 and 7 (‘Computer-assisted language learning gives more flexibility to
language learning’) and items 16 (‘I have faith in computer-based language tests’)
and 17 (‘I have faith in computer-based language exercises’). Probably the correlated
residuals are caused by a content overlap between the items. On the other hand,
items 6, 16 and 17 did not meet the assumptions of normality and followed a
positively skewed distribution. The model was refitted, with free estimation of the
error covariance for the three pairs of items with correlated residuals. This resulted
in a significant improvement of the model fit. Again, modification indices suggested
slight changes in the model construction, but because of model parsimony it was
decided not to include these changes.
Using the validation sample (n2 ¼ 120) the construct structure of the final model
was confirmed. Table 3 presents the major goodness-of-fit indices for the final four-
factor solution for the calibration and validation sample. The values of the indices do
Table 3. A-CALL: Goodness-of-fit indices for the four-factor structure tested with CFA.
n w2 (df) CMIN/DF RMSEA GFI AGFI CFI NNFI

Calibration sample 119 296.85 (161)* 1.84 .084 .81 .75 .89 .88
Validation sample 117 349.46 (161)* 2.17 .100 .78 .71 .84 .81
Full sample 236 424.34 (161)* 2.64 .083 .84 .79 .89 .87
*significant, a 5 .01.
not allow to conclude for a reasonable or good fit. Values for w2 indicate a significant
deviation of the postulated structure and the obtained dataset. Also, the values for
CMIN/DF, ideally close to 1, indicate a discrepancy between our theoretical model
and the data. Values for RMSEA, GFI, AGFI, CFI and NNFI indicate a mediocre
model fit.
Figure 1 displays the standardized factor loadings of the four-factor model for
both samples.
Figure 1. A-CALL: Standardized factor loadings of the four-factor model as obtained with
confirmatory factor analysis for the calibration sample and the validation sample (between
parentheses). Numbering of items is in accordance with the numbering in Appendix 1.
STEP 4: internal consistency and reliability of final A-CALL questionnaire

Table 4 summarizes the descriptive statistics, the reliability (Cronbach’s a) and
Pearson intercorrelations for each A-CALL subscale for the entire sample.
Cronbach’s a coefficients range from .74 to .91, indicating that the final subscales
are internally consistent.
The intercorrelations are moderate to good. The ‘effectiveness of CALL vs. non
CALL’ subscale is positively correlated to the ‘surplus value of CALL’ subscale
(r ¼ .42) and is positively correlated to the ‘teacher influence’ subscale (r ¼ .18).
‘Effectiveness of CALL vs. non CALL’ does not correlate with ‘degree of exhibition
to CALL’ (r ¼ 70.8). The subscale ‘surplus value of CALL’ is positively correlated
to the ‘teacher influence’ (r ¼ .38) and to ‘degree of exhibition to CALL’ (r ¼ .30).
‘Teacher influence’ does not correlate with ‘degree of exhibition to CALL’ (r ¼ .03).
The positive correlations between the subscales and between the subscales and the
total scale score point to the logical consistency of the questionnaire and indicate
that the A-CALL questionnaire is a feasible working instrument but should be the
subject of more confirmatory research.
A-CAL questionnaire
STEP 1: internal consistency and reliability of draft questionnaire
The value for Cronbach’s a is .818, indicating that the internal consistency and
reliability of the entire scale (15 items) is good. Exploratory factor analysis will give
more details on the underlying factor structure of the questionnaire.
STEP 2: exploratory factor analysis

The results of the EFA revealed a three-factor solution, accounting for 49.3% of the
variance. Factor I was labeled ‘computer proficiency’. Five items capture the self-
evaluated proficiency of subjects with CAL (e.g., ‘Compared to others, my speed of
learning to work with a computer is higher’). Factor II was labeled ‘computer
integration’. Four items define the degree of attitude towards integration of
computer use (e.g., ‘I love learning new things when working with computers’).
Factor III was labeled ‘need for computer use’; three items summarize the relative
importance subjects attach to computer use in society (e.g., ‘Given the importance of
computers, you must be able work with computers’).
Table 4. A-CALL: Means (M), standard deviations (SD), reliability (a) and Pearson
correlations between the A-CALL subscales (entire sample).
M (SD) a 2 3 4 5
1. Effectiveness of CALL vs. 17.15 (4.13) .80 .42* .18* 7.08 .59**
non CALL (4 items)
2. Surplus value of CALL (10 items) 45.51 (8.36) .86 – .38* .30** .92**
3. Teacher influence (3 items) 13.97 (2.92) .91 – – .03 .53**
4. Degree of exhibition to CALL 12.54 (3.51) .74 – – – .43**
(3 items)
*p 5 .01 (two-tailed).
STEP 3: Confirmatory factor analysis

Subsequently, confirmatory factor analyses did not confirm the presupposed three-
factor structure. The construct labeled ‘need for computer use’ had very low and
negative covariances with the other two factors and also the items associated with
the factor had low loadings on the construct ‘need for computer use’. Removing this
factor and its associated negatively correlating items led to a model that fitted the
data in a better. The model consists of 9 items, representing two factors. Items 1 (‘I
can work well with computers’) and 2 (‘I have an advanced knowledge of
computers’) have correlated residuals because of very similar wording of the items.
This is also the case for items 4 (‘I have an innate computer knack’) and 5
(‘Compared to others, my speed of learning to work with computers is higher’).
Using the validation sample (n2 ¼ 120), the structure of the final model was
confirmed, but not as strong as was expected. Table 5 presents the major goodness-
of-fit indices for the final two-factor solution for the calibration and validation
sample. The values of the indices do not allow to conclude for adequate or good fit.
Values for w2 indicate a significant deviation between the postulated structure and
the obtained dataset. Also, the values for CMIN/DF, ideally close to 1, indicate a
discrepancy between our theoretical model and the data. Values for GFI, AGFI,
CFI and NNFI indicate a mediocre to good model fit.
Figure 2 displays the standardized factor loadings of the two-factor model for
both samples. As can be seen in this figure, standardized regression coefficients are
high, pointing to high internal consistency.
STEP 4: internal consistency and reliability of final questionnaire

Table 6 summarizes the descriptive statistics, the reliability (Cronbach’s a) and
Pearson intercorrelations for the two A-CAL subscales for the entire sample.
Cronbach’s a coefficients for the subscales are .92 (computer proficiency) and .80
(computer integration), indicating that the subscales are internally consistent. The
intercorrelation between the subscales is moderate and significant (r ¼ .59),
indicating that the two subscales are positively correlated to each other. The
positive correlations between the subscales and the high correlations between
the subscales and the total score on A-CAL point to the logical consistency of the
questionnaire and indicate that the A-CAL questionnaire is a feasible working
instrument, but needs more confirmational research.
A-FLL questionnaire
In line with previous research on attitude, the construct of attitude was split into
three subcomponents: cognitive, affective-evaluative and behavioral. Each
Table 5. A-CAL: Goodness-of-fit indices for the two-factor structure tested with CFA.

Calibration sample 119 39.73 (24) 1.66 .074 .94 .88 .98 .97
Validation sample 117 75.97 (24)* 3.16 .14 .90 .81 .92 .88
Full sample 236 94.98 (24)* 3.54 .103 .93 .87 .96 .94
Figure 2. A-CAL: Standardized factor loadings of the two-factor model as obtained with
confirmatory factor analysis for the calibration sample and the validation sample (between
parentheses). Numbering of items is in accordance with the numbering in Appendix 2.
Table 6. A-CAL: Means (M), standard deviations (SD), reliability (a) and Pearson
correlations between the A-CAL subscales (entire sample).
M (SD) a 2 3
1. Computer proficiency (5 items) 22.41 (6.44) .92 .59* .95*
2. Computer integration (4 items) 21.47 (3.66) .80 _ .82*
component was first analyzed separately and subsequently in relation to the

other subcomponents of attitude in order to create a full questionnaire measuring
attitude towards foreign language learning. Exploratory factor analysis will give
more details on the underlying factor structure of the questionnaire. Because of the
large number of items and for reasons of clarity, we decided to split the analyses per
subcomponent.
Cognitive component – steps 1 to 4

There were 17 items that were related to the cognitive component of attitude towards
foreign language learning. The results of the EFA revealed a two-factor solution,
accounting for 35% of the variance. Factor I was labeled ‘language learning
proficiency’. Six items capture the self-evaluated proficiency of subjects with
language learning (e.g., ‘I’m efficient in learning a language’). Factor II was labeled
‘language learning beliefs’. Four items define the beliefs subjects have related to
language learning (e.g., ‘People with a high IQ are good at learning a language’).
Seven items did not load substantially on one of the two factors and they did not
form separate factors either. These items were related to specific properties of
language learning (e.g., ‘According to me, learning French is easier than learning
English’ and ‘The most important part of language learning is vocabulary’) and were
removed from further analyses.
In a next step, a confirmatory factor analysis was executed on the two-factor
structure with ten items. What strikes one the most is that none of the items that
were supposed to load on the factor ‘language learning beliefs’ loaded

significantly on this factor. Also the correlation between the two factors was
very low (r ¼ .26), indicating that a two-factor structure did not adequately
represent the construct of language learning cognitions. After removing the factor
‘beliefs’ and its associated items, the scale ‘cognition towards foreign language
learning’ was solely defined by general cognitions (seven items). The removal of
one additional item with low loading on the factor (‘I’m able to learn a
language’) and allowing for covariance between the residuals of items 2 (‘I’m
efficient in language learning’) and 3 (‘I consider myself as successful in language
learning’), we obtained a model that fits the data in a good way and has an
adequate representation of the cognitive component of language learning attitude.
Only the RMSEA index indicated a bad fit; this is probably due to the tendency
of this fit measure to favor models with many parameters. Because of the
preliminary nature of this part of the research, no validation analyses were done.
Cronbach’s a for the final scale (6 items) is .94.
Affective-evaluative component – steps 1 to 4

A set of 27 items is related to the affective-evaluative component of attitude towards
foreign language learning. Items were created in line with previous studies focusing
on motivational influences and ‘integrative-ness’ (Gardner, Tremblay, & Masgoret,
1997). The results of the EFA revealed a meaningful four-factor solution, accounting
for 49% of the variance. Factor I, labeled ‘extrinsic motivation’ comprises two
factors that were taken together because all of the associated items represented
extrinsic motivation (four items related to job opportunities and four items related to
compulsory language learning). Factor II was labeled ‘intrinsic motivation’. Seven
items capture the experienced inherent need for language learning (e.g., ‘I consider it
very logical that I learn a language that is spoken in my country’). Factor III was
labeled ‘teacher influence’ and comprises three items that define the importance
subjects attach to teacher characteristics (e.g., ‘The enthusiasm of the (language)
teacher influences my attitude towards the language’). Three items loaded on a
separate factor that could be labeled as ‘influence of culture, people and country on
language learning’. Since this factor only explained 4.7% of the variance in the
sample and with regard to model parsimony, it was decided to leave out the items
associated with this factor. The other six items that were removed did not load on
any of the previously mentioned factors and did not form a separate factor either.
This is probably due to the wording of the items (conditional sentences, e.g., ‘If I
would have more time, then I would learn a new language’).
Following the exploratory factor analysis, a confirmatory factor analysis was
executed on the three-factor structure with 18 items. Items that did not significantly
contribute (i.e., no significant regression weights) were removed from the analysis.
The residuals of the items 9 (‘In a country with two or more official languages, it is
important to speak at least two of the languages’) and 10 (‘I consider it very logical
that I learn a language that is spoken in my country’) were allowed to covary,
because of similar wording. Finally, a model was obtained with three factors
(intrinsic motivation, extrinsic motivation and teacher influence), with fit indices
showing a good to very good model fit.
Because of the preliminary and demonstrative nature of this part of the
research, no validation analyses were done and conclusions must be drawn

tentatively. Values for Cronbach’s a are much more higher in the final scales
(extrinsic motivation – reversed: a ¼ .85; intrinsic motivation: a ¼ .79; and teacher
influence: a ¼ .92).
Behavioral component – steps 1 to 4

The behavioral component of attitude towards foreign language learning consisted
of 20 items, extracted from previous research on exhibition, inhibition and tolerance
of ambiguity in language learning (Johnson, 2001). The results of the EFA revealed a
complex four-factor solution, accounting for 44% of the variance. Factor I (three
items) was labeled ‘inhibition’ and captures the degree of inhibition to foreign
language learning (e.g., ‘I’m often concerned about making mistakes’). Factor II was
could be labeled ‘exhibition’. Four items define the degree of exhibition to foreign
language learning (e.g., ‘Speaking a foreign language is very important for learning
the language’). A third factor could be labeled ‘learning effort’ and comprises six
statements about the efforts subjects are willing to make in learning a language (e.g.,
‘If the learning effort is too much, I’m losing pleasure in language’). A fourth and
final factor (five items) was labeled ‘tolerance of ambiguity’ (e.g., ‘For pleasure,
I often have conversations in a foreign language’).
Two items did not load univocally on one of the four factors and they did not
form a separate factor either. These items were removed from further analysis.
The four-factor structure as revealed by EFA (18 items) was tested in a
confirmatory factor analysis. Five items had very low and non-significant loadings
on the constructs and were therefore removed. Although the factor loadings of the
items on their associated construct were not as high as was expected, the four-factor
structure was confirmed and subscales correlated with each other in a presupposed
way. No modifications in terms of residual covariances have been added. Again,
values for Cronbach’s a are higher in the final scales but (inhibition – reversed:
a ¼ .85; exhibition: a ¼ .64; tolerance of ambiguity: a ¼ .75; and learning effort –
reversed: a ¼ .70).
Full model of attitude towards language learning

As presented in Table 7, the fit measures of the final model, including the
cognitive, affective/evaluative and behavioral component, indicate a mediocre to
acceptable fit. Since this research does not want to present a final validated series
of questionnaires and hence is preliminary in nature, no validation analyses were
done and results must be interpreted with caution.
Figure 3 (a, b, c) displays the standardized factor loadings of the three-
component model of attitude towards foreign language learning. As can be
seen in this figure, standardized regression coefficients or loadings of the items on
the cognitive component are very high, pointing to high internal
consistency. Related to the subscales of the affective/evaluative component, factor
loadings are varying between acceptable and high. The loadings of the items
associated with the subscales of the behavioral/personality component are
acceptable but need further exploration in terms of wording or underlying
construct representation.
Tables 8 and 9 summarize the descriptive statistics, the reliability (Cronbach’s a)

and Pearson intercorrelations for the subscales within each component. The
cognitive component has no subscales and no interrelations can be reported. The
affective-evaluative component comprises three subscales: extrinsic motivation –
reversed, intrinsic motivation and teacher influence, all having an adequate to good
reliability. The subscale ‘extrinsic motivation’ was reversed (i.e., the higher the score,
the lower the extrinsic motivation), this reversed scale is positively correlated to the
subscale ‘intrinsic correlation’, indicating that a low score on extrinsic motivation
Table 7. A-FLL: Goodness-of-fit indices for the subcomponents’ structure tested with CFA
(only calibration sample).

Cognitive 119 11.32 (4) 2.83 .124 .96 .87 .99 .97
Affect-evaluative 119 71.07 (61) 1.17 .037 .92 .88 .99 .98
Behavioral 119 97.72 (59)* 1.66 .074 .88 .82 .92 .90
Figure 3a. A-FLL – Cognitive component: Standardized factor loadings of the one-factor
model as obtained with confirmatory factor analysis for the calibration sample. Numbering of
items is in accordance with the numbering in Appendix 3.
Figure 3b. A-FLL – Affective/evaluative component: Standardized factor loadings of the

three-factor model as obtained with confirmatory factor analysis for the calibration sample.
Numbering of items is in accordance with the numbering in Appendix 3.
correlates with a high score on intrinsic motivation (r ¼ .28). Also, ‘intrinsic

motivation’ is positively correlated to ‘teacher influence’ (r ¼ .39); the higher the
intrinsic motivation the greater the importance subjects attach to the language
teacher. The last component, behavior, consists of four subscales: inhibition –
reversed, exhibition, tolerance of ambiguity and learning effort. The reliability of the
subscales is acceptable but cannot be considered as sufficient or good. Having a look
at the internal consistency the scales ‘inhibition’ and ‘learning effort’ were reversed,
so that both scales contributed to the general behavioral component in a positively
formulated way. The reversed scale ‘inhibition’ does not correlate with exhibition
(r ¼ 7.08), but has positive correlations with ‘tolerance of ambiguity’ (r ¼ .27) and
‘learning effort – reversed’ (r ¼ .25). This means that, the less a subject is inhibited
when learning a foreign language, the higher the tolerance of ambiguity and the
lower the learning effort is rated. The scale ‘exhibition’ correlates positively with
Figure 3c. A-FLL – Behavioral/personality component: Standardized factor loadings of the

four-factor model as obtained with confirmatory factor analysis for the calibration sample.
Numbering of items is in accordance with the numbering in Appendix 3.
‘tolerance of ambiguity’ (r ¼ .45) and finally, tolerance of ambiguity has a positive

relation with learning effort – reversed (r ¼ .22), i.e., the more subjects can deal with
ambiguity in the language learning situation, the less they rate the learning efforts.
Table 9 presents intercorrelations between all the subscales of the three components.
It is shown that, for the affective-evaluative and behavioral component, their
associated subscales contribute significantly and positively to the general component.
Based in this conclusion, we have chosen to work with the general component scores
in order to obtain a certain degree of model parsimony when constructing the
structural equation model that posits attitude towards language learning as
predictive and influenced construct.
Table 8. A-FLL – all components: Means (M), standard deviations (SD), reliability (a) and
Pearson correlations for the cognitive, affective/evaluative and behavioral/personality subscale
(entire sample).
M (SD) a 2 3 4 5 6 7
Cognitive (6 items) 28.37 (7.74) .94
Affective-evaluative (13 items)
1. Extrinsic motivation – 7.66 (3.52) .85 28* .06
reversed (3 items)
2. Intrinsic motivation (7 items) 40.18 (5.00) .79 _ .39*
3. Teacher influence (3 items) 16.89 (3.09) .92 _ _
Behavioral/personality (13 items)
4. Inhibition – reversed (2 items) 8.49 (3.02) .85 7.08 .27* .25*
5. Exhibition (3 items) 17.98 (2.12) .64 _ .45* .01
6. Tolerance of ambiguity (3 items) 14.80 (3.43) .75 _ _ .22*

7. Learning effort – reversed (5 items) 20.78 (4.56) .70 _ _ _
Table 9. A-FLL – all components: Pearson intercorrelations for all subscales (entire sample).
2 3 4 5 6 7 8 9 10
1. Cognitive component .44* .53* .31* .01 .63* .20* .43* .58* .41*
2. Affective component – .61* .86* .63* .43* .04 .62* .52* .09
3. Affective – extrinsic – – .28* .06 .56* .13* .40* .44* .43*
motivation
4. Affective – intrinsic – – – .39* .29* .01 .56* .43* 7.05
motivation
5. Affective – teacher influence – – – – .05 7.06 .30* .21* –
6. Behavioral component – – – – – .58* .41* .73* .18*
7. Behavioral – inhibition – – – – – – – .27* .72*
8. Behavioral – exhibition – – – – – – .08* .45* .25*
9. Behavioral – tolerance – – – – – – – – .01
of ambiguity
10. Behavioral – learning effort – – – – – – – .22*
– –
Discussion
Based on the items of several existing, but not validated questionnaires about
attitude towards foreign language learning and computer-assisted learning we
composed a set of items, resulting in a first version of three questionnaires:
attitude towards computer-assisted language learning (A-CALL), attitude towards
computer-assisted learning (A-CAL) and attitude towards foreign language
learning (A-FLL). These questionnaires were designed to assess the attitude and
its subcomponents towards CALL, CAL and FLL. A cognitive component, an
affective/evaluative component and a behavioral/personality component are
generally agreed to be the three subcomponents of attitude.
Analysis of internal consistency of the preliminary A-CALL questionnaire
indicated that the tripartite model of attitude could roughly be corroborated in our
data. Exploratory factor analysis revealed a five-factor solution that could be

adjusted to a four-factor solution. In a next step, confirmatory factor analysis was
used to validate the model. After testing the model with four factors, we let the
residuals correlate for three pairs of items, because of content and statistical-related
issues. Either a pair with correlated residuals consisted of items with very similar
wording, or the items in a pair both had a positively skewed distribution. After
allowing the residuals of these items to covary, the model was refitted, resulting in a
significant improvement of model fit. The same model was tested using a validation
sample, confirming the earlier defined model structure. However, indices of the
model do not yield very strong confirmation of the model structure as tested in this
study. General model fit was rather mediocre. As a result we obtained a 20-item
questionnaire with a four-factor structure:
(1) effectiveness of CALL vs. non-CALL (four items);

(2) surplus value of CALL (10 items);
(3) teacher influence (three items); and
(4) exhibition to CALL (three items).
We can roughly organize the subscales into the classical three-component

structure of attitude: the cognitive and affective/evaluative component are
strongly interrelated and are represented by the first two scales, ‘effectiveness of
CALL vs. non-CALL’ and ‘surplus value of CALL’. The behavioral component
of attitude is captured in the scales ‘teacher influence’ and ‘degree of exhibition to
CALL’.
The internal consistency of the A-CALL questionnaire and its four subscales was
further confirmed by high values for Cronbach’s a and significant intercorrelations
between several subscales. The more subjects believe in the efficiency of CALL (e.g.,
‘Learning a foreign language assisted by computer is not as good as learning it by
oral practice’, reversed scoring), the more they trusted CALL-applications and have
strong beliefs about the mere value of CALL (e.g., ‘My language learning will
proceed more when this is assisted by a computer’). Also, the more subjects believe in
the efficiency of CALL and have confidence in CALL-applications, the more they are
susceptible to the attitude and proficiency of teachers towards CALL (e.g.,
‘Teacher’s enthusiasm towards CALL largely defines my motivation for using
computers in language learning’). Subjects with high confidence in CALL-
applications also feel less inhibited when learning a language with computer
assistance compared to face-to-face language learning (‘I feel less inhibited when
communicating in the foreign language via computer (chat) than in a face-to-face
situation’).
Two other questionnaires, A-CAL and A-FLL, have been developed in line
with the procedure as used in the construction and validation of the A-CALL.
The model fit of the A-CAL is mediocre to good and the scale shows good
internal consistency. The two subscales, ‘computer proficiency’ and ‘computer
integration’ show a significant positive relationship. Subjects who consider
themselves as highly proficient (e.g., ‘I have an advanced knowledge of
computers’) obtain high scores on computer integration (e.g.,‘I love learning
new things on a computer’).
The last questionnaire that is presented in this research attempts to represent a
validated measure for foreign language learning attitude (A-FLL). Again, this
questionnaire comprises a cognitive, affective/evaluative and behavioral compo-

nent. The cognitive component can be considered as reliable and internally
consistent and represents cognitions about foreign language learning, the
affective/evaluative and behavioral components and consisting of several subscales
representing the underlying constructs. Affect/evaluation related to foreign
language learning is represented by extrinsic motivation, intrinsic motivation
and teacher influence. The lower the extrinsic motivation, the higher the score on
intrinsic motivation, this relationship gives strong indication for the internal
consistency of the affective/evaluative subcomponent. Subjects showing high
intrinsic motivation also attach great importance to teacher behavior or
attitude. A last component of A-FLL is related to behavior and consists of
inhibition, exhibition, tolerance of ambiguity and learning effort. Correlations
between the several subscales show several relationships. The more people are
anxious about learning a new language (degree of inhibition), the less tolerant
they are when involved in ambiguous (language learning) situations and the more
they consider the language learning effort (too) high. Also, the more people like
to be exposed to a foreign language, the less they consider the learning effort as
(too) high.
Conclusion
The influence of learners’ attitudes towards language learning and, more
specifically, computer-assisted language learning, is one of the most frequently
investigated topics in the debate on the effectiveness of CALL. However, too
often in CALL research, a set of questions is considered as a reliable and
representative measurement of what researchers want to measure. Also, questions
are developed in line with software that is used, in line with specific goals of the
studies, or in line with participant characteristics. Since the construct of attitude
cannot be formulated unidimensionally, the need for an operational and
multifaceted definition of the attitude-construct emerged. This research described
in this paper aimed at introducing a standard procedure for constructing and
validating measurement for attitude towards computer-assisted language learning
(CALL), computer-assisted learning (CAL) and foreign language learning (FLL).
More generally, this procedure can also be adopted for the validation of other
CALL-related unobservable constructs that we want to measure. Rather than
focusing on the results of the statistical analyses, we wanted to present the reader
with a clearly formulated and stepwise approach to the several types of statistical
validation of a questionnaire in this research field. The focus of this research was
on the process of developing and validating unobservable or latent constructs in
CALL research.
There are some limitations to this study. First, we intend the A-CALL, A-CAL
and A-FLL to be used in a wide range of situations and for diverse groups of
subjects. For this study however, the validation of the three questionnaires was
limited to a sample that largely consisted of subjects with a certain background in
language learning (e.g., 78% of the participants had foreign language training in
school for at least five hours a week and about two thirds of all participants with
CALL-experience had experience with making CALL-exercises), suggesting that
previous foreign language learning behavior or prior experiences with CALL might
have influenced attitude towards CALL and therefore might have caused some
degree of bias in the results, or in the structure of the questionnaire. Therefore, we

suggest being cautious when using the statistics mentioned in this study as normative
data. Further research is needed with more diverse populations of foreign language
learners, also group analyses are needed to confirm, refine or reject the structure of
the questionnaire for groups with and without CALL-experience, with and without
language learning experience and with and without experience in computer use.
A second limitation is the overall sample size (N ¼ 240) and the sample sizes for
the calibration and validation samples. Although a sample size of more than 200
subjects is generally acceptable when doing confirmatory factor analyzed, the
validity of the results decreases as the sample becomes more heterogeneous. Under
the conditions as described in this research, internal consistency and reliability of all
scales can be considered as adequate to very good. However, for some constructs,
confirmatory factor analyses did not show good fit. It is highly probably that this is
due to a small sample size with regard to the heterogeneity of our sample. More
research with more diverse types of learners and learners’ backgrounds is needed to
elaborate the findings in this research and to confirm or disconfirm the structure of
the models developed in this research.
A final limitation of this study is that it is based on the self-report of attitude.
Relating the scores of subjects on the A-CALL to the information gathered through
tracking and logging data will give a more detailed view of influences of attitude
towards CALL on the effectiveness of CALL for individual learners or groups of
learners.
Future research should focus on multi-group analyses including different profiles
or clusters that exist in (language) learners. As Huskinson and Haddock (2006)
suggest, individuals with highly structured attitudes may have more accessible
cognitions and effects, resulting in stronger influences of these components on the
behavior of individuals. Furthermore, by using a time series design, eventually
accompanied by the inclusion of experimental conditions, attitude towards CALL,
CAL and FLL can be modeled in function of the progress or decline of learners.
Learners’ attitudes towards CALL could be related to the collective intelligence of
learning environments, representing learners’ behavior in an electronic learning
environment, collected by tracking and logging data. By repeatedly analyzing the
combination of this collective intelligence representing learners’ behavior and more
personal user characteristics (such as attitude towards CALL), it is expected to
elaborate and discuss the conclusions of this and other CALL research.
The results of this research fit in the more general conclusion that is drawn from
most CALL-research: a lot remains to be done (Felix, 2008). Our results indicate
support for a multifaceted structure of attitude towards CALL, including cognitions,
affects and behavior. Using the A-CALL and the other questionnaires as a stand-
alone instrument for measuring one’s attitude towards CALL would not fully
express the possibilities of this questionnaire. The development of these ques-
tionnaires was only a first and preliminary step towards the positioning of attitude
towards CALL into a broader framework of encouraging or discouraging factors
that influence the success or otherwise of CALL, such as attitude towards computer-
assisted learning in general and general attitude towards language learning. Gardner
and colleagues (Gardner, 2005; Masgoret & Gardner, 2003) already instigated the
construction of larger models that place second language learning in relation to other
constructs. The construction of larger integrative models also fits in the recent call of
Hubbard (2008) to create more native CALL theories. In his research he reported on
the exceedingly small number of research studies describing the development of

CALL theory. We fully agree with Hubbard’s plea for a conceptual framework in
order to promote theory development in CALL research. Yet, we want to stress the
importance of a usable methodological framework as a second condition for theory
development, confirmation or falsification.
This paper presented a validation procedure for creating valid operationaliza-
tions of non-observable constructs in research on CALL, which is, in our opinion, a
first step towards the construction of testable CALL theory. However, we must bear
in mind that CALL comes in many forms and many applications, resulting in a very
heterogeneous group of CALL-users. One single attitude questionnaire would not
give due to the variations in CALL-users and hence varying aspects of attitude. We
therefore encourage other researchers to develop measurements that mirror attitude
towards CALL and its relevant aspects by using rigorous methods.
Acknowledgements
The authors want to express their appreciation to the two anonymous reviewers for their
valuable comments.
Notes on contributors
Mieke Vandewaetere is a PhD candidate in instructional design and computer-assisted
learning. She received her MA in experimental and theoretical psychology and also followed a
MSc in statistical data-analysis. Her primary interests lie in the area of cognitive psychology,
statistics and data-analysis and computer-assisted learning. Other interests are instructional
design, educational data-mining and intelligent tutoring systems.
Piet Desmet is full professor of French and applied linguistics at the Linguistics Department of
KU Leuven and KU Leuven Campus Kortrijk. His research mainly focuses on computer-
assisted language learning (adaptivity, e-testing, corpusCALL, digital libraries, etc.). He is one
of the board members of the research team itec (interdisciplinary research on technology,
education and communication) of KU Leuven Campus Kortrijk.
References
Arbuckle, J.L. (1995–1999). Amos 16.0. User’s guide. Chicago, IL: SPSS.
Ayres, R. (2002). Learner attitudes towards the use of CALL. Computer-Assisted Language
Learning Journal, 15(3), 241–249.
Bax, S. (2003). CALL – Past, present and future. System, 31, 13–28.
Beatty, K. (2003). Teaching and researching computer assisted language learning. Applied
Linguistics in Action Series. Harlow: Pearson Education.
Bentler, P.M. (1990). Comparative fit indexes in structural models. Psychological Bulletin, 107,
238–246.
Bernat, E., & Gvozdenko, I. (2005). Beliefs about language learning: Current knowledge,
Pedagogical implications, and new research directions. Teaching English as a Second or
Foreign Language, 9(1), 1–21.
Bollen, K.A., & Long, J.S. (1993). Testing structural equation models. Newbury Park, CA:
Sage.
Browne, M.W. (1984). Asymptotic distribution free methods in the analysis of covariance
structures. British Journal of Mathematical and Statistical Psychology, 37, 62–83.
Browne, M.W., & Cuddeck, R. (1993). Alternative ways of assessing model fit. In K. Bollen &
J. Long (Eds.), Testing structural equation models (pp. 136–162). Newbury Park, CA:
Sage.
Byrne, B.M. (2001). Structural equation modelling with AMOS: basis concepts, applications and
programming. New Jersey, NJ: Lawrence Erlbaum Associates, Inc.
Cattell, R.B. (1978). The scientific use of factor analysis in behavioral and life sciences. New
York: Plenum.
Chambers, A., & Davies, G. (2001). New technologies and language learning: A European
perspective. Lisse: Swets & Zeitlinger.
Chapelle, C.A. (2003). English language learning and technology: Lectures on applied linguistics
in the age of information and communication technology. Amsterdam: John Benjamins
Publishing.
Cotterall, S. (1995). Readiness for autonomy: Investigating learner beliefs. System, 23, 195–
205.
DeCarlo, L.T. (1997). On the meaning and use of kurtosis. Psychological Methods, 2, 292–307.
Desmet, P. (2007). L’apport des TIC à la mise en place d’un dispositif d’apprentissage des
langues centré sur l’apprenant. ITL. International Journal of Applied Linguistics, 154, 91–
110.
Desmet, P., & Héroguel, A. (2005). Les enjeux de la création d’un environnement
d’apprentissage électroniques axé sur la compréhension orale à l’aide du système auteur
IDIOMA-TIC. ALSIC. Apprentissage des Langues et Syste`mes d’Information et de

Communication, 8, 281–303.
Felix, U. (2004). A multivariate analysis of secondary students’ experience of web-based
language learning. ReCALL, 16(1), 237–249.
Felix, U. (2005). What do meta-analyses tell us about CALL effectiveness? ReCALL, 17(2),
269–288.
Felix, U. (2008). The unreasonable effectiveness of CALL: What have we learned in two
decades of research? ReCALL, 20(2), 141–161.
Gan, Z. (2004). Attitudes and strategies as predictors of self-directed language learning in an
EFL context. International Journal of Applied Linguistics, 14(3), 389–411.
Gardner, R.C. (1985). Social psychology and second language learning: The role of attitudes and
motivation. London: Edward Arnold.
Gardner, R.C. (2001). Integrative motivation: Past, present and future. Public lecture presented
at Temple University Japan, Distinguished Lecturer Series. Retrieved October 14, 2008,
from http://publish.uwo.ca/*gardner/docs/GardnerPublicLecture1.pdf
Gardner, R.C. (2005). Motivation and attitudes in second language learning. Encyclopedia of
language and linguistics (second edition). Oxford, UK: Elsevier.
Gardner, R.C., Tremblay, P.F., & Masgoret, A.-M. (1997). Towards a full model of second
language learning: an empirical investigation. Modern Language Journal, 81(3), 344–362.
Horwitz, E.K. (1987). Surveying student beliefs about language learning. In A. Wenden & J.
Rubin (Eds.), Learner strategies in language learning (pp. 119–129). Englewood Cliffs, NJ:
Prentice Hall.
Horwitz, E.K. (1988). The beliefs about language learning of beginning university foreign
language students. The Modern Language Journal, 72, 283–294.
Hu, L., & Bentler, P.M. (1999). Cutoff criteria for fit indexes in covariance structure analysis:
Conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1–55.
Hubbard, P. (2008). Twenty-five years of theory in the CALICO journal. CALICO Journal,
25(3), 387–399.
Huskinson, T.L.H., & Haddock, J. (2006). Individual differences in attitude structure and the
accessibility of the affective and cognitive components of attitude. Social Cognition, 24(4),
453–468.
Johnson, K. (2001). An introduction to foreign language learning and teaching. London:
Longman.
Jöreskog, K.G., & Sörbom, D. (1984). Lisrel IV user’s guide. Mooresville, IN: Scientific
Software.
Laurillard, D. (2001). Rethinking university teaching, a conversational framework for the
effective use of learning technologies (2nd Edition). London: Routledge.
Levy, M., & Stockwell, G. (2006). CALL dimensions. Options and issues in computer-assisted
language learning. Mahwah, New Jersey: Lawrence Erlbaum Associates.
Liaw, S.S. (2002). An Internet survey for perceptions of computer and world wide web:
relationship, prediction, and difference. Computers in Human Behavior, 18(1), 17–35.
Liaw, S.S., Huang, H.M., & Chen, G.D. (2007). Surveying instructor and learner attitudes
toward e-learning. Computers and Education, 49, 1066–1080.
Lightbrown, P., & Spada, F. (1999). How languages are learned (rev. ed.). Oxford: Oxford
University Press.
Masgoret, A.-M., & Gardner, R.C. (2003). Attitudes, motivation, and second language
learning: A meta-analysis of studies conducted by Gardner and associates. Language
Learning, 53, 123–163.
Merisuo-Storm, T. (2007). Pupils’ attitudes toward foreign-language learning and the
development of literacy skills in bilingual education. Teaching and Teacher Education,
23(2), 226–235.
Myung, I.J. (2003). Tutorial on maximum likelihood estimation. Journal of Mathematical
Psychology, 47, 90–100.
Nikitina, L., & Furuoka, F. (2007). Beliefs about language learning: A comparison between
novice and intermediate level students learning Russian at a Malaysian university. The
Linguistics Journal, 2(1), 7–27.
Olsson, U.H., Foss, T., Troye, S.V., & Howel, R.D. (2000). The performance of ML, GLS and
WLS estimation in structural equation modeling under conditions of misspecification and
nonnormality. Structural Equation Modeling, 7, 557–595.

Sagarra, N., & Zapata, G.C. (2008). Blending classroom instruction with online homework: A
study of student perceptions of computer-assisted L2 learning. ReCALL, 20(2), 208–224.
Schermelleh-Engel, K., Moosbrugger, H., & Müller, H. (2003). Evaluating the fit of structural
equation models: Test of significance and descriptive goodness-of-fit measures. Methods of
Psychological Research, 8(2), 23–74.
Sengupta, S. (2001). Exchanging ideas with peers in network-based classrooms: An aid or a
pain? Language Learning & Technology, 5(1), 103–134.
Sil International (1999). Language learning attitude questionnaire. LinguaLinks Library,
Version 3.5. Retrieved October 14, 2008, from http://www.sil.org/lingualinks/language
learning/PrepareForLanguageLearning/attitude.pdf
Smith, A.A. (1971). The importance of attitude in foreign language learning. Modern
Language Journal, 55, 82–88.
Tucker, L.R., & Lewis, C. (1973). A reliability coefficient for maximum likelihood factor
analysis. Psychometrika, 38, 1–10.
Wenden, A.L. (1991). Learner strategies for learner autonomy. London: Prentice Hall.
Appendix 1. The attitude towards CALL Questionnaire (A-CALL) – English version

People who learn a foreign language assisted by computer have different attitudes towards that
learning process. We would like to know how your attitude is towards Computer Assisted
Language Learning (CALL). Please read each statement and indicate the extent to which you
agree with the following statements. Please mark your response by circling the number to the
right of each statement from 1 (totally disagree) to 7 (totally agree).
totally totally
disagree agree
1. My language learning will 1 2 3 4 5 6 7
proceed more when this is
assisted by a computer.
2. Learning a foreign language 1 2 3 4 5 6 7

assisted by computer is not as
good as learning it by oral
practice.
3. Computer-based language tests 1 2 3 4 5 6 7
can never be as good as paper-
and-pencil tests.
4. Computer-assisted language 1 2 3 4 5 6 7
learning is less adequate as the
traditional language learning.
5. People who learn a language by 1 2 3 4 5 6 7
computer-assisted learning are
less proficient than traditional
language learners.
learning is a valuable extension
of the classical learning methods.
learning gives more flexibility to
language learning.
learning is as valuable as
traditional language learning.
learning can stand alone.
10. Learning a foreign language by 1 2 3 4 5 6 7
computer constitutes a more
relaxed and stress free
atmosphere.
11. Learning a foreign language by 1 2 3 4 5 6 7
computer enhances your
intelligence.
12. I (would) like learning a new 1 2 3 4 5 6 7
language by computer.
13. Teacher’s attitude towards 1 2 3 4 5 6 7
CALL largely defines my
attitude towards the use of
computers in language learning.
14. Teacher’s enthusiasm towards 1 2 3 4 5 6 7
CALL largely defines my
motivation for using computers
in language learning.
(continued)
Appendix 1. (Continued).
totally totally
disagree agree
15. Teacher’s proficiency of using 1 2 3 4 5 6 7
computers in language learning
largely defines my attitude
towards computer use in
language learning.
16. I have faith in computer-based 1 2 3 4 5 6 7
language tests.
17. I have faith in computer-based 1 2 3 4 5 6 7
language exercises.
18. I feel less inhibited when 1 2 3 4 5 6 7
communicating in the foreign

language via computer (chat)
than in a face-to-face situation.
19. In a face-to-face learning 1 2 3 4 5 6 7
situation (classroom) I often
experience anxiety when
speaking in the foreign
language.
20. For me, the threshold to start a 1 2 3 4 5 6 7
face-to-face conversation is
bigger than starting a virtual
(computer-assisted) conversation.
Scoring:
. Effectiveness of CALL vs. non-CALL: sum score of items 2, 3, 4 and 5 (all reversed).
. Teacher influence: sum score of items 13, 14 and 15.
. Degree of exhibition to CALL: sum score of items 18, 19 and 20.
Appendix 2. The attitude towards CAL Questionnaire (A-CAL) – English version

People who learn by computer have different attitudes towards that learning process. We would
like to know how your attitude is towards Computer Assisted Learning (CAL). Please read each
statement and indicate the extent to which you agree with the following statements. Please mark
your response by circling the number to the right of each statement from 1 (totally disagree) to 7
(totally agree).
totally totally
disagree agree
1. I can work well with computers. 1 2 3 4 5 6 7
2. I have an advanced knowledge of 1 2 3 4 5 6 7
computers.
3. I love to explore the possibilities 1 2 3 4 5 6 7

and qualities of my computer.
4. I have an innate computer knack. 1 2 3 4 5 6 7
5. Compared to others, my speed of 1 2 3 4 5 6 7
learning to work with computers is
higher.
6. I love to use a computer. 1 2 3 4 5 6 7
7. The larger my computer 1 2 3 4 5 6 7
knowledge, the more I love to
work with a computer.
8. I love learning new things about 1 2 3 4 5 6 7
computers.
9. Some content can be learned 1 2 3 4 5 6 7
faster when using a computer.
Scoring:
. Computer proficiency: sum score of items 1 to 5.
. Computer integration: sum score of items 6 to 9.
Appendix 3. The attitude towards Foreign Language Learning Questionnaire (A-FLL)

– English version
People who learn a new language have different attitudes towards that learning process. We
would like to know how your attitude is towards foreign language learning (FLL). Please read
each statement and indicate the extent to which you agree with the following statements. Please
mark your response by circling the number to the right of each statement from 1 (totally
disagree) to 7 (totally agree).
totally totally
disagree agree
COGNITIVE COMPONENT
1. I’m efficient in language learning 1 2 3 4 5 6 7
2. I consider myself successful in 1 2 3 4 5 6 7

language learning
3. I have an innate capacity for 1 2 3 4 5 6 7
learning a language
4. I learn a new language faster 1 2 3 4 5 6 7
than on average
5. I believe I have a special ability 1 2 3 4 5 6 7
for learning a language
Scoring:
. A-FFL - cognitions: sum score of items 1 to 5
AFFECTIVE/EVALUATIVE COMPONENT
1. I only learn a foreign language 1 2 3 4 5 6 7
to succeed or to obtain a
diploma
2. If I was not obliged to learn 1 2 3 4 5 6 7
another language, then I would
not learn one
3. Learning a foreign language is 1 2 3 4 5 6 7
important for your future, but I
do not like it
4. It is important to speak the 1 2 3 4 5 6 7
languages that are spoken
around you
5. The higher my foreign language 1 2 3 4 5 6 7
proficiency, the more I enjoy
speaking the language
6. I feel sympathy for the native 1 2 3 4 5 6 7
speakers of the language I am
learning
7. It is impossible to get to know 1 2 3 4 5 6 7
people if you do not speak
their language
8. Living in a foreign country, you 1 2 3 4 5 6 7
show respect by learning the
language of that country
9. I consider it very logical that I 1 2 3 4 5 6 7
learn a language that is spoken
in my country
10. In a country with two or more 1 2 3 4 5 6 7
official languages, it is
important to speak at least two
of the languages
(continued)
Appendix 31. (Continued).

totally totally
disagree agree
11. The enthusiasm of the 1 2 3 4 5 6 7
(language) teacher influences
my attitude towards the language
12. The language proficiency of the 1 2 3 4 5 6 7
(language) teacher influences my
attitude towards the language
13. The attitude of the (language) 1 2 3 4 5 6 7
teacher influences my attitude
towards the language
Scoring:
. A-FLL – Extrinsic motivation: sum score of items 1 to 3

. A-FLL – Intrinsic motivation: sum score of items 4 to 10
. A-FLL – Teacher influence: sum score of items 11 to 13
BEHAVIORAL/PERSONALITY COMPONENT
1. I am afraid people will laugh at 1 2 3 4 5 6 7
me when I say things wrong
2. I discover some degree of initial 1 2 3 4 5 6 7
resistance when having to start
talking in a foreign language
3. I try to understand people when 1 2 3 4 5 6 7
they are talking in another
language
4. I love learning new things 1 2 3 4 5 6 7
5. Talking in the language you learn 1 2 3 4 5 6 7
is very important in learning it
6. For pleasure, I often have 1 2 3 4 5 6 7
conversations in a foreign
language
7. I love an intensive use of another 1 2 3 4 5 6 7
language (at school, at work)
8. I don’t mind switching to another 1 2 3 4 5 6 7
language
9. I’m losing the pleasure of 1 2 3 4 5 6 7
language learning when learning
effort becomes too big
10. When learning effort decreases, 1 2 3 4 5 6 7
I enjoy a foreign language more
11. I don’t like teachers staying 1 2 3 4 5 6 7
focused on learning vocabulary
and grammar, instead of learning
to converse
12. Learning another language goes 1 2 3 4 5 6 7
way to slow for me
13. It is frustrating that learning a 1 2 3 4 5 6 7
new language is slow in the
beginning
Scoring:
. A-FLL – Inhibition: sum score of items 1 and 2
. A-FLL – Exhibition: sum score of items 3 to 5
. A-FLL – Tolerance of Ambiguity: sum score of items 6 to 8
. A-FLL – Learning effort: sum score of items 9 to 13

2009 - Vandewaetere - Measuring Students Attitude Towards CALL

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

2009 - Vandewaetere - Measuring Students Attitude Towards CALL

Uploaded by

Copyright:

Available Formats

This article was downloaded by: [National Cheng Kung University]

On: 23 April 2013, At: 19:45

Computer Assisted Language Learning

Introducing psychometrical validation

To link to this article: http://dx.doi.org/10.1080/09588220903186547

PLEASE SCROLL DOWN FOR ARTICLE

Full terms and conditions of use: http://www.tandfonline.com/page/terms-and-

Introducing psychometrical validation of questionnaires in CALL

(Received 20 October 2008; ﬁnal version received 20 May 2009)

The great majority of questionnaires measuring non-observable constructs such

*Corresponding author. Email: Mieke.Vandewaetere@kuleuven-kortrijk.be

ISSN 0958-8221 print/ISSN 1744-3210 online

(2003) distinguishes three types of research in CALL, with respectively a focus on

made use of a very speciﬁc tool- or task-oriented procedure involving a

three-component structure, although this structure has never been validated or

(1) the role of the teacher;

(2) the role of feedback;

This was however limited to the construct of self-directed language learning.

Attitude towards CAL (computer-assisted learning)

Attitude towards FLL (foreign language learning)

Aim of this study

obtained by a four-step validation procedure, are described separately.

(1) bachelors students

There were no predetermined exclusion criteria, so all 240 participants that

Mother language Second language Third language

Table 2. Frequencies (N), percentages (%) of usage of CALL-applications.

questionnaire, a set of 24 items was oﬀered to measure attitude towards computer-

sample. Using principal axis factoring with orthogonal varimax rotation we

(1) w2 is the most frequently used goodness-of-ﬁt index. A statistically signiﬁcant

1993) is a ﬁt measure based on population error of approximation. It is

STEP 2: exploratory factor analysis

STEP 3: conﬁrmatory factor analysis

n w2 (df) CMIN/DF RMSEA GFI AGFI CFI NNFI

STEP 4: internal consistency and reliability of ﬁnal A-CALL questionnaire

STEP 2: exploratory factor analysis

STEP 3: Conﬁrmatory factor analysis

STEP 4: internal consistency and reliability of ﬁnal questionnaire

n w2 (df) CMIN/DF RMSEA GFI AGFI CFI NNFI

component was ﬁrst analyzed separately and subsequently in relation to the

Cognitive component – steps 1 to 4

were supposed to load on the factor ‘language learning beliefs’ loaded

Aﬀective-evaluative component – steps 1 to 4

research, no validation analyses were done and conclusions must be drawn

Behavioral component – steps 1 to 4

Full model of attitude towards language learning

Tables 8 and 9 summarize the descriptive statistics, the reliability (Cronbach’s a)

n w2 (df) CMIN/DF RMSEA GFI AGFI CFI NNFI

Figure 3b. A-FLL – Aﬀective/evaluative component: Standardized factor loadings of the

correlates with a high score on intrinsic motivation (r ¼ .28). Also, ‘intrinsic

Figure 3c. A-FLL – Behavioral/personality component: Standardized factor loadings of the

‘tolerance of ambiguity’ (r ¼ .45) and ﬁnally, tolerance of ambiguity has a positive

6. Tolerance of ambiguity (3 items) 14.80 (3.43) .75 _ _ .22*

data. Exploratory factor analysis revealed a ﬁve-factor solution that could be

(1) eﬀectiveness of CALL vs. non-CALL (four items);

We can roughly organize the subscales into the classical three-component

questionnaire comprises a cognitive, aﬀective/evaluative and behavioral compo-

degree of bias in the results, or in the structure of the questionnaire. Therefore, we

the exceedingly small number of research studies describing the development of

towards CALL and its relevant aspects by using rigorous methods.

IDIOMA-TIC. ALSIC. Apprentissage des Langues et Syste`mes d’Information et de

nonnormality. Structural Equation Modeling, 7, 557–595.

Appendix 1. The attitude towards CALL Questionnaire (A-CALL) – English version