System 57 (2016) 79e97

Contents lists available at ScienceDirect

System
journal homepage: www.elsevier.com/locate/system

Measuring latent listening strategies: Development and

validation of the EFL listening strategy inventory
John-Michael L. Nix*
National Taitung University, Center for General Education, Taitung, Taiwan, ROC

a r t i c l e i n f o a b s t r a c t

Article history: The present research identified the structure of latent trait listening strategy use by
Received 24 August 2015 developing and validating an inventory of EFL listening strategic knowledge. Results of
Received in revised form 1 February 2016 factor analysis and multi-dimensional item response theory (MIRT) analysis utilizing re-
Accepted 2 February 2016
sponses from 315 subjects identified and confirmed a two-dimensional structure
Available online 22 February 2016
composed of topedown and bottomeup processing strategy types. These constructs are
shown to have robust correlations with listening comprehension and also exhibit robust
Keywords:
inter-correlation, providing empirical support for the formal model of interactive pro-
Listening strategies
L2 listening
cessing. It was revealed that bottomeup strategies do not exert direct effects on listening
Multi-dimensional item response theory comprehension, but must be mediated by topedown strategies. Results are discussed in
Structural equation modeling light of extant strategy effect studies. Future research to test the generalizability of the
Construct validity inventory across cultural contexts and to gauge its feasibility in pedagogical applications is
suggested.
© 2016 Elsevier Ltd. All rights reserved.

1. Introduction

There is no dearth of research on both language learning strategies and listening strategies. Both strands of research have
witnessed steady activity for decades since Rebecca Oxford's Strategic Inventory for Language Learning (SILL: 1990) and J. R.
Anderson's cognitive model of language processing (1995) first gained wide traction within the field of second language
acquisition (SLA). Since then, the SILL has become the most widely used instrument for surveying language learning strategy
use (Grainger, 1997; Oxford & Burry-Stock, 1995). Review of SILL-based studies by Gao (2004) identifies 18 separate research
projects between 1989 and 2003, and more studies continually appear in print. Meanwhile, the prototypical cognitive pro-
cessing model of perception, parsing and utilization and similarly conceptualized variants such as Rost's (2005) stages of
decoding, comprehending and interpreting, have formed the theoretical bases of numerous L2 listening studies to date.
Weaving these two strands of research has yielded significant strides in assembling a patchwork understanding of L2
listening learning. Research of listening strategies relying on taxonomies derived from general language strategy frameworks
such as the SILL or O'Malley and Chamot's (1990) tripartite classification (cognitive, metacognitive, socio-affective) scheme
have attempted to catalog and classify learners' strategic repertoires or endeavored to modify strategic repertoires through
training regimens.
At the fore stands Vandergrift (1997) who proposed a bi-level interactive schematic of listening processing which melds
the now canonical tripartite scheme of O'Malley and Chamot (1990) with a three-stage cognitive process of listening. The

* National Taitung University, Center for General Education, 369 University Rd., Taitung, 95092, Taiwan, ROC. Tel.: þ886 89 517420.
E-mail address: John.michael.nix@gmail.com.

http://dx.doi.org/10.1016/j.system.2016.02.001
0346-251X/© 2016 Elsevier Ltd. All rights reserved.
80 J.-M.L. Nix / System 57 (2016) 79e97

three stages of listening are divided into “high” and “low” level processing, i.e., perception and parsing of discrete elements is
low level, while comprehension and utilization of parsed information is high level, and when applied across both processing
levels, function as a secondary facet to a simple, but comprehensive framework. Such a framework appears advantageous due
to its simplicity and versatility, and other researchers espouse the bi-level interactive model as well (Buck, 2001; Graham &
Macaro, 2008; Orii-Akita, 2014; Richards, 2008; Tsui & Fullilove, 1998).
Subsequent investigations of the effectiveness of strategic training interventions relying on ad hoc instrumentation
adapted from the SILL, the tripartite classification, or the bi-level processing model have yielded mixed results, leading
scholars to speculate that increased strategy use inevitably corresponds with improvements in listening ability, but the
manner of interaction is still ambiguous. The presence of mixed effects of strategy use indicates the construct is more complex
than a single continuum. If researchers are to advance a complete theory of L2 listening acquisition explicating the manner in
which strategy use yields or fails to yield improvements, they must first develop instrumentation to specifically and objec-
tively measure the trait variable of listening strategy use and identify the dimensional structure. Trait strategy use differs from
state strategy use as the former represents experiential memory of habitual strategic use, a stable long-term memory also
known as strategic knowledge (Vandergrift, Goh, Mareschal, & Tafagodhtari, 2006), while the latter is idiosyncratic with
specific language tasks and often investigated in treatment studies.
The extant taxonomies are extensive and tend toward increasing granularity, but the archetypical strategies implicit in the
psyches of learners must be known to understand how changes in this mindset affect learning. The lacuna in understanding
learners' strategy use exists not because of lack of strategic taxonomies, but because scholars lack insight into the learners'
mental classifications of these strategies. To wit, contemporary taxonomies represent researchers' classifications based on
discrete elemental commonalities of listening filtered through cognitive processing theories and although faithful to the
hypotheses of erudite researchers, they are not necessarily ordered along archetypical dimensions as perceived by the target
population, i.e. L2 learners who lack introspection of their own language acquisition processes. Hsiao and Oxford (2002)
evidence as such, noting incongruences between the five SILL categories and O'Malley and Chamot's (1990) tripartite clas-
sification. They assert Oxford's label of compensation strategy overlaps with O'Malley and Chamot's cognitive inference
strategy, due to the putative intent: listeners must cognitively infer elements when compensating for perception or parsing
gaps. When researchers' labels clash, which one essentially defines the construct? According to Reckase (2009), construct
dimensionality is “dependent on the sample of people that is of interest” (p. 3), therefore, L2 learners' apperceptions of trait
strategy use potentially follow continua dissimilar to researchers' cogitations.
Currently, research adopts strategic frameworks in an ad hoc fashion with insufficient regard to the psychometric qualities
of the instrumentation, often generalizing trait strategy use from state strategy use. The present study aims to build a
foundation for investigation of the trait variable by constructing a prototype listening strategy use inventory with demon-
strable psychometric qualities, meaning that item responses reflect variance in the learners' implicit construct of strategy use,
i.e., a stable trait. As such, it describes the creation and validation of a measurement model of L2 listening strategy use
intended for future research to incorporate into comprehensive structural models which empirically test L2 listening theories.

2. Literature review

2.1. The listening process

Numerous scholars (See Goh, 2000; Nunan, 2002; O'Malley, Chamot, & Küpper, 1989; Sun, 2002) have adopted a three-
stage process of listening loosely based on J.R. Anderson's process of perception, parsing and utilization which ground a
wide swathe of extant L2 listening research. However, the three-step listening comprehension process is limited to one-way
listening, while listening is frequently a two-way interaction; i.e., it is part of interpersonal communication involving real-
time, attentive listening to an interlocutor. Seminal listening comprehension and L2 proficiency research recognized the
need for a response phase with verbal or non-verbal cues to indicate the listener's comprehension of the message (Feyten,
1991). Rost (2005) echoes this idea by succinctly combining these notions, suggesting that the listening process consists of
three simultaneous and parallel processing phases: decoding, comprehension, and interpretation, accompanied by a final
listener response phase. In social constructivist strands of research, listener response is crucial for discourse and negotiation,
while in constructivist-based research, it is the sine-qua-non, as there can be no measurement of listening comprehension
without selection of answer options based on interpretation/utilization of the aural stimuli. Thus in the present study, listening
is hypothesized as a simultaneous mixture of high and low levels of aural processing, culminating in a mental response.

2.2. L2 listening strategies

Listening strategies are methods of managing personal mental and observable behavior to accomplish a listening task
(Goh & Taib, 2006; Graham & Macaro, 2008; Macaro, 2006; Richards, 2008). Listening strategies may be quasi-automatic like
general language strategies, and just as recalled strategies are presumed to illustrate certain aspects of language processing
(Chamot, 2005), listening strategies describe the mental operations involved in listening comprehension. Listening strategy
research requires recollection and consideration of strategy use, thereby uncovering parcels of the usually hidden listening
process. Research adopting verbal protocols (Cross, 2009; Graham & Macaro, 2008; Park, 2010) discovers state listening
strategy use, meanwhile large-scale survey research (Vandergrift, et al., 2006) assays trait listening strategy use.
 J.-M.L. Nix / System 57 (2016) 79e97 81

Seminal strategy research has posited that listening strategies are necessary for development of L2 listening proficiency
(Goh, 1998; Graham & Macaro, 2008; O'Malley et al., 1989; Oxford & Nyikos, 1989; Thompson & Rubin, 1996; Vandergrift,
1997, 1999) via the research assumptions, and current research literature reflects the belief in the utility of listening stra-
tegies. As Seigel (2011) notes, teaching listening strategies equates to teaching “how to listen” and is therefore “valuable”
(p.320).
Listening strategy research originated from L2 strategy research, embodied by descriptive studies seeking to observe the
strategic repertoires of focal groups such as gender (Bacon, 1992), high and low achievers (Teng & Chan, 2008), or effective
and ineffective L2 listeners (Park, 2010). Subsequent research emphasized treatment studies to effect growth of strategic
knowledge and competence (Cross, 2009; Graham & Macaro, 2008; Vandergrift, 2003). After decades of investigation, it was
found that simple, unitary quantities of strategy use do not consistently align with measurements of listening ability.

2.3. Applications of current strategy taxonomies

Park (2010) used a mixed-method analytic approach incorporating both the SILL (quantitative) and canonical tripartite
classification (qualitative) to analyze the strategic listening repertoires of Korean tertiary EFL students. Effective listeners
utilized significantly more strategies than ineffective listeners as delineated by the SILL. Meanwhile ineffective listeners
acquiesced and succumbed to distraction after judging the aural text as too difficult, whereas effective listeners maintained
attention even in the face of distraction by nearby construction work as seen by qualitative analysis utilizing the simpler
framework of metacognitive, cognitive, and socio-affective strategies (p. 7). Both groups reported the cognitive strategy of
prediction; however, effective listeners predicted accurately while ineffective listeners predicted erroneously, leading to
confusion further in the audio stream. Although both groups attempted to parse the audio stream, effective listeners com-
prehended the utterances at the sentence level, while ineffective listeners parsed at the word level and frequently obstructed
themselves by attempting to recall L1 translations. Thus, within the tripartite framework of strategies, effective and inef-
fective listeners both avail themselves of the same strategy types, e.g., meta-cognitive and cognitive, but the aspect discerning
the two groups is strategy implementation. To wit, the tripartite taxonomy of strategies was insufficient to identify participant
group distinctions necessitating an additional facet to the taxonomy of strategies (topedown versus bottomeup) to discern
effective and ineffective listeners' implementation of bottom-up (parsing) listening strategies.
Cross (2009) investigated the listening learning gains of Japanese EFL students trained to use four metacognitive, seven
cognitive and three socio-affective strategies adopted from Vandergrift's (1997) tripartite taxonomy, i.e., O'Malley and
Chamot's (1990) taxonomy adapted to listening. Significant gains were observed within both treatment and control
groups, while treatment group gains over the control were insignificant. Cross postulated that the control group's natural
proclivity for strategies similar to the treatment group and the tendency of some treatment group members to poorly utilize
strategies were causal factors. Also, Cross noted that individual difference variables such as beliefs and learner habits
modulated the efficacy of strategy utilization. A detail of Cross's explanation warrants reiteration: the participants had availed
of listening strategies from the outset, implying that L2 learners may develop notions of how to listen independently of and
prior to explicit instruction provided by educators or researchers.
In contrast to the ambiguity of Cross (2009), Graham and Macaro (2008) reported success in teaching listening strategies
to British learners of French as a second language (approximately third year of high school in the American system) and
success in boosting scores on free recall listening tests. Their study incorporated a limited number of general strategy clusters
which may be classified as bottom-up, cognitive and metacognitive processing. The integration of strategies into cluster
groups plus self-efficacy training in the intervention, teaching participants effective orchestration of the intervention tech-
niques, are key features to which the researchers attribute success.
The three studies detailed herein are representative in that listening strategy based studies narrow the range of strategic
repertoires to isolate and identify efficacious aspects, constituting a fundamental flaw of variable analytic research decried by
Bodie (2009), as study designs inherently focused on particulars preclude surveys of representative types of listening stra-
tegies and their relationships with listening outcomes. Essentially, state listening strategies are used in lieu of traits, rendering
it impossible to generalize learner tendencies for strategy use and concomitant effects on trait listening ability (proficiency).
Accordingly, that listening strategy use begets listening performance is intermittently witnessed; moreover, the strength of
this relationship remains unknown.
Numerous contexts for listening comprehension to occur both inside and outside of the classroom exist, which potentially
expands the scope of learners' implicit listening strategy repertoires (i.e. “Strategic knowledge”: Vandergrift, 2011) beyond
the scope of contemporary research designs. Motivated EFL students often listen to music with English lyrics or study
independently with commercially available material of their choice (O'Bryan & Hegelheimer, 2007). Inside the classroom, the
nature of academic listening also varies considerably, for example, when presented as textbook exercises, or as warm-ups for
language productive tasks, like discussion. During discussion tasks of communicative language teaching (CLT) regimens, the
learner is listening yet again, but inclusion of a live interlocutor offers chances to incorporate socio-affective strategies which
were excluded from teacher-fronted exercises. Finally, L2 learners may undergo speeded listening in the form of examina-
tions. Therefore, the listening strategy inventory in the present study must expand the breadth of strategic repertoires that
typically appear in intervention or “effect” studies to encompass multiple facets of strategies representative of L2 listening.
Vandergrift et al. (2006) provides an appropriate springboard for constructing a psychometrically based inventory of L2
listening strategy use with the first psychometrically tested inventory specifically applicable to trait L2 listening research, the
82 J.-M.L. Nix / System 57 (2016) 79e97

Metacognitive Awareness of Listening Questionnaire (MALQ), designed to assay L2 learners' metacognitive awareness of
target language listening. The imperative for its creation derived from research asserting the role of metacognition in suc-
cessfully utilizing listening strategies. As described in Vandergrift et al. (2006), metacognition encompasses three facets of
knowledge: person knowledge, task knowledge and strategy knowledge, wherein person knowledge describes learners' self-
efficacy beliefs, task knowledge describes learners' awareness of the listening process and factors influencing the processing of
aural input, and strategy knowledge describes learners’ knowledge of extant strategies and their utilization, the precise
construct of interest in the present study. Also, as described in Vandergrift et al., concepts of self-regulation were aggregated
into the MALQ, resulting in a five-factor oblique structure of listening metacognition, a formal concept not embodied by the
MALQ, but an epiphenomenon of the covariances among the factors that emerged from the confluence of items tied to self-
efficacy, self-regulation and knowledge of listening situations. Only the items comprising strategic knowledge are amenable
to the present study under the proviso that transferal from the context of French as a foreign language does not impact
psychometric quality of measurement along continua of the latent trait. Given that dimensionality is contingent on the
population in question, its structure may not be assumed from studies in other contexts. Moreover, the MALQ items comprise
only two strategy use factors aligning with notions of cognitive and metacognitive strategies at the expense of socio-affective
types.
The efforts of Vandergrift et al. (2006) deserve credit as a solid foundation for construction of a novel listening strategy
inventory, as the MALQ describes well-documented strategies such as inferencing, planning, and monitoring. The key issue is
to construct a strategy inventory with sufficient breadth to representatively reflect the strategies extant in the EFL learner
population, while simultaneously avoiding confoundment with self-efficacy, foreign language anxiety, and self-regulation.

3. The present study

Prior research has relied on verbal protocols to elicit narrow repertoires of task-specific strategies for investigation.
However, the present study proposes construction of a novel listening strategy inventory to assay EFL learners' knowledge of
extant strategies, a construct theorized as a trait variable, stable across listening contexts, as knowledge is stored in long-term
memory. This notion of the trait strategic variable contrasts with state strategy use dependent on specific listening tasks
which may only require a subset of an individual's known strategic repertoire. Based on review of listening strategy research,
the present researcher proposes several guidelines for the construction of said trait listening strategy inventories.

3.1. Local independence of construct

Each indicator must be sufficiently differentiated from other indicators to preclude collinearity effects in subsequent
analyses. That is, the wording of items must have sufficient specificity while simultaneously possessing a degree of generality
€ rnyei (2003)
to preserve the trait characteristic of the latent variable. Although notable scholars, e.g., Brown (2001) and Do
have proposed general guidelines for constructing questionnaire inventories, emphasizing panelist content review for
removal of linguistic biases and double-barreled items, etc., no guidelines exist for specifically phrasing items to preclude
statistical local dependence, therefore, a series of IRT analyses were conducted during the construction of the listening
strategic inventories to identify and remove items exhibiting poor measurement. Prototype inventories were expected to be
pared during piloting phases as statistically redundant or poorly constructed items were removed.

3.2. Representativeness of L2 listening contexts

Inventory items should generally reflect situations where L2 listening occurs, without specifying particular instantiations
of listening experience, which would tap a state variable. The items should therefore describe habitual listening practice and
listening use.

3.3. Canonical classification

Inventory items should reflect the trichotomous structure of metacognitive, cognitive, socio-affective language learning
strategies. This structure is advantageous as it is familiar to researchers, parsimonious, and applicable to a range of possible
listening situations.

3.4. Bifurcated classification

Inventory items should reflect the bi-level structure of mental processing simultaneous with the trichotomous classifi-
cation of listening strategies. Observations by Buck (2001) and Vandergrift (1997) that a common underlying dichotomy
exists in taxonomies of listening skills and strategies are astute, intuitive, and amenable to discovery by factor analytic and
statistical modeling techniques.
 J.-M.L. Nix / System 57 (2016) 79e97 83

3.5. Research question

Using these guidelines, a questionnaire called the EFL Listening Strategy Inventory (ELLSI) was constructed to estimate the
strength of the latent trait, listening strategy use, among the population of Taiwanese EFL learners. Development and vali-
dation of a novel psychometric scale entails exploration of the construct's dimensionality, yielding a baseline model followed
by confirmatory analyses which compare alternatives to the hypothesized baseline. Finally, the external validity and
explanatory power of the ELLSI is assessed by correlation with a listening test in a parsimonious structural equation model
(SEM). The following sections detail the development and validation to address the question: What is the best-fitting model
to describe the L2 listening strategy use of EFL learners in Taiwan?

3.6. Constructing the ELLSI

Extant language learning strategy frameworks comprise conflicting typologies describing strategies with multiple cate-
gory labels. Therefore, the objective of the first stage of analyses is to determine the empirically based latent dimensional
structure, that is, to identify EFL learners' actual strategy classification, as opposed to researcher conjectures. Fig. 1 provides a
comprehensive overview of the research-based theoretical facets informing the compilation of strategies queried in ELLSI.
Fig. 1 shows a hierarchy of four facets of listening that influence apperception of an indicator. This structure is utilized
solely to assist in compiling a comprehensive set of ELLSI indicators, and does not constitute any structural assumptions to the
ELLSI measurement model. The superordinate facet of setting describes the situations under which EFL listening may occur,
ultimately affecting the probability of employing a given strategy. The interaction/conversation setting describes listening
occurring with synchronous communication in the channels of face-to-face, telephony/web-chats and video-assisted web-
chatting. The self-study setting encompasses listening when individuals engage in EFL listening activities independently
outside of school settings. Nowadays, individuals may VoIP (voice over Internet protocol) chat with native speakers over
social networking applications, and accordingly self-study shares channels of two-way listening in common with conver-
sation. However, as self-study may be conducted solo with one-way listening, audioevisual media is included as a listening
channel. Finally, academic listening encompasses the activities typically found in classroom settings such as textbook
listening exercises or examinations, and relies exclusively on the channel of audioevisual media. Whether exercise or ex-
amination, this listening is one-way listening where the media playback is controlled by the instructor. These setting and

Fig. 1. Overview of theoretical facets for construction of ELLSI.

84 J.-M.L. Nix / System 57 (2016) 79e97

channel considerations affect the potential range and variety of listening strategies activated during a given listening
experience.
For example, ELLSI indicators of the cognitive strategy, inferencing, can operate at different levels of linguistic processing,
the most subordinate facet. Consider the following items:

a. “I guess the speaker's attitude toward the topic of discussion by noticing redundant words or phrases with similar
meaning.”
b. “I guess the meaning of unknown words by noticing the speaker's tone of voice.”

Item a implies topedown/bottomeup interaction where attention to decoded words and phrasal units yields identifi-
cation of implied aspects of the message, meanwhile item b is confined to bottom-up processing only, where the listener's
objective is merely to fill single-word gaps in the aural input, yet both processing levels constitute “inferencing” cognitive
listening strategies. Consider, also, the following indicators of the cognitive strategy, grouping:

c. “When listening to a difficult passage, I identify chunks of words, or phrases, rather than single words.”
d. “When conversing in English, I identify chunks of words, or phrases, rather than single words.”

Both items indicate a propensity for higher-level processing wherein larger informational units are decoded and compre-
hended, but item c is situated in either the self-study or academic settings, denoting the channel of audioevisual media, while
item d is situated in the interaction setting denoting synchronous, two-way channels of listening. In this manner, descriptors
found in Vandergrift (1997) and Buck (2001) were adapted via application of the facets depicted in Fig. 1 to create a pool of
strategic knowledge items with simultaneous phrasing commonalities and conceptual distinctions.
A Chinese version was constructed based on two rounds of translation by separate translators who are experienced junior
high school EFL teachers in Taiwan. A third Taiwanese school teacher reviewed both translations and arbitrated the wording
to create a final Chinese language rendition administered to participants. Precautions against theoretically vague or arcane
terminology as advised by Brown (2001) and Do €rnyei (2003) were applied during this stage of item construction.
The initial item pool retained a large number of similarly worded items, sometimes differing by only one facet of listening
experience, which poses empirical and practical risks to measurement despite the advantage of the comprehensive theo-
retical scope. Similar items appearing in proximity present a chance of multi-collinearity in the item response matrix exists
due to respondents' failure to notice wording distinctions, thus responding haphazardly. Distal placement of similar items
fails to redress the issue, as respondents would assume careless redundancy on the form, increasing risk of mischievous
response patterns. Hence, an inevitable trade-off exists between scope of the inventory and local dependence of items, which
necessitates a systematic method of trimming the inventory during piloting before use in principal data collection.

3.7. Study design

The present study consists of a pilot phase and main study phase. During the pilot study the full item pool of strategic
knowledge indicators was presented to a sample of junior- and senior-high EFL learners. Unidimensional item response
theory (IRT) analysis was used to trim the ELLSI inventory to a length amenable to incorporation in large-scale cross-sectional
studies, which utilize multiple, simultaneous instrumentation. Based on analysis described in subsequent sections, the ELLSI
inventory was trimmed to 23 items which were then presented to a panel of tertiary level EFL students in Taipei, Taiwan as a
review of face validity because the items should be generalizable to adult and young adult Taiwanese EFL learners. The
subsequent main study utilizing university undergraduate subjects served as cross-validation of the generalizability
assumption. The panelists did not express dissatisfaction with the Chinese wording or content of the items; all items were
perceived as clearly worded and devoid of theoretical jargon.

3.8. Use of IRT analysis

Item response theory (IRT) models, particularly two-parameter (2PL) variants, constitute the standard analytic technique
for latent trait identification in social sciences and have numerous applications for psychometrics besides testing (Edelen &
Reeve, 2007; Fraley, Waller, & Brennan, 2000; Nix & Tseng, 2014). The main advantage of IRT over item factor analysis (IFA)
lies in retaining the categorical probabilities of the Likert response data, rather than assuming a linear scale in the participant
responses (Hambleton, Swaminathan, & Rogers, 1991; Kline, 2005; Maydeu-Olivares, Cai, & Hernandez, 2011).
As the final ELLSI instrument was intended to be composed of reflexive indicators, not formative indicators, in keeping
with the fundamental measurement theory embodied by SEM methodologies (see Blunch, 2008 regarding the distinction
between principal component analysis and IFA), the finalized set of indicators on the questionnaire should not form the
construct of listening strategic knowledge. Rather, the indicators should be representative of the latent construct and elicit
endorsement from individuals who possess measurable quantities of the latent trait, thereby universally reflecting the
presence of the latent trait within the psyches of the respondents.
 J.-M.L. Nix / System 57 (2016) 79e97 85

IRT analysis of pilot data from junior- and senior-high aged EFL learners was presumed to identify the indicators with
the most robust and universal reflective power, as young learners have demonstrated awareness of metacognitive and
cognitive strategies for listening (Goh & Taib, 2006; Vandergrift, 2002) and are capable of strategic learning behavior to
effect academic achievement in general (Flavell, Miller, & Miller, 1993). Moreover, the implicit knowledge that they possess
is relatively nascent and therefore approximates the latent strategic archetypes, the objectives of investigation. Thus, items
yielding high discriminations (equivalent to item factor loadings) would reflect variance in a core, underlying trait common
to all respondents when applied to a more mature population with diverse learning experiences. The versions of the ELLSI
instrument used in piloting and main study phases also included demographic items querying gender, age and years of
formal schooling.

4. Pilot study

Table 1 lists the original 68 indicators in English, with item descriptive statistics, the Chinese translation of which was
piloted with 404 junior- and senior-high Taiwanese EFL students, aged 13e17 years with 7.76 years of formal EFL study on
average. The descriptive characteristics in Table 1 show uniformity in item responses, although several items exhibit notable
skewness and kurtosis, providing preliminary indication of the need to trim suboptimal indicators. These descriptive mea-
sures of normality indicate that inter-item dependence potentially degrades measurement reliability. Additionally, the high
alpha estimate (a > 0.90) suggests statistical redundancy in the inventory (Tavakol & Dennick, 2011). However, robust poly-
serial item-total correlations indicate that items generally reflect the construct despite deviations from normality.
Classical reliability indices, e.g., alpha, cannot discern actual measurement distortion due to inherent biases caused by
lengthy surveys (Cortina, 1993; Schmitt, 1996; Tavakol & Dennick, 2011). From both practical and empirical standpoints, the
complete item pool required trimming with non-classical methods to create a compact and reliable instrument. The response
set next underwent IRT (2PL) analysis with Mplus 7 using a unidimensional model to extract items explaining the greatest
proportion of the construct while minimizing unique variances. Mplus 7 output returns standardized loadings (normal ogive
conversions) in addition to logit scores for items (Muthe n, 2004), allowing researchers to calculate average variance extracted
(AVE) from item estimates. Consistent, high discriminations assist convergent validity of the resultant scale, and 0.50 AVE is
sufficient statistical power to warrant retention of an item (Blunch, 2008; Kline, 2005). The present researchers adopted a
more stringent cut-off value of 0.60 (standardized loading >0.775) because the objective is to retain the items with maximum
potential for latent trait measurement, i.e., maximum effect size. After IRT 2PL analysis, 23 items with sufficient psychometric
characteristics were retained as shown in Table 2.

5. Main study

5.1. Participants and procedures

The main study participants numbered 315 (175 female, 140 male) undergraduates of mixed major of study from a public
university in eastern Taiwan. English is a compulsory subject in Taiwanese schools from the third grade of primary school
(Chang, 2008; Chen, 2006), so these participants had a minimum of eight years formal EFL learning experience.
The 23-item inventory was administered to participants, followed by a short listening comprehension test (10 min), the
scores of which were used to correlate the listening strategy constructs with listening ability for external validation. No prior
listening task was administered to participants prior to the questionnaire as this would confound responses with state
strategy use variables, that is, the participants would bias their responses toward the strategies applied to the most recent
listening task, whereas the ELLSI instrument aims to assay experiential knowledge based on the cumulative English listening
learning experiences of the population. Correlating ELLSI responses with listening test scores creates a parsimonious SEM
whereby the latent trait strategy use variable is presumed antecedent to listening performance, a structure which follows
from prevailing assumptions in listening strategic research. The test instrument consisted of 22 multiple-choice, dichoto-
mously scored stimulus-response listening items modeled after TOEIC and the Taiwanese General English Proficiency Test
(GEPT), a familiar format to the participants.

5.2. Data analysis

Data analysis followed the two-step SEM procedure outlined in Kline (2005) wherein measurement models are first
specified and identified prior to incorporation into a structural model. Analysis at this stage also used IRT methodologies to
preserve the categorical nature of the observed variables. Interested readers can consult Reckase (2009) and Wirth and
Edwards (2007) for a complete history of multidimensional IRT (MIRT) and proofs of its close mathematical identity to
IFA. IRT measurement models essentially constitute re-scaled categorical CFA models which then comprise the final SEM.
Estimation of categorical indicator and latent variables in SEM uses weighted least squares means and variance adjusted
(WLSMV) due to inclusion of disparate scales in the dichotomous test items and polytomous questionnaire items (Beauducel
& Herzberg, 2006; Muthe n, 2004; Wirth & Edwards, 2007).
86 J.-M.L. Nix / System 57 (2016) 79e97

Table 1
Classical reliabilities and statistics of original ELLSI item pool at initial trialing.

Items M SD Skew Kurt. Item-total r a if item

deleted
1) I know what information I need to gather from a conversation in English. 2.16 1.03 0.11 0.55 0.75 0.9841
2) I know why I want to have a conversation with someone in English. 2.17 1.10 0.16 0.62 0.73 0.9842
3) I mentally gauge how much speech I don't understand while I am having an English 2.71 1.05 0.50 0.39 0.44 0.9845
conversation.
4) I mentally gauge how much speech I understand while I am having an English conversation. 2.72 1.01 0.51 0.27 0.54 0.9844
5) I mentally translate what I hear while I am participating in the conversation. 2.44 1.24 0.34 0.85 0.74 0.9842
6) I mentally gauge how much speech I understood after having an English conversation. 2.45 1.03 0.36 0.28 0.75 0.9842
7) I mentally gauge how much speech I didn't understand after having an English conversation. 2.62 1.02 0.47 0.19 0.61 0.9843
8) I make a mental or written summary of what I hear after the conversation. 2.18 1.05 0.24 0.53 0.76 0.9841
9) I translate what I hear in the conversation after the conversation is over. 2.11 1.14 0.10 0.77 0.77 0.9841
10) I mentally rehearse or visualize the English conversation that I am about to have. 1.77 1.25 0.14 0.94 0.73 0.9842
11) I try to relax before having a conversation in English. 2.06 1.19 0.11 0.74 0.70 0.9842
12) I pay attention to the main points of the conversation in English to get a general under- 2.40 1.12 0.30 0.66 0.81 0.9841
standing of what is said.
13) I pay attention to the details of the conversation in English. 2.04 1.06 0.04 0.52 0.73 0.9842
14) When taking tests, if questions are presented before the listening passage, I use them to guess 2.43 1.07 0.32 0.45 0.67 0.9842
the content and meaning of what I will hear.
15) When taking tests, if questions are presented after the listening passage, I use the test in- 2.46 1.06 0.35 0.31 0.70 0.9842
structions to guess the content and meaning of what I will hear.
16) When taking tests, I use words and expressions that I know from some questions to help me 2.77 1.05 0.48 0.55 0.63 0.9843
guess unknown parts of later questions.
17) I use a dictionary to help me understand when having an online English conversation. 1.50 1.19 0.30 0.87 0.65 0.9843
18) I take notes to help me understand when having an online English conversation. 1.44 1.14 0.32 0.75 0.63 0.9843
19) I listen to the other person's speech to determine if he/she has understood me correctly. 2.13 1.02 0.14 0.34 0.81 0.9841
20) I listen to the other person's speech to determine if he/she has misunderstood me. 2.14 0.99 0.23 0.21 0.79 0.9841
21) If allowed, I ask a friend for help when listening to difficult passages. 2.53 1.08 0.50 0.30 0.69 0.9842
22) If allowed, I take notes when listening to difficult passages. 1.91 1.15 0.12 0.78 0.71 0.9842
23) If allowed, I repeat what I hear so that I can become familiar with the words or expressions. 2.29 1.07 0.31 0.40 0.75 0.9841
24) When studying outside class, I listen together with friends to figure out the meaning of the 1.77 1.13 0.05 0.75 0.70 0.9842
passages.
25) When studying outside of class, I pay attention to my feelings about the listening passage. 1.87 1.11 0.08 0.57 0.79 0.9841
26) When studying outside of class, I make sure to choose listening passages/materials that I like. 1.86 1.15 0.04 0.73 0.81 0.9841
27) When listening to a difficult passage outside of class, I group words and expressions 1.77 1.04 0.12 0.42 0.80 0.9841
together based on common features.
28) Before a listening task in class, I ask the teacher questions about the passage. 1.54 1.07 0.36 0.41 0.64 0.9843
29) When listening in class, if the teacher asks the students to work together, I actively coop- 2.20 1.04 0.01 0.55 0.62 0.9843
erate with my groupmates.
30) When listening to a passage in class, I pay attention to my feelings about the passage. 2.04 1.03 0.09 0.33 0.77 0.9841
31) When listening in class, if the teacher pauses the recording, I make sure to ask questions 1.32 1.02 0.40 0.40 0.61 0.9843
about the passage.
32) After a listening task in class, I ask the teacher questions about the passage. 1.45 1.07 0.40 0.39 0.67 0.9842
33) I guess the meaning of unknown words by linking them to known words. 2.14 1.14 0.17 0.72 0.71 0.9842
34) I guess the meaning of unknown words or expressions by noticing redundant words or 2.28 1.07 0.35 0.33 0.77 0.9841
phrases with similar meaning.
35) I guess the meaning of unknown words by noticing the speaker's tone of voice. 2.05 1.06 0.14 0.36 0.78 0.9841
36) I guess the meaning of unknown words by noticing the gestures, actions, or facial ex- 2.23 1.09 0.25 0.50 0.79 0.9841
pressions of the speaker.
37) I guess the speaker's attitude toward the topic of discussion by noticing the intonation or 2.13 1.03 0.23 0.40 0.74 0.9842
stress on words.
38) I guess the speaker's attitude toward the topic of discussion by noticing redundant words or 2.07 0.96 0.16 0.20 0.83 0.9841
phrases with similar meaning.
39) I guess the speaker's intentions by noticing the gestures, actions, or facial expressions of the 2.27 1.03 0.24 0.40 0.78 0.9841
speaker.
40) I judge how well I was able to understand the other person's speech. 2.26 1.00 0.38 0.11 0.77 0.9841
41) I pay attention when the speaker communicates new or important information by noticing 2.29 1.03 0.38 0.27 0.83 0.9840
the intonation or stress on words.
42) I identify the problems I had in understanding the other person's speech. 2.10 1.03 0.08 0.35 0.62 0.9843
43) I ask the speaker questions to clarify what I don't understand. 2.12 1.06 0.24 0.42 0.74 0.9842
44) I ask the speaker to repeat something that I don't understand. 2.23 1.11 0.28 0.48 0.72 0.9842
45) I use personal experience to understand the speaker's meaning and intentions. 2.17 0.99 0.32 0.14 0.83 0.9841
46) I use my knowledge of the world to understand the speaker's meaning and intentions. 2.16 0.98 0.36 0.10 0.81 0.9841
47) I use my knowledge learned from school to understand the speaker's meaning and 2.29 0.99 0.49 0.01 0.85 0.9840
intentions.
48) I mentally visualize meanings or use mental imagery to understand the speaker's meaning. 2.15 1.00 0.27 0.28 0.73 0.9842
49) I set achievable comprehension goals for myself before listening to English, like “under- 1.77 1.04 0.02 0.64 0.63 0.9843
standing 50% is OK”.
50) I pay attention to English words or expressions that are similar to Chinese words or 2.15 1.00 0.10 0.35 0.77 0.9841
expressions.
51) I figure out the relationship between events when listening to a passage. 2.13 0.99 0.21 0.46 0.84 0.9840
 J.-M.L. Nix / System 57 (2016) 79e97 87

Table 1 (continued )

Items M SD Skew Kurt. Item-total r a if item

deleted
52) I make plans to strengthen weak areas of listening comprehension in the future. 1.80 1.09 0.14 0.41 0.72 0.9842
53) When listening to CD texts, I use the title to guess the content or main idea of what I will 2.23 1.10 0.39 0.47 0.79 0.9841
hear.
54) I mentally prepare to listen by reviewing what I know and don't know about the topic. 2.13 1.02 0.09 0.33 0.78 0.9841
55) I create my own endings to a story, and then listen to compare my ending with the actual 1.85 1.08 0.00 0.65 0.59 0.9843
one.
56) I use a dictionary to help me understand when listening to difficult passages. 1.82 1.21 0.10 0.90 0.71 0.9842
57) I tell myself that I can complete the listening task. 2.06 1.08 0.08 0.45 0.74 0.9841
58) I draw pictures to help visualize the meaning when listening to difficult passages. 1.62 1.15 0.20 0.78 0.62 0.9843
59) When conversing in English, I identify chunks of words, or phrases, rather than single words 1.90 1.06 0.01 0.46 0.77 0.9841
that the other person says.
60) When listening to a difficult passage, I identify chunks of words, or phrases, rather than 1.87 1.06 0.05 0.52 0.78 0.9841
single words.
61) When taking notes on a listening passage, I write down key words that identify topics or 1.94 1.06 0.01 0.52 0.68 0.9842
main ideas.
62) I use knowledge of English grammar rules to help me understand difficult parts of a passage. 2.23 1.09 0.18 0.52 0.75 0.9841
63) I use knowledge of English root words and affixes to help me figure out difficult words that I 2.06 1.03 0.13 0.48 0.72 0.9842
hear.
64) When I can't figure out a word that I hear, I substitute other words that I know to see if I can 2.10 1.05 0.14 0.44 0.73 0.9842
catch the meaning of the utterance.
65) When listening to a passage, I predict the ending or outcome from the events described. 2.07 1.04 0.21 0.36 0.76 0.9841
66) I use knowledge of rhyme and rhythm to help me figure out words when listening to En- 2.02 1.10 0.04 0.71 0.68 0.9842
glish songs.
67) When listening to a passage by myself, I pause to ask myself questions about difficult parts, 1.89 1.11 0.04 0.64 0.70 0.9842
and then replay the recording to answer the questions.
68) I use knowledge of English stress and intonation to help me figure out words spoken 2.17 1.07 0.14 0.58 0.77 0.9841
unclearly.

Note. Cronbach's a for the scale is .9844 (95% CI ¼ 0.9822, 0.9865). These statistics are based on trialing with junior- and senior high EFL learners to identify
optimal indicators for incorporation into the current ELLSI instrument.

Table 2
ELLSI items used in present study.

Item Std R2
discrimination
1 I pay attention to the main points of the conversation in English to get a general understanding of what is said. 0.812 .66
2 I listen to the other person's speech to determine if he/she has understood me correctly. 0.815 .66
3 I guess the meaning of unknown words or expressions by noticing redundant words or phrases with 0.786 .62
similar meaning.
4 I guess the meaning of unknown words by noticing the speaker's tone of voice. 0.789 .62
5 I guess the meaning of unknown words by noticing the gestures, actions, or facial expressions of the speaker. 0.801 .64
6 I guess the speaker's attitude toward the topic of discussion by noticing redundant words or phrases 0.835 .70
with similar meaning.
7 I guess the speaker's intentions by noticing the gestures, actions, or facial expressions of the speaker. 0.792 .63
8 I judge how well I was able to understand the other person's speech. 0.795 .63
9 I pay attention when the speaker communicates new or important information by noticing the intonation 0.841 .71
or stress on words.
10 I use personal experience to understand the speaker's meaning and intentions. 0.836 .70
11 I use my knowledge of the world to understand the speaker's meaning and intentions. 0.819 .67
12 I use my knowledge learned from school to understand the speaker's meaning and intentions. 0.845 .71
13 I pay attention to English words or expressions that are similar to Chinese words or expressions. 0.778 .61
14 I figure out the relationship between events when listening to a passage. 0.847 .72
15 When listening to CD texts, I use the title to guess the content or main idea of what I will hear. 0.795 .63
16 When studying outside of class, I pay attention to my feelings about the listening passage. 0.795 .63
17 When studying outside of class, I make sure to choose listening passages/materials that I like. 0.804 .65
18 When listening to a difficult passage outside of class, I group words and expressions together based 0.793 .63
on common features.
19 When listening to a passage in class, I pay attention to my feelings about the passage. 0.775 .60
20 When conversing in English, I identify chunks of words, or phrases, rather than single words that the 0.777 .60
other person says.
21 When listening to a difficult passage, I identify chunks of words, or phrases, rather than single words. 0.785 .62
22 I mentally prepare to listen by reviewing what I know and don't know about the topic. 0.783 .61
23 I use knowledge of English stress and intonation to help me figure out words spoken unclearly. 0.781 .61

Note: Standardized discriminations are estimated from initial 2PL calibrations with junior- and senior-high EFL learners' responses.
88 J.-M.L. Nix / System 57 (2016) 79e97

5.3. ELLSI measurement model

Investigation of the responses began with parallel analysis (Horn, 1965), a type of reiterative exploratory factor analysis
(EFA), to test multidimensionality of the latent construct. Parallel analysis compares the factor structure of the observed data
with the factor structure derived from a large number of randomly simulated, normally distributed observations, with the
rationale being that significant factors underlying actual observations will yield eigenvalues higher than the average and 95th
percentile of 50 or more such random samples (Hayton, Allen, & Scarpello, 2004). Comparison with the 95th percentile
corresponds to a 95% confidence rate, or significant p-value of 0.05.
Parallel analysis results displayed in Fig. 2 suggests two possible latent trait factors within the observed responses dis-
cerned by eigenvalues greater than those calculated for the 95th percentile, as indicated by the point of intersection in the
scree plot; the observed sample yielded 7.819 and 1.708 while the 95th percentile of parallel analyses yielded 1.607 and 1.481.
The inter-factor correlation was robust and significant at 0.535 (p < 0.05). Thus, the significance of the extracted eigenvalues
and correlation suggests the ELLSI data to be multidimensional oblique.
The loadings for the two-factor structure (Table 3) shows that significant crossloading indicators exist among the 23 items,
posing empirical threats to the reliability of measurement and concomitant validity because these item responses cannot be
unambiguously attributed to a single dominant underlying trait, rendering a theoretical conundrum if retained, namely, what
does a response to this item mean? Accordingly, model identification necessitates removal of significant crossloaders, in-
clusive of both practical (loading > 0.30) and statistical (p < 0.05) significance. The resultant inventory is a 15-item scale
comprised of a minor subscale (five items) juxtaposed with a major subscale (ten items). Inspection of the item wordings
leads to interpretation of the two latent constructs as top-down and bottom-up processing which form the baseline mea-
surement model.
Four of the five items comprising the minor dimension are cognitive-type guessing strategies identifying linguistic and
paralinguistic elements such as unknown words (items two and four) and attitudes and intentions of speakers (items six and
seven, respectively) which imply bottom-up processing of input from discrete to integral units and are applicable to both live
interaction and recorded media. Item two is a metacognitive-type monitoring strategy exclusively situated in live interaction
where the listener infers interlocutor's comprehension of a previous turn by the listener. In this respect, the item shares the
objective of identifying the speaker's attitudes/intentions based on processing of discrete elements akin to items six and
seven.
Of the ten items comprising the major dimension, three items are cognitive types which express the utilization of non-
linguistic knowledge to assist in meaning construction (12, 22, and 23), suggesting application of schemas and conceptual
connections to assist aural processing. The strategic processing described in these three items is generalizable across channels
and settings. Three other items are meta-cognitive, monitoring types which express awareness of interest, suggesting that
integration of non-linguistic elements in processing extends to such broad intangibles as affect. The strategic processing
described in these three items is applicable to one-way listening situated in the classroom (19) and in independent study (16
and 17). Another group of three items are cognitive grouping types which express a proclivity to process integrated rather
than discrete elements of aural input (18, 20 and 21). These items are ambiguous as to the criteria listeners apply when
parsing the phrasal units aside from reliance on “common features”, which suggests relatively abstruse knowledge elements
are utilized in processing. Item 20 describes two-way listening in synchronous communication, while items 18 and 21 are
situated in one-way listening contexts. Finally, item one is a cognitive attention type strategy which expresses focus on higher
levels of processing akin to identifying the gist of the aural message. Taken together, these items may be interpreted as
indicating top-down processing.
Based on content review of the 15 items, it can be surmised that the tripartite strategy classification is indeed subsumed by
the overarching bi-level interactive processing framework. Moreover, the facets of one-way/two-way listening and class-
room/independent practice are also distributed across the two putative factors. Congruence with contemporary listening
processing theory stands as a positive prognosis of validity, but theory must be bolstered with empirical support, as de-
terminacy, the fundamental issue in identifying substantive statistical models, requires comparative testing. The identified
factor structure is clearly non-random, but remains unvalidated without model comparisons, given that the baseline model
represents only one possible solution to the observed data matrix. As Kline (2005) notes, models with superb fit may be
erroneous, but equivalent to alternatives which do fit the observed response pattern. Isemonger (2007) also commented
extensively on this methodological shortcoming in factor analyses in the domain of second language acquisition (SLA)
research.
Confirmatory MIRT was utilized to conduct model comparisons with a parsimonious unidimensional variant and a
second-order, hierarchical variant. These two structural configurations stand as equivalent models to the baseline model
because multidimensional models are nested within the parsimonious unidimensional version, and in the latter case, the
inter-correlation of the two dimensions may be caused by a general strategic use variable underlying the bottomeup/
topedown variables. These two possibilities must be tested by comparing model fit to the actual observed data. Indices of
model fit (2 Loglikelihood: -2LL, AIC, BIC, adjusted AIC) measuring degree of discrepancy to the observed data matrix allow
comparisons among baseline and alternative models, with lower values indicating superior fit. The fit indices from this
comparative MIRT analysis are shown in Table 4. Figs. 3 to 5 display the ELLSI indicators configured in the baseline and
alternative models with unstandardized loadings and standard errors. All loadings in baseline and alternative models reached
significance at p ¼ 0.05.
 J.-M.L. Nix / System 57 (2016) 79e97
Fig. 2. Scree plot of ELLSI parallel analysis with university subjects, n ¼ 315.

89
90 J.-M.L. Nix / System 57 (2016) 79e97

Table 3
ELLSI EFA Two-factor Solution for University Subjects, n ¼ 315.

Factors: 1 2
Eigenvalues: 7.819 1.708

Items Loadings
1 0.177 0.505*
2 0.377* 0.256*
3 0.336* 0.238*
4 0.683* 0.003
5 0.720* 0.053
6 0.515* 0.271*
7 0.574* 0.096
8 0.362* 0.363*
9 0.370* 0.357*
10 0.441* 0.316*
11 0.237* 0.365*
12 0.090 0.490*
13 0.234* 0.317*
14 0.287* 0.252*
15 0.264* 0.344*
16 0.038 0.503*
17 0.058 0.598*
18 0.127 0.737*
19 0.190* 0.705*
20 0.125 0.619*
21 0.004 0.658*
22 0.117 0.560*
23 0.047 0.546*

Note. Significant (p < 0.05) loadings marked with *; Primary loadings are in bold.

Table 4
Fit comparisons of baseline and alternative factor structures.

Model Parameter (df) -2LL AIC BIC Adj. BIC

1D 75 (45) 11 343.618 11 493.618 11 775.061 11 537.181
2D oblique 76 (44) 11 193.184 11 345.185 11 630.380 11 389.329
2nd order 75 (45) 11 202.614 11 352.613 11 634.056 11 396.177

As seen in Table 4, the unidimensional trait model yields the least acceptable fit across all the indices, showing the greatest
incongruency with observed data and confirming the parallel analysis findings that trait strategy use is multidimensional.
Although the hierarchical model, also being multidimensional, exhibits better fit than the unidimensional model, it none-
theless exhibits poor fit with respect to the baseline model. This finding demonstrates that three dimensions, a general
strategy use trait underlying the bottom-up and top-down traits, are also incongruent with the observed pattern of responses.
Thus, this stage of confirmatory MIRT identifies the oblique two-dimensional model as the best explanation for the apper-
ceptions of listening strategy use by the participants. Thus, bottom-up and top-down strategies are not dependent on a
general unitary strategic dimension, but rather exist as independent, correlated constructs (r ¼ 0.66, p < 0.05) that mutually
interact, representing an empirical reflection of the formal theory of bi-level interaction.

5.4. External validation of ELLSI

To measure the extent to which these dimensions of listening strategy use could account for variance in listening
comprehension ability, the baseline model was correlated with 22 manifest variables indicating a unitary listening
comprehension trait-ability. Multiple fit indices were used to evaluate the adequacy of the SEM. First, the c2 statistic was
checked for non-significance (p > 0.05) showing that the model does not significantly depart from actual observations. Also,
the c2/df ratio was calculated to account for the effect of large models and sample sizes which bias c2 towards spurious
significance (Bentler, 1990; Maydeu-Olivares et al., 2011). A value of less than 2.0 was taken to indicate good fit (Kline, 2005).
In addition, the Comparative Fit Index (CFI) and the TuckereLewis Index (TLI) greater than 0.90 were also construed as
indicative of good fit (Kline, 2005). Finally, the root mean-square error of approximation (RMSEA) a measure of badness of fit
was consulted as well. RMSEA assesses the amount of model residual error in relation to degrees of freedom, as complex
models yield incrementally good fit as they approach saturation, or zero degrees of freedom, due to mathematical artifacts
(Kline, 2005).
Additionally, a covariate of years of EFL study serves as an indicator of external validation. The literature noted that in-
dividuals may implicitly develop strategic knowledge independent of specific strategy training regimens (Cross, 2009).
 J.-M.L. Nix / System 57 (2016) 79e97 91

Fig. 3. Unidimensional alternative model for ELLSI indicators.

Therefore, an implied, directly proportional relationship exists between length of English study and strength of the listening
strategic traits. Moreover, it is a given assumption that amount of learning experience should have a directly proportional
relation with listening comprehension ability. Accordingly, the amount of EFL study queried in the demographic data section
was placed as covariate to listening ability and listening strategy traits to confirm these hypothetical relationships. The
resultant SEM is shown in Fig. 6 with standardized indicator and path loadings. All loadings of indicator variables to traits
were significant at p ¼ 0.05. Unstandardized path loadings and standard errors are shown in Table 5.
Model fit was good with the exception of c2 (c2 ¼ 1006.865, df ¼ 660, p ¼ 0.00, c2/df ¼ 1.53, CFI ¼ 0.922, TLI ¼ 0.917,
RMSEA ¼ 0.041), but given the positive indications by RMSEA and c2/df, significant difference with the observed data matrix
may be attributable to sample size and multiple parameters. Visual review of the path estimates among the traits and co-
variate provides significant insight on the interaction of listening strategic variables and listening comprehension. Bot-
tomeup strategies are remarkable in their lack of direct effect on listening ability as seen by the non-significant path
(r ¼ 0.08), while top-down strategies have a very strong direct effect on listening outcomes (r ¼ 0.56, R2 ¼ 0.31). This means
that bottom-up strategies exclusively exhibit indirect effects, mediated through top-down strategic processing, which yields a
positive indirect effect of r ¼ 0.35 (R2 ¼ 0.12). The robust interaction between top-down and bottom-up strategies are wit-
nessed again in the inter-correlation (r ¼ 0.62). Robust positive correlations between the length of EFL study and listening
ability (r ¼ 0.38), and among the two strategy use variables (Bottomeup: r ¼ 0.24; Top-down: r ¼ 0.32) corroborate the
validity of the respective constructs measured by ELLSI.

6. Discussion

6.1. Significance of findings

This article described the development and validation of an inventory of trait L2 listening strategy use via parallel analysis
and confirmatory MIRT analysis on two independent samples of EFL learners, finding that the formal theory of bi-level
interaction is witnessed in substantive empirical models. The robust and significant relationships between ELLSI traits of
bottomeup and topedown processing with test-observed listening performance corroborate the validity of the latent trait
measurement model. The present study stands apart from strategic treatment studies in that it identifies learners'
92 J.-M.L. Nix / System 57 (2016) 79e97

Fig. 4. Oblique, two-dimensional baseline model for ELLSI indicators.

spontaneously occurring latent trait interactions antecedent to listening test outcomes without interventions to shape these
processes. In this respect, the present modeling study represents a large-scale, naturalistic, empirical inquiry.
A dichotomous construct reflective of the formal bi-level processing theory wherein L2 listeners engage with aural input
by employing bottomeup and topedown strategies in an interactive manner was shown to explain significant and substantial
variance in listening ability, 12 percent and 31 percent, respectively, which contrasts with Vandergrift et al.'s (2006) results of
r ¼ 0.36 or 13 percent shared variance between the hypothesized model and listening outcomes. Vandergrift et al. (2006) did
not consider the possibility of multidimensionality in listening strategic knowledge when incorporating the construct into the
MALQ, and such misspecification may explain the low correlation with listening ability.
The respective strategy traits identified herein also exhibit differing relations with listening ability in timed testing sit-
uations: bottomeup has no noticeable effect or potentially negative effects if used exclusively, but has salient positive effects
when conjoined with topedown processing, which implies a non-compensatory relationship where both constructs exhibit
mutual dependence. Thus, while top-down is the dominant strategy type with direct effects on listening outcomes, it remains
strongly dependent (39 percent shared variance) on bottom-up processing to yield the facilitative effects, validating the
formal theory of extant literature.
Placed in the broader context of the listening strategic research, why certain variable analytic (treatment) studies fail to
yield positive results is now clear. If subjects are reconditioned by the training regimen to focus exclusively on bottom-up
processing, listening ability will not be noticeably enhanced.
In addition, the ELLSI measurement model possesses demonstrable psychometric qualities corroborated by comparative
testing against alternative models and within the broader SEM network of variables. The covariate of EFL study corroborates
the validity of the trait estimates, as traits vary in direct proportion to length of study, i.e., the longer learners study L2, the
greater their tendency to employ both bottom-up and top-down strategies. The items retained in the final ELLSI instrument
demonstrate acceptable content validity as well, spanning channels of listening (face-to-face, recorded media), settings
(interaction/conversation, independent study, academic), and strategy types (cognitive, metacognitive).
Moreover, the findings in the present study exhibit preliminary indications of generalizability beyond Taiwanese EFL
populations. Revisiting the studies mentioned as indicative of the mixed-effects of strategy training can illustrate the rele-
vance of constructs measured by ELLSI to research outside of Taiwanese tertiary education settings.
 J.-M.L. Nix / System 57 (2016) 79e97 93

Fig. 5. Second-order, hierarchical alternative model for ELLSI indicators.

The model implied relations described herein are congruent with the qualitative component of Park's (2010) study of
Korean learners. Park had witnessed that cognitive aspects of strategy use (guessing) were not crucial per se, but variability
along the topedown/bottomeup continuum were critical to guessing successfully.
In a similar manner, the problems encountered by Cross (2009) with Japanese EFL learners can now be re-examined.
Inspection of the strategy treatment regimen described in Cross's appendix reveals that there are minimal distinctions
across the strategies of “transfer”, “note-taking”, “reconstituting”, “detection”, “selective attention”, “self-management” and
“self-evaluation”, half of the 14-type strategy treatment. Of these seven strategies, reconstituting, the encouragement of
“constructing complete meaning-based interpretations from partial understanding” (p. 175) represents the quintessential
bottom-up processing strategy. Although these strategy treatments span putative cognitive and metacognitive types, Cross
notes that Japanese learners have a culturally mediated tendency to fixate on bottom-up processing, thereby rendering the
possibility of inadvertent focus on bottom-up strategies across the board. Accordingly, a weak and statistically insignificant
effect between strategy use and listening test outcomes was witnessed, just as the present SEM model implies.
Graham and Macaro's (2008) success in strategy training can also be viewed in light of the bi-level processing model.
Graham and Macaro noted from extant literature that metacognition, or “reflection on strategy use” (p. 754) plays a critical
role in successful strategy utilization, so they designed a training regimen to teach strategy clusters and encourage learners to
“see connections between the strategies used and learning outcomes” (p.756). Notably, their training regimen was designed
to give “equal prominence to both top-down and text-based strategies” (p.756), where “text-based” refers to audio texts, i.e.,
bottomeup processing. Thus, it may be conjectured that Graham and Macaro were essentially teaching participants, English
learners of French as a foreign language, to apply bi-level processing, an interaction between topedown and bottomeup that
was also witnessed in the present model.
Review of these studies shows that the formal bi-level processing model can be applied to the research frameworks, and
the present study shows that ELLSI represents a corresponding substantive model which measures learners' proclivities to
process interactively with topedown and bottomeup strategies. Therefore, it may be surmised that the next stage of listening
strategy research should follow a two-pronged approach to elaborate both theoretical and practical applications of the ELLSI
model.
94 J.-M.L. Nix / System 57 (2016) 79e97

Fig. 6. SEM with ELLSI and EFL study as antecedents to listening ability.

6.1.1. Theoretical implications

Future theoretical research should investigate the generalizability of the ELLSI instrument on non-Asian learner pop-
ulations to test the limits of its validity when applied to other cultural contexts. While consensus holds that the bi-level
processing model of listening is manifest across cultures and populations, it remains to be seen whether the ELLSI items
as presently worded can be readily translated to other languages and function accordingly with respect to those learners.
Additionally, subsequent research should place the dual constructs embodied by ELLSI into a comprehensive SEM network
with other relevant constructs. Considering that listening strategies are mental processing behaviors, there are numerous
latent learner variables which are suggested by the research literature to be closely related, e.g., motivation, learning beliefs,
and self-regulation. Integration within a larger structural model may elucidate the multivariate interactions which determine
L2 learners' likelihood to listen with comprehension.

6.1.2. Pedagogical implications

In addition to theoretical investigation, a more pragmatic vein of research should explore the feasibility of integrating the
ELLSI instrument in listening strategy training regimens to conduct self-assessment of strategic knowledge and to formulate

Table 5
Structural relations among latent traits and covariate.

Path Unstandardized SE Standardized R2

Bottom-up with top-down 0.622 0.039 0.622 0.387
Listen on bottom-up 0.100 0.092 0.077 0.006
Listen on top-down 0.719 0.102 0.560 0.314
Listen on EFL study 0.503 0.080 0.379 0.144
Bottom-up on EFL study 0.239 0.088 0.235 0.055
Top-down on EFL study 0.327 0.073 0.316 0.100

Note. Bolded italics denote insignificance at p < 0.05.

 J.-M.L. Nix / System 57 (2016) 79e97 95

approaches to strategy training. In both cases, the short length of the questionnaire and the parsimonious structure constitute
practical features for students and teachers alike.
Just as the SILL (Oxford, 1990) has become a staple component of action research to facilitate autonomy in L2 learners, the
ELLSI inventory can fulfill similar functions with respect to listening strategies per se. As the formal structure of listening
strategic knowledge embodied by the inventory is simple, consisting of only top-down and bottom-up strategies, it is
amenable to language learners who are less inclined toward theoretical introspection of language learning. That is to say,
learners and their teachers who use the instrument as part of a strategy-based approach to language learning can readily
discern a simple pattern of topedown or bottomeup proclivities, or a balance between both. Were a teacher to identify a
group of learners as predominantly bottom-up strategy users, one could then endeavor to habitualize their use of top-down
strategies to achieve balanced interaction (or vice-versa). The ELLSI instrument may be administered as pre- and post-
training measurements or as a series of formative assessments during a term of study to gauge growth in listening strategy
use.
With respect to formulation of strategy training regimens, the ELLSI items imply the adaptation of certain well-known L2
reading strategies for listening training. In this discussion, two robust ELLSI indicators serve as useful examples of extrap-
olating item descriptors into effective strategy training approaches for L2 educators.
Item 17 serves as a case of particular interest as it suggests the importance of learners' affect rather than cognitive aspects
of listening. Moreover, educators may be perplexed regarding the manner in which this item can be formalized as a useful
technique. As explained below, this item expresses the learners' willingness to reiteratively engage cognitively with an aural
text.
The present researcher suggests that self-assessment with ELLSI be used to stimulate a discussion on the utility/value of
the strategies in which the instructor explains that self-selection of preferred material amounts to a conscious decision to
repeatedly listen and possibly discover novel questions and answers.
In the case of textbook listening exercises, the ultimate consequence is temporal and cognitive restriction of listening
practice, as the learner mentally engages the input so far as question responses can be extracted, and the aural input is rarely
revisited once the goal of question response is achieved. In contrast, were the students to focus on a favorite scene in a target
language movie or favorite song, they would increase aural exposure through repetition, and each listening session increases
the potential to extract nuance and metaphor overlooked in previous sessions. Encouraging learners to follow the guidance in
item 17 is essentially teaching the value of “narrow listening” (Krashen, 1996; Vandergrift, 2004). Thus, learners possessing
self-awareness of inclinations to listen to a given passage at the outset exhibit potential to control the depth and scope of
subsequent cognitive engagement.
Item 22 is a rudimentarily worded paraphrase to render the notion of advance organizers comprehensible to language
students. Advance organization is a concept first appearing with respect to reading comprehension approaches (Ausubel,
1960) and later adapted to listening (Herron, Cole, York & Linden, 1998; Vandergrift, 2004). Advance organizers consist of
simple text summaries or visuals both of which serve the purpose of schema activation wherein learners mentally orchestrate
relevant background knowledge into an organized framework of expectations for impending aural information.
The present researcher suggests that strategy training regimens utilize advance organization in the form of mind- and
concept-mapping techniques (Davies, 2011), demonstrated by the instructor. The instructor first constructs a mind-map on
the board to record students' responses to warm-up questions; then the instructor should re-organize the mind-map into a
concept-map, and via guided questioning draw empty nodes to denote information which is missing and/or unknown.
Application of this method serves as a quasi-visual advance organizer operationalizing item 22 by cataloging the knowns and
unknowns. Over the course of a strategy training program the instructor should encourage students to apply this technique
when practicing listening skills independently. When follow-up self-assessment of strategy use is conducted with ELLSI, it
should be explicated that the now familiar mapping technique is an instantiation of item 22. In this manner, the adoption of
the ELLSI instrument facilitates the learning of listening strategies by L2 students, and the assessment of such by their
instructors.

6.2. Limitations

Although MIRT and SEM analyses provide powerful insights for investigation of formal theories, they possess limitations
similar to bivariate studies, namely, representativeness of sampling. Dimensionality of the construct is a reflection of the
characteristics of the population under study, and while the model fit indices and item loadings show a promising picture,
they remain internally consistent and potentially valid so far as Taiwanese tertiary level EFL learners are concerned. Repli-
cations with large samples across various cultural and educational settings are still requisite to ascertain repeated measures
reliability, transferability and validity.
Another area which requires further study for clarification is the interpretation of item loadings as exclusively indicating
bottomeup and topedown processing. Model fit measures of internal reliability presently show that items 18, 20 and 21,
describing grouping and chunking into phrasal units are classified as top-down based on empirically observed patterns of
association with other topedown indicators, yet these items are typically regarded as bottomeup in much of the literature.
This question is linked to the previous issue of repeated-measures reliability and replication in that further research with
multiple samples across various populations is necessary to confirm invariance of these indicatoretrait relations.
96 J.-M.L. Nix / System 57 (2016) 79e97

7. Conclusion

This article has described the development, validation, and pedagogical application of an inventory of trait listening
strategic use by L2 learners. ELLSI has shown practical significance in its correspondence with formal listening theory and
demonstrates a methodology for future researchers to apply SEM techniques for probing the empirical foundations of formal
listening theories. Additionally, ELLSI demonstrates robust psychometric properties when applied to the population from
which it was developed. The multivariate relations uncovered by the ELLSI model show that trait strategy use is two-
dimensional, and variance in bottom-up processing does not imply proportional variance in listening ability. Future
research should explore the generalizability of the ELLSI dimensions in other L2 settings. It is further suggested that sub-
sequent research investigate the feasibility of integrating ELLSI into listening strategy training regimens.

Acknowledgments

The author would like to thank the generous assistance of friends, colleagues and their students who willingly volunteered
their time to participate in this project. In addition, sincere gratitude is extended to the anonymous reviewers whose
invaluable comments assisted the final form of this manuscript. This research was entirely self-funded without assistance
from any funding agencies or grants.

References

Anderson, J. R. (1995). Cognitive psychology and its implications. New York: Freeman.
Ausubel, D. P. (1960). The use of advance organizers in the learning and retention of meaningful verbal material. Journal of Educational Psychology, 51,
267e272.
Bacon, S. M. (1992). The relationship between gender, comprehension, processing strategies, and cognitive and affective response in foreign language
listening. The Modern Language Journal, 76(2), 160e178.
Beauducel, A., & Herzberg, P. Y. (2006). On the performance of maximum likelihood versus means and variance adjusted weighted least squares estimation
in CFA. Structural Equation Modeling, 13(2), 186e203.
Bentler, P. M. (1990). Comparative fit indexes in structural models. Psychological Bulletin, 107(2), 238e246.
Blunch, N. J. (2008). Introduction to structural equation modelling using SPSS and AMOS. Sage.
Bodie, G. D. (2009). Evaluating listening theory: development and illustration of five criteria. The International Journal of Listening, 25, 81e103.
Brown, J. D. (2001). Using surveys in language programs. Cambridge, UK: Cambridge University Press.
Buck, G. (2001). Assessing listening. Cambridge University Press.
Chamot, A. U. (2005). Language learning strategy instruction: current issues and research. Annual Review of Applied Linguistics, 25(1), 112e130.
Chang, Y. F. (2008). Parents' attitudes toward the English education policy in Taiwan. Asia Pacific Education Review, 9(4), 423e435.
Chen, Y. M. (2006). EFL instruction and assessment with portfolios: a case study in Taiwan. Asian EFL Journal, 8(1), 69e96.
Cortina, J. M. (1993). What is coefficient alpha? An examination of theory and applications. Journal of Applied Psychology, 78(1), 98e104.
Cross, J. (2009). Effects of listening strategy instruction on news videotext comprehension. Language Teaching Research, 13(2), 151e176.
Davies, M. (2011). Concept mapping, mind mapping and argument mapping: what are the differences and do they matter? Higher Education, 62(3),
279e301.
Do€rnyei, Z. (2003). Questionnaires in second language research: Construction, administration and processing. Mahwah, NJ: Erlbaum.
Edelen, M. O., & Reeve, B. B. (2007). Applying item response theory (IRT) modeling to questionnaire development, evaluation, and refinement. Quality of Life
Research, 16, 5e18.
Feyten, C. M. (1991). The power of listening ability: an overlooked dimension in language acquisition. The Modern Language Journal, 75, 173e180.
Flavell, J. H., Miller, P. H., & Miller, S. A. (1993). Cognitive development (3rd ed.). Englewood Cliffs, NJ: Prentice Hall.
Fraley, R. C., Waller, N. G., & Brennan, K. A. (2000). An item response theory analysis of self-report measures of adult attachment. Journal of Personality and
Social Psychology, 78(2), 350e365.
Gao, X. (2004). A critical review of questionnaire use in learner strategy research. Prospect, 19(3), 3e14.
Goh, C. C. M. (1998). How ESL learners with different listening abilities use comprehension strategies and tactics. Language Teaching Research, 2(2), 124e147.
Goh, C. C. M. (2000). A cognitive perspective on language learners' listening comprehension problems. System, 28, 55e75.
Goh, C. C. M., & Taib, Y. (2006). Metacognitive instruction for listening in young learners. ELT Journal, 60(3), 222e232.
Graham, S., & Macaro, E. (2008). Strategy instruction in listening for lower-intermediate learners of French. Language Learning, 58(4), 747e783.
Grainger, P. R. (1997). Language-learning strategies for learners of Japanese: investigating ethnicity. Foreign Language Annals, 30(3), 378e385.
Hambleton, R. K., Swaminathan, H., & Rogers, H. J. (1991). Fundamentals of item response theory. Sage Publications, Inc.
Hayton, J. C., Allen, D. G., & Scarpello, V. (2004). Factor retention decisions in exploratory factor analysis: a tutorial on parallel analysis. Organizational
Research Methods, 7(2), 191e205.
Herron, C., York, H., Cole, S. P., & Linden, P. (1998). A comparison study of student retention of foreign language video: declarative versus interrogative
advance organizer. Modern Language Journal, 237e247.
Horn, J. L. (1965). A rationale and test for the number of factors in factor analysis. Psychometrika, 32, 179e185.
Hsiao, T. Y., & Oxford, R. L. (2002). Comparing theories of language learning strategies: a confirmatory factor analysis. Modern Language Journal, 368e383.
Isemonger, I. A. (2007). Operational definitions of explicit and implicit knowledge: response to R. Ellis (2005) and some recommendations for future
research in this area. Studies in Second Language Acquisition, 29, 101e118.
Kline, R. B. (2005). Principles and practice of structural equation modeling (2nd ed.). New York: The Guilford Press.
Krashen, S. D. (1996). The case for narrow listening. System, 24, 97e100.
Macaro, E. (2006). Strategies for language learning and for language use: revising the theoretical framework. The Modern Language Journal, 90(3), 320e337.
Maydeu-Olivares, A., Cai, L., & Hernandez, A. (2011). Comparing the fit of item response theory and factor analysis models. Structural Equation Modeling, 18,
333e356.
Muthe n, B. O. (2004). Mplus technical appendices version 3. Los Angeles, CA: Muthe n & Muthe
n.
Nix, J. M. L., & Tseng, W. T. (2014). Towards the Measurement of EFL Listening Beliefs with Item Response Theory Methods. International Journal of Listening,
28(2), 112e130.
Nunan, D. (2002). Listening in a second language. In Selected papers from the eleventh international symposium on English teaching/fourth Pan-Asian con-
ference (pp. 120e129).
O'Malley, J. M., & Chamot, A. U. (1990). Learning strategies in second language acquisition. Cambridge University Press.
 J.-M.L. Nix / System 57 (2016) 79e97 97

Orii-Akita, M. (2014). The effectiveness of interactive teaching methods in EFL classrooms: a comparison with bottom-up and top-down methods. In
Proceedings of the international symposium on the acquisition of second language speech. Concordia working papers in applied linguistics, 5, 464 (Vol.
477).
Oxford, R. L. (1990). Language learning strategies: What every teacher should know. Boston: Heinle.
Oxford, R. L., & Burry-Stock, J. A. (1995). Assessing the use of language learning strategies worldwide with the ESL/EFL version of the strategy inventory for
language learning (SILL). System, 23(1), 1e23.
Oxford, R. L., & Nyikos, M. (1989). Variables affecting choice of language learning strategies by university students. The Modern Language Journal, 73(3),
291e300.
O'Bryan, A., & Hegelheimer, V. (2007). Integrating CALL into the classroom: the role of podcasting in an ESL listening strategies course. ReCALL, 19, 162e180.
O'Malley, J. M., Chamot, A. U., & Küpper, L. (1989). Listening comprehension strategies in second language acquisition. Applied Linguistics, 10(4), 419e437.
Park, G. P. (2010). Investigation into learning strategies used by effective and less effective EFL learners in Korea. Asian Social Science, 6(8), 3e13.
Reckase, M. D. (2009). Multidimensional item response theory. New York: Springer.
Richards, J. C. (2008). Teaching listening and speaking: From theory to practice. Cambridge University Press.
Rost, M. (2005). L2 listening. In E. Hinkel (Ed.), Handbook of research in second language teaching and learning (pp. 503e528). Mahwah, NJ: Lawrence
Erlbaum.
Schmitt, N. (1996). Uses and abuses of coefficient alpha. Psychological Assessment, 8(4), 350e353.
Seigel, J. (2011). Readers respond: thoughts on L2 listening pedagogy. ELT Journal, 65(3), 318e321.
Sun, K. C. (2002). Investigation of English listening difficulties of Taiwanese students. In Selected papers from the eleventh international symposium on English
teaching/fourth Pan-Asian conference (pp. 518e525).
Tavakol, M., & Dennick, R. (2011). Making sense of Cronbach's alpha. International Journal of Medical Education, 2, 53e55.
Teng, H. C., & Chan, C. Y. (2008, June). An investigation of metacognitive strategies used by EFL listeners. In International conference of the association for
language awareness. Retrieved from http://www.eric.ed.gov.ezproxy.uwplatt.edu/ERICWebPortal/contentdelivery/servlet/ERICServlet.
Thompson, I., & Rubin, J. (1996). Can strategy instruction improve listening comprehension? Foreign Language Annals, 29(3), 331e342.
Tsui, A. B. M., & Fullilove, J. (1998). Bottom-up or top-down processing as a discriminator of L2 listening performance. Applied Linguistics, 19(9), 432e451.
Vandergrift, L. (1997). The strategies of second language (French) listeners. Foreign Language Annals, 30, 387e409.
Vandergrift, L. (1999). Facilitating second language listening comprehension: acquiring successful strategies. ELT Journal, 53(3), 168e176.
Vandergrift, L. (2002). ‘It was nice to see that our predictions were right’: developing metacognition in L2 listening comprehension. Canadian Modern
Language Review, 58(4), 555e575.
Vandergrift, L. (2003). Orchestrating strategy use: toward a model of the skilled second language listener. Language Learning, 53(3), 463e496.
Vandergrift, L. (2004). Listening to learn or learning to listen? Annual Review of Applied Linguistics, 24, 3e25.
Vandergrift, L. (2011). Second language listening: presage, process, product, and pedagogy. In E. Hinkel (Ed.), Handbook of research in second language
teaching and learning (Vol. 2, pp. 455e471). New York, NY: Routledge.
Vandergrift, L., Goh, C. M., Mareschal, C. J., & Tafagodhtari, M. H. (2006). The metacognitive awareness listening questionnaire: development and validation.
Language Learning, 56(3), 431e462.
Wirth, R. J., & Edwards, M. C. (2007). Item factor analysis: current approaches and future directions. Psychological Methods, 12(1), 58e79.