404 FALL 2014

Teaching and Testing L2 Spanish

Listening Using Scripted vs.
Unscripted Texts
Elvis Wagner
Temple University
Paul D. Toth
Temple University

Abstract: Even after years of study, many second language (L2) learners have
difficulty comprehending proficient speakers in real‐world settings. This difficulty stems
partly from the inauthentic nature of the spoken texts learners are exposed to in foreign
language (FL) classrooms—scripted, simplified, and overenunciated “textbook texts.”
This study compared the performance of intermediate university FL Spanish learners on
a listening comprehension test using scripted vs. unscripted spoken texts. An experimen-
tal design was used where learners were assigned to an unscripted group (n ¼ 85) or a
scripted group (n ¼ 86). The groups listened to the same texts, except that the scripted
texts were modified to resemble traditional textbook texts. Results indicated that the
scripted group scored significantly higher than the unscripted group. Implications for the
teaching and testing of L2 listening are discussed.

Key words: Spanish, classroom discourse, foreign language instructor preparation,

listening comprehension

Introduction
An enduring challenge for foreign language (FL) instructors is how to prepare
classroom learners for real‐world communicative contexts. Since the 1970s, the
movement toward communicative language teaching has increasingly defined learn-
ing outcomes in terms of the ability to communicate rather than knowledge of
structural features (see Richards & Rodgers, 2001). Indeed, in the United States, the
National Council of State Supervisors for Languages and ACTFL (2013) currently
conceptualize language learning in terms of functional “can‐do” statements that
describe what learners at different proficiency levels can accomplish in interpersonal,

Elvis Wagner (EdD, Teachers College, Columbia University) is Associate Professor

of TESOL, Temple University, Philadelphia.
Paul D. Toth (PhD, University of Pittsburgh) is Associate Professor of Spanish
Applied Linguistics at Temple University, Philadelphia.
Foreign Language Annals, Vol. 47, Iss. 3, pp. 404–422. © 2014 by American Council on the Teaching of Foreign
Languages.
DOI: 10.1111/flan.12091
Foreign Language Annals  VOL. 47, NO. 3
presentational, and interpretive communi-
cation. Recent decades have also seen a dra-
matic change in the types of texts to which
students have access: Twenty or thirty
years ago, providing target language spoken
texts involved either teacher talk or using
video‐ or audiotapes that were often difficult
to obtain, manipulate, and administer,
and that were also relatively expensive.
Advances in technology have made it
increasingly easier for FL learners to have
almost unlimited access to spoken texts in
the target language through Web sites,
foreign movies, and programs that are avail-
able on television, on DVD, or through
digital streaming.
Nevertheless, even with this great in-
crease in the availability of spoken texts in
the target language, many FL learners still
find it difficult to understand native speak-
ers in real‐world settings, even after having
studied the language for many years. A num-
ber of researchers (e.g., Flowerdew &
Miller, 1997; Gilmore, 2007; Lam, 2002;
Wagner, 2014) have suggested that this dif-
ficulty stems in part from the spoken texts to
which learners are exposed in FL class-
rooms. Because much of the spoken lan-
guage that learners hear is scripted,
simplified, and overenunciated, learners
have great difficulty comprehending un-
scripted, unsimplified, and normally enun-
ciated language outside the classroom. To
explore this issue, this study assessed the
comprehension performance of two compa-
rable groups of intermediate university
learners of Spanish as a foreign language:
One group listened to two unscripted texts,
and the second group listened to versions of
the same texts that had been scripted to
resemble traditional textbook materials.
Review of the Literature
Differences Between Scripted and
Unscripted Texts
It is widely accepted that written and spoken
language have different structural character-
istics (e.g., Chafe, 1982, 1985; Flowerdew,
1994; Halliday, 1985; Lund, 1991; Shohamy
405
& Inbar, 1991; Tannen, 1982). For the pur-
poses of this article, which considers only
spoken texts, these differences can be seen as
analogous to the contrasts between unscript-
ed spoken texts, involving extemporaneous
language with little or no planning, and
scripted spoken texts, in which the text is
written, revised, edited, and polished before
being read aloud. Tannen (1982) argued
that spoken texts can be arranged on a con-
tinuum of orality—with one end represent-
ing informal texts with distinctly oral
features and the other representing highly
literate texts that are planned, written, and
read aloud. Similarly, McCarthy and Carter
(1995) described a continuum of texts
that ranged from “speakerly” to “writerly”
(p. 216).
Wagner (2014) explained that scripted
and unscripted texts differ in three major
ways: First, unscripted texts tend to have
more pauses, fillers, and false starts, or
what Griffiths (1991) referred to as “hesita-
tion phenomena,” than scripted texts. Sec-
ond, lexico‐grammatical characteristics
differ greatly between scripted and unscripted
texts. Finally, there are extensive differences
in the phonological characteristics of scripted
and unscripted texts: Most significantly, the
connected speech that is characteristic of
unplanned spoken texts differs from the
“careful” speech characteristic of scripted
texts. The presence or absence of these three
characteristics in spoken texts can affect the
second language (L2) listening process. Given
that almost all of the research on scripted and
unscripted spoken texts has suggested that
unscripted texts are more difficult for L2
learners (Wagner, 2014), it is important to
consider how the different characteristics of
these two text types might affect listening
comprehension.
Hesitation Phenomena
Hesitation phenomena (Griffiths, 1991) are
the numerous silent pauses, filled pauses,
hesitations, false starts, and redundancies
that are characteristic of unplanned spoken
discourse. These often occur while the
speaker considers what he or she wants to
406
say and how to say it. How these hesitation
phenomena affect overall comprehension is
a matter of debate. On the one hand, it has
been argued that the pauses and fillers in
unscripted texts may be useful for L2 listen-
ers to the extent that pauses and fillers allow
more processing time for the listener
(Lam, 2002; Rubin, 1980). Because a faster
speech rate, usually measured in words per
minute or words per second, results in low-
ered levels of listening comprehension
among learners of a new language
(Griffiths, 1990, 1991; Kelch, 1985), the
presence of hesitation phenomena in a spo-
ken text can, in effect, result in a slower
speech rate. Furthermore, the redundancies
in spoken texts can give listeners another
opportunity to interpret what they have
heard, should they miss it the first time.
Numerous studies conducted with L2 listen-
ers focusing on redundancies as a hesitation
phenomenon found that spoken texts that
contained more redundant language were
helpful to learners in facilitating compre-
hension (Blau, 1990; Chaudron, 1988;
Chiang & Dunkel, 1992; Pica, Young, &
Doughty, 1987). However, other researchers
have argued that spoken texts that contain
numerous pauses or other hesitation phe-
nomena can be more difficult for learners to
process. For example, Voss (1979) and Grif-
fiths (1991) found that L2 listeners had
difficulties decoding filled pauses and false
starts because the learners tried to assign
semantic significance to the hesitation phe-
nomena, which hindered overall compre-
hension. Similarly, Freedle and Kostin
(1999) found that L2 test‐takers had more
difficulty comprehending texts that had
more filled and unfilled pauses than texts
with fewer such pauses.
Lexical and Grammatical Characteristics
In comparison to unplanned spoken
language, written language tends to have
longer idea units (Chafe, 1982, 1985;
Halliday, 1985), more complex syntax
(Chafe, 1982, 1985), and more embedded
clauses (Chafe, 1985). Chafe (1982, 1985)
argued that these characteristics tend to
FALL 2014
make written language that is read aloud
more difficult to comprehend than un-
planned spoken language. However, other
characteristics of written and planned lan-
guage could make it easier to comprehend
for L2 listeners. G. Brown (1995) and Chafe
(1985) described how written language that
is read aloud contains both less slang and
less colloquial language than unplanned
spoken language, so that learners who may
be unfamiliar with informal or colloquial
varieties of the L2 should benefit. Similarly,
McCarthy and colleagues (e.g., Carter &
McCarthy, 1997; McCarthy, 2005; McCar-
thy & Carter, 1995, 2001) argued that writ-
ten and spoken grammatical systems are
extensively different and that FL listeners
who were unfamiliar with the spoken gram-
matical systems that are typical of un-
planned speech would find these kinds of
texts more difficult to comprehend.
Connected Speech
Unplanned spoken texts typically have
concatenated speech that involves phono-
logical modifications including linking, as-
similation, deletion, epenthesis, elision,
contraction, juncture, and reduction
(Celce‐Murcia, Brinton, & Goodwin, 1994;
Crystal, 2004). This phenomenon, some-
times referred to as sandhi variation or re-
duced speech, is now most commonly
referred to as connected speech. J. D. Brown
(2012) argued that connected speech is in-
herent in all native speaker talk, especially in
informal, colloquial conversations, and is
less prevalent in formal speech. According
to Brown, connected speech is “based on a
set of rules used to modify the citation, or
dictionary, pronuncations of words so they
connect and flow more smoothly in natural
speech” (2012, pp. x–xi). There seems to be
a consensus in the literature that the phono-
logical characteristics of connected speech
that are typical of unscripted spoken texts
make these texts more difficult for L2 listen-
ers to process than the overenunciated
speech that is typical of scripted texts
(J. D. Brown, 2012). This would seem to
be especially true in the case of most L2
Foreign Language Annals  VOL. 47, NO. 3
listeners, who probably have had little or no
exposure to this type of speech.
In scripted texts, by contrast, the speak-
ers often enunciate much more clearly and
purposefully avoid the phonological modi-
fications, such as assimilation, deletion, and
reduction, that usually occur throughout
unscripted texts. According to Vandergrift
(2007), perhaps the most challenging aspect
of comprehending spoken input for lower‐
ability L2 listeners is segmenting the sound
stream into meaningful units: “Listeners,
unlike readers, do not have the luxury of
regular spaces that signal where words begin
or end” (p. 194). Thus, by providing listen-
ing texts that are recorded by speakers read-
ing those prescripted texts aloud, textbook
authors make listening easier for learners by
essentially doing much of the segmenting
work for them. Although this may make
such texts more comprehensible, their arti-
ficial enunciation can also hinder the devel-
opment of learners’ ability to understand
language as it is used in real‐world, unscript-
ed contexts (Wagner, 2014).
Second Language Listener
Performance With Different Types of
Texts
As described above, while most of the re-
search and theory has seemed to indicate
that the characteristics of unplanned spoken
language may make such texts more difficult
for L2 listeners, there is no consensus on this
issue, and it remains an empirical question.
However, only a relatively small number of
studies (i.e., Ahn, 1987; Henrichsen, 1984;
Ito, 2001; Read, 2002; Shohamy & Inbar,
1991) have directly compared L2 listeners’
performance on texts that do—and do not—
exhibit these characteristics, and even these
vary dramatically in methodology. Henrichsen
(1984) found that English as a second lan-
guage (ESL) learners had more difficulty com-
prehending spoken language that had
contractions, reductions, and assimilation
than spoken language without these phenom-
ena. However, the spoken input used in
Henrichsen’s study consisted only of sen-
407
tence‐length utterances. Ito (2001) modeled
his study after Henrichsen with comparable
ESL learners and obtained similar results.
Likewise, Ahn (1987) found that Korean
English as a foreign language (EFL) learners
comprehended spoken English where re-
duced forms and connected speech were
not present better than spoken English with
connected speech and reduced forms, again
using spoken input at the sentence rather
than the discourse level, and only considering
the phenomenon of connected speech.
Two other studies (Read, 2002; Shohamy
& Inbar, 1991) also compared L2 learners’
comprehension of different text genres
in work that provides insight into
the scripted vs. unscripted issue under in-
vestigation here. Read (2002) compared the
performance of two groups of Australian
ESL learners, one of which listened to a
scripted monologue while the other heard
the same text presented in a “discussion”
among three speakers. To design these
materials, Read first created the scripted
monologue and then attempted to create
an equivalent unscripted version by having
three speakers plan the content of a discus-
sion in advance without reading from a
script. Read found that the group that heard
the scripted monologue scored significantly
higher than those who heard the unscripted
discussion.
Shohamy and Inbar (1991) investigated
how Israeli EFL learners performed on tests
in which the same information was pre-
sented with three different types of texts: a
news broadcast, a lecturette, and a dialogue.
They assumed that these three text types
represented varying degrees of “orality,”
which was believed to enhance the texts’
“listenability.” The news broadcast was
scripted and read aloud to simulate a news
report. The lecturette was a monologue de-
livered for the purpose of instruction by a
simulated instructor using notes, with oral
features such as repetitions, pauses, and re-
dundancies. Finally, the consultative dia-
logue was “spontaneous,” with an expert
and addressee engaged in continual interac-
tion; this text was deemed the highest in
408
orality. The results showed that the groups
that listened to the dialogue and to the lec-
turette scored significantly higher than the
groups that listened to the news broadcast.
Shohamy and Inbar concluded that the de-
gree of orality of the text significantly affected
test scores: The more “listenable” the text, the
better the test‐takers scored.
The Use of Unscripted Texts in L2
Classrooms
The research described above makes it clear
that unscripted and scripted texts often dif-
fer dramatically and that L2 listeners have
difficulties processing and comprehending
unscripted texts that exhibit characteristics
of unplanned spoken discourse. Several
studies have examined these issues from a
classroom and materials perspective. Gil-
more (2007), for example, reviewed the
use of different types of spoken input for
the teaching of L2 listening and concluded
that even though there have been “appeals
for greater authenticity in language learn-
ing” for more than 30 years (p. 97), there
seems to have been little progress in actually
making this happen. A number of other
researchers have reviewed the types of lis-
tening texts that are included with language
textbooks, usually on a DVD or CD. Flower-
dew and Miller (1997) compared the “spo-
ken lecture” texts found in five different
textbooks with real‐life academic lectures,
analyzed their differences, and concluded
that L2 learners should be exposed to more
unscripted lecture texts in foreign language
classrooms. Similarly, Thompson (2003)
found extensive interactional and discoursal
differences between authentic academic
lectures and textbook academic lectures.
Gilmore (2004) surveyed a number of
FL textbooks and again found major differ-
ences between real‐life conversational
interactions and the interactions in the text-
book texts, although he also claimed that
some of the more recent textbooks had
incorporated at least some of the discourse
features of real‐life spoken language into the
listening texts.
FALL 2014
Numerous researchers have called for
the use of authentic spoken texts in the L2
classroom (e.g., Bacon, 1989; Field, 2008;
Gilmore, 2007, 2011; Meinardi, 2009; Rost,
2011; Wagner, 2013, 2014). Field (2008)
argued that unscripted, authentic spoken
recordings should be used in language class-
rooms “to expose the listener to the natural
cadences of the target language and to train
the learner in the unfamiliar process of ex-
trapolating meaning from a piece of speech
that may only be partly understood”
(p. 277). Meinardi (2009) argued that the
use of authentic spoken texts in L2 class-
rooms would not only help learners improve
their oral proficiency, but also give them “an
understanding of the preferred linguistic pat-
terns, traditions, and culture of the target
country” (p. 316). In contrast to the above
recommendations, a number of scholars (e.g.,
Griffiths, 1991; Guariento & Morley, 2001;
Richards, 2006; Widdowson, 2003) have rec-
ommended against using unscripted, natural
texts, especially for lower‐ability learners, ar-
guing that using these types of texts can
sometimes overwhelm learners, leading to
frustration and a lack of motivation.
Given these competing perspectives, a
number of longitudinal studies have exam-
ined the efficacy of using unscripted spoken
texts. Gilmore (2011) investigated the use of
authentic spoken texts in the L2 classroom
in a longitudinal study where participants
were randomly assigned to a control group,
which received textbook input only, and to a
treatment group, which received predomi-
nantly authentic input over a 10‐month pe-
riod. At the end of the treatment, the two
groups’ scores on a battery of measures were
compared. Gilmore found that the authentic
input group scored higher than the textbook
input group on 8 of the 13 measures of
communicative competence, while he found
no significant differences between the
groups on the remaining five measures.
Gilmore (2011) concluded that the authen-
tic materials used with the treatment group
“were better able to develop a range of com-
municative competencies in the learners”
than the textbook texts (p. 810).
Foreign Language Annals  VOL. 47, NO. 3
Rashtchi and Afzali (2011) performed a
similar study with 45 EFL university stu-
dents in Iran who were divided into two
intact classes. One class, the treatment
group, received instruction on specific fea-
tures of spoken grammar in the listening
texts, with the goal of helping learners to
notice and attend to these features in the
sample text and in subsequent input. The
other class heard the same authentic texts in
class, except without the focused instruction
on spoken grammatical features. At the end
of the 15‐week semester, the experimental
group scored significantly higher than the
control group on a posttest that used au-
thentic texts. Herron and Seay (1991) found
similar results with FL learners of French.
Meanwhile, Matsuzawa’s longitudinal study
(2006) found that explicit teaching of the
reduced forms in unscripted discourse led to
increased comprehension with Japanese
EFL learners, and Brown and Hilferty
(2006) found similar results with Chinese
EFL learners.
In summary, while it is almost univer-
sally recognized that unscripted and
scripted spoken texts have differing essential
characteristics, and while many have argued
that unscripted texts are more difficult than
scripted texts for L2 listeners, there is little
empirical evidence that directly compares
L2 learners’ performance on scripted vs. un-
scripted texts. As Ito (2006) argued, the
research on connected speech is “extremely
limited” (p. 25). Thus, this study sought to
explore further how L2 listeners might differ
in their ability to comprehend scripted vs.
unscripted spoken texts. At the same time,
evidence from longitudinal classroom re-
search suggests that L2 learners may actually
benefit from working with unscripted texts
when instruction focuses on helping them to
cope with some of the texts’ complex struc-
tural features. Because of the dearth of em-
pirical research on how L2 learners might
differ in their ability to process unscripted
spoken texts, the following research ques-
tion was examined: How do learners of
Spanish as a foreign language perform on a
listening comprehension test using un-
409
scripted spoken texts, as compared to simi-
lar learners taking the same test but with
scripted spoken texts?
Method
An experimental design was used where 14
intact classes were randomly assigned to one
of two groups. The independent variable was
the type of audio text that was used as spo-
ken input on the test, either “unscripted” or
“scripted.” The dependent variables were
the group scores on the listening compre-
hension test under each condition. One
group of fourth‐semester university learners
of Spanish as a foreign language took a lis-
tening comprehension test using unscripted
spoken texts, while a comparable group took
the same test with a similar text that was
scripted, modified, and read aloud. The
scores of the two groups on the tests were
compared to see if they differed significantly
in their performance.
Participants
A total of 174 learners of Spanish as a foreign
language at a large American public univer-
sity participated in this study. Each partici-
pant was enrolled in one of 14 intact classes
of an intermediate Spanish course entitled
“Conversational Review.” This course foc-
used on conversational speaking and listen-
ing abilities and was the fourth‐semester
course for most students. The instructors
from 14 classes volunteered to have their
classes participate. Participants were not
compensated for their activities.
About 93 percent of the participants
listed English as their first language on a
preliminary questionnaire. Among the 14 stu-
dents who listed first languages other than
English, three listed Spanish. Data from these
three students were not included, thus reduc-
ing the total number of participants in this
study to 171. The 14 classes of Conversation-
al Review were randomly assigned to one of
the two treatment conditions: Seven classes
were assigned to the “unscripted” group
(n ¼ 85), and seven were assigned to the
“scripted” group (n ¼ 86).
410
The background characteristics of stu-
dents in each group were very similar. The
average age of participants in the unscripted
group was 20.24 years, while the average age
in the scripted group was 20.45 years. In the
unscripted group, 58.8 percent of the par-
ticipants were female, while in the scripted
group, 69.8 percent of the participants were
female. The participants’ self‐assessment of
their proficiency in Spanish was also re-
ported, using a six‐point scale, in which 1
was “lower beginner,” 2 was “upper begin-
ner,” 3 was “lower intermediate,” 4 was
“upper intermediate,” 5 was “lower ad-
vanced,” and 6 was “upper advanced.” The
participants in the unscripted group rated
their proficiency as 3.49 on the six‐point
scale, while the participants in the scripted
group rated their proficiency as 3.53 on the
six‐point scale. The fact that the two groups
were randomly assigned, that participants
all came from the same course, and that
the two groups’ proficiency self‐assessments
were so similar, suggests that the two groups
were comparable.
As in much educational research, this
study used intact classes to form treatment
groups, and thus it was necessary to consider
the possibility of nesting effects. Here,
test‐taking conditions were identical for all
participants, whether they belonged to the
scripted group or the unscripted group. That
is, the researchers administered the test with
the same equipment, without variation by
class. In addition, while each of the 14 par-
ticipating classes had different instructors,
the syllabus for the course was uniform, so
that instructors had little ability to introduce
major variations in the selection and se-
quencing of course topics and instructional
materials. Furthermore, the study took place
within the first few weeks of the semester.
Thus, the possibility of a strong classroom
effect was minimized. Nevertheless, because
intact classes were used, in comparing the
two groups’ performance, it was decided to
perform a nested ANOVA with listening text
type as a fixed factor and classroom as a
random factor, as described in the “Analysis”
section below.
FALL 2014
Materials
All of the materials that we used were creat-
ed especially for this study and consisted of
two different versions of two recorded con-
versations between two female Spanish na-
tive speakers from Peru.
The Unscripted Spoken Texts
For the unscripted versions of the texts, the
two volunteer speakers were instructed to
speak as naturally as possible while record-
ing two role‐plays that would be used for a
listening comprehension test. In both cases,
the prompts were based on those used for
ACTFL’s Oral Proficiency Interview to elicit
language at the intermediate level, which
was assumed to be appropriate for fourth‐
semester learners. The speakers were told
that, for the first text, Un Cuarto para Rentar
[A Room for Rent], one person should act as
a university student looking for a room to
rent in an apartment near her university, and
the second person should act as the owner
who had advertised the room for rent. Based
only on these instructions, the speakers cre-
ated the recording extemporaneously, with-
out further planning. The researchers then
transcribed the recording to document in
detail its discursive characteristics, includ-
ing filled and silent pauses, false starts, ex-
pansions, phatic expressions, and back
channels (utterances made by the listener
to indicate understanding, acknowledg-
ment, etc.). Word boundaries were based
on standard Spanish orthographic conven-
tions, but all repeated words, audible non-
word fillers, and word fragments within false
starts were also transcribed as words. For
example, in the recording excerpt shown in
Example 1, phatic expressions and fillers
such as Pues [Well] in turn (b) and mm‐
hm in turn (d) were each counted as one
word, while in the first and second lines of
turn (f), the fillers pues, bueno [OK], and eh,
as well as the false start pue—, were also
counted as separate words. The resulting
recording was 3 minutes and 36 seconds
in length and consisted of 572 total words
delivered at a speech rate of 2.42 words per
second.
Foreign Language Annals  VOL. 47, NO. 3 411

A similar procedure was followed for
the second unscripted text, Una Amiga Va
de Vacaciones [A Friend Goes on Vacation].
The speakers were instructed to perform a
role‐play where one person approached a
friend to ask if the friend could watch her
house while she went on vacation. From this
information, the two speakers created the
second text extemporaneously, without fur-
ther planning. The same transcription con-
ventions were applied to the recording, and
the resulting text was 3 minutes and 21
seconds in length, with 615 total words de-
livered at a speech rate of 2.94 words per
second.
The Scripted Spoken Texts
Based on the written transcriptions, two
scripts were prepared with all fillers, false
starts, expansions, phatic expressions, and
back channels removed so that the texts
would acquire the more “writerly” charac-
teristics typical of FL listening texts that are
found in most contemporary textbooks and
ancillary materials. The revised written
scripts were then given to the same two
speaker volunteers, and they were asked to
record the text again by simply reading the
text. They were instructed to speak some-
what more slowly, to enunciate clearly in
order to avoid using connected speech, and
to wait to speak until the other had finished
her turn. After one rehearsal reading, the
speakers recorded the texts. The scripted
version of the Un Cuarto para Rentar text
was 3 minutes and 43 seconds long and
consisted of 494 words, resulting in a speech
rate of 2.22 words per second. The scripted
version of the Una Amiga Va de Vacaciones
text was 3 minutes and 39 seconds long,
with 533 words, resulting in a 2.43 words‐
per‐second speech rate. Table 1 provides a
comparison of the scripted and unscripted
versions of the texts.
To more concretely illustrate the differ-
ences between the scripted and unscripted
texts, Example 1 shows the features of the
unscripted recording that were eliminated
from the scripted text in the opening lines of
the first role‐play, Un Cuarto para Rentar.
Here the fillers, false starts, expansive repe-
titions, back channels, and phatic expres-
sions have all been underlined, while
English glosses are provided in brackets un-
derneath each discourse turn. In line (a),
Pilar, the property owner, greets Eunice,
the prospective tenant, and begins with
the phatic exclamative Oh, and a repeated
sí. At the end of the first line, there is a filler,
eh…, and the last sentence in her utterance
repeats and expands the preceding phrase
with qué es lo que buscas [what is it you’re
looking for]. At this point, in turn (b), Eu-
nice inserts an overlapping filler, pues, while

TABLE 1
Characteristics of the Listening Texts
Text Length of Text Number of Speech Rate
Words (Words per
in Text Second)

Un Cuarto para Rentar 3 minutes 36 seconds 572 2.42

(Unscripted) (216 seconds)
Un Cuarto para Rentar 3 minutes 43 seconds 494 2.22
(Scripted) (223 seconds)
Una Amiga Va de 3 minutes 21 seconds 615 2.94
Vacaciones (Unscripted) (201 seconds)
Una Amiga Va de 3 minutes 39 seconds 533 2.43
Vacaciones (Scripted) (219 seconds)
412 FALL 2014

Example 1. Opening lines of Text 1, Un Cuarto para Rentar, “A Room for Rent”
a. Pilar: Oh, sí, sí, Eunice. Sí, estoy rentando un cuarto. Aquí tengo tres
cuartos eh… libres para rentar. Y yo no sé, cuál es que tú
buscas, qué es lo que buscas,
[Oh, yes, yes, Eunice. Yes, I’m renting a room. I have here
three, uh… free rooms to rent. And I don’t know what you’re
looking for, what it is you’re looking for,]
b. Eunice: Pues,
[Well,]
c. Pilar: Y… y quisiera que me digas qué cosa
[And… and I’d like you to tell me what kind of thing] overlap
d. Eunice: mm‐hm
e. Pilar: es lo que quieres.
[it is that you want.]
f. Eunice: Pues, bueno, pue—, yo soy estudiante, y estoy tomando unos
cursos en la universidad, eh… por tres meses, y me gustaría
tener un lugar, me parece que este lugar es conveniente
porque está cerca de la universidad. Y, me gustaría alquilar
un cuarto, eh… por tres meses solamente.
[Well, OK, we—, I’m a student, and I’m taking some courses in the
university, um… for three months, and I’d like to have a place,
it seems that this place is convenient because it’s near the university.
And I’d like to rent a room, uh… for three months only.]

Pilar continues her thought in (c) after a
repeated y [and], perhaps as a result of Eu-
nice’s overlapped preceding turn. Eunice
then utters a back‐channeled mm‐hm in
(d) while Pilar concludes in (e). Eunice’s
extended turn in (f) then begins with three
repeated fillers and goes on to employ two
other uses of the filler eh. In her second line,
she also produces a false start by beginning
and then abandoning the thought me gusta-
ría tener un lugar [I’d like to have a place],
before rephrasing and completing it.
After the unscripted recording was tran-
scribed, each one of these underlined ele-
ments was deleted in the text to be read for
the scripted recording. Any elements from
overlapping turns were consolidated into
noninterruptive single turns.
The Listening Test Instrument
The listening comprehension test was creat-
ed based on a construct definition of L2
listening ability given in Wagner (2002,
2006) and Buck (2001). In these sources,
L2 listening ability was defined as the ability
to listen for both explicitly stated and im-
plicit information in a text. This definition
was operationalized by creating test items
that assessed both types of information.
Eight comprehension items with four re-
sponse options each were created for each
spoken text. The test instrument was piloted
with a group of 14 test‐takers who were
students in a section of Conversational Re-
view but who did not otherwise participate
in the study. After taking the test, the 14 test‐
takers completed a written survey asking
about the difficulty level of the test, the
amount of time given to answer questions,
the instructions, and any especially difficult
or ambiguous questions. Based on statistical
analyses of these data, the test was revised
slightly into its final form. The English
translation of the test instrument is provided
in the Appendix.
Procedures
During a regularly scheduled meeting of each
of the 14 classes, the researchers described
the study in English as a listening compre-
hension test designed for intermediate
Foreign Language Annals  VOL. 47, NO. 3
learners, without raising the issue of un-
planned vs. scripted spoken texts. The par-
ticipants signed informed consent forms and
completed the test. They had one minute to
preview the eight multiple‐choice compre-
hension questions for Task 1, and then the
text was played one time. They then had two
and a half minutes to answer the eight ques-
tions. The procedure was repeated for Task 2.
After completing the test, the participants
completed an 18‐item questionnaire asking
about the spoken texts used on the test and a
five‐item background questionnaire. In total,
the entire procedure took just less than 30
minutes for each class. After finishing the
test, the students were debriefed on the pur-
pose of the study. Within a week of taking the
test, the participants were informed of their
performance on the test, including their per-
centage of correct answers as well as their
class’s average.
Power and Sample Size
In the pilot test that was conducted for this
study, the unscripted group’s mean score
was approximately 9 out of 16, and the
scripted group’s mean score was approxi-
mately 10 out of 16. Based on this rough
estimate of effect size, a power analysis was
conducted, and it was determined that, to
detect an effect of this specific size, a sample
size of 64 participants in each treatment
group in the study would result in a power
level of 0.8 (as suggested by Larson‐
Hall, 2010) to detect an effect of this specific
size. Because the effect size was only an
estimate based on a small sample from the
pilot test, it was decided to use a larger and
thus more conservative sample size of at
least 80 participants in each group. An alpha
level of 0.05 was used in the analyses.
Analyses
SPSS version 17.0 was used for all statistical
analyses. First the internal consistency reli-
ability of each version of the test was esti-
mated using Cronbach’s alpha. Then the
difficulty of each item was examined, as
well as the item‐total correlation for each
413
item with the overall test. Items that were
too difficult or too easy, or that had poor
item‐total correlation with the overall test,
would be considered for deletion.
Descriptive statistics were computed to
examine the central tendency and dispersion
of the two groups on the test. In order to
examine if the two groups differed at a sta-
tistically significant level on their perfor-
mance on the listening tests, a nested
ANOVA was conducted with listening text
type as a fixed, between‐subjects factor and
classroom as a random between‐subjects
factor. The nested ANOVA was used in rec-
ognition of the fact that intact classes were
used in the grouping, which might result in a
nesting effect.1
Results
The internal consistency reliability of the
test was estimated, and the item‐total corre-
lation for each of the items was computed in
order to examine how reliably each of the 16
items was performing in both versions of the
test. All of the items were deemed to be of
appropriate difficulty level, and all were re-
liable in relation to the overall test. For the
unscripted version of the test, the internal
consistency reliability was a ¼ 0.69; for the
scripted version, the internal consistency
reliability was a ¼ 0.68.
Descriptive statistics for the two ver-
sions of the test are shown in Table 2. The
unscripted group’s mean score was 8.21
(SD ¼ 3.03), while the scripted group’s
mean was 9.56 (SD ¼ 3.01). These results
suggest that the test was relatively difficult
for all of the test‐takers in that the mean was
just over 50 percent for the test‐takers in the
unscripted group and just under 60 percent
for the scripted group. In addition, the skew-
ness and kurtosis figures suggest that the
distribution of scores was approaching nor-
mal and that the use of the nested ANOVA
was appropriate. The results of the nested
ANOVA indicated that there was a statisti-
cally significant effect for the text type:
F ¼ 21.455 (1, 13.785; p < 0.001, partial
eta squared ¼ 0.609), and there was no
414
TABLE 2
Descriptive Statistics for the Two Versions of the Test
# of Mean Mean SD Minimum Maximum Kurtosis Skewness Reliability
items (%)
Unscripted 16 8.21 51.3 3.03 2
Version
(n ¼ 85)
Scripted 16 9.56 59.7 3.01 3
Version
(n ¼ 86)
statistically significant effect for the class-
room nesting: F ¼ 0.334 (11, 160;
p ¼ 0.077, partial eta squared ¼ 0.022). The
group of test‐takers who heard the scripted
text scored significantly higher on the com-
prehension test than the group of test‐takers
who heard the unscripted oral text.
Discussion
This study sought to determine how learners
of Spanish as a foreign language performed
on a listening comprehension test using un-
scripted spoken texts, as compared to simi-
lar learners taking the same test with
scripted spoken texts. The results showed
that the 85 test‐takers who heard the un-
scripted texts scored statistically lower than
the 86 who heard the scripted version. The
results provide empirical support for the
widely held belief that authentic, unscripted
texts exhibiting the connected speech, hesi-
tation phenomena, and lexico‐grammatical
characteristics that are typical of unplanned
spoken discourse are more difficult for L2
learners than specially modified texts that
lack many of these characteristics.
Nonetheless, the unscripted group’s sta-
tistically lower scores need not mean that
unscripted texts be avoided among lower‐
proficiency learners, as Richards (2006) and
Guariento and Morley (2001) suggested. In
raw terms, the scripted group’s mean of 9.56
out of 16.00 amounted to slightly more than
one more test item correct on average than
the unscripted group, whose mean was 8.21.
The fact that the unscripted group gave a
FALL 2014
(a)
15 0.22 0.58 0.69
15 0.80 0.08 0.68
fairly robust performance without prior
training on such texts suggests a baseline
of comprehension skill that could be devel-
oped with instructional support. Although
the faster speech rate, speaker overlaps, and
numerous fillers, false starts, expansions,
and back channels may have rendered the
unscripted texts more difficult than the
scripted versions, the partial success of these
intermediate learners suggests that a greater
ease of comprehension may not always be
the optimal criterion in choosing materials
designed to increase proficiency. Thus, rath-
er than consistently lightening the process-
ing burden by simplifying texts with a
carefully read script, we concur with Gil-
more (2011) that greater assistance in cop-
ing with the complexities of natural
discourse will often more aptly facilitate
L2 communication beyond the classroom.
Thus, it is precisely because FL learners
have difficulty with unscripted texts that we
believe they ought to work with them more
often. Given the statistical significance of the
gap between learners who heard more auth-
entic communication and those who heard
more artificial language, the pedagogical sig-
nificance of our results can only mean a call
to more, not less, practice with natural spo-
ken texts. Gilmore’s (2011) longitudinal
study indeed provided evidence that expos-
ing learners to such texts could lead to more
successful learning outcomes, particularly if
explicit instruction focuses on developing
strategies for processing unscripted lan-
guage. Training listeners to consciously re-
flect on the listening process and regulate
Foreign Language Annals  VOL. 47, NO. 3
their learning is exactly what Vandergrift
and Goh (2009, 2012) recommended in
their metacognitive approach to listening
instruction.
Pedagogical Implications
To this end, it is important that instructors
become familiar with the features of target
language natural discourse so that they may
develop explicit, “consciousness‐raising”
lessons around these patterns (Ellis, 2002).
Ultimately, learners must associate the
unique structures of unscripted speech
with both the literal, referential meanings
that are conveyed and the underlying dis-
cursive goals that such structures accom-
plish. Whereas pauses and fillers can be
fairly salient and thus readily attended to,
it may be more difficult for learners to per-
ceive the connection between such elements
and the negotiation of propositions among
speakers in real time, a process that is often
simplified and concentrated into many few-
er turns in carefully read, scripted dialogues.
This was indeed one of the main points of
contrast between the two text types used in
this study, with the development of propo-
sitions in the unscripted texts spread over
many more discourse turns than in the
scripted versions, and many more total
words dedicated to false starts, expansions,
restatements, phatic expressions, and back‐
channeling as well. Furthermore, in addi-
tion to hearing extended vowels and sylla-
bles like um and eh, learners may have been
confused by fillers that coincided with
words or phrases with other literal mean-
ings, such as Spanish este, which also means
“this,” and bueno, which means “good.”
Once instructors understand how lexico‐
grammatical structure and communicative
function relate to each other in unscripted
discourse, they can then help learners dis-
tinguish between: elements that advance the
propositional meaning of the exchange; el-
ements that sustain the speakers’ social rela-
tionship through phatic expressions, back‐
channeling, and politeness; and elements
that simply buy more time to verbalize the
speakers’ thoughts through false starts, fill-
415
ers, and repetitions. Indeed, in order to pen-
etrate the rapid, concatenated speech that
they hear, learners must be able to decipher
the complex yet purposeful functions that
underlie unscripted interactions.
Clearly then, if the goal is to prepare
learners for L2 communicative contexts be-
yond the classroom, continually avoiding
difficult material runs contrary to that pur-
pose. Numerous studies (i.e., Brown &
Hilferty, 2006; Gilmore, 2011; Herron &
Seay, 1991; Matsuzawa, 2006; Rashtchi &
Afzali, 2011) have demonstrated the possi-
bilities for using unscripted spoken texts
effectively in the L2 classroom. Rather than
providing learners a steady diet of simplified
texts that are easier to comprehend, instruc-
tors should be challenged by this study to
provide both more exposure to, and more
support for working with, unscripted texts
through instruction that raises awareness of
their unique features and the discursive
means by which speakers achieve their com-
munication goals. Adapting the task de-
mands to the ability level of the students is
the key to making unscripted listening texts
more approachable for lower‐proficiency
learners. This can be achieved though the
careful management of (1) text length; (2)
the targets of attentional focus; (3) the in-
tended level of comprehension; (4) the num-
ber of listening rounds; and (5) opportunities
for hypothesizing, feedback, and knowledge
consolidation (Wagner, 2014).
For example, educators could support
beginning learners by having them work
with shorter texts, offering two or more
listening rounds, and asking them to focus
at first on identifying recently studied func-
tions of language—perhaps basic greetings
and leave‐takings, daily routines, or person-
al descriptions. Then, instructors could
break new ground by having learners draw
inferences among these islands of familiarity
to derive the broader discursive goals of
speakers or the meaning of unfamiliar words
and phrases. As with all listening tasks,
learners should always have to do something
purposeful with the information they have
gleaned from a text (Johnson, 1982; Lee &
416
VanPatten, 2003). Communicating to learn-
ers what this purpose will be—whether
drawing conclusions about the speaker, or
acting upon the information given in the
text—will help them discern the text
features that are essential to the task while
avoiding the distraction and frustration of
unfamiliar language that is beyond their
reach. J. D. Brown (2012) and Vandergrift
and Goh (2012) have provided excellent
resources for instructors who want to pro-
mote L2 listeners’ ability to comprehend the
types of unplanned spoken language found
in most real‐world settings. Ultimately,
working with unscripted texts comes
down to helping learners to cope with their
inability to understand everything, so that
they will gain confidence in their ability to
understand something, and thereby discover
a grounding in comprehension that will sus-
tain them in natural conversation.
Assessment Implications
This study also has important implications
for the assessment of FL listening. The first
implication has to do with the construct
validity of listening tests (Messick, 1989,
1996). If the goal of a particular test is to
assess how well a learner can understand the
target language as it is spoken in a real‐world
context, then it is important to include texts
that exhibit the characteristics of real‐world
spoken language. By including unscripted
texts on these assessments, one can make
more valid inferences about test‐takers’ lis-
tening ability in that real‐world communi-
cative domain (Wagner, 2013). Another
important implication for assessment has
to do with the consequential aspect of con-
struct validity. Messick (1989) argued that
test developers should consider the social
and educational impact of an assessment.
An assessment’s impact is referred to as
washback, which can be positive or negative.
Washback is the effect that a test or assess-
ment has on instructors, learners, and edu-
cational systems. If large‐scale and high‐
stakes FL assessments use unscripted texts,
instructors may be more cognizant of the
FALL 2014
importance of using unscripted texts in their
classrooms and teaching learners strategies
for processing these kinds of texts. Similarly,
if students know that both the classroom
assessments and the high‐stakes assess-
ments in the L2 include unscripted spoken
texts, then they should be more willing and
motivated to work with these types of texts
during instruction and to seek exposure to
such texts beyond the classroom as well
(Wagner, 2013).
Limitations
There are a number of limitations to this
study that must be acknowledged. In the
interest of minimizing the use of class
time, it was not feasible to give the test‐
takers a pretest to ensure that the two groups
were of comparable listening ability. Fur-
thermore, the reliability of the test instru-
ments used, while relatively high (a ¼ 0.69
for the unscripted test and a ¼ 0.68 for the
scripted test), was lower than ideal. Finally,
the focus of this study was narrow, using
only university learners of an FL at the in-
termediate level, and thus it is difficult to
generalize the results of this study to other
contexts.
It is also difficult to compare the results
of this study with previous empirical studies
investigating the issue, because of methodo-
logical and participant demographic differ-
ences. Studies such as Voss (1979), Griffiths
(1991), and Freedle and Kostin (1999)
found that the presence of hesitation phe-
nomena led to decreased L2 comprehension.
Similarly, a number of studies (Ahn, 1987;
Henrichsen, 1984; Ito, 2001) found that
connected speech also resulted in lower lev-
els of L2 comprehension. While these stud-
ies are informative, they focused on isolating
individual characteristics of unplanned spo-
ken language, and thus it is difficult to com-
pare their results to this study, which
holistically compared performance on
scripted vs. unscripted texts.
The two studies closest to the current
one are Read (2002) and Shohamy and Inbar
(1991). Read (2002) found that the
Foreign Language Annals  VOL. 47, NO. 3
“monologue” group, which listened to a
scripted text read aloud, scored higher
than the “dialogue” group, which listened
to an unscripted dialogue discussion. Again
however, a direct comparison is difficult,
because the current study used the same
speakers and genre for both texts while
Read’s study used different numbers of
speakers and genres in creating two equiva-
lent texts. In addition, in Read’s study, the
scripted version of the monologue text was
created first, and then a “discussion” version
was created. To prepare this version, three
speakers read the source articles for the
monologue, and then “planned in advance
how the discussion would proceed, but it
was not scripted at all” (Read, 2002, p. 110).
This differs markedly from how the texts
were created in the current study.
The results of this study contrast with
those of Shohamy and Inbar (1991), who
found that the groups that heard an un-
scripted discussion or a semi‐scripted lectur-
ette scored significantly higher than the
group that heard a scripted newscast. How-
ever, Shohamy and Inbar’s study had three
different genres of oral texts (a discussion, a
lecturette, and a newscast), and it was un-
clear to what extent the genre of the texts
affected performance. In addition, while the
lecturette and newscast only had one speak-
er, the discussion used two speakers. It is
also unclear if the speakers were the same in
all three texts. Hence, a direct comparison
between Shohamy and Inbar’s study and the
one reported here is not possible.
Clearly, more research is needed to in-
vestigate how the different characteristics of
unplanned spoken texts (i.e., hesitation phe-
nomena, lexico‐grammatical characteristics,
connected speech) affect L2 listening perfor-
mance. Still, the current study provides
much needed empirical evidence for the
widely held belief that FL learners find it
more difficult to process and comprehend
unplanned spoken discourse than spoken
texts that have been written, revised, and
edited, and then read aloud. Researchers
such as Flowerdew and Miller (1997), Gil-
more (2007), and Wagner (2014) have ar-
417
gued that at least part of the reason is that
much, or even most, of the spoken input that
FL learners hear consists of textbook texts,
which lack many of the characteristics of
unplanned spoken language in the real
world. Hence, the results point to the peda-
gogical importance of exploring unscripted
texts with greater instructional support,
rather than continually utilizing simplified
texts that, while not requiring as much sup-
port, lack the characteristics of natural
discourse.
Conclusion
In this study, intermediate‐level learners of
Spanish as foreign language found unscript-
ed listening texts more difficult to compre-
hend than scripted texts. Given that the
unscripted texts had more discourse turns,
as well as many more false starts, expansions,
restatements, phatic expressions, and back‐
channeling than the scripted texts, listeners’
comprehension scores were significantly
lower for these texts than for the scripted
texts. Still, because such features are essen-
tial to natural communication beyond the
classroom, the results provide compelling
pedagogical evidence that unscripted texts
ought to be made an essential component
of L2 listening comprehension tasks. To be
sure, the fact that the two groups’ respective
means were only 51.3 and 59.7 percent sug-
gests that both groups found the test chal-
lenging. Still, the unscripted group’s
performance indicates that there was much
in the text that the learners could compre-
hend, and far from being too overwhelming,
the text might be made more readily accessi-
ble if students could be engaged in classroom
tasks that were designed to raise awareness
about the structural and functional features
of such discourse. Hence, we argue that the
comprehension challenges presented by un-
scripted texts should not be avoided in favor
of more simplified scripted versions; rather,
unscripted texts should be accompanied by
greater amounts of instructional support and
more frequently and actively incorporated
into the L2 curriculum.
418
Acknowledgments
We would like to thank the students in the
14 classes of Conversation Review, as well as
their instructors, for participating in this
study. Thanks also to Poppy Wagner, who
assisted with data input. Thank you to Anne
Marie Hindman and Joshua Klugman for
their assistance with statistical procedures.
Finally, thank you to the anonymous re-
viewers for their insightful comments.
Note
1. This analysis was used at the suggestion
of one of the Foreign Language Annals
reviewers.
References
Ahn, S. (1987). Sandhi‐variation and affective
factors as input filters to comprehension of spo-
ken English among Korean learners (Unpub-
lished doctoral dissertation). University of
Texas, Austin.
Bacon, S. (1989). Listening for real in the
foreign‐language classroom. Foreign Language
Annals, 22, 543–551.
Blau, E. (1990). The effect of syntax, speed,
and pauses on listening comprehension.
TESOL Quarterly, 24, 746–753.
Brown, G. (1995). Speakers, listeners, and com-
munication. Cambridge, UK: Cambridge Uni-
versity Press.
Brown, J. D. (Ed.). (2012). New ways in teach-
ing connected speech. Alexandria, VA: TESOL
International Association.
Brown, J. D., & Hilferty, A. (2006). The effec-
tiveness of teaching reduced forms for listen-
ing comprehension. In J. D. Brown & K.
Kondo‐Brown (Eds.), Perspectives on teaching
connected speech to second language speakers
(pp. 51–58). Honolulu: University of Hawai’i,
National Foreign Language Resource Center.
Buck, G. (2001). Assessing listening. Cam-
bridge, UK: Cambridge University Press.
Carter, R., & McCarthy, M. (1997). Exploring
spoken English. Cambridge, UK: Cambridge
University Press.
Celce‐Murcia, M., Brinton, D., & Goodwin, J.
(1994). Teaching pronunciation: A reference for
teachers of English to speakers of other lan-
FALL 2014
guages. Cambridge, UK: Cambridge University
Press.
Chafe, W. (1982). Integration and involve-
ment in speaking, writing and oral literature.
In D. Tannen (Ed.), Spoken and written lan-
guage: Exploring orality and literacy (pp. 35–
53). Norwood, NJ: Ablex.
Chafe, W. (1985). Linguistic differences pro-
duced by differences between speaking and
writing. In D. Olson, D. Torrance, & A. Hild-
yard (Eds.), Literacy, language and learning
(pp. 105–123). Cambridge, UK: Cambridge
University Press.
Chaudron, C. (1988). Second language class-
rooms: Research on teaching and learning.
Cambridge, UK: Cambridge University Press.
Chiang, C. S., & Dunkel, P. (1992). The effect
of speech modification, prior knowledge and
listening proficiency on EFL lecture learning.
TESOL Quarterly, 26, 345–374.
Crystal, D. (2004). A dictionary of linguistics
and phonetics (5th ed.). Oxford: Blackwell
Publishing.
Ellis, R. (2002). Grammar teaching—Practice
or consciousness raising?. In J. C. Richards &
W. A. Renandya (Eds.), Methodology in lan-
guage teaching (pp. 167–174). Cambridge, UK:
Cambridge University Press.
Field, J. (2008). Listening in the language class-
room. Cambridge, UK: Cambridge University
Press.
Flowerdew, J. (1994). Research of relevance to
second language lecture comprehension—An
overview. In J. Flowerdew (Ed.), Academic
listening: Research perspectives (pp. 7–29).
Cambridge, UK: Cambridge University Press.
Flowerdew, J., & Miller, L. (1997). The teach-
ing of academic listening comprehension and
the question of authenticity. English for Specif-
ic Purposes, 16, 27–46.
Freedle, R., & Kostin, I. (1999). Does the text
matter in a multiple‐choice test of comprehen-
sion? The case for the construct validity
of TOEFL’s minitalks. Language Testing, 16,
2–32.
Gilmore, A. (2004). A comparison of textbook
and authentic interactions. ELT Journal, 58,
363–374.
Gilmore, A. (2007). Authentic materials
and authenticity in foreign language learning.
Language Teaching, 40, 97–118.
Gilmore, A. (2011). “I prefer not text”: De-
veloping Japanese learners’ communicative
Foreign Language Annals  VOL. 47, NO. 3
competence with authentic materials. Lan-
guage Learning, 61, 786–819.
Griffiths, R. (1990). Speech rate and NNS com-
prehension: A preliminary study in time‐ben-
efit analysis. Language Learning, 40, 311–336.
Griffiths, R. (1991). Pausological research in
an L2 context: A rationale and review of select-
ed studies. Applied Linguistics, 12, 345–364.
Guariento, W., & Morley, J. (2001). Text and
task authenticity in the EFL classroom. ELT
Journal, 55, 347–353.
Halliday, M. A. K. (1985). Spoken and written
language. Oxford: Oxford University Press.
Herron, C., & Seay, I. (1991). The effect of
authentic oral texts on student listening com-
prehension. Foreign Language Annals, 24, 487–
495.
Henrichsen, L. (1984). Sandhi‐variation: A fil-
ter of input for learners of ESL. Language
Learning, 34(3), 103–126.
Ito, Y. (2001). Effect of reduced forms on ESL
learners’ input‐intake process. Second Lan-
guage Studies, 20, 99–124.
Ito, Y. (2006). The significance of reduced
forms in L2 pedagogy. In J. D. Brown & K.
Kondo‐Brown (Eds.), Perspectives on teaching
connected speech to second language speakers
(pp. 17–25). Honolulu: University of Hawai’i,
National Foreign Language Resource Center.
Johnson, K. (1982). Communicative syllabus
design and methodology. Oxford: Pergamon
Press.
Kelch, K. (1985). Modified input as an aid to
comprehension. Studies in Second Language
Acquisition, 7, 81–89.
Lam, Y. K. (2002). Raising students’ awareness
of the features of real‐world listening input. In
J. Richards & W. Renandya (Eds.), Methodolo-
gy in language teaching: An anthology of current
practice. Cambridge, UK: Cambridge Univer-
sity Press.
Larson‐Hall, J. (2010). A guide to doing statis-
tics in second language research using SPSS.
New York: Routledge.
Lee, J. F., & VanPatten, B. (2003). Making
communicative language teaching happen (2nd
ed.). New York: McGraw‐Hill.
Lund, R. (1991). A comparison of second lan-
guage listening and reading comprehension.
Modern Language Journal, 75, 196–204.
Matsuzawa, T. (2006). Comprehension of En-
glish reduced forms by Japanese business peo-
419
ple and the effectiveness of instruction. In J. D.
Brown & K. Kondo‐Brown (Eds.), Perspectives
on teaching connected speech to second language
speakers (pp. 59–66). Honolulu: University of
Hawai’i, National Foreign Language Resource
Center.
McCarthy, M. (2005). Fluence and confluence:
What fluent speakers do. The Language Teach-
er, 29, 26–28.
McCarthy, M., & Carter, R. (1995). Spoken
grammar: What is it and how can we teach it?
ELT Journal, 49, 207–218.
McCarthy, M., & Carter, R. (2001). Ten criteria
for a spoken grammar. In E. Hinkel & S. Fotos
(Eds.), New perspectives on grammar teaching
in second language classrooms (pp. 51–75).
Mahwah, NJ: Lawrence Erlbaum Associates.
Meinardi, M. (2009). Speed bumps for authen-
tic listening material. ReCALL, 21, 302–318.
Messick, S. (1989). Validity. In R. Linn (Ed.),
Educational measurement (3rd ed., pp. 13–
103). New York: American Council on Educa-
tion and Macmillan.
Messick, S. (1996). Validity and washback in
language testing. Language Testing, 13, 242–
256.
National Council of State Supervisors for Lan-
guages and the American Council on the
Teaching of Foreign Languages. (2013).
NCSSL‐ACTFL Can‐do statements: Progress in-
dicators for language learners. Alexandria, VA:
ACTFL.
Pica, T., Young, R., & Doughty, D. (1987). The
impact of interaction on comprehension. TE-
SOL Quarterly, 21, 737–758.
Rashtchi, M., & Afzali, M. (2011). Spoken
grammar awareness raising: Does it affect the
listening ability of Iranian EFL learners? Stud-
ies in Second Language Learning and Teaching,
1, 515–531.
Read, J. (2002). The use of interactive input in
EAP listening assessment. Journal of English for
Academic Purposes, 1, 105–119.
Richards, J. (2006). Materials development
and research: Making the connection. RELC
Journal, 37, 5–26.
Richards, J., & Rodgers, T. (2001). Approaches
and methods in language teaching (2nd ed.).
Cambridge, UK: Cambridge University Press.
Rost, M. (2011). Teaching and researching listen-
ing (2nd ed.). Harlow, UK: Pearson Education.
Rubin, A. (1980). A theoretical taxonomy
of the difference between oral and written
420 FALL 2014

language. In R. Spiro, B. Bruce, & W. Brewer Voss, B. (1979). Hesitation phenomena as

(Eds.), Theoretical issues in reading comprehen-
sion (pp. 411–438). Hillsdale, NJ: Erlbaum.
Shohamy, E., & Inbar, O. (1991). Validation of
listening comprehension tests: The effect of
text and question‐type. Language Testing, 8,
23–40.
Tannen, D. (1982). The oral/literate continu-
um in discourse. In D. Tannen (Ed.), Spoken
and written language: Exploring orality and lit-
eracy (pp. 1–33). Norwood, NJ: Ablex.
Thompson, S. (2003). Text‐structuring meta-
discourse, intonation and the signaling of or-
ganization in academic lectures. Journal of
English for Academic Purposes, 2, 5–20.
Vandergrift, L. (2007). Recent developments
in second and foreign language listening com-
prehension research. Language Teaching, 40,
191–210.
Vandergrift, L., & Goh, C. (2009). Teaching
and testing listening comprehension. In M.
Long & C. Doughty (Eds.), The handbook of
language teaching (pp. 393–411). Malden, MA:
Wiley‐Blackwell.
Vandergrift, L., & Goh, C. (2012). Teaching
and learning second language listening: Meta-
cognition in action. New York: Routledge.
sources of perceptual errors for non‐native
speakers. Language and Speech, 22, 129–144.
Wagner, E. (2002). Video listening tests: A pilot
study (Working Papers in TESOL and Applied
Linguistics, Teachers College, Columbia Uni-
versity, 2). Retrieved October 2, 2012, from
http://journals.tc‐library.org/index.php/tesol/
article/view/7
Wagner, E. (2006). Utilizing the visual channel:
An investigation of the use of video texts on tests
of second language listening ability (Unpub-
lished doctoral dissertation). Teachers Col-
lege, Columbia University, New York.
Wagner, E. (2013). Assessing listening. In A.
Kunan (Ed.), Companion to language assessment
(Vol. 1, pp. 47–63). Oxford: Wiley‐Blackwell.
Wagner, E. (2014). Using unscripted spoken
texts to prepare L2 learners for real world
listening. TESOL Journal, 5, 288–311.
Widdowson, H. G. (2003). Defining issues in
English language teaching. Oxford: Oxford
University Press.
Submitted December 12, 2013
Accepted May 12, 2014

APPENDIX

Test Instrument
Text 1—“A Room for Rent”
Directions: You will hear a conversation between two people, Pilar and Eunice. Eunice wants
to rent a room in Pilar’s house. You will hear the recording only once. Then you will have two
and a half minutes to answer the questions. Circle the letter of the correct response.

1. Eunice wants to rent the room for only three months because she ________.

a. is getting married in three months

b. cannot pay more than three months’ rent
c. is studying at university for three months
d. is going to live with her parents after three months

2. Eunice is interested in this apartment because ________.

a. it is close to the university

b. all utilities are included in the rent
c. the rent is lower than other apartments
d. the rooms are large, and she can use the kitchen
Foreign Language Annals  VOL. 47, NO. 3 421

3. Pilar wants to rent the room to a person who ________.

a. does not have pets

b. can pay rent on time
c. does not stay home much
d. respects others and knows how to be a good roommate

4. On a typical day, Eunice ________.

a. attends classes and does not return home until very late
b. has to go to work every day and then go to college at night
c. goes out dancing and drinking at night, and wakes up very late in the mornings
d. studies in her room in the morning, goes to her classes in the afternoon, and goes to
work on weekends

5. The main rule that Pilar has for her renters is that they ________.

a. cannot have overnight guests

b. can use the kitchen, but they have to keep it clean
c. cannot use the refrigerator, except for yogurt and similar types of food
d. have to listen to music quietly in order not to disturb the other people who are studying

6. If Eunice wants to use the phone, what must she do?

a. She must buy prepaid phone cards.

b. She must use her own mobile phone.
c. She can only use it when her roommates are not home.
d. She can only use it between 7 o’clock in the morning and 10 o’clock at night.

7. How often will Eunice use the kitchen?

a. Only for breakfast.

b. Only during the weekends.
c. She will eat at the university every day.
d. For dinner most nights of the week between 8 and 10.

8. What kind of owner is Pilar?

a. She does not like pets.

b. She only cares about money.
c. She does not care what the occupants do in private.
d. She only wants to rent the room to people who are responsible and considerate.

Text Two—“A Friend Goes on Vacation”

1. What is the main problem that Eunice has with the package?

a. She will not be home when it arrives.

b. She does not know when it will arrive.
422 FALL 2014

c. She cannot go to the post office to get it.

d. She does not have the address to send it to.

2. Pilar is jealous of Eunice because Eunice ________.

a. has a handsome husband

b. has a very large apartment
c. has a sister who can help her
d. is going on vacation to Colombia

3. Eunice asks Pilar to go to the apartment instead of her sister because her________.

a. sister does not like dogs

b. sister will be in Bogota this week
c. sister always forgets to water the plants
d. sister cannot go to the apartment at night

4. Eunice wants Pilar go to the apartment on Wednesday, Thursday, Friday, and

Saturday to ________.

a. water plants
b. walk the dog
c. pick up her mail
d. take care of her sister

5. Pilar says, “During the walk, I will walk also, getting some exercise” because Pilar ________.
a. wants to lose weight
b. thinks it is safer to walk with a dog
c. thinks it is very important to exercise every day
d. wants to let Eunice know that it’s not a problem to help her

6. How do they solve the problem of the key?

a. Eunice will leave it under the doormat.
b. Eunice will send a copy of the key to Pilar by mail.
c. Eunice’s sister can open the door of the apartment for Pilar.
d. Eunice’s sister is going to Pilar’s house and will leave a copy of the key for her.

7. Why does Pilar not have plants or pets?

a. She is allergic to them.
b. Her apartment is too small.
c. They make it difficult to go on vacation.
d. She does not have the patience to care for them.

8. Why does Pilar agree to do all these things for Eunice?

a. Pilar loves Eunice’s dog.

b. Pilar is a good friend of Eunice.
c. Eunice will help Pilar after the holidays.
d. Eunice has done many things for Pilar in the past.
Reproduced with permission of the copyright owner. Further reproduction prohibited without
permission.