See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/327670831

The revision and transfer effects of direct and indirect comprehensive corrective
feedback on English-as-a-second-language (ESL) students’ writing

Article  in  Language Teaching Research · December 2018

CITATIONS READS

8 783

2 authors:

Khaled Karim Hossein Nassaji

United Arab Emirates University University of Victoria
11 PUBLICATIONS   80 CITATIONS    116 PUBLICATIONS   4,375 CITATIONS   

SEE PROFILE SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Written Corrective Feedback View project

using technology in L2 pragmatics instruction View project

All content following this page was uploaded by Khaled Karim on 05 October 2018.

The user has requested enhancement of the downloaded file.

802469
research-article2018
LTR0010.1177/1362168818802469Language Teaching ResearchKarim and Nassaji

LANGUAGE
TEACHING
Article RESEARCH

Language Teaching Research

The revision and transfer

1–21
© The Author(s) 2018
Article reuse guidelines:
effects of direct and indirect sagepub.com/journals-permissions
DOI: 10.1177/1362168818802469
https://doi.org/10.1177/1362168818802469
comprehensive corrective journals.sagepub.com/home/ltr

feedback on ESL students’

writing

Khaled Karim
United Arab Emirates University, United Arab Emirates

Hossein Nassaji
University of Victoria, Canada

Abstract
This study investigated the short-term and delayed effects of comprehensive written corrective
feedback (WCF) on L2 learners’ revision accuracy and new pieces of writing (i.e., the transfer
effect of feedback). Three types of feedback were compared: direct feedback and two types
of indirect feedback that differed in their degree of explicitness (i.e., underlining only and
underlining+metalinguistic cues). Fifty-three intermediate level learners of English as a second
language (ESL) were divided randomly into four groups: One direct, two indirect, and a control
group. Students produced three pieces of writing from different picture prompts and revised
them over a three-week period. Each group also produced a new piece of writing two weeks
later. The study included seven sessions: Writing 1, revision of Writing 1, Writing 2, revision of
Writing 2, Writing 3, revision of Writing 3, and Writing 4 (delayed writing). The results showed
that all the three feedback groups significantly outperformed the control group in revision tasks.
Some short-term accuracy improvements were also found on new pieces of writing for direct and
underlining+metalinguistic feedback, but the effects were largely non-significant.

Keywords
comprehensive corrective feedback, direct and indirect feedback, focused feedback, SLA,
unfocused feedback, written corrective feedback

Corresponding author:
Khaled Karim, Department of Linguistics, College of Humanities and Social Sciences, United Arab Emirates
University, P.O. Box 15551, Al Ain, United Arab Emirates
Email: khaledk@uaeu.ac.ae
2 Language Teaching Research 00(0)

I Introduction
Corrective feedback (CF) is a central component of second language (L2) writing pro-
grams around the world. Although many teachers may consider that CF helps L2 learners
improve their writing accuracy (Ferris, 2004), the efficacy of CF has been a much debated
and controversial topic in the field of second language acquisition (SLA). The debate over
the usefulness of written corrective feedback (WCF) heightened with Truscott’s arguments
in 1996 that WCF is ineffective and even harmful and the counter responses to his argu-
ment by a number of other researchers that CF is effective (e.g., Bruton, 2009, 2010;
Chandler, 2003, 2004; Ferris, 1999, 2004, 2006; Ferris & Helt, 2000; Lyster, Lightbown, &
Spada, 1999). Citing a number of studies, Truscott (1996) contended that there is no clear
value of error correction. Using the same research studies, Ferris (1999, 2004), on the other
hand, argued that ‘some potentially positive research evidence on the effects of grammar
correction [had been] overlooked or understated’ (Ferris, 2004, p. 50) by Truscott. She
argued that while some participants in those studies did not benefit from feedback, many
others did. A substantial amount of research since then has been conducted to examine
whether and to what extent WCF assists L2 writing.
Most of the early studies examined the usefulness of feedback versus no feedback.
While some of these studies showed no significant benefit for feedback (e.g., Kepner,
1991; Polio, Fleck, & Leder, 1998; Semke, 1984), the majority concluded that error correc-
tion was indeed beneficial (e.g., Fathman & Whalley, 1990; Ferris, 1997; Lalande, 1982;
Robb, Ross, & Shortreed, 1986; Sheppard, 1992). However, most of these studies focused
on revision accuracy with few measuring the effects of feedback on new pieces of writing
(Ferris, 2010, 2012). Although the revision of an incorrect form may indicate that learners
are able to repair a previously-made error, it does not necessarily indicate that learners are
able to use the correct form when producing new pieces of writing (Ferris, 2004). Thus, to
show feedback has any effects on learning, it is important to examine not only whether
feedback has any effects on L2 revision accuracy but also whether these effects are carried
over to new pieces of writing (Sheen, 2007; Truscott, 1999).
Studies of CF have investigated not only the effect of feedback itself but also that of
different types of WCF. This strand of research has examined the question of whether
some types of CF have greater effects than others (e.g., whether direct and indirect CF
yield different outcomes). Furthermore, many of the previous studies have used feedback
in a focused manner targeting a single type of error rather than a wide range of errors.
While studies of feedback on single errors may show that feedback is effective in improv-
ing learners’ accuracy of the selected target features, the ecological validity of such
research has been questioned as teachers often provide feedback on diverse errors they
observe rather than on errors of a single type (Ferris, 2012; Storch, 2010). The present
research therefore implemented comprehensive (or unfocused) WCF, that is, feedback
that targeted a range of errors rather than single-type errors.
Although an increasing number of WCF studies have examined the effects of differ-
ent types of feedback, much research is still needed to examine not only the differential
effects of feedback types but also under what conditions they enhance L2 development
(Ferris, 2010; Frear & Chiu, 2015). To contribute to the research in this area the present
study examined the efficacy of three types of feedback, notably direct feedback and two
Karim and Nassaji 3

types of indirect feedback, and investigated these effects on both learners’ revision accu-
racy and subsequent, new pieces of writing. The study adopted a research design that
involved multiple feedback and writing sessions, tracking the effects of feedback on
revisions of the previous texts and also on new pieces of writing over time.

II Literature review
A large amount of research has investigated the role of CF in language learning. Although
there was some controversy in the 1990s and early 2000s over the effects of CF, there
now appears to be a consensus that CF is useful. Many of the early studies focused on the
effects of feedback versus no feedback (e.g., Ashwell, 2000; Fathman & Whalley, 1990;
Ferris & Roberts, 2001; Kepner, 1991; Polio et al., 1998; Semke, 1984). Later studies
also compared the effects of different types of feedback (e.g., Bitchener, 2008; Bitchener,
Young, & Cameron, 2005; Bitchener & Knoch, 2009a, 2009b; Mirzaii & Aliabadi, 2013;
Sheen, Wright, & Moldawa, 2009; Shintani & Ellis, 2013).
Two major types of feedback that have received much attention are direct and indirect
feedback. Direct feedback refers to feedback strategies that provide the correct form.
Indirect feedback, however, indicates that an error has occurred but does not provide the
correction. While there seems to be a consensus on the beneficial effects of feedback in
general, there has been a controversy on what type of feedback is more effective (Nassaji,
2016). Some researchers, for example, have argued that direct feedback is more effective
than indirect feedback as it clearly indicates how the error should be corrected (e.g.,
Bitchener, 2008; Ellis, Sheen, Murakami, & Takashima, 2008; Sheen, 2007). Others,
however, have contended that indirect feedback is superior to direct feedback in the long
run as it engages students in problem-solving learning and hence helps them to become
independent learners (Ferris, 2003, 2006).
A number of studies have empirically compared the effectiveness of direct and indi-
rect feedback (e.g., Bitchener, 2008; Bitchener et al., 2005; Bitchener & Knoch, 2009a,
2009b; Lalande, 1982; Mirzaii & Aliabadi, 2013; Robb et al., 1986; Semke, 1984; Sheen,
2007; Sheen et al., 2009, Sheppard, 1992; Shintani & Ellis, 2013). While some have
found no difference between these feedback types (e.g., Robb et al., 1986; Semke, 1984),
others have reported that students who receive indirect feedback make greater improve-
ment than those who receive direct feedback (e.g., Lalande, 1982; Sheppard, 1992). On
the other hand, some have displayed an advantage for direct feedback, particularly when
combined with metalinguistic information (e.g., Bitchener et al., 2005; Bitchener, 2008;
Bitchener & Knoch, 2008, 2009a, 2009b; Ellis et al., 2008; Shintani & Ellis, 2013).
Metalinguistic feedback has been considered to be effective because it facilitates under-
standing of the nature of the error (Bitchener, 2008; Bitchener et al., 2005; Ellis et al.,
2008; Nassaji, 2015). Bitchener et al. (2005) found superiority of the combination of
direct oral metalinguistic feedback and direct written feedback over direct written feed-
back alone. Bitchener (2008) also demonstrated that the two groups that received written
or oral metalinguistic feedback along with direct feedback outperformed the group which
received direct feedback only.
Most of the studies that have found an effect for CF, however, have targeted only a
limited number of errors, rather than a range of target forms. In other words, they have
4 Language Teaching Research 00(0)

used the feedback in a focused manner rather than a comprehensive manner (Ferris,
2012). Focused feedback has been assumed to be more effective than comprehensive (or
unfocused) feedback because the former directs learners’ attention more effectively to a
(pre-)selected target form (e.g., Bitchener & Knoch, 2008; Ellis et al., 2008; Nassaji,
2015; Sheen, 2007; Sheen et al., 2009). Ellis et al. (2008), for example, argued that,
‘learners are likely to attend to correction directed at a single (or limited number of) error
type(s) and more likely to develop a clearer understanding of the nature of the error and
the correction needed’ (p. 356). However, although focusing feedback on a few or single
errors may be effective, the ecological validity of such studies is questionable (Ferris,
2010), as L2 writing teachers do not usually correct only one type of error at a time.
Since L2 learners typically make different kinds of errors, teachers may be more keen to
adopt an approach that is responsive to this reality. Storch (2010) pointed out that
‘researchers who focus only on one structure may find few instances of such structures
in their students’ writing’ (p. 41). Thus, if the aim of WCF is to have direct practical
relevance, the use of feedback should be more representative of what goes on in L2
classrooms.
As comprehensive WCF can be considered to better reflect the reality of most L2
classrooms, some recent studies have attempted to empirically examine and also com-
pare its effect with that of focused feedback (Frear & Chiu, 2015; Hartshorn et al., 2010;
Liu, 2008; Sheen et al., 2009; Storch & Wigglesworth, 2010; Truscott & Hsu, 2008; Van
Beuningen et al., 2008, 2012). The results of these studies have been mixed. Van
Beuningen et al. (2008), for example, examined whether comprehensive feedback helps
the improvement of accuracy in writing from an initial task to its revision and also to a
later writing task (i.e., new text). The study examined the effects of both direct feedback
and indirect feedback, and found that both feedback types when provided on all errors
led to significant accuracy gains. In a subsequent study, Van Beuningen et al. (2012)
investigated the effects of these feedback types on both revisions and long-term accuracy
development. Again findings demonstrated that both direct and indirect WCF led to
improved accuracy in both the revision and in the new pieces of writing. Sheen et al.
(2009) investigated the differential effects of focused and unfocused feedback on the
acquisition of grammatical forms. The target structures for the focused CF treatment
were English definite and indefinite articles, and the unfocused CF treatment targeted a
range of structures including copula ‘be’, regular past tense, and irregular past tense and
preposition. The study found no difference between the two types of feedback. More
recently, Frear and Chiu (2015) also compared the effects of WCF on the accuracy of a
limited number of grammatical structures (i.e., weak verbs) to WCF on all errors. They
found that both the focused and unfocused (or comprehensive) WCF treatments brought
about accuracy gains in their study.
However, the number of studies comparing focused and comprehensive feedback is
still too small to allow a firm conclusion. Furthermore, the mixed findings about the
effect of such feedback may be due to the complexity of WCF and the various factors
that may influence its effectiveness. Indeed, in a recent meta-analysis, Kang and Han
(2015) found that the efficacy of WCF was influenced greatly by factors such as setting
as well as learners’ level of linguistic knowledge. Another reason for the mixed results
could be that these studies have focused on different grammatical forms. Research has
Karim and Nassaji 5

suggested that the type of grammar structure may mediate the effectiveness of feedback
(e.g., Bitchener et al., 2005; Ferris, 2006; Ferris & Roberts, 2001; Van Beuningen et al.,
2012). There have also been differences in the way focused and unfocused WCF have
been defined. For example, some studies have defined focused feedback as feedback on
a limited number of errors types (e.g., prepositions, past simple tense, and definite arti-
cles) (Bitchener et al., 2005) while others have focused more tightly on just one single
type of error (e.g., definite or indefinite articles) (Ellis et al., 2008; Sheen, 2007; Sheen
et al., 2009).
Another issue with most feedback studies concerns the length and duration of the
feedback treatment (Storch, 2010). Indeed, most recent studies in this area (e.g.,
Bitchener, 2008; Bitchener & Knoch, 2008, 2009a, 2009b; Sheen, 2007; Van Beuningen
et al., 2008) have used ‘one-shot’ designs, where feedback is provided only on one occa-
sion and on a single text (Liu & Brown, 2015). One-shot treatments have important limi-
tations. Although they may be methodologically easier to implement, they are problematic
in terms of their implications for learning. Since language learning requires extensive
exposure to feedback, if a study finds no effect for feedback, it cannot be concluded that
feedback is not effective (Storch, 2010), because the exposure to feedback may simply
have been too short. Therefore, to examine feedback effectiveness, studies need to inves-
tigate it over a more extended period, using multiple feedback sessions.

III The present study

To contribute to the studies on L2 WCF, the present study was designed to address some
of the issues discussed above. First, to determine whether the effects of feedback transfer
to new pieces of writing, it examined the effects of feedback not only on students’ revi-
sion accuracy but also on the accuracy of writing new texts. Second, to contribute to the
strand of research on what type of feedback is most effective, the study examined the
effects of three different feedback types including direct as well as two subtypes of indi-
rect feedback, which differed in terms of feedback explicitness. One type of indirect
feedback involved underlining only and the other added metalinguistic cues. As noted
earlier, while some previous studies have examined the various forms of direct feedback
such as direct correction only or direct correction with oral or written metalinguistic
explanation (e.g., Bitchener, 2008; Bitchener et al., 2005; Bitchener & Knoch, 2009a,
2009b; Sheen, 2007; Sheen et al., 2009), few studies have compared the effectiveness of
different types of indirect feedback. Thus, Ferris and Roberts (2001) pointed out that
there is a lack of evidence in research literature about the level of explicitness that is
necessary for WCF to help students:

Specifically, when teachers mark student errors, do they need to indicate the type of error
(wrong verb tense, omitted obligatory article, run-on, etc.) the student has made, or is it
adequate for the teacher to simply underline or circle an erroneous form, leaving it to the
student to diagnose and correct the problem? (Ferris & Roberts, 2001, p. 162)

Unlike most previous studies (e.g., Truscott & Hsu, 2008; Van Beuningen et al., 2012,
Shintani & Ellis, 2013; Shintani, Ellis, & Suzuki, 2014) that provided the CF in only one
6 Language Teaching Research 00(0)

session, the current study involved multiple feedback sessions. This allowed examining
the effects of feedback over time. As noted earlier, one recent line of inquiry has been
whether CF should be selective and focus on one or a limited number of error types at a
time, or be more comprehensive. Although there is certainly no definitive answer to this
question, the present study addressed the role of comprehensive feedback because it
seems to be more ecologically valid (it seems to correspond to the approach taken by
many teachers).

IV Research questions
Given the above aims, the following research questions were addressed:

1. Does comprehensive WCF have any effect on the accuracy of L2 learners’

revision?
2. Does comprehensive WCF have any effect on the accuracy of L2 learners’ new
pieces of writing?
3. Does the effect of the feedback on revision versus new pieces of writing depend
on the type of feedback?
4. Is the effect of the feedback, if any, maintained over time?

V Method
1 Participants
Fifty-three adult intermediate students of English as a second language (ESL) studying
at two ESL schools in Canada participated voluntarily in this study. All of the partici-
pants were enrolled in the intermediate level general English classes. They had been
required to take a placement test to be placed at the appropriate level. Intermediate level
students were selected as they were expected to have sufficient writing proficiency
needed to produce the pieces of writing required. Also, intermediate level students are
more likely to make a range of grammatical and lexical errors in writing (Brown, 2007).
Learners’ proficiency level was determined based on a placement test learners had taken
prior to the study. Although the curricula in these two schools were not identical, the
placement tests designed by the schools were similar and assessed all four proficiency
skills (i.e., reading, writing, speaking and listening). In both schools, the reading compre-
hension portion of the test consisted of two to three reading texts followed by multiple
choice as well as open-ended questions. The writing portion included a picture descrip-
tion and a paragraph-writing task. The speaking and listening tests were administered in
an interview format and included picture descriptions and conversations on given topics.
Most of the participants were of East Asian origin: 39% from South Korea (n = 21), 34%
from Japan (n = 18), and 7.5% from Taiwan (n = 4). Other participants were from Brazil
(n = 3) Saudi Arabia (n = 2), Switzerland (n = 2), Germany (n = 1), Mexico (n = 1),
and Colombia (n = 1). The participants varied in age from 18 to 40 (M = 25.43, SD =
5.0); the majority (81%) were in the age range of 20 to 30. The youngest two were 18 and
19 years old, and there was only one 40-year old participant. There were 17 male (32%)
Karim and Nassaji 7

and 36 female (68%) students. Out of the 53 participants, only 7 reported they had taken
an ESL writing course before.
The feedback sessions were conducted after regular class time. To avoid the potential
effect of (other) writing instruction on the study results, only the students who were not
enrolled in any ESL writing classes during the data collection period participated in study.

2 Research design
Participants were randomly divided into four groups: direct WCF (n = 14), indirect
underline+metalinguistic WCF (n = 14), indirect underline only WCF (n = 13), and no
WCF (n = 12). In direct WCF the errors were corrected by crossing out the erroneous
forms and providing the corresponding target forms above the errors as illustrated by (1).
Underline+metalinguistic WCF was provided by underlining the erroneous forms
and providing metalinguistic information above and below the errors, and in some cases
in the margins without providing the correct form, as illustrated by (2). Underline by
only WCF was provided in the form of underlining the erroneous forms only, which is
illustrated by (3).

(1) Direct WCF:

was drinking
One day, an Emu drink water in front of a house.

(2) Underline+metalinguistic WCF (indirect feedback)

Wrong tense

The Emu then start to follow the man everywhere around the house.

(3) Underline only WCF (indirect feedback)

Emu was very thusty.

3 Treatment procedures
The study included seven sessions: text 1, revision of text 1, text 2, revision of text 2, text
3, revision of text 3, and text 4 (delayed writing). The seven sessions were completed
over a period of six weeks (Table 1). The writing tasks included four narratives written
based on four picture prompts selected from ESL textbooks. Each student composed four
pieces of writing based on the pictures.
The picture prompts used for text 1 consisted of a sequence of eight pictures. The
pictures depicted the story of an Emu who fell in love with a man when he gave it food
out of care, and how the man got rid of the Emu after some ordeals. The picture prompts
for text 2 were a sequence of 10 pictures, which showed the story about a bald man who
was followed and hit by two individuals while walking on the street. The man finally
reached a house but was again hit by a woman. The picture prompts for text 3 included
11 pictures. The pictures showed the story of three friends who planned a whole night out
and how they spent their eventful night together. The picture prompts for the final writing
(text 4) included 10 pictures. The pictures narrated the story of a young man who was
8 Language Teaching Research 00(0)

Table 1. The treatment procedure.

Group A Group B Group C Group D

(direct WCF) (underline+meta (underline (no WCF)
linguistic WCF) only WCF)
Day 1 Text 1 Text 1 Text 1 Text 1
Week 1 Day 2 CF + revision CF + revision of CF + revision of Revision of
of text 1 text 1 text 1 text 1
Week 2 Day 1 Text 2 Text 2 Text 2 Text 2
Day 2 CF + revision CF + revision of CF + revision of Revision of
of text 2 text 2 text 2 text 2
Week 3 Day 1 Text 3 Text 3 Text 3 Text 3
Day 2 CF + revision CF + revision of CF + revision of Revision of
of text 3 text 3 text 3 text 3
Week 6 Text 4 Text 4 Text 4 Text 4

Note. WCF = written corrective feedback.

almost hit by the same car every time he was crossing a road, and who finally planned to
teach the car driver a lesson. All four picture prompts were designed to be narrative so
that they would elicit narrative writing. Following previous studies (e.g., Truscott & Hsu,
2008; Van Beuningen et al., 2008, 2012) the students were not allowed to use dictionaries
or discuss the pictures with other members of the group during the writing sessions.
Production of texts 1, 2, and 3 took place in weeks 1, 2 and 3 of the study, respectively.
The fourth text was written two weeks later. The two-week interval was inserted to examine
the delayed effect of WCF. In day one of week one, participants in all four groups produced
text 1. They were given 30 minutes for this writing task, and were instructed to write at least
10 lines in length or a minimum of 100 words. The mean of total number of words produced
in the texts was 118.60 and the standard deviation was 37.58. The mean text length and
standard deviation per group was: direct (Mean = 112.52; SD = 16.23); underline only
(Mean = 128.47; SD = 53.88); underline+metalinguistic (Mean = 120.16; SD = 33.43);
no CF (Mean = 113.17; SD = 36.75). Once the participants finished writing their texts, the
texts were collected and feedback was provided on all errors. Participants in the treatment
groups received direct WCF, underline+metalinguistic WCF, and underline only WCF,
respectively. Participants in the control group did not receive any feedback. The texts of all
the groups were returned to the participants the next day (day 2 of the study).
Participants in all groups were asked to review and revise their texts. Participants in the
three treatment groups were given ten minutes to review their texts with the corrections
and 30 minutes to revise the first draft of the text 1. They were asked to revise all the errors
corrected. The control group was also given 10 minutes to review their texts and then 30
minutes to revise them. Following Storch and Wigglesworth (2010), to prevent students
from simply copying the corrections or feedback from their original texts, the direct WCF
group received corrections on photocopies of their texts, which were taken away from
them after the ten minutes of review. The original texts, which did not contain WCF, were
then provided to them for revisions. The revised texts were then collected.
Karim and Nassaji 9

4 Scoring procedures
Every text was scored for writing accuracy. Following previous studies (e.g., Chandler,
2003, Truscott & Hsu, 2008, Van Beuningen et al., 2012), the study used an error ratio to
measure overall accuracy. The error ratio was calculated by the total number of errors
divided by the total number of words written × 100. A ratio was used in order to take into
account the differences in the length of each text. As for counting the errors, all errors on
the revised texts were counted. The same procedure was also followed for the new pieces
of writing. The reduction of errors, both in revisions and in new texts was considered to
be partly due to the effect of WCF.
When analysing the data, the effects of feedback on revision accuracy was examined
first. For that purpose, the effect of feedback on editing the same piece of text was meas-
ured (for example, text 1 was compared to the revisions of text 1). Then the efficacy of
the feedback on the new pieces of writing was examined. This was called the transfer
effect of feedback and following previous research (e.g., Truscott & Hsu, 2008; Van
Beuningen et al., 2012) was defined in terms of the reduction of the error rates from one
writing to another. For that purpose, the effect of feedback on each text was examined on
the accuracy of the subsequent text (for example, the accuracy of text 2 was compared to
that of text 1). Both revision and transfer effects were examined on overall accuracy as
well as the two general categories of grammatical and non-grammatical errors (e.g., Van
Beuningen, 2011). The grammatical error category included errors in the syntax and
morphology of a sentence (Van Beuningen, 2011). The non-grammatical category
included word choice errors, capitalization errors, spelling errors, punctuation errors and
other non-grammatical errors.
Intra-rater and inter-rater reliability of the scoring of each error category were then
calculated. In order to check the intra-rater reliability, 50% of the texts were graded a
second time by the same researcher 6 months after the initial scoring and analysis of all
data. Pearson correlation coefficients for the overall scores at two times for the four writ-
ing tasks were: 0.99, 0.99, 0.94, and 0.97 for text 1, text 2, text 3 and text 4 respectively.
The Pearson correlation coefficients for the scores at two times for the three revision
tasks were: 0.99, 0.99, and 0.99 for revision 1, revision 2, and revision 3 respectively. To
check the inter-rater reliability, another scorer, an ESL teacher and a native speaker of
English, scored 20% of the texts individually. Pearson correlation coefficients for the two
scores in the four writing tasks were: 0.98, 0.98, 0.86, and 0.92 for text 1, text 2, text 3
and text 4, respectively. The Pearson correlation coefficients in the three revision tasks
were: 0.99, 0.87, and 0.95 for revision 1, revision 2, and revision 3 respectively.

VI Analysis
To examine the effects of feedback on students’ revisions, a comparison was made
between the four groups on their revision accuracy. For the effect of feedback on new
texts, a comparison was made between the groups on their accuracy of text 1 compared
to text 2, text 2 compared to text 3, and text 3 compared to text 4 (delayed text). Both one
way and mixed methods analysis of variance (ANOVA) were used to find out the effects
of feedback. Separate analyses were conducted for revision effects and transfer effects of
10 Language Teaching Research 00(0)

Table 2. Descriptive statistics: Error rates* of the four groups in text 1 (session 1).

Groups Error rates in text1 (session 1)

M SD
1. Underline+metalinguistic 17.97 6.20
2. Underline only 23.14 10.58
3. Direct WCF 28.53 10.48
4. No WCF 20.65 4.85

Notes. WCF = written corrective feedback. * In percentage (total number of errors / total number of
words × 100).

feedback (i.e., the effects of feedback on new pieces of writing) and also for grammatical
and non-grammatical errors. When a significant difference was found, separate one-way
ANOVAs were conducted to determine where the differences lay. Effect sizes using par-
tial eta-squared (ηp2) were calculated when the results were signficant.

VII Results
First, a one-way ANOVA was performed on the error rates for text 1 to find out if the
students in the four groups began the study with similar writing proficiency. The results
showed a significant difference between the means of error rates among the groups (F(3,
49) = 3.89, p = .014). This suggests that the four groups were not similar in terms of
their writing proficiency (Table 2).
Since there was a significant difference between the means of error rates of the groups in
their first piece of text, gain scores were calculated for both revision and transfer effects. In
other words, we subtracted the error rates of the revisions from the error rates of the first
drafts, and we subtracted the error rates from the later pieces of writing from the error rates
of the earlier ones. Gain analysis has been suggested as a method of analysing pretest–
posttest data when there is a significant difference in the pretest data (Larson-Hall, 2010).
Gain analysis has been preferred to other methods such as ANCOVA because the former
shows the exact improvement by calculating the difference between the pre- and the post-
tests (Maxwell & Delaney, 1990) and it is relatively easy to interpret (Lindstromberg, 2016).

1 Revision effects
For revision accuracy, we first examined feedback effects on overall revision accuracy of
all errors. To do so, a two-way mixed methods ANOVA with time as a within-subject
variable and condition (feedback types) as a between-subject variable was performed.
Before the ANOVAs were conducted, the assumptions of Homogeneity of Variance and
Normality of Distribution were also checked by using Levene’s test and Kolmogorov–
Smirnov test and the results were not significant: (Levene’s test: Gain 1, p = .682; Gain
2, p = .129; Delayed Gain p = .290); (Kolmogorov–Smirnov test: Gain 1, p = .200;
Gain 2, p = .200; Delayed Gain, p = . 119). These indicate that the data met the above
two assumptions required by ANOVA.
Karim and Nassaji 11

The results demonstrated that time had a significant main effect (F(2, 98) = 6.09;
p = .003), suggesting that learners’ peformance differed across different writing ses-
sions. The results also showed that condition (i.e., feedback types) had a significant
effect on revisions (F(3, 49) = 18.65, p = .000). The effect sizes using partial eta-
squared (ηp2) were .11 and .53 for time and condition respectively, representing large
effects. (Following Cohen (1988), values between .01–.05 are considered to be small
effect sizes, between .06 –.13 are considered medium, and values greater than or equal to
.14 are considered large effects.) There was no interaction between time and condition
(F(6, 98) = 1.62; p = .148), suggesting that the effectiveness of different feedback types
did not vary across writing sessions. One-way ANOVAs were then conducted to find out
what feedback type was more effective at each time. The ANOVA revealed that revision
gains were significant in all 3 revision tasks with large effect sizes (in revision 1: F (3,
49) = 11.47, p < .001, ηp2 = .41; in revision 2: F (3, 49) = 11.86, p < .001, ηp2 = .42;
in revision 3: F (3, 49) = 10.04, p < .001, ηp2 = .38).
For revision 1, post hoc pairwise comparisons with Bonferroni correction revealed a
significant difference between the direct WCF and the underline+metalinguistic group,
between the direct WCF and the control group, and also between the direct WCF group
and the underline only group. In revision 2, a significant difference was found between
all the three feedback groups and the control group. The mean values also indicated that
students who received direct WCF performed better than those in the other two treatment
groups. However, the difference was not significant. As for revision 3, there was a sig-
nificant difference between the direct WCF and the control group, and also between the
direct feedback and underline+metalinguistic WCF. Underline only WCF group also
outperformed the control group.
Overall, students who received direct WCF showed a greater reduction in error rates
than students who did not receive any CF in all the three revision tasks followed by those
who received the underline only (in revisions 2 and 3) and underline+metalinguistic
feedback (revision 2 only) (see Figure 1). Table 3 presents the descriptive statistics for
overall accuracy gains in all three revision tasks.
We then examined the differential effects of feedback on the revision of grammatical
and non-grammatical errors. Table 4 presents the descriptive statistics for grammatical
and non-grammatical accuracy gains in three revision tasks. A mixed methods ANOVA
with type of error as a within-subject variable was conducted (including time and condi-
tion as within and between subject variables, respectively). The results confirmed that
time and condition had a significant effect on accuracy improvement in all three revi-
sions. The test further demonstrated that type of errors also had an effect (F(1, 98) =
8.87; p = .004, ηp2 = .15). However, types of errors and feedback types did not have any
interaction (F(3, 98) = 1.65; p = .189). In general, WCF was more effective on the revi-
sion of grammatical errors than non-grammatical errors.

2 Transfer effects (effects of feedback on new pieces of writing)

Similar to revision effects, for transfer effects, we first examined feedback effects on
overall accuracy and then on grammatical and non-grammatical categories. Table 5 dis-
plays the descriptive statistics for overall accuracy gain scores for all groups.
12 Language Teaching Research 00(0)

Figure 1. Overall accuracy gains in three revision tasks.

Table 3. Descriptive statistics: Overall accuracy gains in revision 1, 2 and 3 by group.

Revision 1 Revision 2 Revision 3

M SD M SD M SD
1. Underline+metalinguistic 12.04 6.78 11.85 5.51 8.30 4.82
2. Underline only 13.34 8.16 12.19 5.17 11.72 7.02
3. Direct WCF 22.20 8.43 14.44 5.82 17.20 10.86
4. No WCF 5.61 5.12 3.47 1.88 2.31 1.70

Note. WCF = written corrective feedback.

Like the revision effects, first a two-way mixed methods ANOVA was used with time as
a within-subject variable and feedback condition as a between subject variable. The results
demonstrated that time did not have any significant main effect (F(2, 98) = 2.08; p =
.130). There was no significant main effect for condition either (F(3, 49) = 1.73; p = .173).
However, there was an interaction of time and feedback condition (F(6, 98) = 2.50; p =
.027, ηp2 = .13), suggesting that the effects of different feedback types varied over time.
To examine the interaction effects, one-way ANOVAs were conducted to find out
where the effect was. These analyses revealed that the direct group showed a notable
increase in accuracy gains in text 2, but the difference among the groups did not reach
statistical significance (F(3, 49) = 2.38, p = .081). However, the accuracy gain scores in
the four groups varied significantly in text 3 (F(3, 49) = 3.26, p = .029, ηp2 = .16). Post-
hoc pairwise comparisons using Bonferroni tests revealed a significant difference
between underline+metalinguistic WCF and direct WCF group for gains in text 3.
Students who received underline+metalinguistic WCF showed a greater error-ratio
reduction than those who received direct WCF in the third text. In fact, rather surpris-
ingly, the error ratio increased slightly in the latter group. Underline+metalinguistic
Karim and Nassaji 13

Table 4. Descriptive statistics: Revisions of grammatical & non-grammatical errors.

Revision 1 Revision 2 Revision 3

M SD M SD M SD
A: Grammatical errors:
1. Underline+metalinguistic 6.89 4.10 7.79 4.22 5.07 4.72
2. Underline only 7.44 7.21 7.12 3.58 6.30 4.72
3. Direct WCF 13.98 6.20 8.5 4.75 10.72 7.43
4. No WCF 2.46 5.12 2.23 2.29 1.26 1.59
Total 7.90 6.96 6.57 4.48 6.00 6.04
B: Non-gammatical errors:
1. Underline+metalinguistic 5.15 4.96 4.05 2.35 3.22 2.58
2. Underline only 5.90 5.92 5.06 3.27 5.42 3.56
3. Direct WCF 8.21 4.15 5.88 4.83 6.47 4.71
4. No WCF 3.15 4.19 1.24 2.59 1.04 1.92

Note. WCF = written corrective feedback.

Table 5. Descriptive statistics: Short-term and delayed overall accuracy gains by group and
session.

Groups Text 1 to Text 2 to Text 3 to text

text 2 text 3 4 (delayed)

M SD M SD M SD
1. Underline+metalinguistic 1.66 7.77 5.21 7.11 1.77 5.61
2. Underline only 4.87 11.11 1.30 9.08 5.22 6.67
3. Direct WCF 10.88 8.46 –2.82 7.25 7.66 11.60
4. No WCF 4.36 10.14 3.47 3.83 2.16 5.09

Note. WCF = written corrective feedback.

WCF group also showed more improvement than underline only WCF group in text 3
but the difference was not significant. There were no significant differences in accuracy
gains between other groups in text 3. No significant difference was also found among the
groups in their gains in text 4 (delayed writing) (see Figure 2). To examine delayed
effects, we also compared accuracy gains from text 2 (week 2) to text 4 (week 6) and also
text 1 to 4 and found no significant difference among the four groups.
We then examined the transfer effects of feedback on grammatical and non-
grammatical errors. Table 6 displays the descriptive statistics for both error types in all
groups and across sessions. Mixed methods ANOVA with type of error as the within-
subject variable showed a significant effect for error type (F(1, 98) = 6.86; p = .01, ηp2
= .12), suggesting that improvement of accuracy as a result of feedback was influenced
by error types. In general, WCF was more effective on grammatical errors than non-
grammatical errors. It also showed an interaction between time and feedback types,
14 Language Teaching Research 00(0)

Figure 2. Overall accuracy gains per treatment (short-term and delayed)

confirming that the transfer effect of feedback types varied over time. The one-way
ANOVA revealed that the four groups significantly varied in the grammatical accuracy
gains from text 1 to text 2 (F(3, 49) = 2.80, p = .049, ηp2 = .15). Post-hoc multiple
comparisons using Bonferroni revealed that the students who received direct WCF sig-
nificantly outperformed students who received underline+metalinguistic WCF in text 2.
No significant difference was observed in grammatical accuracy gains among the four
groups in texts 3 and 4 (the delayed writing).

VIII Discussion
This study investigated the effects of different comprehensive WCF types on students’
revisions and new pieces of writing. The findings demonstrated that the feedback
improved learners’ accuracy with both direct and indirect feedback significantly reduc-
ing learner errors in subsequent revisions. These findings add to the results of previous
research that has found significant effects of WCF on revision tasks (e.g., Ashwell, 2000;
Bitchener, 2008; Bitchener & Knoch, 2008, 2009a; Ferris & Roberts, 2001; Sachs &
Polio, 2007). Previous studies, however, have used feedback on single errors (e.g.,
Bitchener, 2008; Bitchener & Knoch, 2008, 2009a, 2010b; Sheen, 2007, 2010a) or if on
multiple errors, the feedback was provided on one writing task (Ellis et al., 2008; Truscott
& Hsu, 2008; Van Beuningen et al., 2012). The present study used comprehensive feed-
back and also provided feedback on multiple writing tasks over time. Thus, it led to a
number of additional findings.
As for the revision effects, overall, direct WCF was more successful than indirect
feedback. Differences were also found in the effects of different types of indirect feed-
back. However, these effects varied over time. For example, while both the
underline+metalinguistic feedback group and the underline only group outperformed
the control group in revision 2 and 3, in revision 3, only the underline only feedback
outperformed the control group. This could possibly be partly due to the fact that the
underline only group made more errors than the underline+metalinguistic group
Karim and Nassaji 15

Table 6. Descriptive statistics: Short-term grammatical and non-grammatical accuracy gains by

group and session.

Text 1 to Text 2 to Text 3 to text

text 2 text 3 4 (del)

M SD M SD M SD
A: Grammatical errors:
1. Underline+metalinguistic .908 4.12 3.19 4.62 1.53 4.16
2. Underline only 3.48 6.63 −.015 6.38 4.32 5.96
3. Direct WCF 7.49 6.53 −2.52 7.97 5.70 7.98
4. No WCF 2.83 7.14 1.20 3.37 .030 4.51
B: Non-gammatical errors:
1. Underline+metalinguistic .761 6.26 2.01 4.11 .242 3.32
2. Underline only 1.38 7.87 1.31 5.87 .896 3.60
3. Direct WCF 3.39 4.74 −.293 4.50 1.96 5.07
4. No WCF 1.53 5.70 2.26 3.98 2.13 3.39

Note. WCF = written corrective feedback.

according to their error rates and therefore they conceivably had more room for
improvement than the other group (see Nassaji, 2016). Another reason could be that
due to their greater proficiency (as shown by their error rates), the
underline+metalinguistic group made fewer errors over time (e.g., in text 3 than in
text 2). Since they produced fewer errors, the difference in their performance decreased
in subsequent texts as compared to the control group.
With regard to transfer effects of WCF on new texts, the evidence is less compelling,
as no statistically significant differences in accuracy gains emerged between the feed-
back groups and the control group. If feedback had any effect on the accuracy of new
texts, this appeared to vary across tasks and error categories. For example, the direct
WCF produced some accuracy gains from text 1 to text 2. However, this effect did not
reach statistical significance. The effects also varied depending on whether the errors
were grammatical or non-grammatical. The four groups, for example, significantly var-
ied in the grammatical accuracy gains from text 1 to text 2 with direct WCF outperform-
ing underline+metalinguistic WCF. They also varied in their gains in text 3 with the
underline+metalinguistic WCF outperforming the direct WCF group. However, no
effects were found between these feedback groups and the control group. As for the two
different types of indirect feedback, the underline+metalinguistic WCF performed
slightly better than underline only WCF group from text 2 to 3; however, the difference
was not significant. No significant gain was found among the groups in text 4 (the
delayed writing) compared to text 3, suggesting that the feedback did not have any
delayed transfer effects.
The fluctuations across different texts could be partly related to the differences in task
features, task difficulty and as shown, the nature of the errors made. Since four different
picture prompts were used for the four narrative wiring tasks, it might have led to texts
that might have differed in terms of the kind of language forms required. For example,
16 Language Teaching Research 00(0)

different picture prompts may have required vocabulary of different type and degrees of
difficulty or familiarity. If the task-essential vocabulary was quite different in four writing
tasks, this might indicate a task effect and could explain some of the differences in the
effect of feedback on both revision and accuracy of new texts. In a recent study de Jong
and Vercellotti (2016) found that speaking performance varied with prompts that were
similar in narrative structure, storyline complexity, and number of elements. Since in the
present study feedback targeted a broad range of linguistic features, it might also have
caused cognitive overload for some learners and interrupted their feedback processing
(Van Beuningen, 2011). In other words, processing comprehensive CF could have been
cognitively demanding for some learners. Some of the differences may also have to do
with other factors such as the participants’ background and their past experience with the
WCF types. For example, it is possible that some participants might not have previously
received CF in the form of underlining or metalinguistic information. Indirect WCF
groups, especially underline only WCF, might need more instruction or training on inter-
preting the error codes (Ferris, 2004). Learners’ expectations of CF could be another fac-
tor. As Amrhein and Nassaji (2010) showed, while teachers may indirectly correct errors,
learners may expect that the error be corrected more directly. Thus, as these researchers
pointed out, teachers need to openly discuss the use of WCF with learners and make sure
that they understand the purpose and value of different ways of correcting errors. Learners
also need to be motivated to engage with the CF provided as CF loses its beneficial roles
if learners misperceive the goal of CF (Van Beuningen, 2011). Furthermore, learners’
attitude and motivation towards CF might influence the success of involvement with CF
(Storch, 2010; Storch & Wigglesworth, 2010; Swain & Lapkin, 2002).
Although some differences in accuracy gains from one text to the next were observed
between some of the feedback groups in this study, no significant difference in improve-
ment was found between the feedback groups and the control group, which makes it hard
to judge whether the improvements should be attributed to the feedback procedures as
such or to other factors. Despite this, the reduction of error rates in some of the tasks
under some of the feedback treatments (e.g., the considerable improvement from text 1
to text 2 after receiving direct feedback on text 1—see Table 5) is encouraging. Clearly,
the potential transfer effects of different feedback types merits further examination in
future research.

IX Conclusions and implications

In summary, the findings of the present study suggest that both direct and indirect compre-
hensive WCF can improve learners’ accuracy during the revisions of the same texts. The
effectiveness of focused WCF has already been established in the previous research (e.g.,
Bitchener, 2008; Bitchener & Knoch, 2008, 2009a, 2009b, 2010; Ellis et al., 2008; Sheen,
2007; Sheen, 2010b; Shintani & Ellis, 2013; Shintani et al., 2014). The present study is one
of the very few (e.g., Truscott & Hsu, 2008; Van Beuningen et al., 2012) that have provided
CF treatment on all existing errors in students’ writing tasks and examined its effectiveness
for the improvement of accuracy in new writing tasks. Sheen et al. (2009) advocated the
use of focused feedback asserting that ‘focused CF can contribute to grammatical accuracy
in writing’ (p. 556). The finding of the present study, however, showed that unfocused (i.e.,
Karim and Nassaji 17

comprehensive) CF can also have important effects on L2 writing accuracy. In other words,
while the effectiveness of focused CF is already established in previous research, the find-
ings of the current study add to the smaller body of evidence that comprehensive CF can
also be a useful pedagogical procedure for L2 teachers to help learners improve their accu-
racy in writing.
As noted earlier, in this study, direct WCF showed consistent positive effects on revi-
sion accuracy across all tasks. This finding is important and contributes to the theoretical
debate over direct and indirect WCF types for L2 development. As mentioned earlier,
Ferris (2004, 2006) has argued that indirect feedback is superior to direct feedback as
indirect feedback makes students engage in problem-solving learning and more pro-
found forms of language processing. Some other researchers, on the other hand, have
argued that the more direct forms of WCF are superior to indirect WCF as the former
may help learners to better notice their interlanguage problems, presenting them with
clearer information about the mismatch between the target and nontarget forms (e.g.,
Bitchener & Knoch, 2010b). The findings of this study seem to support the argument in
favor of the former, albeit mostly so when it comes to increasing the accuracy of a revised
text. However, the data also indicate that the effectiveness of such feedback may vary
depending on the nature of the task.
The feedback effects mostly pertained to grammatical errors. This could be explained
by the distinction between treatable and untreatable errors. Ferris (1999) suggested that
written CF might be more effective when directed at ‘treatable errors’, that is, errors relat-
ing to language forms that occur in ‘a patterned, rule-governed way’ (p. 6) (e.g., verb tense
and form, subject-verb agreement, article usage) than ‘untreatable’ errors (e.g., word
choice, unidiomatic sentence structure, missing or unnecessary words). The findings of
this study seem to support this hypothesis by showing that feedback was more effective in
the case of language forms that follow a set of syntactic and morphosyntactic rules than
those that do not, such as word choice. Similar findings have also been reported in other
studies that have shown greater accuracy gains when corrective feedback targeted treata-
ble grammatical errors than untreatable non-grammatical errors (e.g., Ferris & Roberts,
2001; Nassaji, 2011; Bitchener et al., 2005). Of course, the distinction between treatable
and non-treatable errors is not clear cut and can vary depending on the nature of the gram-
matical rules. For example, within grammatical rules, we may have those that are more
transparent than others such as past tense -ed versus the use of definite articles. There are
also rules that may be easy to explain and teach but not easy to learn such as third person
singular ‘s’ (Nassaji, 2011). These differences can influence the effectiveness of different
feedback types and therefore can also explain some of the differences found in the results
of this study related to the effect of feedback on different texts.
The present study has certain limitations that should be kept in mind when interpret-
ing its results. First, the study was conducted outside the regular classroom context.
Thus, while the findings can have pedagogical implications for WCF, they cannot be
directly applied to classroom contexts. Therefore, studies using the same design in regu-
lar classroom settings would yield results that could be generalized in a more direct way
to classroom contexts. The sample size was also relatively small. In addition, students
produced texts of different lengths. To address length variation, proportions of errors
(ratio of error rates per 100 words) were used in the analysis. Nevertheless, differences
18 Language Teaching Research 00(0)

in length could be considered as a factor and hence together with the small sample size
could be taken to partly explain some of the non-significant results.
To assess learners’ level of L2 proficiency, classroom placement tests were used.
However, placement tests might not be fully accurate in terms of assessing language
proficiency. A replication study using the same design but more robust language profi-
ciency tests to measure learners’ initial linguistic and writing abilities would be helpful.
To measure delayed effects, there was only a two-week gap between the third text and
the final text. Such a gap or interval would not be sufficient to provide evidence for long-
term accuracy gain. Therefore, to determine long-term effects, future studies could use
longer intervals between texts. Furthermore, although attempts were made to have suc-
cessive writing tasks with similar levels of difficulty, there is a possibility that partici-
pants responded to these tasks differently at different times. It is also worth noting that,
although picture prompts were provided to ensure the content of the participants’ texts
would be comparable, text length varied considerably, with some students keeping their
narratives to a bare minimum.
The direct WCF group received corrections on photocopies of their texts, which were
taken away from them after the reviewed the feedback. Although this was a methodo-
logical strategy to prevent learners from copying the text, it is likely to have positively
influenced learning as suggested by a large body of memory research on the effects of
retrieval. It is also probably not what many teachers actually do when they provide WCF
in their classrooms, and therefore may question its ecological validity. Of course, eco-
logical validity is a matter of degree, and any research, depending on the amount of
control it has over its design, would be more or less ecologically valid.
Finally, different language forms may respond differently to different feedback types
in ways that have not yet been properly understood. Feedback effects also depend on a
number of variables including the nature of the target structures, their complexity, fre-
quency, salience, learners’ prior knowledge, etc. as well as a host of other contextual and
affective variables. Therefore, although studies of feedback effectiveness including the
current study provide insights into the relative effectiveness of some feedback forms in
some contexts, they do not and cannot establish a clear superiority of one type of feed-
back in all contexts. Thus, the use of different feedback types should be viewed more as
a matter of suitability rather than superiority (Al-Rubai’ey & Nassaji, 2013).

Acknowledgements
We are grateful to Professor Frank Boers, the co-editor of Language Teaching Research, and the
anonymous reviewers for their insightful and constructive comments on earlier drafts of this article.

Funding
This research received no specific grant from any funding agency in the public, commercial, or
not-for-profit sectors.

References
Al-Rubai’ey, F., & Nassaji, H. (2013). Direct and indirect metalinguistic feedback: A matter of
suitability rather than superiority. In Mahmoud, M., & R. Al Mahrooqi (Eds.), Issues in TEFL
in the Arab World (pp. 203–232). Muscat: Sultan Qaboos University Press.
Karim and Nassaji 19

Amrhein, H.R., & Nassaji, H. (2010). Written corrective feedback: what do students and teachers
prefer and why? Canadian Journal of Applied Linguistics, 13, 95–127.
Ashwell, T. (2000). Patterns of teacher response to student writing in a multi-draft composition
classroom: Is content feedback followed by form feedback the best method? Journal of
Second Language Writing, 9, 227–257.
Bitchener, J. (2008). Evidence in support of written corrective feedback. Journal of Second
Language Writing, 17, 102–118.
Bitchener, J., & Knoch, U. (2008). The value of written corrective feedback for migrant and inter-
national students. Language Teaching Research Journal, 12, 409–431.
Bitchener, J., & Knoch, U. (2009a). The relative effectiveness of different types of direct written
corrective feedback. System, 37, 322–329.
Bitchener, J., & Knoch, U. (2009b). The contribution of written corrective feedback to language
development: A ten month investigation. Applied Linguistics, 31, 193–214.
Bitchener, J., & Knoch, U. (2010). Raising the linguistic accuracy level of advanced L2 writers
with written corrective feedback. Journal of Second Language Writing, 19, 207–217.
Bitchener, J., Young, S., & Cameron, D. (2005). The effect of different types of corrective feed-
back on ESL student writing. Journal of Second Language Writing, 9, 227–258.
Brown, D.H. (2007). Teaching by principles: An interactive approach to language teaching. 3rd
edition. New York: Longman/Pearson Education.
Bruton, A. (2009). Designing research into the effect of error correction in L2 writing: Not so
straightforward. Journal of Second Language Writing, 18, 136–140.
Bruton, A. (2010). Another reply to Truscott on error correction: Improved situated designs over
statistics. System, 38, 491–498.
Chandler, J. (2003). The efficacy of various kinds of error feedback for improvement in the accu-
racy and fluency of L2 student writing. Journal of Second Language Writing, 12, 267–296.
Chandler, J. (2004). A response to Truscott. Journal of Second Language Writing, 13, 345–348.
de Jong, N., & Vercellotti, M.L. (2016). Similar prompts may not be similar in the performance
they elicit: Examining fluency, complexity, accuracy, and lexis in narratives from five picture
prompts. Language Teaching Research, 20, 387–404.
Ellis, R., Sheen, Y., Murakami, M., & Takashima, H. (2008). The effects of focused and unfocused
written corrective feedback in an English as a foreign language context. System, 36, 353–371.
Fathman, A.K., & Whalley, E. (1990). Teacher response to student writing: focus on form versus
content. In Kroll, B. (Ed.), Second language writing: Research insights for the classroom
(pp. 178–190). Cambridge: Cambridge University Press.
Ferris, D. (1997). The influence of teacher commentary on student revision. TESOL Quarterly,
31, 315–339.
Ferris, D. (1999). The case for grammar correction in L2 writing classes: A response to Truscott
(1996). Journal of Second Language Writing, 8, 1–11.
Ferris, D. (2003) Responding to writing. In Kroll, B. (Ed.), Exploring the dynamics of second lan-
guage writing (pp.119–140). Cambridge: Cambridge University Press.
Ferris, D. (2004). The ‘grammar correction’ debate in L2 writing: Where are we, and where do
we go from here? (and what do we do in the meantime …?). Journal of Second Language
Writing, 13, 49–62.
Ferris, D. (2006). Does error feedback help student writers? New evidence on the short- and long-
term effects of written error correction. In Hyland, K., & F. Hyland (Eds.), Feedback in second
language writing: Contexts and issues (pp. 81–104). Cambridge: Cambridge University Press.
Ferris, D. (2010). Second language writing research and written corrective feedback in SLA.
Studies in Second Language Acquisition, 32, 181–201.
Ferris, D. (2012). Written corrective feedback in second language acquisition and writing studies.
Language Teaching, 45, 446–459.
20 Language Teaching Research 00(0)

Ferris, D., & Roberts, B. (2001). Error feedback in L2 writing classes: How explicit does it need
to be? Journal of Second Language Writing, 10, 161–184.
Ferris, D., & Helt, M. (2000) Was Truscott right? New evidence on the effects of error correc-
tion in L2 writing classes. Unpublished paper presented at the Conference of the American
Association for Applied Linguistics, Vancouver, BC, Canada.
Frear, D., & Chiu, Y.H. (2015). The effect of focused and unfocused indirect written corrective
feedback on EFL learners’ accuracy in new pieces of writing. System, 53, 24–34.
Hartshorn, K.J., Evans, N.W., Merrill, P.F., et al. (2010). Effects of dynamic corrective feedback
on ESL writing accuracy. TESOL Quarterly, 44, 84–109.
Kang, E., & Han, Z. (2015). The efficacy of written corrective feedback in improving L2 written
accuracy: A meta-analysis. The Modern Language Journal, 99, 1–18.
Kepner, C.G. (1991). An experiment in the relationship of types of written feedback to the devel-
opment of second-language writing skills. Modern Language Journal, 7, 305–313.
Lalande, J.F. (1982). Reducing composition errors: an experiment. Modern Language Journal,
66, 140–149.
Larson-Hall, J. (2010). A guide to doing statistics in second language research using SPSS. New
York: Routledge.
Lindstromberg, S. (2016). Inferential statistics in Language Teaching Research: A review and
ways forward. Language Teaching Research, 20, 741–768.
Liu, Y. (2008). The effects of error feedback in second language writing. Arizona working papers
in SLA and Teaching, 15, 65–79.
Liu, Q., & Brown, D. (2015). Methodological synthesis of research on the effectiveness of correc-
tive feedback in L2 writing. Journal of Second Language Writing, 30, 66–81.
Lyster, R., Lightbown, P., & Spada, N. (1999). A response to Truscott’s ‘What’s wrong with oral
grammar correction.’ Canadian Modern Language Review, 55, 457–467.
Maxwell, S.E., & Delaney, H.D. (1990). Designing experiments and analyzing data: A model
comparison perspective. Pacific Grove, CA: Brooks/Cole.
Mirzaii, M., & Aliabadi, R.B. (2013). Direct and indirect written corrective feedback in the context
of genre-based instruction on job application letter writing. Journal of Writing Research, 5,
191–213.
Nassaji, H. (2011). Correcting student’s written grammatical errors: The effects of negotiated
versus non-negotiated feedback. Studies in Second Language Learning and Teaching, 1,
315–334.
Nassaji, H. (2015). The interactional feedback dimension in instructed second language learning:
Linking theory, research, and practice. London: Bloomsbury.
Nassaji, H. (2016). Anniversary article: Interactional feedback in second language teaching and
learning: A synthesis and analysis of current research. Language Teaching Research, 20,
535–562.
Polio, C., Fleck, C., & Leder, N. (1998). ‘If only I had more time’: ESL learners’ changes in lin-
guistic accuracy on essay revisions. Journal of Second Language Writing, 7, 43–68.
Robb, T., Ross, S., & Shortreed, I. (1986). Salience of feedback on error and its effect on EFL
writing quality. TESOL Quarterly, 20, 83–95.
Sachs, R., & Polio, C. (2007). Learners’ use of two types of written feedback on a L2 writing revi-
sion task. Studies in Second Language Acquisition, 29, 67–100.
Semke, H. (1984). The effects of the red pen. Foreign Language Annals, 17, 195–202.
Sheen, Y. (2007). The effect of focused written corrective feedback and language aptitude on ESL
learners’ acquisition of articles. TESOL Quarterly, 41, 255–283.
Sheen, Y. (2010a). The role of oral and written corrective feedback in SLA. Studies in Second
Language Acquisition, 32, 169–179.
 Karim and Nassaji 21

Sheen, Y. (2010b). Differential effects of oral and written corrective feedback in the ESL class-
room. Studies in Second Language Acquisition, 32, 201–234.
Sheen, Y., Wright, D., & Moldawa, A. (2009). Differential effects of focused and unfocused writ-
ten correction on the accurate use of grammatical forms by adult ESL learners. System, 37,
556– 569.
Sheppard, K. (1992). Two feedback types: Do they make a difference? RELC Journal, 23, 103–110.
Shintani, N., & Ellis, R. (2013). The comparative effect of direct written corrective feedback and
metalinguistic explanation on learners’ explicit and implicit knowledge of the English indefi-
nite article. Journal of Second Language Writing, 22, 286–306.
Shintani, N., Ellis, R., & Suzuki, W. (2014). Effects of written feedback and revision on learners’
accuracy in using two English grammatical structures. Language Learning, 64, 103–131.
Storch, N. (2010). Critical feedback on written corrective feedback research. IJES, 10, 29–46.
Storch, N., & Wigglesworth, G. (2010). Learners’ processing, uptake, and retention of corrective
feedback on writing: Case studies. Studies in Second Language Acquisition, 32, 303–334.
Swain, M., & Lapkin, S. (2002). Talking it through: two French immersion learners’ response to
reformulation. International Journal of Educational Research, 37, 285–304.
Truscott, J. (1996). The case against grammar correction in L2 writing classes. Language Learning,
46, 327–369.
Truscott, J. (1999). The case for ‘the case for grammar correction in L2 writing classes’: A response
to Ferris. Journal of Second Language Writing, 8, 111–122.
Truscott, J., & Hsu, A.Y. (2008). Error correction, revision, and learning. Journal of Second
Language Writing, 17, 292–305.
Van Beuningen, C.G. (2011). The effectiveness of comprehensive corrective feedback in second
language writing. Unpublished doctoral dissertation, University of Amsterdam, Netherlands.
Available at: http://dare.uva.nl/en/record/374645 (accessed September 2018).
Van Beuningen, C.G., De Jong, N.H., & Kuiken, F. (2008). The effect of direct and indirect correc-
tive feedback on L2 learners’ written accuracy. International Journal of Applied Linguistics,
156, 279–296.
Van Beuningen, C.G., De Jong, N.H., & Kuiken, F. (2012). Evidence on the effectiveness of
comprehensive error correction in second language writing. Language Learning, 62, 1–41.

View publication stats