Krasnoshchekova. Narrative Competence of Adults L2 Russian Learnes

You might also like

Download as pdf or txt
Download as pdf or txt
You are on page 1of 25

Journal of Psycholinguistic Research

https://doi.org/10.1007/s10936-018-9622-3

Narrative Competence of Adult L2 Russian Learners

Sofia Krasnoshchekova1 · Kseniia Kashleva2 

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Abstract
Narrative competence is an essential part of language proficiency. Research of narrative
competence has both a theoretical and empirical value. Our study aims to assess narrative
competence of adult L2 Russian learners and to investigate the relationship between their
narrative competence and their language proficiency. For assessment, we used the Multi-
lingual Assessment Instrument for Narratives adapted for the Russian language. We also
designed a scale for assessing microstructure in Russian narratives. The study uses both
qualitative and quantitative analysis. The results show that macrostructural narrative sub-
competence of L2 Russian learners does not depend on their language proficiency (except
for an ability to produce structurally shorter episodes at higher level) and microstructural
narrative subcompetence of L2 Russian learners depends on their language proficiency
only in some ways. Our study contributes to the theory of narrative competence in L2
acquisition.

Keywords  Second language acquisition · Narrative competence · Language proficiency ·


Russian as a second language

Introduction

Studying narrative competence is essential for a wide range of problems connected with
second language acquisition (SLA) in adults. Acquiring a second language, learners pro-
gressively upgrade their language proficiency. An inherent feature of language proficiency
is narrative competence. However, a relationship between narrative competence and lan-
guage proficiency is not clear. This relationship should be investigated further.

In this study were used the results obtained by S. Krasnoshchekova in the research, supported by
the Russian Foundation for Basic Research (Project No. 18-012-00650 “Semantic Categories in the
Grammar System of Russian Language”).

* Kseniia Kashleva
kkashleva@hse.ru
1
Higher School of International Educational Programs, Peter the Great Saint Petersburg Polytechnic
University, 29 Politekhnicheskaya Ulitsa, Saint Petersburg, Russia 195251
2
Faculty of Humanities, School of Linguistics, National Research University Higher School
of Economics, 21/4 Staraya Basmannaya Ulitsa, Moscow, Russia 105066

13
Vol.:(0123456789)
Journal of Psycholinguistic Research

There are volumes of published studies describing language proficiency, starting from
Carroll’s seminal paper (1961) on ten aspects of language competence and Lado’s book
(1961) on language testing. The views on language proficiency can be broadly divided into
two groups: structural/generative and communicative/functional approaches. The former,
following the classic papers, refers to language proficiency as to linguistic knowledge and
appropriate usage of language means, or ‘the capacity to use language’ (Llurda 2000: 96),
while the latter incorporates pragmatic, communicative, and sociolinguistic competences
(Cummins 1980; Verhoeven and de Jong 1992; Harley et al. 1990). The modern ideas on
language proficiency mostly based on the communicative approach (Read 2015). Hulstijn
(2011) analysed language proficiency in native and nonnative speakers and defines lan-
guage proficiency in general as ‘the extent to which an individual possesses the linguistic
cognition necessary to function in a given communicative situation, in a given modality’
(Hulstijn 2011: 242). Although a generally accepted definition of language proficiency is
lacking, many researchers use this term without clarifying what they mean by ‘language
proficiency’. It is especially typical of sociological or economic studies (Hochman and
Davidov 2014). A lot of researchers also use ‘language proficiency’ interchangeably with
‘language fluency’ or with ‘the level of proficiency’. Hulstijn argues that the notion of the
level of proficiency is sensible only in educational context and is not useful for explaining
individual differences in language acquisition (Hulstijn 2011: 242). His idea may be proved
by the fact that there are many guidelines to language tests providing a range of procedures
for language proficiency assessment (Bachman and Palmer 1996; Alderson 2005; Lyons
and Dadey 2017; Goldschmidt and Hakuta 2017).
In this paper, language proficiency is defined as an ability to use appropriate language
means effectively in all communicative situations (Pakulak and Neville 2010; Shuai 2014;
De Clercq and Housen 2017). As language proficiency can be represented as a set of com-
petences, it is necessary to investigate them not only integrally, but also individually. One
of these competences is narrative competence.
A narrative being a story or a chain of connected events is a form of communication that
is based on the specific communicative motives. Tomasello postulated that there are three
general communicative motives: requesting, informing, and sharing (Tomasello 2014). A
narrative encapsulates two of three motives—informing and sharing. It makes narrative
competence an essential part of language proficiency.
Narrative competence, broadly regarded as an ability to produce and to comprehend
narratives, is defined in different ways: as “a process of constructing topic predicate struc-
tures” (van Oers 2007: 299) within a structural approach, as a capacity “to produce and
validate personal <…> stories” (Zanazanian and Popa 2018: 365) within a personal-based
approach, or as an ability to form an organized, well-structured, and coherent text (Pinto
et al. 2018).
In the present study, narrative competence is considered as an ability to produce a coher-
ent story, to organize it around a theme and to use appropriate language devices, making a
story comprehensible for an interlocutor (a listener or a reader; Montanari 2004; Pavlenko
2006).
Research into narrative competence has a long history. Labov and Waletzky (1967)
has laid the foundation for investigating narratives. They have assumed narrative elements
(thesis, orientation, the chain of events, evaluation, conclusion, and coda). Another semi-
nal paper has established that a speaker needs four types of information: about macro and
microstructure, content, and context (Shapiro and Hudson  1991). Later, new elements to
the structure of narrative were added (Ochs and Capps 2001). Narratives are also consid-
ered as a way of thinking (Wilkinson et al. 1993).

13
Journal of Psycholinguistic Research

Special attention was given to narrative competence in children. To assess narrative


skills in children, who acquire one or more languages from birth or from early age, the
Multilingual Assessment Instrument for Narratives (MAIN) was created (Gagarina et  al.
2012, 2015). A great deal of previous research into narrative competence has focused
on studying narratives produced by monolingual or bilingual children (John-Steiner and
Panofsky 1992; Berman 1995; Schneider and Hayward 2010; Justice et al. 2010; Gorman
et al. 2016), including language proficiency assessment based on narratives from children
(Tilstra and McMaster 2007). A big number of researchers have investigated language
impairment of children using narrative analysis for studying different psychological or cen-
tral nervous system abnormalities (Schneider et al. 2006; Justice et al. 2006; Thorne 2017).
Others have questioned reliability and accuracy of different narration-based dynamic
assessment tasks (Peña et al. 2006; van der Veen and Poland 2012; Petersen et al. 2017).
Extensive research has also been made in different aspects of narrative competence in
L2 learners (Akinci et al. 2001; Pavlenko 2006; Almgren and Manterola 2016). However,
less research has been Russian-based, and researchers have focused on the acquisition of
Russian morphology, mostly (Kempe and MacWhinney 1998; Romanova 2017).
Recent studies have left unanswered the question if narrative competence depends on
language proficiency of L2 Russian learners. Narrative competence may be divided into
two subcompetences: macrostructural and microstructural. Macrostructural subcompetence
deals with a plot of a story. A speaker has to master macrostructural subcompetence to
organize a story around a theme and to make a story coherent. Coherence is absolutely nec-
essary. Only sentences that are united by one topic and with special means can be regarded
as a text, not just a set of sentences. These special language means are used by speakers
if they master microstructural subcompetence. Mastering microstructural subcompetence
makes a story comprehensible and clear.
It results in the following research question: do macrostructural and microstructural sub-
competences depend on language proficiency and to what extent?
From the question of this study, the following hypothesis comes: macrostructural sub-
competence does not depend on language proficiency and microstructural subcompetence
depends on language proficiency to some extent.
The purpose of this research is to compare L1 adults’ narratives with L2 adults’ narra-
tives to investigate the relationship between their narrative competence and their language
proficiency.

Method

A number of techniques have been developed to evaluate narratives. One of the most well-
known tools for assessing narratives is making a story based on pictures. Different authors
created their own variations (Schneider et al. 2006) or based their analysis on the wordless
picture book Frog, Where Are You? (Karlsen et al. 2016).
For this study, we adopted the Multilingual Assessment Instrument for Narratives
(MAIN). The MAIN allows assessing narratives in three modes: telling, retelling, and tell-
ing after listening to a model story (Gagarina et al. 2012). The MAIN was chosen because
it focuses on macrostructure (which is language-independent): story structure components,
structural complexity, and internal state terms (IST). The MAIN can be used to assess
narratives from bilingual and monolingual children. We adopted this method for L2 adult
learners. We used only Section I (Production). We did not ask any questions required in

13
Journal of Psycholinguistic Research

Section II (Comprehension) as this section focuses on psychological abilities of children


and their cognitive skills.
Our analysis of macrostructure was based on the MAIN, but for microstructure we
developed our own assessment system. As microstructure is language-specific, this system
reflects typical features of the Russian language (see “Appendix” section). It consists of
four main parts: Morphosyntax, Reference, Complexity, and Vocabulary. The Morphosyn-
tax part focuses on sentences, so this part includes the assessment of syntactic features in
a sentence. The Reference part provides the assessment table for referential cohesion. The
Complexity part provides the assessment tool for sentence types used in a narrative and
links between them. The Vocabulary part focuses on vocabulary used in L2 narratives.
It consists of two parts: the assessment of usage mistakes (wrong meaning, non-existent
words, and code-switching) and the assessment of IST in total.

Participants

The project used a group of native Russian speakers (L1 group) and a group of L2 learners
of Russian. The L1 group consisted of 37 students. The gender distribution in this group
favoured girls (73%) over boys (27%). All of the native speakers were aged between 18 and
22. The number of L1 group narratives is 72. Of the sample of L2 learners of 49 respond-
ents, 21 were female and 28 male. L2 learners were aged between 17 and 37. The number
of L2 narratives is 89.
Among 49 L2 learners there were 26 Chinese native speakers, 4 Vietnamese native
speakers, 3 Mongolian native speakers, and 2 native speakers of each of the following lan-
guages: Arabic, Turkish, Italian, and Bulgarian; plus 1 native speaker of each of the fol-
lowing languages: Korean, Tajik, Norwegian, Hindi, German, Pashto, Macedonian, and
Portuguese. As for their language background, most L2 participants (85.71%) have been
studying Russian for more than a year. What is more, at the moment of the experiment they
have been studying L2 in target language environment for 2–4 months. Therefore, we can
consider the participants proficient enough to produce short narratives on general topics.

Procedure

The narratives from both L1 and L2 speakers were collected by adapting the procedure
used by Gagarina et al. (2012). The order of tasks was the following: the first task was to
write an original story (producing), and the second one was to write a story that they heard
(reproducing). The order of tasks was made to avoid the interference of a story for repro-
ducing on production of an original story.
Firstly, we collected background information about the L2 participants: their age, gen-
der, first language, and how long they have been studying Russian. After that, the L2 par-
ticipants were given the first stimulus. It was a set of pictures—either the set ‘Baby Birds’
or the set ‘Baby Goats’ from the MAIN. The L2 participants were asked to write their own
story in Russian based on the given set of pictures.
Then, they were given the second stimulus. The second set of pictures was either the
set ‘Cat’ or the set ‘Dog’ from the MAIN. An experimenter read a story in Russian based
on the given set twice and the L2 participants were asked to write their reproduction of the
story they had listened to. They were not allowed to use any dictionaries or ask the experi-
menter about any words they did not remember.

13
Journal of Psycholinguistic Research

The procedure for the L1 participants was the same, excluding questions about their first
language and about time of studying Russian.
There was no time limit for doing both tasks, however, the participants usually required
between 35 and 50 min to produce their narratives. After that, we transcribed their written
narratives.

Data Analysis

In order to identify typical features of L2 Russian learners’ narratives, we assessed their


macrostructure and microstructure.
Firstly, we measured text length (in words), sentence length (in words) and the number
of sentences in a text for L1 and L2 narratives. Then we calculated an average rate for each
value.
Secondly, we assessed the macrostructure of each narrative. For doing that, we used
the MAIN scoring sheets. The MAIN scoring sheets consist of three parts: part A (‘Story
structure’), part B (‘Structural complexity’), and part C (‘Internal state terms’). The MAIN
gives 17 points at most for part A. One point is given for each value mentioned in the scor-
ing sheet. In part B, the number of specific sequences (for instance, ‘Attempt’ + ‘Outcome’)
is counted. In part C, the total number of IST in a text is measured in tokens. So each nar-
rative got a number of points for its macrostructure. After that, again, we measured an aver-
age rate for each value.
Following this, we assessed microstructure in narratives. Mistakes in L2 narratives were
taken into consideration if and only if mistakes affected communication. We did not reduce
scores for the same mistakes within one narrative more than once. The first step was to
assess morphosyntax and reference in sentences. Each sentence in a narrative was assigned
with 100 points at the beginning of the assessment. If a sentence was not clear and under-
standable, a score for coherence in this sentence was 0. If there were any mistakes men-
tioned in a scoring table (see “Appendix” section), the number of points for this sentence
was reduced according to the appropriate table. We attained a result for each sentence in a
narrative and then we calculated an average rate for both morphosyntax and reference in a
narrative by the formula Av = T/N, where Av is an average score, T is a sum of scores for all
sentences in a narrative, and N is a number of sentences in a narrative. Then, the same pro-
cedure was performed for assessing complexity of a narrative. Each narrative was assigned
with 100 points at the beginning of the assessment. If a text was not finished, a total score
for complexity was 0. We reduced and added points according to the appropriate scoring
table. This means that the total score for complexity of a narrative can exceed 100 points.
And finally, we assessed vocabulary in narratives. Each narrative, again, was assigned
with 100 points and we reduced the score according to the appropriate scoring table. IST
was calculated in tokens. Having all these figured, we were able to calculate an average rate
for each value in both L1 and L2 narratives.

Results

The purpose of this experiment was to investigate narrative competence of L2 adult learn-
ers. We used two quantitative tools for that: the Multilingual Assessment Instrument for
Narratives (MAIN) designed by Gagarina et al. (2012) and our own assessment tool.

13
Journal of Psycholinguistic Research

As some students wrote only one text (producing or reproducing), it was important to
check if these unpaired texts have any influence on the final results. For that reason, we
performed all the tests twice—with and without unpaired texts. The tests were performed
with SPSS (IBM Corp. 2015). The results for the corpora without and with unpaired texts
were the same, mostly. This allowed us to compare L1 narratives with L2 narratives, using
the corpus with unpaired texts, and to consider the results obtained as reliable. However, if
the results for the corpus without unpaired texts and for the corpus with unpaired texts dif-
fer, it is mentioned specifically.
Comparing L1 and L2 narratives by any value, we took into consideration three param-
eters: L1 of foreign students, gender in both groups of participants, and type of narratives.
As Chinese learners of Russian make 53% of L2 group, it was essential to compare L1
narratives with all L2 narratives, with narratives of Chinese learners (CH) only, and with
narratives of non-Chinese learners (Non-CH) only. With regard to gender, we compared L1
male with L1 female narratives and L2 male with L2 female narratives to find out if there
are any differences. As for the type of narratives, we compared L1 producing (P) with L1
reproducing (RE), L2 producing with L2 reproducing, L1 producing with L2 producing,
and L1 reproducing with L2 reproducing.
The first set of analyses examined the volume of narratives. Table 1 shows the results for
L1 and L2 narratives from a formal point of view.
It should be mentioned that no significant difference was found between L1 male and L1
female narratives and between L2 male and L2 female narratives (p > .05). No significant
difference was also found between L1 producing and L1 reproducing (p > .05) or between L2
producing and L2 reproducing (p > .05). Therefore, these results were not included in Table 1.
It can be seen from the data in Table 1 that L1 narratives and L2 narratives are signifi-
cantly different from each other by text length and sentence length. The results show that
L1 narratives are typically longer in words than L2 narratives. Sentences in L1 narratives
are also longer than in L2 narratives. However, there are slight variations in these results
depending on L1 of foreign students or the type of narratives. Table 1 shows that sentence
length significantly differs between L1 producing and L2 producing, while there is no dif-
ference between L1 reproducing and L2 reproducing. The results also show that a signifi-
cant difference in text length and sentence length between L1 narratives and L2 narratives
is due to Chinese students mostly.
What is interesting about the data is that there is no significant difference in the number
of sentences that both L1 and L2 participants used to produce a story, regardless of the
type of narratives or L1 of foreign students.
For the corpora without unpaired texts the results were the same, except for text length
(the p value for non-Chinese narratives was less than .05).
As explained earlier, the analysis of narrative content consisted of macrostructure and
microstructure assessment. Table 2 presents the summary statistics for macrostructure in nar-
ratives of L2 learners compared with L1 narratives. (For complete IST statistics see Table 3.)
The results show that there is no significant difference between L1 male and L1 female nar-
ratives. No significant difference was also found between L2 male and L2 female narratives
(p > .05), except for IST: a difference between L2 male and L2 female narratives has a certain
trend toward significance (p < .05). Because of this, these results were not included in Table 2.
As shown in Table 2, L1 narratives and L2 narratives are identical in many ways. L1
participants tend to mention when the events of a produced story took place. Similarly,
L2 participants find important to refer to when a story happened, regardless of their
native language (CH or non-CH). However, there is a difference that has a certain trend

13
Table 1  Length of narratives
Journal of Psycholinguistic Research

Mean score for L1 Mean score for L2 nar- Independent t-test (by L1) Mean score for L1 Mean score for Independent
narratives (by L1) ratives (by L1) narratives (by type) L2 narratives (by t-test (by type)
type)

Text length (in words) 111.04 All 88.58 All p < .0001 All 111.04 All 88.58 All p < .0001
CH 85.45 CH p < .0001 P 116.11 P 85.6 P p = .001
Non-CH 92.1 Non-CH p = .007 RE 105.69 RE 91.64 RE p = .035
Sentence length (in words) 12.53 All 9.87 All p < .0001 All 12.53 All 9.87 All p < .0001
CH 8.92 CH p < .0001 P 12.41 P 9.22 P p < .0001
Non-CH 10.94 Non-CH p = .02 RE 12.65 RE 10.54 RE p = .051
Number of sentences in a text 9.43 All 9.68 All p = .68 All 9.43 All 9.68 All p = .68
CH 10.4 CH p = .212 P 10 P 9.45 P p = .592
Non-CH 8.88 Non-CH p = .47 RE 8.83 RE 9.89 RE p = .214

13

Table 2  Macrostructure in narratives
Mean score for L1 Mean score for L2 Independent t-test (by Mean score for Mean score Independent
narratives (by L1) narratives (by L1) L1) L1 narratives for L2 narra- t-test (by type,

13
(by type) tives (by type) L1 vs. L2)

Time .82 All .76 All p = .395 All .82 All .76 All p = .395
CH .72 CH p = .234 P .84 P .62 P p = .027
Non-CH .81 Non-CH p = .896 RE .8 RE .91 RE p = .185
Place .7 All .54 All p = .045 All .7 All .54 All p = .045
CH .55 CH p = .126 P .86 P .73 P p = .138
Non-CH .52 Non-CH p = .077 RE .51 RE .34 RE p = .124
Goals 1.43 All 1.31 All p = .428 All 1.43 All 1.31 All p = .428
CH 1.51 CH p = .644 P 1.3 P 1.33 P p = .846
Non-CH 1.1 Non-CH p = .066 RE 1.57 RE 1.29 RE p = .229
Attempts 2.15 All 2.1 All p = .707 All 2.15 All 2.1 All p = .707
CH 2.21 CH p = .712 P 2.35 P 2.2 P p = .389
Non-CH 1.98 Non-CH p = .297 RE 1.94 RE 2 RE p = .786
Outcomes 2.18 All 2.35 All p = .163 All 2.18 All 2.35 All p = .163
CH 2.45 CH p = .062 P 2 P 2.1 P p = .504
Non-CH 2.24 Non-CH p = .682 RE 2.37 RE 2.59 RE p = .165
IST 3.36 All 2.63 All p < .0001 All 3.36 All 2.63 All p < .0001
CH 2.77 CH p = .013 P 2.84 P 2.38 P p = .059
Non-CH 2.48 Non-CH p < .0001 RE 3.91 RE 2.89 RE p = .001
Part A in total 10.64 All 9.7 All p = .01 All 10,64 All 9,7 All p = .01
CH 9.49 CH p = .401 P 9.35 P 8.76 P p = .174
Non-CH 8.31 Non-CH p < .0001 RE 10.31 RE 9.11 RE p = .025
Attempt + Outcome (AO) 1.02 All .97 All p = .665 All 1.02 All .97 All p = .665
CH 1 CH p = .869 P 1.22 P .93 P p = .093
Non-CH .93 Non-CH p = .571 RE 1 RE .83 RE p = .457
Journal of Psycholinguistic Research
Table 2  (continued)
Mean score for L1 Mean score for L2 Independent t-test (by Mean score for Mean score Independent
narratives (by L1) narratives (by L1) L1) L1 narratives for L2 narra- t-test (by type,
(by type) tives (by type) L1 vs. L2)

Goals (G) .17 All .11 All p = .368 All .17 All .11 All p = .368
CH .13 CH p = .609 P .29 P .16 P p = .187
Non-CH .1 Non-CH p = .327 RE .03 RE .07 RE p = .432
Goal + Attempt (GA) or Goal + Outcome (GO) .72 All .49 All p = .048 All .72 All .49 All p = .048
CH .51 CH p = .148 P .51 P .53 P p = .889
Journal of Psycholinguistic Research

Non-CH .48 Non-CH p = .054 RE .94 RE .45 RE p = .007


Goal + Attempt + Outcome (GAO) .54 All .7 All p = .152 All .54 All .7 All p = .152
CH .87 CH p = .015 P .49 P .64 P p = .289
Non-CH .52 Non-CH p = .889 RE .6 RE .77 RE p = .316
Mean score for L1 narratives Independent t-test (by type, P Mean score for L2 narra- Independent t-test (by
(by type) L1 vs. RE L1) tives (by type) type, P L2 vs. RE L2)

Time All .82 p = .682 All .76 p = .001


P .84 P .62
RE .8 RE .91
Place All .7 p = .001 All .54 p < .0001
P .86 P .73
RE .51 RE .34
Goals All 1.43 p = .229 All 1.31 p = .842
P 1.3 P 1.33
RE 1.57 RE 1.29
Attempts All 2.15 p = .045 All 2.1 p = .279
P 2.35 P 2.2
RE 1.94 RE 2

13

Table 2  (continued)
Mean score for L1 narratives Independent t-test (by type, P Mean score for L2 narra- Independent t-test (by
(by type) L1 vs. RE L1) tives (by type) type, P L2 vs. RE L2)

13
Outcomes All 2.18 p = .023 All 2.35 p = .004
P 2 P 2.1
RE 2.37 RE 2.59
IST All 3.36 p < .0001 All 2.63 p = .056
P 2.84 P 2.38
RE 3.91 RE 2.89
Part A in total All 10,64 p = .037 All 9,7 p = .466
P 9.35 P 8.76
RE 10.31 RE 9.11
Attempt + Outcome (AO) All 1.02 p = .068 All .97 p = .724
P 1.22 P .93
RE 1 RE .83
Goals (G) All .17 p = .005 All .11 p = .241
P .29 P .16
RE .03 RE .07
Goal + Attempt (GA) or Goal + Outcome (GO) All .72 p = .023 All .49 p = .576
P .51 P .53
RE .94 RE .45
Goal + Attempt + Outcome (GAO) All .54 p = .428 All .7 p = .461
P .49 P .64
RE .6 RE .77
Journal of Psycholinguistic Research
Table 3  Microstructure in narratives
Mean score for L1 Mean score for L2 narra- Independent t-test (by L1) Mean score for Mean score for L2 Independent
narratives (by L1) tives (by L1) L1 narratives (by narratives (by type) t-test (by type,
type) L1 vs. L2)

Mistakes in morpho-syntax − 1.89 All − 37.21 All p < .0001 All − 1.89 All − 37.21 All p < .0001
CH − 43.66 CH p < .0001 P − 1.84 P − 36.2 P p < .0001
Non-CH − 30 Non-CH p < .0001 RE − 1.94 RE − 38.23 RE p < .0001
Journal of Psycholinguistic Research

Morpho-syntax 99.77 All 95.98 All p < .0001 All 99.77 All 95.98 All p < .0001
CH 95.55 CH p < .0001 P 99.77 P 96.09 P p < .0001
Non-CH 96.45 Non-CH p < .0001 RE 99.76 RE 95.86 RE p < .0001
Reference − .36 All − 1.93 All p < .0001 All − .36 All − 1.93 All p < .0001
CH − 1.74 CH p < .0001 P − .43 P − 1.69 P p = .012
Non-CH − 2.14 Non-CH p = .001 RE − .28 RE − 2.18 RE p < .0001
Complexity
Complex sentences 7.4 All 7.1 All p = .71 All 7.4 All 7.1 All p = .71
CH 6.47 CH p = .324 P 7.38 P 6.07 P p = .259
Non-CH 7.81 Non-CH p = .687 RE 7.43 RE 8.16 RE p = .516
Compound sentences 2.64 All 3.22 All p = .157 All 2.64 All 3.22 All p = .157
CH 3.49 CH p = .104 P 2.76 P 2.8 P p = .937
Non-CH 2.93 Non-CH p = .486 RE 2.51 RE 3.66 RE p = .068
Mistakes in complexity − .19 All − 1.62 All p < .0001 All − .19 All − 1.62 All p < .0001
CH − 1.32 CH p < .0001 P − .27 P − 1.51 P p < .0001
Non-CH − 1.95 Non-CH p < .0001 RE − .11 RE − 1.73 RE p < .0001
Complexity in total 117.1 All 116.8 All p = .923 All 117.1 All 116.8 All p = .923
CH 118.6 CH p = .67 P 120 P 114 P p = .11
Non-CH 114.76 Non-CH p = .622 RE 114.06 RE 119.63 RE p = .329

13
Journal of Psycholinguistic Research

toward significance between L1 and L2 producing (p < .05) and there is a significant dif-
ference between L2 producing and L2 reproducing (p < .01).
On the question of spatial orientation, a difference between L1 narratives and L2
narratives in general cannot be considered as significant, while a difference between L1
producing and L1 reproducing (p < .01) and a difference between L2 producing and L2
reproducing (p < .0001) is significant. In the corpora without unpaired texts a difference
between L2 producing and L2 reproducing is also significant (p < .01).
As for describing actions of story characters (goals, attempts, and outcomes), the
results of L1 participants are comparable to those of L2 participants. It is interesting that
there is also no difference in some sequences within one episode (Attempt + Outcome
and Goal + Attempt + Outcome) and in number of episodes with goals only between L1
narratives and L2 narratives. On the other hand, L1 narratives are different from L2
narratives in some respects. L2 participants are less likely to produce a sequence of
Goal + Attempt or Goal + Outcome within one episode. These differences have a certain
trend toward significance (p < .05). In the corpora without unpaired texts, the results
were the same, except for goals (the p-value for non-Chinese and L1 narratives is less
than .05), attempts (the p-value for difference between L1 producing and L1 reproduc-
ing is more than .05), and GA/GO (the p-value for difference between L1 reproducing
and L2 reproducing is less than .05).
Finally, L2 participants tend to use fewer IST for describing a characters’ inner state.
This relates to lexical proficiency (see Table 4).
Further analysis focused on microstructure. Table 3 illustrates some of the main charac-
teristics of the L2 learners’ narratives in comparison with native speakers.
No significant difference was found between L1 male and L1 female narratives,
between L1 producing and L1 reproducing, and between L2 producing and L2 reproduc-
ing (p > .05). No significant difference was also found between L2 male and L2 female
narratives (p > .05), except for mistakes in morphosyntax between L2 male narratives and
L2 female narratives (p < .05). This difference in morphosyntactical mistakes may be due
to the length of narratives—although there is no significant difference in text length, sen-
tence length or in the number of sentences between L2 males and L2 females (p > .05), L2
female narratives are usually longer than L2 male ones (average male narrative text length
is 85.12, average female narrative text length is 93.46).
As can be seen from Table  3, L1 narratives differ from L2 narratives in a number of
important ways. The most important and the most obvious thing is the difference in gram-
matical proficiency. There is significant difference between L1 and L2 narratives in refer-
ence, morphosyntax, and mistakes in morphosyntax (p < .0001). L2 learners demonstrate
a wide range of mistakes due to the strategies of overgeneralization, simplification, and
chaotic use of grammar markers.
Regarding complexity, L1 and L2 narratives are surprisingly equal. T-tests found no sig-
nificant differences in mean scores on complex or compound sentences in L1 and L2 narra-
tives. Predictably, there is a significant difference in mistakes made by L1 and L2 speakers.
In the corpora without unpaired texts the results were the same.
The final step of analysis was to assess vocabulary in narratives. The results are sum-
marized in Table 4.
There was no significant difference between L1 producing and L1 reproducing or
between L1 male and L1 female narratives (p > .05). No significant difference was also
found between L2 male and L2 female narratives (p > .05), except for IST in total (p < .05).
Table  4 shows that there is a significant difference in vocabulary used by L1 and L2
speakers. Mistakes in usage of lexical items can be easily explained by a low level of

13
Table 4  Vocabulary in narratives
Mean score for L1 Mean score for L2 Independent t-test Mean score Mean score Ind. t-test (by Ind. t-test (P
narratives (by L1) narratives (by L1) (by L1) for L1 nar- for L2 nar- type) L2 vs. RE
ratives (by ratives (by L2)
type) type)
Journal of Psycholinguistic Research

Usage mistakes (wrong meaning) − .3 All − 2.46 All p < .0001 All − .3 All − 2.46 All p < .0001 p = .02
CH − 3.2 CH p < .0001 P − .38 P − 1.89 P p < .0001
Non-CH − 1.64 Non-CH p < .0001 RE − .23 RE − 3.04 RE p < .0001
Usage mistakes (non-existent words) 0 All − 1.32 All p < .0001 All 0 All − 1.32 All p < .0001 p = .04
CH − 1.45 CH p < .0001 P 0 P − .89 P p < .0001
Non − 1.19 Non-CH p < .0001 RE 0 RE − 1.77 RE p < .0001
− CH
Words from other languages (code switching) 0 All − .06 All p = .083 All 0 All − .06 All p = .083 p = .549
CH − .04 CH p = .323 P 0 P − .04 P p = .368
Non-CH − .1 Non-CH p = .16 RE 0 RE − .09 RE p = .16
Vocabulary in total 99.7 All 96.16 All p < .0001 All 99.7 All 96.16 All p < .0001 p = .005
CH 95.32 CH p < .0001 P 99.62 P 97.18 P p < .0001
Non-CH 97.07 Non-CH p < .0001 RE 99.77 RE 95.09 RE p < .0001
IST in total 11.33 All 7.01 All p < .0001 All 11.33 All 7.01 All p < .0001 p = .313
CH 7.62 CH p < .0001 P 12.05 P 6.58 P p < .0001
Non-CH 6.33 Non-CH p < .0001 RE 10.57 RE 7.45 RE p = .002

13
Journal of Psycholinguistic Research

language proficiency as vocabulary acquisition directly depends on it. More interesting is a


significant difference in IST usage. L2 speakers tend to use fewer IST than native speakers.
In the corpora without unpaired texts, the results were the same, except for usage mis-
takes—wrong meaning and total vocabulary (in both cases the p-value for a difference
between L2 male and L2 female narratives was less than .05).
The narratives of L1 learners were also compared with L2 narratives regarding the level
of L2 learners. Our data contains narratives of A1, A2, B1, and B2 learners of Russian. As
the number of A1 narratives was rather small (N = 4), they were not compared with L1 nar-
ratives. The number of A2 narratives is 37, of B1 narratives is 19, and of B2 narratives is 29.
Let us now consider the narratives of A2 level in comparison with L1 narratives. The
results show that there is a certain trend to significance for a GAO sequence (p = .014). A
significant difference between L1 and A2 narratives was found out in IST usage in part A
(p = .01), text length in words (p = .001), sentence length in words (p = .003), vocabulary
in total (p < .0001), wrong meaning of words (p < .0001), non-existent words (p = .001),
IST in total (p = .001), morphosyntactical mistakes (p < .0001), morphosyntax in total
(p < .0001), mistakes in complexity (p = .001), and referential mistakes (p < .0001).
A comparison of the narratives of B1 level and L1 narratives reveals that a certain trend
to significance is in part A in total (p = .016) and in a GA/GO sequence (p = .039). A signif-
icant difference was discovered in IST usage in part A (p = .006), sentence length in words
(p < .0001), vocabulary in total (p < .0001), wrong meaning of words (p = .001), IST in
total (p = .001), morphosyntactical mistakes (p < .0001), morphosyntax in total (p < .0001),
and mistakes in complexity (p = .002).
Turning now to the experimental evidence on the narratives of B2 level, it can be seen
that there is a certain trend to significance text length in words (p = .022). The narratives
of B2 level significantly differ from L1 narratives in sentence length in words (p = .001),
vocabulary in total (p < .0001), wrong meaning of words (p < .0001), non-existent words
(p < .0001), IST in total (p < .0001), morphosyntactical mistakes (p < .0001), morphosyntax
in total (p < .0001), mistakes in complexity (p < .0001), and referential mistakes (p = .005).

Discussion

An initial objective of the study was to identify the relationship between narrative compe-
tence and language proficiency of adult L2 Russian learners.
The current research found that significant differences between L1 and L2 narratives lie
in text length and sentence length, used vocabulary, and morphosyntax. In general, these
differences do not depend on gender of participants, L1 of foreign students or the type of
narratives. Slight variations can be due to the fact that there are several unpaired texts (only
producing or only reproducing) written by participants.
The results indicate that the text length varies because of the length of sentences
only. As for text length in words, the lowest result was for the narratives of A2 level, as
expected (text length = 85.3). Texts become longer in words from A2 to B1 (text length
in B1 = 94.42) and after that stay more or less the same at higher levels (text length in
B2 = 92.69). Sentence length in words is approximately the same for all L2 narratives. The
data show that average sentence length in L1 producing significantly differs from L2 pro-
ducing, while there is no difference between L1 reproducing and L2 reproducing in average
sentence length. The possible explanation may lie in the type of narratives. For L2 learners,
it may be more complicated to produce long sentences than to reproduce them.

13
Journal of Psycholinguistic Research

Although the sentence length can depend on language proficiency, language proficiency
not necessarily influences the text length regarding the question of plot construction. The
data shows that L2 learners, regardless of their level, and native speakers used more or less
the same number of sentences to make a story. Because of this, it may be claimed that an
ability to organize a story around a theme (to make a plot) appeared to be unaffected by
language proficiency.
On the other hand, differences in used vocabulary and morphosyntax might be explained
by these values directly connected with language proficiency.
L2 learners often have problems with choosing words correctly—acquired vocabu-
lary usually depends on the level of a language user. It is strongly proved by the acquired
results: L2 narratives, regardless of level (A2, B1 or B2), significantly differ from L1 nar-
ratives by vocabulary in total, wrong meaning of words, and non-existent words. It may
seem surprising that these parameters in L2 narratives do not change from level to level,
but these results may be caused by the method itself which is quantitative. It may mean that
although all tested levels demonstrated the same results for a difference between L2 and L1
narratives (p ≤ .0001), a difference between levels may be qualitative.
Interestingly, only few L2 respondents gave names to the characters of their stories,
while it is quite typical of native Russian speakers. Other specific features of narratives
produced by native speakers are: irony, evaluation or conclusions, exclamations, diminu-
tives, gender variation, oppositions, emoticons, and imitation of the style of fairy tales.
Similar to vocabulary, L2 narratives, regardless of level, significantly differ from L1 nar-
ratives in morphosyntactical mistakes (p < .0001) and morphosyntax in total (p < .0001).
However, typology of mistakes is different. For instance, at B2 level L2 learners may use
the nominative case instead of the dative case to mark a semantic subject in grammatically
impersonal sentences, while at A2 level they may use the nominative case in all cases.
Another example is confusing parts of speech at A2/B1 level, while at B2 level L2 learners
do not make such mistakes. The study shows that typical morphosyntactic mistakes of L2
learners in general are various mistakes in agreement: wrong arguments (incorrect case) or
mistakes in subject-verb agreement (mostly due to the problems with acquisition of gen-
der agreement). Another common mistake is inability to distinguish between perfective
and imperfective aspects. One of the reasons for that may be the acquired language itself.
Rich Russian morphology makes L2 users keep in mind many grammar rules of agreement
between parts of a sentence. Although there remains a significant difference between L1
narratives and L2 narratives regardless of level, the number of morphosyntactical mistakes
decreases from level to level (A2 = 40.43, B1 = 38.21, B2 = 33.72).
As for reference, the higher their level is, the fewer referential mistakes L2 learners
make (the number of mistakes at A2 level is 2.49, at B1 level is 1.79, and at B2 level is
1.44). Despite that, L2 narratives still significantly differ from L1 narratives (p < .0001).
Most reference mistakes in L2 learners are caused because of two reasons. Firstly, it is
a gender or number disagreement or incompetence in a gender agreement. The results sug-
gest that it is the same lack of grammatical competence that causes mistakes in a subject-
verb agreement. Sentence (1) from an L2 narrative illustrates this point clearly.

(1) coбaкa видилa бaкeт. Oн дyмaл, чтo здecь ecть кoлбaca?


sobaka vidila baket. On dumal, čto zdes’ est’ kolbasa?
dog.F see.PRET.F packet he think.PRET.M that here is sausage
‘Then the dog (F) saw (F) the packet. He(M) thought (M) if there was a sausage.’

13
Journal of Psycholinguistic Research

Such mistakes cannot be fully regarded as referential ones as they do not demonstrate an
inability to bind anaphoric pronouns correctly. Instead, this type of mistake may be due to
low proficiency in using grammar gender and may be classified as morphosyntactic. It is
necessary to mention that similar mistakes were presented in two L1 narratives:

(2) Кoт гyлял пo бepeгy oзepa. Oнa yвидeлa бaбoчкy < …>


Kot gulyal po beregu ozera. Ona uvidela babočku
cat walk.PRET.M at bank lake.GEN she see.PRET.F butterfly
‘A cat (M) was walking (M) at the bank of the lake. She (F) saw (F) a butterfly.’

However, Russian native speakers made these mistakes by distraction, “forgetting” or


not paying attention to the gender of a character’s name in previous lines.
Secondly, L2 referential mistakes are caused by the use of third-person possessive pro-
nouns instead of possessive reflexives and vice versa:

(3) Maмa кoзёл и cвoи peбёнки cкaзaли блaгoдapнocть вopoнe.


mama kozyol i svoi rebenki skazali blagodarnost’ vorone
mother goat.M and oneself’s.PL children say.PRET.PL thank crow.DAT
‘The Mother-goat and self-owned children said thank you to the crow.’
(Instead of that, it should be in Russian: mama i ee deti—‘mother and her children’.)

(4) Oн xoтeл дocтaть eгo мять.


On hotel dostat’ ego myač
He want.PRET.M get.INF his ball
‘He wanted to get someone’s ball.’
(It should be in Russian: On hotel dostat’ svoy myach.—‘He wanted to get his own
ball.’)

Russian reflexives are person-unmarked and are usually bound by the subject of a clause
(Timberlake 1980; Avrutin 1994; Rappaport 1986, 2004; Grashchenkov and Grashchen-
kova 2006). Although binding properties of Russian reflexives are not unique in world lan-
guages (König et al. 2013), Russian reflexives are broadly regarded to be a challenge for L2
learners (Czeczulin 2007).
Interestingly, L2 adult learners are less likely than Russian monolingual children of
early age to make third-person pronoun binding mistakes. One such mistake is ambiguous
binding, where two or more potential antecedents are present in the left context, and it is
not clear to the listener which one is meant by the speaker. Ambiguous binding was found
only twice in L2 learners and 3 times in L1 speakers:

(5) Mышь бeжaлa cлишкoм быcтpo,


L2
Mysh’ bezhala slishkom bistro,
mouse.F run.PRET.F too fast
coбaкa нe дoгнaлa eё и oнa yдapилacь гoлoвoй.
sobaka ne dognala eyo i ona udarilas’ golovoy
dog.F not catch.PRET.F she.ACC​ and she hit.PRET.F head
‘A mouse (F) ran too fast, a dog (F) did not catch it and she (F) hit her head.’
(It is not clear who hit its head, a mouse or a dog.)

13
Journal of Psycholinguistic Research

(6) и eгo бeзмятeжнoгo бpaтцa зaмeтилa злaя и гoлoднaя лиca.


L1
i ego bezmyatezhnogo bratza zametila zlaya i golodnaya lisa
and his calm brother.ACC​ notice.PRET.F bad and hungry fox
Maть-кoзa ee нe зaмeтилa, oнa пилa вoдy из oзepa.
Mat’-kosa ee ne zametila, ona pila vodu iz ozera
mother-goat she.ACC​ not notice.PRET.F she drink. water from lake
PRET.F
‘and a bad and hungry fox.F noticed his calm brother. The Mother-goat.F did not notice her.F, she.F was drink-
ing from the lake.’
(It is not clear who was drinking, a goat or a fox.)

Another possible mistake for children is to introduce a referent with an anaphoric pro-
noun (an anaphoric pronoun is used in a demonstrative way, without any antecedent) (Gül-
zow and Gagarina 2007). By contrast, there is no-antecedent anaphora in our data for L2
learners.
These results suggest that basic principles of anaphoric binding are acquired in early
childhood in L1 and are transferred to L2 without any difficulty. Thus, it can be assumed
that unambiguous anaphoric binding belongs to cognitive competence rather than to a lin-
guistic one.
Based on this assumption, we may claim that the significant difference in reference in
adults’ L1 and L2 narratives is a result of both low grammatical proficiency in L2 speakers
and specific binding features of Russian possessives that cause confusion in L2 learners.
Turning now to complexity of narratives, it is necessary to mention that L2 learners may
construct sentences that are unnatural in Russian:

(7) caбaкa yвидeл мышкy cидит пoд дepeвoм.


sabaka uvidel myshku sidit pod derevom
dog.F see.PRET.M mouse.ACC​ sit.PRES under tree
‘A dog saw a mouse sitting under the tree.’
(In Russian it should be: sobaka uvidela, čto myshka sidit—‘a dog saw that a mouse sits’.)

L2 participants are sometimes not able to produce correct complex sentences because
of problems with constructing subordinate clauses. However, there are more similarities
between L1 and L2 narratives in complexity. Although there is a significant difference in
mistakes between L1 and L2 narratives (regardless of L2 level), there is no significant dif-
ference in the number of clauses in complex or compound sentences in L1 and L2 narra-
tives. It may be due to the fact that ability to construct complex and compound sentences is
a cognitive ability and is not connected with language proficiency. Both groups of partici-
pants produced equally long sentences. It may permit an exclusion of sentence complexity
out of adult’s language proficiency criteria.
On the question of narrative macrostructure, our results indicate that L1 and L2 nar-
ratives are equal in most cases. However, the results demonstrated that the L2 narratives
of A2 and B1 levels differ from L1 narratives in some aspects of macrostructure (a GAO
sequence for A2 and a GA/GO sequence for B1), B2 narratives do not differ from L1 nar-
ratives in macrostructure. Surprisingly, at A2/B1 level L2 learners are more likely to use a
GAO sequence than native speakers. At B2 level L2 learners use this structure as L1 speak-
ers do. In general, it was found out that native speakers and L2 learners at B2 level are

13
Journal of Psycholinguistic Research

more likely to shorten a structure of an episode, using only two elements. Other variations
discovered were not significant.
Together with similarities between L1 and L2 narratives in complexity, it may mean that
the level of language proficiency of L2 participants did not prevent them from producing
a coherent and well-structured story. An adult L2 learner already masters two elements of
narrative competence (to produce a coherent story and to organize it around a theme) in his
or her native language, and is able to transfer both these skills to L2. These results are in
line with other research assuming that L2 writing proficiency is more correlated with L1
writing proficiency than with L2 linguistic knowledge (Schoonen et al. 2003).
What can be also interesting is the investigation of possible syntactic priming effects in
L2 narratives at reproducing. As our participants first heard and then reproduced a story
in written, it may seem that syntactic priming will affect their narratives. Surprisingly,
the results show no presence of syntactic priming—no significant difference was found
between L2 producing and L2 reproducing (p > .05) for microstructure (see Table 3).

Limitations and future research

As this research is naturally limited by the number of participants, it is more qualitative


than quantitative. The results may be considered as reliable, despite that L2 group was
bigger than L1 group. The ratio of male and female participants was not the same for two
groups, but tests did not show any significant differences between male and female narra-
tives, in both L1 and L2 groups.
This study represents interesting results. The results suggest that L2 speakers are less
likely than native speakers to use IST. One of the reasons can be the limited acquired lexi-
con of L2 learners. However, as IST describes feelings, emotions, mental activities etc.,
IST usage is strongly connected with cognitive and psychological abilities of a speaker
(Siller, Swanson, Serlin, & George 2014). We were not able to investigate possible cogni-
tive or psychological impairments of L2 participants in this research so it was not possible
to discover the precise underlying reasons.
Another result that could not be clearly explained within this research is the difference
between L1 and L2 speakers in naming of characters and in the usage of stylistic devices
in narratives. It is probable that L2 learners pay more attention to the construction of a
story (a base) and may have no possibility to enhance the story with stylistic devices. The
reason could also lie in different storytelling traditions. For future research, it is necessary
to assess the effects of native language and native culture on production of L2 narratives.

13
Journal of Psycholinguistic Research

Conclusion

The aim of the present research was to examine the relationship between narrative compe-
tence and language proficiency of L2 Russian speakers. This study investigated the ques-
tion if macro and microstructural narrative subcompetences depend on language profi-
ciency and to what extent.
The investigation of microstructure has shown that a significant difference between
L1 and L2 narratives lies in reference and morphosyntax. Generally, both referential and
morphosyntactical mistakes are determined by low language proficiency of L2 speak-
ers. The study shows that reference and morphosyntax are getting better from A2 to B2
level, however, this improvement is not enough—a significant difference in comparison
with L1 narratives remains even at B2 level. It can be concluded that these skills are not
completely developed till proficient levels (C1/C2).
Language proficiency also influences used vocabulary. There is a significant differ-
ence between L1 and L2 narratives (regardless of level) in vocabulary, so it can be sug-
gested that this skill is also not completely developed at A2/B1/B2 level.
This study has also shown that L2 adult narratives have no significant difference
in macrostructure and complexity with L1 adult narratives. Therefore, language profi-
ciency of L2 learners does not influence macrostructure and complexity in their nar-
ratives that are core elements of a narrative. Nevertheless, one observation was made.
The MAIN method supposes that for children completeness of narrative macrostruc-
ture depends on a speaker’s language proficiency level. It means that highly proficient
children generally produce episodes with a full structure (GAO, or goal—attempt—out-
come). However, L1 adults tend to reduce this structure to GA or AO sequences only,
keeping the necessary meaning. L2 learners at B2 level are close enough to L1 speak-
ers in the ratio between complete and incomplete episodes, while students at A2/B1
level are more likely to produce complete episodes than native speakers do. It may mean
that an ability to shorten narrative structure without loss in meaning is a component
of narrative competence that depends on language proficiency level and discriminates
between lower and higher proficient L2 learners. Aside from that, no other significant
difference was found in macrostructure between L1 speakers and L2 learners regardless
of their level.
It seems possible that macrostructural subcompetence of L2 Russian learners does not
depend on their language proficiency, except for an ability to produce structurally shorter
episodes at B2 level, and microstructural subcompetence of L2 Russian learners depends
on their language proficiency in some ways (lexical and grammatical competences).
This research extends our knowledge of producing adult L2 Russian narratives. A key
strength of the present study was the comparison of adult L1 narratives with adult L2
narratives. This study has several practical applications. Firstly, it provides the assess-
ment tool for microstructure for the Russian language that can serve as an example for
other languages. Secondly, the results may be used in teaching Russian as a foreign lan-
guage. This research show strengths and weaknesses of L2 narratives and demonstrates
possible difficulties for L2 learners of the Russian language. Thirdly, the data from adult
L1 Russian narratives can be used in further research for comparison with narratives
from other L2 learners, L1 children, heritage Russian speakers, and L1 speakers with
language impairments.

13
Journal of Psycholinguistic Research

Appendix: Assessment Tools for Microstructure

Morphosyntax

See Table 5.

Table 5  Scoring sheet for morphosyntax


Value If no

Apprehensibility − 100
(total score for coherence
in the sentence is 0)
Value If absent Mistakes
Subject (if necessary) − 4/each clause (sentence) − 2 incorrect case
− 2 incorrect number
Mistakes are taken into consideration if
and only if it is NOT possible to under-
stand that this is a subject.
Object (if necessary) − 4/each clause (sentence) − 2 incorrect case
− 2 incorrect number
Mistakes are taken into consideration if
and only if it is NOT possible to under-
stand that this is an object.
Other arguments (if necessary) − 4/each clause (sentence) − 2 incorrect case
− 2 incorrect number
Mistakes are taken into consideration if
and only if it is NOT possible to under-
stand that this is an argument.
Attribute (if necessary) − 6/each clause (sentence) − 2 incorrect number
− 2 incorrect gender
− 2 incorrect case
Verb (if necessary) − 10/each clause (sentence) − 2 incorrect tense
− 2 incorrect aspect
− 2 voice (for each)
− 4 no agreement with the subject
Predicate other than verb (if − 6/each clause (sentence) − 4 no agreement with the subject
necessary) − 2 incorrect case
Word order − 2
Conjunctions between parts of − 2/each
a sentence (e.g. homogeneous
predicates)
Wrong grammar class (e.g. an − 2/each
adverb instead of an adjective)
Prepositions (wrong usage) − 2/each − 2/each
Double negation − 2/each
Adjectives (if necessary) − 2 − 2 incorrect degree of comparison
Participle (if necessary) − 6 − 2 incorrect aspect
− 2 incorrect voice
− 2 incorrect tense
Adverbial participle (if neces- − 6 − 2 incorrect aspect
sary) − 2 incorrect voice
− 2 wrong usage

13
Journal of Psycholinguistic Research

Reference

See Table 6.

Table 6  Scoring sheet for reference


Value If absent Mistakes

Personal pronouns (if necessary) − 6 − 2 no antecedent


− 2 wrong gender
− 2 ambiguous binding
Possessive pronouns − 8 − 2 incorrect number
− 2 incorrect gender
− 2 ambiguous binding
− 2 no antecedent
Demonstrative pronouns (if necessary) − 4 − 2 ambiguous binding
− 2 wrong pronoun
Relative pronouns (if necessary) − 4/− 8 − 2 ambiguous binding
− 2 wrong case (if necessary)
− 2 wrong pronoun
− 2 number (if necessary)
Reflexive pronouns (if necessary) − 4 − 2 wrong pronoun
− 2 ambiguous binding
Indefinite pronouns (if necessary) − 4 − 2 wrong pronoun
− 2 wrong series
Not proper or correct reference (e.g. dangling or − 2/each
ambiguous reference)

Complexity

See Table 7.

Table 7  Scoring sheet for complexity


Value Score Mistakes

Complex sentence 2/each clause − 2 no tense agreement


− 2 wrong order of clauses
− 2 if absent when necessary
Compound sentence 2/each clause − 2 no tense agreement
− 2 wrong order of clauses
Link words/connectors (if appropriate/necessary) − 2/each if absent
Conjunctions between sentences/clauses (if appropri- − 2/coordinative
ate/necessary) − 2/subordinative
(wrong usage)

13
Journal of Psycholinguistic Research

Vocabulary

See Table 8.

Table 8  Scoring sheet for vocabulary


Value Score

Usage mistakes (wrong meaning) − 1/each (if mistakes do


not affect communica-
tion)
− 2/each (if mistakes
affect communication)
Usage mistakes (non-existing words) − 2/each
Words from other languages (code switching) − 2/each
IST
IST Total number in tokens

References
Akinci, M.-A., Jisa, H., & Kern, S. (2001). Influence of L1 Turkish on L2 French narratives. In L. Verho-
even & S. Strömqvist (Eds.), Narrative development in a multilingual context studies in bilingualism.
Amsterdam: John Benjamins Publishing. https​://doi.org/10.1075/sibil​.23.
Alderson, J. C. (2005). Diagnosing foreign language proficiency: The interface between learning and
assessment. London: Continuum.
Almgren, M., & Manterola, I. (2016). The development of narrative skills in learners of Basque as a second
language. Education Inquiry, 7(1), 27–46. https​://doi.org/10.3402/edui.v7.27627​.
Avrutin, S. (1994). The structural position of bound variables in Russian. Linguistic Inquiry, 25(4), 709–
727. Retrieved November 20, 2018 from https​://www.jstor​.org/stabl​e/41788​82.
Bachman, L. F., & Palmer, A. S. (1996). Language testing in practice. Oxford: Oxford University Press.
Berman, R. A. (1995). Narrative competence and storytelling performance: How children tell stories
in different contexts. Journal of Narrative and Life History, 5(4), 285–313. https​://doi.org/10.1075/
jnlh.5.4.01nar​.
Carroll, J. B. (1961). Fundamental considerations in testing for English language proficiency of foreign stu-
dents. Testing. Washington, DC: Center for Applied Linguistics.
Cummins, J. (1980). The cross-lingual dimensions of language proficiency: Implications for bilingual edu-
cation and the optimal age issue. TESOL Quarterly, 14(2), 175–187. https​://doi.org/10.2307/35863​12.
Czeczulin, A. O. (2007). The Russian reflexive in second-language acquisition: Binding preferences and
L1 Transfer. Ph.D. dissertation. University of Pittsburgh. Retrieved November 20, 2018 from https​
://d-schol​arshi​p.pitt.edu/9022/1/czecz​ulina​o_etd20​07.pdf.
De Clercq, B., & Housen, A. (2017). A cross-linguistic perspective on syntactic complexity in L2 develop-
ment: Syntactic elaboration and diversity. The Modern Language Journal, 101, 315–334. https​://doi.
org/10.1111/modl.12396​.
Gagarina, N., Klop, D., Kunnari, S., Tantele, K., Välimaa, T., Balčiūnienė, I., Bohnacker, U., & Walters,
J. (2012). Part I. MAIN: Multilingual assessment instrument for narratives. ZAS Papers in Linguis-
tics, 56, 1–139. Retrieved November 20, 2018 from http://publi​katio​nen.ub.uni-frank​furt.de/front​door/
index​/index​/docId​/34782​.
Gagarina, N., Klop, D., Kunnari, S., Tantele, K., Välimaa, T., Balčiūnienė, I., et al. (2015). Assessment of
narrative abilities in bilingual children. In S. Armon-Lotem, J. de Jong, & N. Meir (Eds.), Assessing
multilingual children (pp. 243–269). Bristol: Multilingual Matters.
Goldschmidt, P., & Hakuta, K. (2017). Incorporating English learner progress into state accountabil-
ity systems. Washington DC: Council of Chief State School Officers. Retrieved from: https​://eric.
ed.gov/?id=ED578​899.

13
Journal of Psycholinguistic Research

Gorman, B. K., Bingham, G. E., Fiestas, C. E., & Terry, N. P. (2016). Assessing the narrative abilities of
Spanish-speaking preschool children: A Spanish adaptation of the narrative assessment protocol. Early
Childhood Research Quarterly, 36, 307–317. https​://doi.org/10.1016/j.ecres​q.2015.12.025.
Grashchenkov, P., & Grashchenkova, A. (2006). Possessive reflexives in Russian. In Proceedings of FASL-
15. Toronto, Ann Arbor. Retrieved November 20, 2018 from https​://istin​a.msu.ru/publi​catio​ns/artic​
le/33459​69/.
Gülzow, I., & Gagarina, N. (2007). Noun phrases, pronouns and anaphoric reference in young children nar-
ratives. ZAS Papers in Linguistics, 48, 203–223. Retrieved November 20, 2018 from https​://publi​katio​
nen.ub.uni-frank​furt.de/opus4​/front​door/index​/index​/docId​/31017​.
Harley, B., Allen, P., Cummins, J., & Swain, M. (Eds.). (1990). The development of second language pro-
ficiency (Cambridge Applied Linguistics). Cambridge: Cambridge University Press. https​://doi.
org/10.1017/CBO97​81139​52456​8.
Hochman, O., & Davidov, E. (2014). Relations between second-language proficiency and national identifi-
cation: The case of immigrants in Germany. European Sociological Review, 30(3), 344–359. https​://
doi.org/10.1093/esr/jcu04​3.
Hulstijn, J. H. (2011). Language proficiency in native and nonnative speakers: An agenda for research and
suggestions for second-language assessment. Language Assessment Quarterly, 8(3), 229–249. https​://
doi.org/10.1080/15434​303.2011.56584​4.
IBM Corp. Released 2015. IBM SPSS statistics for windows, Version 23.0. Armonk: IBM Corp.
John-Steiner, V., & Panofsky, C. (1992). Narrative competence: Cross-cultural comparisons. Journal of
Narrative and Life History, 2(3), 219–233. https​://doi.org/10.1075/jnlh.2.3.03com​.
Justice, L. M., Bowles, R. P., Kaderavek, J. N., Ukrainetz, T. A., Eisenberg, S. L., & Gillam, R. B. (2006).
The index of narrative microstructure: A clinical tool for analyzing school-age children’s narra-
tive performances. American Journal of Speech-Language Pathology, 15(2), 177–191. https​://doi.
org/10.1044/1058-0360(2006/017).
Justice, L. M., Bowles, R., Pence, K., & Gosse, C. (2010). A scalable tool for assessing children’s lan-
guage abilities within a narrative context: The NAP (Narrative Assessment Protocol). Early Childhood
Research Quarterly, 25(2), 218–234. https​://doi.org/10.1016/j.ecres​q.2009.11.002.
Karlsen, J., Geva, E., & Lyster, S. (2016). Cognitive, linguistic, and contextual factors in Norwegian sec-
ond language learner’s narrative production. Applied Psycholinguistics, 37(5), 1117–1145. https​://doi.
org/10.1017/S0142​71641​50005​1X.
Kempe, V., & MacWhinney, B. (1998). The acquisition of case-marking by adult learners of Russian and
German. Studies in Second Language Acquisition, 20, 543–587. https​://doi.org/10.1017/S0272​26319​
80040​45.
König, E., Siemund, P., & Töpper, S. (2013). Intensifiers and reflexive pronouns. In Dryer, M. S., &
Haspelmath, M. (Eds.), The world atlas of language structures online. Leipzig: Max Planck Institute
for Evolutionary Anthropology. Retrieved November 20, 2018 from http://wals.info/chapt​er/47.
Labov, W., & Waletzky, J. (1967). Narrative analysis. In J. Helm (Ed.), Essays on the verbal and visual arts
(pp. 12–44). Seattle: University of Washington Press.
Lado, R. (1961). Language testing. London: Longmans, Green, and Co.
Llurda, E. (2000). On competence, proficiency, and communicative language ability. International Journal
of Applied Linguistics, 10, 85–96. https​://doi.org/10.1111/j.1473-4192.2000.tb001​41.x.
Lyons, S., & Dadey, N. (2017). Considering English language proficiency within systems of accountabil-
ity under the every student succeeds act. Washington, DC: Council of Chief State School Officers.
Retrieved November 20, 2018 from https​://www.nciea​.org/sites​/defau​lt/files​/pubs-tmp/Consi​derat​
ions%20for​%20ELP​%20ind​icato​r%20in%20ESS​A_03271​7.pdf.
Montanari, S. (2004). The development of narrative competence in the L1 and L2 of Spanish–English bilin-
gual children. International Journal of Bilingualism, 8(4), 449–497. https​://doi.org/10.1177/13670​
06904​00800​40301​.
Ochs, E., & Capps, L. (2001). Living narrative: Creating lives in everyday storytelling. Cambridge, MA:
Harvard University Press.
Pakulak, E., & Neville, H. J. (2010). Proficiency differences in syntactic processing of monolingual native
speakers indexed by event-related potentials. Journal of Cognitive Neuroscience, 22(12), 2728–2744.
https​://doi.org/10.1162/jocn.2009.21393​.
Pavlenko, A. (2006). Narrative competence in a second language. In H. Byrnes, D. H. Weger, & K. Sprang
(Eds.), Educating for advanced foreign language capacities: Constructs, curriculum, instruction,
assessment (pp. 105–117). Washington: Georgetown University Press.
Peña, E. D., Gillam, R. B., Malek, M., Ruiz-Felter, R., Resendiz, M., Fiestas, C., et  al. (2006). Dynamic
assessment of school-age children’s narrative ability: An experimental investigation of classification

13
Journal of Psycholinguistic Research

accuracy. Journal of Speech, Language, and Hearing Research, 49(5), 1037–1057. https​://doi.
org/10.1044/1092-4388(2006/074).
Petersen, D. B., Chanthongthip, H., Ukrainetz, T. A., Spencer, T. D., & Steeve, R. W. (2017). Dynamic
assessment of narratives: Efficient, accurate identification of language impairment in bilingual students.
Journal of Speech, Language, and Hearing Research, 60(4), 983–998. https​://doi.org/10.1044/2016_
JSLHR​-L-15-0426.
Pinto, G., Tarchi, C., & Accorti Gamannossi, B. (2018). Kindergarteners’ narrative competence across
tasks and time. The Journal of Genetic Psychology, 179(3), 143–155. https​://doi.org/10.1080/00221​
325.2018.14537​75.
Rappaport, G. C. (1986). On anaphor binding in Russian. Natural Language and Linguistic Theory, 4(1),
97–120. https​://doi.org/10.1007/BF001​36266​.
Rappaport, G. C. (2004). The syntax of possessors in the nominal phrase: Drawing the lines and deriving
the forms. Possessives and beyond: Semantics and syntax. In J.-Y. Kim, B. H. Partee, & Y. A. Lander
(Eds.), UMOP 2x (University of Massachusetts Occasional Papers) (pp. 243–261). Amherst: GLSA
Publications.
Read, J. (2015). Assessing English proficiency for university study. London: Palgrave Macmillan. https​://doi.
org/10.1057/97811​37315​694.
Romanova, N. (2017). Processing of gender and number agreement in Russian as a second language. Studies
in Second Language Acquisition, 39, 97–128. https​://doi.org/10.1017/S0272​26311​60000​12.
Schneider, P., & Hayward, D. (2010). Who does what to whom: Introduction of referents in children’s sto-
rytelling from pictures. Language, Speech, and Hearing Services in Schools, 41, 459–473. https​://doi.
org/10.1044/0161-1461(2010/09-0040).
Schneider, P., Hayward, D., & Dubé, R. V. (2006). Storytelling from pictures using the Edmonton narrative
norms instrument. Canadian Journal of Speech-Language Pathology and Audiology, 30(4), 224–238.
Retrieved November 20, 2018 from https​://www.cjslp​a.ca/detai​l.php?ID=936&lang=en.
Schoonen, R., Van Gelderen, A., De Glopper, K., Hulstijn, J., Simis, A., Snellings, P., et al. (2003). First
language and second language writing: The role of linguistic fluency, linguistic knowledge and meta-
cognitive knowledge. Language Learning, 53, 165–202. https​://doi.org/10.1111/1467-9922.00213​.
Shapiro, L. R., & Hudson, J. A. (1991). Tell me a make-believe story: Coherence and cohesion in young
children’s picture-elicited narratives. Developmental Psychology, 27(6),  960–974.  https​://doi.
org/10.1037/0012-1649.27.6.960.
Shuai, L. (2014). The effects of different levels of linguistic proficiency on the development of L2 Chi-
nese request production during study abroad. System, 45, 103–116. https​://doi.org/10.1016/j.syste​
m.2014.05.001.
Siller, M., Swanson, M. R., Serlin, G., & George, A. (2014). Internal state language in the storybook nar-
ratives of children with and without autism spectrum disorder: Investigating relations to theory of
mind abilities. Research in Autism Spectrum Disorders, 8(5), 589–596. https​://doi.org/10.1016/j.
rasd.2014.02.002.
Thorne, J. C. (2017). Accentuate the negative: Grammatical errors during narrative production as a clin-
ical marker of central nervous system abnormality in school-aged children with fetal alcohol spec-
trum disorders. Journal of Speech, Language, and Hearing Research, 60(12), 3523–3537. https​://doi.
org/10.1044/2017_JSLHR​-L-17-0128.
Tilstra, J., & McMaster, K. (2007). Productivity, fluency, and grammaticality measures from narratives:
Potential indicators of language proficiency? Communication Disorders Quarterly, 29(1), 43–53. https​
://doi.org/10.1177/15257​40108​31486​6.
Timberlake, A. (1980). Reference conditions on Russian reflexivization. Language, 56, 777–796. https​://doi.
org/10.2307/41348​8.
Tomasello, M. (2014). Origins of human communication (2nd ed.). Cambridge: MIT Press.
van der Veen, C., & Poland, M. (2012). Dynamic assessment of narrative competence. In B. van Oers (Ed.),
Developmental education for young children, international perspectives on early childhood education
and development. Dordrecht: Springer. https​://doi.org/10.1007/978-94-007-4617-6_7.
van Oers, B. (2007). Helping young children to become literate: The relevance of narrative competence for
developmental education. European Early Childhood Education Research Journal, 15(3), 299–312.
https​://doi.org/10.1080/13502​93070​16797​18.
Verhoeven, L., & de Jong, J. H. A. L. (1992). Modeling and assessing language proficiency. In L. Verho-
even & J. H. A. L. de Jong (Eds.), The construct of language proficiency: Applications of psycho-
logical models to language assessment. Amsterdam: John Benjamins Publishing Company. https​://doi.
org/10.1075/z.62.

13
Journal of Psycholinguistic Research

Wilkinson, L. C., Silliman, E. R., Nitzberg, L. A., & Aurilio, M. (1993). Narrative analysis: Filter-
ing individual differences in competence. Linguistics and Education, 5(2), 195–210. https​://doi.
org/10.1016/0898-5898(93)90036​-A.
Zanazanian, P., & Popa, N. (2018). Using a narrative tool to help Quebec English-speaking students produce
personal histories of belonging. Learning Landscapes Journal: Teaching and Learning with Stories,
11(2), 365–379. Retrieved November 20, 2018 from https​://www.learn​ingla​ndsca​pes.ca/index​.php/
learn​land/artic​le/view/969.

Publisher’s Note  Springer Nature remains neutral with regard to jurisdictional claims in published maps and
institutional affiliations.

13

You might also like