Second and Foreign Language

Assessment
JAMES E. PURPURA
Teachers College, Columbia University
Program in Applied Linguistics and TESOL
Department of Arts and Humanities
525 West 120th St.
New York, NY 10027
Email: jp248@tc.columbia.edu

This article summarizes some of the main issues, concerns, and debates that have ensued over the years
in the field of L2 assessment and shows how past concerns have shaped contemporary L2 assessment
research and practice. The article first describes what L2 assessment is and what it entails, arguing that
notions of L2 assessment have been broadened over the years to keep pace with contemporary inter-
pretations of how information from assessments in large-scale or classroom contexts is used to make
decisions. It then describes in some detail four approaches to construct definition, showing how these
approaches have influenced what gets assessed. Finally, the paper discusses a range of other topics of
current and future importance for L2 assessment theory, research, and practice.
Keywords: L2 assessment; L2 testing; L2 validation; L2 construct definition; models of L2 proficiency

OVER THE YEARS, SOCIETAL, ECONOMIC,
geopolitical, and technological forces in the work-
place have increased the knowledge, skills, and
abilities (KSAs) that people need to perform their
jobs. We are now asked to read, listen, and syn-
thesize large amounts of information from several
sources via multiple modalities; search for infor-
mation, judge its accuracy, and evaluate its appli-
cability; and use communication technologies to
collaborate in teams whose members represent
a diverse global community (National Research
Council, 1999, 2001). Importantly, many of us are
asked to do this in a second, foreign, or heritage
language (L2), requiring competencies for com-
municating ideas and establishing relationships
in culturally respectful ways. Such demands have
sparked the need to have disciplinary skills, along
with linguistic skills, to communicate in complex
ways across diverse contexts via multiple modal-
ities. To succeed in this environment, L2 users
must demonstrate that they have the skills needed
The Modern Language Journal, 100 (Supplement 2016)
DOI: 10.1111/modl.12308
0026-7902/16/190–208 $1.50/0

C 2016 The Modern Language Journal
to process information, reason from evidence,
make decisions, solve problems, self-regulate, col-
laborate, and learn—and they need to do this in
their L2. This reality has created performance de-
mands for all aspects of education, but especially
for assessment, where the public depends on as-
sessments to do everything from diagnosing learn-
ing needs to verifying the achievement of stan-
dards in educational and professional settings.
To keep pace with changing workforce and ed-
ucational demands, the field of L2 assessment has
been tasked with determining whether L2 users
can perform high-level skills in the L2, or whether
they have acquired sufficient linguistic resources,
along with other relevant competencies, to ben-
efit from L2 and disciplinary instruction. The
changing nature of work and education has also
required L2 assessments to tap into a broader
range of competencies over the years, competen-
cies that measure more complex linguistic, so-
ciocognitive, and sociocultural skills and, to some
extent, the ability to engage affectively with top-
ical content on assessments in new ways. Finally,
these changes have led educators and testing
experts to rethink the KSAs they aim to tap
into and to reconsider how assessments might be
James E. Purpura
designed, developed, and validated before being
used for decision making in a range of contexts.
Besides debating the theoretical components
of L2 proficiency, L2 educators and testing
experts have also capitalized on new commu-
nication and digital technologies (e.g., Skype,
digital video clips) and analytic procedures (e.g.,
automated scoring, corpus analyses) for inclusion
into the assessment system, thereby impacting
design, development, delivery, scoring, and vali-
dation. They have also embraced the use of new
and sophisticated measurement practices into
assessment systems. And they have used these
innovations in both large-scale and classroom
contexts. However, despite these advances and
the many theoretical and methodological affini-
ties assessment experts have with other areas of
applied linguists (e.g., L2 analysis, acquisition,
use, pedagogy), L2 assessment as a field has
often struggled to influence how other applied
linguists (e.g., SLA researchers) conceptualize,
operationalize, and validate the assessments they
use or even how other applied linguists (e.g.,
discourse analysts) might benefit from trans-
disciplinary research with testers. Rather than
seeing assessment as an organic part of applied
linguistics, L2 assessment is still often viewed
as an afterthought, or as a craft. Consequently,
the real potential that L2 assessment has for
transdisciplinary understandings and practices in
applied linguistics is, in my view, yet to come.
This article provides an overview of the main is-
sues, concerns, and debates in contemporary L2
assessment theory, research, and practice. I will
first describe what L2 assessment is and what it en-
tails, arguing that notions of L2 assessment have
broadened over the years to keep pace with differ-
ent interpretations of how information from as-
sessments in large-scale or classroom contexts is
used to make decisions. I will then describe four
approaches to construct definition and show how
these approaches have influenced what gets as-
sessed. While this discussion might reflect an ad-
mittedly narrow but traditional conceptualization
of assessment as a ‘test,’ these debates have clear
relevance for assessments in nontesting situations.
Finally, I will discuss a range of other topics of cur-
rent and future resonance.
WHAT DOES LANGUAGE ASSESSMENT
ENTAIL?
Language assessment is a broad term referring
to a systematic procedure for eliciting test and
nontest data (e.g., a teacher checklist of student
performance) for the purpose of making infer-
191
ences or claims about certain language-related
characteristics of an individual. In other words,
the term assessment refers not only to formal tests
such as the TOEFL or an end-of-chapter evalua-
tion, but also to other methods of obtaining in-
formation about KSAs such as by observing L2
performance during pair work or by asking learn-
ers to report their understandings and uncertain-
ties. The goal behind all L2 assessments is to elicit
L2 performance from an individual under certain
conditions so that performance consistencies can
be interpreted and used to produce records such
as scores, verbal descriptions, or mental notes. In-
terpretations from these records are then used as
evidence for making decisions.
One quality of all L2 assessments, according
to Bachman and Palmer (2010), is that assess-
ments are based on substantive grounding. In
other words, they are designed to elicit informa-
tion rooted in a principled and verifiable body of
content, coming from a lesson in a textbook, a
syllabus, standards, or a model of L2 proficiency.
A second quality is that assessments are goal-
oriented in that they have an intended purpose,
even though in many spontaneous, classroom-
based assessments embedded in instruction the
purpose may be more implicit than explicit. A
final quality of all assessments is that the proce-
dures used to elicit information involve varying
degrees of systematicity, ranging from very con-
trolled tests to far less controlled assessments as in
routinized teacher protocols during instruction.
More specifically, an L2 test, measure, or
measurement instrument is a special type of
assessment designed to control: (a) the type
of L2 behavior elicited from a content domain
under certain conditions, (b) the systematicity
of the elicitation procedure, and (c) the stan-
dardized procedures for scoring performance
and generating performance records. Other
measures designed to assess social-psychological
or demographic factors like student affective
dispositions are sometimes referred to as mea-
surement scales, but for all intents and purposes,
these have characteristics similar to tests, and are
treated as such. For example, the ACTFL oral
interview1 is both an assessment and in the form
of a test (often used synonymously), whereas the
principled observation of student performance
in class is an assessment, but never a test.
Claims or inferences about a user’s L2 at-
tributes, made on the basis of an assessment,
relate to constructs, such as L2 knowledge, L2
ability, or linguistic complexity, whose definition
must be constructed from research, theory, stan-
dards, accumulated experience, or principled
192
practice. Constructs cannot be observed directly,
but can only be inferred from concrete opera-
tionalizations of the construct by means of tasks
or other less formal protocols (e.g., naturalistic
conversations) that elicit performance data. In
L2 assessment, we are specifically interested in
eliciting evidence of L2 performance under
certain conditions so that we can make claims
about what L2 learners or users know, what
L2 skills they have, and the extent to which
they can use L2 resources, along with a host of
other resources (e.g., topical, sociocognitive),
to communicate effectively within or across
contexts.
Technically speaking, assessment refers only to
the elicitation and collection of performance data
for some specific purpose; however, a discussion
of ‘what assessment is’ should also address what
we do with the performance records once col-
lected. This gives rise to the topic of interpretation.
As assessment data are gathered and records gen-
erated with respect to some specific purpose, we
need to consider the meaning of these records
in terms of the claims they are intended to make
about an individual’s attributes or some other fea-
ture of interest (e.g., the characteristics of L2 pro-
duction). The process of deriving and justifying
meaning from assessment records is at the heart
of validation. These record-based interpretations
are then used as an evidentiary basis for making
assessment (or research) decisions. The system-
atic use of evidence for the purpose of making de-
cisions is then referred to as evaluation. According
to Bachman and Palmer (2010), “Evaluation in-
volves making value judgments and decisions on
the basis of information, and gathering informa-
tion to inform decisions is the primary purpose for
which language assessments are used” (p. 21, orig-
inal emphasis).
As L2 educators and researchers, we use inter-
pretations of assessments to make a wide range
of decisions. In classrooms, this involves decisions
about readiness (i.e., diagnosis), progress, mas-
tery, retention, and promotion, or decisions about
teaching, learning, and curricula. In schools, we
make decisions about proficiency, selection, and
placement. And in society, we make decisions
about certification and accountability. All these
decisions come with consequences to the stake-
holders, or the individuals mostly impacted by the
decision. For example, university selection deci-
sions based on proficiency test scores are obvi-
ously high stakes given the major consequences
these decisions have on the applicants’ lives,
whereas decisions about whether learners need
further practice based on a quiz are low stakes since
The Modern Language Journal, 100, Supplement 2016
the consequences to the students’ lives are rela-
tively minor.
As might be imagined, using score-based inter-
pretations to make high-stakes decisions does not
occur in a sociocultural or political vacuum, even
in classroom contexts. Rather, the use of assess-
ment information cannot be fully understood or
evaluated without considering the specific use(s)
for which assessments are intended, as well as the
potential consequences of these uses. Therefore,
a discussion of the consequences of assessment
use, especially as this relates to issues of fairness
and social justice, is currently part and parcel of
a discussion of what assessment is (see Kunnan,
2004; McNamara & Ryan, 2011).
This naturally leads to a discussion of the
consequences of assessment misuse, given the
number of situations in which assessments are
(un)intentionally used in ways for which they were
not intended or in ways in which social benefi-
cence is not prioritized, thereby leading to nega-
tive consequences for stakeholders. This situation
may arise, for example, in research contexts when
readily available measures from previous studies
are used for purposes other than the ones for
which they were designed. Since the two contexts
are different, the construct may be measured too
broadly or narrowly for use in both contexts, re-
sulting in low score variability and uninterrupt-
able results. The unjustified score interpretations
and ensuing decisions may well lead to spurious
claims with unintended, negative consequences
(Purpura, Brown, & Schoonen, 2015). A similar
situation may arise in classroom settings where er-
roneous assumptions of learning are made about
a student’s KSAs based on inadequate perfor-
mance data. Given the importance of the social
consequences of assessments, we will discuss this
topic in greater detail later on.
Finally, current views of ‘what L2 assessment is’
would certainly recognize ‘assessment’ in its sim-
plest form as a principled collection of informa-
tion elicited under certain conditions for some in-
tended purpose, leading to performance that can
be scored or characterized verbally, analyzed, and
interpreted to produce assessment records. How-
ever, few assessment experts would now agree that
assessment begins and ends with data collection,
scoring, and analysis. Most now recognize that as-
sessments are used to make decisions in real-world
contexts, involving factors outside the assessment
itself, where the probability of making the ‘right’
decision about individual attributes or research
questions is not only a function of the quality of
the assessment and the related interpretations,
but also a function of the use of the assessment for
James E. Purpura
some intended purpose, together with the conse-
quences of making those decisions. This broad-
ened scope of L2 assessment has opened the way
for an ever-burgeoning diversity of ideas that now
falls under the rubric of L2 assessment. This evolu-
tion also comes at a time in history when the fun-
damental scientific principles, philosophical and
sociopolitical assumptions, and technological op-
portunities for assessment are being rethought,
thereby revealing the true potential that assess-
ment has not only for beneficial and constructive
purposes, such as the improvement of teaching
and learning, but for safeguards against compro-
mises to assessment quality or unethical misuses
of assessment information.
In sum, the dynamism and intellectual diversity
of L2 assessment are charted by a virtual explosion
of literature in this subfield of applied linguistics.
This includes at least three journals, four book se-
ries, two encyclopedias, several handbooks, and a
companion, all dedicated exclusively to L2 assess-
ment topics.
APPROACHES TO CONSTRUCT DEFINITION:
BROADENING THE CONSTRUCT OF L2
PROFICIENCY
Probably the most compelling and enduring
challenge in L2 assessment is the quest to un-
derstand and define the construct of L2 profi-
ciency, so that meaningful interpretations of an
individual’s performance consistencies can be at-
tributed, to the extent possible, to L2 resources,
along with other internal and external factors,
responsible for communicative success across a
range of tasks and contexts. This has led as-
sessment theorists in educational measurement
and applied linguistics (e.g., Bachman, 2005;
Chapelle, 2008; Kane, 2006; Messick, 1989) to
propose several approaches to defining constructs
so that these approaches can serve as the basis
not only for meaningful interpretation of perfor-
mance consistencies, but also for assessment de-
sign and operationalization, interpretation, and
use. In some cases, these approaches have led
to frameworks or ‘underlying models’ of theoret-
ical constructs, where specific components and
their possible interrelationships are hypothesized.
These models have not been conceptualized as
universal prescriptions of the construct, given the
breadth and complexity of the proposals, but
as a heuristic for informing conceptual assess-
ment frameworks for any given context, includ-
ing those involving elicitation methods other than
traditional tests (i.e., assessments embedded in
instruction and mediated through interaction).
193
In an effort to contextualize how past concerns
convene to create opportunities for current and
future assessment understandings, practice, and
research activity, I will describe four approaches to
construct definition: (a) trait-centered, (b) task-
based, (c) interactionist, and (d) sociointerac-
tional, and illustrate how these approaches shape
how L2 educators and assessment experts can tai-
lor their assessments to a diversity of assessment
purposes.2
One question surrounding differences in how
L2 proficiency has been defined relates to a con-
sideration of the components of L2 knowledge
and their interrelationships. Another concerns
the role that context plays in characterizing an
examinee’s KSAs based on performance consis-
tencies. A third involves sociocognitive and af-
fective factors underlying L2 assessment perfor-
mance, and how these factors mediate between
an examinee’s KSAs and their ability to respond
to a task. A final question—and one receiving too
little attention in my opinion—is the role that
topical, content, or disciplinary knowledge and
the conveyance of meaning play in the L2 pro-
ficiency construct. Dialectics surrounding these
questions have led to a broadening of the L2 con-
struct in light of growing interpretation needs,
and to refinements in the construction, delivery,
and use of assessments. As these four approaches
shape current thinking in L2 assessment, I will de-
scribe each with an eye toward how contemporary
theory, research, and practice follow suit.
What Gets Assessed: Trait-Based Approach
to Construct Definition
The trait- or construct-based approach to con-
struct definition interprets performance consis-
tencies as a reflection of an examinee’s KSAs as
conceptualized within some theoretical model of
the construct of interest (Chapelle, 1998). To as-
sess the construct, items or prompts, for example
in a test, are drawn from a pool of tasks in the
target language use (TLU) domain (e.g., the aca-
demic context), and then specified so that task
characteristics constrain the performance to re-
flect the trait. This selection process allows for the
KSAs needed to perform these tasks to general-
ize across other tasks in a wide range of unspeci-
fied contexts. In this approach, interpretation of
performance is thus a function of the specified
construct, assuming that the trait will generalize
across contexts. While this has been the dominant
paradigm in L2 assessment for years, researchers
have begun to question the generalizability of
traits across real-world L2 proficiency contexts.
194
Numerous examples of the trait-based ap-
proach in the L2 assessment literature can be
found, beginning with Lado (1961) and Car-
roll (1961), who proposed a skills-and-elements
model in which the individual L2 elements (e.g.,
phonology, structure, lexis) could be assessed
separately while performing language skills. The
assumption in this approach was that L2 pro-
ficiency transpires by internalizing simple, dis-
crete units of the L2 before acquiring complex
sequences, the accumulation of which consti-
tuted proficiency. This led to a discrete-point ap-
proach to assessment, which is still a common
method of assessment in many contexts today, es-
pecially where curricula and instructional meth-
ods assume simple-to-complex notions of devel-
opment. For example, in classroom-based gram-
mar assessment, where the goal is for learners
to recognize the differences between discrete
forms and their meanings, the skills-and-elements
model and discrete-point testing would provide
an adequate basis for that interpretation. Also,
discrete-point testing of isolated items with refer-
ence to a theoretical construct is the basis for iden-
tifying L2 production features measured in cur-
rent automated scoring protocols of writing and
speaking.
Another example of a trait-based model of L2
proficiency was proposed by Oller (1979), who
conceptualized L2 proficiency not as a set of
components, but as a unitary, general proficiency
factor, integrated with perceptual and productive
processing.3 This was referred to as pragmatic ex-
pectancy grammar, or “any procedure or task that
causes the learner to process [and produce] se-
quences of linguistic elements in a language that
conform to the normal contextual constraints of
that language and which requires the learner to
relate sequences of linguistic elements via prag-
matic mappings to the extralinguistic context”
(p. 38). This model addressed how L2 elements
relate to pragmatic meanings associated with an
external context of situation, thereby foreshadow-
ing current notions of pragmatic knowledge and
meaning potential. It also identified perceptual
and productive processing as a means by which
learners are able to make moment-by-moment
predictions about what an interlocutor is likely
to understand or say in interaction, especially in
context-reduced situations, again presaging cur-
rent inquiry into the role of L2 processing in per-
formance on assessments, as seen today in the
work of Doughty (2014), focusing on aptitude,
and Phakiti (2003) and Purpura (1999, 2014a,
2014b), focusing on strategy use and test per-
formance. Finally, rejecting discrete-point testing,
The Modern Language Journal, 100, Supplement 2016
Oller proposed an integrative approach to assess-
ment, where scores are interpreted as measures
of overall proficiency—a practice not so dissimi-
lar from holistic assessment today.
While Oller’s notion of global proficiency was
subsequently rejected for a multicomponential
depiction of L2 ability, involving a general higher-
order factor along with several distinct factors
(Bachman & Palmer, 1982; Vollmer & Sang,
1983), depictions of L2 proficiency as an inte-
grated factor have recently resurfaced, to some
extent, with contemporary proposals for inte-
grated skills assessments or task-based L2 assess-
ments, topics to be discussed later in further
detail.
Another example of a trait-based approach
to L2 proficiency was offered by Canale and
Swain (1980) and later by Canale (1983) with
their model of communicative competence (CC),
which defined proficiency as knowledge of gram-
matical resources and the capacity to use these
resources to understand and generate written
and spoken texts in sociolinguistically appropri-
ate ways. This model specified grammatical, soci-
olinguistic, and discourse competence, along with
strategic competence, defined in terms of com-
pensatory strategies, that is, strategies that serve to
mediate between competence and performance.
By specifying sociolinguistic and discourse com-
petence, Canale and Swain acknowledged how
context shaped L2 use, again broadening the
notion of L2 proficiency. Today, communicative
language teaching and assessment are still the
mainstay of L2 education.
Extending the CC model, Bachman (1990),
with later refinements by Bachman and Palmer
(1996, 2010), provided another trait-based ap-
proach to construct definition with their model
of communicative language ability (CLA). In this
model, L2 ability was conceptualized as an individ-
ual’s capacity for language use, as inferred from
performance consistencies on tasks assumed to be
generalizable across other tasks in the TLU do-
main. L2 ability was thus seen as an underlying
competence to be accessed during future perfor-
mance, but not as a prediction of L2 ability for a
specific TLU context.
In their model L2 ability was defined theo-
retically in terms of L2 knowledge and strate-
gic competence. L2 knowledge consisted of
organizational and pragmatic knowledge, where
organizational knowledge involved grammatical
and textual knowledge, and pragmatic knowledge
included functional and sociolinguistic knowl-
edge. Strategic competence was seen as strategies
underlying language use. In this model, CLA is
James E. Purpura
understood to be activated by the need to per-
form tasks drawn from the ‘context of situation’—
where tasks represent “restricted or controlled
versions of the contextual features that determine
the nature of language performance” (Bachman,
1990, p. 112). That said, this model does not spec-
ify how L2 ability interacts with tasks in specific
contexts, that is, how L2 ability might relate to
tasks in the academic domain. Nonetheless, the
CLA model is still one of the most comprehensive
models of L2 proficiency and has been the source
of inspiration for several conceptual assessment
frameworks today.
What Gets Assessed: Task-Centered Approaches
to Construct Definition
A second approach to construct definition
is referred to as a task-centered, task-based, or
performance-based approach to L2 assessment
(Clark, 1972, 1978; McNamara, 1996; Norris et al.,
1998; Skehan, 1998), where performance consis-
tencies are assumed to reflect the examinee’s abil-
ity to accomplish open-ended, written or spoken
tasks within some real-life TLU domain. Task spec-
ifications represent critical features of the task
and context, such as the setting (e.g., a phar-
macy), the communicative context (e.g., a new
prescription), the communicative goal (e.g., un-
derstanding instructions), and so forth. Task com-
pletion is judged mostly from criteria drawn from
real-world standards of performance (e.g., criteria
from aviation), and the components of L2 knowl-
edge may or may not be isolated and assessed
(McNamara, 1996). In this regard, McNamara dis-
tinguished strong from weak performance assess-
ments (PAs) depending on the real-word crite-
ria used and noted that most L2 PAs were ‘weak’
assessments, since they failed to use real-world
assessment criteria. Thus, performance in this
approach may be interpreted as a predictor of the
ability to perform similar tasks in real-life settings
and/or as a reflection of the ability underlying
performance.
Task-centered approaches are all rooted in the
work of Clark (1972) and Jones (1985), whose
goal was to measure L2 proficiency in a direct
manner, that is, by requiring examinees to per-
form tasks similar to those in the real-life do-
main (Clark, 1972, p. 121). Instead of focusing
on L2 ability components, they described PA as
a measure of an examinee’s ability to communi-
cate for some pragmatically useful purpose within
a real-life setting by performing tasks that “dupli-
cate as closely as possible the setting and opera-
tion of the real-life situations in which the pro-
195
ficiency is normally demonstrated” (Clark, 1975,
p. 10). The examinee’s ability is then based on
raters’ judgments of the performance, as de-
scribed in a scoring rubric. Performance on these
assessments is seen as a predictor of future task
performance.
McNamara (1996) referred to Clark and Jones’s
approach to assessment as the strong version of
PA. In describing the weak version, he empha-
sized that the focus was not so much on the can-
didate’s ability to complete surrogate, real-world
test tasks as predictors of future task success;
rather, the focus was on the task’s success in elic-
iting L2 performance on assessments in contexts
and conditions similar to those of the intended
real-life performance, so that raters could then
judge the performance with reference to the KSAs
of interest by means of a holistic or analytic rubric.
PA is currently the most common approach to
assessing L2 production and is one of the most
thoroughly researched areas of the field of L2 as-
sessment, especially as this concerns the effects
of different sources of variance on performance
scores.4 Some current examples of PAs are seen
in the Occupational English Test5 and the Canadian
Language Benchmarks.6
Another example of a task-centered approach
to construct definition is seen in the work of
Norris et al. (1998) and Brown et al. (2002)
and is referred to as task-based language assess-
ment (TBLA). An extension of the direct method,
TBLA interprets performance consistencies as the
accomplishment of real-life tasks within a specific
contextual domain, given the abilities that can-
didates bring to the task, so that future perfor-
mance can be predicted. “Task” is the “fundamen-
tal unit of analysis motivating item selection, test
instrument construction, and the rating of task
performance” (Long & Norris, 2000, p. 60). The
competencies needed to perform tasks are not
drawn, per se, from a theoretical model of L2 abil-
ity, rather from the KSAs needed to accomplish
tasks at different levels of success. TBLA typically
assesses performance based on can-do statements,
like those in the Common European Framework of
Reference,7 rather than on capacity-for-use state-
ments, where performance reflects what learners
know based on a theoretical construct. Thus, the
focus in TBLA is not on ability level, but on perfor-
mance and task outcome (Skehan, 1998). A cur-
rent example of a TBLA is seen in the Certificate of
Dutch as a Foreign Language (Van Gorp & Deygers,
2014).8
Skehan (1998) proposed a somewhat different
version of TBLA in which he is less concerned
with tasks as predictors of real-world performance
196
than with tasks as triggers of language and L2
processes during task performance. This perspec-
tive highlights the potential interactions among
language, task, and cognitive variables. Tasks
in this approach are designed with reference
to code complexity, cognitive complexity, and
communicative stress, rather than according to
contextual features such as setting and purpose.
Also, instead of using can-do statements as per-
formance indicators, performance is assessed by
means of capacity-for-use scales involving rater
judgments of performance, as in traditional PA,
or by describing the features of L2 production
involving the complexity, accuracy, and fluency
(CAF) of the output. Interestingly, Skehan also
proposed the use of processing-driven scales,
where performance moderators relating to pro-
cessing (e.g., lexical searches) are specified in L2
production rubrics. However, this idea has, unfor-
tunately, yet to gain attention in the assessment
literature.
Although Skehan’s (1998) proposal to use
L2 production features (i.e., CAF) as a mea-
sure of performance consistencies has had con-
siderable resonance in the SLA literature (e.g.,
Ellis & Barkhuizen, 2005), this approach to con-
struct definition has had little influence on L2
testing practice, since complexes of L2 pro-
duction features (e.g., percentage of error-free
clauses) are virtually impossible to interpret and
use in operational assessment contexts. Also, in
examining the effects of task characteristics and
performance conditions related to cognitive de-
mands on difficulty associated with CAF, Iwashita,
McNamara, and Elder (2001) found that the
performance conditions hypothesized by Skehan
(and others) failed to affect CAF as predicted.
Nonetheless, the use of these features has gained
considerable momentum in the development of
automated scoring systems of speaking and writ-
ing and their use in the enhancement of human
scores (Deane, 2015). Also, the use of L2 produc-
tion features in validation research to compare
the characteristics of tasks, contexts, or domains
in assessment contexts with those in TLU contexts
has also emerged as an informative source of ev-
idence in validation (see Cumming et al., 2006;
Knoch, Macqueen, & O’Hagan, 2014).
In sum, TBLA in general has gained consid-
erable recognition over the years as L2 teachers
have become more interested in task-based lan-
guage teaching as an extension of communicative
language teaching. This approach has also gen-
erated considerable research interest in SLA and
L2 assessment (see Van den Branden, Bygate, &
Norris, 2009; Van Gorp & Deygers, 2014).
The Modern Language Journal, 100, Supplement 2016
What Gets Assessed: Interactionist Approach
to Construct Definition
A third approach to construct definition is re-
ferred to as the interactionist approach to con-
struct definition, as proposed by Chapelle (1998).
This approach addresses the need to relate
performance consistencies to “traits, contextual
factors, and to their interactions” (p. 34). It
interprets L2 performance as a combination of
the trait of L2 knowledge and the characteristics
of the context in which L2 ability is elicited and to
which performance generalizes, all the while as-
suming that the interaction between the trait and
contexts is facilitated by metacognitive strategy
use (Chapelle, 1998). In other words, when learn-
ers or users are presented with goal-oriented L2
assessment tasks in some relevant context, their
L2 knowledge is activated along with the cognitive
processes needed to perform the task. Therefore,
since score meaning derives from the interactions
among L2 knowledge, contextual factors, and
strategy use, the interactionist approach requires
specification of (a) the traits, (b) the context, and
(c) the strategies required for construct-relevant
performance. As Chapelle (1998) points out, “the
trait components can no longer be defined in
context-independent, absolute terms, and contex-
tual features cannot be defined without relevance
to their impact on underlying characteristics”
(p. 43). The interactionist approach has signifi-
cantly broadened the construct of L2 proficiency
and is currently the model underlying the assess-
ment framework for the TOEFL iBT.
Chalhoub–Deville (2003) carried Chapelle’s
(1998) ideas on context and L2 ability a step fur-
ther by arguing for a representation of L2 ability
as an ‘ability-in-individual-in-context’ construct,
rather than as an individually focused ‘cognitive
or within language user’ representation of L2 abil-
ity. She maintained that the context of L2 use
sometimes activates an examinee’s L2 ability at
times, and at other times, the examinee’s L2 abil-
ity influences facets of the context, concluding
that “ability and context features are intrinsically
connected and it is difficult or impossible to dis-
entangle them” (p. 372).
An example of the interactionist approach
is seen in Douglas’s (2000) work on language
for specific purposes (LSP) assessment. Argu-
ing that “the nature of language knowledge
may be different from one domain to another”
(p. 24), Douglas proposed a special purpose lan-
guage ability construct in which performance
consistencies reflect the strategic engagement of
L2 ability resulting from tasks situated within
James E. Purpura
specific purposes contexts. He also argued that
since background knowledge plays such an im-
portant role in these assessments, special purpose
language ability needed to specify background
knowledge as part of the construct. So, drawing
on Hymes (1974), he extended Bachman and
Palmer’s (1996) task characteristics framework to
include critical features of context. Douglas also
provided applied linguists with a concrete exam-
ple of how the L2 knowledge trait, task/context,
and their interaction through strategy use could
be incorporated into assessment construction. Im-
portantly, this included background knowledge as
part of the L2 proficiency construct.
Another example of the interactionist ap-
proach is seen in the work of Purpura (2004,
2014c). Arguing that the fundamental linguistic
resources of communication involve not only
grammatical forms, but also the meaning po-
tentials associated with these forms when used
alone or in context, he proposed a model of L2
proficiency in which performance consistencies
can be attributed not only to the selective use of
grammatical forms in conveying and understand-
ing literal, semantic, or propositional meanings
associated with the topic of communication, but
also to a range of intended or implied prag-
matic meaning potentials (e.g., sociolinguistic,
sociocultural, psychological/affective, rhetorical)
encoded in the communicative context. To elicit
these meaning potentials, tasks then need to spec-
ify contextual features of communication (e.g.,
the setting, the communication goal), together
with relevant sociolinguistic features (e.g., partici-
pant roles), sociocultural features (e.g., New York
City culture), psychological/affective features
(e.g., participant attitudinal or emotional dispo-
sitions), and rhetorical (e.g., genres) features of
the context. Finally, to mediate between L2 knowl-
edge, topical or disciplinary knowledge, and task
engagement, examinees need to access process-
ing capabilities associated with both the mind’s
cognitive architecture (e.g., attention, percep-
tion, memory) and its functions (e.g., processing,
strategies). This expansion of the L2 proficiency
construct has implications for LSP assessment,
scenario-based assessment, and the assessment
of language and content integrated learning. An
example of this approach will be found in the
newly created Community English Placement
Test at Teachers College, Columbia University.
In sum, the construct of L2 ability has evolved
over the years to keep pace with contemporary
theories, empirical research, and increasing de-
mands to align assessment interpretation with tar-
get language use in real-life contexts. The inter-
197
actionist view of L2 ability has provided theoret-
ical grounding for incorporating traits, contex-
tual factors, and processing into construct def-
inition, thereby acknowledging that a person’s
L2 ability is sociocognitively activated by goal-
oriented tasks embedded within TLU contexts.
What is lacking in this approach, however, is a
consideration of a meaning component—or what
meaning potentials or topical/disciplinary con-
tent are being communicated. Functional knowl-
edge (Bachman & Palmer, 2010) allows us to ex-
amine the purpose of meaning conveyance, but not
the content of meaning conveyance. This is im-
portant if L2 educators and assessment experts
want to assess the extent to which communication
is substantively meaningful or content responsi-
ble (Douglas, 2000; Purpura, 2004; Snow & Katz,
2014). Thus, besides specifying the trait compo-
nent, the context, and the interactions, I believe
it is critical to specify the meaning component by
addressing the role that topic, theme, and con-
tent (topical or disciplinary) play in L2 perfor-
mance as elicited from contextually rich tasks, de-
signed within social–interactional, transactional,
academic, or professional TLU domains.
What Gets Assessed: Sociointeractional Approaches
to Construct Definition
A fourth approach to construct definition
might be referred to as the sociointeractional ap-
proach to construct definition. In this approach,
performance in the TLU domain involves a lo-
cal, social activity (e.g., pitching a new product
idea to associates) in which individuals interact
on a moment-by-moment basis to jointly con-
struct meaning to perform a goal-oriented activ-
ity (McNamara, 1997). Embedded in this activ-
ity occur indigenous (L2 and content) assessment
activities that can be associated with indigenous
assessment criteria. Assessment thus involves an
evaluation of the degree to which participants
effectively achieve the communicative goal by co-
constructing construct relevant meanings and ap-
propriately engage in discursive practices (e.g.,
turn-taking, preference organization, repair) ex-
pected within the specific TLU domain.
An example of this approach occurs in the work
of Jacoby and McNamara (1999), who reported
on how physicists in a graduate seminar evaluated
and provided feedback on conference presenta-
tion rehearsals. These rehearsals were intended
to socialize participants into the practices, goals,
and habits of mind of this particular community.
Based on an analysis of the interactions, partici-
pants oriented to several assessment criteria (e.g.,
198
newsworthiness, clarity, content accuracy, techni-
cal delivery, overall quality) used to judge perfor-
mance in the discursive practice of presenting a
conference talk. Jacoby and McNamara argued
that in this context, an independent set of assess-
ment criteria would have been inappropriate for
the assessment goal.
Another example of this approach is seen in the
work of He and Young (1998), who, in refining
Kramsch’s (1986) notion of interactional compe-
tence (IC), described performance consistencies
“not as an ability within an examinee,” but as the
joint co-construction of “abilities, actions, and ac-
tivities of all participants” (p. 5, italics in the orig-
inal), while performing assessment tasks. IC is,
thus, conceived as a product of both interlocutors,
since interaction is locally managed on a moment-
by-moment basis between participants. Interlocu-
tors are assumed to acquire “practice-specific IC”
rather than “general, practice-independent com-
municative competence” (p. 7), suggesting that
performance on interactive speaking tasks does
not generalize beyond the specific event.
While the claim that IC is an attribute of talk-in-
interaction rather than an attribute of individuals
using resources to jointly co-construct meaning,
knowledge, and action within some discursive
practice is problematic for most assessment ex-
perts. IC and the notion of indigenous assessment
criteria have certainly broadened the L2 profi-
ciency construct in that they have highlighted the
role of co-construction (e.g., topic management,
turn-taking) and intersubjectivity in spoken
communication, thereby leading L2 assessors to
reexamine interaction in test construction and
validation. They have also underscored the need
to understand and consider indigenous scoring
criteria from the TLU domain in the assessment.
A somewhat different example of the socioin-
teractional approach is seen in the work of
Lantolf and Poehner (2004) on dynamic assess-
ment (DA), where performance consistencies
are conceptualized in terms of changes in L2
development as a result of interventions provided
by a mediator (assessor), who, during the course
of instruction, attempts to mediate (i.e., assist)
the narrowing of learning gaps. The assump-
tion underlying DA is that mediations, provided
through co-constructed talk-in-interaction with
a more capable person, will lead to the inter-
nalization of new understandings. DA, unlike
other conceptualizations, is located within the
teaching/learning activity to provide mediation
between the learner and task, instead of after
the activity. While DA has typically been used
and researched inside classrooms, current efforts
The Modern Language Journal, 100, Supplement 2016
have been interestingly extended to applying
DA principles to large-scale assessment contexts,
especially in contexts where feedback, scaffolded
assistance, and other forms of mediation are
delivered through intelligent, computer-assisted
L2 assessment protocols (Poehner, 2014).
Where Do We Go From Here? Since its inception,
the field of L2 assessment has engaged in lively
debates over the ongoing challenge of construct
definition and how best to rethink constructs
to support contemporary needs and purposes
for which assessments may be put to use. These
developments have direct implications for as-
sessment practice. This section attempted to
summarize these major issues and debates by
considering how past understandings continue to
inform current thinking, research, and practice,
and how they define future agendas for how the
construct of L2 proficiency might be assessed.
In the next section, I will discuss some selected
areas of current interest and debate within the
assessment community.
SELECTED AREAS OF CURRENT
ASSESSMENT ACTIVITY
Aside from the debates on approaches to con-
struct definition, L2 educators and assessors have
continued to explore and improve ways to design,
develop, deliver, score, analyze assessments, and
report assessment information in light of current
understandings. They have also taken advantage
of new opportunities for assessment through tech-
nological advances and new methods of educa-
tional measurement for statistical modeling. Fi-
nally, they have pursued critical inquiries on the
use of assessments as fair and just measures of
performance. Since the research contribution in
these areas is overwhelming, I will focus on a few
selected areas of interest.
The Assessment of Individual Elements and Skills
Assessing Pronunciation. After years of margi-
nalization, the assessment of pronunciation has
resurfaced as an important area of inquiry and
practice in L2 assessment. This stems, for exam-
ple, from the need to evaluate speech samples
from asylum seekers who claim they are from
repressed groups, from demands for the intelli-
gibility of international teaching assistants, from
the desire to represent several varieties of En-
glish in standardized exams, and from an interest
in measuring phonological control through auto-
mated scoring protocols of L2 speaking (Isaacs,
James E. Purpura
2014). While the focus is no longer on accent
reduction or the attainment of native-like stan-
dards, current research in this area is concerned
with understanding what phonological features
have the greatest effect on intelligibility and
comprehensibility, in other words, what features
contribute most to communication breakdowns.
Another area of inquiry relates to the influence of
rater characteristics, such as phonological mem-
ory, on the ability to judge L2 pronunciation. A
final area concerns the use of speech recognition
technology to relate automated scores of intelli-
gibility to scores from human judgments (Isaacs,
2014). In sum, we are likely to see much more
work on the assessment of pronunciation in the
future.
Assessing Pragmatics. Most applied linguists
would agree that communication in real world
contexts involves not only the ability to commu-
nicate literal propositions, but also the ability to
decipher and convey nuanced pragmatic mean-
ings that encode, inter alia, the interpersonal re-
lationship of interlocutors, their emotional and
attitudinal stance, presuppositions about what in-
terlocutors know, and assumptions about how to
interact in a particular setting (Purpura, 2004).
However, until recently, the assessment of prag-
matic knowledge has received relatively little at-
tention, even though pragmatic competence is
implicitly assessed in all L2 assessments.
Empirical research on pragmatics assessment
began with the work of Hudson, Detmer, and
Brown (1995), who investigated the sociolinguis-
tic component of pragmatic knowledge by design-
ing discourse completion tasks, role plays, and
self-assessments to measure the contextual vari-
ables of relative power, social distance, and abso-
lute ranking of impositions in the context of three
speech acts. Building on this work, Roever (2005)
developed a web-based multiple-choice test of
pragmalinguistics intended to measure not only
the sociolinguistic variables of politeness, but also
conversational implicature and situational rou-
tine formulae. Current research in the area (see
Roever, 2014) has also begun to look at pragmatic
awareness and notions of interactional compe-
tence, drawing on conversation analysis (Ross &
Kasper, 2013).
Another approach to the assessment of prag-
matic knowledge is seen in Purpura (2004). Sit-
uating pragmatics assessment within a model of
CLA, he defined pragmatic competence as the
ability to use L2 resources, together with context, to
understand and convey meanings beyond what is
literally said. The source of these implied prag-
199
matic meaning potentials may be contextual, so-
ciolinguistic, sociocultural/intercultural, psycho-
logical/affective, or rhetorical. The contextual
features driving pragmatic ability include the in-
terlocutors’ beliefs and presuppositions about the
temporal, spatial, and social characteristics of the
settings, their knowledge of appropriate subject
matter, their knowledge of appropriate registers,
and so forth (Levinson, 1983). Critical to this
model is the role of context without which the
ability to interpret extended or exophoric mean-
ings encoded in utterances would be lost. This
conceptualization again represents a significant
broadening of the construct of L2 proficiency
with implications for the construction of test tasks,
scoring, and validation.
Grabowski (2009, 2013) used Purpura’s model
to design contextually rich, reciprocal role-play
tasks intended to measure the ability to con-
vey sociolinguistic, sociocultural, and psycholog-
ical meanings with rubrics intended to measure
these meanings. She found that the tasks suc-
ceeded in eliciting the meanings of interest, and
that performance could be reliably judged. The
tasks also produced scalable scores across differ-
ent proficiency levels. Grabowski concluded that
while pragmatic knowledge can be measured at all
proficiency levels, it is critical to do so at the ad-
vanced levels to avoid construct underrepresenta-
tion. She also noted the challenges, but not the
impossibility, of administering and scoring recip-
rocal tasks in operational testing situations.
Currently, the assessment of pragmatic knowl-
edge has emerged as an area of strong interest in
L2 assessment research, and one that is being in-
vestigated for incorporation into large-scale L2 as-
sessments.
Assessing Integrated Skills
Integrated Skills Assessment. Research on the di-
visibility of L2 proficiency clearly showed that
L2 ability consists of a higher-order factor along
with several interrelated components (Bachman
& Palmer, 1982). As a result, L2 proficiency assess-
ments have traditionally been designed to mea-
sure the four skills, together with grammar and vo-
cabulary, in separate sections or embedded in the
assessment of productive skills. If the individual
skill scores on these assessments produced strong
correlations, the assumption was that a combined
score could serve as an estimate of overall profi-
ciency. However, recent descriptions of the L2 de-
mands needed in academic and workplace con-
texts have shown that besides needing to per-
form skills independently, in the vast majority of
200
contexts, learners need to express meanings
through the integration of L2 skills (Leki & Car-
son, 1997). In other words, source information is
obtained from reading or listening and then inte-
grated into speaking and writing. Thus, increas-
ing numbers of assessments (e.g., the TOEFL
iBT) have designed tasks in which two or more
modalities are integrated, and where perfor-
mance consistencies are interpreted as the com-
bination of these constructs within a contextual
space, as one might expect in a synthesis task.
Empirical research on integrated skills has
taken some interesting pathways. One strand has
attempted to understand the extent to which
reading and writing are separate or integrated
constructs. For example, Sawaki, Quinlan, and
Lee (2013) examined performance on TOEFL in-
tegrated writing tasks in which examinees used
source material content from the reading and lis-
tening sections to write a response to the inte-
grated items. They found that, in fact, both the
reading/listening comprehension factor and the
writing factor were subsumed within a higher-
order integrated comprehension and writing fac-
tor, which served to explain source-text-based
writing and text comprehension. Another set of
studies examined the processes and strategies that
examinees associate with performing integrated
skills assessment. Plakans (2009) found that writ-
ers asked to perform integrated skills tasks showed
composing processes that clearly reflected ‘dis-
course synthesis.’ Still another interesting line of
inquiry examined the role of source-text charac-
teristics of writing ability on L2 tests (Cumming
et al., 2006).
In sum, demands for L2 assessments that mea-
sure competencies and processes similar to those
in the TLU domain have motivated the assess-
ment of integrated L2 skills (Cumming, 2014). As
Plakans (2013) notes, this represents a step away
from conventional tests of the four skills toward
constructive principles of knowledge integration
and synthesis. While many questions still remain
regarding what constructs are actually being as-
sessed by complex tasks requiring integration or
how best to assess them, the ability to integrate
information is fundamental to language use and
a sure area of increased research interest in the
future.
Scenario-Based Language Assessments. While the
work on integrated skills assessments has high-
lighted the importance of tasks designed to mea-
sure the learners’ ability to understand source
material to communicate ideas in writing and
speaking (e.g., reading to summarize in writ-
The Modern Language Journal, 100, Supplement 2016
ing), this perspective has concentrated somewhat
narrowly on skill integration and performance.
Scenario-based assessment (SBA) takes this a step
further by orchestrating skill integration within a
thematically coherent, socially familiar, purpose-
driven scenario in which participants are required
to complete a sequence of subtasks intended to
reflect the habits of mind underlying the over-
arching scenario goal (e.g., work with peers to
pitch an idea about creating a green school; Saba-
tini & O’Reilly, 2013). The completion of such a
goal requires a range of L2 skills in which exam-
inees have to understand, summarize, synthesize,
and integrate information across texts and modal-
ities; evaluate the relevance and quality of the
synthesized information; and integrate the syn-
thesized information into coherent, goal-oriented
spoken or written responses. Capitalizing on the
potential of computer delivery, SBA also provides
opportunities to measure component skills pre-
sumed to enable performance of the higher order
integrated skills, topical/disciplinary knowledge
across the scenario, their need for and response
to feedback and assistance, and a range of demo-
graphic and social–psychological dispositions.
Several researchers in mainstream assessment
(e.g., Bennett, 2010) have worked on the use of
scenarios in disciplinary content learning. Among
them, Sabatini and O’Reilly (2013) described how
scenarios in the context of L1 literacy assess-
ment could be used to provide a comprehen-
sive measure of reading ability designed to tap
into a wide span of macro and micro reading
competencies with the potential of providing di-
agnostic information to examinees. The compo-
nent skills part of the assessment targeted gen-
eralizable features related to basic processing of
text (e.g., word recognition), while the integrated
skills part aimed to capture the ability to ac-
complish performance on purpose-driven, the-
matically related, integrated tasks, called global-
integrated, scenario-based assessments (GISAs).
O’Reilly and Sabatini (2013) described how sev-
eral dimensions of the reading ability construct
could be organized and measured through sce-
narios. They state:
This orchestration is achieved by incorporating a
scenario-based design that organizes the assessment
around a central theme and goal for reading (e.g.,
work with fellow students to study for an exam
or prepare a presentation on a science or history
topic) (. . .) a set of diverse sources (e.g., blogs,
Web sites, videos, charts and diagrams, traditional
text genre excerpts), and a sequence of subtasks to
achieve the final goal (e.g., evaluate sources, identify
important or relevant ideas, integrate information
James E. Purpura
across sources, make decisions, edit a wiki). In this
manner, GISA is designed to resemble the types of
reading activities one might engage in school, work,
or leisure. (pp. 2–3)
Although L2 assessment experts have only
begun to develop SBAs of L2 ability, this type of
assessment, in my view, is theoretically appeal-
ing for its capacity to measure L2 ability within
thematically coherent, goal-driven contexts re-
quiring KSAs that one would expect to find in
real-life L2 task completion. It also offers op-
portunities to incorporate the measurement of
performance moderators, such as background
knowledge, into the assessment protocol, along
with a learning-oriented component9 designed
to provide feedback and assistance for those who
need it. In my view, SBA provides a plausible solu-
tion to the theoretical conundrum of designing
tasks that account for the trait, context, content,
and the ensuing sociocognitive and affective
interactions.
Assessment, Learning, and Instruction
While much discussion in L2 assessment has fo-
cused on assessments external to the classroom,
the role that assessment plays inside classrooms
has emerged as an exciting new area of inquiry, es-
pecially as this relates to assessments embedded in
learning and instruction. Traditionally, the focus
of L2 classroom-based assessment (CBA) has been
on the principled construction of tests, designed
mostly to measure achievement for summative
purposes (e.g., a grade) and administered at the
end of instruction. This focus has unsurprisingly
resulted in several books devoted to the develop-
ment of assessments of individual L2 components
(e.g., assessing grammar), skills (e.g., assessing
writing), and methods (e.g., assessing language
for specific purposes).10 At the same time, the em-
phasis of CBA research in both mainstream edu-
cation and L2 assessment has also been motivated
by a push to understand and promote forma-
tive assessment (FA) or assessment FOR learning
(AFL; i.e., providing information to improve), as
opposed to summative assessment or assessment
OF learning—AOL (i.e., providing information
on mastery; Black & Wiliam, 1998). The L2 CBA
research has focused primarily on the teacher and
teacher assessment literacy—that is, teachers’ un-
derstandings of assessment and assessment pro-
cesses related to the identification and narrow-
ing of learning gaps in instruction through FA. To
that end, several studies have examined the role
of teacher knowledge, experience, and beliefs in
201
L2 assessment (e.g., Leung, 2009; Rea–Dickins,
2004), teacher assessment processes (e.g., Davi-
son, 2004; Rea–Dickins, 2001), and L2 assessment
methods (Cheng, Todd, & Huiqin, 2004). Other
research, concentrating more on learners, has
looked at the value of self- and peer assessment
(Patri, 2002), the role of diagnostic (Alderson
et al., 2014) or dynamic (e.g., Poehner, 2014)
assessment in promoting teaching and learning,
or the role of technology in learning and assess-
ment (Chapelle & Douglas, 2006). Finally, Hill
and McNamara (2011) used data from two for-
eign language classrooms to propose a framework
for researching CBA processes, which addressed
questions related to the teacher perspective, such
as what teachers do, what teachers look for, and
what theory or standards they use. Interestingly,
they also addressed the learner’s perspective, that
is, what understandings learners have regarding
learning and assessment of the foreign language.
More recently, Turner and Purpura (2015) have
proposed a multidimensional approach to CBA,
entitled learning-oriented assessment (LOA), in
which the goal is to understand the complexities
of how information from assessments (e.g., tests,
observations, class discussions, naturalistic talk-
in-interaction, peer feedback, self-assessment,
projects, portfolios, or moment-to-moment eval-
uations of performance embedded in dialogic in-
struction) serve to trigger L2 processes in the ulti-
mate resolution of learning gaps. Importantly, this
approach acknowledges that assessment in class-
rooms is multifaceted, involving not only many
different dimensions (e.g., the context) that con-
tribute to the learning process, but also several
agents (e.g., students, teachers, peers, comput-
ers). In this respect, Turner and Purpura move
beyond dichotomous depictions of CBA as for-
mative/summative or AOL/AFL, toward an ap-
proach that characterizes assessment, learning,
and instruction, while different, as intrinsically in-
tertwined. They identify seven critical intercon-
necting dimensions of CBA. These include: the
contextual dimension (i.e., the social, cultural, or
educational context of learning), the elicitation
dimension (i.e., the method used to elicit perfor-
mance), the proficiency dimension (i.e., the KSAs
being targeted and tracked), the cognitive or
learning dimension (i.e., the sociocognitive char-
acteristics underlying performance and learn-
ing), the affective dimension (i.e., attitudinal and
emotional dispositions engaged in performance
and learning), the interactional dimension (i.e.,
the interactional attributes of assessment-related
communication—turn-taking, preference struc-
ture, etc.), and the instructional dimension (i.e.,
202
the instructor’s content knowledge, pedagogical
content knowledge, and assessment literacy).
While considerable research on LOA is cur-
rently under way, the importance of this topic
as an emerging area of inquiry is highlighted
by several recent conference symposia,11 a three-
day roundtable in 2015 at Teachers College,
Columbia University on LOA theory and prac-
tice in large-scale and classroom contexts,12 and a
symposium on classroom-based L2 assessment at
the 2016 Georgetown University Round Table on
Language and Linguistics.
Language Assessment in Global Contexts
Over the last few decades, societal, economic,
and geopolitical forces around the globe have
converged to create the need for L2 certification
for individuals wishing to pursue education, em-
ployment, asylum, and citizenship. This has re-
sulted in the creation of high-stakes standardized
L2 assessments designed for these uses, followed
by a substantial body of research relating to the
use of these exams for their intended purposes.
As the need for high-quality exams administered
around the world has forced testers to rethink
some of their practices, I will briefly highlight
some of the current issues.
Assessing English as a Lingua Franca. English
has become the de facto global lingua franca
given its extensive use around the world by native
and nonnative speakers. In fact, increasing num-
bers of speakers from different first languages
use English to communicate, something that
many think should be reflected to some degree
in assessments. For example, Harding (2011a)
highlighted the need to incorporate L2 accents
of English into the listening sections of large-scale
exams to reflect the diversity of accents in English-
speaking contexts. Research on nonnative En-
glish speakers has also highlighted how nonnative
speakers use phonological accommodations to
enhance intelligibility in L2 communication
(Jenkins, 2000). Research from corpus analyses
has shown how certain nonnative-like lexicogram-
matical features have little effect on communica-
tion (Seidlhofer, 2011). More recently, research
from pragmatics analyses has examined accom-
modations with respect to miscommunication
and the role of code-switching as a means of
preserving or broadening one’s linguistic and
cultural identity. This research has called into
question, as Jenkins and Leung (2014) point out,
the use of the native-like standard of competence
with respect to grammatical and pragmatic norms
on assessments. It has also highlighted the need
The Modern Language Journal, 100, Supplement 2016
to include ‘native’ and lingua franca varieties
of English in assessments, especially with those
claiming to provide a global measure of L2 pro-
ficiency (Brown, 2014). These needs have been
tempered by concerns that the inclusion of ‘some’
native varieties of English and virtually all lingua
franca varieties into high-stakes assessment could
introduce construct-irrelevant variance into the
assessment and ultimately incur negative conse-
quences for examinees (Harding, 2011b). While
this debate is far from over, research in the area
has initiated discussions for a reconsideration
of what L2 proficiency means, especially in EFL
contexts, and for further research on the effect
of these features on test performance.
Common European Framework of Reference (CEFR).
The CEFR, originally designed by the Council
of Europe (2001) to provide “a common basis
for the elaboration of language syllabuses, cur-
riculum guidelines, examinations, textbooks, etc.
across Europe” (p. 1), provides several sets of L2
performance indicators across six levels of pro-
ficiency. These indicators are then used to align
L2 assessments to the framework so that profi-
ciency levels can reflect common understandings
across contexts in terms of what candidates are
able do with the L2 (Council of Europe, 2009).
The CEFR has been used extensively across Eu-
rope but has more recently also been used in con-
texts outside of Europe as a set of standards for
curricular reform. Research in the area has exam-
ined the CEFR scale functionality, procedures for
aligning tests with the CEFR, scale use in several
demographic contexts, and the limitations of the
CEFR (Council of Europe, 2002; see also a special
issue in Language Testing—volume 22). Another
body of research has criticized the theoretical un-
derpinnings of the CEFR, its use as a policy instru-
ment, and its use with non-European languages
(Fulcher, 2004). Nonetheless, the CEFR has had
an enormous effect on how L2 teaching and as-
sessment relate to and can be aligned with a set
of external standards, and it can be credited with
raising assessment literacy across the globe.
The Values and Social Consequences of Assessment Use
As mentioned previously, L2 assessors have
long been interested in the consequences of
assessment use, especially as this relates to fair-
ness, social justice, and ethics. This has mostly
been argued from the perspective of linking
test validity to empirical or rational evidence
supporting the beneficial consequences of test
use. However, much of the recent research in
this area has actually focused on test misuse. For
James E. Purpura
example, one potential misuse of assessment oc-
curs when tests are used to make decisions about
individuals or groups for whom the assessment
was not originally intended. This situation may
unintentionally lead to harmful consequences for
stakeholders affected by those exams (Shohamy,
2001a). This situation is brought into focus
when assessments are used to make high-stakes
decisions in educational contexts or contexts
related to citizenship, immigration, asylum,
or naturalization (Eades, 2009; Kunnan, 2009;
McNamara & Roever, 2006; Shohamy, 2001b). In
the educational context, Leung and Rea–Dickins
(2007) discuss the contradictions and capacities
of teacher assessment as a policy instrument
for the assessment of English literacy with the
National Curriculum in the United Kingdom.
They describe the consequences for pedagogy
and curriculum provision when policy makers
seem more interested in the political benefits of
educational success than in assessment findings
that reveal the actual technical and educational
issues related to educational reform for L2 learn-
ers. In the citizenship context, McNamara and
Ryan (2011) use the role of English literacy in
the Australian citizenship test as a context for
distinguishing between fair assessments (i.e.,
qualities related to the test properties) and just
assessments (i.e., qualities related to the ethical
values embodied in assessment use), arguing that
while the technical standards of an assessment
might be fair, they may still be ethically unjust to
stakeholders, and vice versa.
Since the late 1990s, the social consequences
of assessment use have been a deep and endur-
ing concern for L2 assessors with two special issues
(Language Testing in 2004 and Language Assessment
Quarterly in 2007). This topic is likely to remain in
the fore for years to come.
Technology and Language Assessment
Advances in computer and telecommunica-
tions technologies have profoundly changed the
ways we work, learn, and socialize. These trends
have significantly impacted the competencies we
now need, and those to be assessed. These same
technological advances have allowed assessors to
design, deliver, and score assessments that not
only reach beyond conventional practices, but
also provide opportunities for assessing constructs
that have previously eluded us. This section dis-
cusses some of the current work in this area.
Technology and Assessment Design. Some of the
most exciting applications of technology to L2 as-
203
sessment come from new constructs that can be
assessed, and how assessments can be delivered
on computers and other digital devices. Once be-
yond the reach of most assessors, several programs
are now available (e.g., Google apps) with author-
ing tools for the creation of all sorts of assess-
ments. As a result, L2 educators and assessors can
fairly easily create enriched task environments, in-
corporating into the assessment audio files, digi-
tal video clips, wikis, podcasts, limited web access
through hyperlinks, interactivity through video
chats or messaging, simulations of face-to-face
interviews or phone conversations, editing and
proofreading tools (e.g., spell check), and other
resources (e.g., dictionary). The availability of this
technology has, to some extent, revolutionized
how assessments can be designed and delivered,
the caveat always being that the usefulness of these
features for expanding constructs and augment-
ing authenticity should be determined by valida-
tion research.
Also available to assessment developers are writ-
ten and spoken corpora that can be used in the
construction of assessments to examine grammat-
ical, semantic, functional, and discourse features
at several levels of proficiency and across general
and specific language use domains. These cor-
pora are now available online for use in devel-
oping, for example, options for multiple choice
questions or for selecting graded texts (Cobb,
2009). In fact, several large corpora have been
compiled for use in large-scale assessment. ETS
developed the TOEFL 2000 Spoken and Written
Academic Language Corpus to help item writers
assess language drawn from a variety of contexts
in the real-life academic domain (Biber et al.,
2004). Cambridge ESOL and Cambridge Univer-
sity Press developed a learner corpus as a resource
for designing test items and investigating lexico-
graphic research. This corpus consisted of written
exam scripts by nonnative English speakers along
with the associated questions; the scripts repre-
sent several general, academic, and professional
English tests at six CEFR levels. Barker (2014) pro-
vides an excellent review of the use of corpora in
assessment.
Considerable research has targeted the role of
technology and new media in L2 assessment as
well. Chapelle and Douglas (2006) provided a
comprehensive overview of computer-based L2 as-
sessment in which discussions included how new
technologies influence construct definitions and
other features of the assessment. Ockey (2009)
highlighted the need to disentangle the interplay
among learner abilities, technical affordances,
and multimodal texts so that L2 constructs can
204
be reexamined. Gruba (2014) called for broader
validation research agenda on new media and L2
assessment by means of an assessment use argu-
ment. Given the potential of these new technolo-
gies for the development of assessments and the
questions that they pose, I think we have only be-
gun to see what the future holds.
Automated Scoring of Speaking and Writing. New
digital technologies have facilitated the design,
delivery, and collection of assessment data by
means of a growing number of tools and ap-
plications that allow for assessment responses to
be automatically scored and analyzed. As a re-
sult, several large-scale L2 assessment programs
have designed automated scoring and evalua-
tion programs of speaking or writing. As Burn-
stein (2013) points out, automated scoring pro-
grams only produce scores, whereas automated
evaluation systems also offer diagnostic feedback
usually related to linguistic features. One widely
used system for automatic essay scoring was de-
veloped at the Educational Testing Service (ETS)
and is called e-rater.13 This system uses a natu-
ral language processing methodology to detect
text structure, linguistic structure, and other fea-
tures, combined with statistical protocols to gener-
ate essay scores. Despite criticisms that e-rater fo-
cuses too much on linguistic features rather than
on messages, research on comparisons between
e-rater and human essay scores have shown re-
markably high rates of agreement (96% of the
time; Chodorow & Burstein, 2004), attributed per-
haps to the relationship between “ease of text
production” and the “ability to mobilize cogni-
tive resources to address rhetorical and concep-
tual problems” (Deane, 2015, p. 7). Since e-rater
is more efficient than humans in focusing on mi-
cro features, e-rater scores are increasingly used
alongside human scores to produce essay scores.
Although many testers are still skeptical about
assigning scores based on a complex of pro-
duction features without the inclusion of robust
meaning-based contributions to the scores, con-
siderable research in the area is under way and is
likely to continue as new technologies emerge.
CONCLUSION
In this article I have attempted to provide an up-
date on some of the major issues, concerns, and
debates in L2 assessment, as I see them. I have
shown how past concerns shape current solutions,
and then how current solutions create opportu-
nities for advancement. I have tried to argue that
‘past’ concerns often have a way of resurfacing for
The Modern Language Journal, 100, Supplement 2016
current consideration. Thus, I began the article
by clarifying how notions of what constitutes L2
assessment have significantly expanded over the
years as assessment concerns have moved beyond
the technical quality of an instrument to the use
of that instrument in decision making. I then pro-
vided a rather in-depth look at a critical concern
for L2 assessors—approaches to construct defini-
tion and how they influence what actually gets as-
sessed. Finally, with an eye toward the future, I
rather subjectively highlighted a few areas of as-
sessment activity that have attracted considerable
research interest and enthusiasm. Finally, given
the limits of such a review, I admit I may have left
out some areas of critical concern for L2 testers,
such as the debates regarding the different ap-
proaches to validation or the latest quantitative
methods and their application to assessment vali-
dation. This review was never meant to be conclu-
sive or fully comprehensive, but it is my hope that
it has succeeded in highlighting the exciting work
that L2 assessors are currently engaged in and the
work that still remains before us.
NOTES
1 See the American Council on the Teaching of
Foreign Languages Interview Familiarization Man-
ual (2012): http://www.languagetesting.com/oral-
proficiency-interview-opi
2 For further reading on these approaches, see Bach-
man (2007) and Chapelle (1998).
3 While Oller does not specifically mention ‘pro-
ductive processing,’ we can assume that pragmatic ex-
pectancy grammar also applied to production.
4 For a current review of PA, see Yu (2013).
5 See https://www.occupationalenglishtest.org
6 See https://www.google.com/search?client=safari
&rls=en&q=Canadian+Language+Benchmarks&ie=
UTF-8&oe=UTF-8
7 For more information on the CEFR, see http://
www.coe.int/t/dg4/linguistic/cadre1_en.asp
8 See http://www.cnavt.org/files/alg.folder_EN.pdf
9 For more information on SBA, see http://www.
ets.org/research/topics/reading_for_understanding/
assessments
10 See the Cambridge Language Assessment Series or
Routledge’s New Perspective on Language Assessment
Series.
11 See LOA symposia at AAAL 2013, BAAL 2014, and
LTRC 2015.
12 For more information on LOA, see http://www.
tc.columbia.edu/tccrisls/
13 For more information, see https://www.ets.org/
erater/about
James E. Purpura
REFERENCES
Alderson, J. C., Haapakangas, E.–L., Huhta, A., Niemi-
nen, L., & Ullakonoja, R. (2014). The diagnosis of
reading in a second or foreign language. New York:
Routledge/Taylor & Francis.
American Council on the Teaching of Foreign Lan-
guages. (2012). Oral Proficiency Interview familiariza-
tion manual. White Plains, NY: ACTFL. Accessed 17
April 2015 at http://www.languagetesting.com/
oral-proficiency-interview-opi.
Bachman, L. F. (1990). Fundamental considerations in lan-
guage testing. Oxford: Oxford University Press.
Bachman, L. F. (2005). Building and supporting a case
for test use. Language Assessment Quarterly, 2, 1–34.
Bachman, L. F. (2007). What is the construct? The di-
alectic of abilities and contexts in defining con-
structs in language assessment. In J. Fox, M.
Wesche, D. Bayliss, L. Cheng, C. E. Turner, & C.
Doe (Eds.), Language testing reconsidered (pp. 41–
71). Ottawa: University of Ottawa Press.
Bachman, L. F., & Palmer, A. (1982). The construct val-
idation of some components of communicative
proficiency. TESOL Quarterly, 16, 449–465.
Bachman, L. F., & Palmer, A. S. (1996). Language testing
in practice. Oxford: Oxford University Press.
Bachman, L. F., & Palmer, A. S (2010). Language assess-
ment in practice. Oxford: Oxford University Press.
Barker, F. (2014). Using corpora to design assessment.
In A. J. Kunnan (Ed.), The companion to language as-
sessment (pp. 1452–1476). Oxford, UK: John Wiley
& Sons.
Bennett, R. (2010). Cognitively based assessment of, for,
and as learning (CBAL): A preliminary theory of
action for summative and formative assessment.
Measurement, 8, 70–92.
Biber, D., Conrad, S. M., Reppen, R., Byrd, P., Helt,
M., & Clark, V. (2004). Representing language use
in the university: Analysis of the TOEFL R
2000 spo-
ken and written academic language corpus (TOEFL R
Monograph No. MS-25). Princeton, NJ: Educa-
tional Testing Service.
Black, P., & Wiliam, D. (1998). Assessment and class-
room learning. Assessment in Education: Principles,
Policy & Practice, 5, 7–74.
Brown, J. D. (2014). The future of world Englishes in
language testing. Language Assessment Quarterly, 11,
5–16.
Brown, J. D., Hudson, T. D., Norris, J. M., & Bonk, W.
(2002). An investigation of second language task-based
performance assessments. Honolulu, HI: University of
Hawai‘i Press.
Burnstein, J. (2013). Automated essay evaluation and
scoring. In C. A. Chapelle (Ed.), The encyclope-
dia of applied linguistics: Assessment and evaluation.
Wiley–Blackwell.
Canale, M. (1983). On some dimensions of language
proficiency. In J. W. Oller, Jr. (Ed.), Issues in lan-
guage testing research (pp. 333–342). Rowley, MA:
Newbury House.
205
Canale, M., & Swain, M. (1980). Theoretical bases of
communicative approaches to second language
teaching and testing. Applied Linguistics, 1, 1–
47.
Carroll, J. B. (1961). Fundamental considerations in
testing for English proficiency of foreign students.
In Testing the English proficiency of foreign students
(pp. 30–40). Washington, DC: Center for Applied
Linguistics.
Chalhoub–Deville, M. (2003). Second language interac-
tion: Current perspectives and future trends. Lan-
guage Testing, 20, 369–383.
Chapelle, C. A. (1998). Construct definition and validity
inquiry in SLA research. In L. F. Bachman & A.
D. Cohen (Eds.), Interfaces between second language
acquisition and language testing research (pp. 32–70).
New York: Cambridge University Press.
Chapelle, C. A. (2008). The TOEFL validity argument.
In C. A. Chapelle, M. Enright, & J. Jamieson
(Eds.), Building a validity argument for the Test of En-
glish as a foreign language (pp. 319–352). London:
Routledge/Taylor & Francis.
Chapelle, C. A., & Douglas, D. (2006). Assessing language
through computer technology. Cambridge: Cambridge
University Press.
Cheng, L., Todd, R., & Huiqin, H. (2004). ESL/EFL
instructors’ classroom assessment practices: Pur-
poses, methods, and procedures. Language Testing,
21, 360–389.
Chodorow, M., & Burstein, J. (2004). Beyond es-
say length: Evaluating e-rater’s performance on
TOEFL R
Essays (TOEFL R
Research Report RR-
73, ETS Research Report RR-04-04). Princeton,
NJ: Educational Testing Service.
Clark, J. L. D. (1972). Foreign language testing: Theory and
practice. Philadelphia: Center for Curriculum De-
velopment.
Clark, J. L. D. (1975). Theoretical and technical consid-
erations in oral proficiency testing. In R. Jones &
B. Spolsky (Eds.), Testing language proficiency (pp.
10–24). Arlington, VA: Center for Applied Lin-
guistics.
Clark, J. L. D. (1978). Direct testing of speaking proficiency.
Princeton, NJ: Educational Testing Service.
Cobb, T. (2009). Complete Lexical Tutor. Montréal: Uni-
versity of Québec. Accessed 7 September 2015 at
http://www.lextutor.ca/
Council of Europe. (2001). Common European Frame-
work of Reference for Languages: Learning, teach-
ing, assessment. Cambridge: Cambridge University
Press.
Council of Europe. (2002). Common European Frame-
work of Reference for Languages: Learning, teaching,
assessment—Case studies. Cambridge: Cambridge
University Press.
Council of Europe. (2009). Manual for relating language
examinations to the Common European Framework of
Reference for Languages. Strasbourg, France: Coun-
cil of Europe.
Cumming, A. (2014). Assessing integrated skills. In A. J.
Kunnan (Ed.), The companion to language assessment
(pp. 216–229). Oxford, UK: John Wiley & Sons.
206
Cumming, A., Kantor, R., Baba, K., Eouanzoui, K., Er-
dosy, U., & James, M. (2006). Analysis of discourse
features and verification of scoring levels for indepen-
dent and integrated prototype written tasks for the new
TOEFL R
test (TOEFL R
Monograph No. MS-30).
Princeton, NJ: Educational Testing Service.
Davison, C. (2004). The contradictory culture of
teacher-based assessment: ESL teacher assessment
practices in Australian and Hong Kong secondary
schools. Language Testing, 21, 305–334.
Deane, P. (2015). On the relation between automated
essay scoring and modern views of the writing con-
struct. Assessing Writing, 18, 7–24.
Doughty, C. (2014). Assessing aptitude. In A. J. Kunnan
(Ed.), The companion to language assessment (pp. 25–
46). Oxford, UK: John Wiley & Sons.
Douglas, D. (2000). Assessing language for specific purposes:
Theory and practice. Cambridge: Cambridge Univer-
sity Press.
Eades, D. (2009). Testing the claims of asylum seekers:
The role of language analysis. Language Assessment
Quarterly, 6, 30–40.
Ellis, R., & Barkhuizen, G. (2005). Analysing learner lan-
guage. Oxford: Oxford University Press.
Fulcher, G. (2004). Deluded by artifices? The Common
European Framework and harmonization. Lan-
guage Assessment Quarterly, 1, 253–366.
Grabowski, K. (2009). Investigating the construct validity of
a test designed to measure grammatical and pragmatic
knowledge in the context of speaking. (Unpublished
doctoral dissertation). Columbia University, New
York.
Grabowski, K. (2013). Investigating the construct valid-
ity of a role-play test designed to measure gram-
matical and pragmatic knowledge at multiple pro-
ficiency levels. In S. Ross & G. Kasper (Eds.), Assess-
ing second language pragmatics. New York: Palgrave
Macmillan.
Gruba, P. (2014). New media in language assessment.
In A. J. Kunnan (Ed.), The companion to language
assessment (pp. 955–1012). Oxford, UK: John Wiley
& Sons.
Harding, L. (2011a). Accent and listening assessment: A val-
idation study of the use of speakers with L2 accents on an
academic English listening test. Frankfurt am Main,
Germany: Peter Lang.
Harding, L. (2011b). Accent, listening assessment and
the potential for a shared-L1 advantage: A DIF per-
spective. Language Testing, 29, 163–180.
He, A. W., & Young, R. (1998). Language proficiency
interviews: A discourse approach. In R. Young &
A. W. He (Eds.), Talking and testing (pp. 1–24).
Philadelphia/Amsterdam: John Benjamins.
Hill, K., & McNamara, T. (2011). Developing a com-
prehensive, empirically based research framework
for classroom-based assessment. Language Testing,
29(3), 395–420.
Hudson, T., Detmer, E., & Brown, J. D. (1995). Devel-
oping prototypic measures of cross-cultural pragmatics.
Honolulu, HI: University of Hawai‘i Press.
The Modern Language Journal, 100, Supplement 2016
Hymes, D. (1974). Foundations in sociolinguistics: An
ethnographic approach. Philadelphia: University of
Pennsylvania Press.
Isaacs, T. (2014). Assessing pronunciation. In A. J.
Kunnan (Ed.), The companion to language assessment
(pp. 140–155). Oxford, UK: John Wiley & Sons.
Iwashita, N., McNamara, T., & Elder, C. (2001). Can
we predict task difficulty in an oral proficiency
test? Exploring the potential of an information-
processing approach to ask design. Language
Learning, 51, 401–436.
Jacoby, S., & McNamara, T. (1999). Locating compe-
tence. English for Specific Purposes, 18, 213–241.
Jenkins, J. (2000). The phonology of English as an interna-
tional language. Oxford: Oxford University Press.
Jenkins, J., & Leung, C. (2014). English as a lingua
franca. In A. J. Kunnan (Ed.), The companion to
language assessment (pp. 1605–1616). Oxford, UK:
John Wiley & Sons.
Jones, R. (1985). Some basic considerations in test-
ing oral proficiency. In Y. Lee, A. Fok, R. Lord,
& G. Low (Ed.), New directions in language testing
(pp. 77–84). Oxford, UK: Pergamon.
Kane, M. (2006). Validation. In R. Brennen (Ed.), Educa-
tional measurement (4th ed., pp. 17–64). Westport,
CT: Greenwood.
Knoch, U., Macqueen, S., & O’Hagan, S. (2014). An in-
vestigation of the effect of task type on the dis-
course produced by students at various score levels
R
in the TOEFL iBT writing test. ETS Research Report
Series, 2014, 1–82.
Kramsch, C. (1986). From language proficiency to in-
teractional competence. Modern Language Journal,
70, 366–372.
Kunnan, A. J. (2004). Test fairness. In M. Milanovic
& C. Weir (Eds.), European language testing in a
global context (pp. 27–48). Cambridge: Cambridge
University Press.
Kunnan, A. J. (2009). Testing for citizenship: The U.S.
naturalization test. Language Assessment Quarterly,
6, 89–97.
Lado, R. (1961). Language testing. New York: McGraw–
Hill.
Lantolf, J. P., & Poehner, M. E. (2004). Dynamic assess-
ment: Bringing the past into the future. Journal of
Applied Linguistics, 1, 49–74.
Leki, I., & Carson, J. (1997). “Completely different
worlds”: EAP and the writing experiences of ESL
students in university courses. TESOL Quarterly, 31,
39–69.
Leung, C. (2009). Developing formative teacher assess-
ment: Knowledge, practice, and change. Language
Assessment Quarterly, 1, 19–41.
Leung, C., & Rea–Dickins, P. (2007). Teacher assess-
ment as policy instrument: Contradictions and ca-
pacities. Language Assessment Quarterly, 4, 6–36.
Levinson, S. (1983). Pragmatics. Cambridge: Cambridge
University Press.
Long, M. H., & Norris, J. M. (2000). Task-based language
teaching and assessment. In M. Byram (Ed.),
James E. Purpura
Encyclopedia of language teaching (pp. 597–603).
London: Routledge.
McNamara, T. F. (1996). Measuring second language per-
formance. London: Longman.
McNamara, T. F. (1997). ‘Interaction’ in second lan-
guage performance assessment: Whose perfor-
mance? Applied Linguistics, 18, 446–466.
McNamara, T. F., & Roever, C. (2006). Language assess-
ment at school: Social values and policy. Language
Learning, 56, 203–245.
McNamara, T. F., & Ryan, K. (2011). Fairness versus jus-
tice in language testing: The place of English lit-
eracy in the Australian citizenship test. Language
Assessment Quarterly, 8, 161–178.
Messick, S. (1989). Validity. In R. L. Linn (Ed.), Educa-
tional measurement (3rd ed., pp. 13–103). New York:
American Council on Education and Macmillan
Publishing Company.
National Research Council. (1999). The changing na-
ture of work: Implications for occupational analysis.
Committee on Techniques for the Enhancement
of Human Performance: Occupational Analy-
sis. Commission on Behavioral and Social Sci-
ences and Education. Washington, DC: National
Academy Press.
National Research Council. (2001). Building a workforce
for the information economy. Committee on Work-
force Needs in Information Technology. Board on
Testing and Assessment; Board on Science, Tech-
nology, and Economic Policy; and Office of Scien-
tific and Engineering Personnel. Washington, DC:
National Academy Press.
Norris, J. M., Brown, J. D., Hudson, T., & Bonk, J. (2002).
Examinee abilities and task difficulty in task-based
second language performance assessment. Lan-
guage Testing, 19, 395–418.
Norris, J. M., Brown, J. D., Hudson, T., & Yoshioka,
J. (1998). Designing second language performance
assessments (Vol. SLTCC Technical Report #18).
Honolulu, HI: Second Language Teaching and
Curriculum Center, University of Hawai‘i at
Mānoa.
Ockey, G. (2009). Developments and challenges in the
use of computer-based testing (CBT) for assessing
second language ability. Modern Language Journal,
93, 836–847.
O’Reilly, T., & Sabatini, J. (2013). Reading for understand-
ing: How performance moderators and scenarios impact
assessment design (Research Report No. RR-13-31).
Princeton, NJ: Educational Testing Service.
Oller, J. W. (1979). Language tests at school: A pragmatic
approach. London: Longman.
Patri, M. (2002). The influence of peer feedback on
self- and peer-assessment of oral skills. Language
Testing, 19, 109–131.
Phakiti, A. (2003). A closer look at the relationship of
cognitive and metacognitive strategy use to EFL
reading comprehension test performance. Lan-
guage Testing, 20, 26–56.
Plakans, L. (2009). Discourse synthesis in integrated
second language assessment. Language Testing, 26,
561–587.
207
Plakans, L. (2013). Assessment of integrated skills. In C.
A. Chapelle (Ed.), The encyclopedia of applied linguis-
tics. Wiley–Blackwell.
Poehner, M. (2014). Dynamic assessment in the class-
room. In A. J. Kunnan (Ed.), The companion to lan-
guage assessment (pp. 677–692). Oxford, UK: John
Wiley & Sons.
Purpura, J. E. (1999). Strategy use and language test perfor-
mance: A structural equation modeling approach. Cam-
bridge: Cambridge University Press.
Purpura, J. E. (2004). Assessing grammar. Cambridge:
Cambridge University Press.
Purpura, J. E. (2014a). Cognition and language assess-
ment. In A. J. Kunnan (Ed.), The companion to
language assessment (pp. 1452–1476). Oxford, UK:
John Wiley & Sons.
Purpura, J. E. (2014b). Language learner styles and
strategies. In M. Celce–Murcia, D. Brinton, & A.
Snow (Eds.), Teaching English as a second or foreign
language (4th ed., pp. 532–549). Boston: National
Geographic Learning/Cengage Learning.
Purpura, J. E. (2014c). Assessing grammar. In A. J.
Kunnan (Ed.), The companion to language assessment
(pp. 100–124). Oxford, UK: John Wiley & Sons.
Purpura, J. E., Brown, J. D., & Schoonen, R. (2015). Im-
proving the validity of quantitative measures in ap-
plied linguistics research. Language Learning, 65,
36–73.
Rea–Dickins, P. (2001). Mirror, mirror on the wall: Iden-
tifying processes of classroom assessment. Lan-
guage Testing, 18, 429–462.
Rea–Dickins, P. (2004). Understanding teachers as
agents of assessment. Language Testing, 21, 249–
258.
Roever, C. (2005). Testing ESL pragmatics. Frankfurt am
Main, Germany: Peter Lang.
Roever, C. (2014). Assessing pragmatics. In A. J. Kunnan
(Ed.), The companion to language assessment (pp.
125–139). Oxford, UK: John Wiley & Sons.
Ross, S., & Kasper, G. (Eds.). (2013). Assessing sec-
ond language pragmatics. New York: Palgrave
Macmillan.
Sabatini, J., & O’Reilly, T. (2013). Rationale for a
new generation of reading comprehension assess-
ments. In B. Miller, L. Cutting, & P. McCardle
(Eds.), Unraveling reading comprehension: Behavioral,
neurobiological, and genetic components (pp. 100–
111). Baltimore, MD: Brookes Publishing.
Sawaki, Y., Quinlan, T., & Lee, Y.–W. (2013) Understand-
ing learner strengths and weaknesses: Assessing
performance on an integrated writing task. Lan-
guage Assessment Quarterly, 10, 73–95.
Seidlhofer, B. (2011). Understanding English as a lingua
franca: A complete introduction to the theoretical nature
and practical implications of English used as a lingua
franca. Oxford: Oxford University Press.
Shohamy, E. (2001a). The power of tests: A critical perspec-
tive on the uses of language tests. London: Pearson.
Shohamy, E. (2001b). Democratic assessment as an al-
ternative. Language Testing, 18, 373–391.
Skehan, P. (1998). A cognitive approach to language learn-
ing. Oxford: Oxford University Press.
208
Snow, A., & Katz, A. (2014). Assessing language and con-
tent. In A. J. Kunnan (Ed.), The companion to lan-
guage assessment (pp. 230–247). Oxford, UK: John
Wiley & Sons.
Turner, C. E., & Purpura, J. E. (2015). Learning-
oriented assessment in second and foreign lan-
guage classrooms. In D. Tsagari & J. Baneerjee
(Eds.), Handbook of second language assessment (pp.
255–272). Boston: De Gruyter Mouton.
Van den Branden, K., Bygate, M., & Norris, J. M.
(Eds.). (2009). Task-based language teaching: A
reader. Philadelphia/Amsterdam: John Benjamins.
The Modern Language Journal, 100, Supplement 2016
Van Gorp, K., & Deygers, B. (2014). Task-based language
assessment. In A. J. Kunnan (Ed.), The companion
to language assessment (pp. 578–593). Oxford, UK:
John Wiley & Sons.
Vollmer, H. J., & Sang, F. (1983). Competing hypotheses
about second language ability: A plea for caution.
In J. W. Oller, Jr. (Ed.), Issues in language testing
research (pp. 29–79). Rowley, MA: Newbury House.
Yu, G. (2013). Performance assessment in the class-
room. In A. J. Kunnan (Ed.), The companion to lan-
guage assessment (pp. 615–630). Oxford, UK: John
Wiley & Sons.