LANGUAGE TESTING

AND ASSESSMENT

UNIT CODE: NALTA5112

CREDIT POINTS: 3
LEVEL: POST GRADUATE
 WHAT DID YOU LEARN
FROM PREVIOUS LESSON?
LINEAR STAGES OF TEST DEV.

A DESIGN stage:
A
Test purpose, Test
CONSTRUCTION A TRY-OUT stage
content and Test
stage
Method

TEST TEST TEST TRIALS

SPECIFICATIONS MATERIALS DATA
TEST DEVELOPMENT
REVIEW
To practice as a tour guide in Korea or a taxi driver in
Wales, you are now required to pass a language test.
For each job, make a list of the purposes for which you
would use language. You may also wish to list the
kinds of language functions, grammar, or vocabulary
that might be particularly relevant to the context.
SPECIFICATION

Test context & purposes

Test structure

Test tasks

(Carr, 2011)
TEST CONTEXT & PURPOSES
CONSTRUCT DEFINITION
CONSTRUCT DEFINITION
TEST STRUCTURE
TEST TASKS
• the task format(s) to be used to assess each construct, or
each aspect of each construct
• how many of each task (items, passages, prompts, etc.) to
use in each section
TEST TASKS
TEST TASKS
TEST TASKS
Speaking & writing prompts:
• Provide instructions for how to write those prompts (purpose
of the communication, the topics of the prompts, and how
topic-specific the prompts can be)
• Make clear what sort of response is desired (length: a few
sentences, an entire paragraph, or a properly organized
essay…)
TEST TASKS
Vocabulary & grammar:
• the distinction between the ability to recognize a word, or
comprehend a structure, and to use that word or structure
accurately.
• consider the range of vocabulary or grammatical structures that
have been acquired.
• consider the degree of mastery that has been achieved over the
words or structures that are actually produced.
TEST TASKS
TEST TASKS
TEST TASKS
APPROACHES TO ITEM WRITING
Psychometric

Authentic

Systematic

Critical

(Fulcher & Davison, 2012)

PSYCHOMETRIC APPROACH
“Reproduce similar items in multiple-choice format. Do it
objectively.”

• 2nd half of the 20th century

• a quantitative, objective, and outcome-oriented view of

testing

• MCQ, T/F, matching

• criticized for giving too much weight to students‟

knowledge of the linguistic system
AUTHENTIC APPROACH
“Search for authentic items. Avoid the multiple-choice
format.”
• qualitative, subjective, and process-oriented views of
testing
• incorporate the real world, value practical language
skills, and choose items which appear integrative,
authentic, communicative, and pragmatic
• direct tests of speaking and writing, comprehension tests
of extended discourse, cloze tests.
• criticized for subjective scales, unreliable raters, and
impractical test administration
SYSTEMATIC APPROACH
“See the whole picture of test development. Do it systematically.”
• testing practices as contingent, as unpredictable from complex
variables, and as open to different methods of item creation
and validation
• design test items as “the „response as evidence‟ to the
construct as language knowledge, or as ability for use within
specified parameters laid out in a framework and test
specification”
• test specifications can provide detailed information on each
design of test items
• test items need to be well-discussed and clearly characterized
in the stage of test specification creation.
CRITICAL APPROACH
“Create items to help test takers. Think socio-politically.”
• objective test formats grant language tests power.
• political aspects embedded in the process of test design
and use
• called for ethical awareness in language testing
• focuses on test misuse
• pay more attention to the dimension of item design
driven by intended or unintended misuses of the test.
TASK FORMAT
• What a test looks like in terms of what sort of input
test takers must process, and how we expect them to
respond (e.g. choosing an option, writing one word,
or performing a role-play)
• Similar terms: task type, item format, test method,
response format, method, technique.
 FIRST CLASSIFICATION:
Types of response that a particular task calls for

Selected response Constructed response tasks

tasks … call for the examinee to write or say
something in response.
… require the
Limited Extended
examinee to choose
production tasks production tasks
the correct answer
… call for examinees … require
from among
to provide a short responses that are
several options.
answer, which may longer than one
be as short as one sentence or
word, or as long as a utterance.
sentence
SECOND CLASSIFICATION:
Item and prompts

Items Prompts

… a question that requires a … asks the test taker to

short answer, or the provide an extended
selection of an appropriate response.
response (selected response or … most commonly use in
limited production tasks) speaking and writing
assessments.
THIRD CLASSIFICATION:
Passage-based or independent
Passage-based
… require examinees to read
or listen to material in order
to perform the task.
… the questions cannot be
answered or tasks cannot be
performed, without
comprehending the passage.
Independent
… do not require the
processing of any additional
material.
… test takers can respond to
the task solely on the item or
prompt itself.
SOURCES OF TEST TASKS

Construct-based test development

Task-centered test development

Construct-based test Task-centered test development
development
• starts out by considering • starts with deciding which tasks
the construct that is from TLU domain – typically
being assessed, and real-world language use tasks -
selecting or creating are of greatest interest.
tasks that will tap into it. • sampling is necessary because not
everything can be included in the
test.
• adaptation is needed because we
are usually simulating real-life
tasks in a classroom, not
following students around in the
real world and scoring how they
do „out there.‟
→ adopt both approaches
GENERAL SUGGESTIONS ON
ITEM WRITING
• Keeping questions clear and construct-relevant
• Not giving away the answers
• Maintaining item dependence
EXAMPLE
Both items are based on the same listening comprehension
passage.
EXAMPLES
SELECTED RESPONSE TASKS
• Multiple choice
Identify item stem, options, key, distractors.
There ______ four chairs in the kitchen.
A. has C. is
B. have D. are
SELECTED RESPONSE TASKS
• Multiple choice
- putting as much information as possible into the stem,
but restricting it to only the information needed to pose
the question.
- avoiding negatives, particularly not, whenever possible,
as these can make the questions more difficult to read.
- Words such as not, except, etc. should be highlighted very
obviously, preferably in bold face and either italicized,
underlined, or printed in CAPITAL LETTERS.
SELECTED RESPONSE TASKS
- Options should all come from the same semantic field, from
different semantic fields, or the fields should be balanced
equally
- Semantic similarity among options is associated with
increased item difficulty

E.g:
In the story, the man bought two ___________ tickets.
A. airplane
B. bus
C. movie
D. train
EXAMPLES
SELECTED RESPONSE TASKS
• True-False
• Matching
• Ordering tasks
LIMITED PRODUCTION TASKS
Can be either passage-based or independent.
• Short-answer questions
- force students to produce the correct forms, not
merely recognize them
- be careful about including grammar or vocabulary
use as part of the scoring criteria when assessing
reading or listening ability.
- take into consideration the expected level of writing
ability of the students who will be taking the test
- the key should include a list of all acceptable
answers.
LIMITED PRODUCTION TASKS
• Fill-in-the-blank / gap-fill items
• Sentence writing or combining tasks
• Completing a blank or incomplete graphic organizer
• Completing an incomplete outline
EXTENDED PRODUCTION TASKS
• Recall tasks: students write as detailed a summary
as they can remember after listening to or reading a
passage.
• Written summaries: test takers are allowed to refer
to the passage (or notes in listening) when writing.
• Dictation: integrates listening ability with writing
ability
• Note taking: may be appropriate in academically
oriented language programs
EXTENDED PRODUCTION TASKS
• Interview: can be highly scripted with specific
prompts or much more free-formed. However, the
issue of discourse authenticity (not like what happens
in real-world) has led to interest in alternative
approaches to speaking assessment: group oral
interview
• Group oral interviews: pairs or small groups of
students discuss one or more specific topics
EXTENDED PRODUCTION TASKS
• Role-play tasks: assess learners‟ ability to engage in
various speech acts, or to perform a wider variety of
language functions, in a wider range of situations and
about a wider range of topics.
• Monologue
• Reading aloud / Oral reading: assess pronunciation
ability including both segmentals (vowels and
consonants) and suprasegmentals (stress, rhythm,
intonation, connected speech etc.) and reading fluency.
EXTENDED PRODUCTION TASKS
• Portfolio: collections of student work from a course,
and are useful for showing a student‟s development
and improvement.
• Translation: as a means of assessing reading,
writing, speaking, listening, grammar, vocabulary,
discourse competence, or sociolinguistic
competence.