PRETEST:

WHAT ARE THINGS OR STANDARDS YOU SHOULD

CONSIDER IN CONSTRUCTING DESIGNING A TEST?
MAKE A CREATIVE GRAPHIC ORGANIZER OF YOUR
IDEAS.
 CRITERIA &
TYPES OF TEST

Prepared by: Maricris L. Refugio

MAEd-Language Instruction
 S U M M A R Y:

01 02 03 04 05

VALIDITY VS. TEST INSTRUCTION BACKWASH TYPES OF TEST

DISCRIMINATION
RELIABILITY TO THE TESTEE EFFECT
 01
RELIABILITY
VS. VALIDITY
“A Valid Test is always reliable but a
Reliable test is not necessarily Valid”
 RELIABILITY
• CONSISTENCY, DEPENDENCE, & TRUST
• Results should be dependable. 1) EQUIVALENCE

• Should remain stable, consistent, not be different

when the test is used on different days. 2) STABILITY

Three (3) Aspects:

3) INTERNAL
1. Reliability of the test itself. CONSISTENCY

2. Reliability of the way in w/c it has been

marked.

3. Reliability of the way in w/c it has been

administered.
 ASPECTS OF RELIABILITY

01 02 03

INTERNAL CONSISTENCY/
EQUIVALENCE STABILITY
HOMOGENEITY
The amount agreement similar scores are
between two or more tests the extent to which items
obtained with repeated on the test are measuring
that are administered at testing with the same
nearly the same point in the same thing.
group of respondents.
time.
 FACTORS THAT AFFECT TEST RELIABILITY

01 LENGTH OF THE TEST

The more items on the test, the
more reliable it is considered to be.

02 ADMINISTRATION OF THE TEST

Classroom setting (lighting, seating
arrangements, lack of intrusive noise etc.)

03 AFFECTIVE STATUS OF STUDENTS

Test anxiety can affect student’s test result.
 VALIDITY
“Is this really going to test what I
want to know?”
“Whether or not the test measures
what it claims to measure”
Like reliability, there are also some
factors that affect the validity of the
test scores.
Includes:
1. CONTENT (how well
performances demonstrate the
specified learning domain)
2. CONSTRUCT (theoretical
principles are reflected in the test
design)
3. FACE (extent to w/c a test
appears to measure what is
intended)
 FACTORS IN THE TEST
● Unclear directions to students to respond to the test.
● The difficulty of reading vocabulary and sentence
structure.
● Too easy or too difficult test items.
● Ambiguous statements in the test items.
● Inappropriate test items for measuring a particular
outcome.
● Inadequate time provided to take the test.
● The length of the test is too short.
● Test items not arranged in order of difficulty.
 Factors in test
Administration & Scoring
● Unfair aid to individual students, who ask for help.
● Cheating by students during testing.
● Unreliable scoring of essay type answers.
● Insufficient time to complete the test.
● Adverse physical and psychological condition at the
time of testing.

Factors related to students:

● Test Anxiety
● Physical and Psychological state
 02
DISCRIMINATION
Davies (1999) suggests that, in general, an item would
be expected to have a difficulty index of 30-70% to be
useful in discriminating between candidates. He points
out, however, that this will vary depending on test
purpose.
 DISCRIMINATION
• distinguishes between strong and weak students.

• All assessment is based on the comparison, either between one

student and another or between students as they are now and as they
were earlier.
• An important feature of a good test is its capacity to discriminate among
different students or the same student at
the performance of
different points in time.
• The extent of the need for discrimination varies according to the
purpose of the test.
 03
TEST INSTRUCTION
TO THE TESTEE
According to an article in Thoughtco, not following directions on
a test is one of the top ten mistakes students make on
tests. While not a cure-all, writing clear instructions on a test
will help to prevent that from happening.
 TOP 6 GUIDELINES IN WRITING
CLEAR TEST INSTRUCTIONS

01 02 Clearly
03 Requesting
Specific Help
& Concise Stated Goals Encouraged

04 05 06
Instructions Split
Within Complex Give clear
Questions Questions context
 Specific
01 & Concise

Note: For my examples, I will be using literature

tests “The Lion, the Witch, and the Wardrobe.”

Based on our classroom discussion on what tactics

the White Witch used to trick Edmund into believing
her, answer the following questions:
02 Clearly Stated
Goals

Here is an example of directly telling the students the goal that

I have in the subsequent questions:

For the following questions, think about how Edmund

responded to the White Witch after he enjoyed the Turkish
Delight and hot drink. Then answer the questions in light of
your notes and classroom discussion.
 Requesting
03 Help
Encouraged
 Instructions
04 Within
Questions

Here is an example of providing the instructions for

answering the question within the question:

When you think about the White Witch trying to be nice to

Edmund, but yet her evil tendencies emerge several times
throughout her attempt to gain his trust, how do you think
he was able to trust her without seeming to be affected by
the evil that she couldn’t hide?
________________________________________________
 Split
05 Complex
Questions

Example
While this is more effective in more complex subjects such as math or science,
it is also effective in literature. Here is an example:

How did the White Witch counsel Edmund with regard to the position of the 4
human children’s roles in Narnia?
__________________________________________________
How did Aslan counsel Peter, Susan, and Lucy with regard to the position of
the 4 human children’s roles in
Narnia?__________________________________________________
How do you think the Pevensie children came together in fulfilling their roles in
Narnia in spite of the very different counsel they received?
__________________________________________________
06 Give clear
context

Example
Here is an example of how context helps the students to recall specifically what
I wanted them to regarding the questions that followed:

From our discussion on Chapter 4, where the White Witch spent the
entire chapter tricking Edmund into trusting her so he would be
willing to do anything she bade him, answer the following questions:
 04
BACKWASH EFFECT
Huhges defines backwash as "the impact of language
testing on teaching and learning", and points out that
language testing can have good or bad effects on teaching
and learning.
 BACKWASH EFFECT
• Influence or reaction of testing on teaching (including teaching and learning) in the field of
education. Prodromou (1995:13) “Backwash is the direct or indirect influence of testing on
teaching methods.”
• Mcnamara (2003:73 “the backwash effect of testing is the impact of testing on teaching.
Later, British applied linguists introduced the concept of "backwash" into the field of
linguistics.
• Mesisck “effect of testing on teachers and students engaged in language teaching to
varying degrees to do things they did not need to do, which should be interpreted as
"backwash effect".
• Wall and Anderson “test developers and candidates not only believe that language testing
has an important impact on teaching and learning, but also investigate the impact of
specific areas, positive and negative directions and predictions.”
• Backwash has the potential to affect not only individuals,
but also the entire education system and all aspects that
testers need to investigate. Therefore, the effect of
backwash is much more complex than that of simple
teaching tests .

• Common point is that scholars use "backwash" to explain

the impact of language testing on teaching and learning.

• Testing plays an important role in backwash of teaching

objectives, content, methods, and even reliability and
validity.
BACKWASH EFFECT
 BACKWASH EFFECT
• Dual characteristics:

1. Scientific Testing can have a positive impact on teaching, play a

positive backwash role, can point out the correct direction of teaching,
ensure and promote the realization of teaching objectives
2. Unscientific testing has a negative impact on teaching, plays a
negative backwash role, may lead teaching into a wrong direction,
affect or even hinder the realization of teaching objectives.
 05
TYPES OF TEST
THE NEXT STEP INORDER TO FIND APPROPRIATE TESTING METHODS IS
TO ANALYZE THEM ACC. TO LEARNING TARGETS
(3) Different Targets (which are basically the main skills one needs to learn)
in a language:
2. COMMUNICATIVE
1. LANGUAGE
SKILLS
SUBSYSTEMS a. Listening
a. Phonology b. Speaking
b. Vocabulary c. Reading
c. Grammar Comprehension
d. Spelling d. Writing
3. CULTURAL COMPETENCE
a. Facts &History
b. Traditions
c. Prevailing Attitudes
 The first distinction can be made between

LONG ANSWER SHORT ANSWER

• Essay A. Open Answer B. Closed Answer
• Prediction • Information Gap • Multiple Choice
• Monologue • Gap Filling And Completion • True/False
• Role play • Cloze • Matching Exercises
• Problem solving • Interviews
 ESSAY

• The testee is given a topic, such as ‘Positivity Amidst Pandemic', and

asked to write an essay of a specific length.
• A good test of general writing abilities.
• It is relatively easy to provide a topic and tell the class to write an
essay about it but marking is extremely difficult and time-consuming.
PREDICTION
• This activity resembles gap filling, but requires critical thinking skills, as
students imply information they haven’t got during the learning process.
• Prediction questions are directed toward material not yet read. As
students read, they look for clues that help them decide what might
come next in the text.
• Predictions help students set expectations for reading, use text to aid
comprehension, and to compare their thinking with what the author has
written
MONOLOGUE

• The testee is given a topic or question and asked to speak about it for a
minute or two.
• This tests oral fluency in 'long turns' - something not everyone can do in
their mother tongue!
• It also tests overall knowledge of pronunciation, grammar and
vocabulary.
• To choose a topic and allot it is not so difficult; to assess is very difficult
indeed, demanding concentration and a very clear set of criteria and
weighting system
 ROLEPLAY

• Role-playing is a form of assessment that requires students to take

a role. This can include acting; as in skits, plays, or fake interviews, or
taking a role as in an accountant, journalist; or world leader and
performing the duties of that position for an amount of time.
PROBLEM SOLVING

• The teacher give a role for one student to be someone who has a
real-life problem, and another student will be an expert who can give
an advice to fix the problem.
• Build Confidence
• They learn to look at challenges from a fresh perspective. Therefore,
they take more calculated risks. If students practice problem
solving consistently, they can develop greater situational and social
awareness. Additionally, they learn to manage time and develop
patience
INFORMATION GAP

• An information gap task is a technique in language teaching where

students are missing information necessary to complete a task or solve a
problem, and must communicate with their classmates to fill in the gaps.
• It is often used in communicative language teaching and task-based
language learning.
• type of communicative activity in which each participant in the activity
holds some information other participants don't have and all participants
have to share the information they have with other participants in order to
successfully complete a task or solve a problem.
GAP-FILLING & COMPLETION

• The testee has to complete a sentence by filling a gap or adding

something.
• A gap may or may not be signalled by a blank or dash; the word to be
inserted may or may not be given or hinted at.
• This usually tests grammar or vocabulary, as in the examples.
• It is tedious to compose, though not so difficult as multiple-choice; it is
more easily administered in writing than in speech; the marking is usually
simple. You may need to be aware that there is more than one possible
right answer. .
 CLOZE
• It is fairly easy to
prepare and administer.
• Marking can be tricky:
you may find it difficult
sometimes to decide if a
specific item is
'acceptable' or not..
 INTERVIEWS

• The teacher will give a role to the

students who want to get a job in job
interview, and a teacher acts as
someone who wants to interview
them.
 TRUE/FALSE QUESTIONS
• pretty straight forward.
• A statement is given which is to be marked true
or false.
• This may also be given as a question, in which
case the answer is yes or no.
• This does not directly test writing or speaking
abilities: only listening or reading. It may be
used to test aspects of language such as
vocabulary, grammar, content of a reading or
listening passage. It is fairly easy to design;
• Also easy to administer, whether orally or in
writing, and to mark.
MATCHING EXERCISES

• Require the learner to match the elements of an exercise, e.g. problem-

solution, question-answer, the right way.
• The testee is faced with two groups of words, phrases or sentences; each
item in the first group has to be linked to a different item in the second
• . This usually tests vocabulary, and is rather awkward to administer orally:
thus it is best presented written on the board or on paper, though responses
may be either oral or in writing.
• Items can be time-consuming and difficult to compose, and again, there
may be alternative 'right' answers to any particular item. Answers are fairly
easily checked.
MULTIPLE CHOICE QUESTIONS

• This may be used for the same testing purposes as true/false items; it does test
rather more thoroughly since it offers more optional answers and is obviously
very easy to mark.
• It is administered more conveniently through writing but note that since the
reading of the question-and-options is fairly time-consuming, the process of
comprehension of the actual question items may take more time and effort than
the point ostensibly tested, which raises problems of validity.
• Another important problem is that good multiple-choice questions are
surprisingly difficult to design: they often come out ambiguous, or with no clear
right answer, or with their solutions over-obvious. They are to be approached
with caution!
 OTHER
METHODS 0F
LANGUAGE
TESTING
 DICTATION
• The tester dictates a passage or set of words; the testee writes them down.
• This mainly tests spelling, perhaps punctuation, and, perhaps surprisingly
on the face of it, listening comprehension: people can only usually write
words down accurately from dictation if they understand them. It does not,
however, test other writing skills or speech, and involves very little reading.
• It may supply some information on testees' passive knowledge of
pronunciation, grammar and vocabulary.
• It is very easy to prepare and administer; it is relatively easy to mark,
though there may be a problem deciding how much weight to attribute to
different mistakes
 REWRITING

• A sentence is given; the testee rewrites it, incorporating a given change of

expression, but preserving the basic meaning.
• This tests the same sort of thing as transformation, but is likely to reflect
more thorough knowledge of the target items, since it involves
paraphrasing the entire meaning of a sentence rather than transforming a
particular item.
• It is, however, more difficult to compose, and the marking may be more
subjective. It is, as its name suggests, usually done in writing
TRANSFORMATION

• A sentence is given; the testee has to change it according to some given

instruction.
• This item is relatively easy to design, administer and mark, but its validity
may be suspect.
• It tests the ability of the testee to transform grammatical structures, which
is not the same as testing grammar: a testee may perform well on
transformation items without knowing the meaning of the target structure
or how to use it in context.
• Marking is fairly straightforward.
 TRANSLATION

• The testee is asked to translate expressions, sentences or entire passages

to or from the target language.
• A technique which, at the time of writing, is for various reasons rather
unpopular, but in my opinion undeservedly so.
• In a monolingual class whose teacher also speaks the learners' mother
tongue, the translation of a 'bit' of language to or from the target language
can give very quick and reliable information on what the testee does or
does not know, particularly when it involves entire units of meaning
(phrases, sentences) within a known context.
 QUICK PREVIEW:
VALIDITY TEST INSTRUCTION BACKWASH TYPES OF TEST
DISCRIMINATION
VS. RELIABILITY TO THE TESTEE EFFECT

01 02 03 04 05
A.Open Answer SHORT ANSWER
LONG ANSWER •Information Gap
• Gap Filling And Completion
• Essay • Cloze B. Closed Answer
• Prediction •Interviews • Multiple Choice
• Monologue •Matching, • True/False
• Role play •True/False Questions • Matching Exercises
• Problem solving •Multiple Choice
THANKS
FOR
LISTENING!
 REFERENCES
✓ https://www.researchgate.net/publication/3430987
40_Language_Test_Construction_and_Evaluation

✓ https://www2.vobs.at/ludescher/testing/types_of_t
est_techniques.htm?fbclid=IwAR0_Lm5FnLh5ziQEDe
MJg8Yd9JdzN6ygcz-CDNDG4dySKbTeGtg78OgRrPo

✓ https://alhoward.wordpress.com/2011/11/29/a-valid-
test-is-always-reliable-but-a-reliable-test-is-not-
necessarily-
valid/?fbclid=IwAR02cpJnZVelJAEB0au0fSMPewGBw
xn1jKE7ZGtX0wPsneN1zI9tlcog5Dw

✓ file:///C://55916440.pdf
POST TEST
• How would you make sure of the effectiveness of
your test-construction?
• What are other test methods you used in language
teaching? (Give atleast 5)
• How do you improve its quality?
• As a language teacher, how would you obtain a
positive backwash effect of your test construction?