Multiple Choice Item

Construction
D. Michele Jacobsen
Assistant Professor
dmjacobs@acs.ucalgary.ca

Test Development Process

1. Statement of Goals
2. Content Outline
3. Table of Specifications
4. Item Selection
5. Item Construction
6. Composition of Instructions
7. Development of Answer Sheets
8. Construction of Answer Keys
9. Test Administration
10. Test Revision

Todays Focus
3. Table of Specifications
4. Item Selection
5. Item Construction

Two Important Characteristics of Tests

Reliability
consistency
free from extraneous sources of error

Validity
how well a test measures what it is
supposed to measure

Formative vs. Summative Tests

Formative
monitor progress toward goals within a
course of study

Summative
assess overall achievement of course
goals

A. Table of Specifications
Blue print for test
Purpose
ensure proper emphasis given to all
elements of a course of study
content validity

Guide for writing items

Table of Specifications (2)

Start with Instructional Objectives
Biology 30: Circulation Unit Test
1.
Identifies parts of circulation
system.
2.
Explains function of parts in
relation
to whole system.
3.

Distinguishes between circulation

system and other bodily systems.

Levels

Of

Understanding

Knowledge Comprehension Application

Biology 30: Circulation

Need to Consider and Decide:

Length of Test
Weight to be given to each
objective
Weight to be given to each level of
taxonomy
Estimate number of items in each
cell

Biology 30: Circulation

L evels
Content
Parts
Functio
n
Relation
to other
systems

Of

Understanding

Knowledge Comprehension A pplication

10

15

15

10

15

15

15

15

45

B.

Item Selection

Types of Items
Objectively Scored (Selection)

true/false
completion
matching
multiple choice

Subjectively Scored (Supply)

interpretive exercises
essay

C.

Item Construction

Selected Response Test Items

Item construction skills for valid and
reliable measures of student
achievement.
Guidelines which apply to all types
Specific Suggestions for writing each type
Advantages and Disadvantages of Each

Guidelines for Writing Objective Items

1.

Construct at appropriate level of

difficulty for examinees

2.

Include Items at appropriate level of

difficulty for purpose of test.

3.

Test significant elements of a

course.

4.

Write independent items.

II. Guidelines for Writing Objective Items

5.

Construct questions free from

extraneous reasons for problems.

6.

Communicate the question in clear,

concise language.

7.

In the correct alternatives,

paraphrase statements from the text.

8.

Exclude clues to correct answer.

III. Guidelines for Writing Objective Items

9.

Provide one correct answer.

10.

Edit the Items.

Alternate Response Items

Involves the selection of one of two
alternatives
true / false
yes / no
right / wrong
fact / opinion
Mainly for Knowledge & Comprehension
Can be written at higher levels

True / False
Word statements clearly. Vague or ambiguous wording
will confuse students.
Avoid overgeneralizing.
Poor:

Heavy smoking causes lung cancer.

Better:

Heavy smoking often causes lung cancer. T F

T F

Avoid Trick questions.

i.e., General Wolseley led Canadian troops to Manitoba in
1870.
3 tricks: Colonel, British, Not yet a province

Do not use trivial statements to pad out the number of

questions and marks to arrive at a predetermined level.

True / False
Statements should be entirely true, or
entirely false:
Unacceptable:
In King Lear, Regan ordered Gloucesters eyes to be
plucked out and Gloucester died when he jumped off
the cliff of Dover.

Acceptable:
In King Lear, Regan ordered Gloucesters eyes to be
plucked out. (T)
In King Lear, Gloucester died when he jumped off
the cliff at Dover. (F)

True / False
Avoid using universal descriptors such as
never, none, always, and all.
Testwise students will recognize that there are few
absolutes.

Avoid negative words, as they are often

overlooked by students.
Poor:
Better:

It was not unheard of for Henry VIII to close

monasteries in England. T F
Henry VIII closed some monasteries in England.
(T)

True / False
Do not include two ideas in one statement unless you are
evaluating students understanding of cause and effect
relationships.
Poor:
Better:

Porpoises are able to communicate because

they are mammals.
T F
Porpoises are mammals.
T F
Porpoises are able to communicate.
T F

Provide a T and F beside each statement and ask

students to circle correct answer.
Avoids problem of students writing illegible letters.

True / False & Variations

Include more false than true statments in any given test
and vary the number of false statements from test to test.
tendency to mark more statements true than false.
discrimination between those who know the content
and those who do not is greater for false expressions.
Avoid using negative statements.
Under the demands of the testing situation, students
may fail to see the negative qualifier.

Matching Items
Consist of
a column of premises
a column of responses
directions for matching the two.
Similar to multiple choice, but easier and more
efficient to construct
Can be written to assess Knowledge,
Comprehension, Application, Analysis level
behaviors

Guidelines for Writing Matching Items

Provide clear instructions on how to indicate the
correct answers.
Indicate whether the same response can be used
more than once.
Maintain grammatical consistency within and
between columns.
within a column: either sentence or point form
between columns: one or the other
Ensure that any matching question appears entirely
on one page.

2. Guidelines for Writing Matching Items

Provide an unequal number of premises and
responses
reduces guessing and elimination
increases measure of comprehension

Avoid designing questions which require

students to draw lines between premise and
response.
confusing for student and marker
provide space for letter or number answers

3. Guidelines for Writing Matching Items

Make sure lists are homogeneous.
i.e., do not include items testing names, dates, and
events.
Instead, make every response plausible

Make the wording of the premises longer than

the wording of the responses.
Identify the items in one list with numbers and
those in the second list with letters.

Example:
Directions:

1. On the line to the right of each phrase in

Column I, write the letter for the word in
Column II that best matches the phrase.
2. Each word in Column II may be used once,
more than once, or not at all.

Column I
1. Name of the answer in
addition problems.
2. Name of the answer in
subtraction problems.
3. Name of the answer in
multiplication problems.
4. Name of the answer in
division problems.

Column II
A. Difference
B. Dividend
C. Multiplicand
D. Product
E. Quotient
F. Subtrahend
G. Sum

Short Answer Test Items

Typically, the student is asked to reply with a
word, phrase, name, or sentence, rather than a
more extended response.
Direct Questions / Short Answer
Who is the current Prime Minister of Canada?
Incomplete Sentences / Fill In the Blanks
The current Prime Minister of Canada is _____?

Items are fairly easy to construct and mark

Assess mainly knowledge, comprehension, and
some application.

Guidelines for Writing Short Answer Items

Questions must be carefully worded so that all
students understand the specific nature of the
question asked and the answer required.
Poor:

Wellington defeated Napoleon in _____ ?

Better: In what battle fought in 1815 did Wellington

defeat Napoleon?
OR
In what year did Wellington defeat Napoleon at
Waterloo?

II. Guidelines for Writing Short Answer Items

Word completion or fillin questions so that missing
information is at, or near the end of, the sentence.
Makes reading and responding easier.
Poor: In the year ______ , Canada turned 100 years
old.
Better: Canada turned 100 years old in the year
_____ .
Instructions and teachers expectations about filling in
blanks should be made clear. Indicate whether each
blank of equal length represents one word or several
words, whether long blanks require sentences or
phrases, and whether synonymous terms are accepted.

III. Guidelines for Writing Short Answer Items

When an answer is to be expressed in numerical units, the
unit should be stated.
Poor: If a room measures 7 metres by 4 metres, the
perimeter is _____ .
Better: If a room measures 7 metres by 4 metres, the
perimeter is _____ metres (or m).
Do not use too many blanks in completion items. The
emphasis should be on knowledge and comprehension,
not mind reading!
Consider:
In the year _____ , Prime Minister _________ signed the
__________ , which led to a ___________ which was
__________ .

Multiple Choice Items

Interpretive Exercise
Guidelines for Writing
Advantages & Disadvantages

Terminology: Multiple Choice

1

1. The capital city of Canada is

a. Vancouver
b. Montreal
3
4
c. Toronto
*d. Ottawa
2

1. Stem: presents the problem

2. Keyed Response: correct or best answer
3. Distracters: appear to be reasonable answers to the
examinee who does not know the content
4. Options: include the distracters and the keyed
response.

Ambiguity
Extrinsic
Desirable quality in
multiple choice
items
Outside the item
Allows
discrimination
between those who
know material and
those who do not.

Intrinsic
Undesirable
Should be avoided
Inside the item
poor wording
more than one answer
Even those who know
content have difficulty
choosing correct answer

Types of Multiple Choice Items

Correct Answer*
Only one correct response
Best Answer
requires examinee to select alternative
closest to being correct
fine distinctions
Multiple Answer
More than one correct or best answer

Interpretive Exercise
Usually begins with verbal, tabular or graphic
information which is the basis for 1 or more
multiple choice questions.
map, passage from a story, a poem, a cartoon

Can challenge students at various levels of

understanding
application, analysis, synthesis, evaluation

Exercise contains all information needed to

answer questions
Readily adaptive to the more important
outcomes of disciplines.

Interpretive Exercises (cont)

Examples
If student answers incorrectly it is because
they have not mastered the thinking or
reasoning required by the question, NOT
because they failed to memorize background
information.
Math questions: give students the formulas,
test ability to apply concepts, rather than
ability to memorize formulas.

Guidelines for Writing:

Multiple Choice Items
State stem in the form of a question.
Weak
Canada is
a)
a country
b)
where you live
c)
between the Pacific and Atlantic oceans
d)
all of the above
Better
Between which two oceans is Canada located?
a)
Atlantic and Indian
b)
Atlantic and Pacific
c)
Pacific and Indian

Guidelines for Writing: Stems

Place most of the subject matter in the Stem
ensures full statement of problem

Eliminate extraneous material from the Stem

goal is to measure student achievement, not to present
new material
maximize use of time for demonstrating understanding,
not reading ability

Avoid Negatively phrased Stems

students may miss the qualifier
use only when learning outcome requires this type of
differentiation

I. Guidelines for Writing: MC

Ensure similarity among alternatives
with regard to:
grammatical structure
length
mode of expression

Grammatical errors provide

unintentional clues to the answer
When in doubt, students will select the
longest alternative as the correct answer

Example: Length of Alternatives

Neurotics are more likely than psychotics to
a.
b.
c.
*d.

be dangerous to society
have delusional symptoms
be dangerous to themselves
have insight into their own inappropriate
behavior but nevertheless feel rather
helpless in terms of dealing with their
difficulties

II. Guidelines for Writing: MC

Make one of the alternatives the most clearly
correct or best answer
exception: multiple answer form
reduces intrinsic ambiguity
reduces frustration during test

Make distracters plausible

desire to attract students who really do NOT know the
answer to the question
create distracters from elements of the correct response
improves reliability of item

Example: Undemanding Distracters

Mickey Mouses two nephews are
a) Huey, Dewey and Louie
b) Clarabelle Cow
*c) Morty and Ferdy
d) Abbott and Costello
The Role of Humour?

III. Guidelines for Writing: MC

Avoid parallel language between the Stem and
the Correct Response
gives clues to keyed response
emphasizes testwiseness, not knowledge

Randomly distribute answers across the

alternative positions
inexperienced test writers emphasize b and c
alternatives (hide the answer!!)
do NOT use an interpretable order of keyed responses

IV. Guidelines for Writing: MC

Use qualifiers such as all of the
above and none of the above
sparingly
testwise students will use process of
elimination to select answer
do NOT use to pad out the distracters
because you cannot think of another
one.

Advantages of Multiple Choice Items

allow more adequate sampling of content.
tend to more effectively structure the problem
to be addressed
items can be more efficiently and reliably
scored than supply items
different response alternatives can provide
diagnostic feedback (item analysis)
items can be constructed to address various
levels of cognitive complexity

Disadvantages of Mult.Choice Items

difficult & time consuming to construct good
items
leads to emphasis on other selected response item types

can lead the instructor to favour simple recall

of facts
high degree of dependence on students
reading and instructors writing ability
can be difficult to achieve clarity of expression

measuring synthesis and evaluation can be

difficult
inappropriate for measuring outcomes that
require skilled performance

Wrap-Up