Assemble Test

There are four steps in assembling a good test.

1. Collect test questions

A good approach to collecting test questions is to create a question card or

file, on which test questions are written or recorded each time you prepare a
lesson plan or at the end of each teaching session (Worthen et al., 1999). This
would make it easier and more effective to create questions to assess the
important aspects of the instruction. A question card or file may include the
question(s), answer(s), distractors, course name, unit title, and other information.

2. Review test questions

The question cards or files also help you review and refine the questions.
Worthen et al. recommend using the following questions to review the questions.

● Is the test format appropriate for what you want to test?

● Is the intent of the question unambiguous?
● Is the question straightforward?
● Is the difficulty level of the question appropriate?
● Does the question have one "best" answer?
● Is the question free from bias?
● Is the question free from any irrelevant clue?
 The above questions focus on creating individual questions. When those
questions are combined together into a test, other problems may arise. Worthen
et al. further suggest considering the following issues.

● Is the group of questions representative of the course content?

● Do questions sufficiently cover the material taught?
● Is the group of questions free from unnecessary overlap and
duplication?
● Are all questions independent?
● Is the time required to answer the items appropriate?

3. Format test

The following are general guidelines for formatting the test (Kubiszyn &
Borich, 2003; Worthen et al., 1999).

● Questions with similar formats should be grouped together.

● Arrange questions from the easiest to the most difficult.
● Make the test as readable as possible (e.g., spacing, clear illustrations,
font size and type, keep the question stem and options on the same
page, etc.).
● Avoid predictable response patterns (e.g., TFTFTFTF, TTFFTTFF, etc.).

4. Prepare directions for test

Clear directions are essential (Worthen et al., 1999). Make sure all students
understand the test directions, especially those who have limited reading ability
or who are not a native speaker. Good test directions should explain what
students need to do, how to do it, where to write down the correct answer, how
the given information is used, etc. Consider the following guidelines in writing
directions.

● Provide a specific set of directions for each different type of question

format.
● Explain the basis for scoring (e.g., partial credit, point deduction due to
grammar errors, spelling mistakes, etc.).
● Specify the amount of time allowed on the test.

If testing is done outside class or outside the learning management system,

inform students where, when, and how to take the test. If a testing center is used,
provide necessary information to help students plan (e.g., open hours, locations,
contact information, fees, etc.).

Reflection

● How often do you review your test questions?

● How do you format your test questions?
● Are the directions for all your tests clear and concise?

Analyze and Revise Test

Test questions that follow all the guidelines discussed earlier may still have
problems. Problematic questions are not necessarily the ones that students
complain about most. A more systemic approach is needed to analyze the test
questions.
Item Analysis

An item analysis provides the systematic approach to examine the tests to

determine if individual questions function the way they were intended (Worthen
et al., 1999). The analysis may be more appropriate for multiple-choice and
true/false questions. Worthen et al. suggest the results from the item analysis be
used along with the professional judgment to analyze and revise the test, guided
by the following questions.
● What percentage of students pass or fail a specific question?
● Do some question options get selected more or less frequently than
expected?
● Are there any questions that high scorers do poorly and/or low scorers
do well?
● Are there any questions that appear to be generally misunderstood,
responded randomly, or miskeyed?

Here is how to perform the item analysis using an example from Worthen et
al. (1999, pp. 266-277).

Example: An American History test using the multiple-choice format is given to 32

students. The following are steps to conduct item analysis.

● Grade the test. If you use a system, the grading may be done
automatically.
● Rank students' performances from the highest to lowest.
● Keep ten highest scores in one group (high-scoring group). Use around
25% of the class as a reference.
● Keep ten lowest scores in another group (low-scoring group).
● Exclude the rest of the scores from further analysis.
● For each test question, count and write down the number of students in
the high-scoring group who selected each question option (see example
below).
● Do the same for the students in the low-scoring group.
Multiple-Choice Question Item Analysis
Who was the most influential in Number of high Number of low
convincing the French to assist the scorers (HS) scorers (LS)
Americans during the Revolutionary War choosing each choosing each
with England? option option
(10 students) (10 Students)
a. George Washington 0 1
b. Marquis de Lafayette 1 4
c. Benjamin Franklin (correct answer) 8 2
d. Benedict Arnold 0 0
e. Count de Rochambeau 1 3

No student from the HS group chose George Washington, but one student
from the LS group chose that option. Eight students from the HS group chose
Benjamin Franklin, but only two students from the LS group chose that option.

In this example, ten students (8 from HS and 2 from LS) choose the correct
answer. Many from the HS group got the answer correctly. Many of those in the LS
group did not select the correct answer. This is what you should expect from a
good question. If most students from the HS group missed this question, the
question may not function the way you've planned. If most students in both
groups get it correctly, the question may not distinguish HS from LS well. So, the
item analysis provides useful information to help you decide whether a revision is
needed.