Item Analysis

BY: DEBBIE E. BANDAHALA

 Item Analysis
-  is an important (probably the
most important) tool to increase test
effectiveness. Each items contribution is
analyzed and assessed. To write
effective items, it is necessary to
examine whether they are measuring
the fact, idea, or concept for which they
were intended
 Item Analysis
It is a technique that will enable us to
assess the quality or utility of an item.
It does so by identifying distractors or
response options that are not doing
what they are supposed to be doing.
It is ideally suited for examining the
usefulness of multiple choice formats.
Consider the following Questions and
Answers on item analysis:
Question: Why do we administer tests?
Answer: To find out who has mastered
the material and who has not (i.e. to
discriminate between the groups)
Question: Will a test discriminate
between the groups better if each item
on the test discriminate between those
who did not do well on the test overall?
Answer: Absolutely! If more students
who do well on the test overall answer
an item correctly- positive
discrimination index-that item helps the
overall discrimination ability of the test.
If this is true for all items (they are all
positively discriminating), the test will
do a good job of discriminating between
those who know their stuff and those
who don’t. To the extent that students
who do poorly on the overall test
answer individual items correctly-
negative discrimination index-the test
loses its ability to discriminate.
Question: How can I tell whether the
key for any question is chosen more
frequently by the better students (i.e.,
the item is positively discriminating) or
the poorer students (i.e., the item is
negatively discriminating)?
Answer: Follow the procedure described
on the next slide.
 Discrimination Index
Discrimination Index- measure of the extent
to which a test item discriminates or
differentiates between students who do well
on the overall test and those who do not do
well on the overall test.
There are three types of discrimination index:
1. Positive discrimination index-those
who did well on the overall test chose the
correct answer for a particular item more
often than those who did poorly on the
overall test.
2. Negative discrimination index-
those who did poorly on the overall test
chose the correct answer for a
particular item more often than those
who did well on the overall test.
3. Zero discrimination index- those
who did well and those did poorly on
the overall test chose the correct
answer for a particular item with equal
frequency.
To determine each item’s discrimination index
(D), complete the following steps:
1. Arrange the papers from highest to
lowest score.
2. Separate the papers into an upper
group and a lower group based on total
test scores. Do so by including half of
your papers in each group.
3. For each item, count the number in
the upper group and the number in the
lower group that chose each
alternative.
4. Record your information for each
item in the following form. The
following data is the example; the
asterisk indicates the keyed option):
Example for item X (Class size = 30)
Options A B C* D
Upper 1 0 11 3
Lower 2 0 7 6

5. Compute D, the discrimination index,

by plugging the appropriate numbers
into the following formula:
 (Number who got item correct in upper group) – (Number who got item correct in lower group)
D=
Number of students in either group (if group sizes are unequal, choose the higher number)

Plugging in our numbers we arrive at

D = 11-7 = 0.267
15
Interpretation:
Our discrimination index (D) is 0.267, which is positive. More students who did well on the
overall test answered the item correctly than students who did poorly on the overall test.

6. Compute P, the difficulty index, by

plugging the appropriate numbers into the
following formula:
 (Number who got item correct in upper group) + (Number who got item correct in lower group)
P=
Total Number of students taking the test

Plugging in our numbers we arrive at

P = 11+7 = 0.60
30

Interpretation:
An item with p = 0.60 and D = 0.267 would be considered a moderately difficult item
that has positive (desirable) discrimination ability. When p levels are less than about
0.20, the item is considered relatively difficult. When p levels are above 0.80, the item is
considered relatively easy. We noted earlier that B option need to be replaced, since no
one chose it. But what about A and D? are they acceptable? The answer is yes, but can
you figure out why? Because more student in the lower group chose them than did
people in the upper group.

Note: Test construction experts try to build tests that have most items between p levels of
0.20 and 0.80, with an average p level of about 0.50. Some experts insist that D should
be at least 0.30, while others believe that as long as D has positive value, the item’s
discrimination ability is adequate.
 Another Item sample:
Example for item Y (Class size = 28)
Options A* B C D
Upper 4 1 5 4
Lower 1 7 3 3
D = .214 P = .178 or 0.18
Question: Should this item be eliminated?
Answer: No. since it is positively discriminating. However, it is difficult item: only
18% of the class got it right.
Question: Should any distractors be eliminated or modified?
Answer:Yes; distractors C and D have attracted more students who did well on
the test overall. If these distractors are modified or replaced, a good item will
be made even better. Remember, in order for an item to discriminate well,
more students who do well on the test choose the correct answer than
students who do poorly(the correct answer should be positively
discriminating) and fewer students who do well on the test should choose
each distrators than students who do poorly(the distractors should be
negatively discriminating).
 Another Item sample:
Example for item Z (Class size = 30)
Options A B* C D
Upper 3 4 3 5
Lower 0 10 2 3
P = 0.467 D = -0.40
Question: Should this item be eliminated?
Answer: Yes! The item is moderately difficult, approximately 47% of the class got
it right , but it discriminates negatively. Remember, one of the reasons for
testing is to discriminate between those students who know their stuff and
those who do not. On this item more students who knew their stuff (who did
well on the overall test) chose the incorrect options than the correct answer.
Question: Should any distractors be eliminated or modified?
Answer; Since we have already decided to eliminate the item, this is a moot
question.