DISTRACTOR ANALYSIS

17/12/2013

2 Comments

Distractor analysis is an extension of item analysis, using techniques that

are similar to item difficulty and item discrimination. In distractor analysis,
however, we are no longer interested in how test takers select the correct
answer, but how the distracters were able to function effectively by drawing
the test takers away from the correct answer. The number of times each
distractor is selected is noted in order to determine the effectiveness of the
distractor. We would expect that the distractor is selected by enough
candidates for it to be a viable distractor. What exactly is an acceptable
value? This depends to a large extent on the difficulty of the item itself and
what we consider to be an acceptable item difficulty value for test times. If we
are to assume that 0.7 is an appropriate item difficulty value, then we should
expect that the remaining 0.3 be about evenly distributed among the
distractors. Let us take the following test item as an example:

In the story, he was unhappy

because…………

A. it rained all day

B. he was scolded
C. he hurt himself
D. the weather was hot
Let us assume that 100 students took the test. If we assume that A is the
answer and the item difficulty is 0.7, then 70 students answered correctly.
What about the remaining 30 students and the effectiveness of the three
distractors? If all 30 selected D, the distractors B and C are useless in their
role as distractors. Similarly, if 15 students selected D and another 15
selected B, then C is not an effective distractor and should be replaced.
The ideal situation would be for each of the three distractors to be selected by
10 students. Therefore, for an item which has an item difficulty of 0.7, the ideal
effectiveness of each distractor can be quantified as 10/100 or 0.1. What
would be the ideal value for distractors in a four option multiple choice item
when the item difficulty of the item is 0.4? Hint: You need to identify the
proportion of students who did not select the correct option.

From a different perspective, the item discrimination formula can also be used
in distractor analysis. The concept of upper groups and lower groups would
still remain, but the analysis and expectation would differ slightly from the
regular item discrimination that we have looked at earlier. Instead of expecting
a positive value, we should logically expect a negative value as more students
from the lower group should select distracters. Each distractor can have its
own item discrimination value in order to analyse how the distracters work and
ultimately refine the effectiveness of the test item itself. If we use the above
item as an example, the item discrimination concept can be used to assess
the effectiveness of each distractor. If a class has 100 students, we can form
upper and lower groups of 30 students each. Assume the following are
observed:

The values in the last column of the table can once again be interpreted
according to how we examined item discrimination values, but with a twist.
Alternative A is the key and a positive value is the value that we would want.
However, the value of 0.33 is rather low considering the maximum value is 1.
The value for distractor B is 0 and this tells us that the distractor did not
discriminate between the proficient students in the upper group and the
weaker students in the lower group. Hence, the effectiveness of this distractor
is questionable. Distractor C, on the other hand, seems to have functioned
effectively. More students in the lower group than in the upper group selected
this distractor. As our intention in distractor analysis is to identify distractors
that would seem to be the correct answer to weaker students, then distractor
C seems to have done its job. The same cannot be said of the final distractor.
In fact, the positive value obtained here indicates that more of the proficient
students selected this distractor. We should understand by now that this is not
what we would hope for.

Distractor analysis can be a useful tool in evaluating the effectiveness of our

distractors. It is important for us to be mindful of the distractors that we use in
a multiple choice format test as when distractors are not effective, they are
virtually useless. As a result, there is a greater possibility that students will be
able to select the correct answer by guessing as the options have been
reduced.
The values in the last column of the table can once again be interpreted
according to how we examined item discrimination values, but with a twist.
Alternative A is the key and a positive value is the value that we would want.
However, the value of 0.33 is rather low considering the maximum value is 1.
The value for distractor B is 0 and this tells us that the distractor did not
discriminate between the proficient students in the upper group and the
weaker students in the lower group. Hence, the effectiveness of this distractor
is questionable. Distractor C, on the other hand, seems to have functioned
effectively. More students in the lower group than in the upper group selected
this distractor. As our intention in distractor analysis is to identify distractors
that would seem to be the correct answer to weaker students, then distractor
C seems to have done its job. The same cannot be said of the final distractor.
In fact, the positive value obtained here indicates that more of the proficient
students selected this distractor. We should understand by now that this is not
what we would hope for.

Distractor analysis can be a useful tool in evaluating the effectiveness of our

distractors. It is important for us to be mindful of the distractors that we use in
a multiple choice format test as when distractors are not effective, they are
virtually useless. As a result, there is a greater possibility that students will be
able to select the correct answer by guessing as the options have been
reduced.