Lesson 6 Item Analysis and Validation

Course Outcomes:

 distinguish the uses of item analysis, validity, reliability, difficulty index, discrimination index
(PO-Cc)
 determine the validity and reliability of given test items (PO-Dd) (PO-Ec)

LESSON 6.1: Item Analysis: Difficulty Index and Discrimination Index

Intended Learning Outcomes (ILO’s)

The students will be able to…

1. explain the concepts and principles governing item analysis and validation
(CLO-K.3)
2. determine the quality of a test item by its difficulty index, discrimination
index and plausibility of options (CLO-S.2)

INTRODUCTION

The concern of this lesson is that, in order to insure that the final version of the test would
be useful and functional, the teacher normally prepares a draft of the test which is subjected to
item analysis and validation. He/She tries out the draft test to a group of students of similar
characteristics as the intended test takers (try-out phase). From the try-out group, each item
will be analyzed in terms of its ability to discriminate between those who know and those who
do not know and also its level of difficulty (item analysis phase). The item analysis will provide
information that will allow teacher to decide whether to revise or replace an item (item revision
phase). Finally, the final draft of the test is subjected to validation if the intent is to make use of
the test as a standard test for a particular unit or grading period. (Navarro, Rosita, et.al.,2019)

Preliminary Questions:
 1. Based on the pictures, explain your criteria of choosing a product.
2. Are your criteria of choosing the product important to your life?

I. CAPTIVATE

1. What can you say about the pictures?

2. Can you relate that pictures to your life as a student?

II. CONNECT

What is an item analysis?

Item analysis is a process of examining class-wide performance on individual test items.

What are the characteristics of item analysis?

Item analysis is the examination of individual items on a test, rather than the test as a
whole,
for its difficulty, appropriateness, relationship to the rest of the test, etc.
Item analysis is useful in helping test designers determine which items to keep, modify, or
discard on a given test; and how to finalize the score for a student .
If you improve the quality of the items on a test, you will improve the overall quality of the
test – hence improving both reliability and validity. (https://www.geneseo.edu/sites/default/files)
Item analysis test the effectiveness of the items by studying the student’s responses of
each item.
Uses of Item Analysis
1. Efficient class discussion of the test results.
2. Remedial work.
3. General improvement of classroom instruction.
4. Increased skills in test construction.
5. Constructing test bank.

Benefits of Item Analysis

1. It gives useful information for class discussion of the test. For instance, easy items can
be treated lightly, responses to difficult items can be explained more comprehensively,
and defective items can be pointed out to the students.
2. It gives data for helping the students to improve their learning method. For instance, if a
student commits a number of incorrect responses to several items, this reveals
misconceptions of the learning tasks which should be taken into consideration for
remedial work.
3. It gives insights and skills which lead to the construction of better test items for future
use. (Calmorin, 2011)

There are three common types of item analysis which provide teachers with three
different types of information:
1. Difficulty Index - Teachers produce a difficulty index for a test item by calculating the
proportion of students in class who got an item correct. (The name of this index is
counter-intuitive, as one actually gets a measure of how easy the item is, not the
difficulty of the item.) The larger the proportion, the more students who have learned the
content measured by the item.
Difficulty Index - It refers to the proportion of the number of students in the upper and
lower groups who answered an item correctly.

Level of Difficulty of an Item

Index Range Difficulty level
0.00 – 0.20 Very Difficult
0.21 – 0.40 Difficult
0.41 – 0.60 Average/Moderately Difficult
0.61 – 0.80 Easy
0.81 – 1.00 Very Easy

Formula:
Index of difficulty ( Di) = RU + RL
NU + NL
Where: Di = Difficulty index,
RU = number of students who got the correct answer from the upper group
RL = number of students who got the correct answer from the lower group
NU = total number of students from the upper group
NL = total number of students from the lower group
 2. Discrimination Index - The discrimination index is a basic measure of the validity of an
item. It is a measure of an item's ability to discriminate between those who scored high
on the total test and those who scored low. Though there are several steps in its
calculation, once computed, this index can be interpreted as an indication of the extent
to which overall knowledge of the content area or mastery of the skills is related to the
response on an item. Perhaps the most crucial validity standard for a test item is that
whether a student got an item correct or not is due to their level of knowledge or ability
and not due to something else such as chance or test bias.
Discrimination Index - It refers to the power of the item to discriminate the students
between those who scored high and those who scored low in the overall test. It is the
basis of measuring the validity of an item. This index can be interpreted as an indication
of the extent to which overall knowledge of the content area or mastery of the skills is
related to the response on an item.

Level of Discrimination
Index Range Interpretation Action
-1.0 – -.50 Can discriminate but item is Discard
questionable
-0.51 – 0.45 Non-discriminating Revise
0.46 – 1.0 Discriminating item Include
Formula:
Index of discrimination (ID) = ___RU – RL__
NG
Where: ID = Index of Discrimination
RU = number of students who got the correct answer from the upper group
RL = number of students who got the correct answer from the lower group
NG =number of students in either the upper group or lower group

Types of Discrimination Index

1. Positive Discrimination happens when more students in the upper group got the item
correctly that those in the lower group.
2. Negative Discrimination occurs when more students in the lower group got the item
correctly than those in the upper group.
3. Zero Discrimination happens when a number of students in the upper group and lower
group answer the test correctly are equal, hence, the test item cannot distinguish the
students who performed well in the overall test and the students whose performance are
very poor.
3. Analysis of Response Options - In addition to examining the performance of an entire
test item, teachers are often interested in examining the performance of individual
distractors (incorrect answer options) on multiple-choice items. By calculating the
proportion of students who chose each answer option, teachers can identify which
distractors are "working" and appear attractive to students who do not know the correct
answer, and which distractors are simply taking up space and not being chosen by
many students. To eliminate blind guessing which results in a correct answer purely by
chance (which hurts the validity of a test item), teachers want as many plausible
distractors as is feasible. Analyses of response options allow teachers to fine tune and
improve items they may wish to use again with future classes.
(http://www.specialconnections.ku.edu/)
Item Analysis Procedures
There are several item-analysis procedures that may be applied, but for informal
achievement tests the UL method (Upper/Lower method) is the simplest procedure. In other
words, only the upper group (U) and lower group (L) scores are considered. The middle or
average group is held in abeyance.
Example: The table will show the result of item # 20 taken by 45 students in Mathematics
test which is subject for item analysis.
Table 6.1. Record the scores of the students with their corresponding answer to item no. 20.
Students Score Answer Students Score Answer Students Scor Answer
e
1 42 D 16 69 A 31 60 D
2 35 C 17 70 C 32 88 C
3 48 D 18 55 B 33 75 D
4 69 C 19 80 C 34 46 A
5 75 A 20 96 C 35 70 A
6 80 C 21 55 D 36 54 C
7 47 B 22 98 C 37 70 C
8 35 C 23 54 C 38 85 C
9 75 C 24 70 B 39 52 B
10 89 B 25 68 A 40 65 D
11 90 C 26 44 A 41 40 A
12 58 D 27 38 D 42 36 D
13 60 D 28 40 C 43 79 C
14 45 C 29 66 D 44 89 C
15 60 A 30 77 B 45 90 C

* C – correct answer for item no. 20

The following steps outline a simple but effective procedure for item-analysis. Consider the
table data.
Step 1. Arrange the test scores from the highest to the lowest.
Table 6.2. (Continued)
Score Answer Score Answer Score Answer
98 C 70 C 54 C
96 C 70 B 54 C
90 C 70 A 52 B
90 C 70 C 48 D
89 C 69 C 47 B
89 B 69 A 46 A
88 C 68 A 45 C
85 C 66 D 44 A
80 C 65 D 42 D
80 C 60 D 40 C
79 C 60 A 40 A
77 B 60 D 38 D
75 A 58 D 36 D
75 D 55 B 35 C
75 C 55 D 35 C
 Step 2. Get one-third of the papers from the highest score and one-third from the lowest
scores. The former group is called the upper group (U) and the latter, lower group (L). Set aside
the middle group. In the foregoing example, 15 papers from the upper group and another 15
papers from the lower group should be analyzed. The scores and answers of these papers for
upper and lower group are as follows:
Upper Group Lower Group
Table 6.3. One-third of the papers from the highest score and one-third from the lowest scores.
Score Answer Score Answer
98 C 54 C
96 C 54 C
90 C 52 B
90 C 48 D
89 C 47 B
89 B 46 A
88 C 45 C
85 C 44 A
80 C 42 D
80 C 40 C
79 C 40 A
77 B 38 D
75 A 36 D
75 D 35 C
75 C 35 C

Step 3. Count the number of students in the upper and lower groups, respectively, who
choose the options.

Step 4. Record the frequency from Step 3. The frequency may also be recorded on the item
card or on a separate sheet.
These are as follows:
Table 6.4 Item 20 - number of students in the upper and lower groups, respectively, who
choose the options.

Options A B *C D

Upper (15) 1 2 11 1

Lower (15) 3 2 6 4

*Correct answer
 Step 5. Estimate the index of difficulty. In determining the index of difficulty, simply use the
following formula:

∑X
Index of difficulty = --------- x 100
N
Where: ∑X = Sum of right responses of upper (RU) and lower (RL) groups
N = Number of students both from the upper and lower groups

In the above example, there are 11 students from the upper group who got the correct
answer and 6 from the lower group. By substituting the formula, the index of difficulty of the
above example is 57%, which means the item is average or moderately difficult. To illustrate:

1) Index of difficulty ( Di) = RU + RL

NU + NL
= 11 + 6
15 + 15
= 17
30
= 0.5666
= 0.57 (Average/moderately difficult)

∑X
2) Index of difficulty = --------- x 100 Where:
N ∑X = RU + RL
= 11 + 6 x 100 N = NU + NL
30
= 17 x 100
30
= 0.5666 x 100
= 57%

Di = RU + RL
NU + NL

where: Di = Difficulty index,

RU = number of students who got the correct answer from the upper group
RL = number of students who got the correct answer from the lower group
NU = total number of students from the upper group
NL = total number of students from the lower group

Difficulty refers to the percentage of getting the right answer of each item. The smaller the
percentage , the more difficult the item is. The higher the value of the index of difficulty, the
easier the item. Hence, more students got the correct answer and more students mastered the
content measured by the item.
 Step 6. Estimate index discriminating power. In estimating the index of discriminating power,
consider the difference of the right response between the upper group (RU) and right response
from lower group (RL) divided by the number of pupils in each group (NG). The formula is as
follows:

Index of discrimination (ID) = ___RU – RL__

NG Given:
= ___11 – 6___ RU = 11
15 RL = 6
= 5 ___ NG = 15
15
= 0.3333
= 0.33 (Discriminating)

The index of discriminating power obtained is 0.33 in which Item 20 is reasonably good item
but possibly for improvement. The discriminating power of an item is not more 1.0.
Positive discrimination is obtained when more students in the upper group got the item
correctly that those in the lower group.

Another example :

1) Index of discrimination (ID) = ___RU – RL__

NG Given:
= ___15 – 0___ RU = 15
15 RL = 0
= 15 ___ NG = 15
15
= 1.0
If the discriminating power is 1.0, the item is 50 percent (50%) level of difficulty. That is
the upper 15 got the correct answer and the lower 15 did not. This explains that test makers
are advised to prepare test items of 50 percent (50%) level of difficulty in order that the
maximum discriminating power of 1.0 can be attained.

2) Index of discrimination (ID) = ___RU – RL__

NG Given:
= ___ 7 – 9___ RU = 7
15 RL = 9
= - 2 ___ NG = 15
15
= - 0.13

The above discriminating power of an item is negative 0.13 (-0.13) where in upper group
has lesser right response, 7, than the lower group, 9. This kind of item must be improved.
Negative discriminating power is obtained when more students from the lower group got
the correct answer than the upper group.
 3) Index of discrimination (ID) = ___RU – RL__
NG Given:
= ___ 9 – 9___ RU = 9
15 RL = 9
= 0 ___ NG = 15
15
= 0.0
Zero discriminating power (0.0) is obtained when the correct answer of the upper and
lower groups are the same. Item having zero (0.0) discriminating power, that is, the right
responses of the upper and lower groups have the same frequency, must be improved.

Action Table
Difficult Level Discriminating Level Action
Not Discriminating Reject
Very Difficult / Difficult Moderately Discriminating Revise
Discriminating Accept
Not Discriminating Revise
Moderately Difficult Moderately Discriminating Revise
Discriminating Accept
Not Discriminating Reject
Very Easy / Easy Moderately Discriminating Revise
Discriminating Revise

Step 7. Determine the effectiveness of each option. All options including distractors (wrong
response) must be analyzed to determine if they are effective. A good distractor attracts more
students from the lower to choose as their answer than in the upper group. Table 6.4 shows
that
Option C, the correct answer functions effectively because more students from the upper group
choose the correct answer and few from the lower; Option A is also effective because only one
pupil is attracted from the upper group as wrong response and three from the lower group;
Option B is fair because both the upper and lower groups have the same frequency of two (2);
Option D are good distractor. (Calmorin, 2011)

Measure of Attractiveness (MA)

In a multiple choice type of question it is desirable that the distractors be equally attractive
to the non – informed test takers. Since the distractor is the incorrect option, it should attract
more of the lower group than the upper group.

Measure of Attractiveness (MA) = ___AL – AU__

NU + NL
where: MA = Measure of Attractiveness
AU = number of test takers who choose the wrong option from the upper group
AL = number of test takers who choose the wrong option from the lower group
NU = total number of students from the upper group
NL = total number of students from the lower group
If MA is positive, then the option is an effective distractor;
If MA is negative, then the option is a confusing distractor
 If MA is zero, then the option is ineffective
Example: The author of “The Raven” is:
Options U L AL – AU MA
A. Edgar Allan Poe 6 2 -4 – 0.20
B. Elizabeth Browning 1 5 4 0.20
C. Robert Frost 0 2 2 0.10
D. Ernest Hemmingway 3 1 -2 – 0.10

The correct answer is A. This is not a confusing option since A is the correct answer (not
a distractor). Apart from the keyed response, the only confusing option is D (Ernest
Hemmingway is a novelist, not a poet). All the other distractors are effective. We must replace
Hemmingway by another American poet.
Ambiguous Item - This happens when more students from the upper group choose equally
an incorrect option and the keyed answer

Further reading, please click the link below.

https://www.slideshare.net/RonaldQuileste/item-analysis-discrimination-and-difficulty-index

III. COLLABORATE

Exercise # 1 on Item Analysis

Test I. True / False test. Write True if the statement is correct, False if incorrect. Write the
answer on the space provided for.
1. Item with zero discriminating power must be retained. 1. ___
2. A distractor is good if more students from the lower group got the correct answer. 2. ___
3. If 71 percent of the class got the correct answer, this means that the item is difficult. 3. ___
4. Item with negative discriminating power should be retained. 4. ___
5. Item with positive discriminating power must be discarded. 5. ___
6. Item analysis gives data to help students to improve their study habits. 6. ___
7.The maximum discriminating power of an item is 1.0. 7. ___
8. If an item has 28 percent difficulty, it is said as easy item. 8. ___
9. Item analysis gives insights and skills which lead to the construction of better test
items for future use. 9. ___
10. A zero discriminating power of an item is obtained when the upper and lower
groups have the same score of correct responses. 10.___

Exercise # 2 on Item Analysis

Consider a multiple choice type of test of which the following data were obtained from 60 students.
Item Options
A *B C D Total
1 3 13 2 2 Upper 1/3 (20)
5 9 5 1 Lower 1/3 (20)
The correct answer is B. Compute the difficulty index and index of discrimination.
 IV. CREATE

Ed 216 – Assessment in Learning 1

Name : ________________________________________ Date : _______________

Curr.Year : ____________________________ Score: _______________
I. Multiple Choice.
Direction: Read each problem carefully and determine the correct answer from the given
options. Check the box from the choices.

1. Item with positive discriminating power must be discarded.

[ ] True [ ] False
2. Item analysis gives data to help students to improve their study habits.
[ ] True [ ] False
3.The maximum discriminating power of an item is 1.0.
[ ] True [ ] False
4. A zero discriminating power of an item is obtained when the upper and lower groups have the
same score of correct responses.
[ ] True [ ] False
5. If 71 percent of the class got the correct answer, this means that the item is difficult.
[ ] True [ ] False
6. Item with negative discriminating power should be retained.
[ ] True [ ] False
7. If an item has 28 percent difficulty, it is said as easy item.
[ ] True [ ] False
8. Item analysis gives insights and skills which lead to the construction of better test items for
future use.
[ ] True [ ] False
9. Item with zero discriminating power must be retained.
[ ] True [ ] False
10. A distractor is good if more students from the lower group got the correct answer.
[ ] True [ ] False
11. What principle of test construction is violated when one places very difficult items at the
beginning; thus creating frustration among students particularly those of average ability and
below average?
[ ] All the items of particular type should be placed together in the test.
[ ] The items should be phrased so that the content rather than the form of the statements
will determine the answer.
[ ] All items should be approximately 50 percent difficulty.
[ ] The items of any particular type should be arranged in an ascending order of difficulty.

12. In his second item analysis, Mr. Gonzales found out that more from the lower group got the
test item 15 correctly. What does this mean?
[ ] The item has become more valid
[ ] The item has become more reliable
[ ] The item has a positive discriminating power
[ ] The item has a negative discriminating power
13. If the proportion passing for the upper and lower group is .90 and .30 respectively, what is
the discrimination index?
[ ] 0.40 [ ] 0.50
[ ] 0.60 [ ] 0.70
14. If a test item has a difficulty index of 0.06, how would you describe the test item?
[ ] It is very easy. [ ] It is very difficult
[ ] It is moderately difficulty. [ ] It is difficult
15. Ms. Mirasol who is a high school teacher in English conducted an item analysis of her test.
She found out that four of the items of the test obtained the following difficulty and
discrimination indices as follows:
Item Number Difficulty Index
1 0.58
2 0.92
3 0.09
4 0.93
Which of the above items should be discard in her item pool?
[ ] Item 1 [ ] Item 2
[ ] Item 3 [ ] Item 4

II. Problem Solving.

Given are the total number of students who responded correctly in a multiple choice
examination from the upper and the lower group of 60 examinees. Complete the table below
by filling in the needed/unknown information. Show your solutions in another piece of paper.

Item No. RU RL Di Interpretation ID Interpretation Action

1 9 3
2 10 10
3 8 12
4 0 3
5 6 8
6 13 2
7 15 15
8 4 8
9 7 3
10 2 11
 V. ASSIGNMENT
Below are scores of 48 students in Item 28 English test. Look for the a) index of difficulty and
b) index of discrimination. Use sheets of paper. Follow the item analysis procedures.

1. Is item 28 easy or difficult? Why?

2. Is the item be discarded or improved? Why?
3. Are all the options function effectively? Why?

Students Score Answer Students Score Answer Students Scor Answer

e
1 52 D 17 69 A 33 60 D
2 85 B 18 70 C 34 88 B
3 78 B 19 55 B 35 75 D
4 69 C 20 80 C 36 46 B
5 75 A 21 96 B 37 70 A
6 80 C 22 55 D 38 54 C
7 47 B 23 98 B 39 70 C
8 85 B 24 54 C 40 85 B
9 75 C 25 70 B 41 52 B
10 89 B 26 68 A 42 65 D
11 90 B 27 48 A 43 40 A
12 58 D 28 38 D 44 36 D
13 60 D 29 40 C 45 79 C
14 55 C 30 66 D 46 89 B
15 60 A 31 77 B 47 90 B
16 78 A 32 43 B 48 89 B

References:
Books:
Navarro, Rosita L., Santos, Rosita G., Corpuz, Brenda B. (2019). Assessment of Learning 1.
Fourth Edition. Quezon City: Lorimar Publishing, Inc.
Navarro, Rosita L., Santos, Rosita G., Corpuz, Brenda B. (2019). Assessment of Learning 1.
OBE and PPST Based. Fourth Edition. Quezon City: Lorimar Publishing, Inc.
Navarro, Rosita L., Santos, Rosita G., Corpuz, Brenda B. (2017). Assessment of Learning 1.
OBE and K12 Based. Third Edition. Quezon City: Lorimar Publishing, Inc.
De Guzman, Estefania S. and Adamos, Joel L. (2015). Assessment of Learning 1. Quezon City:
Adrian Publishing Co., Inc.
Gonzales, Jacobo O. and Nocon, Rizaldi C. (2015). Essential Statistics. 2015 Edition. Quezon
City: MaxCor Publishing House Inc.
Corpuz, Brenda B. and Salandanan, Gloria G. (2015). Principles of Teaching 2 with TLE.
Lorimar Publishing, Inc.
Navarro, Rosita L. et al. (2012). Assessment of Learning Outocmes. Second Edition. Lorimar
Publishing, Inc.
Buendicho, Flordeliza C. (2011). Assessment of Learning 1. Rex Bookstore
Santos, Rosita de-Guzman , Ph.D. (2007).Assessment of Learning 2. Lorimar Publishing ,
Inc., Quezon City.
Calmorin, Laurentina P. (2004). Educational Research Measurement & Evaluation. (3rd ed.)
National Bookstore
Padua, Roberto N. et.al. (1997). Educational Evaluation and Measurement. Katha Publishing
Coy.
Calmorin, Laurentina P. (1994). Educational Research Measurement & Evaluation. National
Bookstore
Calderon and Gonzales. (1993). Measurement and Evaluation. National Bookstore.

Website:

https://www.geneseo.edu/sites/default/files
http://www.specialconnections.ku.edu/
https://www.slideshare.net/RonaldQuileste/item-analysis-discrimination-and-difficulty-index

Prepared by: DR. LUZVIMINDA T. PATINDOL