Rita C. Ramos, R.N.

Assistant Professor
University of the Philippines Open University
Faculty of Management and Development
Studies
 Medical Surgical Nursing covers greater
percentage in the total number of hours in
the nursing curriculum
 Statistics show s that there has been

increasing percentage of failing among test

takes in the Nurse Licensure Examination
( PRC 2010 )
1. To establish the validity and reliability of the
instrument using CTT and IRT framework
2. To determine the dimensionality measure of
items.
3. To compare the item difficulty and item
discrimination of the Medical Surgical
Nursing Achievement Examination using
CTT and IRT
 137 4th year nursing students form a private
university in Manila.
 Selected on the basis of the completion of all

subjects in Medical Surgical nursing

 Nursing Achievement Test for Medical
Surgical Nursing ( NAT- MSN ) is comprised
of 219 items of multiple choice
 Parallel to Nursing Practice III, IV and 5 of the

Nurse Licensure Examination

 Test III
Client in Pain
Peri- operative Care
Alterations in Human Functioning

Test IV
Alterations in Human Functioning Medical
Client in Biologic Crisis
Emergency and Disaster Nursing Surgical Nursing

Test V
Disturbances in Perception and Coordination
 Test objectives for each major categories and
its subsets were derived from course syllabi
 11 topics derived from three major categories

( Nursing Practice 3, 4 & 5 )

 Nursing subjects have both theoretical and

clinical components ( Theory and Related

Learning Experience )
 Total hours summed up to 408 hours in the 11
topics ( Theory and Related Learning Experience )
 Total item computed for each subtopic were
distributed according to five domains of New
Bloom’s Taxonomy .
 The final structure and draft of the achievement
examination was arranged according to the main
three parts : Nursing Practice III ( 100 items ),
Nursing Practice IV (100 items ) and Nursing
Practice V ( 19 items )
 Seeking of permission
 Test administration with Research assistant
 Reliability of the achievement examination
utilizing framework of Classical Test Theory
using SPSS version 11.5 ( Chicago , Illinois ).
 Item difficulty and discrimination were

computed and analyzed according to formula

 WINSTEP version 3.69 ( Linacre 2010 ) was

used to assess the following

:unidimensionality, hierarchical ordering of
items, person reliability and separation, and
item reliability and separation.
IRT
Person reliability .76

Item reliability .97

Classical Test Theory

Cronbach’s Alpha .7546

 The sample yielded a person reliability of .76

 This implies that items are working well

together to consistently reproduce a
participant’s score
 The sample produced a person separation
statistics of 1.78.
 The strata formula was used to determine the

number of distinct strata ( HP=( 4GP +

1 )/3 ).
 Thus it resulted to strata equalled to 2.70.
 The sample can be grouped and separated

into three distinct ability groups.

 The item separation was 5.71 when
computed using the stated above formula
( ( HP =( 4GP +1 )/3 ) resulted 7.94.
 Findings suggested that the test items can be

categorized into eight subgroups.

 The item reliability of the said achievement
is .97.
 The SZTD resulted to .00 ZSTD less than 0

indicated greater predictability.

 Mean SD

.282 .122
CTT difficulty

IRT difficulty 0.137854 .1209

 There were 139 items ( 63.47 % ) considered
as Average; 80 items ( 36.52 % ) difficult.
 Majority of reasonably good items, marginal

items and poor items were all from test 3.

 Resulted to the following : of the 219 items, 9
items were poor items ( 4, 10% ) , 8 items
were marginal ( 3.65 % ), 27 items ( 12.37 % )
were reasonably good items, 11 items ( 5 % )
were good items and 164 items ( 74.88 %)
were very good items.
 The nine poor items and difficult were as
follows : items # 106, 185, 92, 157, 6, 72,
117, 129 and 147.
 Reflects the matching of ability with difficult
of item.
 The mean of items was 1 logit below the

sample.
 The ability of the participants was higher than

the all the difficult items,

 There were two potential gaps ( located 106,
185, 72 and 92 );from 92 and 157;
additionally from 157 and 197 and 6 )
 The gaps are not significant considering that

is less than 2.00 logits

 Mean SD

0.564729 0.245878
CTT
discrimination

1.02386 0.310584
IRT
discrimination
 The utilization of two frameworks in testing
intensified and strengthened the stability of
the achievement examination.
 CTT and IRT yielded reliable results : .7546

and .97 respectively

 IRT has person reliability which is one of the
limitations of CTT.
 Person reliability shows the consistency

across participant’s score.

 Generated three distinct strata.
 Test items can be categorized into eight

subgroups instead of 12 subjects instead of

12 subtopics.
 The results of CTT and IRT are almost
indistinct.
 Majority of the difficult items identified in

CTT are synonymous with IRT results

 IRT can predict the probability of each

student to answer such item correctly or

incorrectly based on the logit. Hence , it
provides individual assessment instead of
group.