Psychologica

l Assessment
Mr. Symon Carpiso, RPm, MP
UTILITY
What is Utility?
• In everyday language, we use the term utility to refer to the usefulness
of some thing or some process
• In the language of psychometrics, utility (also referred to as test
utility) means much the same thing; it refers to how useful a test is
• Utility in the context of testing and assessment as the usefulness or
practical value of testing to improve efficiency.
UTILITY ANALYSIS
What is Utility Analysis?
• A utility analysis may be broadly defined as a family of techniques
that entail a cost–benefit analysis designed to yield information
relevant to a decision about the usefulness and/or practical value of a
tool of assessment.
How is Utility Analysis Conducted?
• Expectancy data - expectancy table can provide an indication of the
likelihood that a testtaker will score within some interval of scores on
a criterion measure—an interval that may be categorized as “passing,”
“acceptable,” or “failing.”
• Taylor-Russell tables provide an estimate of the extent to which
inclusion of a particular test in the selection system will improve
selection.
• Naylor-Shine tables entails obtaining the difference between the
means of the selected and unselected groups to derive an index of
what the test (or some other tool of assessment) is adding to already
established procedures
METHODS OF SETTING CUT SCORES
The Angoff Method
• Devised by William Angoff (1971), the Angoff method for setting
fixed cut scores can be applied to personnel selection tasks as well as
to questions regarding the presence or absence of a particular trait,
attribute, or ability.
The Known Groups Method
• Also referred to as the method of contrasting groups, the known
groups method entails collection of data on the predictor of interest
from groups known to possess, and not to possess, a trait, attribute, or
ability of interest.
Item Response Theory
• Each item is associated with a particular level of difficulty. In order to
“pass” the test, the testtaker must answer items that are deemed to be
above some minimum level of difficulty, which is determined by experts
and serves as the cut score.
• Item-mapping method, arrangement of items in a histogram, with each
column in the histogram containing items deemed to be of equivalent
value.
• Bookmark method, delivering a test to a representative sample (or
population) of examinees, and then calculating the difficulty statistics for
each item.
End of Topic 5. Congratulations!!!!