Professional Documents
Culture Documents
Psychological Testing and Assessment 9th Edition Cohen Solutions Manual instant download all chapter
Psychological Testing and Assessment 9th Edition Cohen Solutions Manual instant download all chapter
Psychological Testing and Assessment 9th Edition Cohen Solutions Manual instant download all chapter
https://testbankdeal.com/product/psychological-testing-and-
assessment-9th-edition-cohen-test-bank/
https://testbankdeal.com/product/psychological-testing-and-
assessment-an-introduction-to-tests-and-measurement-8th-edition-
cohen-solutions-manual/
https://testbankdeal.com/product/psychological-testing-and-
assessment-an-introduction-to-tests-and-measurement-8th-edition-
cohen-test-bank/
https://testbankdeal.com/product/psychological-testing-
principles-and-applications-6th-edition-murphy-test-bank/
Psychological Testing History Principles and
Applications 7th Edition Gregory Test Bank
https://testbankdeal.com/product/psychological-testing-history-
principles-and-applications-7th-edition-gregory-test-bank/
https://testbankdeal.com/product/psychological-testing-
principles-applications-and-issues-8th-edition-kaplan-test-bank/
https://testbankdeal.com/product/mastering-modern-psychological-
testing-theory-and-methods-1st-edition-reynolds-test-bank/
https://testbankdeal.com/product/psychological-assessment-and-
theory-tests-8th-edition-kaplan-test-bank/
https://testbankdeal.com/product/foundations-of-psychological-
testing-a-practical-approach-5th-edition-miller-test-bank/
1
Chapter 7: Utility
Chapter 7
Utility
Chapter Outline
I. What is Utility?
A. Factors That Affect a Test’s Utility
Psychometric soundness
Costs
Benefits
Self-Assessment
Term to Learn
Utility: In the language of psychometrics, utility (also referred to as test utility) refers to how
useful a test is.
Ehreke, L., Luck, T., Luppa, M., et al. (2011). Clock drawing test: Screening utility for mild
cognitive impairment according to different scoring systems. International Psychogeriatrics,
Vol 23(10), 1592–1601
O'Meara, A., Davies, J., & Hammond, S. (2011). The psychometric properties and utility of the
Short Sadistic Impulse Scale (SSIS). Psychological Assessment, Vol 23(2), 523–531.
Reel, K. H., Lecavalier, L., Butter, E., & Mulick, J. A. (2012). Diagnostic utility of the Pervasive
Developmental Disorder Behavior Inventory. Research in Autism Spectrum Disorders, Vol
6(1), 458–465.
Is the concept of utility as it relates to tools of assessment absolute or relative? Can the same test
be considered of great utility in one situation but of little or no utility in another situation?
Here is a list of questions that may be used to stimulate class discussion, as well as critical and
generative thinking, with regard to some of the material presented in this chapter.
1. Introduce the topic of test utility to the class by stimulating a discussion of the costs
(monetary and otherwise) and benefits (monetary and otherwise) of various things familiar
to students. What are the costs and benefits (in the broadest sense), for example, of the
following situations?
a. Owning a credit card
b. Obtaining an academic degree
c. Serving in the military
d. Owning an automobile
e. Investing in a business
2. A valid test is said to be psychometrically sound for a particular purpose, and a useful test
has utility for a specific purpose. To amplify this point, present to the class a scenario
wherein a particular test of cognitive ability has been found to have great utility in
selecting members of a high school debate team. Now, pose the question of how much
utility the same test might have in various other selection situations. For each situation,
students should respond with one of the following opinions: “yes”, “no”, or “maybe.”
Will this test of cognitive ability be useful or not in selecting applicants for the groups
listed below?
a. Law school
b. Art school
c. A police hostage negotiation unit
d. A middle school gifted program
e. Executive level positions in a labor union
f. Actors in a theme park who spend their work day dressed in a character costume
3. As noted in the chapter, it is seldom possible to have the lowest false positive rate, the
lowest false negative rate, and the highest hit rate, without incurring any costs of testing.
The coexistence of the lowest selection costs, the highest hit rate, and the lowest miss rate
is usually not in the cards. In many instances, test users must make a judgment, for
example, about the respective desirability of different types of “misses.” Is a possible false
negative preferable over a false positive? Is a possible false positive preferable over a false
negative?
In each of the scenarios described below, students should assume that the test mentioned is
valid for predicting success on the criterion of interest. The students’ task is to express an
opinion about the relative desirability of a false negative and a false positive and then
explain the reasoning that led to that opinion. So, for each of the situations given, students
respond to the question, “Is a false negative preferable over a false positive?” The
response should also address the question of whether a false positive is preferable over a
false negative.
In-Class Demonstrations
Bring (or assign students to bring) to class newspaper articles, scholarly articles, trade
journals, news clips, or web references that discuss the utility of using a particular test or
intervention. For example, on February 10, 2008, the website http://www.msnbc.com
featured a story about a new test designed to detect 10 genes in men who have the highest
risk for developing prostate cancer. The article included a statement by a corporate chief
executive that “this is a test with significant clinical utility for improving and personalizing
the screening and treatment of one of the most common cancers.” Have students discuss
the different aspects of this test’s utility.
Invite a guest speaker to class. The guest speaker could be one of the following:
Invite a local consumer of psychological test data from a corporate organization who
can discuss test utility in terms of organizational decision making.
Invite a local expert on test utility from academia who can provide the class with
illustrations of how considerations of test utility have had “real-life” consequences.
1. Role-play exercises
a. Students play the role of subject matter experts to set cut scores.
receive credit for a given course when they transfer from another university. Such
decisions are usually made based on judgments relating to how similar one course is
to the other. With this as a background, take the following steps: (1) Stimulate a class
dialogue regarding the basic facts about utility a student transferring into your tests
and measurement class would have to know in order to be given credit for knowing
the utility chapter. (2) Have students create a 10-item test to measure knowledge of
the chapter. (3) Have students set a cut score to indicate which score on the test will
be deemed satisfactory (in this latter task, students are divided into separate groups
and advised to employ the Angoff method to develop a cut score). (4) Compare the
various cut scores that the groups of students have independently set. How similar
are they? What is the lowest cut score? What is the highest cut score? The members
of the various groups can then debate the issue until they reach some sort of
consensus.
b. Students play the role of consultants advocating for the use of a test.
Have students or teams of students identify a tool of assessment that could be used to
select individuals for jobs, select individuals for educational programs, diagnose
individuals with a particular disorder, or treat individuals with a particular disorder.
The Buros Institute of Mental Measurements’ Mental Measurements Yearbook or
Tests in Print are good sources to find tests. Next, students play the role of a
consultant who will make a presentation to sell assessment services to a corporate
client. The students should do the following tasks before the role play:
• Establish a specific context for which the testing, assessment, or intervention
would be done (e.g., selecting computer programmers for a software
development company).
• Research and estimate a realistic base rate and selection ratio for their
particular context.
• Prepare a presentation that includes recommendations for the cost-effective use
of the test for selection purposes.
• Include recommendations whether the test should be used alone or in
conjunction with other tests.
o These recommendations should also include whether a compensatory,
multiple hurdle, or some kind of hybrid strategy should be employed.
• Finally, students should provide estimates of utility including projected false
positive rates, false negative rates, and return on investment.
Identify a test that could be used to select individuals for a particular position (job or
educational position). For this particular test, the plaintiff claims that evidence
regarding the validity of the test scores for making the selection decision is scanty
and generally low (0.25). The defendant claims that even scores that demonstrate low
validity coefficients can have utility and that procedures such as validity
generalization are applicable to the current situation. One-third of the class will play
the role of the plaintiff in the dispute, one-third of the class will play the role of the
defendant in the dispute, and the remaining one-third of the class will play the role of
the judge and jury. Give the students the details (e.g., test name, type of position for
which the selection is being made) in advance so that all students can thoroughly
research the issues and the concepts involved with regard to the dispute. A trial will
be held in class, focusing on the concepts involved. Students playing the role of the
plaintiff and those acting as defendant may elect one representative to present their
evidence (e.g., experts, research results obtained from reputable sources, etc.). The
judge and jury members will be responsible for summarizing research results and
doing any research that is necessary after the case is presented. They may deliberate
in their decision and render a judgment, to be declared at the beginning of the next
class.
A number of references are cited in the text to support the argument that the top-down
selection policies can carry with them consequences of adverse impact (Cascio et al., 1995;
DeCorte & Lievens, 2005; McKinney & Collins, 1991; Zedeck et al., 1996). On the other
side of the coin, the top-down selection policies may also carry with them real benefits to
the organization that uses them for selection. The task here is to prepare students for a
debate on the pros and cons of top-down selection and critically examine the adverse
impact argument. The class can be divided into two groups with one group as the “pro top-
down,” and the other one as “con top-down.” Specific examples of situations wherein a
top-down selection policy would or would not be desirable should be cited.
a. A corporate HR department
Visit the human resources department of a local or large organization that uses
psychological tests and other assessment instruments for employee selection,
b. A university office
Visit the office at a local university or any other educational institution that uses
testing to help make selection or advancement-related decisions for students. Arrange
a class session with a representative from the institution to elaborate on how tests and
assessment methods help the institution make better decisions.
Suggested Assignments
The “Close-up” section in this chapter provides an example of the case when the cost of
testing applicants is substantially outweighed by the benefits of the testing. For class
discussion, have students generate a list of hypothetical situations in which the use of test
scores would have negative utility; that is, the cost of the testing would far outweigh the
benefits of using the test scores to make decision.
3. Read-then-Discuss Exercises
Have each student review the daily newspaper, watch the news, or review news-
related websites looking for news articles that refer to cut scores or utility analysis.
Students should bring in the articles or write a short summary of the news stories and
discuss them during the next class session. Ideally, student presentations should
mention whether any of these approaches—cut scores, multiple hurdle, or top-down
selection—were presented. Students should also opine on whether or not the most
appropriate methods were employed. If not, how might the estimated utility of the
measure change with a change in method?
Assign an article by Phelps (1999), which can be found on the following website:
http://www.siop.org/tip/backissues/Tipapr99/4Phelps.aspx. After everyone has read
it, conduct a class discussion on the pros and cons of using a standardized test for
employment selection decisions.
4. Research-then-Report Exercises
Many tests available to consumers are used to make dichotomous decisions (e.g.,
pregnancy tests, drug tests, nicotine tests, tests for urinary tract infections, etc.).
Students should research and report on the utility-related aspects of any one of these
tests. The report should include, for example, the false positive and false negative
rates for the test, as well as a discussion of the implications of these types of errors
for the consumer.
Students should select a utility-related topic from the scholarly literature and write a
report on it. The following are some sample topics:
Rather than bringing test users or developers into class, instructors can assign
students to interview test users or developers. The interview should address the
following questions: How does one determine the utility of the use of scores on a test
or the use of a particular intervention? How is the utility of test scores estimated to
make different decisions? How does one determine the appropriate cut scores to use?
How does one estimate and minimize false positive and false negative rates?
Media Resources
On the Web
A noncomprehensive sampling of some of the material available on the World Wide Web.
2. Online tutorials
For an online tutorial by David Lane that discusses the “usefulness” of adding a test or
other predictor in multiple regression (i.e., the incremental validity of the predictor), go to
the following website:
http://davidmlane.com/hyperstat/prediction.html.
An online clinical decision-making calculator (by Rob Hamm, Ph.D.) that calculates
sensitivity, specificity, false positive, false negative, and other factors related to utility can
be found on the following website:
http://www.fammed.ouhsc.edu/robhamm/cdmcalc.htm.
An online tutorial entitled “Utility and Decision Making” on the Web Interface for
Statistics Education (WISE) is made available by Claremont Graduate University. To use
this tutorial, your browser must be enabled with Java and Java Script. Access WISE
tutorials at CGU’s website: http://wise.cgu.edu.
To gain more insights on employee selection, visit the websites listed below.
• www.siop.org
• www.shrm.org
To gain more insights on educational testing, visit the website given below.
• www.ets.org
To gain more insights on the use and abuse of statistics in the media, visit the website
given below.
• www.stats.org
References
Cascio, W. F., Outtz, J., Zedeck, S., & Goldstein, I. L. (1995). Statistical implications of six
methods of test score use in personnel selection. Human Performance, Vol. 8(3), 133–164.
Daw, J. (2001). Psychological assessments shown to be as valid as medical tests. APA Monitor,
Vol. 32(7), 46–47.
De Corte, W., & Lievens, F. (2005). The risk of adverse impact in selections based on a test with
known effect size. Educational and Psychological Measurement, Vol. 65(5), 643–664.
Fisher, C. A. H., & Larner, A. J. (2007). Frequency and diagnostic utilityof cognitive test
instrument use by GPs prior to memory clinic referral. Family Practice, Vol. 24(5), 495–
497.
Jæger, M. M. (2007). Economic and social returns to educational choices: Extending the utility
function. Rationality and Society, Vol. 19(4), 451–483.
Kazmierski, J., Kowman, M., Banach, M., et al. (2008). Clinical utility and use of DSM-IV and
ICD-10 criteria and the Memorial Delirium Assessment Scale in establishing a diagnosis of
delirium after cardiac surgery. Psychosomatics: Journal of Consultation Liaison Psychiatry,
Vol. 49(1), 73–76.
Kingston, D.A., Firestone, P., Moulden, H.M., & Bradford, J. M. (2007). The utility of the
diagnosis of pedophilia: A comparison of various classification procedures. Archives of
Sexual Behavior, Vol. 36(3), 423–436.
Kooij, J., Boonstra, A. M., Swinkels, S. H., et al. (2008). Reliability, validity, and utility of
instruments for self-report and informant report concerning symptoms of ADHD in adult
patients. Journal of Attention Disorders, Vol. 11(4), 445–458.
McKinney, W. R., & Collins, J. R. (1991). The impact on utility, race, and gender using three
standard methods of scoring selection examinations. Public Personnel Management, Vol.
20(2), 145–169.
Meyer, G. J., Finn, S. E., Eyde, L D., Kay, G. G., Moreland, K. L., Dies, R. R., Eisman, El J.,
Kubiszyn, T. W., & Reed, G. M. (2001). Psychological testing and psychological
assessment: A review of evidence and issues. American Psychologist, Vol. 56(2), 128–165.
Phelps, R. P. (1999). Education Establishment Bias? A Look at the National Research Council's
Critique of Test Utility Studies. The Industrial-Organizational Psychologist (TIP), Vol.
36(4), 37–49.
Smith, T. Gildeh, N., & Holmes, C. (2007). The Montreal Cognitive Assessment: Validity and
utility in a memory clinic setting. The Canadian Journal of Psychiatry/La Revue canadienne
de psychiatrie, Vol. 52(5), 329–332.
Tomanik, S. S., Pearson, D. A., Loveland, K. A., et al. (2007). Improving the reliability of autism
diagnoses: Examining the utility of adaptive behavior. Journal of Autism and Developmental
Disorders, Vol. 37(5), 921–928.
Zamboanga, B. L., Horton, N. J., Tyler, K. M. B., et al. (2007). The utility of the AUDIT in
screening for drinking game involvement among female college students. Journal of
Adolescent Health, Vol. 40(4), 359–361.
Zedeck, S., Cascio, W. F.,Goldstein, I.L., & Outtz, J. (1996). Sliding bands: An alternative to
top-down selection. In R. S. Barrett (Ed.), Fair employment strategies in human resource
management (pp. 222–234). Westport, CT: Quorum Books/Greenwood Publishing.
Title: Henkivakuutusherroja
Romaani
Language: Finnish
Romaani
Kirj.
VEIKKO KORHONEN
— Ai perhana!
— Ka, sinä lempoko siinä. Kun kulkee kuin lehmä kadulla, alkoi
Varsala purkaa sisuaan.
— Älä.
— Vai pääsi poika ylenemään. No, nythän sinä saat tuntea, miten
ihanaa on olla henkivakuutusherrana. Useimmiten toiset tekevät
työn, ja me saamme niittää sadon, ainakin mitä tantiemiin tulee.
Meidän on tosin pidettävä huoli kilpailusta ja siitä, että asiamiehet,
joille varsinainen työnteko kuuluu, eivät pääse laiskottelemaan,
mutta sehän tehtävä onkin kaikkein helpoimpia. Ja ajatteles, ethän
sinä enemmän kuin minäkään, tullessani Leimausyhtiön
palvelukseen ole suorittanut mitään tutkintoja tällä alalla, ainoastaan
ovelasti päättänyt muutamia tukkikauppoja maalaisten kanssa. Niistä
minut napattiin tähän toimeen. Huomasivat kai, että minussa on
liikeneroa ja että osaan olla ovela. Ja hyvin minä olen
menestynytkin, osaan herättää ihmisissä, varsinkin maalaisissa
luottamusta. Sen vaikuttaa ulkoasuni. Minä näes en komeile
vaatteilla. Olen sattunut usein toisten yhtiöitten samaan luokkaan
kuuluvien virkamiesten kanssa samaan aikaan hankinnalle ja tullut
huomaamaan, että kovin koreaa herrasmiestä katsellaan kansan
seassa hiukan sekavin tuntein. Minulla on siitä tuoreita esimerkkejä.
Viime viikolla yövyin erääseen taloon, jossa hieroin kymmenen
tuhannen vakuutusta, jota oli samana päivänä kärttänyt sinullekin
hyvin tunnettu Rientoyhtiön Keikaus. Kuulin miten isäntä arveli
emännälleen: — Tuo näyttää oikealta mieheltä! Se Keikaus oli niin
hieno mies ja isovatsainen. Taitaa syödä vakuutettaviensa varoja.
Mitäs, jos ottaisimmekin vakuutuksen Leimaus yhtiössä. Ja aamulla
se vakuutus päätettiin.
— Jospa se lienee niin kuin sanot. Mutta mitä muuta sanoisit vielä
ihmisiä piinatessa tarvittavan?
Varsala naurahti.
— Häh?
— Häh?
— Nousenhan minä.
— Ei ole.
— Kyllä.