Utility Analysis Revised: The ROI Of Better HR Selection Methods

Posted by Rene Klep

In a previous article, I have explained and advocated the use utility analysis for

estimating the financial return of learning interventions. As highlighted in this

previous article, utility analysis refers to a specific tool designed to estimate the

institutional gain or loss anticipated to a company from various HR interventions

designed to enhance the value of the workforce (Sturman, 2003). Next to

determining the effects of training interventions, utility analysis can also be used to

assess the (financial) impact of selection decisions. In this article, I want to share

another personal experience in which I used utility analysis to show the added

value (or Return on Investment) of using a more sophisticated selection device for a

specific group of employees.

Patent Attorneys at Philips

A number of years ago I was working as an HR manager for Philips Intellectual

Property & Standards. Philips has continually been investing in new technologies
for over 100 years. This has allowed the company to create a substantial portfolio

of intellectual property (IP). One of the key jobs in this department is the role of a

“Patent Attorney”. Patent Attorneys are specialists in intellectual property,

trademarks, design rights, copyright laws, and patents. They represent researchers

or companies in the patent application process, assist with patent infringement and

act as litigators to protect their client’s rights to an invention. Compared to others

roles I had been dealing with before, this role had some unique characteristics:

 Philips invested substantially in training with the aim to grow (junior)

engineers in a number of years towards the role of Patent Attorney (they

needed to qualify for a Dutch and European examination). Associated with

this was the notion that Philips had to be very selective in picking out

those candidates who were able to fulfill this role. In more psychometric

terms: the selection ratio was relatively low.

 Most patent attorneys stayed in the company for a long time (high tenure).

 The job was well-paid compared to other professionals with an academic

degree (scarcity on the market).

The Utility Analysis Equation

Why are these latter characteristics of the role important? These variables play an

important role in determining the cost and benefit of a selection procedure. These

parameters are incorporated in the utility formula (called the Brogden-Cronbach-

Gleser formula), which is depicted below.

In its basis form the formula is:

If we apply this formula for assessing the utility of selections decisions the

parameter “dt” is not used as it is about the performance improvement of an

existing population (it is used to determine the impact of training interventions like

we saw in my previous article). If we express the formula above in words, we might

conclude the following:

 The more applicants (N) selected using a particular selection procedure,

the greater the total utility

 A higher mean predictor score (Zx) will lead to a higher utility. Likewise,

a smaller selection ratio will lead to higher utility. The calculation of Zx can

be derived from the selection ratio (the number of people to be hired,

 divided by the total number of total applicants),

or 

 Selection procedures with higher validity will have greater utility than

procedure with lower validity (r)

 Jobs that have a larger SDy, either because they are more complex or

because they have more impact on organizational results, will benefit

more from the use of valid selection procedures than jobs with a lower

SDy. In the previous article, I have explained the meaning of the SDy-

parameter being the added value in dollars/euros of an employee who

performs one standard deviation above average (that is, compared to the

average performer)

 Utility will be higher, when the cost is lower.

In sum: according to the Brogden-Chronbach-Gleser model, the utility of a selection

system is the portion of the total dollar/euro value added by those selected that is

due to that particular selection system.

Applying the formula to Patent

Attorneys
When I became HR manager in Philips Intellectual Property & Standard, the

department was already using a combination of structured interviews and

psychological testing (combining intelligence and personality tests). However, some

managers were having doubts about the added value of the psychological testing in

the procedure, mainly because of the additional costs associated with this choice.

With the formula above we were able to show the added value of the psychological

testing despite the fact the cost per applicant were much higher.

So how can we make calculations?

Based on Cascio and Boudreau’s book “Investing in People” (2010), the Society for

Human Resource Management (SHRM) has developed software to make quick

calculations to compare the outcomes of different numbers of parameters in

our utility formula.

For our example of Patent Attorneys, we compared the utility of a procedure

without psychological testing (Existing, E) and a procedure with psychological

testing (Replacement, R).  In our calculation we assumed two parameters being

different:

 The validity (r) of the replacement procedure (R) is 0,4 compared to 0,2 of

the existing procedure (E). This assumption is based on psychometric

literature, which indicates higher (predictive) validities for selection

procedures combining several psychological tests.

 The cost of the replacement procedure (R) is 2000 euro per applicant, the

cost of E is 500 euro per applicant.

For sake of comparability, other parameters were kept the same (selection ratio,

tenure, SD of Job Performance and number of selected employees)

Now we can better understand why the low selection ratio, the long tenure and the

relatively high pay (affecting the SDy value) of Patent Attorneys mentioned earlier

are so important. The outcome of the formula shows that investing in a good

selection device is way more important for these roles than for many other kinds of

jobs.

If we look at the outcome below the replacement procedure (with psychological

testing) show a “per-year gain” in utility per selectee of 7,735 euro!

Of course, you can have a long debate about which figures to put in the tool but for

me that is not the essence of this method. It is not about finding or filling in the
correct number, it’s more about showing the relative differences between

alternatives based on realistic assumptions.  If you combine the findings in

literature about validity of selection devices, standard deviations of job

performance, and cost estimates of selection procedures in your own company,

you are able to make convincing business cases.