STAFFING ANALYTICS

Matthew Bidwell

WHARTON ONLINE
The Staffing Cycle

Basic facts about stang processes

The value of analysis
A7ri8on Hiring
Possible analy8c approaches

Internal Mobility
and Career
Development

WHARTON ONLINE
Hiring: Predicting Performance with Incomplete Data

A7ri8on Hiring

Internal Mobility
and Career
Development

WHARTON ONLINE
Selection: A Question
Which of the following methods of evaluating job candidates is
most effective at predicting subsequent performance?
Which is least effective?

1. Job knowledge tests

2. Cognitive ability tests
3. Personality tests
4. Reference checks
5. Structured interviews
6. Unstructured interviews
7. Work samples
8. Integrity tests
WHARTON ONLINE Source: Ryan and Tippins 2004
Getting Selection Right

Correla,on with subsequent performance (0-1)

Work Samples 0.54

Cogni8ve Ability Tests 0.51
Structured Interviews 0.51
Job Knowledge Tests 0.48
Integrity Tests 0.41
Unstructured interviews 0.31
Personality test (conscien8ousness) 0.31
Reference Checks 0.26

WHARTON ONLINE Source: Ryan & Tippins (2004)

Fine-tuning your selection

Performance Predictors
Performance Evalua8ons Background/experience
Objec8ve Performance Metrics Test scores
- Sales Interview performance
- Produc8vity - Specic ques8ons
- Customer Sa8sfac8on - Specic interviewers
A7ri8on
Rate of promo8on

WHARTON ONLINE
Predicting New Hire Performance: Hints, Tips, Issues
1. Comparing Apples with Apples
The work
The location
The manager / unit
The level
Time in the Job
2. Disentangling Influences
Beware spurious correlations
Apply common-sense / understand the mechanism
3. Accounting for Selection
Who got hired
Who stayed
WHARTON ONLINE
Approaches to Predicting Hire Performance
Best:
Use mul8variate regression to separate out
inuences of dierent characteris8cs
Apply selec8on correc8on to account for who
was hired and a7ri8on from sample
Good:
Use mul8variate regression to separate out
inuences of dierent characteris8cs Y = 3.2 + 0.2X1+.5X2+ 1.8 X3
Y = 3.2 + 0.2X
BeDer:
Compare characteris8cs of best and worst Y
performers within same cohort and job

Okay:
Compare characteris8cs of best and worst
performers
Test for sta8s8cal signicance
X
WHARTON ONLINE
How Does Data Analysis Compare to Human Judgment?

The Bad News

Combination of various tests and selection methods leaves much
of performance unexplained

The Worse News

Implementation of algorithms reduced turnover in call centers
Turnover was lower the less often managers over-ruled the
algorithm

WHARTON ONLINE Ryan and Tippins, 2004; Homan, Kahn, Li, 2015
Bringing Data to Internal Mobility: Beyond the Peter
Principle

A7ri8on Hiring

Internal Mobility
and Career
Development

WHARTON ONLINE
The Peter Principle

In time, every post is occupied by an employee who is incompetent

to carry out its duties
Peter and Hull, 1969
OR:
How well does success in the current job predict performance in a
higher level job?
OR:
What does predict success in higher level jobs?

WHARTON ONLINE
Analyzing Promotability

Requirements Poten,al
Mul8ple dimensional Which dimensions of lower
performance indicators level performance best predict
- Output measures performance in the higher level
- Competence job?
- Assessments

WHARTON ONLINE
Evaluating Staffing Options

Which routes lead to be7er

performance?
What is the eect on cost?

WHARTON ONLINE
A Practical Example

Personnel data from large Focus on effect of how

investment banking division workers entered their current
job (hired versus promoted)
Wide variety of functions
Use very detailed job controls
Annual snapshots of
to compare workers entering
employees from 2003-2009
similar jobs by different routes
Performance Evaluations
Compensation Study only jobs that
Job Can be entered by
promotion
I observe being filled

WHARTON ONLINE Bidwell, 2011

Does it matter how people enter jobs?
Performance Pay
Hires performed substantially worse New hires receive 18% more
than similar promotes compensation than promotes
75% less likely to get top rating
Pay gap only closes very slowly
270% more likely to get lowest
(up to 7 years)
rating
Takes 3 years to acquire similar
performance to those promoted into
the job

WHARTON ONLINE Bidwell, 2011

Comparing Hiring Inside Firms

Internal Pos8ng Sponsorship

Internal market Social network

Manager posts job & Manager iden8es

invites interested candidates through her
candidates to apply Vs. personal network
Creates compe88on for Appoints preferred
jobs within the rm candidate to the job

WHARTON ONLINE Keller, 2015

Comparing Hiring Inside Firms

Creates unconven8onal
career paths
Internal Pos8ng Leads to higher
Internal market performance ra8ngs:
Larger pool of
Manager posts job & candidates
invites interested Disciplines decision-
candidates to apply making
Creates compe88on for
Associated with higher
jobs within the rm salaries (3% - 6%)

WHARTON ONLINE Keller, 2015

 ISSUES OF CAUSALITY

Matthew Bidwell

WHARTON ONLINE
Why We Care About Causality
People who enter jobs through Should we avoid pos8ng?
formal pos8ng perform worse

People who have been in the job Should we move people around
longest have lower performance more?

People who have taken a training Should we send more people to

program perform be7er training?

People who have taken a training Should we send more people to

program show greater training?
performance improvements
WHARTON ONLINE
Two Types of Causality Problems

Omitted Variable Bias Reverse Causality

O
X Y
X Y
Do we only post when jobs are hardest to Are our highest performing people gelng
ll? promoted out of the job leaving middle
performers?
Do people only get trained following dips in
their prior performance? Are our highest performing people being
trained?

WHARTON ONLINE
The Central, Underlying Question

What is leading to difference in our main

predictor variable?

WHARTON ONLINE
Approaches to Addressing Causality

Conduct an experiment
Effective-
ness Exploit natural sources of
randomization

Look for evidence to rule

out alternatives

Measure and control for

omitted variables

Difficulty

WHARTON ONLINE
Approaches to Addressing Causality
Measure and control for omitted variables
Conduct omitted
Collect data on possible an experiment
variables
Effective- and
ness Exploit
Includenatural
in regressions
sources of
randomization
Create matched pairs with similar values
Examine within person changes to hold
Look for evidence to rule
person constant
out alternatives

Measure and control BUT:

for omitted variables Not everything can be measured

Difficulty

WHARTON ONLINE
Approaches to Addressing Causality

Conduct an experiment
Effective-
ness Exploit natural
Whatsources
would of
be some implications
randomization
of alternative explanations?
Can you find evidence for or
Look for evidence to against those explanations in the
rule out alternatives data?

Measure and control for

omitted variables

Difficulty

WHARTON ONLINE
Approaches to Addressing Causality
Natural Experiments
change your X variable in Conduct an experiment
Effective-
ways that shouldnt also
ness affect Y Exploit natural sources
Mimics assignment to of randomization
treatment vs control group in
genuine experiment Look for evidence to rule
Allows for assessmentoutofalternatives
causal effects
Measure and control for
BUT:
omitted variables
You need to be lucky

Difficulty

WHARTON ONLINE
Approaches to Addressing Causality
Randomly assign individuals/
jobs to treatment and control Conduct an experiment
Effective- groups (ensuring balanced
characteristics of each group)
ness Exploit natural sources of
Test whether results in two
randomization
groups are different
BUT:
Look for evidence to rule
You need to persuade people to
out alternatives
let you do it
Measure
Veryand control for
time-consuming
omitted variables

Difficulty

WHARTON ONLINE
Understanding and Managing Attrition
Problems
Hiring Costs
Training Costs
Loss of Critical Knowledge
ADri,on Hiring
Levers
Impact on Customer
Relationships Inform hiring strategy
Target interventions
Internal Mobility Improve conditions
and Career Address unmet needs
Development Train managers
Focus retention efforts

WHARTON ONLINE
Understanding Attrition A Simple View

People leave their jobs because there is something else that they
would rather be doing

Attractiveness of outside Satisfaction with current

opportunities job situation
Demand for skills
Industry / regional
Vs. Perception of future
opportunities/trajectory
growth
in organization
Planned career evolution

WHARTON ONLINE
Why People Move Some Basic Predictors

Inverse correla,on w turnover

Supervisor rela8onship 0.25
Job sa8sfac8on 0.22
Role conict 0.22
Promo8on opportuni8es 0.16
Stress 0.13
Co-worker sa8sfac8on 0.13
Pay 0.11

0 0.05 0.1 0.15 0.2 0.25 0.3

WHARTON ONLINE Allen, Bryant and Vardaman 2010

Process Perspectives on Turnover
Direct contact
from recruiter
Comparison Dissatisfaction
between existing with job
Decision
job and viable
to quit Decision to
alternative
search for a job Awareness of
Plan to move at attractive
certain age/
alternatives
experience

Dissatisfaction
with job

WHARTON ONLINE Adapted from Lee and Mitchell, 1994

Turnover as a Search Process
Assumptions:
1. We will enjoy and be
better at jobs that are a
better fit with our abilities
and preferences
2. We can only assess fit
once we are actually in
the job
3. If we turn out to be a
poor fit, we will quit
WHARTON ONLINE
on
Jovanovic,
1979
Implications:
1. Probability of turnover
decreases the longer people
have spent in the job
Have learned whether it is a
good fit or not
2. Rate of turnover falls as
workers get older
More likely to know what fits
and what doesnt
Based
Some Common Predictors of Turnover

Manager
Pre-hire background
Type of work / project / function
Performance evaluations
Geography
Social network behavior

WHARTON ONLINE
Approaches to Predicting Attrition

Best:
Use of survival / hazard rate models to test
which factors accelerate risk of exit

Good:
Use mul8variate regression to predict who
reaches each milestone

BeDer:
Comparison of % leaving before specic
milestones
- 3 months
- 6 months
- 1 year
Okay:
Comparisons of % a7ri8on across 8me and
across units
Test for sta8s8cal signicance

WHARTON ONLINE
The Survival Model

% of
sample
s8ll at
rm

Explanatory Variable

Time

WHARTON ONLINE