Benchmarking for Organizational

Excellence in Addiction Treatment

2009 SAAS National Conference/

NIATx Summit
Paul M. Lefkovitz, Ph.D.
July 30, 2009
 The Measurement of
Organizational Performance

• Performance is measured in all

addiction treatment settings
• How helpful is performance data?
• Does data drive organizational vision?
• Does data drive organizational decision-
making?
The Limitations
of Outcomes
• A thermometer
reading would be of
no value as a
measure of your
health…
• If you didn't know
that 98.6 is the
“average”
temperature!
 The Limitations of
Outcomes in Your Organization

• Similarly, knowing that

your missed
appointment rate is
27% is of little value in
evaluating the health
of your agency…if
you don’t know how
others are performing
Data without Meaning
• We are deluged
with data without
meaning
• Data without a
context is just a
bunch of numbers
• What is the
answer?
The Role of Benchmarking
• Benchmarks
provide the vital
external context for
understanding your
outcomes
 Benchmarking Defined

• Gift and Mosel (1994) define benchmarking

as “the continual and collaborative discipline
of measuring and comparing the results of
key work processes”

Gift, R. and Mosel, D. (1994). Benchmarking in Health Care, American Hospital Association
 A Tiered Model
of Benchmarking

• Descriptive Benchmarking

• Comparative Benchmarking

• Process Benchmarking

• Each builds on the preceding type

 Descriptive Benchmarking

• Most common form of benchmarking

• Industry overviews, government reports
• Static snapshot of industry performance
• No formal or statistical comparisons
between an organization’s performance
and industry norms
• “Eyeball analysis” can be misleading
Comparative Benchmarking

• Formal comparison of an organization’s

performance against a descriptive
benchmark
• Typically reported as percentile
rankings but can be reported in graph
form also
 The Importance of Formal
Comparative Benchmarks
100

80

60

40

20

0
Raw Scores Percentile A Percentile B
Group Average 50 50 50
Organization 55 51 99
Comparative Benchmarking
in Your Daily Life

• What examples of comparative

benchmarking have you encountered in
your own day-to-day activities?
 Important
Benchmarking Concepts

• Benchmarking differs from formalized

research or laboratory science
• Benchmarking views data as a vehicle
rather than a destination
• Benchmarking generates ideas and
hypotheses---not answers
• Benchmarking is a communal activity
 Benchmarking as
an Impetus to Action

• Human nature gives benchmarked data

its power
• Raw data tells no stories
• Benchmarked data draws comparisons
• Nobody wants to “bring up the rear”
• Benchmarked data prompts action
 The Limitations
of Benchmarking

40 • Benchmarks profile
35 your organization
30 against others and
25 help identify
20 opportunities for
15 improvement
10
5
0
Improve? How?

• Benchmarks do
not give any
indication as to
how to improve
 The “How”:
Process Benchmarking
• Based on a simple premise: The methods of
top performers differ from those of others
• Process benchmarking systematically
compares tactics of top performers with
those of others
• Methods that distinguish between top
performers and others may be regarded as
potential best practices
 Process
Benchmarking Methodology
• Typically performed in workshop settings
• Guide to Participation distributed in
advance to allow participants to respond
to questions that will be posed about
methods used
• Audience polling technology used to
permit anonymous involvement and
immediate analysis of data
 Surprises Abound

• What is expected
usually does not
materialize
• Top performers do
not know why they
perform well
• Subtle factors often
prove to be powerful
 Process
Benchmarking In Action

• The identification
of “best practices”
 Internal Benchmarking
• All concepts also apply to internal benchmarking
• Descriptive, comparative or process
benchmarking can be conducted within an
organization
• Examples of internal comparative benchmarks:
– Productivity differences among staff
– Access differences among locations
– Unit cost differences among programs
– Staff satisfaction differences among leaders
 Benchmarks:
Where are They?
• Readily
accessible,
relevant and
affordable
comparative
benchmarking
data is hard to
come by
 Benchmarking for Organizational
Excellence in Addiction Treatment

• SAAS, NIATx, and Behavioral Pathway

Systems are partnering to sponsor an
addiction-specific national
benchmarking initiative
• BPS specializes in behavioral health
and human services benchmarking and
has numerous state and national
benchmarking initiatives underway
 Objectives of
Benchmarking Initiative
• Provide SAAS with information it needs to advocate
for its members
• Provide participating organizations with
individualized comparative benchmark data that can
serve as a vital context for understanding measured
outcomes
• Provide a powerful vehicle for the identification of
best practices and organizational improvement
through process benchmarking and the application
of NIATx principles
 Benchmark
Selection Process

• National interest survey conducted

• Interest
• Availability of Data
• 354 respondents
• Survey findings guided Steering
Committee decisions
 Scope of Benchmarks

• Comprehensive range of benchmarks

• Operational, clinical, organizational
climate, and financial domains of
performance
• 21 benchmark dimensions
• Approximately 150 input metrics
 Operational Benchmarks
• Initial Access
• Length of Stay/Utilization, by Level of Care
• Subsequent Access (Number of days from
intake to first treatment appt)
• Average Caseload Size of Clinician, by
Level of Care
• Outpatient Productivity
• Average Group Size
 Clinical Benchmarks

• Engagement/Retention, by Level of Care

• Outpatient No-Show/Cancellation Rates
• Client Satisfaction/Perceptions of Care
• Degree of Engagement with Recovery
Support Services
• Involvement of Significant Others
 Organizational Climate

• Staff Retention/Turn-Over
• Staff Morale/Satisfaction (Measure and
scoring to be provided)
• Percent of Staff Position Vacancies
(Counselors)
 Financial Benchmarks
• Cost per Unit of Service, by Level of Care
• Salaries, by Role
• Administrative Overhead as a Percent of
Total Expenses
• Payer Mix
• Current Ratio (Assets Divided by Liabilities)
• Net Days in Accounts Receivable
• Days of Cash on Hand
 Data
Submission
• On-line benchmarking survey
• Aggregated anonymous data-no complicated
encounter-level data
• No software needed
• User-friendly, encrypted and secure
• Available 24/7
• Submit relevant and available data--no
reporting requirements
• State-of-the-art on-line data validation
 Benchmarking
Reports

• Standard Report
• Executive Summary Report
• Organizational Climate Report
• Run Charts
 Standard
Benchmarking Report
• Normative Data
– Sample Size
– Mean
– Median
– Standard Deviation
• Comparative Data
– Overall Percentile Rankings
– Peer Group Comparisons (budget size, geographic
area, setting) “Apples to Apples”
• Previous Scores
Standard Benchmarking Report
 Executive Summary Report

• Designed for busy senior leaders and

boards
• Graphic representation of scores
• Brief, key highlights
Executive Summary Report
Executive Summary Report II
 Run Chart

• Track your data over time

• Trend analysis
• Examine interplay among different
benchmarking dimensions
 Run Chart
Re-Admissions within 30 Days

13.40
12.76 UCL (12.65)
Re-Admission Percentage

UWL (11.05)
10.76
10.10
9.60
9.20
8.76 8.50
8.40
Mean (7.86)
7.30
6.76 6.50
6.20
5.80
5.10
4.76 LWL (4.66)
4.20

LCL (3.07)
2.76

r
st
ril

r
ay
ry

ly
ne
y

ch

er

be

be
be
ar

gu
Ju
Ap
ua

ob
M
ar

Ju

em

em
em
nu

Au
br

ct
Ja

ov

ec
pt

O
Fe

Se

D
Reporting Period
 Accommodations
for Multiple Locations

• Satellite locations easily accommodated

• Each location can be benchmarked
independently to produce a separate
report
• No limit to number of additional
locations
 Moving Beyond
“the Numbers”
• Benchmarking is not just about data
• When data is generated, real fun begins
• Identify potential best practices through process
benchmarking and other vehicles
• Enhance organizational performance through
NIATx principles
• Emphasis on shared learning from one another
and from “top performers”
• A learning community
 Shared Activities and
Available Resources
• Monthly audio-conferences/user support
meetings
• Monthly newsletter to educate and inform
• Telephonic and E-Mail user support
• Free telephonic consultation in
interpreting your data and developing
improvement strategies
• Articles and other resources
 One Year Subscription Fee

• Annual Subscription Fee: $1,000-

includes all aspects of initiative
• Conference Special for those that sign-
up on site-$900
 Questions?

Contact:
Paul M. Lefkovitz, Ph.D.
President, Behavioral Pathway Systems
877-330-9870 (Toll-Free)
plefk@bpsys.org
www.bpsys.org
BPS is Booth #33 in the Exhibit Hall