Creating Effective KPIs

Article published in DM Review Magazine

June 2006 Issue

By Wayne Eckerson

One of the most common questions people ask about performance dashboards is, How do we define
effective key performance indicators (KPIs)? The answer is important because KPIs govern how employees
do their jobs.

Agents of Organizational Change

The adage "What gets measured, gets done" is true. KPIs focus employees' attention on the tasks and
processes that executives deem most critical to the success of the business. KPIs are like levers that
executives can pull to move the organization in new and different directions. In fact, among all the tools
available to executives to change the organization and move it in a new direction, KPIs are perhaps the
most powerful.

Subsequently, executives need to treat KPIs with respect. As powerful agents of change, KPIs can drive
unparalleled improvements or plunge the organization into chaos and confusion. If the KPIs do not
accurately translate the company's strategy and goals into concrete actions on a daily basis, the
organization will flounder. Employees will work at cross purposes, impeding each other's progress and
leaving everyone tired and frustrated with little to show for their efforts.

Suboptimized Processes

A trucking company, for example, that measures performance by the percentage of on-time shipments
may drive hauling costs skyward because the KPI does nothing to discourage dispatchers from sending out
half-empty trucks to meet their schedules. To keep costs in line, the company needs to add a second KPI
that measures the percentage of unused cargo capacity in outgoing trucks, and it needs to revise the first
KPI so it emphasizes meeting customer expectations for fast, reliable shipments rather than just on-time
deliveries. This combination of KPIs gives dispatchers leeway to contact customers and renegotiate
shipping schedules if they know the customer may be flexible.

The Zen of Development

Crafting sound KPIs is more of an art than a science. Although there are guidelines for creating effective
KPIs (see sidebar), they do not guarantee success. A KPI team may spend months collecting
requirements, standardizing definitions and rules, prioritizing KPIs and soliciting feedback - in short,
following all the rules for solid KPI development - but still fail. In fact, the danger is that KPI teams will
shoot for perfection and fall prey to analysis paralysis. In reality, KPI teams can only get 80 percent of the
way to an effective set of KPIs; the last 20 percent comes from deploying the KPIs, seeing how they
impact behavior and performance, and then adjusting them accordingly.

Metrics used in performance dashboards are typically called key performance indicators because they
measure how well the organization or individual performs against predefined goals and targets. There are
two major types of KPIs: leading and lagging indicators. Leading indicators measure activities that have a
significant effect on future performance, whereas lagging indicators, such as most financial KPIs, measure
the output of past activity.
Leading indicators are powerful measures to include in a performance dashboard but are sometimes
difficult to define. They measure key drivers of business value and are harbingers of future outcomes. To
do this, leading indicators either measure activity in its current state (i.e., number of sales meetings
today) or in a future state (i.e., number of sales meetings scheduled for the next two weeks), the latter
being more powerful because it gives individuals and their managers more time to influence the outcome.

For example, Quicken Loans identified two KPIs that correlate with the ability of mortgage consultants to
meet daily sales quotas: the amount of time they spend on the phone with customers and the number of
clients they speak with each day. Quicken Loans now displays these two current-state KPIs prominently on
its operational dashboards. More importantly, however, it created a third KPI based on the previous two
that projects whether mortgage consultants are on track to meet their daily quotas every 15 minutes. This
future-state KPI, which is based on a simple statistical regression algorithm using data from the current-
state KPIs, enables sales managers to identify which mortgage consultants they should assist during the
next hour or so.

Challenges to Creating KPIs

Some of the challenges with creating effective KPIs include process nuances, activity measurement,
accurate calculations and lifecycle management.

Process Nuances. The problem with many KPIs is that they do not accurately capture the nuances of a
business process, making it difficult for the project team to figure out what data to capture and how to
calculate it.
For example, executives at Direct Energy requested a repeat-call metric to track the efficiency of field
service technicians, but it took the project team considerable time to clarify the meaning of the KPI. For
example, field service technicians primarily repair home energy equipment, but they can also sell it. So, is
a repeat call a bad thing if the technician also brings literature about replacement systems or makes a
sale? Or, what if a homeowner only lets a technician make minor repairs to an aging system to save
money but calls shortly afterward because the home's furnace broke down again?

Most business processes contain innumerable nuances that must be understood and built into the KPI if it
is to have any validity, especially if the KPI is used as a basis for compensation. The worst-case scenario
is when employees discover these nuances after the KPIs have been deployed, which stirs up a hornet's
nest of trouble and wreaks havoc on both the performance management system and compensation
policies.

Accurate Calculations. It is also difficult to create KPIs that accurately measure an activity. Sometimes,
unforeseen variables influence measures. For example, a company may see a jump in worker productivity,
but the increase is due more to an uptick in inflation than internal performance improvements. This is
because the company calculates worker productivity by dividing revenues by the total number of workers
it employs. Thus, a rise in the inflation rate artificially boosts revenues — the numerator in the KPI — and
increases the worker productivity score even though workers did not become more efficient during this
period.

Also, it is easy to create KPIs that do not accurately measure the intended objective. For example, many
organizations struggle to find a KPI to measure employee satisfaction or dissatisfaction. Some use
surveys, but often employees do not answer the questions honestly. Other companies use absenteeism as
a sign of dissatisfaction, but these numbers are skewed significantly by employees who miss work to
attend a funeral, care for a sick family member or stay home when day care is unavailable. Some experts
suggest that a better KPI, although not a perfect one, might be the number of sick days taken because
unhappy employees often take more sick days than satisfied employees.

Natural Lifecycle. It's also important to note that a KPI has a natural lifecycle. When first introduced, the
KPI energizes the workforce, and performance improves. Over time, the KPI loses its impact and must be
refreshed, revised or discarded. Thus, it is imperative that organizations continually review KPI usage.

Performance dashboard teams should automatically track KPI usage with system logs that capture the
number of users and queries for each KPI in the system. The team should then present this information to
the performance dashboard steering committee, which needs to decide what to do about underused KPIs.
For example, Hewlett Packard Technology Solutions Group (TSG) holds quarterly meetings to review KPI
usage, which it tracks at a detailed level. "If a KPI isn't being accessed, we go back to the owners and see
whether they still want it. If not, we remove the KPI," says Martin Summerhayes, program manager.

KPIs are powerful agents of organizational change, but creating effective KPIs is challenging; it is more art
than science. While there are many guidelines that can steer teams in the right direction, ultimately,
teams must put the KPIs in practice and see what behaviors they drive. Then, the teams need to
continually refine and refresh the KPIs to ensure that they are positively affecting organizational change,
not undermining it.

...............................................................................

For more information on related topics visit the following related portals...
Corporate Performance Management (CPM) and Scorecards and Dashboards.

Wayne Eckerson is director of research at The Data Warehousing Institute, the industry's premier provider
of in-depth, high-quality training and education in the data warehousing and business intelligence fields.
He can be reached at weckerson@tdwi.org.

http://www.dmreview.com/article_sub.cfm?articleId=1056305

Downloaded: 27-06-06