7

ANALYZING AND
1NTERPRETING MEASURES

surement system may be well designed and reliable but if the data
A rnea '
d information are not proper1y analyzed and interpreted, the benefits
a~ovided will be limited. Althou~h rigid rules for analyzing and interpreting
performance measures and the1r related data cannot be defined, heeding
;he following guidelines will help assure the data is analyzed correctly
and the right conclusions are drawn.
"It is a capital mistake to theorize before one has data ."
Arthur Conan Doyle

Variation and Statistical Analysis

All performance measures will exhibit some variation. At the lower levels
of detail, this variation can be quite large (±30% or more from week to
week), even if a production process is in control. This is called "common
cause" variation, because it is an inherent characteristic of the process
resulting from the variation in its individual components. In addition, there
can be "special cause" variation created by external forces or changes
within the process. Not recognizing that some variation is inevitable, will
result in managers chasing noise and oscillating between elation and
depression when performance has not really changed at all.
The first rule to follow when interpreting performance measures is to
not react to short term deviations until the reasons for the deviation are
understood. If the deviation is within the normal range, there has been
no change in performance at all. If it is a very large aberration, something
unusual has happened and the cause should be determined. In most cases,
spec,a
· 1 problems or circumstances will be known by those responsible •

113
 " •
-
operational
, the performanc .
,or

Performance Measurement: Increasing Total Product·IVlty

e measure.
1'b'l'ty
11 1s one
Understanding what is happening in
. . .
of the first require me nts m properly .
c

.
one's
inter.
area of respons For exampie , if it appears pe ri o rm ance has irnprovect
Preting measures. k but no changes have been made to proce

---- .i5(I()
IIPJ

Analyzing and Interpreting Measures

SALES • 4 Mell moving average

115
L 1::- A .,: t A ;:~,
t few wee s, . d' . sses,
during the pas ,, . robably a spike that w1 11 1sappear m the next few
I,
the "improvement is p 3500
weeks. . ra h or run chart will provide a good sense of the 30'.ll
A simple hne
I vanauon m a
p process. This is one reason why performan
. d f . ce
i•
norma Id ays be put on run charts mstea o relymg solely
I
2flXJ
asures shou aIw I . 0n
me h f hat is perceived by managers as unusua circumstances
reports. Muc o w
is really normal behavior.
, 20Xl \:~~~~;~··· ···\:/······ . f ·f-··.. ·····\~/···· \·r···:···J"'·········
While discussing the problems of manag ing in ventories, one 1500 . . . ... . .. . .. ... ... . .. .. • .. .. · ·· ·· · ·· · ·~ · 1· · · · · · · ··· ····· · ····\f......-~··f·· ····· ······
manager exclatmed: "1bis business is hard to manage. Take this
1CXXl ········· ····· ·· ······· ···· ············ ···· ·········· ··························- ····· ···· ····
order, for example. 1bis guy wants 20,000 p oun ds by Fn day but
be hasn't ordered this product in six months! We get exceptions
-
500
like this all the time!"
0 20
1be manager was correct in saying that eve nts like the unusual order 5 10 15
happened all the time, but that meant they weren't exceptions at all, just
normal behavior. This became quite evident whe n a 4-week moving
average showed that demand for each product was actually quite stable Figure 7-1 Weekly and 4-Week Moving Average of Sales
as shown by Figure 7-1. Other techniques such as exponential smoothing
can be used to reduce the amount of variation in measures and present In a broad sense, if you are using numbers, you are using statistics.
a more meaningful picture of performance. Statistical analysis is a powerful tool, but it must be properly used to add
Although these simple techniques of dealing with variation are suffi- value to data. Since improper use of statistics can lead to incorrect
cient in many cases, the only way to really understand and properly conclusions, companies should make sure the people doing statistical
interpret variation is to use statistical analysis. That subject is beyond the analysis have the appropriate knowledge and experience when important
scope of this book, but there are many good books and training courses decisions or complicated situations are involved. Appendix D provides a
available on statistical process control. One cautionary note is in order; good example of how improper statistical analysis can lead to conclusions
th at is, a little knowledge can be a dangerous thing.
that have no merit.
My favorite story about the misuse of statistics occurred at a
sDemtnar conducted by Edwards Deming. After the seminar, Dr. Identifying Relationships
eming was answeri .
a•nn b ng questions, when a marketing manager Identifying
h .
re1at1onsh1ps. between variables is important for understanding
,..,,roac ed him 1b .
sales control cha. e manager proudly showed Dr. Deming hrs Look'
ow a process works and also for identifying the causes of problems.
gruffly satd· "Tb rt;Dr. Deming took one glance at the chart and t iall 10~ for relationships should be part of analyzing any process, espe-
right mind· at 5 a nice chart, but why would anyone in their Y t ose that have several external or internal variables that might affect
tn control• want a control chart on sales? You don 't want sales pro
CUstcess p er ormance. JThis also includes customers, because particular,
' you Want safes if ,
realize that?!" When l out o control - going up.I Don t you war 0 mem ca n m · fl uence quality measures such as compIamts,
· returns_,
leavtng the roo ast seen, the devastated manager was ranty cl . •
room to spare. m by crawling under the door with considerable
Scatt ~tms, and general satisfaction.
two ~r diagrams are a simple way of identifying a relationship between
variables. This consists of plotting points for different values of the
_.Alli
116 Operational Performance Measurement: Increasing Total Prod uct1v1ty
..

..
. .. ..
--- Anal_>-zing and Interpreting Measures
Complaints · percent of units sold

117
.. Model % Plant %
.. .. .. .
Chain %
.. B
.. ... .
B
106A 1.1 1 0.9 K 0.6
.. -.. ... '

117B 2.6 2 0.8 s 0.8

A A 419B 0.8 3 2.4 p 0.7
NO RELATIONSHIP RELATIONSHIP
777A 0.5 4 0.9 T 0.8
Figure 7-2 Scatter Diagram
w 2.1
two variables as shown by Figure 7-2. If there is a strong relationship
between two variables, it will probably be indicated by a scatter diagram.
If the relationship is weak, it may not be readily apparent, especially if Figure 7-3 Complaint Stratification Analysis
there are other factors influencing the behavior of the variable of interest.
More sensitive and precise ways of identifying relationships are regres- considered. Unless cause-effect relationships are absolutely known, it is
sion analysis and Taguchi design-of-experiments methods. These tech- not wise to make assumptions before stratifying data. For example, if
niques can quantify the influence of several related variables on a process analyzing manu facturing product quality, stratifying data according to
or single performance measure. There are several powerful and inexpen- vendors, part numbers, product groups, process step, operator, shift, day
sive statistical analysis software packages available for using these analyt- of the week, department, supervisor, type of material, temperature in the
ical methods. plant, design engineer, sales representative, customer, or any other asso-
Stratifying data is another way of identifying relationships between ciated variable would be appropriate.
variables. Stratification consists of cutting the data into layers according Stratifying data can yield some unforeseen results. For example, an
to the different variables in a process. For example, to analyze customer analysis of defects in a metal finishing process showed that during one
complaints about toasters, it might be appropriate to look at them by period of four weeks, the defects on one of three plating lines were only
model number, which plant made them and which retail chain sold them. one-third of what was normal. Further investigation revealed that during
1 th
~ is case, the distribution of compl~ints to units sold might look like those four weeks, a chemical component that was normally very stable
Figure 7-3.
went out of control on the line with the lowest defects. In fact, it was at
The analysis indicates a possible relationship with model 117B, plant 3, only half of its specified level. What this meant was that the proces~ r~n
and store chain W but th Id
better when the variable was out of tolerance than when it was with in
1 • . .
be . ' ese re attonsh1ps are far from certain. It cou
st
~h~m W ju happened to sell many more 117B units than the o ther specifications!
ml e s a~d that plant 3 just happened to make mostly 117B units. It is
a so possible there · thin · W
Add'ttlonal
· investigation revealed the specifi ca c·ion may have been
mistakenly m· ts no g wrong with 117B units, but chain ~orrect at one time but several changes had been made to the pr?~ess
in th ' • d · d the ongmal
data does no:Sreprer:ed the units in its advertisements. Stratifying the e past few years. Further analysis and testtng etermme
been narrowed~upp y e answer, but the choice of where to look has specjfi • cifi • should be changed to a
fi cation was incorrect and the spe catton · .
ifi t'10n "fishing
When looking for 1 . . &Ure close to what was discovered by accident. This SCrat ca d d
variable in a process hre a~onships, it should be kept in mind that any e~pedition" identified a relationship that was totally unexpecte :
parameters that aren't as t e potential to affect it. Even inputs or process Yielded a b' Th bl might have been discovered by o er
1g return. e pro em . d b anyone
supposed to change or affect a process should be means, but since the specification was not being quest1one y ,
........._
118 Operational Performance
it is more l1 e Y
time with
. th

Measurement: Increasing Total Productivity

----
'k I the process would have continued running. for a Iong
•fi t,·on before the problem was discovered
ong spec, ca b .

---- Analyzing and Interpreting Measures

Physical processes, however, can be very complicated. Consid-
difficul~- stigation and analysis may be required to determine the rela-
erable _inv~tween variables in physical processes.

119
. 1 ehwr lpful to summan·ze quality and . waste pro !ems by broad t1onships
It ,s. a so e as being caused by machines, operators, vendors, cus.
categones such temal or .
mterna 1factors hardware or software, and. controllabJe
, .
tomers, ex S tifying data by broader categories 1s similar to
Determining Process Capability
uncontrollable. tra k Th .
or . th bo Id rs instead of the roe s. ere is no guarantee Understanding the capability of a production process i·s .
looking at e u e h'l I k' h I · very important
.
stratificat1on w1.11 veal anything worthw i e; not oo mg,
re . be " d owever, guar- for both control an d p annmg purposes. For a process such as ru·
. ships that do exist will . . . . I cu ng
antees any reIatmn . . not
I . 1ounth . sheets of metaI to d imensions, stallstica methods can be used to determine
When looking for relationships, a cntica pom~ at must be kept in the capability of the process to meet specifications.20
. d 1s
mm, . that 1·f two variables seem to be related, it does not necessarily For large and complex processes, the capability question is "What
mean they are related. If it appears A _and are related, there are three performance level can be maintained by the process?
possible reasons for the apparent relationship: To answer this question, it may be necessary to select a period that
seems to represent normal operating conditions and make a judgment
1. A and B are not related at all. The apparent relationship is the from the performance measures. If managers are staying in touch with
result of pure chance or coincidence. what is happening, these judgments can be quite accurate.
2. A and B are related, but A does not cause B or vice-versa. Instead, In terms of improving performance, determining the capability of a
they share a common bond through some other variables. process is often not the question. No matter what it is, it may need to be
3. A and B have a cause-effect relationship. improved.
Theoretically, when all the special problems in a process have been
In other words, correlation does not mean causality. Just because two fixed, the process is operating at its capability. But this limit only applies
things happen at the same time, it doesn't mean they are related. As the to the process in its current configuration. In that sense, the real question
story goes: managers often need to address is: "At what point should attempts to
incrementally improve the capability of a process be abandoned in favor
Last Saturday morning, I was aroused out of bed by my neighbor of a radical restructuring (or reengineering) of the process?"
who was in bis back yard dancing around, chanting, and Consider a complex piece of equipment, which has ten quality variables
beating a drum. that are all not within specification. By bringing each of the variables
"Herb! What in the world are you doing?" I yelled. within specification, quality problems are cut in half. Is this as good as
~e equipment can do? In its present state, yes. But perhaps an additional
"I'm doing this ancient African dance to keep the elephants improvement could be made to the machine by installing more powerful
away, " be answered.
motors. Is this all that can be done? Not necessarily. Maybe gears could
"B ttb · •
u · ere tSn t an elephant Within 10,000 miles of us!" I replied. be changed, better tools could be used, or laser-guided alignment mech-
"Gosh, I didn 't think it would be that effective, " says Herb.
anisms could be installed. All of these changes would increase the capa-
bility of the machine 1 not just fix something that is broken.
To avoid drawing simil . If several changes were made, the performance of the eq~ipment might
between var,·abl ar erroneous conclusions, apparent relationships look like Figure 7-4, which shows the improvement ~rov1ded by each
es must be v rified b
works. This· ma . . e Y understanding how the process
Y require mve5r · ·t ~uccessive change. It might seem further significant improvements are
must be done if th igauon and even experimentation, but 1 impossible after making Change 4. This could be true - unless, of course,
e process is · d
Fortunately • most busmess
. proc going to be understood and improve · another way is found to improve the machine's performance. h d
of cancer, so verifyin esses are not as complicated as the causes As h· .. h n a process has reac e
g cause-effect relationships is usually not terribly
. t is example illustrates determmmg w e f . d t
its . ' · matter o JU gmen •
Practical limit for incremental improvement, ,s a
120 Operational Performance Measurement: Increasing Total PrOd Uct1vity
Event Defects{%)

.
' --- 25000

Analyzing and Interpreting Measures

-··· · ······------- . .. .. . ................................. .
3.5

121
Start 10.0 ERRORS and SHIPMENTS
20000 ··························· ·········~ 3
Change 1 5.0 ll
C:
715000
. 2.5 iI:!
Change 2 4.1 ---•;•A••-~- .......... .
8.
I -----·+ 2
I 10000
I
3.4 •••••••••••••• ; ·- •••••••••• • ·errorisiart"ilicreas'1g • •••.•.• 1.5 t
.
Change 3
Change 4 2.8
ii 5000 -
- .....---
. --
Shipments • ' ' 13,600 Md 14,500
unbJweek
- - - - ..... -...... --......... -........ - . . ....... -......... - . g
w
Figure 7-4 Process Improvement Steps 0 ~ -t-~~-+--:;---+-~-+--+---+-l----t---1----.1....- - 0.5
0
1 3 5 7 9 11 13 15
Week
If the person making that judgment understands how well the process is
performing and its improvement history, that determination will probably
be quite accurate. If no measures are in place, making these decisions is Figure 7-5 Determining Capacity From Quality and Product Output
just a guessing game.
In situations where workloads are changing because of changes in
sales, product mix, staffing, or processes, changes in quality performance
Determining Production Capacity
can indicate production capacity or capabilities are being exceeded.
How much work a process can do in a given period is its production Although it is usually less pronounced, the same behavior can be observed
capacity, which is a type of capability. Knowing this figure is obviously in equipment performance when it is overworked. When quality problems
essential for effective planning and management. When production capac- start to increase and there are no other identified causes, it is a strong
ity is exceeded, the following will happen. indication that the production capacity of the process (or the people in it)
has been reached.
1. If the process is limited by equipment capacity, work will pile up
in front of the limiting steps of the process. Mk'
a mg Comparisons
2
- If the process is limited by labor capacity, work may pile up in
Executives
ments oft. en want to compare the performance of different depart-
front of the bottleneck step(s) in the process, but the work may
al~o get done while the quality of the work suffers - rework and or units, especially when it is time for raises promotions or
re1ects will increase. bonuses
Th · B ut h ow can this be accomplished on a meaningful' basis? '
Yardst~r: are no ~asy answers. Variance against budget is one common
fonsequently, an estimate of production capacity can be derived from assu ic . ' but this assumes the budget was valid and the operating
~tp1;1odauncdu· production output data. By plotting quality at various levels the mp~ions made when the budget was prepared were satisfied during
on output as h b . • lity ve penod in question. The same is true for using actual performance
problems will be ' s own Y Figure 7-5, an increase m qua.
seen when the od • begi!lS thersus ob·Ject1ves
. as either a measure of performance or for companng .
to be exceeded. In this pr uctton capacity of the process_ noctf rformance of different units. If the objectives were reasonable and
14,000 units per we k Th~se, the production capacity is approximately
co di?~ of consequence changed this comparison is valid, but those
fairly accurate estim:re· f ts method is not precise, but it will provide. a n ltlons are diflicult to achieve ,or venfy.
.
a process because th . o capacity. It can also identify bottleneck steps in 0
' e1r quality _c first. items": o~her basis for comparison is quality. In this case, there are two
peuormance is going to break down f mterest - the current level of performance and the rate of
 _, ance Measurement: Increasing Total Produ .. Anal ·
...,
122 operational Pe11orm ct1v1ty ----- yzing and Interpreting Measures • 123
.
t tes of improvement is always valid, assu .
change. Companng ve been made to what is being measuredming no
substantial cha~gecals 1alated For example, if division A reduced 't or ho'W

for example, a process having one step and f ·1

thousand produced would have a failure rate af t?re rate of one unit
1
. . dex is cu . I s qu 1· P~ ppm. A process having the same failure r~te bl,OOO? x 1,000,000 =
0
a qua1ity in d division B reduced its quality costs fifty p a tty l, might have the same 1,000 ppm rating . ' ut with three serial
t n percent an , ercent . steps, m terms of outp t .
costs e 'bl conclusion that B outperiormed A in terms f , II but each step would have to be operating at roughly 100 u qua_1ity,
would be a senEs1 if they started at different levels of qualityo quality 000 ppm end result. ppm to achieve
improvement. ve . . . Perfor. the 1, . .
the conclusion is still vahd. Counting actual errors 1s straightforward but d t . .
mance, ,
·ng quality per,ormance · a b so Iu te terms 1s
m · another . f , e ermmmg the oppor
ities for error 1s o ten not so easy. For example how m .. -
Com pan I'd .
ely similar operations, va I comparisons are easy to mak b ·
matter tu n h • k' . • any opportunmes
For clos b h e, Ut , r error are t ere m ma mg a piece of furniture? D
10 • . • o you count the
nits that have different products,thprocesses, or. ot ' comparing qua1ity ·
pi·eces , the ind1v1dual steps, or what? How about measu rmg . the typmg.
r
,or u . .
erformance is much more an art an 1t ts a science. Do you measure errors per word pe 1.
error rate in a report?
? h . k. , r me, per
p In theory, conformance quality of different production processes could page, or per letter. W at you ptc 1s going to make a dramatic difference
be compared by measuring non-conformance and adjusting that fi in the calculated error rate, but the process and its performance charac-
for the complexity of the process. The common way of measuring 8Ure teristics are going to remain unchanged.
. . ( ) non-
conformance is in terms o f parts-per-mt11 10n ppm which is ·ust a An approach that can be used to arrive at a ppm figure, is to assume
. 1 more
convenient way of expressing very small percentage figures as wh that before quality improvement efforts start, all processes are operating
1
numbers (0.01% • 100 ppm). This figure can also be converted too e at four-sigma (6,210 ppm). This is the performance level of the average
statistical "sigma" equivalent as shown by Figure 7-6. (These figures all a company and operation as determined by surveys of processes such as
for a 1.5 sigma drift in the process.) ow writing up orders, airline baggage handling, and payroll processing.21
Then, the divisor in the ppm equation can be chosen so the initial figure
FAILURE PARTSPER is within the given range.
SIGMA RATE-% Mill.ION Consider a complex assembly process that is producing 2,000 units
per week, with an error rate of 160 defects per week. On a per-unit-
3 6.6810 66,810 produced basis, this is an error rate of 8% or 80,000 ppm, which seems
like poor performance. However, if there are a thousand opportunities
4 0.6210 6,210 for error on each unit, the error rate falls to 0.008%> or 80 ppm, which is
very good performance. There is no way to determine the actual oppor-
5 0.0233 233 tunities for error, but assuming the process is about average, the current
6 0.00034
defect rate of 8% would equate to 6,210 ppm. This yields the following
3.4 formula for measuring the assembly process quality performance in ppm:
Figure 7-6 Sigma Rating and Parts-per-Million Equivalent ppm - (defects/units produced) X (77,625)
°
Statistical methods can be used to determine the non-con rmancegiven rate If there were several different assembly plants, they could all use
for
d processes having continuous variables such as drilling hole; to ances s_imilar formulas, and if they all started improvement efforts at ~he sa~e
1mens1ons
. h . . a steel plate. For measuring discrete no n -comorm ality
in time, their relative progress would be directly reflected _in their quaho/
h
sue as itt1ng. , one's isthumb with a hammer' the formula to me asure qu Ppm index. Eveiyone will be on the same scale as far as improv.em~nt is
in pans-per-million
concerned but this will not mean that units with the same quality mdex
Ppm • (actual errors/opportunities for error) x , '
tllveryneces~arily be operating at the same absolute level r°~ ~erforf a~:~
index will depend on the initial average level Oe ects w ic
( 1 000 000)
 •
---
111111
124 Ope..-ra·:.:t1::·o.:.:n=-al:...P_e:...ri:...o_rm_an_c_e_M
_e_
as_u_re_m
_en_t_:_l_n_cr_e_a_s_
in..!eg~li::o::t:.
al:_:P
----
:_:r'..::o~d~uct1v1ty
. . Analyzing and Interpreting Measures 125
---
could easily be off by 100/o or more) and the assumption that the stanin
• ents a certain absolute level of performance. 8
pomt repres .
Comparisons can also be made against benchmarks, but it is not D epartment A Department B Department C
practical to develop good benchmarks for every process. Even if it Were
possible, unless the benchmarked p~ocesses were closely similar, compar- Problem Points Problem Points Problem Points
ing levels of performance would still be a matter of. educated judgme ,___
It is possible to make reasona bly accurate ~ompansons against bench- Al 1256 Bl 457 Cl 802
marks, but it requires thoroughly understanding both the company's and
the benchmark's processes to be able to make those judgments with any A2 1022 B2 256 C2 421
degree of accuracy. I
The most valuable comparisons for a company would be comparing its A3 878 B3 145 C3 303
key performance factors to its competitors' and to benchmarks that are trul
best in class and/or the best in the industry. This is not easily accomplished 819 B4 102 C4
A4 254
but many companies have been able to do it in selected areas. '
- cs 102
A5 693 B5 79
Departmental and Process Perspectives
Monitoring performance on a departmental basis is necessary for establish- I Figure 7.7A Top Two Departmental Problems Based on Quality Costs
ing accountability, providing feedback, seeing appropriate actions are taken,
and the development of individuals and the organization. Although this is
an important use of performance measures, looking at performance from
only the departmental perspective may overlook important process issues.
For this reason, it is important to periodically put all the performance Process
measures of a production process together and review the total process.
1n this case, the focus should be on identifying the limiting elements and Problem Points
improvement priorities for the total process. When viewed from a total
process perspective, priorities may look quite different than they do from Al 1256
a departmental viewP9int.
For example, assume there is a process involving three departments 1022
A2
A, B, and C. The weekly quality costs of the top five problems in each
department are shown by Figure 7-7A. The top two priorities in each 878
A3
department _are relatively clear. However, from a process perspective, five
th
of ~ top SIX problems are in A, one is in C, and none are in B as shown 819
A4
bit Figure 7-7B. From the process viewpoint most resources should be
a ocated to department A, a small portion sh;uld go to C and department 802
B should get whatever can't be used by A and C Giving 'each department Cl
: equal share of resources would result in ap~lying about one third of
em to areas of little return. AS 693
Analyzing performance from . . • rtant
for efficient allocation of a process perspective IS very impo 1 ------------- on Quality
orientation of most com an:eso~r:es. However, with the departmentae Figure 7-7B Top Six Process Problems Based
is specifically d . dp ies, it 15 not likely to happen unless someon
esignate to do so.
 ,,.
----
-
l26 1 operational Performance Measurement: lncreasing liotal Prod uo·IVity
Analyzing and Interpreting Measures 127
zooming In and out ·rnprovement in performance. However if
. th matical certainty that higher level, or aggregate some ed1 the gains will not last. , processes weren't
It 1s a ma e . show less . . th th • 1
vanauon an eir ower level com, Performanc (hang •
measures WI11 h . Pone e Zoarning out .can show patterns that can't be seen by ki th
. trUe of the time it takes to see c anges m perform nts. lne . . , Ioo ng at e
same IS ance p A case 10 pomt 1s a manu,acturing producti·on 1· h
• •
·t is important for managers to penod1cally zoom in · orihis deta Us · . d . me t at was
reaso, n l di . . onth xperiencing quality an production downtime problems. Efforts to
measures. Probing into trends an arge vanat1ons in detail perfore detail
• • d ~prove performance were effective in . some areas, but the shaping
measures can identi fy areas where acuon 1s nee ed or wher rnance machine seemed to defy any attempts to improve its performance. When
·I I I · e manag
support would be benefi c1a. ta so .increases
h .a manager's underst ernent performance data fo~ a four-month period was analyzed, it showed
of what affects a process and what 1s appenmg within it. anding
repeated problems w1~ gears, chains, motors, and other parts in one of
Similarly, looking at individual trees is no way to see a for st drives inside the machme. Most of the parts had been replaced or adjusted
c;2Y! and ~ g~ er view of what is happening in ;: r Zooming ral times in the four-month period.
fle~eot,_.:_o r:process should also be done every few mo~ company, seVe . . .
th When this data was reviewed wtth the operators and maintenance
-case, longer-term
- trends and relationships between me asures s. h In this technicians, everybody reached the same conclusion: continuing to repair
!h!:..primary focu§. When taking the broader view ue st . s ould be
the drive on a piecemeal basis was not going to work; it was worn out
following need to be asked from a total companY perspective. ' q '.ons like the
and a major overhaul was needed. The overhaul was accomplished with
an all-out weekend effort and performance dramatically improved.
• Where is the workload increasing and why? It could be argued that someone should have reached the same
• Where has progress stopped? conclusion without analyzing any data. Perhaps so, but the repairs made
• Which strategic objectives are being/not be ing achieved? on the machine were spread over three shifts with two maintenance
• Is any realignment of priorities in order? men and one operator on each shift. No one could possibly see the
• Where do we need to apply more or less resources? whole problem, and given the level of activity in the plant, no one
• Which areas look as if they are starting to weaken? could be expected to remember the details of what happened weeks
• Which departments or groups need help? ago and put them together. Even with the data, the pattern of behavior
was far from obvious because many different parts and actions were
Zooming out also means taking a longer-term view of performance. involved.
What may seem to be big gains or losses in performance last month, may
look quite different when the past twelve months is considered.
Looking for Opportunities as Well as Problems
_A company that was trying different pricing strategies and promotions
to mcrease customer volume and a product's sales illustrates how different The previous example of the plating process, where defects decreased
pe~ormance can look from a broader and longer perspective. TeSlS of when the chemicals were not within specification, illustrates the value
vanous offers in a few markets indicated some increased sales 10 to IS%. of looking for opportunities as well as problems. While solving problems
Hhow~v_e~, when these promotions were implemented on a broader scale, ~ay be the most important aspect of improving performance, the other
t ey 101tially met • • mers side of the coin should not be forgotten. Managers tend to get upset
0 expectations
r sa1es over a six- th .but failed to yield any increase m CUShtOt the
Price and moo penod. The reason this happened was t ath . aotl ask questions when performance is worse than normal, but when
promotional h e elf herformance is better than expected, no one seems to ask why it
purchases r d. c anges caused some customers to mov ., appened .
,orwar m tim This d custom"
volume, when actuall e. created the illusion of increase t 11ie
same type of Y no new customers were buying the produc ·
Questions like "What is different?" or "What has changed since last
improve perfo;:ponse can be encountered in just about any effort or 10
Week~"· need to be asked. The answers may be just as difficu It to determme
·
st as "'~en trying to solve a problem but they can also be just as valuable.
ju giving any ;:~:- The "~awthorne effect" of making any chan~~ iJJ
With looking for problems no~al variation must be taken into account
rmance issue more emphasis will usually res in Order to avoid
. discovering
. ' an opportunity
· that doesn't exiSt ·
 128 ~~u~
v_
.._
~_
·_
·_
-_______
Analyzini and lnterpretinB Measures 129
. P rformance Measures
Interpreting e
Quality and Productivity
Context do not exist in a vacuum. They are af' ,
measures . od . 1ectect by lity and productivity are strongly related, but they do not necessarily
Performance f' a company or its pr uct1on processes ""
. that a ,ects · weath ~ua s move in the same direction. There are four possible combinations
anything . d' ruptions unusual customer requests corn . er,
ly hne 1s ' ' Petit ,
strikes, supP. bo t anything else on the face of the earth c 0 rs
~;v:Janges in ~ua~ity and productivity that have different meanings and
important imphcat1ons:
. and iust a u h' • an cause
actions, . . . performance measures. Anyt mg of this natu
I deviations m . . re lllUst
arge
be art of th e con text considered when mterpretmg measures .
t. ~rod~ct ~ t y increases and productivity decreases. Quality
P
For examp 1 e, 'f the quality performance of a manufacturing pla nt has
1 . h . . 1s bemg improved by brute force, not by improving production
been decreas1·ng slightly for the past SIX. mont .s, 1t 1s 1ogical . to cone!ude processes. Intensive inspection and rework will improve quality,
the manager IS • doing a poor job of tmprovmg quality. But if furth er but only at the expense of lower productivity.
invest1ga . ti'on reveals that during those. four months
. the plant introduced 2. Product and process quality increases, but productivity does
, maio
,our . r new products • which required ma1or changes to manufactun·ng not increase. This means performance has improved, freeing up
processes, his performance ma~ h~ve actually be_en ou~tanding. That is resources that are not being utilized. Putting it another way, quality
why it is good practice to note significant changes m env1ro~mental factors has improved, but work has expanded to fill the time available.
or unusual circumstances on charts when they occur. Besides explaining The resources released by improved quality need to be identified
what caused particular behavior, these notes can help managers predict and reallocated to other activities.
what will happen under similar conditions in the future. 3, Product quality, process quality, and productivity are all
Context is important, but making allowances for poor performance can increasing. Process improvement is working and resources are
also be overdone. Minor events should never be used to make excuses either being reallocated or absorbed by increases in the workload.
for large changes in performance. Furthermore, a logical cause-effect However, this does not mean all free resources available are
being used.
relationship should always be identified when interpreting performance
measures. After all, everything can't be blamed on the weather. 4. Productivity and quality are both decreasing. To paraphrase
a popular beer commercial, it doesn't get any worse than this! The
The same is true when performance is unusually good. This could also
performance measures should tell you where the problems lie. Get
be caused by favorable circumstances. Even if the circumstances are not them corrected as soon as possible.
controllable, understanding what happened can lead to new opportunities.
Proper interpretation requires managers to approach the task objectively and
resist the urge to attribute all poor perforrnance to uncontrollable factors and Strategy, Operations, and Profits
all good performance to superior management skills and decision-making. Strategy, operational performance, and profits are firmly linked together.
A simplified summary of this relationship is shown by Figure 7-8. A
Two maggots, Fred and Sam, were thrown off a garbage truck s_uccessful strategy can be viewed as being in the right place at the right
as it turned a corner. Fred fell near a dead squirrel that had time, whether by accident or design. With a strong strategy, it is difficult
th
been rown under a bush. Sam fell into a storm drain where not to make money, although poor operational performance can still erode
~:ere was no food at all. After a day-long struggle, Sam managed margins. On the other hand, a company that finds itself in the wrong
craw/ out of the dratn. Disheveled skinn11 and hungry, he place at the wrong time can be operationally excellent and still have very
cou1d b I , JI • t
h arey move. As luck would have it his friend Fred JUS low profits .
s:1:/t~;,d 1
~ be Passtng by. "Ht, Fred! You ~ure look good!" said Having been associated with companies at both ends of the scale, I
natl . .,. ou re so sleek and fat, but I'm so hungry I could eat can personally attest to the validity of Figure 7-8. Apple Computer is a
s. ,o what d
"Brans °
1 and You attribute your success?" · d
gOOd example a company going from a strong strategic position to a very
Weak one, although the quality of the company's operational performance
...
Personality. ., Personality, " said Fred. 'Just brains an
at either point is not clear.
,.
 --
-
operational Performance Measurement: Increasing Total Product·IV1ty
Operations

.
Profits -
" Analyzing and Interpreting Measures
Although constructing a strategy-operations-profit chart for a given

131
strategy mpany may not be practical, the concept is still useful. It says that
High k owing where a company 1·1es on its
co . strategy-operations grid will deter-
Excellent
Strong :ine what type of actions should be taken to increase profits. Excellent
Poor Moderate erations cannot fully compensate for a weak strategy. Likewise, a strong
Strong ~frategy can compensate for poor operations, but it can also be under-
Excellent Moderate mined if it is poorly executed. Since the effectiveness of a strategy is more
Weak
difficult to assess than operational performance, the quality and produc-
Poor Low tivity of operations should never be an unknown quantity. Furthermore,
Weak
it certainly makes sense to maintain operational performance at high levels
to remove that variable from the performance equation. With proper
performance measures in place, even if operational performance deteri-
Figure 7-8 Strategy, Operations Performance, and Profits
orates, it will be a known quantity that can be taken into account when
of course, both strategy and operational performance can have an infinite assessing a strategy's effectiveness.
number of degrees of perfection. Conceptually, this is shown by Figure 7-9. Determining where a company lies on an operations excellence scale
Maximum profit occurs at the point of maximum strength of strategy and cannot necessarily be done in absolute terms, but a good sense of a
operations; minimum profit (or maximum loss) occurs at the opposite company's position can be gained from the following information:
extreme. Any point in-between will have some other level of profitability.
Although where a company lies on the strategy-operations grid will deter- • Company performance measures. Knowing where you are and
mine its level of profitability, a given profit level can be attained by multiple where you came from is a prerequisite for determining your place
combinations of strategy effectiveness and operational performance. on the strategy-performance grid. Furthermore, knowing how much
performance has improved in recent years can be compared to
what has happened in related industry groups to get some idea
of whether ground is being gained or lost.
[1 • Information published by industry groups, professional asso-
~-., ., .,;.-<
.•. -··· t ,,_ ciations, government sources, trade groups, investment firms,
.-····· +•+ --- '\... ~efl\ and similar sources. Almost every business sector of any size
slS~:':~-······ ',,__ '\... ei-c/3~
'(
. .•········\~'~,~'
.. •···· . ., ., ---- •

publishes figures about sales, productivity, and other performance

variables that can be used for comparing rates of improvement.
Magazine, newspaper, and professional journal articles. Com-
·,+ ~ - - - - ~J,' ' .,.,,
.,.-,..:.t~'UO
Qy'v
"
+,- panies with outstanding operations are frequently discussed in
trade and business publications. Industry Week, for example, has
an annual Best Plants award and devotes an issue of the magazine
to that subject.
• Benchmarking with companies that have similar processes, but
are not competitors.
• Customer feedback from third-party surveys and from information
--
collected directly from customers.
• Competitive intelligence. A lot of information is available and
..
legally attainable if someone looks and asks for it. It is amazing
what can be learned about an industry and companies within it
The Relationship Between Strategy, Operatio ' With a little effort.
· ns an d Profits
' ,.,...- 11111
M asurement: Increasing Total Produq· .
. I Performance e iv,ty
132 1 Operauona
---------------'----"----..:..L.:..::.::..:3?...:.:..::::::::::.:..:::--=~::.::.
Analyzing and Interpreting Measures 133
. downward spiral in the early 1980s, ilitis
X which was Ill a knowing the coordinates of their str lrates i
xero , f anagers bl d h . ateo,, pt action is needed to prevent further erosion. Had General Motors
. p<>rtance o rn . ·all the company ame t e1r misfonu or
the ,m . . IniU y, th k Th nes o pro; and Chrysler heeded the signals sent by their small losses of marke~
operations p<>51uon~ their products on e mar et. en they made n
:1
:~:r; in the early 1960s, how differe~t their fortunes might be today! The
the Japanese durn~ gThe first, was that the Japanese were able to o
same is true of the U.S. commercial electronics industry • which has
shocking discoven;~at it cost Xerox to make them. The second Was essentially disappeared.
their products for f lity was 2()<!/4) of sales revenue.22 t
Every company should have measures of market share because it says
's cost o qua , •
the company 's design and manu1acturmg operations we 50 much about their competitive position. Getting an accurate measure of
the company . h re not
ClearIY, . . th"s Xerox turned its Roe ester, NY plant up .d rket share may be very difficult in many cases. However, a measure of
· · Reahzmg 1 , s1 e-
competittve. d II procedures to involve every employee _ a m :ether market share is increasing or decreasing can be obtained by
down and change'd aff23 Xerox reversed its . , d
1ortunes an has long .
ove
rtainlY pat o • since comparing a company's sales growth with its industry's growth. Changes
that ce competitive position. The key elements in makin . market share should also correlate with customer and non-customer
turned to a strong k . . I g
re
this and other strategic decisions, was nowmg . , its
, own eve! of per'1or- :tisfaction surveys. If customers are saying they are very satisfied but market
' d mparing that to its competitors per1ormance. These initial share is decreasing, something very important is being missed in the
mance an co . · I k
. efforts were later revised to not JUSt• oo at competitors' but satisfaction surveys, or the surveys are missing a large segment of the market.
companson 24
to uncover the best practices wherever they exist.
If a company knows how well it is satisfying its customers' requirements Relating Operational and Financial Measures
and its performance relative to its competitors', it can make informed
decisions about addressing operations, strategy, or both. If it also knows Operational and financial performance measures should generally track
what levels of performance "best-in-class" companies are achieving in key each other, but there are reasons why they could be showing different
areas, it will know what performance is attainable, how far behind it is, patterns, especially in the short-term.
and what it has to accomplish to achieve superiority.
Certainly, all of this information cannot be obtained in precise terms • Since there can be significant time lags between changes in oper-
and numbers, but companies have demonstrated that enough quantitative ating performance and when these changes appear in accounting
and qualitative information can be obtained to make effective strategic figures, operational measures will generally lead, or predict, finan-
and operational decisions. If a company does not have this information, cial results. Poor quality reported today may not show up as
it is just guessing about where it is and where it needs to go. 1ne increased costs for several weeks when the actual rework takes
performance measures and benchmarks won't improve performance, solve place. The effect of returns, extra service calls, and added freight
lhe problems, or define a viable strategy but they will tell a company costs may not show up for months. Many of these costs may not
where it stands and h th et1" n, be seen at all because they get thrown into broad cost categories.
must be taken. · W ere per1ormance must improve or o er a Ou-
C '
Similarly, if productivity and quality increase, it may take months
for increased sales to absorb the under-utilized capacity or to
reallocate people to other areas. Until then, unit costs may show
The ,..,,.,._ of Marlcet Share , '"""' little or no improvement. In both manufacturing and services, the
work-in-process pipeline must be emptied before the full impact
M,,ket """' s , key lndkaio,- of the elfect,ven'."s
. O f a company
d .n the short
s " on of changes in performance will be evident in financial reports. This
•nd <Ompetiti,e po,jtlon. M,rl<e( sh,re can be mcrease ' ssible uni"'
by ""'"•ng """'• but in the long..- term, that is not,!';,oduc!>
• COmp,ny h,s • - •dvant,ge ,elative to the value of ' spects, th

delay could be as long as several months.

• Financial account structures will probably reflect organizational
structure rather than production processes, making it difficult to
· Of • COmp•ny's Diarl<et sha,e is important, bUt m
•tt · many . re re,.;ng r, correlate the two sets of measures .
- "8nili<>nt "P<a of - sha,e is whether it is me . , ~ • Administrative or indirect manufacturing costs may be allocated to
ma.....,g, it says the company has_ a :: irue"
"'"'•~ge mthe man..plac:c_
• - • · If k •
If decreasing, the oppo&te
goods or services according to formulas that are not related to
how processes work. For example, indirect costs are commonly
 ....,..-- ...
-
rfi nee Measurement: Increasing Total Productivit
134 ~rational Pe orma ---J'.___ Analyzing and Interpreting Measures 135
II0 cdonte h basl·s of units or dollars produced. This can result higher than what it has been. Most major corporate downsizings have
a ~ _e lume products a much smaller share of ind·
. giving Iow vo h 1req f ·Jed to produce what was expected, according to Peter Scott-Morgan,
m they often consume a large s are of these serv-
costs because th h h tees. :S~ociate director of Arthur D. Little.25
Accounting. reports aggregate costs at ave t. e same . name , even
though they may be driven or caused by entirely d1f~erent events.
"ght osts for instance, may be caused by nonnal shipments, rush Determining Priorities
Frei c , ct· d h"
shipments from vendors, or expe 1te s 1p_ments to customers Because there will always be more problems and opportunities than there
because of problems throughout the production process. Each of are resources available to pursue them, managers must always think in
these costs is different from the others, but they all wind up in the terms of priorities. Priorities for improving performance, or changes in
same bucket of "freight costs." Then, no one is held accountable for those priorities, should be one of the regular outputs of analyzing perfor-
these costs, because no one really understands where they came from. mance measures. Assuming a measurement system has the capability of
• Since accounting reports are usually produced on a monthly basis, determining the relative impact of performance variables, priorities should
they have a built-in smoothing factor that may mask some changes be relatively clear in terms of costs or profit opportunities. However, costs,
in performance. Any special charges, adjustments to accounts, or sales, and profits should not be the only factors considered in establishing
timing problems can also cause financial measures to deviate from priorities. Potential risk, the investment required, the payback period, how
operational measures. well projects support strategic objectives, the availability of required
resources, and other factors must also be part of the priority setting process.
Modifying the financial account structure so it coincides as closely as From a customer perspective, both the importance of a particular
possible to the operational measures, is one step that can be taken to variable and its current level of satisfaction must be considered when
make it easier to relate financial and operational performance measures. setting priorities. As Figure 7-10 shows, where a performance factor lies
It may also be possible to summarize operational performance data in on an importance vs. satisfaction grid should determine what action is
ways that correspond more closely with financial measures. What can be taken. This table should always be considered when determining priorities
done to better relate financial and operational measures, depends on the to prevent resources being allocated to projects that are totally unnecessary
systems a company has in place. The primary objective is to verify the in the first place.
acco~nting and operational measurement systems are tracking each oth~r
nd
a_ tf they are not, to understand why. The issue is not which system is ACTION REQUIRED
IMPORTANCE SATISFACTION
nght or wrong because they are designed to provide different information
and accomplish different objectives. High High Maintain current level
nda of performance.
. A s~co ry objective of relating operational and financial measures
ts to estimate th fi • . Q ("tu
e nancia1impact of the key operational measures. ua 1''
costs (the costs of · . lly High Low Potentially very costly situation.
be e il inspection, prevention rework and waste) can genera
as Y re Iated t0 fi · ' ' 30% Priority for action and additional
of sal . nancial results. With these costs running 20 to
es m most comp · nent resources.
of financial . anies, quality costs should be a standard compo d
ana1ys1s The · f (es an Possibly too many resources being
selling costs • · impact o customer dissatisfaction on sa Low High
ts not easily ifi d storner applied. Reapply elsewhere,
satisfaction data could . quant ed, but analysis of sales an cu
In order to pro reld reasonable estimates.
financial measures per Y relate operational performance measures ;
how key process ' managers need to understand both systems as we ae

·th
5
- Low Low

if possible.
Low priority items.Address if time
I es Work Th'18 becaus and resources permit .
~ie current level of und~rs ~an be a demanding requirement, endy ....__
..........___
olhmg to brag about . ta nd mg of costs and operations is appar Id
seem the success rate o~o~grea_t many companies. If it were, it :i~ube Figure 7-1 O Importance - Satisfaction Priority Grid
CUiling efforts in many companies wo
....a........
136 Operational Performanc
e Measurement: Increasing Total Product· .
~ ty
• _____________A_na__ly!.:z:.:.:in.:!:g~a:.:.:nd::..'..'.ln~te:.'.Jrp!:'.'.r~et~in~g~M~e~a~su~re~s~•~.!_13~7

"'
evaluated from the broader perspecuv
. . must aIso be d e of th
Prionues 'd sub-optimization an to assur e resou e vieW can ke~p short-ten~ problems in perspective and prevent over-
to avo1 . rces
total companY f most return. As discussed earlier, what look are cting to mmor bumps m the road.
rea . . . 'd
allocated to ~e areafs om a departmental perspective may be immat like A case m point was a vice presi ent who was hyperventilating because
. rtant issue ro ena( ·
an 1mpo ocess By the same token, where comparative( in he saw a scrap rel'<:>rt that li~ted 200 parts that had been cut to the wrong
of the totaI pr · . Y1arg
terms be oduced with small investments, smaller problem e dimensions. When it was pointed out that the plant produced over 15 000
returns can pr s and
'ties should be captured. components that we~k and the plant manager had drastically red~ced
opi:~:ble period must be selected when determining priorities. If too scrap and rework during the past year, his attitude quickly changed. Rather
h rt a time frame is selected, the data sample may be too small to he sheepishly he said, "Well, 1 guess it's a good thing I didn't call Bill and
~e;resentative; if the time frame is too long, much of the data may he chew him out. Maybe I should be congratulating him for the improvements
too old to reflect current conditions. In most operating environments th he has made." Probably so, but that should have already happened several
last 1 to 6 months is usually the most meaningful, but it all depen~ e times during the past year.
the response of the production process and what changes have beon When reviewing performance measures, look at all the measures at
made to it. Priorities should be determined from data taken durin en the same level to see if they fit together and reflect what has been
.. 1 ga happening. Mixing lower level detail measures in with key performance
period that represents current cond1tions, norma operation, and is Ion
enough to provide a representative sample of data. g measures can lead to confusion and incorrect conclusions. The relative
importance of performance measures and their cause-effect relationship
Comparing Results Against a Forecast must always be kept in mind. It is usually possible to find some detail
measure that is exhibiting exceptionally good or bad performance, but
If managers understand how a process works, its current situation, and this cannot be the explanation for everything happening in a company.
what is being done to improve its performance, they should be able to Cause-effect relationships must be understood and performance meas-
forecast results with a reasonable degree of accuracy for the next 3 to 6 ures must be reviewed from that perspective to verify the changes in
months. Forecasts will always be too high or too low, but continually top level measures are explained by lower level measures. If there are
missing short-term forecasts by a wide margin indicates a poor under• any conflicts between the top level and detail measures, then further
standing of the process and/or the situation when the forecasts were investigation is required to resolve the discrepancy. Since mistakes can
made. When this happens, special efforts should be made by the respon· happen anyplace within the data processing chain, anything that doesn't
sible managers to understand what is causing the difference. Of course, make sense should not be accepted until it is either corrected or
unforeseen events could invalidate the forecast, but exceptions cannot explained.
happen all the time.
Team Consensus
LoOk At the Whole Picture , ,.a
Looking at the whole picture of performance in terms of breadth, depth,
The best safeguard for assuring proper mterpretatto
. . n of penorm
·ust a piece
and time will help assure correct interpretation, but there is no way to
measures, is to look at the whole picture of performance, 1.r behavior
or two. Businesses and their processes are very complex an t ;1
mance of
~arantee it. Given the same information, different people will reach
different conclusions because they have different knowledge and experi-
cannot be explained with only one or two variables. The per or ality or
ence. For this reason, using a small team of qualified individuals to discuss
any department or operating unit cannot be judged by th e qdu rs and
and reach a consensus on the meaning of performance measures is a
quantity
. of its
. outputs alone. The quality of work mpu . ts ' venkload' an d
suPPort services must also be considered, along with the wor 0
fOOd practice. This is best accomplished when the team members do not
relevant external factors. nds eel threatened by the performance measures and can feel free to be
~ nd ld and objective during their deliberations. A culture that encourages
. Look·
. ing_ at the whole picture, includes looking at longer-term tre
nd longer onesty, openness, and "telling it like it is," will help ensure performance
incon1unctton With most recent performance. Taking a broader a
measures are correctly interpreted.
138 Operational Performance Measurement: Increasing Total Productivity

SUMMARY

Get your facts first, then you can distort them as you please.
Mark Twain

Mark Twain makes a good point, especially for politicians. In business

however, the objective is to get the facts, interpret them correctly, and
take appropriate action.
Getting the facts, requires analyzing the performance measures and
their data. Moving averages, exponential smoothing, control charts, and
other statistical methods are useful for accounting for the normal variation
found in performance measures. Scatter diagrams, stratification, and regres-
sion analysis are effective ways to identify relationships, but these rela-
tionships may be causal or only casual. Any apparent relationship should
be regarded as coincidence until a logical connection has been identified
or otherwise established.
Other methods of analysis are making comparisons, looking at perfor-
mance from both a department and process perspective, zooming in and
out to see both the trees and the forest, and looking for opportunities as
well as problems. The objectives of analyzing performance measures and
data are to identify relationships, and to determine process capability,
work capacity, and priorities.
After getting the facts, they must be interpreted. Understanding how
a process works and the cause-effect relationships between process vari-
ables are basic requirements for properly interpreting performance meas-
ures. It is also important to understand the relationships between quality
and productivity; strategy, operational performance, and profits; and finan-
cial and operational performance measures. If these elements are poorly
understood, wrong conclusions will be drawn and faulty decisions will
be made.
Looking at the whole picture of performance will help assure correct
interpretation, but there is no way to guarantee it. Since different people
will interpret the same information in different ways, a team approach
can increase the likelihood of arriving at the right conclusion. Even if an
incorrect decision is made, the performance measures will send signals
that what was supposed to happen is not happening, and that changes
in strategy or action plans are required.