ASQ+Quality+Progress+ +feb+2019

FE B RUARY 2019 | QUALIT YPROG RE SS .
COM
Six Sigma Combined AS9100D:

Makes Metrics A Successful
Mining Help Gauge Transition
Safer Customer
Loyalty
|FEBRUARY 2019
page 24 page 52 page 58
The official publication of ASQ

LEAN AND SIX SIGMA
Follow this
18-step process
for lean and Six
Sigma portfolio
management
VOLUME 52 | NUMBER 2
REGISTER NOW
asq.org/lssc
Lean and Six Sigma This year’s sessions and

workshops will focus on:
in the Digital Age
• The “How” of Lean and Six Sigma:
Join ASQ to explore the challenges and Methods for Implementing Projects
opportunities presented by exponential and Initiatives
changes in the workforce, the workplace, • The Application of Lean and Six Sigma
and the markets organizations serve. in Nonmanufacturing Environments
This year’s dynamic program will offer • The Human Side of Lean and Six Sigma
best practices, tested solutions, and proven • Lean and Six Sigma Fundamentals
results for applying lean and Six Sigma • Advanced Content Masters Series
tools and principles in the digital age.
You can’t afford to miss it! ASQ members - save $200!
Register today: asq.org/lssc
CHECK OUT THESE
NEW BOOKS FROM
ASQ QUALITY PRESS!
THIS MONTH’S PICKS!
Data Quality: Risk Management Using
Dimensions, Failure Mode and Effect
Measurement, Strategy, Analysis (FMEA)
Management, and This guide is for those who want fast
information regarding failures and
Governance how to deal with them. It explores
Data are the foundation on the evaluation process of risk by
which organizations rest in utilizing FMEA. The intent is to make
this competitive age. This book the concepts easy to understand
balances technical details and and to explain why FMEA is used in
higher-level qualitative discussions many industries with positive results.
with case studies, illustrations, and real-world examples.
Item: H1553
Item: H1552
Learn more about these books through the

Quality Press bookstore at asq.org/quality-press.
GROW YOUR
LEADERSHIP SKILLS
ASQ’s Emerging Quality Leaders program develops executive knowledge, soft skills, and leadership
experience within a select group of rising quality professionals, utilizing both virtual and on-site
learning. This 12-month program gives participants the ability to embrace engaging and thought-
provoking challenges to serve as the next generation of quality and improvement leaders.
OPEN ENROLLMENT for the 2019-2020

APPLY TODAY! program ends on March 15, 2019.
Visit asq.org/emerging-quality-leaders
or contact emergingqualityleader@asq.org
for information.
FEATURES
16 18 Steps to Six Sigma Project Success 34 A Match Made in Maintenance Heaven
This step-by-step guide will help Six Sigma A case study showcases how one organization
practitioners align their projects to their from the oil and gas industry combined quality
organization’s business strategy and and maintenance management programs to
guarantees that the project stays headed achieve equipment reliability and drive its
in the right direction. maintenance philosophy.
by Marcelo Machado Fernandes, Jennifer by Ray Stoute
Hurst, Jiju Antony, João Batista Turrioni
and Messias Borges Silva 42 Slam Dunk
There’s a new way of using statistics and
24 Mining Matters analytics to identify undervalued professional
Six Sigma methods can provide a rock-solid basketball players and build a winning roster
way to help mining companies unearth in fantasy basketball leagues.
metrics and analytics to help make their by George Recck, I. Elaine Allen,
operations safer. Adam Kershner, Zachary Mittelmark
by Rajeev Chadha and Julia E. Seaman
INSIDE
February 2019 EXCLUSIVES
VOLUME 52 ● NUMBER 2 at qualityprogress.com
More on Mining
Additional figures to illustrate how Six
Sigma was used to assess the risks in
underground mining operations, the
topic of “Mining Matters” (pp. 24-33)
On the Radar
More graphics to supplement this
month’s Try This Today column
(“Risk on the Radar,” p. 64).
Digital Option
Experience QP in its digital format—just
another way to access the magazine
when you’re on the go.
qualityprogress.com ❘ February 2019 QP 3

INSIDE
February 2019
6 8 15 52
5 Foreword 13 Career Coach 60 Marketplace

Smarter selection. Carving out your ideal
career path. 62 Footnotes
6 Seen & Heard
15 My Quality Story 64 Try This Today
7 Expert Answers 5S and kanban sparks Risk priority and radar charts.
Statistics and stability big growth for a small
studies. business.
8 Progress Report 52 Statistics Spotlight

Getting in ship-shop Gauging customer loyalty
shape. by combining NPS and
PLUS CSAT metrics.
Getting to Know …
Jami Kovach 58 Standard Issues
The successful launch of
12 Mr. Pareto Head the AS9100D series of
standards.
13
Advertising Production Article Submission Process express permission of Quality

Barbara Mitrovic Quality Progress is a peer- Progress. For permission, write
reviewed publication with 85% Quality Progress, PO Box 3005,
Digital Production Specialist of its feature articles written by Milwaukee, WI 53201-3005, call
Julie Wagner quality professionals. To learn 414-272-8575 x7406, fax 414-272-
more about article submission and 1734 or email reprints@asq.org .
Editor in Chief and Publisher Design GLC the manuscript review process,
glcdelivers.com Photocopies and Reprints
Seiche Sanders visit www.qualityprogress.com Article photocopies are available
@ASQ_Seiche Media Sales Naylor LLC under “Tools and Resources” and from ASQ at 800-248-1946. To
866-277-5666 click on Author Guidelines. purchase bulk reprints (more than
Associate Editor Mark Edmund
@ASQ_Mark Editorial Offices 414-272-8575 Photocopying Authorization 100), contact Barbara Mitrovic at
Mail Authorization to photocopy items ASQ, 800-248-1946.
Assistant Editor Lindsay for internal or personal use or the
Dal Porto @ASQ_Lindsay Quality Progress/ASQ List Rentals
600 N. Plankinton Ave. internal or personal use of specific Orders for ASQ’s member and
Manuscript Coordinator Milwaukee, WI 53203 clients is granted by Quality nonmember buyer lists can be
Valerie Ellifson Progress provided the fee of $1 purchased by contacting Michael
Telephone Fax per copy is paid to ASQ or the
Copy Editor Amanda Hankel 800-248-1946 414-272-1734 Costantino at the Infogroup/Edith
Copyright Clearance Center, 222 Roman List Management Co.,
Email editor@asq.org Rosewood Drive, Danvers, MA 402-836-6626 or fax
Creative Services Manager
01923, 978-750-8400. Copying 845-620-1885.
Cathy Milquet
for other purposes requires the
Quality Progress (ISSN 0033-524X) is published monthly by the American Society for Quality, 600 N. Plankinton Ave., Milwaukee, WI 53203. Editorial and advertising
offices: 414-272-8575. Periodicals postage paid at Milwaukee, WI, and at additional mailing offices. Institutional subscriptions are held in the name of a company,
corporation, government agency or library. Requests for back issues must be prepaid and are based on availability: ASQ members $18 per copy; nonmembers $26 per
copy. Canadian GST #128717618, Canadian Publications Mail Agreement #40030175. Canada Post: Return undeliverables to 2835 Kew Drive, Windsor, ON N8T 3B7. Prices
are subject to change without prior notification. © 2019 by ASQ. No claim for missing issues will be accepted after three months following the month of publication of the
issue for domestic addresses and six months for Canadian and international addresses.
Postmaster: Please send address changes to ASQ, PO Box 3005, Milwaukee, WI 53201-3005. Printed in USA.
4 QP February 2019 ❘ qualityprogress.com

FOREWORD
The quality inside Quality Progress
Steering Your Selection

18 steps to wise project choices
Does anyone you know struggle with directional dyslexia? that ASQ also publishes a quarterly specialty title focused on lean
A condition where one is so confused by directions that they’re and Six Sigma? And that the magazine was recently re-launched
always making wrong turns and getting lost? Whose best friend with new departments and a new title? Packed with case studies,
is their GPS? If you’ve ever been there, you know it can be a help- tips and ideas you can use on the job, Lean & Six Sigma Review’s
less and frustrating feeling to have no sense of where you're going. introductory issue is open access for members to peruse and can
Even the most seasoned quality professionals can experience be found here: asq.org/pub/sixsigma/index.html.
a similar sense of disorientation when faced with a tangle of peo- In the age of Amazon, quicker, more accurate delivery of online
ple, processes and potential projects all vying for improvement purchases has become a customer requirement organizations
attention. The true success of a project often hinges on whether simply can’t avoid. Read about how these expectations are shap-
you chose the right thing to begin with. But the right road often ing customer loyalty in “Shape Up or Ship Out,” p. 8, and weigh in
isn’t obvious. via the Reaction Gauge questions posted on my.ASQ.org.
This month’s cover story is designed to put you on the right
track. In “18 Steps to Six Sigma Project Success,” p. 16, become
a project-selection pro by following the 18-step process that the
authors have devised to help inform lean and Six Sigma project
identification. The framework, synthesized from a multitude of
sources, will send you in the right direction regardless of whether
you have an established lean and Six Sigma program in your
organization, or are looking to launch one anew.
This month’s issue theme is lean and Six Sigma, a perennially Seiche Sanders
popular topic for many in the quality community. Did you know Executive Editor & Associate Publisher
TCC Daniella A. Picciotti, QMS Alliance QP Editorial Review Board

Directors Randy Brull, Chair
Edwin Garro PXS Performance Administrative Committee
Excellence Solutions Brady Boggs, Randy Brull, Larry Haugh,
Christianna Hayes Purple Innovations Jim Jaquess, R. Dan Reid, Richard Stump
Jane Keathley Keathley and Company,
LLC Technical Reviewers
James Kittredge Adaptimmune US Naveen Agarwal, Ashraf Ali, Suresh Anaganti,
ASQ’s Vision Scott Moeller GI Supply M. Onur Artan, Andy Barnett, Matthew Barsalou,
By making quality a global priority, an Luis Morales Verizon Telematics Inc. David Bonyuet, David Burger, Brooks Carder,
organizational imperative and a personal ethic, Nancy Nouaimeh Abela & Co Bernie Carpenter, L.N. Prabhu Chandrasekaran,
the American Society for Quality becomes the Susan Peiffer HSHS Sacred Heart Ken Cogan, Linda Cubalchini-Travis, Ahmad
community for everyone who seeks quality Hospital Elshennawy, Mark Gavoor, Kunita Gear, Daniel
technology, concepts or tools to improve Gideon Roth Cabiran Gold, T. Gourishankar, Roberto Guzman, Ellen
themselves and their world. Paulo Sampaio Universidade de Minho Hardy, Lynne Hare, Victoria Jones, Trevor Jordan,
Lars Sorqvist Sandholm Associates Ray Klotz, T.M. Kubiak, William LaFollette, Scott
ASQ Administration and the Royal Institute of Laman, Pradip Mehta, N.S. Narahari, Arind
CEO William J. Troy Technology Parthasarathy, Larry Picciano, Gene Placzkowski,
Wanda Sturm Bearded Fox Brewing Tony Polito, Peter Pylipow, Imran Ahmad Rana,
Senior Leadership
Company Narahari Vittal Rao, John Richards, James
Chair Benito Flores, Universidad De Monterrey
Vijai Venkata Daikin Rooney, Ayman Sakr, Manboubeh Samghabadi,
Chair-Elect Austin Lin, Google Allen Wong Abbott Nutrition Brian Scullin, Abhijit Sengupta, Amitava
Past Chair Elmer Corbin, IBM Corp. Sengupta, Mohit Sharma, A.V. Srinivas, Adrian
Administration Tan, Joe Tunner, B. Vaithiyanathan, Manu Vora,
Treasurer M ark Moyer, Moyer Consultant
William J. Troy, CEO Keith Wagoner, Jack Westfall, Doron Zilbershtein
Group, LLC
Seiche Sanders, Publisher
To promote discussion of issues in the field of quality and ensure coverage of all responsible points of view, Quality Progress publishes articles representing conflicting and
minority views. Opinions expressed are those of the authors and not necessarily of ASQ or Quality Progress. Use of the ASQ logo in advertisements does not necessarily
constitute endorsement of that particular product or service by ASQ.

SEEN&HEARD
Reader reactions from around the world
WORKING IN HARMONY
“Statistics Spotlight: The Music of the Spheres” (January 2019, pp. 44-45) is a great article. It
is an example of metaphor becoming model. I particularly appreciate the tune-up exercise. It
not only drives alignment, but it also permits errors as vehicles for learning to improve the
harmony. Thank you!
Andy Moysenko, Chelmsford, MA
IMPROVING PERCEPTIONS
“Standard Issues: Prove Your Worth” (December 2018, pp. 65-69)
tackles the main challenge faced by employees and the person
leading an organization’s quality assurance (QA) program. The
topics covered in the column call for an entire book. The column
THE REACTION GAUGE

could further be improved by citing an example of how an orga-
nization benefited from one of the approaches mentioned. The
attributes are a must-have for the QA department leader.
Raymond Medina, Santa Rosa, Philippines this month's question
ê
CORRECT CONCLUSIONS With the proliferation of two-day and same-day delivery

In response to “Statistics Spotlight: Asking the Right Question” options, online shoppers have come to expect faster, flexible
and more convenient shipping. So much so that customer
(September 2018, pp. 48-49): Good read! I always teach this to satisfaction depends on it—98% of people who shop online
my industrial engineering community here in the Philippines. say it affects their brand loyalty. What delivery experiences—
Felix Veroya, Batangas, Philippines good or bad—have impacted your satisfaction with a brand?
What new shipping options do you see for the future?
A GOOD GUIDE
In response to “Salary Survey: Influential and Impactful” (Decem-
ber 2018, pp. 16-24): Very detailed information for quality Join the discussion on myASQ at my.asq.org, or
managers, and it’s helpful in career development. on LinkedIn at www.linkedin.com/groups/3633.
Jitao Hu, Ningbo, China

Subject matter experts take on your quality-related queries
THIS MONTH'S QUESTION degradation line fitted on the stability data and on
the appropriate statistical interval.
Stability studies are Establishing and monitoring stability data across the product life cycle
an integral part of ensures product quality and minimizes patient risk. Hence, a systematic
any pharmaceutical approach must be adopted in the presentation and evaluation of the stability
information. The stability information should include, as appropriate, results
manufacturing system.
from the physical, chemical, biological and microbiological tests, including
Monitoring this data those related to attributes of the dosage form (for example, dissolution rate
across the life cycle of for solid oral dosage forms).
the product is equally The adequacy of the mass balance must be assessed. Factors that can
cause an apparent lack of mass balance must be considered, including, for
important. What
example, the mechanisms of degradation and the stability-indicating capa-
statistical tools should be bility and inherent variability of the analytical procedures.
part of a stability study? Where applicable, an appropriate statistical method should be
employed to analyze the long-term primary stability data. The purpose
of this analysis is to establish, with a high degree of confidence, a retest
OUR RESPONSE period or shelf life during which a quantitative attribute will remain within
acceptance criteria for all future batches manufactured, packaged and
Stability of a pharmaceutical product must stored under similar circumstances.
be established before seeking regulatory To provide a valid statistical inference for the estimated retest period
approval for commercialization. The pur- or shelf life, the statistical method used for data analysis must consider
pose of the stability study is to establish shelf the stability study design. The statistical evaluation of the pharmaceutical
life and to propose a retest period of a drug stability data must include regression analysis, analysis of variance, general
product. A complete stability study includes linear model and the computation of several statistical intervals (confidence,
a long-term stability study performed at prediction and tolerance).
intended storage conditions and an accel- Regression analysis is considered an appropriate approach to evaluating
erated stability study performed under the stability data for a quantitative attribute and establishing a retest period
stressed conditions (at higher temperatures or shelf life. The nature of the relationship between an attribute and time will
and relative humidity). determine whether data should be transformed for linear regression analysis.
The shelf life of a pharmaceutical product The relationship can be represented by a linear or nonlinear function on an
can be defined as the time the drug product arithmetic or logarithmic scale. In some cases, a nonlinear regression can
is expected to remain within the specifica- better reflect the true relationship.
tions defined for its purity and strength. However, nonlinear and other special linear models (square root, 1/x, log,
For an attribute known to decrease exponential, polynomials) generally are avoided unless specifically indicated
with time, International Council for Har- from the data, historical experience or a careful analysis of the residuals.
monization of Technical Requirements for Finally, stability is a key consideration when setting product specification
Pharmaceuticals for Human Use guidelines acceptance limits. Stability and rates of degradation must be compared to
state that the shelf life is calculated as the the upper specification limit and lower specification limit during early devel-
time at which the lower, one-sided, 95% opment and throughout the product life cycle.
confidence limit for the mean degradation
REFERENCE
curve intersects the acceptance criterion. 1. International Council for Harmonization of Technical Requirements for
For an attribute known to increase with Pharmaceuticals for Human Use (ICH), “Guidance for Industry: Q1A(R2) Stability
time, the shelf life is calculated as the Testing of New Drug Substances and Products,” ICH, www.fda.gov/downloads/drugs/
guidances/ucm073369.pdf.
time at which the upper confidence limit
intersects the acceptance criterion.1
Hence, the estimation of pharmaceuti- This response was written by Suresh Anaganti, CEO, Lab to Pharmacy LLC,
cal drug quality is based on the adequate Palmyra, PA.

A digest of
trends, research
& late-breaking
news
CUSTOMER SERVICE
Shape Up or
Ship Out Package delivery is playing a bigger role
than ever in customer satisfaction
The days of seven-day shipping are shoppers agree their loyalty to a brand is be able to correct a deliv-
long over. Today, it’s all about next-day influenced by the organization’s shipping ery address after placing an
or even same-day delivery. We live in a abilities. The survey also found that 84% order, and 53.2% thought they
time in which we can order something would cut ties with a brand if they had just should be able to redirect a
on Amazon.com at lunch and reasonably one bad delivery experience—a whopping package after it shipped.2
expect it to be waiting at our front door 34% increase from 2017.1 Meanwhile, when there’s a
when we get home from work. Not only do customers want faster delivery delay, 52% of customers
A 2018 survey conducted by Convey, delivery, but they also want more flexible expect the organization to make
a delivery experience management orga- delivery. For example, in 2018, nine up for it by refunding or dis-
nization, found that an astonishing 98% of out of 10 people thought they should counting the price of shipping.3

Alan Daniels has been named the new Technical Advi-
sory Group (TAG) 176 Quality Management and Quality
New @ ASQ Assurance chair. In addition, Susan Briggs will continue
to serve as chair of TAG 207 Environmental Management
What's on our minds
for the 2019-2021 term. TAGs 176 and 207 are responsible
for the U.S. voice in the revision of standards such as ISO
9001 and ISO 14001. If you are interested in joining a TAG,
please e-mail standards@asq.org.
Meeting demand
These factors have made the delivery world a lot “Modern e-commerce
more cutthroat in recent years. According to Con- retailers need a solution
vey’s chief marketing officer, Kirsten Newbold-Knipp, for delivery as fast as
Amazon is to blame. two hours.”
“Delivery expectations have clearly increased —Andrew Chung
year-over-year as Amazon continues to raise the bar
and customers take the bait,” said Newbold-Knipp.
“Today, simply tracking a package isn’t enough.
Retailers and brands that want to thrive need to
invest in people, processes and tools that positively
impact last-mile delivery and customer loyalty.” 4
With increased customer expectations and higher- UPS and FedEx see rates rise by 32%. Rates
than-ever competition among retailers, how do No matter how quickly a for FedEx’s similar Smart-
organizations meet customer demands? According retailer packages and ships Post arrangement are also
to Andrew Chung, a journalist, the answer involves an order, part of the process going up. The USPS also
that last mile. still depends on the mail raised its rates this past
“Modern e-commerce retailers need a solution for delivery service. Two of those October, with its Parcel
delivery as fast as two hours,” Chung said. “Last- services—UPS and FedEx— Select package rates rising
mile warehouses facilitate the movement of goods in have recently imposed a 4.9% between 9% and 12%.”10
the supply chain to the final destination. Being closer average rate hike for 2019. That adds up quickly—in
to the consumer decreases supply-chain costs while The carriers have imposed the 2018, UPS alone averaged 31
minimizing the time to complete delivery.”5 same rate increase every year million deliveries each day.11
With 41% of the U.S. retail market share, Amazon for the past 10 years. How- Although the rate increases
has set a precedent for fast delivery.6 It’s no longer ever, it’s targeted this year at won’t be detrimental to
a luxury, but an expectation, which makes it increas- residential customers, and the organizations like Amazon—
ingly difficult for smaller retailers to keep up. increases could significantly which charges an annual fee
“While consumer expectations around delivery impact free shipping.9 for free shipping—smaller
are rapidly increasing, their delivery experiences Journalist Rich Duprey organizations will feel the
are declining,” Chung said. “Fast delivery is import- reported that what will impact impact. According to Spend
ant to 99% of U.S. consumers when making online free shipping the most is the Management Experts, a trans-
purchases. At the same time, an increasing number rate hike UPS imposed on portation spend management
of consumers feel frustrated with the lack of profes- SurePost, which is used in col- consultant, Parcel Select,
sionalism and accuracy when having items delivered. laboration with the U.S. Postal SmartPost and SurePost are
For retailers looking to exceed expectations, there Service (USPS) for faster last- the services retailers use most
is a large opportunity to capture customer loyalty mile deliveries. for free shipping.12
when delivery is done right.”7 “For packages over one
Despite these challenges, there are some things pound, rates are rising by The future of delivery
organizations can incorporate into their last-mile 9%; for those under a pound, So how can retailers afford to
facilities that can help, such as locating logistics they’re going up 9.34%,” offer fast and free delivery?
facilities near major roadways and cities, and Duprey said. “Surcharges The answer is technology.
ensuring the facilities can adequately house the to certain rural or diffi-
organization’s products.8 cult-to-access locations will continued on page 10 

Shape Up
Or Ship Out H E A LT H C A R E
 continued from page 9

Unnecessary
Medical Services
Many organizations are experimenting with using
autonomous vehicles and drones to deliver packages.
Last year, for example, Amazon was issued a pat-
ent for a delivery drone the organization hopes will
improve delivery reliability and speed. The retailer’s Cost $200 Billion
In United States
goal is to be able to deliver packages in 30 minutes.13
Walmart has worked with several organizations,
including Waymo and Ford, to pilot grocery delivery
programs. In its current partnership with Udelv, the
organizations have established online grocery deliv-
ery in about 100 metropolitan areas.
Each Year
“We want to make sure we stay on the cutting Overuse of low-value, unnecessary services costs the U.S.
edge of grocery delivery by exploring what’s new healthcare system more than $200 billion each year, according
and next,” said Tom Ward, senior vice president of to an estimate included in research by the Joint Commission.
Walmart’s digital operations in the United States.14 The estimate is part of a new study that was released last
These technologies are relatively new and still in the month detailing a commission initiative to reduce unnecessary
experimental phase, but they offer a promising future care and raise awareness among clinicians and the public about
for the fast, free delivery we’ve all come to expect. issues surrounding healthcare overuse.
—compiled by Lindsay Dal Porto, assistant editor The campaign, called “Choosing Wisely,” aims to identify and
measure low-value care for patients not only to save money, but
REFERENCES
1. “Convey Survey: Last Mile Delivery—What Shoppers Want also to avoid potential harm to patients and inconvenience of
and How to #SaveRetail,” BusinessWire, Nov. 7, 2018, unnecessary care.
https://tinyurl.com/ydgyjs9b. For more from the study, released last month, visit https://tinyurl.
2. Krista Garcia, “Online Shoppers Want More Control Over com/joint-comm-choose-wise.
Deliveries,” eMarketer, Nov. 7, 2018 https://tinyurl.com/
yapaf54s.
3. “Convey Survey: Last Mile Delivery—What Shoppers Want
and How to #SaveRetail,” see reference 1.
4. Ibid. ASQ
5. Andres Chung, “The Importance of Last-Mile Facilities
ASQ Sells Shares

in the Supply Chain,” SupplyChainBrain, Dec. 19, 2018,
https://tinyurl.com/ybq9k2k9.
6. Ibid.
In ANAB to ANSI
7. Ibid.
8. Ibid.
9. Rich Dubrey, “Will UPS and FedEx Rate Hikes Doom
Free Shipping?” Fool, Jan. 1, 2019, https://tinyurl.com/
yde3ys24. ASQ sold its share of ownership in the ANSI-ASQ National
10. Ibid. Accreditation Board LLC (ANAB) to American National Stan-
11. FreightWaves “Parcel Carriers Shine During Peak Delivery dards Institute (ANSI) in late 2018.
Period, Firm’s Data Shows,” Benzinga, Jan. 7, 2019, https:// ANAB had been owned equally by ANSI and ASQ since 2005.
tinyurl.com/ybexlfnd. The purchase results in ANAB becoming a wholly owned subsid-
12. Dubrey, “Will UPS and FedEx Rate Hikes Doom Free iary of ANSI, registered as a separate legal entity, and renamed
Shipping?” see reference 9. the ANSI National Accreditation Board (ANAB).
13. Hamza Shaban, “Amazon Is Issued Patent for Delivery While ANSI and ANAB will continue business as usual in the
Drones That Can React to Screaming Voices, Flailing
near term, ANSI intends to move its accreditation of conformity
Arms,” Switch, March 22, 2018, https://tinyurl.com/
y7hh2vl2. assessment bodies into ANAB under the auspices of ANSI own-
14. Lauren Thomas, “Walmart Taps Udlev for Latest ership, creating a single entity to lead accreditation services.
Driverless Car Tests to Deliver Groceries,” CNBC, Jan. 8, For more information about ANSI and the ANSI National
2019, https://tinyurl.com/y6w5sgfr. Accreditation Board, visit www.ansi.org/accreditation/anab.

GETTING TO KNOW…
Jami Kovach
QUALITY-RELATED current position education
NEWS FROM AROUND Associate professor at the Uni-

versity of Houston (UH).
Doctorate in industrial engineer-
ing from Clemson University in
South Carolina.
THE WORLD What was your World Conference on Recent awards or
—Powered by LexisNexis Quality and Improve-
introduction to ment (WCQI) and honors?
quality? UH’s student branch
Top 10 Most Reliable Automakers councilor for ASQ’s I recently became an
Greater Houston ASQ fellow.
This year’s Consumer Reports Car Reliability Survey A quality class I took Section.
collected data from over 500,000 members to in college, which led
to a summer intern-
determine what problems vehicle owners can ship and then to my Personal:
What notewor-
expect. Problems range from troublesome engines first job in industry.
thy activities or Married.
to infuriating infotainment systems, transmission achievements
issues and other troubles. Read more about the Is there a teacher
outside of ASQ do What are your
top 10 automakers here: https://tinyurl.com/ who influenced
you participate in? favorite ways to
top-ten-automakers. you more than
others? relax?
I’m an Academician
in the International
GM to Cut Thousands of Jobs, My dissertation advi- Academy for Quality. Binge watching shows
Close Plants In U.S. and Canada sor, Rae Cho, because I received the 2017 on Netflix.
he taught me how to Outstanding Educator
General Motors (GM) is poised to end production at publish my research. Award from the
five plants in the United States and Canada, kill off Southwest Decision What books are
several passenger cars and slash 15% of its salaried
Sciences Institute. you currently
What’s the best And I was part of
reading?
workforce in a sweeping cost-cutting plan designed career advice
the team that was
awarded second
to boost profits and adjust to changing tastes in you’ve ever place in the Project I’m currently reading
vehicles. For more information about the changes, received?
Management Insti- Michelle Obama’s
tute’s 2016 Teaching Becoming. One of my
visit https://tinyurl.com/GM-cuts-jobs. Case Competition. favorite authors is
Work hard, play hard. Malcolm Gladwell.
Amazon Customer Data Was Have you had any-
Leaked Ahead of Black Friday Previous note- thing published? What was the last
Amazon experienced an embarrassing data lapse worthy jobs? movie you saw?
as it headed into one of its busiest periods on Black I’ve published more
Working as a product than 50 articles “Feminists: What Were
Friday. The e-commerce giant emailed customers on and process improve- in a wide range of They Thinking?” It’s a
Nov. 21 to tell them their names and email addresses ment engineer for peer-reviewed jour- documentary looking
several years in the nals that describe my back at feminism
had been leaked on its website, due to a “technical U.S. textile industry research studying the and how the times
error.” Read the full story here: https://tinyurl. before getting my application of process are different, but not
Ph.D. improvement and necessarily better.
com/Amazon-data-leak. process design/rede-
sign methods through
Are you active case studies con- Quality quote:
ducted in industries
in ASQ? including animal care,
To get a roundup of the week’s most noteworthy “Providing value
banking, construction,
stories delivered to your inbox every Friday, requires cultivating
I’m currently serving healthcare, juvenile
subscribe to the QNT Weekly e-newsletter at justice, logistics a relentless pursuit
as editor of Lean & Six
asq.org/newsletters. and transportation, of excellence.”
Sigma Review, chair of
the technical program manufacturing and
committee for ASQ’s oil and gas.

News Briefs
The American Society for have made to improve
Nondestructive Testing will healthcare quality. Visit
hold its annual research https://nahq.org/about/
H E A LT H C A R E
symposium April 1-4 in healthcare-quality-week
GROUP LISTS 118

Garden Grove, CA. The four- in the coming months for
day event addresses key updates and details on the
TOP U.S. HOSPITALS research, development and

innovation of advancing non-
weeklong celebration.
The Leapfrog Group, a national nonprofit health- destructive evaluation (NDE) The International Accred-
care advocacy group, recently released its annual technologies, and bridges itation Forum (IAF) just
list of top U.S. hospitals based on safety and quality gaps in NDE research, finished celebrating its 25th
measurements. engineering and technology anniversary last year. IAF
In 2018, a total of 118 hospitals were recognized transfer. Find more details at is the world association
because of performance in different areas of hos- www.asnt.org. of conformity assessment
pital care, including preventing infections, reducing accreditation bodies and
C-sections, using technology to ensure safer care, The 24th annual American other bodies interested in
and instituting leadership policies and practices. Productivity and Quality conformity assessment in
“Leapfrog is proud to recognize the recipients of Center Knowledge Manage- the fields of management
our most elite award for safety and quality,” said ment Conference will be held systems, products, services,
Leah Binder, president and CEO of the Leapfrog April 29 to May 3 in Hous- personnel and other similar
Group. “We’re encouraged by the hard work of top ton. Visit www.apqc.org/ programs of conformity
hospitals, as well as all of the hospitals that compete apqcs-2019-knowledge- assessment. It was founded
for this award. Their transparency and determination management-conference in January 1993 by repre-
delivers the best possible care in their communities.” for details on speakers and sentatives of accreditors of
The states of Florida, California, New Jersey and the more than 20 practi- quality system certification
Texas were each home to 12 or more hospitals that tioner-led sessions. bodies from Australia, New
received the Leapfrog distinction. Zealand, Canada, Japan,
The National Association for Mexico, the Netherlands,
Healthcare Quality has set the United Kingdom and the
For a complete list, visit https://tinyurl.com/ Oct. 20-26 as Healthcare United States. ANAB (then
leapfrog-top-hosp.
Quality Week to mark the known as RAB) was among
contributions professionals the founders of IAF.
Mr. Pareto Head By Mike Crossen

Advice to advance your career
E D U C AT I O N
What’s Your
Hedgehog?
How to determine your ideal career path by Darrell Baldock
I consider myself an expert when it comes to career devel- its day searching for food and maintaining its home.
opment. Why? Because my definition of an expert is someone When the fox attacks, the hedgehog rolls up into a ball
who has tried and failed several times. Let me explain: and is protected by its sharp spines. The fox retreats to
Although I have had my share of successes, my career path the forest to plan its next attack. This becomes a daily
wasn’t a straight line and at times wasn’t very fulfilling. It left ritual and the outcome never changes—the hedgehog
me wanting more. It was good, but it wasn’t great. always wins. The hedgehog isn’t stupid. In fact, just the
In his book Good to Great,1 Jim Collins talks about the opposite—it understands the brilliance of simplicity.
hedgehog concept, which states that the world is made up In his book, Collins shows how the success
of hedgehogs and foxes. The fox knows many things, but the of some great organizations is based on simple
hedgehog knows one big thing. concepts. We can apply the hedgehog concept to
The fox uses its cunning and speed to surreptitiously attack career development by constructing a work life that
the hedgehog. But the hedgehog is a simple creature, spending meets three tests:

1. Best in the world: Are you doing work for which you have FIGURE 1
a God-given talent and perhaps could become one of the
best in the world (baby steps—maybe we start with your
local geography) in applying that talent? Do you feel like you
Circles of the
were born to do it? Think about your days in grade school
and the subjects you liked or the ribbons you won. This hedgehog concept
should give you a few clues.
For example, I loved to read and preferred projects over
tests. I won a couple of public speaking contests and was Best
a finalist in the Ontario Greater Windsor science fair two Passion in the
world
years in a row. Do
But I ignored these clues while I stumbled through high this!
school and instead took courses I didn’t like. I ended up
getting a business degree, majoring in finance, even though
math was my least favorite subject. Economically
What clues are you ignoring? viable
2. Economically viable: Can you be paid well for what you do?
We should all enjoy the experience of thinking, “I get paid to
do this? Am I dreaming?”
I remember working on a project where we flew in a con-
sultant from Great Britain. He was the global expert for the opportunity to speak to groups to show them
inventory management module. We paid him an outrageous how lean principles can change lives. And
sum for a few days’ work. He earned it because he was the although it is still early in my consulting career,
subject matter expert. Meanwhile, I was the generalist project I am confident that I can make a living by influ-
manager, earning a fraction of that amount. encing others to adopt and implement LSS. Every
Is there a need in the marketplace for your skill set or day I wake up excited because there is so much
a niche you can exploit? unrealized potential. Now more than ever, we
3. Passion: Are you doing work you’re passionate about and need this approach in business, government
absolutely love to do? Do you look forward to getting up and even in our personal lives.
every morning and throwing yourself into your work, and If you could drive toward the intersection of these
believe in what you’re doing? three circles and translate that intersection into a
Later in my career, I pursued a career in law. I volunteered simple concept that guides your life choices, then
at the local courthouse and worked with the duty counsel. you’d have your hedgehog—the job you should work
The duty counsel worked in the courthouse all day, and then toward (see Figure 1).
went home at night and read law articles until the wee hours I hope my experiences and the hedgehog concept
of the morning. He told me the laws and precedents were help you make the right choices as you head down
always changing, and he needed to stay current. I liked law, your career path. Don’t settle for a good career
but quickly realized I would never have his passion for it. when you can have a great career.
However, I do have a passion for continuous improvement
and can remember reading books by Wayne Dyer and Denis
REFERENCE
Waitley in university. 1. Jim Collins, Good to Great: Why Some Companies Make
Today, my personal library includes books by Tim Ferris, the Leap and Others Don’t, HarperBusiness, 2001.
Tony Robbins, Stephen Covey, Ray Dalio and Dale Carnegie,
among others. I could spend every waking hour reading and
practicing new ways of doing things.
This is where lean Six Sigma (LSS) comes in. I received my
Darrell Baldock is the
first LSS training about 10 years ago. I progressed through director of Lean Canadian in
Yellow, Green and Black Belts, and recently became a certi- Ontario, Canada. He earned
fied Master Black Belt. I finally found my mission in life. a master’s degree in business
strategy from the University
LSS has allowed me to learn from masters such of Windsor in Ontario, and is
as Taiichi Ohno and W. Edwards Deming. I take every a member of ASQ.

Personal stories from quality professionals
LEAN AND SIX SIGMA
Clean-Up
Time by Kate Jakubas
How 5S and kanban helped a small business

grow fourfold in just two years
Quality knowledge isn’t widely considered a critical startup skill, but lean Continuous improvement
and Six Sigma helped me grow my manufacturing business over an order of and continuous growth
magnitude in two years. In May, we moved from our original
I founded Meliora Cleaning Products in 2013 after a decade of working for 1,500-square-foot factory to a space four
larger manufacturing organizations. During that time, I learned about and used times bigger, and we are fulfilling cus-
lean and Six Sigma, and became an ASQ-certified Green Belt and Black Belt. The tomer orders 10 times larger in volume
certifications were mostly an academic exercise. They ticked the “professional than those just two years ago.
development” box and gave me something to feature on my résumé. Occasion- We recently had the pleasure of tak-
ally, I applied Six Sigma or lean tools to a project, saving the organization money ing personal friends who also are lean
or improving quality. manufacturing engineers on a tour of our
Now that I own a small business, eliminating waste and improving quality isn’t plant. We are now implementing further
just an interesting exercise—it’s critical to the health of my business and paying changes based on their observations.
my employees on time. I use my quality training every day. Without a large staff to conduct internal
audits, we are lucky to have outsiders
5S and kanban evaluate our processes so we can con-
The tool and method that have helped me the most are 5S and kanban. tinue improving.
In our factory, 5S means keeping things as orderly as possible and making We will continue to make improvements
sure everyone knows where everything is. A bilingual production staff has been and look for ways to implement the best
especially helpful in encouraging us to reduce the number of verbose written parts of big-organization quality at our
procedures in favor of photos, color coding and other visual factory manage- tiny manufacturing facility.
ment systems.
Kanban is our system for keeping product flowing. We do the bare minimum
when it comes to scheduling production and don’t forecast based on anticipated
product mix. Instead, we circulate labeled trays. Each finished product is stored
Kate Jakubas is the
on labeled trays. Trays are emptied to fill customer orders, and an empty tray founder of Meliora
indicates that it’s time to make more of that product. This allows us to use our Cleaning Products
time most effectively and only make enough product to keep up with current in Chicago, and an
advisor to William
customer orders. Hooper Consulting
Our system is extremely basic, but also extremely effective. We improve our in Indiana. She earned a master’s degree
systems by adding, subtracting and adjusting kanban amounts. Rarely is there in environmental engineering at the Illinois
Institute of Technology in Chicago. Jakubas
a day that colored electrical tape isn’t used to identify a storage area or product- is a member of ASQ and an ASQ-certified Six
specific piece of equipment. Sigma Green Belt and Black Belt.

F E AT U R E
SIX SIGMA
Just the
Facts
Quality profes-
sionals tend to
struggle with
selecting and
prioritizing Six
Sigma projects.
Recognizing the
opportunity, the
authors created a
Six Sigma project
portfolio manage-
ment model to
help quality pro-
fessionals select
appropriate proj-
ects and ensure
their success.
The 18-step model

isn’t limited to any
specific indus-
try or program
maturity level.

Plan your route for
projects that deliver
| by Marcelo Machado Fernandes,
Jennifer Hurst, Jiju Antony,
João Batista Turrioni and
Messias Borges Silva

F E AT U R E
SIX SIGMA
Selection and prioritization of Six Sigma projects (SSP) is to the CEO’s staff.5 For example, MBBs should be
one of the most frequently discussed issues in Six Sigma sitting at the table with senior executives to properly
literature. (“Six Sigma literature” refers to the references listed understand their voices and translate them into
at the end of this article.) If an SSP is poorly selected, positive significant projects, as well as walking the processes
results are delayed and frustration may build throughout the (gemba walk) to collect improvement opportunities
organization. This is one of the main factors separating success- straight from the shop floor.
ful SSPs from other types of improvement projects.1,2 Step two is focused on the MBB filtering out
To explore this research opportunity, the project portfolio improvement initiatives that clearly have no link to
management (PPM) theory was used to cover not only project the organization’s business strategy,6 such as those
selection, but also other important elements of an SSP, such as not associated with common and unknown causes,7
project prioritization, resource allocation, constant performance as well as immeasurable problems8 and problems
monitoring, risk management and project portfolio balancing. better solved with a different method.
The dynamism created by PPM is particularly important for To illustrate this topic, let’s look at an example
Six Sigma. SSP selection must be a dynamic process,3 so it is from a global automotive organization. The orga-
useful to identify a fragment of PPM theory and use it—along nization was facing a quality outbreak related to
with the Six Sigma literature that discusses criteria for project the diameter of copper cables in a wiring harness
selection—to build a conceptual model for Six Sigma project production line. The plant manager suggested open-
portfolio management (SSPPM). ing a lean Six Sigma project to contain the problem.
Proposed here is a framework for SSPPM that can be used In this type of emergency, the recommendation is to
by any Six Sigma practitioner aiming to maximize the chances act using a task force or immediate kaizen approach
of not only selecting the right projects, but also guaranteeing rather than the structured define, measure, analyze,
they are aligned with business strategy, properly progressing improve, control method.
to deadlines and reaching the organization’s expected results. Step three is to classify the PSSPs. The most
common categories are Green Belt (low-medium
SSPPM model complexity) and Black Belt (medium-high com-
Many articles from Six Sigma literature were used as a starting plexity) projects.9 Any other justified form of
point for the SSPPM model. However, it is important to rein- categorization can be used, such as product line,
force that our collective practical experience also contributed customers affected or departments involved.10
significantly to designing this 18-step SSPPM framework. If the organization has a Yellow Belt (YB) pro-
Figure 1 (p. 20) presents the SSPPM conceptual model, which gram, low-complexity projects can be conducted
is structured in three phases: the main process (steps one using the SSPPM method. Nielsen Media Research
through 12), auxiliary process one (steps 13 through 16) and (Nielsen), the global market research organization,
auxiliary process two (steps 17 and 18). Table 1 depicts the con- implemented this approach, and a YB wave recently
nection between the main steps and components of the SSPPM took place in the organization: More than 20 associ-
model and the Six Sigma literature. ates were certified as YBs, and six projects brought
Main process. Step one of the main process is the potential significant improvement to the organization in terms
SSP (PSSP) continuous screening. The Master Black Belt (MBB) of quality and cycle time.
or Six Sigma program leader must provide and coordinate differ- Step four of the model must be carried out by the
ent ways of identifying new continuous improvement initiatives Belt who will lead the project. At this stage, the Belt
to be evaluated as potential new improvement projects.4 should consider information related to the levels and
The MBB should avoid portfolio biases by drawing on sources behavior of the problem to be analyzed. At this step,
that are as varied as possible, from workers on the shop floor the project manager also should estimate resources
required to develop the project, as well as the base-
line cost associated with not solving the problem.11
At minimum, the size of the problem should be esti-
If a Six Sigma project is poorly selected, positive mated or, in some cases, the baseline stability should
results are delayed and frustration may build be evaluated and the initial baseline sigma level
throughout the organization. should be estimated.

TA B L E 1
SSPPM literature
Cooper, Edgett and Kleinschmidt (1997)

Project Management Institute (2008)
Bengt, Wiklund and Edgeman (2001)

Lynch, Bertolino and Cloutier (2003)
Smith, Blakeslee and Koonce (2002)

review summary
Archer and Ghasemzadeh (1999)
Jones, Parast and Adams (2010)

Kumar, Antony and Cho (2009)
Coronado and Antony (2002)
Snee and Rodebaugh (2002)
Harry and Schroeder (2000)

Bertels and Paterson (2003)
Tanik and Sen (2012)

Goh and Xie (2004)
Cooper (1994)
Cheng (2009)
Authors
SSPPM steps
Potential Six Sigma project continuous X X
screening
Potential Six Sigma project pre-evaluation X X X
Potential Six Sigma project categorization X X
Potential Six Sigma project detailed X
information gathering
Potential Six Sigma project evaluation X X X X X X X
(individual)
Potential Six Sigma project evaluation X
(interaction)
Potential Six Sigma project prioritization X
Six Sigma project portfolio balancing X
Six Sigma project portfolio resource X
allocation
Six Sigma project portfolio graphical X
representation
Six Sigma project portfolio final adjustments X
Six Sigma project portfolio presentation X
Business strategy evaluation X
Significant change in business strategy X
evaluation
Six Sigma project portfolio selection criteria X
(and criteria weights) reevaluation
Current Six Sigma project portfolio X
reevaluation
Six Sigma project portfolio performance X X X
evaluation
Feedback on Six Sigma project portfolio X X
according to its performance
SSPPM = Six Sigma project portfolio management

This list of authors and their works can be found in this article's list of references.

F E AT U R E
SIX SIGMA
FIGURE 1 Step five is an individual evalua-
Conceptual model for SSPPM

tion of each PSSP, which should be
performed by the MBB.12 The indi-
vidual evaluation is based on criteria
Auxiliary process 1 Main process Auxiliary process 2 found in the Six Sigma literature: link
to customer requirements,13 link to
13 1 17 business strategy,14 potential finan-
Business Potential Six 7 Six Sigma cial return,15, 16 available resources
strategy Sigma project Potential Six project portfolio
evaluation continuous Sigma project performance considering the project scope17
screening prioritization evaluation and potential for ending in a short
period of time.18
14 2 8
Potential Six Six Sigma 18 Step six involves evaluating
No Significant Sigma project project portfolio No Yes PSSP interactions by investigating
change? pre-evaluation balancing Positive
performance? interdependences among PSSPs.
3 9 This step, emphasized by authors
Yes Potential Six Six Sigma Norman Archer and Fereidoun
Sigma project project portfolio
15 categorization resource Ghasemzadeh,19 outlines the
Six Sigma allocation possibility to present a project that,
project portfolio 4
selection on its own merit, has low priority
Potential Six 10
criteria and Sigma project Six Sigma but enables another initiative.
criteria weights detailed project portfolio
reevaluation Ultimately, it is a project that
information graphical
gathering representation will significantly contribute to a
16
high-priority project. For steps five
Current 5 11
Six Sigma Potential Six Six Sigma and six, a multicriteria matrix similar
project portfolio Sigma project project to the classic Pugh matrix is used
reevaluation evaluation portfolio final
(individual) adjustments to prepare the initial or preliminary
list of SSPs. At this point, any tools
6 12 reviewers can use to clearly check
Potential Six Six Sigma
Sigma project project portfolio the effect of each project (individu-
evaluation presentation ally and collectively) are welcomed.
(interaction)
Step seven consists of prioritiz-
ing PSSPs and creating a list of all
initiatives that includes information
SSPPM = Six Sigma project portfolio management
related to the criteria used in
steps one through six. This is
an important task in support-
Management can rebalance a new portfolio based ing portfolio balancing.20 It
on how well current projects are performing. is important that proper
visibility is given to the
list of projects. Nielsen
is now using Google
Suite (docs, slides
and sheets) instead

FIGURE 2 officially presenting it to the
Nielsen Six Sigma project portfolio

executive team.24
Step 12 is the SSPP
presentation and the final
Green Belt Black Belt
step in this phase of the
Act now model. It represents the
9
portfolio’s formalization to
8
everyone involved,25 includ-
7 ing the Belts, project team
6 members, sponsors, cham-
pions and stakeholders.
Impact
5
After steps 11 and 12, the
4 Avoid
Simple lean proper potential changes
concepts/tools 3 must be implemented in the
2 bubble chart.
1 According to the Six Sigma
0
literature, SSPPM is dynamic
0 1 2 3 4 5 6 7 8 9 in many ways. For example, it:
Complexity ++ Continuously evaluates
Latin America India SEANAP Europe
improvement initiatives.
++ Allows for terminating or
Greater China NA East NA West halting current projects.
++ Alters criteria or how crite-
ria are weighed according
NA = North America SEANAP = South East Asia, North Asia and Pacific to changes in business
Note: This model was adapted from Nielsen Media Research.
strategy.
Management can rebalance
of Microsoft Office, so the fact that files to reinforce that portfolio management a new portfolio based on how
can be easily shared with other associates is dynamic, so at any time, additional well current projects are per-
increases the chance of proper awareness resources can be added or reduced forming. Therefore, portfolio
on the first version of the Six Sigma project according to the portfolio’s performance. communication should not
portfolio (SSPP). Step 10 is the graphical representation be the last step in the main
Step eight is related to portfolio balanc- of the SSPP. The objective is to offer the process. Rather, it is a task
ing. It helps the MBB decide which projects best view of the project’s characteristics, that should be performed
will receive the organization’s financial scope, risk and potential benefits. Often, whenever the portfolio under-
resources, per category.21 It primarily con- it takes the format of a bubble chart, with goes a modification. Indeed, if
siders the organization’s strategic needs. each project represented with bubbles someone changes a portfo-
This is the point at which the word “poten- in a two-dimensional scenario. The axes lio’s structure without giving
tial” is dropped and the project is referred can be complexity and effect (potential proper notice to everyone
to thereafter as an SSP. Indeed, this step gains), for example. The size and color of involved, the expectations of
is where the first version of the portfolio is the bubble also may have meaning, such the project team and stake-
presented to the organization. as region, development time or resources holders may diverge, resulting
Step nine refers to resource allocation to involved.23 Figure 2 is a sample SSPP in a poor contribution to
develop SSPs. The resources to be allocated bubble chart adopted from Nielsen. It project development from
should not only be human but also any other uses different shapes and colors to refer champions and sponsors.
important resources,22 such as machines, to different regions around the world. Auxiliary process one.
labs and special materials. It also means Step 11 lays out the adjustments to the In steps 13 to 16, all quality
that, conceptually, the resources initially portfolio. After defining the initial version, initiatives must be linked to
allocated to conduct SSPs should not be risk analysis and graphical representa- the business strategy.26 It is
displaced during project development to tion, the MBB can, if necessary, adjust the up to the MBB to understand
conduct day-to-day activities. It is important portfolio to optimize the final figure before and regularly monitor the

F E AT U R E
SIX SIGMA
organization’s business strategy and identify any significant REFERENCES

1. Thomas Bertels and George Patterson, “Selecting Six Sigma Projects
changes that could affect the SSPP in any way. If there is
that Matter,” Six Sigma Forum Magazine, Vol. 3, No. 1, 2003, pp. 13-17.
significant change in the business strategy, the criteria and 2. Jiju Antony and Ricardo Banuelas, “Key Ingredients for the Effective
criteria weight must be reevaluated and, consequently, the Implementation of Six Sigma Program,” Measuring Business
Excellence, Vol. 6, No. 4, 2002, pp. 20-27.
SSPP must be reevaluated.27 This is just one more reason
3. Murat Tanik and Ali Sen, “A Six Sigma Case Study in a Large-Scale
why MBBs should be seated at the table with senior lead- Automotive Supplier Company in Turkey,” Total Quality Management
ership: The further apart they are, the more likely the SSPs & Business Excellence, Vol. 23, No. 3, 2012, pp. 343-358.
4. Bertels, “Selecting Six Sigma Projects that Matter,” see reference 1.
will become irrelevant.
5. Robert G. Cooper, Scott J. Edgett and Elko J. Kleinschmidt, Portfolio
Auxiliary process two. In steps 17 and 18, the MBB must Management for New Products, McMaster University Press, 1997.
assess the performance of the SSPP. The objective of this 6. Bertels, “Selecting Six Sigma Projects that Matter,” see reference 1.
7. Ronald D. Snee and William F. Rodebaugh Jr., “Frontiers of Quality:
phase is to collect information to support decisions related to
The Project Selection Process,” Quality Progress, September 2002,
a new portfolio balancing, as well as to exclude or continue pp. 78-80.
projects, or place them on hold.28-30 8. Donald P. Lynch, Suzanne Bertolino and Elaine Cloutier, “How to
Scope DMAIC Projects,” Quality Progress, January 2003, pp. 37-41.
Nielsen, for example, implemented three regular gate
9. Ricardo Banuelas Coronado and Jiju Antony, “Critical Success
reviews, a project charter at the beginning of the Six Sigma Factors for the Successful Implementation of Six Sigma Projects
journey, a mid-project review to analyze causes and potential in Organizations,” TQM Magazine, Vol. 14, No. 2, 2002, pp. 92-99.
10. Robert G. Cooper, “Third-Generation New Product Processes,”
solutions, and a final review to evaluate whether projects
Journal of Product Innovation Management, Vol. 11, No. 1, 1994,
were ready for certification. pp. 3-14.
11. Norman Archer and Fereidoun Ghasemzadeh, “An Integrated
A successful SSP Framework for Project Portfolio Selection,” International Journal
of Project Management, Vol. 17, No. 4, 1999, pp. 207-216.
This 18-step SSPPM model serves as a guide for all Six 12. Maneesh Kumar, Jiju Antony and Byung Rae Cho, “Project Selection
Sigma practitioners who want to guarantee their SSPs are and its Impact on the Successful Deployment of Six Sigma,” Business
Process Management Journal, Vol. 15, No. 15, 2009, pp. 669-686.
properly aligned to business strategy and properly pro-
13. Thong Ngee Goh and Min Xie, “Improving on the Six Sigma
gressing toward their goals. It is intended to be applied by Paradigm,” TQM Magazine, Vol. 16, No. 4, 2004, pp. 235-240.
MBBs and the Six Sigma program leader. 14. Dick Smith, Jerry Blakeslee and Richard Koonce, Strategic Six Sigma:
Best Practice From Executive Suite, Wiley, 2002.
It is important to note that this topic is equally critical for
15. Mikel J. Harry and Richard Schroeder, Six Sigma: The Breakthrough
organizations that don’t have a robust Six Sigma or lean Six Management Strategy Revolutionizing the World’s Top Corporation,
Sigma program in place. It is a great method for building Doubleday, 2000.
16. Erick C. Jones, Mahour Mellat Parast and Stephanie G. Adams,
the program using Philip Crosby’s mentality of doing things
“A Framework for Effective Six Sigma Implementation,” Total Quality
right the first time. It also can be applied for individual Management & Business Excellence, Vol. 21, No. 4, 2010, pp. 415-424.
project selection, guaranteeing that resources will be well- 17. Klefsjö Bengt, Håkan Wiklund and Rick Edgeman, “Six Sigma Seen
as a Methodology for Total Quality Management,” Measuring
spent on things that converge to the final business strategy.
Business Excellence, Vol. 5, No. 1, 2001, pp. 31-35.
The model isn’t limited to any specific industry, and 18. Snee, “Frontiers of Quality: The Project Selection Process,”
can be piloted and applied to manufacturing and service see reference 7.
19. Archer, “An Integrated Framework for Project Portfolio Selection,”
organizations.
see reference 11.
20. Cooper, “Third-Generation New Product Processes,” see reference 10.
21. Cooper, Portfolio Management for New Products, see reference 5.
If there is significant change in the

business strategy, the criteria
and criteria weight must
be reevaluated.

22. Project Management Institute (PMI), The Standard for Portfolio Padhy, Ramakrushna K., and Sadananda K. Sahu, “A Real Option Based Six
Management, second edition, PMI, 2008. Sigma Project Evaluation and Selection Model,” International Journal of
23. Cooper, Portfolio Management for New Products, see reference 5. Project Management, Vol. 29, No. 8, 2011, pp. 1091-1102.
24. Cooper, “Third-Generation New Product Processes,” see reference 10. Snee, Ronald D., and Roger W. Hoerl, Leading Six Sigma: A Step-by-Step
25. PMI, The Standard for Portfolio Management, see reference 22. Guide Based on Experience With GE and Other Six Sigma Companies, FT
26. Jung-Lang Cheng, “Six Sigma and TQM in Taiwan: An Empirical Study Press, 2002.
of Discriminate Analysis,” Total Quality Management & Business Voss, Chris, Nikos Tsikriktsis and Mark Frohlich, “Case Research in Operations
Excellence, Vol. 20, No. 3, 2009, pp. 311-326. Management,” International Journal of Operational and Production
27. Cooper, “Third-Generation New Product Processes,” see reference 10. Management, Vol. 22, No. 2, 2002, p. 195.
28. Archer, “An Integrated Framework for Project Portfolio Selection,” Womack, James P. and Daniel T. Jones, Lean Thinking—Banish Waste and
see reference 11. Create Wealth in Your Corporation, Free Press, 2003.
29. Cooper, “Third-Generation New Product Processes,” see reference 10. Yin, Robert K., Case Study Research: Design and Methods, third edition, Sage
30. Tanik, “A Six Sigma Case Study in a Large-Scale Automotive Supplier Publications, 2003.
Company in Turkey,” see reference 3.
BIBLIOGRAPHY
Alsmadi, Majed, Brian Lehaney and Zulfiqar Khan, “Implementing Six Sigma
in Saudi Arabia: An Empirical Study on the Fortune 100 Firms,” Total
Quality Management & Business Excellence, Vol. 23, No. 3-4, 2012, Marcelo Machado Fernandes is a Master
pp. 263-276. Black Belt (MBB) at Nielsen Media
Anand, Gopesh, Peter T. Ward and Mohan V. Tatikonda, “Role of Explicit Research in Cotia, Brazil. He earned a
and Tacit Knowledge in Six Sigma Projects: An Empirical Examination doctorate in industrial engineering from
of Differential Project Success,” Journal of Operations Management, São Paulo State University in Brazil.
Vol. 28, 2010, pp. 303-315. Fernandes is an ASQ-certified MBB and
Antony, Jiju, “Is Six Sigma a Management Fad or Fact?” Assembly Automation, a member of ASQ.
Vol. 27, No. 1, 2007, pp. 17-19.
Antony, Jiju, Maneesh Kumar, Christian N. Madu, Douglas C. Montgomery
and Sung Hyun Park, “Common Myths of Six Sigma Demystified,”
International Journal of Quality & Reliability Management, Vol, 25,
No. 8, 2008, pp. 878-895. Jennifer Hurst is Business Process
Brazilian Institute of Geography and Statistics (IBGE), “Data Base Research,” Improvement Center of Excellence
IBGE, www.sidra.ibge.gov.br/bda/tabela. leader at Nielsen Media Research in
Brun, Alessandro, “Critical Success Factors of Six Sigma Implementations Oldsmar, FL. She earned a master’s
in Italian Companies,” International Journal of Production Economics, degree in computer information
Vol. 131, No. 1, 2011, pp. 158-164. systems from Georgia State University
Cheng, Chen-Yang, and Pu-Yan Chang, “Implementation of the Lean Six in Atlanta. Hurst is an ASQ-certified
Sigma Framework in Non-Profit Organizations: A Case Study,” Total Black Belt and a member of ASQ.
Quality Management & Business Excellence, Vol. 23, No. 3-4, 2012,
pp. 431-447.
Eisenhardt, Kathleen M., “Building Theories From Case Study Research,”
Academy of Management Review, Vol. 14, No. 4, 1989, pp. 532-550.
Eisenhardt, Kathleen M., and Melissa E. Graebner, “Theory Building From
Cases: Opportunities and Challenges,” Academy of Management Journal, Jiju Antony is a professor of quality
Vol. 50, No. 1, 2007, pp. 25–32. management at Heriot Watt University
Goh, Thong Ngee, P.C. Low, Kwok-Leung Tsui and Min Xie, “Impact of in Edinburgh, Scotland. He earned a
Six Sigma Implementation on Stock Price Performance,” Total Quality doctorate in quality management from
Management & Business Excellence, Vol. 14, No. 3, 2003, pp. 753-763. the University of Portsmouth in England.
Hart, Susan, Erik Jan Hultink, Nikolaos Tzokas and Harry R. Commandeur, Antony is a senior member of ASQ.
“Industrial Companies’ Evaluation Criteria in New Product Development
Gates,” Journal of Product Innovation Management, Vol. 20, No. 1, 2003,
pp. 22-36.
Jonas, Daniel, “Empowering Project Portfolio Managers: How Management
Involvement Impacts Project Portfolio Management Performance,” João Batista Turrioni is a professor
International Journal of Project Management, Vol. 28, 2010, pp. 818-831. at the Federal University of Itajubá
Linderman, Kevin, Roger G. Schroeder, Srilata Zaheer and Adrian S. in Minas Gerais, Brazil, and earned a
Choo, “Six Sigma: A Goal Theoretic Perspective,” Journal of Operational doctorate in engineering from São
Management, Vol. 21, No. 2, 2003, pp. 193-203. Paulo State University.
Martinsuo, Miia, and Päivi Lehtonen, “Role of Single-Project Management
Achieving Portfolio Management Efficiency,” International Journal of
Project Management, Vol. 25, 2007, pp. 56-65.
Meskendahl, Sascha, “The Influence of Business Strategy on Project Portfolio
Management and its Success—A Conceptual Framework,” International
Journal of Project Management, Vol. 28, No. 8, 2010, pp. 807-817.
Nair, Anand, Manoj K. Malhotra and Sanjay L. Ahire, “Toward a Theory of
Managing Context in Six Sigma Process-Improvement Projects: An Action Messias Borges Silva is a professor at
Research Investigation,” Journal of Operations Management, Vol. 29, No. 5, São Paulo State University and earned a
2011, pp. 529-548. doctorate in chemical engineering from
the University of Campinas in Brazil.

MINING
F E AT U R E
SAFETY
MATTERSHow Six Sigma is being deployed to assess high-risk

underground mining operations and make activities safer
| by Rajeev Chadha
Just the
Facts While switching my career from auto-
motive to the mining sector, I realized
A mining organi-
zation’s ultimate that the mining of precious metals and
goal must be commodities is inherently risky. But
to achieve zero
fatalities, even during my onboarding, the employer
though mining is showed me that mining operations in
an inherently risky
activity. North America are statistically safer than
the average workplace in manufacturing,
Leading indica-
tors are more construction and other sectors.
relevant than Over the years, the mining sector has
lagging indicators
to achieve this driven down instances of time lost due to
common goal injury, but more work can be done when
and to diminish
low-occurrence, it comes to eliminating serious injuries
high-severity and deaths. In one case in 2009, the bot-
incidents.
tom of a cone-shaped bin holding raw ore
A collaborative let go, spewing an estimated 500 tons
approach among
mining organi- of raw ore. A 40-year-old mine operator
zations that taps was spared the brunt of the impact, but
into the power of
Six Sigma for met- another young miner was crushed under
rics and analytics the ore. Emergency response crews had
is needed to reach
the goal of human to dissolve the water-soluble ore and sift
safety by sharing through it to retrieve the body, according
lessons learned
and experiences. a company spokesperson.

F E AT U R E
SAFETY
After some investigation, the structural integrity of the

40-year-old mine came under question after a gallery that
Priority must be to prevent fatalities. Although the
transports ore from the head frame to the mill collapsed. No
industry has made great efforts to reduce injuries,
one was in the gallery at the time, so there were no injuries. The
fatalities continue to occur, so the thinking around
company undertook a major inspection of all structural steel
mining must shift. The health, safety and well being
components at that time. “Obviously, they didn’t catch this one,”
of people are first.
a member of the process and plant maintenance team told the
Miner Journal in an interview in December 2009.1 “This storage
bin is the original storage bin. Inspections are done frequently, TA B L E 1
Black Belt (BB) to understand
and the structural steel in the mill has been regularly replaced.” and validate the key beliefs
The two operators affected were working a midnight shift
and assigned to the storage area to hammer at the bin to
Injury and in the workplace to lay the
groundwork for change.
dislodge the ore as the ore flowed downward through the cone-
shaped bin onto a conveyor belt. “It is common practice for severity scores The fundamental drivers
for improvements in health
employees to bang on the bin,” mine workers later explained to and safety are human safety
Severity score
the Miner Journal. Injury type (1 - 10 scale) and our value system. People
A mining organization’s goal must shift from achieving zero come first, meaning that
No incident 0
harm to achieving zero fatalities. Leading indicators are more caring for the people in the
Near miss 1
relevant than lagging indicators to achieve this common goal mining industry should be the
and to diminish low-occurrence, high-severity incidents. The First aid 2 driver of all decision making.
current practices are only allowing us to reduce the injury rates Recordable 4 Next comes the human value
instead of ensuring the elimination of all causes of fatalities. Lost time 8 system. Health and safety are
A collaborative approach among mining companies is needed values, not priorities. A priority
to reach the goal of human safety by sharing lessons learned can change over time, but a
and experiences. value doesn’t. If the mining industry hopes to obtain
Six Sigma deployments in mine safety are primarily related and secure the social license to operate, health and
to the safety analysis that will prevent mine accidents and safety must always be at the core of the business.
fatalities by observing and evaluating the work environment, Priority must be to prevent fatalities. Although the
work conditions, production timing and miners’ habits. While industry has made great efforts to reduce injuries,
the individual mining company’s health and safety department fatalities continue to occur, so the thinking around
provides on-the-job counseling to individual miners and assists mining must shift. The health, safety and well-being
in correcting unsafe or potentially hazardous conditions and of people are first and foremost. Improving health
actions, people often misunderstand the primary responsibility and safety is centered on building relationships with
of the health and safety department as policing for mine safety. every person at every level: It is about building trust.
Its job is to conduct professional-level miner training and It is about enacting the belief that human life and
develop safety programs to create a safer work environment well-being are more important than production or
and develop instructions for safe work practices. Safety Audits economic value.
mandate that safety is everyone’s responsibility, and individuals Values often live close to beliefs. Some common
must stop the work if the conditions in their work environment beliefs at many underground mine sites in Western
are perceived to be unsafe. When operators obtain quality Canada fall under the following domain and can be
training like this and perform on-the-job safety analysis, a considered local paradigms of mine safety:
significant reduction in surface and underground accidents and ++ A series of small incidents happen before a major
fatalities results. incident ensues. This indicates that when there
are continuous safety violations visible in the
Values and beliefs form of small incidents—such as near misses, cuts,
For a Six Sigma professional, safety analysis starts from the bruises and equipment damage—the occurrence
environmental setting because each work environment is dif- of a major incident is highly likely.
ferent. People think differently, and their opinions cumulatively ++ Safety incidents happen more frequently when
develop a culture based on common beliefs. When one chal- production support activities—such as plant
lenges beliefs, it is parallel to challenging work culture, which maintenance, start-up preparations and shutdown
can lead to positive change. Therefore, it is important for a activities—are increased.

FIGURE 1
I-chart of mean severity scores of normal production (A) and

maintenance shutdown (B) periods between 2010 and 2013
I-chart of severity score by shutdown period
1A 2B 3A 4B 5A 6B 7A
1
12
x̅ = 0.84 x̅ = 0.88
10 x̅ = 0.82
x̅ = 0.59 1 1 1 1 x̅ = 0.62 1 x̅ = 0.61

x̅ = 0.56
Individual value
8 1
11 11 1 11 1 1 1 1
6
11 1 1 1
1 1 11 1 11 111 1 1 1 11 1 1 1 1 11
4 1 1 1
1
10 10 011 0 11 0 11 11 12 12 12 13
20 /2
0
/2 /2 /2 /20 /20 /20 /20 /20
1/ 19 /7 27 15 /3 22 10 8 15
6/ 9/
1 4/ 8/ 12 3/ 7/ /2 2/
10
Date
Normal mine production = A

Annual maintenance shutdown = B
Two-week shutdown preparation period before/after = C
++ Safety incidents happen more with new, less-experienced To deliver excellence in safety performance,
employees and contract workers, compared to seasoned a Six Sigma professional must fully integrate safety
operators. with the business because working in silos does not
++ Faster job rotation is a factor in improved productivity. How- provide the desired outcomes. The safety function
ever, this productivity can compromise safety because time must prove to add value to the business and work
spent on personnel training is not enough to make the job toward a world-class safety culture that helps
rotation safe. place the business at a distinct advantage among
The Six Sigma approach for ensuring mine safety is well its competitors.
defined in literature. The process determines the root causes The best way to achieve this is through improve-
contributing to the hazard or injuries so each of them can ment of systems that are already in place and
be targeted and improved. By using mistake-proofing monitored by management for continuous business
(poka-yoke) tools, a process’ safety performance can be performance.
significantly improved.
This requires testing and awareness among everyone Mine site safety data organization
involved, as well as the institutionalization of these practices Finding comprehensive data and statistics on mining
across the mining operation. These remedies must be built health and safety is a major challenge. Organizations
into the process specifications and procedures, and the safety measure and define injuries or accidents differently.
department should create awareness across all groups and Due to the lack of a common standard, there is no
functions to prevent injuries or a specific hazard type.2,3 one-stop shop for safety data for the industry on

F E AT U R E
SAFETY
a global level. Even when data can be extracted from sus-

tainability reports from different organizations, mining The mine site safety data are primarily discrete and
associations and other organizations, comparing those require a simple hypothesis to be tested to draw
numbers can lead to inaccurate conclusions because of the major inferences about mine safety.
difference in reporting calendars and the definition of the
indicators. This leads to the growth and development of
local safety paradigms. It also creates barriers in compiling British Standard (BS) OHSAS 18001 is an inter-
information and creating consistency across the industry, nationally applied standard for occupational health
leading to the development of local safety standards. and safety management systems. In cases of lost
Currently, most of the usable data is based on post-in- time and higher injury, proper documentation
cident reports detailing incident statistics, fatalities and and reporting to regulatory agencies are required
health problems. However, these provide a snapshot of by the provincial laws and mining regulations.
the past, and are not necessarily relevant to the future.
Most of the existing mine safety data are based on post- Six Sigma study on a
incident reports, the so-called lagging indicators. They mine site's safety
include fatality rates, incident statistics and health problems, To carry out a Six Sigma study on a mine site's
providing indications of past trends that aren’t necessarily performance, a team of BBs and Green Belts
representative of future performance. However, in the mine (GB) began by focusing on additional safety
safety database, it is mandated that all incidents on mine resources for turnarounds, or annual mainte-
sites—big and small—must be reported and documented nance shutdowns and plant clean-up work. This
to ensure success of the safety management system. involved completing an analysis of trends, if any,
In most cases, incidents are classified, as per Occupa- around incidents during annual maintenance
tional Health and Safety Assessment Series (OSHAS)/ turnarounds and the two-week preparation time
ISO 18000 guidelines, into seven categories: before and two-week start-up time afterward.
1. Near miss. All of the site safety reports and data gen-
2. First aid. erated on anything that was included in the
3. Consultation. annual reports—whether a small first aid, near
4. Property damage. miss, or injury or death—and four years of safety
5. Major injury. data were collected and compiled based on the
6. Critical injury. incidents’ safety scores. The data were from a
7. Fatality. continuous period that included normal production

operation, an annual maintenance shutdown and two week- Study analysis and results
long shutdown preparation periods. These time periods The first graphical presentation of data is done in
were identified and marked as A, B and C, respectively. the form of I-charts. Three different I-charts were
The site safety data from 2010-2013 were identified, prepared to understand the overall severity score
marked and compiled to perform graphical analysis and trend of the mine site.
statistical testing to unveil facts and correlations. The The first I-chart (Figure 1, p. 27) shows the severity
data were segregated into subperiods associated with scores on a continuous scale with the mean severity
their operational activities, and a time-period leg- scores of normal production (A) and maintenance
end was used to maintain the continuity of data. shutdown (B) periods. The average severity varies
++ Normal mine production = A. between 1.56 and 0.88. In other words, the severity
++ Annual maintenance shutdown = B. of incidents was highest during the shutdown of
++ Two-week shutdown preparation period before/after = C. 2012 and lowest during the shutdown of 2011. How-
For example, legend combination 1A is the first period ever, the normal operation of the mine during 2012
(2010) normal mine production, and 12C is the 12th period and 2013 records the lowest mean severity of 0.62
(2012) two-week maintenance shutdown preparation period and 0.61, respectively.
before or after. Table 1 (p. 26) shows a breakdown of injury While looking at the severity during the actual
type and the related severity score. The severity scores maintenance work period (B) and the two week-long
are internal safety standards set for the organizationwide periods before and after (C), you can see a differ-
ISO 18000 program and to each of the mine’s sites. ent scenario. The mean severity of maintenance
FIGURE 2
I-chart of mean severity scores of maintenance shutdown (B)

period and two weeks
1C 2B 3C 4C 5B 6C 7C 8B 9C 10C 11B 12C
1
12
x̅ = 1.14
10 x̅ = 0.93
x̅ = 0.86
x̅ = 0.79
Individual value
8
x̅ = 0.63 x̅ = 0.57
1 x̅ = 0.57 x̅ = 0.55 1
6 x̅ = 0.5
1 x̅ = 0.29 1
1 1 1 1 1
4 x̅ = 0.25
x̅ = 0.14 1 1
1 1 1 111
2
11
10 10 10 10 10 0 11 11 12 12 12
/2
0
/2
0
/2
0
/2
0
/2
0
/2 20 20 20 20
25 16 13 /4 26 17 8/ 9/ 0/ 2 /
/ 9/ / 11 / / 1/ /2 /2 /1
8 10 11 10 1 10 11 12
Date


F E AT U R E
SAFETY
work periods are 0.25, 0.63, 0.55 and 0.93, and it has steadily with less difficulty. People are willing to share best
increased since 2010. This might be correlated with a new main- practices that are making them safe.
tenance workforce, including new on-site contractors. On the contrary, safety incidents happen more
This indicates there may be a correlation between new frequently when production support activities—such
employees and contract workers and safety incidents, as noted as plant maintenance start-up preparations and
earlier. This shows some signs of our paradigm that safety inci- shutdown activities—are increased beyond their nor-
dents happen more with new employees and contract workers mal level. These activities are random and discrete,
as compared to seasoned operators. See Figure 2, p. 29. and often unique in their makeup and planning.
Figure 3 is the I-chart of all data points representing all 16 peri- Pressure to finish maintenance quickly is an
ods. The lowest mean severity is 0.25 and the maximum is 1.14. important contributor in creating unsafe conditions
The mine site safety data are primarily discrete and require a when safety procedures are violated and shortcuts
simple hypothesis to be tested to draw major inferences about are adopted. These reasons and others—such as
mine safety. One valid hypothesis is to test whether the various tempered tools, operator fatigue and inadequate
time periods are significantly different. To do this, we can per- job preparation—validate the common notion
form a single-factor analysis of variance (ANOVA) and a t-test. that safety incidents happen more frequently
See Online Figures 1 and 2, which can be found on this article’s when production support activities—such as plant
webpage at qualityprogress.com. maintenance start-up preparations and shutdown
The t-test and one-way ANOVA results show there is no activities—are increased beyond their normal level.
statistically significant difference in the severity of incidents The paradigm that a series of small incidents hap-
between the different time periods. This weakens our safety pens before a major incident is difficult to validate.
culture paradigm that says safety incidents happen more It requires investigation and analysis around major
frequently when production support activities—such as plant incidences to draw conclusions. A few fatal incidents
maintenance, start-up preparations and shutdown activities, in similar underground mining environments were
are increased beyond their normal levels. See Online Figures reviewed to analyze this paradigm. However, there
3 and 4. were signs of small safety incidents recorded by the
In a continuous operation—such as rock cutting, conveying, mining companies.4
crushing, screening and refining—human activities are typically
repetitive and recorded in the subconscious mind. Because
of this, the hazards are known and well defined. Further, the Job rotation increases the variety of tasks required
safety records are tangible in these activities, and continuous because the worker takes on more duties, enlarging
improvement programs can be implemented in routine and the physical demands and adding variety to the job.

FIGURE 3
I-chart of mean severity scores of all the maintenance

shutdown periods (A),(B) and (C) between 2010 and 2013
1A 3C 6B 7C 9C 11C 13C 15C
11
12
x̅ = 1.14
10 x̅ = 0.86 x̅ = 0.85 x̅ = 0.84 x̅ = 0.93
x̅ = 0.82
x̅ = 0.61 x̅ = 0.57 x̅ = 0.57 1 x̅ = 0.61
1 1 1 x̅ = 0.62 1
x̅ = 0.42
Individual value
8 x̅ = 0.55 1 x̅ = 0.5
x̅ = 0.25
11 11 1 11 1 1 1 1
6
11 1 1 1
1 1 11 1 11 111 1 1 1 11 1 1 1 1 11
4 1 1 1
1
10 10 011 0 11 0 11 11 12 12 12 13
20 /2
0
/2 /2 /2 /20 /20 /20 /20 /20
1/ 19 /7 27 15 /3 22 10 8 15
6/ 9/
1 4/ 8/ 12 3/ 7/ /2 2/
10
Date

That means when the continuous propagation was severed when 900 kilograms of rock fell on him, pinning
of safety violations is visible in the form of small him to a conveyor belt. Goddard was paralyzed from the
incidents—such as near misses, cuts, bruises and waist down and suffered two broken arms.6
equipment damage—the occurrence of a major inci- In mines, faster job rotation is a factor in improved produc-
dent is highly likely.5 The following examples validate tivity. However, it works against safety because time spent on
these local safety paradigms: personnel training is not enough to make the job rotation safe.
++ In Saskatchewan, a fire trapped more than 50 According to Ergonomics Plus, an injury prevention and ergo-
miners underground for about 15 hours until nomics consulting firm,7 job rotation is the structured interchange
emergency workers extinguished the blaze. No of workers between different jobs, requiring workers to rotate
one was injured in this February 2014 incident. among different workstations or jobs at certain time intervals.
++ Andrew Hann, a 25-year-old Newfoundland man, Job rotation increases the variety of tasks required because
died after falling about 20 meters from a steel the worker takes on more duties, enlarging the physical
scaffold in July 2013. Hann was employed by a demands and adding variety to the job. The objective is to pro-
construction company contracted to work on a vide a consistent and systematic method of developing formal
mine expansion project. job rotations that are based on the requirements of the jobs
++ Edward Artic, a 59-year-old electrician, died in being rotated.
a mine’s above-ground mill in May 2010 when a Research on ergonomics shows that workers who partake
load being lifted into the building struck his head. in medium-complexity rotations have high job satisfaction.8
++ In 2006, 29-year-old Paul Goddard’s spinal cord However, a high or low-complexity rotation, which many

F E AT U R E
SAFETY
FIGURE 4
T-test to identify significance of the mean severity scores of

the maintenance shutdown periods
2-sample t-test for the mean of severity scores and 2severity scores
summary report
Do the means differ? Statistics Severity score 2Severity score

0 0.1 > 0.5 Sample size 542 554
Mean 0.76384 0.64801
Yes No 90% CI (0.6585, 0.8691) (0.55396, 0.74207)
P = 0.177 Standard deviation 1.4879 1.3436
The mean of severity score is not significantly different from the mean
of 2severity score (p > 0.1). Difference between means* 0.11582
90% CI (-0.025254, 0.25690)
90% confidence interval (CI) for the difference * The difference is defined as severity score - 2severity score
Does the interval include zero?
Comments
0.00 0.06 0.12 0.18 0.24 • Test: There is not enough evidence to conclude that the
means differ at the 0.1 level of significance.
• CI: Quantifies the uncertainty associated with estimating the
Distribution of data
Compare the data and means of the samples. difference from sample data. You can be 90% confident that
Severity score
the true difference is between -0.025254 and 0.25690.
• Distribution of data: Compare the location and means of
samples. Look for unusual data before interpreting the results
of the test.
2Severity score
-2 0 2 4 6 8 10 12
workers prefer, leads to fewer accidents and days of sick leave. safety performance, some historical knowledge of the
It is important to remember that job rotation does not eliminate original statistical improvement tools and industrial
ergonomic risk factors. processes is necessary.9
While job rotation is an effective control measure for jobs that It is important for a Six Sigma practitioner to
have been identified as high-risk jobs, it is not desirable that define proper metrics in any Six Sigma exercise.
ergonomic risk factors are hidden by administrative controls. Developing leading measures that could more
Obviously, ergonomic problems should be addressed and fixed accurately predict future events would be extremely
through training whenever possible. Despite the caution this helpful to the mining and metals industry. The idea
solution presents, it is important to recognize the many great is to identify trends that reveal whether conditions
benefits to implementing job rotation in the workplace. leading to an accident are increasing. The aim is
to be able to take preemptive measures before an
Preemptive measures accident happens.
Six Sigma is the evolution of data-driven process improvements Traditionally, safety has been framed as a business
that have been used extensively to improve the performance of priority, with the case for safety often made in terms
manufacturing and chemical industries. Although Six Sigma tools of enhanced productivity and reduced costs. Infor-
can help produce significant and sustainable improvements in mation about safety management measurements

3. S.K. Sinha, “Risk Management in Mines: The Six Sigma
Way,” Indian Institute of Coal Management, University
It is important for a Six Sigma practitioner to define proper of Wollongong Australia, 2008, http://ro.uow.edu.au/cgi/
metrics in any Six Sigma exercise. viewcontent.cgi?article=1025&context=coal.
4. The Miner-Journal, see reference 1.
5. Alex Macpherson, “Miner Suffered ‘Life-Threatening
Injuries’ in Underground Accident at Agrium’s
generally has not been shared among organizations because,
Vanscoy Potash Mine,” Saskatoon Star Phoenix, Aug.
sometimes, it’s perceived to aid competitors. 9, 2016, http://thestarphoenix.com/business/mining/
Consider safety as a value that extends beyond employees— miner-in-critical-condition-following-underground-
accident-at-agriums-vanscoy-potash-mine2.
to all the people with whom the industry interacts, including
6. The Miner-Journal, see reference 1.
contractors, families and communities. Although this article has 7. Mark Middlesworth, “A Step-by-Step Guide to Job
validated some beliefs, such as safety incidents happen more Rotation,” ErgoPlus, July 3, 2015, http://ergo-plus.com/
job-rotation.
with new and less-trained employees, a resolution to the issue
8. Ibid.
of health and safety in the mining industry will come only when 9. Sinha, “Risk Management in Mines: The Six Sigma Way,”
broader industrywide paradigms are created. see reference 3.
The safety performance in mining has evolved to the current
BIBLIOGRAPHY
paradigm wherein behavior plays a crucial role in achieving Hodge, Anthony, “Shifting the Paradigms of Health
safety targets. Yet, we still see a gap between where the and Safety in Mining,” Cornerstone, Vol. 2, No. 2,
Summer 2014, pp. 4-8, http://cornerstonemag.net/
industry is today and achieving zero fatalities. Priorities change,
shifting-the-paradigms-of-health-and-safety-in-mining.
especially in response to external pressures. Values should not.
Safety is not a competitive issue, which is why injuries and
fatalities in any organization affect the entire industry. There-
fore, we must act openly and welcome all mechanisms that Rajeev Chadha worked as
the Six Sigma continuous
will help us share what we learn along the way. Leaders at all improvement lead at Mosaic
levels must set the tone that enables good health and safety Potash ULC Canada in
performance. Saskatoon, Saskatchewan.
He holds master’s degrees
in process engineering
REFERENCES and business management
1. The Miner-Journal, Koskie Publications, Esterhazy, Saskatchewan, from the Indian Institute of
Dec. 7, 2009. Technology in New Delhi. An ASQ fellow, Chadha
2. Patricia O’Rourke, Using Six Sigma in Safety Metrics, OSHA Academy, is a professional engineer and ASQ-certified quality
www.oshatrain.org/notes/pat.html, Jan. 21, 2002. auditor, engineer and Six Sigma Black Belt.

F E AT U R E
MAINTENANCE MANAGEMENT
Improve equipment reliability and availability by

integrating two management systems | by Ray Stoute

Just the
Facts Maintenance management
embraces several asset
Organizations in
the oil and gas management principles that
industry must emphasize short, medium
provide near-con-
tinuous service, and long-term perspectives,
which is why it is juggling capital vs. operational
crucial for these
organizations to expenditure to harness the
implement quality spend, as well as a project
and maintenance
management management philosophy to
programs. prioritize it all and achieve the
By combining intended deliverables.
these manage- Does quality management
ment programs,
an organization play a useful or even practical
can achieve role in this equation?
equipment reli-
ability and drive To answer this question,
its maintenance consider Massy Wood’s (for-
philosophy.
merly Massy Wood Group)
The author success story, which took
presents a case
study of quality place in Trinidad and Tobago.
and maintenance Massy Wood is a partnership
management
integration from between the Massy Group
the oil and gas conglomerate—a Trinidad-born
industry.
entity that also has made
inroads in the Caribbean and
Latin America—and the Wood
organization (formerly Wood
Group)—a major global player
in maintenance, engineering
and construction management,
and selective procurement for
those industries.
At the time of the study,
which was between 2006
(the initiation of the contract)
and 2009 (which culminated
with Massy Wood’s ISO 9001
certification), maintenance
management was the primary
service offered by Massy
Wood, and the onus was on
the organization to get it right
the first time. This was one
reason it aligned the principles
and controls of an International
Organization for Standardiza-
tion (ISO) quality management
system (QMS) during the
initiation phase of maintenance
management implementation.

F E AT U R E
Quality in the maintenance industry and existing—or “brown- They are closely managed so
The marriage of quality assurance (QA) and quality field”—environments. These as not to exceed the project
control (QC) in the oil and gas industry, particularly terms are commonplace in management anchors of
in the manufacturing and construction management the petrochemical industry, time, cost and quality. By
environments, is a natural and symbiotic relation- and Massy Wood functions comparison, maintenance
ship. QA and industry-specific QC techniques are comfortably in both areas. management professionals
fully ingrained in such contracts internationally. include reliability engineers,
Asset maintenance management—by definition— Maintenance focus mechanical technicians and
focuses on the upkeep of facilities that often are When maintenance and equipment specialists.
designed to provide as near-continuous operational quality are juxtaposed, The common ground here,
service as possible in the provision of hydrocarbon herein lies the dilemma of though, is the need for plan-
and its derivatives. It is this base product that is whether to rely on one or ners and schedulers, sound
responsible for making and breaking hundreds of the other when it comes health, safety and environ-
organizations in the energy sector, depending on to asset and equipment ment (HSE) management
which end of the global monetary scale oil and gas reliability and availability. systems and, of course, QMSs.
can command. Quality professionals in
Like quality management, maintenance man- the construction industry, for Types of
agement also has its own specialisms, theories and example, are dedicated to maintenance
gurus that make it a standalone profession in previ- myriad specialized tasks, such organizations
ously undeveloped—or “greenfield”—environments as nondestructive examina- There are four general types
tions, welding inspections of maintenance organizations:
and tank, pipe and pressure reactive, preventive, predic-
When maintenance and quality are juxtaposed, vessel inspections, which tive and productive. These
herein lies the dilemma of whether to rely on one are all part and parcel of the organizations, their key char-
or the other when it comes to asset and equipment day-to-day project activities acteristics and quality overlap
reliability and availability. of the oil and gas industry. are summarized in Table 1.

TA B L E 1
Types of maintenance organizations

Maintenance organization Key characteristics Quality overlap
++ Equipment runs until failure. ++ Inefficient or
nonexistent quality
++ Frequent unscheduled jobs. management system
(QMS).
Reactive ++ Lack of maintenance key performance indicators (KPI).
++ No collaboration
++ Inefficient recordkeeping/equipment history. between maintenance
++ Little or no inventory control. and quality.
++ Emphasis on preventive maintenance (PM) activities (weekly, monthly, ++ QMS implemented but
quarterly maintenance). limited in adding value
to the maintenance
++ Planned turnarounds/shutdowns. function.
Preventive ++ Computerized maintenance management system (CMMS) in place.
++ Minimal KPIs in place.
++ Short-term work scheduling.
++ Inventory control present but focuses on short-term activity.
++ PM activities are based on time and use. ++ QMS implemented
contributes to the
++ Planned turnarounds/shutdowns. maintenance strategy
effectiveness via
++ Advanced CMMS in place.
Predictive performance trending,
++ Wider use of KPIs, includes trending of equipment shutdown/failure. root cause analysis
(RCA) and risk analysis
++ Short, medium and long-term work scheduling. (RA).
++ More intelligent inventory control with specialized or critical spares included.
++ Condition-based and reliability-based maintenance (PM) activities in place. ++ QMS implemented
aligns with the
++ Corrective maintenance activities influenced by the above. organization’s business
strategy.
++ Minimal unscheduled/emergency work.
++ Cost of poor quality
++ Planned turnarounds/shutdowns not as intrusive or extensive. is now an inherent
Productive ++ Innovative use of CMMS. measurement
in addition to
++ Robust use of wide-ranging KPIs. performance trending,
RCA and RA.
++ Short, medium and long-term work scheduling.
++ Inventory control includes critical spares and alignment with budgetary costs
and capital expenditure.
Types of maintenance strategies optimizing the useful life was the overarching element.
An organization’s classification in Table 1 depends of equipment or a system.” It was at this design stage
on its maintenance strategy (or mix of strategies). In the case of Massy Wood, that quality assurance played
The different maintenance strategies include reactive, the organization began a its most crucial role, ensuring
preventive, predictive and proactive. The strategies, detailed gap analysis of its that the organization was on
their key characteristics and quality overlap are sum- major client at the time, which par with its world-class status.
marized in Table 2 (p. 38). influenced the maintenance This was orchestrated
These strategies all have a governing attribute strategies applied. The keeping in mind that the
that triggers when and what type of maintenance analysis was done under the functions of maintenance and
takes place. They also all fall under the umbrella framework of an overall man- quality management are sim-
definition of “maintenance,” which is defined as agement system, of which ilar in that the most effective
“activity that is geared toward sustaining and maintenance management systems are developed when

F E AT U R E
TA B L E 2
Types of maintenance strategies

Maintenance organization Key characteristics Quality overlap
++ Equipment allowed to run to failure. ++ Eliminates advantageous quality
techniques such as root cause
++ No advanced planning performed. analysis (RCA) and skewed audit
results.
Reactive ++ Appropriate for certain applications if more cost-effective to
replace than fix. ++ Escalated quality costs (calibration
and inspection).
++ High maintenance costs due to unplanned maintenance and
downtime.
++ Maintenance activity performed periodically prior to failures, ++ Allows for the use of RCA, auditing
which achieves desired reliability and availability. and trending, and adjustment of key
performance indicators (KPI).
++ Time-based intervals used.
Preventive ++ RCA works together with FMEA.
++ Mean time between failures (MTBF) used to dictate maintenance
intervals.
++ Use of failure mode and effects analysis (FMEA).
++ More noninvasive than preventive maintenance (PM) due to ++ Allows for the use of RCA, auditing
reliance on data collection, such as: and trending, and adjustment of
KPIs.
++ Specialized monitoring techniques, such as vibration and oil
Predictive analysis, thermography and acoustic measurements. ++ RCA works together with FMEA.
(condition-based)
++ These measurements detect the onset of degradation that may ++ Calibration used in all strategies,
lead to a failure. and even more so with the use of
specialized equipment.
++ Use of MTBF and FMEA.
++ Emphasizes the use of PM. ++ RCM is an example of maintenance
Proactive and quality in full alignment.
(reliability-centered ++ Also known as a PM strategy.
maintenance (RCM))
++ Focus is on preventing failures.
there is a cross-section of disciplines con- plan-do-check-act cycle (Figure 2, p. 40).

tributing to the building of these systems. Both systems in each of the cycle steps
So, it is only fitting that the most effective accomplish these tasks:
maintenance strategies can be realized
when there is overlap with the quality Plan
function at the design stage, as depicted in ++ Establish equipment history and
Tables 1 and 2. life cycle.
This results in a synergistic relationship ++ Develop key or critical operating
whereby quality influences maintenance as a procedures.
value-added filter that, after superimposed, ++ Develop a risk mitigation and manage-
results in a combined, efficient system rather ment outline.
than two coexisting entities with redundant ++ Identify critical equipment and
processes aimed at accumulating identical or critical spares.
overly similar information. ++ Determine subcontractor and vendor
Figure 1 illustrates the maintenance strat- pre-qualifications.
egy hierarchy.
Do
Plan-do-check-act ++ Analyze failures (working in tan-
The common denominators of the two dem with root cause analysis).
management systems are categorized in the ++ Measure reliability indicator.

++ Manage data (computerized). ++ Determine material traceability. management support, orga-
++ Determine technician com- ++ Evaluate subcontractors and vendors. nizational culture and the
petency and assessment. effectiveness of the maintenance
++ Perform risk mitigation Act strategy in place that allows
and management. ++ Identify continuous improvement organizations to transcend from
++ Manage materials, ware- initiatives. one level to the next.
house and logistics. ++ Develop summary of lessons learned.
Having these elements in place Benefits of integration
Check through this combined lens makes Massy Wood’s system focused
++ Identify key performance indicators. the difference in attaining a reactive, on an integrated approach that
++ Verify audits. preventive, predictive or productive required the implementation
++ Determine customer satisfaction maintenance organization. The distinc- of best practices to achieve the
and gather feedback. tion among these organizations is their desired level of equipment reli-
++ Manage data (computerized). ability to sustain long-term maintenance ability and drive the maintenance
++ Calibrate inspection, measuring cost savings and high equipment reliabil- philosophy. In addition, the orga-
and test equipment (IMTE). ity and availability. It is a combination of nization could benchmark against
FIGURE 1
Maintenance strategy hierarchy

Proactive
strategy
Predictive
(condition-based)
strategy
Integrated maintenance system controls
Preventive
strategy
Reactive
strategy
No
strategy
Reliability

F E AT U R E
not only its global operations, but also industry best FIGURE 2 control, process simplifica-
practices in maintenance and quality. As a result, the tion and contributions to
organization saw operations benefits from as early
as the implementation phase and throughout the
Plan-do-check-act cycle operational lessons learned.
++ Understanding the bound-
execution of the core maintenance business. These aries where maintenance
benefits included: ended and quality began—
++ Differentiating between a preferred ratio of and vice versa—also was
planned and scheduled maintenance vs. reac- important. The run-to-
tive and unplanned maintenance allowed for failure maintenance
proper analysis of failure modes on occurrence strategy mentioned earlier
(or those allowed to occur, such as in the case typically is an instance of
of the run-to-failure strategy for non-critical, the maintenance strategy
easily replaceable equipment). Such trending working as it should, rather
allowed Massy Wood to improve its inven- than an opportunity to
tory and spares-management programs. launch a failure investiga-
++ Finding a balance between preventive and tion. Where these failures
corrective maintenance activities should be a become recurrent and
fundamental metric in any maintenance system. fail too soon in the useful
Because it was aligned with the QMS (via the life of the equipment, the
monitoring and measurement clause of ISO 9001), situation must be reviewed
however, the trending and criticality of failures to determine whether there
was investigated using fault tree analysis and root Finding a balance between is a quality issue rendering
cause analysis. Calibration and categorization of preventive and corrective the cost-effectiveness of
IMTE as critical or not also influenced this task. maintenance activities should the strategy ineffective.
++ The governing activities in the maintenance sys- be a fundamental metric in any ++ Maintenance quality per-
tem were occasionally reviewed for effectiveness. maintenance system. sonnel were not embedded
This was achieved by implementing a risk-based in the execution teams, as
internal audit schedule that focused on the higher married well with the HSE would normally occur in a
areas of impact to production and efficiency. applications and empha- construction environment.
Tailoring the schedule frequency played a crucial sis of the business. This was a deliberate move
role and was influenced by the number of non- For example, a recur- because, given its nature,
conformities, such as breakdowns and operational ring seal failure identified maintenance execution can
upsets that occurred. the existence of abrasive be governed by the mainte-
++ The percentage of time dedicated to continuous foreign particles—for nance strategy.
improvement (rather than pure problem solving) which the seals were not The quality depart-
may be somewhat of a luxury in maintenance designed—rather than ment was engaged via
management, but is an absolute requirement of incorrect seal thickness or a intervention in the form
ISO 9001. Therefore, continuous improvement ini- manufacturing defect. of surveillance audits to
tiatives were documented as an attainable target ++ 5S was used to identify ensure compliance with
in performance charters to encourage pushing and eliminate waste, such the integrated system’s
this envelope, with positive results. The de- as inefficient job planning, controls. The reliability
bottlenecking of a workshop repair process was time delays, nonconforming engineer function also was
one area actioned using this method. material, equipment failure a strategic ally of the QA/
++ Because of the integration with quality, main- and contractor mismanage- QC department through
tenance performance indicators were used as ment. This was a beneficial fault tree and root cause
a prompt or trigger for corrective or improve- amalgam with the mainte- analyses of perceived trou-
ment actions, rather than simply as a database nance strategies that were ble areas.
repository. In other words, a proactive—rather ultimately implemented and ++ Competence often is taken
than reactive—system was achieved by segre- was most visible in materi- for granted, but ISO 9001
gating leading and lagging indicators. This also als management, inventory highlights the need for it in

any QMS. With Massy Wood’s characteristic that helped
core business being that of immensely. It was this tailor-made
service provision, people and integrated system that determined
their competence played an Massy Wood’s success, and was
enormous role in the success responsible for it becoming a
of the business. As a result, a front-runner in Trinidad and Toba-
defined competency manage- go’s industrial landscape.
ment system was implemented ++ Fit-for-purpose systems. Given
that focused on not only the the unique variables of oil and gas
core crafts of maintenance tech- organizations, and their subsets
nicians (mechanical, electrical of operational environments and
and instrumentation), but also specific end products, no two
the specialized crafts (reliabil- integrated management systems
ity engineers) that supported are the same. Massy Wood’s story
the data collected by the focuses on providing mainte-
technicians. nance management services in
an offshore and onshore oil and
Criteria for success gas production environment in a
The benefits of an integrated natural but corrosive atmosphere.
approach paint a positive picture. ++ Eliminate redundancy through
The fundamental criteria for getting integration. Integrating the qual-
such a project right the first time, ity and maintenance management
regardless of the industry, are: systems, which were focused on
++ Management support. What the pillars of quality and HSE,
Massy Wood had working in its and entwined with the adopted
REFERENCE
favor was buy-in from its own maintenance strategy, produced 1. John M. Campbell, “Effective Maintenance
top management, including effective outcomes, minimized Management,” www.jmcampbell.com.
involvement in the steering com- redundant measurements and
mittee activities (led by quality). optimized the following results:
BIBLIOGRAPHY
Without this, the endeavor ++ Fewer, more detailed “Effective Maintenance Management—
would have failed immediately. procedures. OM-21,” John M. Campbell & Co., www.
++ Choosing the right maintenance ++ Reduced equipment jmcampbell.com/applied-maintenance-
management-om21.php.
strategy. Massy Wood used a downtime. International Organization for Standardization
combination of maintenance ++ Documentation of the cost of (ISO), ISO 9001:2015—Quality management
strategies for maintaining its quality (prevention, appraisal systems—Requirements.
Nyman, Don, and Joel Levitt, Maintenance
facilities. More than 95% of its and rework costs). Planning, Coordination, & Scheduling, second
systems used a combination ++ Fewer, more effective audits. edition, Industrial Press Inc., 2010.
of preventive, predictive and ++ Integrated business risk man-
proactive maintenance. The agement analysis.
remaining 5% used a run-to-fail- ++ Individual strength of compo- Ray Stoute is a contract
ure model. Some of the strategies nent systems. It is important to delivery manager and
were inherited from the client, note that each element of the former quality manager at
Massy Wood in Trinidad and
and others were suggested and integrated system was robust Tobago. He earned an MBA
subsequently adopted. enough to stand on its own. But from Heriot-Watt University,
++ Early integration. The quality had there not been an integrated Champs Fleurs Campus, in
Trinidad and Tobago and a
element was brought to the table approach with quality manage- bachelor of science degree
at an early stage, which laid the ment being front and center, in industrial engineering from the University of
foundation for an integrated the stellar outcomes wouldn’t the West Indies in Kingston, Jamaica. Stoute is an
ASQ-certified quality engineer and a senior member
management system gov- have been realized to their full of ASQ, as well as a certified Project Management
erned by quality principles—a potential. Professional.

F E AT U R E
ANALYTICS
Sl am
D un k
d
d u n d e r va l u e
o fin
d a n a lytics t a s ketball
r ov e s y b
Im p a nta
r your f
am
e n , Ad
e r s f o la ine All
play an
, I. E m
Recck Julia E
. Sea
G eorge d
| b y a n
leaguer, Zachary Mittelm
ar k
e
Kershn

e
Just ths
Fact
t-
y baske
Fantas gues are
ball lea ing more
becom r, and rank-
popula d analytics
ings animportant
are an managing
p a r t of .
a team
tify the
To iden yers to
best pla r your
select foteam,
fantasy ating
a new r has been
systemped.
develo
yer
This plaon model
valuati es
examinuch an
how m ual player
individ utes to a
contrib overall
team’s al per-
field goe and free
centagpercentage.
throw

F E AT U R E
ANALYTICS
In the past 30 years, we have entered the world of big data. TA B L E 1
Fantasy league points awarded

The magnitude and accessibility of data—and the ability to
analyze those data—have advanced dramatically. This advance-
ment of data processing power can be seen in diverse fields,
including investing, web analytics and sports metrics.
Following this trend, fantasy sports have significantly devel-
for PTS and AST: 2016-2017
oped, both in terms of sophistication and popularity. Rankings Team PTS Points Team AST Points
and analytics in sports leagues are everywhere and becoming
SGFF 8,752 9 Mr. C 2,083 9
more important. One of the easiest places to see this is in
fantasy sports because the data are public, and there are online SalesDog 8,719 8 SalesDog 2,044 8
business and betting opportunities. The difficulty in summariz- Mr. C 8,352 7 The Illini 1,954 7
ing this data for choosing players for fantasy sports teams lies The Illini 8,235 6 FengD 1,919 6
in identifying which metrics are best at discriminating between Crew 8,156 5 Crew 1,917 5
the highest quality players and the rest of those in the league.
Viper 8,119 4 Viper 1,877 4
This article focuses on fantasy basketball, introducing a new
method to estimate the overall quality of a given basketball Ghost 8,012 3 SGFF 1,792 3
player, contrasted in tandem with current procedures used by Cobra 7,658 2 Cobra 1,627 2
sports websites to estimate player value. This approach has led FengD 7,284 1 Ghost 1,207 1
a few of this article's authors to three championships and two
AST = assists PTS = points scored
second-place finishes in the last eight years in a mature fantasy
basketball league.
cap, or have position or play time requirements, to
Fantasy basketball overview restrict and control team makeups. Most leagues
Previously overshadowed by baseball and football, fantasy even allow trading to occur among GMs.
basketball has been growing. In fact, fantasy basketball During the season, after the draft is completed,
revenue surpassed that of fantasy football in 2016 at FanDuel, GMs set their starting lineups for the starting period.
and it was projected to do so in 2017 at DraftKings. These two This can be done on a daily, weekly or monthly basis,
sites represent the largest fantasy gambling outlets.1 In addi- or just once for the whole NBA season, depend-
tion, a recent ruling by the U.S. Supreme Court gave states the ing on the fantasy league preferences. Thereafter,
ability to authorize sports betting, which all but nine states statistics for the players in a GM’s starting lineup
have now allowed.2 accumulate and are automatically ranked. The
Many believe the rapid growth of fantasy basketball is due to overall goal is to earn the most points in the league
sports fans’ desire to remain engaged with sports and fan- to win the season.
tasy leagues during Major League Baseball’s off-season. Like Despite the potential to win cash, the return
baseball, the National Basketball Association (NBA) has many on time invested is typically lower than minimum
games played daily, many traditional as well as new statistical wage. Thus, participating in a fantasy league is done
scoring measurements, and the advent of digital tracking to purely out of enthusiasm for the game and bragging
provide endless amounts of data per game or player. In addi- rights. Many leagues even have intermediate prizes
tion, data analysis opportunities are numerous: Most data in (monthly or daily), which can keep a team out of the
basketball are larger in magnitude than other sports, and much race for first place engaged in the league. These are
are continuous or pseudo-continuous in comparison to baseball. the typical eight categories involved in calculating
While it varies by group and hosting site, anywhere from four an overall fantasy NBA team ranking:
to 20 people can form a fantasy basketball league during the ++ Assists (AST).
NBA season. The participants act as general managers (GM) ++ Blocks (BK).
for their respective teams. These GMs meet to draft a roster of ++ Field goal percentage (FGP).
players from the pool of NBA players. After a player is drafted, no ++ Free throw percentage (FTP).
other GM can draft that player for his or her team. More recently, ++ Points scored (PTS).
drafts are typically held online. Drafts may be held all at once, ++ Three-pointers made (3PT)
over several days or even over several weeks. Many leagues ++ Total rebounds (TRB).
place dollar values on players and have a corresponding salary ++ Steals (ST).

Table 2 (p. 46) shows how the points earned in
each ranked statistical category throughout the
season are totaled to determine the league’s ending
standings. It is not necessary for a GM to win the
General most points in every category, but rather have the

highest average value per ranked category. Ideally,
the GM should aim to score a seven across the cate-
Calculation gories for the best chance to win the league.
Valuing players
of Added A player’s value for fantasy leagues is based on his
statistics in the ranking categories. In today’s mod-
Value ern setting, actual NBA data are readily available
online from sources such as CBS Sports, ESPN and
Tables 4 and 5 show how to calculate this added Yahoo Sports, among others. Most of these sites
value to FGP or FTP for an individual player. Below provide rankings for NBA players based on their
are the equations to do so for any player where proprietary formulas that they promote for use in
AvgFG and AvgFT represent the team’s average fantasy leagues. However, while these sites provide
number of field goals and free throws multiplied by
four to represent the other four players. AvgFGA a generalized overview of the league, not all fantasy
and AvgFTA represent average attempts, while leagues measure the same categories, so rankings
FB, FGA, FT and FTA represent those made by the may not be precisely tailored for a specific league.
individual player being evaluated. We were interested in deciphering these rankings
for two reasons:
(4*AvgFG)+FG AvgFG 1. To determine their relevance to the eight
FG Add = –
(4*AvgFGA)+FGA AvgFGA fantasy scoring categories.
2. To evaluate the quality of current ranking
(4*Avg FT)+FT Avg FT schemes and identify higher quality metrics
FT Add = –
(4*Avg FTA)+FTA Avg FTA
for choosing players.
For sake of brevity, all further analyses in this
After applying this calculation to the top 50 play- article will use the CBS Sports rankings.
ers, we can build the following regression models:
++ Predict FG Add based on FG made, FG missed and Multivariable modeling
FG attempted Multivariable linear regression models are used
++ Predict FT Add based on FT made, FT missed and
FT attempted. to evaluate the current ranking metrics using a
separate model by year for nine NBA seasons. Each
model calculates a coefficient for each of the eight
metrics as predictors of the overall CBS rank. Using
this method, each predictor’s coefficient controls for
the effect of the other seven metrics, and its statis-
Calculating league standings tical significance in the overall model is evaluated.
Table 1 illustrates how league points were awarded for PTS and Both the size of the coefficient and its consistency
AST categories based on the statistics accumulated over the over multiple years are important.
2016-2017 season of a mature fantasy NBA league. This model uses the last nine NBA seasons (from
The table shows that regardless of whether a player wins a 2008-2009 to 2016-2017) based on the top 100
category by one or 1,000, only nine points are assigned to the players as ranked by CBS Sports. Table 3 (p. 47)
category—thus, you need to win a category by just a single unit offers this nine-year history of regression coeffi-
to accumulate nine points. You might think of this in the same cients for each statistical category and their
way as winning a bid on a contract: A business bidding for ser- relative strength.
vices would want to offer just enough to beat its competitors, The strength of a category can be measured by
without sacrificing too much of its own resources. the size (weight) of the regression coefficients.

F E AT U R E
ANALYTICS
TA B L E 2 Looking only at the individual regres-
Final fantasy league standings: 2016-2017

sion coefficients and p-values for the
covariates for the 2016-2017 season (the
last column), we can use these coeffi-
Rank Team 3PT AST BK FGP FTP PTS ST TRB Total cients to form the ranking model
1 st
Illini 3 8 7 9 3 6 6 8 50 that follows.
The regression equation for 2016-2017
2nd SalesDog 9 3 8 5 2 8 5 3 43
to predict CBS rank is:
3 rd
Viper 8 6 4 3 9 4 3 5 42 CBS Sports rank = 212.6 + 0.22(3PT)
3 rd
Cobra 7 5 2 8 8 2 8 2 42 + 0.03(TRB) + 0.04(AST) + 0.29(ST)
5th Mr. C 4 9 9 6 1 7 1 4 41 + 0.35(BK) + 0.01(PTS) + 0.57(FGP) +
5 th
SGFF 6 4 3 2 6 9 2 9 41 0.34(FTP)
With the regression analysis results in
5th Ghost 5 7 1 1 7 3 9 1 34
Table 3, we have shown that sports web-
7th FengD 1 2 6 7 5 1 7 6 35 sites, such as CBS Sports, do not consider
9 th
Crew 5 1 8 1 4 1 2 3 25 FGP and FTP as high-quality metrics in
AST = assists BK = blocks FGP = field goal percentage FTP = free throw percentage developing their player rankings. This is a
PTS = points scored 3PT = three-pointers made TRB = total rebounds ST = steals major flaw given the importance of these
statistics as fantasy point categories. Our
subsequent analysis introduces a new
The change in importance in a particular category can be seen in the colors method to more accurately value players
of the heatmap (with red indicating the most important categories and of the highest quality for fantasy leagues
white identifying the least important categories). and further analyze CBS Sports’ method
Note that some categories are highly significant (p < 0.001) but do not to rank players.
add much weight to the overall ranking because of the small size of the
coefficient. In the heatmap, coefficients of 0.4 or greater are colored in red, Needle moving for a
coefficients between 0.3 and 0.39 are colored in dark orange, coefficients new valuation model
between 0.2 and 0.29 are colored in light orange, and the remaining coeffi- “Needle moving” represents a new
cients have no coloring. approach to valuing success in the
Throughout the years, it’s easy to note the consistently strong (low) FGP and FTP categories, raising their
p-values for the first six categories (3PT, TRB, AST, ST, BK and PTS) value in player evaluation. Essentially,
throughout the nine-year history. However, FGP and FTP emerge as we asked ourselves, “How much does
statistically insignificant most of the time. In the three seasons that FTP is player X improve a team’s overall FGP or
significant, its significance is borderline enough that it is still not a strong FTP?” In baseball, a similar term—wins
predictor of rank, especially in comparison to the six significant variables. above replacement—has also become a
The high p-values in the six years that FTP is statistically insignificant popular fan statistic.
also indicate that its occasionally low p-value is likely more a fluke than To answer this question, we calculated
anything statistically meaningful. There are also variabilities in the strength the added (or lost) value each player
of the associations, as noted in the heatmap colors in Table 3. 3PT, ST and contributes to the team’s overall FGP or
BK are consistently stronger than the other three statistics that show con- FTP as follows:
sistent significance. 1. We calculated the average field goal
These results indicate that FGP and FTP are not strong predictors of (FG) made and free throw (FT) data
rank, even though they are an important aspect of the fantasy scoring for the top 50 players to create a
component. This is further illustrated by comparing an initial model that baseline of the average player. We
includes FGP and FTP to a new model without FGP and FTP. Here, the chose the top 50 players as opposed
R-squared value in the initial model was 81.5%, whereas the R-squared to the top 100 to include those players
value in the new model that excludes FGP and FTP is 80.7%. Such a small likely to play most of the games.
decrease in this value indicates that FGP and FTP, while statistically signifi- Thus, their average is the profile of
cant variables, contribute a very small amount to explaining variation the average player they are likely
in rankings. to replace.

TA B L E 3
Heatmap and significance of regression coefficients 2008-2017

for each category
Category 2008-09 2009-10 2010-11 2011-12 2012-13 2013-14 2014-15 2015-16 2016-17
3PT 0.34*** 0.36*** 0.40*** 0.44*** 0.30*** 0.31*** 0.25*** 0.21*** 0.22***
TRB 0.07*** 0.07*** 0.08*** 0.10*** 0.04*** 0.05*** 0.06*** 0.05*** 0.03***
AST 0.08*** 0.08*** 0.10*** 0.12*** 0.09*** 0.08*** 0.05*** 0.05*** 0.04***
ST 0.23*** 0.32*** 0.31*** 0.43*** 0.27*** 0.33*** 0.27*** 0.25*** 0.29***
BK 0.29*** 0.33*** 0.43*** 0.54*** 0.41*** 0.41*** 0.36*** 0.33*** 0.35***
PTS 0.02*** 0.02*** 0.01*** 0.02*** 0.01* 0.02 0.02** 0.01** 0.01***
FGP 0.07 0.05 0.21 0.26 0.30 0.07 0.26 0.52 0.57
FTP 0.25 0.21 0.30* 0.18* 0.29 0.06 0.35 0.20 0.34*
* < 0.05 ** < 0.01 *** < 0.001

AST = assists BK = blocks FGP = field goal percentage FTP = free throw percentage
PTS = points scored 3PT = three-pointers made TRB = total rebounds ST = steals
2. We incorporated the FG and FT data for an indi- FT Add= 0.097 + 0.074(FT) - 0.06(FTA)
vidual player of interest. Using these regression coefficients, we can now create a new
3. The change in FGP or FTP from the four-player statistic for the percentage categories: field goal score (FGS)
team to the five-player team that includes player and free throw score (FTS).
X represents how much does player X moves the FGS= (1.8*FG) - (0.87*FGA)
needle. We call this the added (or subtracted) Each field goal made receives 1.8 points, while a penalty of
value that player contributes to (or costs) the team. 0.87 points is assigned to each field goal attempted.
The scope of the analysis that follows focuses on FTS= (7.4*FT) - (6*FTA)
the 2016-2017 season. Table 4 (p. 48) shows a calcu- Each free throw made receives 7.4 points, while a penalty
lation for NBA player Kevin Durant’s added value to of six points is assigned to each free throw attempted. We use
a four-player team’s FGP. We define a player’s added these scores in our construction of a new model: the player
(or subtracted) value calculated in step three as FG valuation model.
Add (when calculating the change in FGP) or FT Add
(when calculating the change in FTP). New player valuation model
Table 5 (p. 48) gives the same calculation using FT For a team to have the best chance of winning the league, the
added value rather than for FGs. Again, Durant rep- team should try to achieve a “seven,” or close to it, in each cat-
resents player X. As in Table 4, the values for players egory. Given that there are typically eight statistical categories,
one through four represent the averages for the top we can assume that a score of 56 (eight categories x seven)
50 players. See the sidebar “General Calculation of points should win the league. Of course, the number of points
Added Value” for more explanation. it takes to win the league will vary slightly by season, but 56
Between the two years for which we performed points is a good goal to set.
the earlier calculations, there was a small increase in To build the player valuation model, we begin with PTS as a
the added value, although this was not significant. baseline value. From Table 1, in 2016-2017, to “beat the seven”
We use the coefficients from these models to calcu- in points, 8,352 points were needed. In addition, to “beat the
late our new player valuation model. seven” in assists, 1,954 points were needed. The ratio of these
Using the data from Tables 4 and 5, we get the two-point totals shows that each assist is worth 4.3 points.
following regression equations: With this ratio, the individual category weights for each
FG Add = -0.02 + 0.018(FG) - 0.0087(FGA) statistical category are determined in “points.” These weights

F E AT U R E
ANALYTICS
TA B L E 4 TA B L E 5
Needle moving—FG 2016-2017 Needle moving—FT 2016-2017

Player # FG made FG missed FG attempts FGP Player # FT made FT missed FT attempts FTP
Player 1 492 586 1,078 45.6% Player 1 266 72 338 78.7%
Player 2 492 586 1,078 45.6% Player 2 266 72 338 78.7%
Player 3 492 586 1,078 45.6% Player 3 266 72 338 78.7%
Player 4 492 586 1,078 45.6% Player 4 266 72 338 78.7%
Team total 1,968 2,344 4,312 45.6% Team total 1,064 288 1,352 78.7%
Durant Durant
698 683 1,381 50.5% 447 51 498 90.0%
“Player X” “Player X”
New FGP New FTP
New New
2,666 3,027 5,693 46.8% 1,511 339 1,850 81.7%
team total team total
Kevin Durant’s added Kevin Durant’s added
1.2% 3%
value to FGP value to FTP
FG = field goals FGP = field goal percentage FT = free throws FTP = free throw percentage
are used to calculate a rating for each player. Note that these Figure 1 is a histogram that shows the number
category weights vary slightly from year to year based on of players plotted against the NVR for the 2016-
that season’s rankings, similar to the regression coefficients. 2017 season. Therefore, after getting past the elite
However, as with regression coefficients—except in the case of players, this model can be used to find undervalued
a significant trend—this variation is small and therefore can be players to acquire or overvalued players to avoid.
used to determine next season’s points thresholds. The histogram shows most players in the 90-150
We call this newly calculated rating the new valuation rating range with two elite players, James Harden and
(NVR), which can be calculated as shown below. This rating Russell Westbrook, who are well above 220.
incorporates the new approach to FGS and FTS introduced in For example, DeMar DeRozan was ranked 72nd by
the needle moving section. CBS, but had an NVR of 150 that ranked 23rd overall.
NVR= (1*PTS) + 11.7(3PT) + (2.8*TRB) + (4.3*AST) + 17*ST) + This suggests DeRozan may be a better fantasy
(21.5*BK) + [(2.5*FG) - (1.1*FGA)] + [(7.4*FT) - (6*FTA)] draft pick than CBS expected. This is from the fact
This new rating system helps identify undervalued—or that his FG and FT scores may have been underval-
falsely overvalued—players from traditional ranking systems ued by CBS.
by using points as a baseline value. In addition, players are There are many more examples of differences
rated on a per-game basis, which accounts for how frequently between the CBS rank and the NVR, and these
they are playing. This can often drastically increase (or players may represent the under or overvalued draft
decrease) a player’s value, causing an inconsistency between picks for a fantasy team. While optimizing single
the rating and the CBS ranking. In addition, the new FGS and player value is important, complementing across the
FTS metrics incorporate the added (or lost) value each player combined team statistics is also important to score
brings to the team. in every ranked category. Draymond Green, ranked
The player valuation model using the rating will help a fantasy by CBS as seventh overall, had an NVR of 149, which
league participant analyze the value of each individual player ranks 25th because he had negative FG and FT scores.
to construct a well-rounded team capable of winning in many Isaiah Thomas, Kawhi Leonard, Damian Lillard
scenarios. Because it is nearly impossible to attempt to win all or Jimmy Butler would all serve as better picks
eight categories, this model values players to “beat the seven,” over Green, given that they all had an NVR of 175 or
which, if achieved in all statistical categories, nearly guarantees better, with strong FG and FT scores. Note that all
winning the league overall. of these players played between 74 and 76 games,

which is important because CBS creates its ranking rank, whereas the remaining two (in this case, FGS and FTS) are
based on aggregate totals rather than per-game not significant predictors. This result confirms that CBS Sports
totals, so there is a dampening effect for players is not incorporating FG and FT data into its player rankings,
who played many games versus players with a few indicating a possible flaw in its rankings.
good games. While the FGS and FTS metrics we created are not statisti-
cally significant predictors of CBS rank, they help identify the
New predictor model for CBS rank added (or subtracted) value an individual player brings to a
Using the newly calculated score values (FGS and given team and increase the quality of player rank by giving
FTS) for the top 100 players, we return to pre- more weight to consistency of play and added value to a team.
dicting CBS rank using a regression analysis. The
purpose of this second attempt at predicting CBS Player valuation model and imbalances
rank is to analyze the p-values for FGS and FTS, Despite the plethora of data now available to analysts, popular
to evaluate whether websites such as CBS Sports fantasy sports websites such as CBS Sports, ESPN and Yahoo
are incorporating FG and FT data in any way to Sports are at a loss for how to use some of that data to rank
calculate rankings. We already established that the players. Through numerous regression models over a nine-year
raw percentages are not significant, but we now span, we saw that the percentage categories (FGP and FTP)
analyze whether the weighted scores we created are not statistically significant predictors of player rank.
would be meaningful impactors in predicting We created a new rating system—rather than a ranking
CBS rank. system—to identify hidden value in players (or even some-
Table 6 (p. 50) shows the regression output times, overvalued players). To do this, we began by what we
for predicting CBS rank based on the same eight call “needle moving,” which asks how much an individual player
categories, except with scores (FGS and FTS) rather contributes to a team’s overall FGP or FTP. This player’s contri-
than points (FGP and FTP). bution may either increase or decrease the team’s percentage.
Here, as with our initial model discussed in Tables Not only does this help to identify higher-quality players, but
2 and 3, the first six categories (3PT, TRB, AST, ST, by building a regression model to predict this added value (“FG
BK and PTS) are highly significant in predicting CBS Add” or “FT Add”), we can use those regression coefficients
FIGURE 1
Histogram of new valuation ratings 2016-2017

16
14
12
Number of players
10
0
60 90 120 150 180 210
Rating

F E AT U R E
ANALYTICS
TA B L E 6 REFERENCES
1. Daniel Roberts, “The Growth in Fantasy Sports Will Not Come
Revised model coefficients From Football,” Yahoo! Finance, Sept. 28, 2017, https://tinyurl.com/
yahoo-fantasty-basket.
2. The Supreme Court of the United States, “Murphy, Governor of New
and p-values: 2016-2017 Jersey, et al. v. National Collegiate Athletic Assn. et al.” opinion, May 24,
2018, www.supremecourt.gov/opinions/17pdf/16-476_dbfi.pdf.
Coeffi- P-value
Predictor cient
AST = assists
3PT 0.23 0.00 BK = blocks
TRB 0.03 0.01 FGP = field goal George Recck is director of the math
percentage resource center at Babson College in
AST 0.04 0.00 FTP = free throw
percentage Wellesley, MA, and founder of Total
ST 0.28 0.00 PTS = points scored Information Inc., an information
3PT = three-pointers consulting businesses providing service
BK 0.35 0.00 made to small businesses. He holds an MBA
TRB = total rebounds from Babson College.
PTS 0.02 0.01 ST = steals
FGS 0.04 0.06
FTS 0.01 0.36
I. Elaine Allen is professor of

biostatistics at the University of
California, San Francisco, and emeritus
to create a new metric for the field goal and free professor of statistics at Babson College.
throw categories: FGS and FTS. She is also director of the Babson Survey
Thus, we created the player valuation model Research Group. She earned a doctorate
in statistics from Cornell University in
that seeks to rate each player based on per-game Ithaca, NY. Allen is a member of ASQ.
performance. Seeking to “beat the seven” in each
statistical category (excluding FGs and FTs), we
calculated the ratio of points to a given category,
using the values that achieved “the seven.” This ratio
represents that statistical category’s weight. By
combining these weights with the FGS and FTS met- Adam Kershner is a senior at Babson
College with a concentration in business
rics we created, we arrive at the new valuation rating. analytics. After graduation, he will
For future research, the player valuation model begin a career in consulting at Ernst &
can be adjusted to identify trends before the regres- Young in transaction advisory services.
sion analysis. For example, the nine-year history of
regression coefficients shows an increase in value
and a devaluing of three pointers as more and more
players become high-quality three-point shoot-
ers. Our category weights in the player valuation
Zachary Mittelmark is a senior
model agree with this, and also show the same associate at PNC Riverarch Capital in
trend. However, this model would be more useful if Pittsburgh. He has a bachelor’s degree
in business administration from Babson
it could identify this trend first, among several other
College.
potential trends.
You also might use the player valuation model in
conjunction with scheduling imbalances, where NBA
teams win several consecutive games, by taking Julia E. Seaman is research director
of the Quahog Research Group and a
advantage of the imbalances in a week-to-week for- statistical consultant for the Babson
mat. This could greatly enhance your fantasy team’s Survey Research Group at Babson
performance and competitive advantage over other College. She earned her doctorate
in pharmaceutical chemistry and
players in the league who are only using the website pharmacogenomics from the University
rankings to select players for their roster. of California, San Francisco.

CONNECT WITH QUALITY
LEADERS AND PEERS AT myASQ!
What is myASQ?
myASQ is a centralized online community providing timely, relevant, and
customizable engagement for members of ASQ and quality professionals
around the globe.
At myASQ, you’ll be able to stay informed of community news and
events, initiate and contribute to discussions about quality topics important
to you, and learn the information you need to succeed.
How do I access myASQ?

Access myASQ at my.asq.org, signing in using your asq.org username
and password. If you don’t have an asq.org username and password,
register for immediate access. Complete the myASQ Welcome Tour,
which includes steps to update your profile and account settings, and
introduce yourself in the dedicated discussion board.
Visit my.asq.org today

to get started!
Solving quality quandaries through statistics
Statistics Spotlight 1213231315447133415341545721157945495241262

124345464755497541275299949514654237204317
91545154134541213231315447133415341545721157945495241262
124345464755497541275299949514654237204317
91545154134541213231315447133415341545721157945495241262
124345464755497541275299949514654237204317
91545154134541213231315447133415341545721157945495241262
124345464755497541275299949514654237204317
91545154134541213231315447133415341545721157945495241262
124345464755497541275299949514654237204317
91545154134541213231315447133415341545721157945495241262
124345464755497541275299949514654237204317
9154515413454
The Complete
Picture
Together, NPS and CSAT metrics can better gauge customer loyalty
by Christopher A. Seaman, I. Elaine Allen and Julia E. Seaman
The net promoter score (NPS) was proposed by Calculation and eight are called passives.
Frederick Reichheld in 2003 as “the one number of the NPS The original question metric
you need to grow.” 1 It is now ubiquitous in market- Reichheld’s single question has 11 levels, though it can
ing and for reporting organization growth. to measure satisfaction of a appear with just 10 levels
It is based on a one-question survey that asks customer (“How likely is it (one to 10).
the respondent the likelihood of recommending a that you would recommend While the NPS respondent
product or organization from zero to 10, using an [brand or organization X] to a may choose between up to 11
extended Likert scale. However, given the process of friend or colleague?”) has the levels, the results are grouped
creating one's score, this metric is condensed into a respondent choosing a num- into just three. Specifically,
dichotomous two-point framework, and much of the ber between zero and 10, with the NPS that is reported is
information collected is ignored.2 Even Reichheld zero labeled “not at all likely” calculated as the percent-
now admits that his initial findings were imperfect, and 10 labeled “extremely age of promoters minus the
and a causal connection between the NPS and likely.” The distribution of the percentage of detractors and,
growth has not been validated.3 responses is calculated and according to Reichheld, any
Despite the evidence against its use, the NPS has summarized in each numbered positive score is good and
become ingrained in business analytics. It is an easy category and transformed into indicates growth, with a score
score to calculate, compare and interpret, driving a single statistic. of 50 or more being stellar.
its use and adoption. It is now its own industry with The respondents, who The middle group—the
specialized graphics and benchmarks tailored to give ratings of nine or 10, are passives—are ignored. None
industries and the Fortune 500.4 In this article, we combined and called promot- of the ordinal nature of this
statistically explore the NPS to demonstrate the ers, while respondents with scale is being used because
variability and indeterminate value, highlighting how ratings from zero to six are it is being reduced to just
any one NPS can be derived from many different called detractors. Those in the two levels (promoter or
survey scenarios. middle with ratings of seven detractor) without regard to

FIGURE 1
Net promoter scores (NPS) for different

percentages of passive respondents
With 100 respondents—21 ways to get an NPS of 60
Detractor % Passive % Promoter %
100%
75%
Clearly, it is important
to understand the
50% percentage of your
sample the net promoter
score represents, as well
25% as the actual score.
0%
0% 20% 40% 60% 80%
Passive percentage (%)
any differences between a The score stays the same if one pro- The possible combinations for these NPS
definitive detractor (= 0) and moter and one detractor both change values of 10, 30 and 60 are shown in
a marginal detractor (= 6). their ratings to become passives. plots in Figures 1-4 (pp. 53-55).
As the number of respondents These plots illustrate that although an
How many ways increases, the number of combinations NPS might be identical to another NPS,
can we obtain the with the same score increases as well. making any conclusions that they are
same score? With 10 respondents, there are five measuring the same thing—when in one
For the NPS, businesses ways to achieve an NPS of 10. With 100 case, all or most of the respondents are
want to maintain and report respondents, meanwhile, there are 46 promoters or detractors, and in another
as high a score as possible, different combinations of promoters, case, they are mostly passives—would
ideally above 50. However, passives and detractors. Also, even with be incorrect. It is rare for any business to
there are multiple ways of the same number of responses, the provide details around its reported NPS,
getting the same NPS with number of combinations possible for a so determining where one is on these
different combinations of particular score increases as the score examples is almost impossible.
promoters, passives and gets closer to zero. With 100 respon- Another important issue with the NPS
detractors depending on dents, there are 46 combinations for an is its lack of confidence bounds. This
the score and the number of NPS of 10, 36 ways for an NPS of 30 and interval estimate would be calculated
ratings. This is because the 21 combinations for an NPS of 60. based on the total number of respon-
method of calculation does Clearly, it is important to understand dents in the promoter and detractor
not account for total num- the percentage of your sample the NPS categories. The properties of a variety of
bers or the passive group: represents, as well as the actual score. methods of examining interval estimates

124345464755497541275299949514654237204317
91545154134541213231315447133415341545721157945495241262
124345464755497541275299949514654237204317
91545154134541213231315447133415341545721157945495241262
124345464755497541275299949514654237204317
91545154134541213231315447133415341545721157945495241262
124345464755497541275299949514654237204317
91545154134541213231315447133415341545721157945495241262
124345464755497541275299949514654237204317
91545154134541213231315447133415341545721157945495241262
124345464755497541275299949514654237204317
9154515413454
FIGURE 2

Detractor % Passive % Promoter % of the NPS are given in by
Brendan Rocks,5 a data
100%
scientist, and show that as the
number of passives increase—
regardless of the total
75%
respondents and the value
of the NPS—the confidence
interval will increase, and
50%
there will be less confidence
in the calculated NPS value.
The precision of your NPS
25%
is only as good as those
respondents included in its
0% calculation. If only 10% of
your respondents are either
0%
4%
8%
12%
16%
20%
24%
28%
32%
36%
40%
44%
48%
52%
56%
60%
64%
68%
72%
76%
80%
84%
88%
promoters or detractors
Passive percentage (%) that are calculating your
NPS, adding the 90% who
are passive respondents
FIGURE 3
will increase your precision,
Net promoter scores (NPS) for different but provide no information

about your brand. Using the
percentages of passive respondents total number of respondents

will give you a narrower
confidence interval than
With 100 respondents—36 ways to get an NPS of 30 your actual value indicates.
Detractor % Passive % Promoter %
An alternative to
100%
the NPS: the CSAT
Another metric that can be
used with similar data is the
75%
customer satisfaction (CSAT)
metric. In theory, CSAT mea-
sures a different sentiment
50%
than the NPS. CSAT attempts
to capture whether the
25% customer is content with his
or her interaction or purchase,
while the NPS is concerned
0% with brand loyalty.
The difference with CSAT is
0%
2%
4%
6%
8%
10%
12%
14%
16%
18%
20%
22%
24%
26%
28%
30%
32%
34%
36%
38%
40%
42%
44%
46%
48%
50%
52%
54%
56%
58%
60%
62%
64%
66%
68%
70%
the prompt for the customer

Passive percentage (%) is slightly different: “How

FIGURE 4

satisfied were you with your Detractor % Passive % Promoter %
experience today?” with a
100%
scale from one (worst) to
10 (best) for the customer
to choose from. Other
75%
ranges are possible with
CSAT (one-to-three scales
or one-to-five scales are
50%
common), but we will con-
tinue to use the one-to-10
scale shared with the NPS.
25%
Calculating CSAT is even
more straightforward than
the NPS: Simply take the 0%
average of all the ratings. This
0%
2%
4%
6%
8%
10%
12%
14%
16%
18%
20%
22%
24%
26%
28%
30%
32%
34%
36%
38%
40%
means that it is possible to
have different CSAT scores Passive percentage (%)
and standard deviations for
the same NPS and number
TA B L E 1
of responses. Table 1 demon-
strates some of the values
possible in even a small set CSAT vs. NPS vs. Std Dev
of responses with
Score Responses Score Responses Score Responses
the same NPS.
While your NPS equals 10 1 4 1 1
for all three of these survey 2 2 2
responses, CSAT is different 3 3 3
and, more concerning, its
4 4 4 Despite the evi-
variability changes an order
5 5 5 dence against its
of magnitude from 0.3 to 4.5,
6 6 4 6 use, the NPS has
indicating a different pattern of
become ingrained
CSAT and number of custom- 7 1 7 1 7
in business ana-
ers recommending a brand 8 8 8 9 lytics. It is an easy
to other potential customers.
9 9 5 9 1 score to calculate,
Incorporating variabil-
10 5 10 10 compare and inter-
ity into the estimate of
Total 10 Total 10 Total 10 pret, driving its use
CSAT is important, and
and adoption.
including the passives in CSAT 6.1 CSAT 7.6 CSAT 8.1
the calculation is essential, NPS 10 NPS 10 NPS 10
the interpretation of the
Std Dev 4.5 Std Dev 1.5 Std Dev 0.3
CSAT and the NPS together
may give additional insight CSAT = customer satisfaction
NPS = net promoter score
into customer behavior. Std Dev = standard deviation

124345464755497541275299949514654237204317
91545154134541213231315447133415341545721157945495241262
124345464755497541275299949514654237204317
91545154134541213231315447133415341545721157945495241262
124345464755497541275299949514654237204317
91545154134541213231315447133415341545721157945495241262
124345464755497541275299949514654237204317
91545154134541213231315447133415341545721157945495241262
124345464755497541275299949514654237204317
91545154134541213231315447133415341545721157945495241262
124345464755497541275299949514654237204317
9154515413454
BIBLIOGRAPHY
Birkett, Alex, “What Is Customer Satisfaction Score
(CSAT)?” Aug. 16, 2018, HubSpot, https://tinyurl.com/
birkett-blog-nps.
CJM Research, “Limits of the Net Promoter Score
(NPS) System,” May 13, 2016, https://tinyurl.com/
Seeing a complete picture cjm-research-nps.
The popularity of the NPS is not likely to change in the near future. While Hayes, Bob E., “Customer Loyalty 2.0,” Quirk’s Marketing
Research Review, 2008, https://tinyurl.com/
further evidence disproves its initial purpose, NPS reporting has become nps-hayes-quirk.
a standard and expected brand analysis. As we have demonstrated here, Spool, Jared M., “Net Promoter Score Considered Harmful,”
however, the NPS may not tell the whole story, and there may be a large User Interface Engineering, 2017 https://tinyurl.com/
uie-spool-nps.
difference between two brands with the same score. An alternative to the
NPS, the CSAT, does show the variability in the scores, and examining it in
the context of a given NPS is helpful.
NPSs don’t tell the whole story on their own. Tracking NPS is better than
having no measurement of consumer sentiment, but to understand how Christopher A. Seaman is
your customers feel, you must look at the distribution of the ratings. There- director of data science of the
fore, while the NPS can be a good comparator tool, it is useful to try to look Quahog Research Group in
Oakland, CA, and a statistical
behind the numbers to see a complete picture. consultant for Babson Survey
Finally, by asking “How likely is it that you would recommend [brand or Research Group at Babson
organization X] to a friend or colleague?” you are asking the respondent College in Wellesley, MA. He
earned his master’s degree
to judge the usefulness of the product or brand for someone else. What if in mathematics and cryptography and is all but
I think that the product is far too simple for my complex needs, but would dissertation (ABD) in mathematics from the City
work for my friend’s simple application? It would be an error to assume University of New York.
that I was loyal to the product, I would probably always avoid it.
Likewise, I could love it, but think my own needs are unique, and therefore
I might not recommend it for my friends. Of course, if I have little under- I. Elaine Allen is professor
of biostatistics at the
standing of what my friends actually need, I’d tend to give some middle
University of California, San
ground ranking, which would show no relationship to the quality of the Francisco, and emeritus
product, but rather my not understanding the specific needs of my friends. professor of statistics at
Babson College. She is also
This can be far worse when ranking a specific product rather than a brand
director of the Babson Survey
because the match of product to user can be specific. Research Group. She earned
a doctorate in statistics from Cornell University in
REFERENCES
Ithaca, NY. Allen is a member of ASQ.
1. Fredrick Reichheld, “The One Number You Need to Grow,” Harvard Business Review,
December 2003, https://tinyurl.com/hbr-nps-reichheld.
2. Daniel B. Schneider, Matt Berent, Randall Thomas and Jon S. Kosnick, “Measuring Julia E. Seaman is research
Customer Satisfaction and Loyalty: Improving the ‘Net-Promoter’ Score,” 2008, director of the Quahog
Semantics Scholar, https://tinyurl.com/schneider-cust-sat. Research Group and a
3. Timothy L. Keiningham, Bruce Cooil, Tor W. Andreassen and Lerzan Aksoy, “A statistical consultant for the
Longitudinal Examination of Net Promoter and Firm Revenue Growth,” Journal of Babson Survey Research
Marketing, Vol. 71, July 2007, pp. 39-51. Group at Babson College.
4. Customer Guru, “Net Promoter Benchmarks for Fortune 500 Companies,” 2018, She earned her doctorate in
https://customer.guru/net-promoter-score/fortune-500. pharmaceutical chemistry
5. Brendan Rocks, “Interval Estimation for the ‘Net Promoter Score,’” American and pharmacogenomics from the University of
Statistician, Vol. 70, No. 4, 2016, pp. 365-372. California, San Francisco.

SIX SIGMA: BUILD IT BETTER,
SMARTER, AND FASTER
Whether you’re new to quality or an experienced professional, ASQ has the
lean and Six Sigma solutions you need to improve processes, productivity, and
profits at every level. You’ll find introductory courses and certifications to more
strategic executive-level training.
Choose the format that fits your schedule and budget—from classroom or virtual
training led by leading subject matter experts to web-based, self-paced courses.
asq.org/leansixsigma
VISIT for more INFORMATION!
Standards-related developments and activities
Standard Issues
AS9100
We Have Liftoff
A look at what made the transition to the AS9100D series of standards a success
by L.L. “Buddy” Cressionnie
The transition to the AS9100D series of quality FIGURE 1
Integrated transition activities

management system standards for aviation,
space and defense (ASD) organizations ended
Sept. 15, 2018. Here’s a look at how it went:
The international ASD transition rate was 99.1%
as of Sept. 16, 2018, and continues to climb. Many
groups deserve thanks for this success, including 1. Standards
developed and
the organizations implementing the changes, the released
certified auditors performing the transition audits,
and the certification bodies processing the audit
artifacts and issuing the certificates.
To transition the industry to the AS9100D series of
standards, several complex and integrated activities
had to occur (see Figure 1): 4. Organization 2. Auditor
implementation training and
and transition certification
1. Standards developed and released
AS9100D development was the culmination of
four years of work. The deployment support
material was enhanced for the AS9100D series
transition. Expanded materials were provided
years before the AS9100D series of standards was 3. Certification process
published on the International Aerospace Quality established: ABs/CBs/
OASIS
Group (IAQG) website.
ISO 9001:2015, which is the AS9100 series base-
line, was published in September 2015. September
2016 started the two-year transition period from
AB = accreditation bodies
the AS9100C series of standards to the AS9100D CB = certification bodies
series of standards. OASIS = Online Aerospace Supplier Information System
2. Auditor training and certification were required to pass the training to be designated as competent to
A critical component of AS9100 series conformity perform audits.
assessment is ensuring auditor knowledge of the Probitas, the auditor authentication body in the Americas, was
new requirements and consistency in their appli- required to update the auditor certification status. Initially, there was
cation. The IAQG partnered with its contracted concern that a significant number of auditors wouldn’t transition to the
training developer, Plexus, to produce online train- new requirements. However, as seen in Table 1, the number of auditors
ing modules to train and test auditors’ knowledge qualified to perform audits on the 2016 standard has increased for
of the new requirements. All aerospace auditors AS9100, AS9110 and AS9120.

3. Certification process established
Accreditation bodies (AB) have a responsibility to ensure their accred-
ited certification bodies (CB) are operating within the industry scheme
when requirements for a standard change. For the AS9100D series
ASD Expectations
The International Aerospace Quality Group iden-
transition, ABs monitored the accreditation status of CBs to meet tifies the following points as expectations for the
industry requirements. aviation, space and defense (ASD) industry:
++ A focus on product safety.
The certification process that was established starts with CBs ensuring
++ Improved first-pass quality. Do it right the first
their processes, which are verified by ABs, meet industry requirements. time—no exceptions.
The CBs communicate requirements to certified organizations and their ++ On-time delivery of products and services.
auditors regarding the certification transition. CBs provide a timescale ++ Repeatable processes that deliver value.
for certification and required competency of the audit team, and create ++ A low cost of quality and responsibility of quality
shared by all employees.
certification decision-making resources for their certified organiza-
++ Less oversight by regulatory agencies as their
tions. CBs then conduct process-based transition audits of the revised confidence in ASD processes builds.
requirements. The CB technical committee reviews the audit artifacts ++ Cost savings and improved shop floor efficiency.
and AS9100D series certifications are released. —L.L.B.C.
Additionally, the Online Aerospace Supplier Information System’s
(OASIS) audit and certificate database, called OASIS Next Generation, Organizations didn’t struggle with many of the
was upgraded to improve functionality and released for use during tran- new concepts identified by the industry as risk areas.
sition audits. OASIS improvements provide additional search capability This may be due, in part, to the communication and
and mistake proofing when auditors populate AS9101F forms when deployment support materials available to organiza-
performing CB audits. tions regarding the transition.
4. Organizational implementation and transition Working together

Organizational implementation varied and typically included training Overall, the organizations implementing the
and understanding the new requirements, gap assessment of the new changes, the certified auditors performing the
requirements, implementation, internal audits and management reviews. transition audits and the CBs all did an excellent job
The distribution of nonconformances issued by CBs during AS9100 series transitioning to the AS9100D standards as planned.
transition audits are illustrated in Online Figure 2 (which can be found on The successful result is due to the coordination and
this column's webpage at qualityprogress.com). The top five drivers were: partnership of all interested parties working together.
1. Clause 8.4: Control externally 3. Clause 7.1: Resources. The IAQG continues to stress the ASD expectations
provided processes, products 4. Clause 8.1: Operational planning for the industry (see the sidebar, “ASD Expectations”)
and services. and control. and to ensure auditors and organizations understand
2. Clause 8.5: Production and 5. Clause 10.2: Nonconformity and and evaluate the effectiveness of AS9100D series
service provision. corrective action. requirements.
TA B L E 1 L.L. “Buddy” Cressionnie is
Number of certified aerospace auditors

president of ASD Expertise,
LLC, with industry leadership
positions of Americas
Aerospace Quality System
(as of 10/3/2018) Committee (AAQSC) chair and
Certified auditors AAQSC leader of requirements,
projects and AS9100. He is
2009 2016 active in standards development
9100 1,080 1,316 as a liaison member to the International
Organization for Standardization (ISO) Technical
9110 202 224 Committee (TC) 176, including writing ISO 9001:2015
and ISO 9004:2018, and participating in the ISO
9120 354 367
9001 Interpretations Committee.

Newly released products and tools
Marketplace
FUSE
Fuse offers
extended ranges
Schurter extends the rated current range of its SHF
6.3x32 down to 500 mA for a total 19 rated currents,
ranging from 500 mA to 32 A, including the newest 500
mA, 630 mA and 800 mA versions.
The SHF 6.3x32 is a ceramic fuse for higher voltages with fast tripping charac-
teristics. The high dielectric strength and high breaking capacity of 1500 A at rated
voltage of up to 500 VAC/VDC exemplifies the high performance of this fuse series,
while also allowing for new areas of application.
Besides applications in the energy and industrial sector, the SHF 6.3x32 is
suitable for the protection of three-phase systems. The compatibility with existing
form and fit products on the market makes the SHF 6.3x32 an ideal substitution for
applications with demand for increased ratings in the 6.3x32 mm size.
www.schurterinc.com | 800-848-2600
SWITCH
User-
LASER MODULES
Modules designed for

alignment applications friendly
remote
switch
BEA Lasers has introduced
two new additions to their
MIL series of laser diode
modules for alignment CBS ArcSafe has released the Chicken
applications. The MIL RA Switch brand of remote switching
alignment modules feature devices. The Chicken Switch’s design is
a low profile 3/8” rugged lightweight and compact. It is a remote
laser housing, fitted with switch kit for select low-voltage elec-
an M12 connector. A PVC trically operated control switches and
jacketed cable two meters circuit breakers.
long, integrated power sup- Under CBS ArcSafe, the Chicken
ply and optional mounting Switch devices will retain their original
bracket complete the sys- root part number from Martek Limited
tem. The optional mounting (for example, CS01) but will feature the
bracket is available in sensor style or as a multi-adjustable “LB” bracket. The modules added prefix RSK (remote switch kit),
can be obtained in straight or right-angle configuration. which designates that the devices are
The MIL RA series laser diode system is available in standard 520 nm (green) sold as a complete solution or kit. The
or 635 nm (red), with 1 mW or 5 mW output. The MIL RA series laser modules are Chicken Switch does not require a sep-
typically used for alignment and leveling applications. They are ideal for counting, arate remote switch operator as legacy
machine vision, edge detection, paper web, riveting and water jet cutting alignment. CBS ArcSafe equipment does.
www.bealasers.com | 800-783-2321 www.chickenswitch.com | 877-472-3389

ANGLE ENCODER
Sealed angle encoder ideal

for machining applications
Heidenhain’s sealed angle encoder can be used as an alternative to modular angular encoders for
use in motion control machining applications that require a larger-than-usual through-hole. The
RCN 6000 series absolute sealed angle encoders with integral bearings have a large hollow shaft
diameter of 180 mm, allowing larger-scale machines to take advantage of high resolution and
signal quality.
Common applications for the RCN 6000 encoders are large-scale rotary tables and radio
telescopes that can adapt their design to the encoder while the integrated stator
coupling ensures alignment of the components involved in the scanning
process, even if there are deviations in the shaft coupling.
The RCN 6000 series encoders uses Heidenhain’s metallur process
(in which the graduation is applied directly to the bearing ring) and a
reflected light scanning method, which gives it compact dimensions
and larger mounting tolerances when compared to other products of
its type. These integrated features—along with the compact design—
minimize mounting difficulty while making the most of its unique shape.
www.heidenhain.us | 847-884-4761
PHOTODIODES
Photodiodes convert broadband

light to photocurrents
Marktech Optoelectronics’ InGaAs/InP PIN photodiodes are
designed to convert broadband light into photocurrents within
the VIS-SWIR range. In addition to their low noise, high sensitivity
and high-speed response, they feature wide standard spectral
ranges of 0.6 μm to 1.7 μm (1.7 μm material), low dark current and
high efficiency, typically 0.6A/W. Standard model MTPD1346D-xx
series broadband PIN photodiodes are offered in active area sizes
from 0.1 mm to 3.0 mm.
Each is packaged in a hermetically sealed, three-pin TO-46
metal can with flat lens cap, with a choice of either through-hole
or surface mounting. The photodiodes may be integrated seam-
lessly, together with Marktech SWIR emitters, as a single, compact package.
Marktech MTPD1346D-xx series InGaAs/InP broadband PIN photodiodes are designed to support applications in
which high-speed performance, high data rates, low dark current, small capacitance and smaller active area sizes
are required. Typical applications include aerospace, automation, autonomous vehicles, high-speed communica-
tions, industrial controls, medical, material and chemical analysis, security and wearables.
www.marktechopto.com | 518-956-2980

Agile Methods for Safety-Critical Systems: A Primer Using Medical Systems
NANCY VAN SCHOOENDERWOERT AND BRIAN SHOEMAKER ❚ CREATESPACE INDEPENDENT PUBLISHING
PLATFORM ❚ 2018 ❚ 130 PP. ❚ $40 (BOOK).
Quality and safety are integral The culture of learning and the ability to document incremen-
aspects of software-driven medical tally is the benefits of agile. This helps fulfill legal and regulatory
and diagnostic devices. The authors obligations pertaining to various aspects including traceability,
lament that ensuring quality, safety safety, risk and design. The fourth chapter deals with applying
and performance through validation and documentation often agile principles. There is no one way of implementing agile and
is viewed by organizations as an unavoidable cost and not as they share several agile implementation situations.
a benefit. The authors saw an alternative to this outlook in an The book details forming agile teams, documentation and
approach used by agile teams that integrated quality control risk management. The rest of the book details agile project
activities into the development process leading to better and implementation, such as planning techniques, and impact and
safer products. story-mapping techniques. The last part of the book discusses
The book is organized into 12 chapters. The first part of how to track progress and accelerate learning when using agile.
the book provides a detailed explanation of the agile method The last chapter discusses maintaining agile.
and its origins. The authors contend that agile is a mindset This book is written in a simple and readable manner with
and, in the software world, is established through four val- emphasis on understanding and implementation. Theory is kept
ues, grounded by 12 principles, and manifested through many to a minimum. This book should be in the reading list of quality
different practices. The dual feedback loop comprised of agile professionals and those appearing for the ASQ certification exams.
management practices and agile technical practices is discussed. —Rangarajan Parthasarathy, Harvard, IL
Additions to your quality library
Footnotes
Quality Experience Telemetry: How to Effectively Use Telemetry for Improved
Customer Success
ALKA JARVIS ❚ LUIS MORALES ❚ JOHNSON JOSE ❚ ASQ QUALITY PRESS ❚ 2018 ❚ 204 PP. ❚ $45 MEMBER ❚ $75 LIST
(BOOK).
The telemetry method to enhance customer satisfaction by methods. Chapter three covers the use
proactively recognizing and addressing problems experienced of data, its classification, quality, protec-
by the end user throughout the product or service life cycle has tion, security and storage, and the use of various databases. It
been rarely used. Telemetry is “an automatic way of collecting also includes a case study on the use of automation leading to
data at remote sites or locations and transmitting the data to improved customer experience.
collectors at the receiving equipment for monitoring, analyzing, Chapters four and five focus on the application of telemetry
and taking appropriate actions based on the insights pro- in a software development organization and the required talent
vided by the data analytics.” The insights gained through this for successful integration. Also included is a list of risk factors
adoptive method will result in more cost effective and efficient considered in designing a telemetry-enabled system. In the
customer service. concluding chapter the previous five chapters are summarized
The book is structured in six chapters. Chapter one provides and include examples of the application.
a basic introduction to several topics, such as customer experi- This book is a valuable resource for those are tasked with
ence, technological tipping points for productivity and growth, development and implementation of a telemetry-based system
proactive versus corrective and preventive actions, and change in organizations with the capacity and desire for enhanced and
in management. sustainable customer loyalty programs beyond the use of tradi-
Chapter two provides a detailed overview of telemetry and tional methods of measuring customer satisfaction.
telemetry structures and compares past and newly developed —Herzl Marouni, Houston

PROFESSIONAL SERVICES For information on placing an ad, contact Media Sales at 866-277-5666.
TQM ASSOCIATES INC. Robust Quality – Powerful Integration

“we’re here to support you” of Data Science and Process Engineering
by Rajesh Jugulum
• Established in 1994 This book provides a strong connection between the concepts
• Women-Owned in data science and process engineering to ensure better
quality levels and takes you through a systematic approach
• Quality Assurance Professionals to measure holistic quality with several case studies.
• Across the U.S. and Worldwide Video link: https://vimeo.com/278471141 • Buy from Amazon or CRC press
For more details visit: www.rajeshjugulum.com
• Temporary or Permanent
• Source Inspection
• Surveys
• Audits
Looking for Quality
• Expediting Professionals?
800-424-4729 3990 Old Town Ave. #C109
Fax 619-297-3251 San Diego, CA 92110 Place a Recruitment Ad in
tqmassociates.com email: stephk@tqminc.net
Quality Progress and reach
more than 100,000 readers
from all over the world!
Contact Media Sales
at 866-277-5666.
Advertisers Index
ADVERTISER PAGE PHONE WEB
CyberMetrics Corp. OBC 800-777-7020 www.cybermetrics.com
Write Here, Write Now

Do you aspire to be an author? Is 2019 the year you’ll see your first byline? Perhaps
you’ve been published before and you have an itch to write again?
Either way, now that QP’s latest editorial calendar has been released, it’s the perfect
time to consider writing about any of the featured topics editors have planned for 2019:
January May September

Building a Better Workplace Standards and Auditing Women in Quality
February June October
Lean and Six Sigma The Strength of Teams Quality 4.0
March July November
Innovation and Creativity Careers: Finding Your Path Risk and Risk Management
to Success
April August December
Leading Change Basic Tools, New Twists Salary Survey
Visit qualityprogress.com and click “Author Guidelines” at the bottom of the page for more details. Make sure to submit the
article three to six months before the issue date to allow time to review, revise and prepare for publication.
If you’ve missed the deadline or have an idea or topic that doesn’t fit the theme, that’s OK. Quality Progress publishes every
feature accepted through our review process. Visit our website for details on the review and submission process.
qualityprogress.com❘ ❘February
qualityprogress.com January 2019 55
QP 63
Risk on
A new twist on an established
quality concept
The Radar
Combine likelihood, consequence and detectability with a radar chart by James L. Gooding
Traditional risk assessments use two dimensions of FIGURE 1

analysis: the likelihood that a threat or hazard event
will happen, and the consequence if the event does
happen.1 Using graded number scales—increasing val-
Risk priorities for pipeline safety
ues for rising likelihood or consequence—risk analyses
can be presented as plots of consequence vs. likelihood. as visualized in a radar chart
Contemporary risk assessments include a third RPN (LCD) RPN (LC)
dimension—detectability, which is the degree of
difficulty of sensing an event through monitoring for Corrosion
process alerts or warnings.2,3 A detectability num- 100
ber scale at its low end means timely detection is
certain; the high end means timely detection is nearly
All other causes Excavation damage
impossible.
10
Likelihood (L), consequence (C) and detectability
(D) are multiplied together into a risk priority number
(RPN) as:
1
RPN (LC) = L x C or RPN (LCD) = L x C x D.
Other outside
The two RPNs are clearly distinguished on a force damage
Incorrect operation
radar chart.
Example from pipeline safety

Online Table 1 (which can be found on this column's web-
page at qualityprogress.com) summarizes risk analyses Natural force Material/weld/
damage equipment failure
applied to natural gas pipeline incidents in the United
States during 2016.4 Likelihood values reflect relative
RPN = risk priority number
frequencies of incidents, whereas consequence values
increase from slow leaks to catastrophic explosions.
REFERENCES
Low detectability values denote excellent alerting
1. American National Standards Institute (ANSI) and American Society of Safety
from in-line inspection technologies, whereas high val- Engineers (ASSE), ANSI/ASSE Z690.2-2011—Risk Management Principles and
ues indicate poor alerting, including ill-advised practices Guidelines—National Adoption of ISO 31000:2009, Jan. 11, 2011.
2. R. Dan Reid, “Standards Outlook: FMEA—Something Old, Something New,”
that evade or defeat monitoring. Such detectabili-
Quality Progress, May 2005, pp. 90-93.
ty-based distinction explains why leak/rupture detection 3. Jim Breneman, Dan Burrows and Mark Durivage, “In Focus,” Quality Progress,
has renewed emphasis in pipeline risk analysis. October 2017, pp. 18-24.
4. “DIMP Implementation: What Gets Measured Gets Done,” National Association
Traditional consequence vs. likelihood (Online Fig-
of Pipeline Safety Representatives and US DOT PHMSA Office of Pipeline
ure 1) does not provide differentiation by detectability Safety, October 2017, https://primis.phmsa.dot.gov/dimp/docs/WRGC_DIMP_
as does the RPN radar chart using a logarithmic Update_08.29.2017.pdf.
axis (Figure 1). For example, natural force damage
(such as from severe floods) might not be detected
if physical access to the pipeline is impeded (RPN
James L. Gooding is managing director of Geoclime,
(LCD/LC) = 32/8) and rates a higher risk priority LLC, in Seabrook TX, which provides quality and
than material/weld/equipment failure, in which risk consulting for the energy, water, science and
monitoring is more immediate (RPN (LCD/LC) = engineering industries. He also has worked in the
natural gas and electric-power industries. Gooding
18/9). The radar chart improves risk prioritization by is a senior member of ASQ and an ASQ-certified
detectability using RPN (LCD). manager of quality/organizational excellence.

UPCOMING
CERTIFICATION
EXAMS
• Calibration Technician
• Pharmaceutical GMP
Professional
• Quality Auditor
• Quality Engineer
• Quality Improvement
Associate
• Quality Process
Analyst
• Six Sigma Green Belt
• Software Quality
Engineer
GET YOUR
ASQ CERTIFICATION
Looking for a way to stand out in 2019? ASQ certification can help with
that. Choose from 18 certifications that are sure to not only differentiate
you from your competition, but can also result in a higher salary or help
land a promotion. It’s time to become a recognized expert in your field.
Apply for the April 2019 exams today!
2019 UPCOMING APPLICATION DEADLINE – March 8

TESTING WINDOW – April 5 – 21
For more information or to apply for an exam, visit asq.org/cert.

FREE DOWNLOADS
of the World’s Best
Calibration Management Software
With a solid 30-year track record and countless auditor recommendations, GAGEtrak calibration
management software has improved quality at over 15,000 facilities worldwide, assuring unwavering
accuracy, efficiency and standards compliance. And now, with the addition of GAGEtrak Lite to our
product line, we have a software solution to meet every need and budget. Contact us now to discuss
which one is right for your company or download both and see for yourself.
GAGEtrak and GAGEtrak Lite

Two levels of calibration management. One auditor-recommended name.
Download for free at: GAGEtrak.com/downloads 1-800-777-7020 +1 (480) 922-7300

ASQ+Quality+Progress+ +feb+2019

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

ASQ+Quality+Progress+ +feb+2019

Uploaded by

Copyright:

Available Formats

FE B RUARY 2019 | QUALIT YPROG RE SS .

Six Sigma Combined AS9100D:

page 24 page 52 page 58

The official publication of ASQ

Lean and Six Sigma This year’s sessions and

Learn more about these books through the

OPEN ENROLLMENT for the 2019-2020

qualityprogress.com ❘ February 2019 QP 3

5 Foreword 13 Career Coach 60 Marketplace

8 Progress Report 52 Statistics Spotlight

Advertising Production Article Submission Process express permission of Quality

4 QP February 2019 ❘ qualityprogress.com

Steering Your Selection

TCC Daniella A. Picciotti, QMS Alliance QP Editorial Review Board

qualityprogress.com ❘ February 2019 QP 5

THE REACTION GAUGE

CORRECT CONCLUSIONS With the proliferation of two-day and same-day delivery

6 QP February 2019 ❘ qualityprogress.com

qualityprogress.com ❘ February 2019 QP 7

8 QP February 2019 ❘ qualityprogress.com

qualityprogress.com ❘ February 2019 QP 9

 continued from page 9

ASQ Sells Shares

10 QP February 2019 ❘ qualityprogress.com

NEWS FROM AROUND Associate professor at the Uni-

qualityprogress.com ❘ February 2019 QP 11

GROUP LISTS 118

TOP U.S. HOSPITALS research, development and

Mr. Pareto Head By Mike Crossen

12 QP February 2019 ❘ qualityprogress.com

qualityprogress.com ❘ February 2019 QP 13

14 QP February 2019 ❘ qualityprogress.com

LEAN AND SIX SIGMA

How 5S and kanban helped a small business

qualityprogress.com ❘ February 2019 QP 15

The 18-step model

16 QP February 2019 ❘ qualityprogress.com

qualityprogress.com ❘ February 2019 QP 17

18 QP February 2019 ❘ qualityprogress.com

Cooper, Edgett and Kleinschmidt (1997)

Bengt, Wiklund and Edgeman (2001)

Smith, Blakeslee and Koonce (2002)

Archer and Ghasemzadeh (1999)

Jones, Parast and Adams (2010)

Harry and Schroeder (2000)

Tanik and Sen (2012)

SSPPM = Six Sigma project portfolio management

qualityprogress.com ❘ February 2019 QP 19

FIGURE 1 Step five is an individual evalua-

Conceptual model for SSPPM

20 QP February 2019 ❘ qualityprogress.com

Nielsen Six Sigma project portfolio

qualityprogress.com ❘ February 2019 QP 21

organization’s business strategy and identify any significant REFERENCES

If there is significant change in the

22 QP February 2019 ❘ qualityprogress.com

qualityprogress.com ❘ February 2019 QP 23

MATTERSHow Six Sigma is being deployed to assess high-risk

24 QP February 2019 ❘ qualityprogress.com

After some investigation, the structural integrity of the

26 QP February 2019 ❘ qualityprogress.com

I-chart of mean severity scores of normal production (A) and

x̅ = 0.59 1 1 1 1 x̅ = 0.62 1 x̅ = 0.61

* < 0.05 < 0.01 * < 0.001