F E AT U R E

BIG DATA

26 QP September 2017 ❘ qualityprogress.com

The Deal With
BIG
DATA
Move over! Big data analytics
and standardization are the
next big thing in quality  Just the
Facts
by Michele Boulanger, More and more
organizations
Wo Chang, have realized
Mark Johnson the important
role big data
and T.M. Kubiak plays in today’s
marketplaces.

Recognizing this
shift toward big
The era of big data is upon us. While providing a data practices,
formidable challenge to the classically trained quality quality profes-
sionals must step
practitioner, big data also offers substantial opportunities up their under-
for redirecting a career path into a computational and standing of big
data and how
data-intensive environment. organizations
The change to big data analytics from the status quo can use and take
advantage of
of applying quality principles to manufacturing and their transactional
service operations could be considered a paradigm data.
shift comparable to the changes quality professionals Standards groups
experienced when statistical computing packages realize big data is
here to stay and
became widely available, or when control charts were are beginning to
first introduced. develop founda-
tional standards
for big data and
big data analytics.

qualityprogress.com ❘ September 2017 QP 27

F E AT U R E
BIG DATA

The challenge for quality practitioners is to recognize
this shift and secure the training and understanding nec-
essary to take full advantage of the opportunities.
What’s the big deal?
What exactly is big data? You’ve probably noticed that
big data often is associated with transactional data
sets (for example, American Express and Amazon),
social media (for example, Facebook and Twitter) and,
of course, search engines (for example, Google). Most
formal definitions of big data involve some variant of the
four V’s:
++ Volume: Data set size.
++ Variety: Diverse data types residing in multiple
locations.
++ Velocity: Speed of generation and transmission of
data.
++ Variability: Nonconstancy of volume, variety and
velocity.
This set of V’s is attributable originally to Gartner Inc.,
a research and advisory company,1 and documented
by the National Institute of Standards and Technology
(NIST) in the first volume of a set of seven documents. 2
Big data clearly is the order of the day when the quality
practitioner is confronted with a data set that exceeds
the laptop’s memory, which may be by orders of
magnitude.
In this article, we’ll reveal the big data era to the qual-
ity practitioner and describe the strategy being taken by
standardization bodies to streamline their entry into the
exciting and emerging field of big data analytics. This is
all done with an eye on preserving the inherently useful
quality principles that underlie the core competencies of
these standardization bodies.

FIGURE 1

NIST big data reference architecture

Information value chain

System orchestrator

Big data application provider

Data consumer
Data provider

Preparation/
Collection curation Analytics Visualization Access
IT value chain

Big data framework provider

Processing frameworks: Computing and analytic frameworks
Security and privacy

Messaging/ Streaming Resource

communications Batch management
Management

Platforms: Data organization and distribution

Indexed storage
File systems

Infrastructures: Networking, computing and storage

Virtual resources
Physical resources

LEGEND
NIST = National Institute of
Big data information flow Standards and Technology
Service use
Software tools and algorithms transfer

28 QP September 2017 ❘ qualityprogress.com

Primary classes of big data problems
The 2016 ASQ Global State of Quality reports included
a spotlight report titled “A Trend? A Fad? Or Is Big Data
the Next Big Thing?”3-6 hinting that big data is here to
stay. If the conversion from acceptance sampling, con-
trol charts or design of experiments seems a world away
from the tools associated with big data, rest assured
that the statistical bases still apply.
Of course, the actual data, per the four V’s, are
different. Relevant formulations of big data problems,
however, enjoy solutions or approaches that are sta-
tistical, though the focus is more on retrospective data
and causal models in traditional statistics, and more
forward-looking data and predictive analytics in big
data analytics.7
Two primary classes of problems occur in big data:
++ Supervised problems occur when there is a depen-
dent variable of interest that relates to a potentially
large number of independent variables. For this,
regression analysis comes into play, for which the typ-
ical quality practitioner likely has some background.
++ Unsupervised problems occur when unstructured
data are the order of the day (for example, doctor’s
notes, medical diagnostics, police reports or internet
transactions). Unsupervised problems seek to find the
associations among the variables. In these instances,
cluster and association analysis can be used. The qual-
ity practitioner can easily pick up such techniques.
The world of standardization
and big data analytics
Many fields of expertise share an interest in big data
analytics, including statistics, artificial intelligence, infor-
mation systems, and other fields in which the data are
collected in great quantities (marketing, quality, finance,
supply chain and engineering, for example).
The convergence of so many technical areas moti-
vates standardization activities in areas such as
terminology, algorithms and reporting. A search for
big data analytics on the International Organization
for Standardization (ISO) website, in fact, identified 54
items that include standards, reports and guidelines.
Most of these come from Joint Technical Committee
1 (JTC 1) Information Technology. Others come from
vertical committees, such as information security man-
agement, energy, transportation and farming.
ISO/IEC JTC 1 has established Working Group 9
(WG 9) to develop foundational standards for big
data, including reference architecture and vocabulary
standards for guiding big data efforts throughout JTC 1
upon which other standards can be developed.
NIST is the key contributor to JTC 1/WG 9. Its goal is
to develop a standard interoperability framework that
offers scientists and other experts an architecture to
ease the process of big data analytics—whether during
the collection, storage, analysis, or deployment of mod-
els or any other phase encountered in the big data arena.
The NIST Reference Archi-
tecture has been documented
in a seven-volume publication Many fields of expertise
that can be downloaded from share an interest in big
the NIST website and is ref- data analytics, includ-
erenced as the NIST Big Data ing statistics, artificial
Interoperability Framework. 8 intelligence, informa-
Figure 1 shows a high-level tion systems, and other
view of the architecture. fields in which the data
The application provider are collected in great
in Figure 1 is the area in which quantities.
ISO Technical Committee
(TC) 69 Applications of Statistical Methods, statisticians
and quality practitioners, are positioned. There are three
opportunities to interface with other fields of expertise:
1. The data provider provides or feeds data or informa-
tion into the big data system.
2. The framework provider executes certain statistical
algorithms while protecting the privacy and integrity
of the data.
3. The data (end) consumer in this framework consti-
tutes end users or other systems that use the results
of the big data application provider.
It is interesting to see that visualization—usually a
step in the analysis that follows the data preparation—is
now a step following the actual analysis of the data. The
rationale is that visualization of the entire data set is
unwieldy, if not impossible, while visualization of results
at the post-analysis (that is, the presentation) stage
facilitates understanding.
Joint partnership with
JTC 1/WG 9 and NIST
Given the multidisciplinary nature of big data analytics, it
is a natural step for ISO/TC 69 to establish a liaison with
JTC 1/WG 9. A small joint team of TC 69 experts and JTC
1/WG 9 experts, along with NIST experts, was assembled
with the following business and analytics objectives:
1. To mimic—as much as possible—the operation of a
data mining or business analytics team assembled to
address a case study selected by the team.
qualityprogress.com ❘ September 2017 QP 29
F E AT U R E
BIG DATA

FIGURE 2

Preliminary roadmap of potential standards for big data

Prepare/
Collect Analyze Visualize Access
curate

Scoring full data set and

Sampling guidance Assess data interface between application
++ Random quality provider and framework
++ Stratified provider
Outliers Choice of analytical techniques
identification based on storage platforms
Validation (TR on survey/
Ensure quality of Ensure quality of Ensure quality of
compilation of the existing
input processes input processes input processes
scientific literature and
for data quality for data quality for data quality
standards on guidance for use
New CRISP New CRISP New CRISP New CRISP New CRISP

CRISP = cross-industry standard process

TR = technical report

2. To exercise the reference architecture developed by
NIST (Figure 1).
3. To use ISO/TC 69’s existing statistical standards
where appropriate for the case study and to identify
gaps in standards within TC 69 to conduct the case
study. These gaps will be important elements to
define a roadmap of future standards for TC 69 to
develop.
The team selected a case study to better understand
the challenges, “get its hands dirty” and achieve these
objectives. The case study chosen was a fraud detection
problem pertaining to Medicare payments to providers.
The 2012 Medicare public data set was used for the
case study and consisted of more than 9 million records
aggregated by unique provider codes (NPI) and proce-
dure (HCCPS) codes.
Further, the data set consisted of 29 columns,
including a unique identifier for the provider, address,
credentials, services aggregated by procedure, unique
beneficiaries of these services, average amount sub-
mitted for repayment by Medicare, average allowed by
Medicare and the average paid by Medicare, along with
standard deviations of costs for a given procedure code.9
A specific set of analytic objectives was developed
for the case study, referring to the analytics goals
themselves:
++ Identify suspicious patterns of medical activities
reflected in the sampled data.
++ Document algorithms to be used on the full data set,
as well as the updated and appended data sets.
++ Ensure that the tools, techniques and algorithms
apply to other large data sets and, in particular, are
scalable (that is, capable of handling different data set
sizes from different locations) to data sets stored on
multiple nodes.
The team identified three key learning opportunities:
1. Analysts working on the application provider layer
usually do not have the capability to analyze the total-
ity of the data. This is due, in part, to the large size of
the data, which may be stored in multiple nodes and
potentially owned by different parties that use differ-
ent security systems. This characteristic of big data
requires new methods that deal with the application
of learning from sample data to the original data set.
This, in turn, requires standardization so all parties
know how to interact.
2. Programming code (that is, code used to prepare
and curate the full data set) must be presented to
the information systems practitioner in a language
(JavaScript, C, Python or some other programming
language) that can be executed. This programming
code is passed between the application provider layer
and the data provider (see Figure 1).
3. Because different experts have different back-
grounds, time and a willingness to understand each
expert’s point of view is required. Consequently, the
dynamics of the team and the importance of the team
leader cannot be minimized.

30 QP September 2017 ❘ qualityprogress.com

TC 69 and big data
TC 69 decided to create an ad hoc group to explore the
feasibility of developing standard methods in the area of
big data analytics and in the context of the partnership
with ISO JTC 1 / WG 9 and NIST for the reference archi-
tecture they are developing.
The first item on the charter of that ad hoc group was
to conduct a gap analysis to assess the applicability
of 102 existing TC 69 standards and 34 documents in
various stages of development.10
Gap analysis: This task consisted of developing a set
of requirements characterizing the treatment of big data
to evaluate the appropriateness of the TC 69 documents
in the big data world. All 136 documents within TC 69
were evaluated against these requirements.
It was recognized that all 59 control charts, process
management and acceptance sampling documents can
be useful in big data analytics, especially in the prepa-
ration and curation phase, as illustrated in Figure 1. This
phase relates to controlling the processes generating
the data or assessing the quality of the data.
TC 69 has 63 of the 136 (46%) documents that can be
viewed as relevant. None of these documents, however,
were initially developed specifically to address the big
data environment. Therefore, they are likely to require
some level of interpretation and modification to accom-
modate big data.
Preliminary roadmap development: The next item
for the ad hoc group consisted of creating a preliminary
roadmap of documents that TC 69 should consider to
standardize big data analytics. The roadmap shown in
Figure 2 was developed following a discussion on the
impact of the potential changes to existing standards
that will be needed to address big data or the develop-
ment of brand new documents.
This roadmap is shown to define a portfolio of doc-
uments to be developed in such a way that they link
to each other and create a consistent set of tools and
procedures. The roadmap is expected to be revised as
standards are developed.
Challenges in big data analytics
The challenges with regard to big data analytics are
formidable and, hopefully, not insurmountable. A few
key examples include:
Problem formulation—Successful problems are
solved, first and foremost, by defining the problem cor-
rectly and succinctly. This is particularly true with regard
to big data analytical problems. With big data sets, the
number of data fields can be enormous, the data fields
may not be fully understood, necessary data fields may
be missing or a direct means of identifying the solution
is not available.
The Medicare fraud case study mentioned earlier
illustrates the difficulty that can be incurred when
trying to formulate a meaningful
problem statement—particularly
with regard to identifying the The challenges with
solution. Initially, the problem or regard to big data
question to address was, “Where analytics are formi-
does Medicare fraud exist?” After dable and, hopefully,
significant and time-consuming not insurmountable.
analyses, it became clear that a
direct answer to this question did
not exist because the analysis not only pointed toward
real outliers, but potential fraudulent activities.
The reason for this dilemma WAS the simple fact that
no data exist that can readily confirm the presence of
fraud. Statistical tools can only suggest the presence
of fraud. Therefore, the problem statement must be
reformulated as follows: “Where does the potential for
Medicare fraud exist?”
After such situations are identified statistically, further
investigation is reduced to the hands-on approach. Such
an approach might include visits to medical provider
offices, patient interviews and record reviews.
Skill sets—In the past, a quality practitioner would
often analyze processes using a spreadsheet or special-
ized software such as Minitab, JMP and SAS. Problems
were usually limited to analyzing a single process using
small structured data sets located in a single database
on a single desktop computer.
Frequently, the quality practitioner would produce
a variety of charts (such as control charts and Pareto
charts) and occasionally run a simple linear regression or
two. Problems dealing with determining the underlying
relationships that might exist among multiple variables
(that is, data fields) were often avoided. This occurred
because the quality practitioner did not possess the
appropriate statistical skill level, or the analytical tools
were not available or did not exist.
Enter big data sets, and the landscape changes
significantly as described earlier. You can easily see from
Figure 1 that such a framework necessitates the involve-
ment of information technology specialists, process
subject matter experts (SME), statisticians, quality
practitioners and possibly many others. These skill sets
are not mutually exclusive; However, the individual
qualityprogress.com ❘ September 2017 QP 31
F E AT U R E
BIG DATA
disciplines are necessary to effectively analyze a prob-
lem involving big data.
Teamwork—Due to the complexity of data mining
software, analytical software, statistical techniques,
hardware platforms, data structures and the
knowledge required to understand the underlying
processes that generate data, it is imperative that orga-
nizations implement a team-based approach.
The Venn diagram shown in Figure 3 illustrates the
need for interactions, communications and teamwork.
Of course, the degree of interactions and communica-
tions among the team members depends on the nature
of the problem and the level of skills each member
possesses.
Note that Figure 3 has introduced a team member
called the end consumer. This team member is gener-
ally the individual who issued a request for data analysis
or posed a question or problem to be addressed. This
individual receives the output of the big data analysis.
In some cases, the five roles introduced in Figure 3
may be reduced. For example, the quality practitioner
may be the end consumer and the process SME and the
statistician—all depending on the skill level achieved
in each discipline. Regardless of the number of roles
involved, continuous and effective communication is
required, and teamwork is paramount.
Data integrity and quality—This issue is the scorn of
big data. Any meaningful analysis demands a high level
of the integrity and quality of the data. Often, this is not
the case in the big data era. Such was the situation with
the Medicare fraud case study data.
Due to multiple points of data entry (that is, medical
providers across the country and beyond), the data for
a single data field was expressed in different ways. For
example, the “MD” suffix for a medical provider may
have been entered as “MD” or “M.D.” Generally, any
analytical software recognizes these data as distinct
data elements. Such situations—and there are many of
them—must be identified separately and corrected. This
requires significant time and effort, as well as teamwork
among the IT practitioner and other team members.
Given these challenges, consider how difficult it would
be for a global organization to analyze quality data
across the organization.
For example, suppose multiple product lines
exist within a site, multiple sites comprise a divi-
sion, multiple divisions comprise a business unit and
multiple business units comprise the total global
organization. Such a structure is not uncommon. If the
32 QP September 2017 ❘ qualityprogress.com
FIGURE 3
Teamwork facilitates
successful big data
applications
Process SME
IT practitioner
End
consumer
Statistician
Quality practitioner
SME = subject matter expert
global organization grew through mergers and acqui-
sitions, it would be reasonable to conclude that the
collective quality data across the organization would
constitute a problem of enormous inconsistency. Now
expand the scenario to include multiple global organiza-
tions (think industry), and the analysis grinds to a halt.
In such situations, successful results can only be
obtained when data are defined and interpreted in a
uniform manner. For example, the United States has
developed a unified set of state abbreviations (for exam-
ple, PA, FL, AZ and IL). This set of abbreviations is now
universally accepted and embedded in all data collection
systems—often through the use of drop-down menus.
Similarly, other examples can be readily found.
Incorrect data will never be eliminated, but it can
be minimized. This is a substantial challenge in the big
data era.
More contributions necessary
At its annual meeting in June in Cape Town, South
Africa, TC 69 launched three new work items on big data
analytics on:
++ Statistical terminology.
++ Methodology to flesh out the application provider in
Figure 1 and enhance the development of CRISP-DM
++ Model validation in which a different approach to
 traditional statistical techniques was determined to Michele Boulanger is an
be needed due to the large volume, variety, velocity associate professor of statistics
and variability of the data. and international business at
Rollins College in Winter Park,
Many challenges remain for TC 69. One of the most FL. She is chair of ISO/TC 69,
difficult ones is the recruitment of analytical experts, past chair and current member
information system experts and SMEs to participate in the of the U.S. Technical Advisory
Group (TAG) to ISO/TC 69, and
development phase of these documents. a member of the Z1S Standards
The partnership with ISO/JTC 1/WG 9 and NIST is crucial Committee. She is an academician in the International
to maintain connectivity to their expertise as needed by the Academy for Quality.
statistically inclined participants. With an exciting pro-
gram ahead, the challenge consists of recruiting additional
experts on a volunteer basis and selling the advantages Wo Chang is the digital
of standardization and the learning that comes with it to data advisor for the NIST IT
management. Laboratory. He chairs the ISO/
IEC Joint Technical Committee
Quality practitioners have an opportunity to join the (JTC) 1/ working group (WG)
group as SMEs initially while they assimilate the sophis- 9 on big data, the Institute
ticated techniques that make up the core of big data of Electrical and Electronics
Engineers Big Data Governance
analytics. Experts are needed in three areas targeted for and Metadata Management group, and the ISO/IEC
documentation including terminology, methodology and JTC 1 Standards Committee (SC) 29 WG 11 (MPEG)
model validation. Multimedia Preservation ad hoc group.

REFERENCES AND NOTES

1. Frank Buytendijk and Thomas W. Oestreich, “Organizing for Mark Johnson is professor
Big Data Through Better Process and Governance,” Gartner of statistics at the University
Research Report, March 10, 2015, www.gartner.com/doc/3002918/ of Central Florida in Orlando.
organizing-big-data-better-process. He is a fellow of the American
2. National Institute of Standards and Technology (NIST), “NIST Big Statistical Association, elected
Data Interoperability Framework Version 1.0,” report, September member of the Information
2015, https://bigdatawg.nist.gov/V1_output_docs.php. The Sciences Institute, a chartered
framework includes big data definitions, taxonomies, use cases statistician with the Royal
and requirements, security and privacy, architecture white paper Statistical Society, chair of ISO/
survey, reference architecture and a standards roadmap. TC 69/SC 1 on terminology, member of the U.S. TAG
3. ASQ and American Productivity and Quality Center (APQC), “The to ISO/TC 69 and statistics coordinator for the ASQ
Global State of Quality 2 Research: Discoveries 2016,” https://asq. Statistics Division. He has received the Brumbaugh
org/quality-resources/research/global-state-of-quality/reports. Award, two Shewell Awards and the Jack Youden Award
4. ASQ and APQC, “Spotlight Report: KPIs Key to Successful from ASQ. He has also received the Shin Excellence in
Supply Chain,” https://asq.org/quality-resources/research/ Research Award, sponsored by the Geneva Association
global-state-of-quality/reports. and the International Insurance Society.
5. ASQ and APQC, “Spotlight Report: A Trend? A Fad? Or Is Big Data
the Next Best Thing?” https://asq.org/quality-resources/research/
global-state-of-quality/reports.
6. ASQ and APQC, “Spotlight Report: Innovation and Quality Go
Hand in Hand,” https://asq.org/quality-resources/research/ T.M. Kubiak is a president and
global-state-of-quality/reports. founder of the consulting firm,
7. Galit Shmueli, “To Explain or to Predict,” Statistical Science, 2010, Performance Improvement
Vol. 25, No. 3, pp. 289-310. Solutions, located in Fort
8. NIST, “NIST Big Data Interoperability Framework,” see reference 2 Mohave, AZ. He is an ASQ
9. A description of the data is provided in the file Fellow, a member of the U.S.
PartBNationalSummaryReadmeFile2012.pdf available at www.cms. TAG to ISO/TC 69, a member of
com. the Z1S Standards Committee and the delegated expert
10. The program of work and existing standards are being handled by on Six Sigma for the U.S. TAG to ISO/TC 69.
ISO/TC 69. See www.iso.org/iso/home/standards_development/
list_of_iso_technical_committees/iso_technical_committee.
htm?commid=49742 for more information.

qualityprogress.com ❘ September 2017 QP 33