See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/319466212

Toward understanding outcomes associated with data quality improvement

Article  in  International Journal of Production Economics · September 2017

DOI: 10.1016/j.ijpe.2017.08.027

CITATIONS READS

5 261

4 authors:

Benjamin Thomas Hazen Fred Karl Weigel

Fachhochschule Oberösterreich Lipscomb University
103 PUBLICATIONS   2,262 CITATIONS    17 PUBLICATIONS   93 CITATIONS   

SEE PROFILE SEE PROFILE

Jeremy D. Ezell Bradley C Boehmke

Auburn University Air Force Institute of Technology
10 PUBLICATIONS   430 CITATIONS    20 PUBLICATIONS   60 CITATIONS   

SEE PROFILE SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Toward Developing Competencies for a Healthcare Administration Program: A Delphi Study View project

Sustainability & Big data analytics View project

All content following this page was uploaded by Bradley C Boehmke on 11 October 2017.

The user has requested enhancement of the downloaded file.

 International Journal of Production Economics 193 (2017) 737–747

Contents lists available at ScienceDirect

International Journal of Production Economics

journal homepage: www.elsevier.com/locate/ijpe

Toward understanding outcomes associated with data quality improvement

Benjamin T. Hazen a, Fred K. Weigel b, *, Jeremy D. Ezell c, Bradley C. Boehmke a,
Randy V. Bradley d
a
Department of Operational Sciences, Air Force Institute of Technology, Wright-Patterson Air Force Base, OH, USA
b
Department of Management, Lipscomb University, Nashville, TN, USA
c
Department of Computer Information Systems and Business Analytics, James Madison University, Harrisonburg, VA, USA
d
Department of Marketing and Supply Chain Management, The University of Tennessee, Knoxville, TN, USA

A R T I C L E I N F O A B S T R A C T

Keywords: Business analytics is driving the way organizations compete. However, the decisions made as a result of any
Data quality analytics process are greatly dependent on the quality of the data from which they are based. Scholars suggest
Quality control several data quality improvement methods, and commercial software is now available that can help improve data
Supply chain innovation
quality. Although organizations are motivated to find ways to improve analytic capabilities and using these
Aviation logistics
methods has been shown to improve some measures of data quality, there is little understanding of the tangible
Case study
and intangible outcomes of employing these methods. In this study, we explore outcomes that arise from a data
quality improvement process implementation in an operations management environment. Over a three-year
period, we conducted a longitudinal single case study at an organization that maintains a large fleet of
aircraft, collecting and analyzing qualitative interviews and observations. The findings suggest outcomes (both
positive and cautionary) associated with the implementation of data quality improvement processes. These
include increased stakeholder commitment to data quality and business analytics, as well as an over-emphasis on
program metrics to the peril of operational outcomes, among others. From the basis of our findings, we construct a
research agenda, which includes testable Propositions regarding outcomes of data quality improvement.

1. Introduction their business decision-making processes. Gathering, storing, and making

1.1. Motivation for study
Decision-making professionals within modern organizations provide
management with strategic analysis and operational recommendations
based on data collected and stored by the firm. The volume of data
flowing through the contemporary firm can be overwhelming, moti-
vating new ways of thinking about how data are organized and analyzed
(Hey, 2010). The importance of high-quality data for decision-making
provides a strong impetus for organizations to actively improve all as-
pects of their business analytics activities. Scholarly research increasingly
suggests that business analytics is quickly becoming a separate capability,
which an organization can leverage to compete in the marketplace (Akter
et al., 2016; Chen et al., 2012; Davenport, 2006). For example, LaValle
et al. (2010), suggest that an organization's analytics capability is a sig-
nificant marketplace differentiator; the top performing firms in their
study were twice as likely as low performing firms to apply analytics to
sense of voluminous data through knowledge generation is a complex
process (Wamba et al., 2015), however, and there are many opportunities
throughout the process for data integrity to be compromised.
High-quality data are paramount to optimal analytics performance,
particularly decision-making effectiveness. The ramifications of low-
quality data have been noted in the literature for decades (e.g., Wang
and Strong, 1996), and efforts continue today to find ways to provide
higher quality data and more contextual information about the data to
improve decisions (e.g., Price and Shanks, 2011). For instance, research
has focused on a variety of means to monitor and enhance data quality
(Lenz and Borowski, 2010; Pierchala et al., 2009; Redman, 1992, 2001,
2008; Sparks and OkuGami, 2010a,b). But, even monitoring and
awareness of the sub-par quality of data used for decision-making are not
sufficient unless the organization takes active steps to correct data quality
issues. It is this active effort to enhance data quality, that is to improve
the completeness, accuracy, consistency, and timeliness (Batini et al.,
2009; Haug and Arlbjørn, 2011; Jones-Farmer et al., 2014) that should

* Corresponding author. One University Park Drive, Department of Management, Nashville, TN 37204-3951, USA.
E-mail addresses: Benjamin.hazen@live.com (B.T. Hazen), fred.weigel@lipscomb.edu (F.K. Weigel), ezelljd@jmu.edu (J.D. Ezell), bradleyboehmke@gmail.com (B.C. Boehmke),
rbradley@utk.edu (R.V. Bradley).

http://dx.doi.org/10.1016/j.ijpe.2017.08.027
Received 27 September 2016; Received in revised form 15 July 2017; Accepted 30 August 2017
Available online 4 September 2017
0925-5273/© 2017 Elsevier B.V. All rights reserved.
B.T. Hazen et al.
lead to more effective decision-making augmented by improved analytic
performance. Given the importance of data for strategic decision-making
by the modern firm, actions taken by the firm to improve data quality
should be impactful in many areas of the firm, in both positive and
negative ways. Unfortunately, extant literature provides little evidence
regarding the consequences of implementing data quality improve-
ment methods.
1.2. Research objective and question
Although it is intuitive that efforts to enhance the completeness, ac-
curacy, consistency, and timeliness of data should lead to positive out-
comes, current research insufficiently addresses this conjecture. Research
must move beyond merely improving data monitoring and awareness
activities. Rather, leadership and managers need to understand the im-
pacts gained from data quality improvement projects and processes.
Consequently, the purpose of this study is threefold: (1) To investigate
outcomes realized from the adoption and employment of data quality
improvement initiatives; (2) To illustrate the after-effects and conse-
quences of such actions; and (3) to clarify how these quality improve-
ment steps impact decision-making outcomes in the organization. Thus,
our primary research question is: How does the employment of data
quality improvement methods impact a firm's analytics and busi-
ness processes?
To answer this question, we approached the study from a post-
positivist worldview (Creswell, 2013). This worldview is appropriate
because it was our intent to condense the ideas we identified into a small
group of testable Propositions; we also felt that using multiple methods of
observing and analyzing the phenomenon was necessary for the clearest
understanding (Benbasat et al., 1987; Creswell, 2013; Dub
e and Par
e,
2003). Additionally, having access to view the creativity of personnel at
all levels of the organization leads to a rich yield of data (Voss et al.,
2002). Thus, the knowledge generation of the organizations, coupled
with the direct observations of an embedded research provide for supe-
rior theory generation than analysis from an outside perspective, such as
what might be found through survey data collection (Benbasat et al.,
1987; Voss et al., 2002). Finally, we believed taking a more inclusive,
holistic view of the organization would provide greater insights into the
data quality improvement methods (Voss et al., 2002). In contrast, we
concluded that an external view of the organization would not provide
sufficient insight to properly understand the phenomenon or develop
strong propositions.
Drawing from the post-positivist worldview, we address the research
question through a longitudinal case study that highlights both positive
and cautionary impacts associated with the implementation of a data
quality improvement program. From the basis of our findings, we
construct a research agenda, which includes testable Propositions
regarding outcomes of data quality improvement.
The remainder of this article is headed with a literature review of the
concepts behind data quality, its measures, its impact on the business
analytics cycle, and extant data quality improvement methods. Subse-
quently, we review the case study methodology that guided our inves-
tigation, data collection, and analysis. In our findings section, we present
our results and results-based Propositions that we hope will guide future
research and scholarship in this area. After that, we discuss both the
theoretical and managerial implications of the study results and conclude
with remarks upon the study's limitations and potential future research
activities.
2. Background and literature review
2.1. Background: data quality monitoring and improvement
As technology evolves at its characteristically rapid pace, information
systems' capacities to store more data, from more sources, yielding ever-
larger datasets requiring ever more sophisticated analyses (Wang et al.,
738
International Journal of Production Economics 193 (2017) 737–747
2016) grows just as fast. The analysis of large data sets is a growing and
competitively necessary trend in business and industry (Chen et al.,
2012). While business analytics continues to proliferate, the nature and
structure of the data are changing as a firm's information infrastructure
evolves from one that primarily stores structured quantitative data to one
that relies on a combination of structured quantitative and unstructured
qualitative data (Tan et al., 2015). As the types of data and data sources
increase, so does the potential for data quality problems (Hazen
et al., 2014).
The quality of decisions within the firm depends on the quality of the
data that are analyzed (Dyson and Foster, 1982; Warth et al., 2011). Data
quality issues have long been present in organizations and negatively
impact even the simplest analytics efforts (Porter and Rayner, 1992),
such as using data warehousing and evidence-based, data-grounded
analysis for organizational transformation (Cooper et al., 2000). These
problems are increasing in proportion to the scale of information tech-
nology and data usage within the modern business enterprise. Data
quality problems arise from those activities surrounding the collection,
storing, retrieval, and processing of data within the firm, areas which are
part of the overall business analytics cycle (Dutta and Bose, 2015). It
would be these activities, then, that should be the focus of any data
quality improvement processes.
A review of the literature reveals numerous articles discussing the
problem of data quality (Ballou and Pazer, 1985; Ballou et al., 1998;
Huang et al., 1999; Pipino et al., 2002; Redman, 1996; Wang and Strong,
1996). Both the academic and practitioner literature has long stated the
need for improved data quality; however, the literature regarding applied
methods for measuring, monitoring, and controlling data quality is still in
early stages of development (Bose, 2009; Warth et al., 2011). Whereas
the use of methods to monitor and improve the quality of manufacturing
and service processes is well-researched and implemented in practice,
there has been an only limited use of methods to monitor and improve
the quality of the data used to manage these manufacturing and service
processes. Even more limited is research on the implications of using such
data quality improvement methods.
Although this literature stream is still emerging, there are some
proposed means by which a firm can monitor and improve its overall data
quality. For instance, statistical process control charts have evolved to
consider more complex and sophisticated processes and data types (Ho
and Quinino, 2013; Jones-Farmer et al., 2014; Ou et al., 2012; Wu et al.,
2009). Control charts have been introduced to speed the signaling pro-
cess, increase probabilities, minimize false alarms, control for multi-
dimensional process measures, consider complex data structures, and
work in high dimensional, online applications. More comprehensive
applications of data quality improvement are included in the work of
Redman (1992, 1996, 2001), who recommends using methods to
monitor accuracy over time. Pierchala et al. (2009) published a report on
data quality improvement methods used by the National Highway Traffic
Safety Administration to monitor the data quality in the Fatality Analysis
Reporting System and identified some measures of data quality
improvement.
Sparks and OkuGami (2010a,b) considered a more sophisticated
method for flagging biased measures in monitoring the consistency of
large volumes of data. They made the distinction between spatial, tem-
poral, and multivariate consistency within data and recommended three
different types of monitoring methods to use for different scenarios. In
another application, Lenz and Borowski (2010) developed an automatic
error tracking and correction application based on using a commercially
available data warehouse integration, auditing, and reporting tool.
Although scholars have suggested the use of data quality control
mechanisms, the applications have not been widely adopted in practice.
In many cases, the implementations of such control processes are rudi-
mentary. In other cases, these applications fail to account for de-
velopments regarding the theoretical and practical dimensions of data
quality present in the literature (Ezell et al., 2016). This might be due, in
part, to a lack of evidence to suggest the efficacy and tenability of using
B.T. Hazen et al.
data quality improvement processes in practical business settings. This
research examined the implementation and use of an application similar
to that proposed by Lenz and Borowski (2010), which combines auto-
mated error-finding with manual data correction.
2.2. Data and decision quality improvement
Quality improvement methods have long been closely associated with
process improvement initiatives (Mitra, 2008; Roberts, 1959), including
applications in information systems (Curtis et al., 1992; Weller, 2000).
These methods can be employed as a way to identify areas or processes
within the overall business analytics lifecycle (Dutta and Bose, 2015) that
are in need of improvement, with a focus on controlling and improving
data quality. Once properly diagnosed, perhaps through root-cause
analysis or other quality improvement methods, solutions can be
implemented to improve data quality. Similar to what is suggested by
Pierchala et al. (2009), once process issues have been ruled out as causes
of poor data quality, other issues can be investigated for potential im-
provements, including technological and information system issues.
Bartlett (2013), discusses the importance of data management in the
analytically-enabled organization and notes how specific data quality
improvement approaches can be paired with traditional summary sta-
tistics to monitor the quality of data as it enters the data warehouse
(2013, p. 241).
With competitive business decisions increasingly made based on in-
sights extracted from the analysis of massive data sets as opposed to
traditional statistical sampling (Mayer-Sch€ onberger and Cukier, 2013),
the use of sophisticated structured data quality monitoring tools is
essential for the modern enterprise. It intuitively follows that the quality
of data used for decision-making is related to the quality of the decision
made when using such data. This relationship has received some dis-
cussion in the literature (Ballou and Tayi, 1999; Cooper et al., 2000;
Fisher and Kingma, 2001; Forgionne, 1999; Raghunathan, 1999; Shin,
2003; Wang and Strong, 1996; Wixom and Watson, 2001); yet, empirical
evidence that delineates this relationship is deficient.
Firms experience organizational learning when data gathered or
generated (from internal or external sources) are applied to organiza-
tional processes (Argyris and Sch€ on, 1999; Cohen and Levinthal, 1990;
Cook and Yanow, 1993; Crossan et al., 1999). When these data are
increasingly used to guide future learning and organizational activities,
firms improve their absorptive capacity (Pavlou et al., 2006; Roberts
et al., 2012). The gathering and storing of data is a critical part of the
Business Analytics cycle, and organizations continually need to gather
and create new information to innovate, remain competitive, and stay
ahead of market peers (Castiaux, 2007; Drucker, 1991; Grant, 1996a).
The absorptive capacity literature highlights the positive impact that this
continuous data gathering can have not only on data quality but also on
the firm's ability to integrate that data into its routines and processes
(Roberts et al., 2012). With higher volumes of data, whose quality the
firm continuously acts to improve, decision makers throughout and
beyond the organization can potentially make better decisions (Shin,
2003; Wixom and Watson, 2001).
2.3. Behavioral and analytics cycle improvements
Management support is consistently recognized as contributing to
increased performance for firm processes and initiatives in which such
support is garnered (e.g., Hwang and Schmidt, 2011; Wixom and Watson,
2001). This aligns well with expectation-value theories which link indi-
vidual choices and performance to their value of a task and cognitive
expectation of success (Wigfield and Eccles, 2000). The literature sug-
gests that implementation of a data quality improvement program serves
as both a proxy for and signal of management support of enhanced data
quality specifically, and the business analytics process in general. If the
top management team voices their support for a corporate initiative and
includes allocation of resources to support that initiative, organizational
739
International Journal of Production Economics 193 (2017) 737–747
members can reasonably expect to value that initiative and see it as
critical to their personal and professional success. Management scholars
have also focused on the development of appropriate metrics for moni-
toring critical processes (Port and Bui, 2009; Singh et al., 2009), with the
understanding that established metrics can motivate employee behavior
(Rungtusanatham, 2001). In short, behavior within the organization is
expected to be driven by what is measured (Fenton and Pfleeger, 1998;
Powell and Woerndl, 2008) and by what is visibly supported by the top
management team (Hwang and Schmidt, 2011; Wigfield and Eccles,
2000; Wixom and Watson, 2001).
When there are unknown levels of data quality, those using the data
to support decision-making must spend time and effort to verify their
data, correct any issues, or find alternate data sources during the decision
process (Forgionne, 1999). To this end, information signaling the quality
of data has been shown to be useful (Fisher and Chengalur-Smith, 2003).
However, gathering such information comes at a cost and can often slow
the decision-making process (Even et al., 2010a, 2010b; Price and
Shanks, 2011). A reasonable expectation—stemming from the liter-
ature—is that organizations actively engaged in processes targeting in-
creases in data quality will off-load this labor from the decision makers
and speed up strategic and operational decision-making.
For firms to realize benefits of using business analytics, the data
systems themselves must be viewed as useful and relied upon (i.e., used)
to make quality decisions. Delone and McLean's (1992, 2003) notion of
information systems success provides a foundation for this concept. One
of the tenets of the Information Systems Success Model lies in the inter-
relationship between information quality and system quality in leading to
intention to use and user satisfaction to produce the overall net benefits
(Weigel and Hazen, 2014).
Isson and Harriott (2013), note the importance of governance of data
as it is produced and used throughout the organization, calling data
quality and the understanding of the organization's foundational pro-
cesses for ensuring high-quality data “the heart of business analytics” (p.
77). When poor data quality is rampant and when individuals within the
organization feel that they cannot count on quality data, use of business
analytics tools to make decisions will likely decrease. Cooper et al.
(2000), Shin (2003), and Wixom and Watson (2001), provide notable
examples, among others (see Ballou and Tayi, 1999), of the negative
effect poor data quality can have on users' perceptions and use of infor-
mation technology systems and tools implemented for business analytics.
This idea is based partly on tenets of the Technology Acceptance Model
(Davis, 1989; Davis et al., 1989), with its basis on the Theory of Reasoned
Action (Fishbein and Ajzen, 1975). When an information technology is
perceived as being useful—as may be the case when data quality is
perceived to be high—individuals are more likely to use it (Venkatesh
and Bala, 2008; Venkatesh et al., 2012; Weigel et al., 2009). Also,
organizational focus on data quality, along with resultant process
changes, improves the work environment and increases feelings of
employee empowerment in those who use the analytical tools, which
subsequently increases tool usage (Kim and Gupta, 2014).
Redman (2008), makes clear that efforts to improve data quality
require an organization-wide effort, and the benefits yielded from this
effort can be plentiful. Data flows through the firm similar to a
manufacturing process (Ballou et al., 1998; Wang and Kon, 1993), and its
improvement is not limited to just one department or division (Kahn
et al., 2002). The issue of data quality in the enterprise is one founda-
tional part of an enterprise's business analytics plan (Bartlett, 2013),
moving the organization further along its path through the analytics
maturity model (Davenport and Harris, 2007; Isson and Harriott, 2013).
Efforts to improve data quality enable those throughout the company to
assess not just where data originate and how they are processed and
stored, but what data should be collected. Improved data quality is not the
only antecedent to better decision-making (McAfee and Brynjolfsson,
2012), but such improvements driven by managerial focus and changes
to organizational culture do play an important role (Lin and McDo-
nough, 2011).
B.T. Hazen et al. International Journal of Production Economics 193 (2017) 737–747

Finally, efforts to improve the quality of data used by the firm are not 25,000 records per month at this location.
entirely related to improved decision-making in a linear fashion. With its
roots in economic theory, the law of diminishing returns—originally
formulated by Ricardo, Malthus, West, and Torrens during correspon- 3.2. Data collection
dence over land rent related to corn-wage rates (Hollander, 1997)—
provides a foundation for the concept that the organization must find a Fig. 1 details the timeframe of data collection activities that occurred
balance between needed data quality and efforts to achieve further im- during this study. We placed one researcher in the organization during
provements. The information systems and management literature have execution of the DIP to observe the transformation. The rest of the
noted how limited resources can act as an organizational factor, limiting research team focused on research design and data analysis for the
the extent and success of a technology or process (Cohen and Levinthal, duration of the study. The embedded researcher directly observed the
1990; Fichman, 2004b). For instance, the effort to achieve an 80% DIP activities, strengthening the overall participatory aspects and rigor of
quality improvement might be worth the benefit, but increasing from the study. Embedding the researcher in the organization has its basis in
80% to 90% might come at a cost that is not, and could potentially impact general systems theory, with a view that phenomena can only be un-
the visibility of success of the data quality improvement project itself. derstood by viewing it as a whole rather than looking at individual parts
The literature suggests that organizations must determine how far they in isolation (Bertalanffy, 1968). The extension of the general systems
need to drive their data quality efforts, and at what point further efforts theory for this study flows from this concept that by placing the
will not yield entirely beneficial results. researcher in the system in a participatory role as well as a simultaneous
researcher role, the understanding would be greater than an external
view only (McIntosh, 2010; Patton, 2015).
3. Method
The embedded researcher made observations over a three-year
period. The first six months preceded the establishment of the DIP.
In consideration of the nature of the phenomenon of interest, we
During that period, the researcher gathered observations and informa-
chose an inductive single case study approach for our analysis (Patton,
tion regarding historical data quality to construct a baseline picture of the
2015; Yin, 2014). The focus of this longitudinal case study is on the data
before-state. Implementation began at the seven-month point, and
quality problem viewed in the operations management perspective. A
personnel immediately implemented all aspects of the DIP. However, it
U.S. Air Force (USAF) base in the continental United States that main-
took approximately six months of change management to affect changes
tains a large fleet of aircraft was the chosen site for this case study. The
at the organization. The final two years consisted of frequent review of
suitable unit of analysis is the organizational level and the appropriate
the process, and assessment of behaviors and outcomes. We objectively
commander granted permission for this study. We collected data
measured and compared data quality with before-state measures. The
regarding outcomes associated with the implementation of a data quality
researcher directly observed meetings and discussed changes and out-
improvement program. The base is representative of other USAF bases
comes with participants, to include management, DIP managers, and
throughout the world and is similar to civilian airfields from the
shop supervisors. Key participants also recorded and reported their ob-
perspective of airfield operations, aircraft maintenance, and operations
servations regarding behavioral changes and other outcomes.
management. Indeed, the general nature of the business practices and
In addition, using a purposive (nonprobability) sampling approach
operations supported by the data systems and data quality improvement
(Patton, 2015), the embedded researcher interviewed ten users repre-
program under investigation are similar to most production settings.
senting six different user bases over the course of nine weeks at the
2½-year-mark of the study. Of those interviewed, two informants are
3.1. Case study background
administrators who are responsible for AirData system management,
including the DIP. Also included in the sample of interviewees were DIP
With the assistance of a variety of interrelated information systems,
monitors, who are responsible for correcting errors in the database.
the USAF manages the world's largest fleet of aircraft. AirData (a pseu-
Maintenance technicians have, as their primary responsibility, the task of
donym) is the system dedicated to managing and documenting mainte-
inputting data after performing maintenance actions; therefore, the
nance activities for eight specific airframes and is used by over 30,000
researcher interviewed maintenance technicians as well. Interviews also
personnel worldwide. Data inputs are transmitted from each base into a
included maintenance managers who are responsible for overseeing the
centralized mainframe, where data are assimilated and interfaced with a
maintenance technicians and using reports to make operational-level
variety of other USAF and Department of Defense systems. Outputs of
AirData consist of informational reports that assist managers in making
fleet management and aircraft sustainability decisions, as well as aircraft
status determinations that directly impact operational capabilities. 6 Months Before Adoption:
Of note, AirData data are not used solely for routine reports; they are Pre-Improvement Observations
used for a myriad of purposes. In addition to scheduled reporting pur-
poses, for example, the data are used to document and predict fleet-wide Immediately Preceding
aircraft maintenance requirements. Additionally, operations managers at adoption:
higher echelons forecast aircraft availability using AirData data, which Baseline Assessment of Data
informs decisions regarding overseas operations, deployments, and other
long-term missions. Using AirData data, operations managers at the base
Post-Adoption:
level can determine the ideal aircraft employment schedule. Illustrating
Observations Regarding Data
another use of the data, logistics managers discern the best locations at Improvement Process
which to stage and house each aircraft for maximum effectiveness Implementation and Management
and efficiency.
Before being transmitted to the mainframe, data from the base are
subjected to a Data Integrity Program (DIP) process. This process consists 2 Years Post-Adoption:
of two steps. First, an automated program flags data records that are Improvement Process Review
suspected of containing errors regarding accuracy, consistency, and Outcome Assessment
completeness, and timeliness. Second, appointed DIP monitors manually
assess the flagged record to determine if there is indeed an error and
make any required corrections. The AirData system processes roughly Fig. 1. Major Milestones during Data Collection.

740
B.T. Hazen et al.
decisions. Ensuring there was a full spectrum of perspectives, the
researcher interviewed senior managers who are responsible for
tactical-level decision-making. To round out the complement of data
consumers and decision-makers, the researcher included senior execu-
tives who use AirData to make strategic-level aircraft fleet decisions in
the interviewee pool. The purpose of these interviews was to gather
additional data to refine the coding scheme and emerging nomological
network as well as to ensure that no other significant phenomena were
excluded. Therefore, the researcher employed both semi-structured and
unstructured interview techniques. As part of the interview, there were
five common questions that the researcher asked all participants (see
Appendix); subsequently, in the interview, the researcher asked partici-
pants to talk freely about anything else related to the DIP.
3.3. Validity and reliability
Validity and reliability were considered at all phases of the research.
To establish construct validity, our assessment included multiple sources
of evidence (Krippendorff, 2004). As previously mentioned, we captured
archival data from AirData and related systems, interviews, participant
observations, direct observations, and before/after data regarding data
quality. Further, the researcher interviewed employees at all levels
related to AirData and the DIP. To strengthen our analysis, key in-
formants at the organization reviewed drafts of the case study for errors
in content and chronology (Yin, 2014). The feedback of these informants
was paramount in the refining process and considerably improved the
accuracy of the findings.
Participants of the study also assessed the data; seeking and inte-
grating their feedback in assessing the findings were essential to the rigor
of the study. To ensure internal validity, we addressed rival explanations
in the findings of this study (Shadish et al., 2002, p. 508). In addition to
including the participants in assessing the data, we used power quotes
from the interviews; our intent was to develop the Propositions as a
means to demonstrate support for cause-effect relationships.
The approach to enhancing external validity of this single case study
was to use theory and literature to aid in Proposition development.
Comparing, contrasting, and integrating findings to extant theory and
developing Propositions based on both extant theory and the emerging
data allowed for more generalizable findings (Shadish et al., 2002).
Focusing on enhancing reliability, we established and adhered to a
detailed protocol for the case study (Yin, 2014, p. 240). Another view on
reliability is to consider the consistency between the results and the data
(Merriam and Tisdell, 2016, p. 251). To ensure this consistency, we
triangulated results through multiple methods of data collection and
analysis; by providing a thorough audit trail of the process, we
strengthened the study further (Merriam and Tisdell, 2016). Combined
with the interviews and participant-observations, there are ample
methods with which we sufficiently triangulated multiple sources of data
in order to enhance validity and reliability of the findings (Merriam,
1998; Yin, 2014).
4. Results and discussion
The analysis and discussion take the form of Proposition development
and presentation. We describe the findings in relation to extant research
and theory to form each proposition. The findings are presented in this
section to describe implications for practitioners and scholars.
4.1. Data and decision quality improvement
In the case study interviews, organizational members noted that poor
data quality originated from initial data entry, not data systems or
transfer processes. As part of the overall data improvement process in the
organization, members first investigated root-causes and identified base
process issues (Pierchala et al., 2009). In this case study, solutions to
address data quality problems included the addition of processes such as
741
International Journal of Production Economics 193 (2017) 737–747
training for data entry and data management personnel. Whether found
in root cause analysis or continuous monitoring and control, interview
subjects reported that a focus on improving data quality through studying
the overall process by which data are generated and flow through the
organization (Curtis et al., 1992) led to better data quality. By under-
standing the overall process, organizations can focus on the aspects of
data quality that are most important to the data consumers (Wang and
Strong, 1996; Bitterer and Newman, 2007). Understanding these di-
mensions of data quality allows an organization to operationally define
measures for these dimensions and apply tools to actively monitor for
data quality problems (Hazen et al., 2014). Thus, supported by literature
and as revealed through the case study interviews, we propose
the following:
Proposition 1a. Organizations that implement data quality improvement
methods will experience an increase in data quality.
In our study, we found that decision-makers realized that higher-
quality decisions were an outcome of the DIP. Decisions regarding
part-ordering, fault troubleshooting, aircraft mission planning, and
similar decisions upon which data from AirData are based improved, as
both decision-makers and those who manage decision-makers reported.
As one manager mentioned, “I can actually count on the data now,
instead of just guessing as to whether or not it can be trusted. The de-
cisions I make are more accurate now, and that shows up in our metrics.”
Another more senior manager relayed, “My parts managers are making
better decisions, and I can see that across some of our metrics.” Indeed,
managers in this case study overwhelmingly supported the notion that
the enhanced levels of data quality improved their decision-making.
These findings support extant literature, which states that the quality
of data used for decision-making is related to the quality of the decision
made when using such data (Ballou and Tayi, 1999; Cooper et al., 2000;
Wang and Strong, 1996; Wixom and Watson, 2001). Thus, DIP efforts
allow organizations to identify and remedy root causes of poor data
quality, which can result in quality improvements for both the data and
decisions (Curtis et al., 1992; Weller, 2000; Pierchala et al., 2009).
Proposition 1b. Organizations that implement data quality improvement
methods will experience increased decision quality.
Our analysis revealed that improvements to data quality throughout
the organization could benefit not only data generated internally within
the firm, but also the means by which the firm captures and interprets
information from both inside and out. Our interviews found that efforts
to improve data quality improved the ability of the organization as a
whole to learn, supporting the literature's suggested benefits of increased
absorptive capacity of the firm (Roberts et al., 2012). At the organiza-
tional level, the findings of our study suggest that data quality
improvement efforts grounded in changes to data generation and moni-
toring processes, and the information systems that enable them, yield
benefits. Due to the nature of the focal organization and the fact that data
at the organizational-level are transmitted and shared across other
business units inside and outside of the USAF, the improved data at the
focal organization was shown to have some positive cascading effects,
where the enhanced level of quality bolstered the view of data to that of a
knowledge resource (Grant, 1996b). This is because the users of the data
often saw data as having little value in its previous, poor-quality state;
however, as quality improved, so did the usefulness of the data, both to
the focal organization and its partners.
Although the unit of analysis is the organizational level, the research
revealed both vertical and horizontal implications to instituting the DIP,
which enabled other organizations to capitalize on the focal firm's
knowledge resources. For instance, senior managers in a lateral organi-
zation noted how the higher quality data derived from the focal orga-
nization helped to forecast workload within their organization, and led to
reduced inventory costs derived from more accurate service life infor-
mation. Similarly, managers at a vertical organizational unit remarked
that higher quality data achieved using DIP allowed for faster, more
B.T. Hazen et al.
reliable decision-making. These findings support the literature that pos-
tulates that benefits from improved data quality are not limited to just
one department or division but can have cascading impacts across the
entire value chain (Redman, 2008; Kahn et al., 2002).
Proposition 1c. Data quality improvement initiated by one organizational
element will positively affect decision quality in intra- and inter-organizational
elements with which the initiating element shares data.
4.2. Improvement in data quality behavioral aspects
The research literature suggests that there are also indirect means by
which data quality improvement methods can be used to enhance data
quality and contribute to data-driven business analytics. The result of our
analysis suggests that instituting data quality improvement initiatives is a
means for top managers to demonstrate support for and emphasize the
importance of business analytics in their organization, and drive
employee expectations (Hwang and Schmidt, 2011; Wigfield and Eccles,
2000; Wixom and Watson, 2001), which leads to greater commitment
toward high quality data. When management initiates a program for
monitoring and improving data quality, this may indicate interest in the
targeted data management processes, signaling to those who input and
manage data throughout the organization that the process is a manage-
rial priority. For instance, one technician stated, “now that we have to do
[the DIP process], I assume leadership really cares about the data.”
Another remarked, “I know that managers at headquarters are using a lot
of data to make decisions. I suppose they need the data to be good to do
that, so we're trying to do better.”
The DIP became a high-visibility item, and DIP compliance was
emphasized through meetings and establishment of DIP metrics. This
emphasis provides a corollary to Wixom and Watson's (2001) finding of
the association between high levels of data quality and high levels of
perceived net benefits. One manager noted, “I think that the workforce is
getting the point that the data doesn't just go into some ‘system’ and that
people are actually relying upon it to support decisions and to make their
lives easier, too.”
Proposition 2a. Organizations that implement data quality improvement
methods will be viewed by their employees and others as being committed to
high-quality data.
This level of management support also implies support of business
analytics that relies heavily on the quality of data on which it runs. Thus,
Proposition 2b. The outcome of the aforementioned level of management
support will be higher levels of data quality.
Proposition 2c. The outcome of the aforementioned level of management
support will be higher levels of decision quality.
In a similar vein, use of data quality improvement methods implies
specific metrics that are being monitored as well as visible signals to
employees that management is monitoring their ability to achieve those
metrics (Rungtusanatham, 2001). We witnessed this phenomenon in our
case study. Analysis of interview responses revealed that the metrics
established with the use of the DIP had an effect on employees of
establishing a subjective norm, thus affecting employees' focus because
those who work with data felt that their actions were being closely
monitored. Ajzen's theory of planned behavior (Ajzen, 1991, 2011) ad-
dresses this effect of subjective norm on one's intention toward a
behavior, and ultimately the behavior itself (1991, 2011). One supervisor
noted, “if my section's DIP numbers are bad, then I have to explain it.”
Another mentioned, “They are rating us on this now, so we have no
choice but to do it.” As such, data workers provided greater attention to
processes involved in data entry, compilation, and management. For
instance, a technician mentioned, “it's easier to just do it right the first
time. If the DIP says I have errors, then I just got to go in and do it over
again.” The before-and-after data quality metrics support such
improvement.
742
International Journal of Production Economics 193 (2017) 737–747
Additionally, the improvement may be impacted by the data workers'
increased perceived behavioral control, or “perceived ease … of performing
the behavior” (Ajzen, 1991, p. 188). At the commencement of the DIP
tool, when initial data quality metrics were first tracked, initial error
rates averaged above 30%. After implementation, error rates before
correction average four percent and are corrected using DIP to an average
of 0.15%. This suggests that even the mere the existence of the program
itself continues to motivate data quality improvements and influence
workers' attitudes toward enhancing data quality, indicating an effect of
employees' attitudes toward behavior (Ajzen, 1991, 2011). Indeed, as one
technician noted, “I never thought anyone looked at this stuff before. I
guess they need [the data] to be right so that we may take our time now.”
Proposition 3a. Organizations that implement data quality improvement
methods will experience positive behavior changes in their employees, partic-
ularly those involved in data handling processes (3a).
Proposition 3b. The careful level of attention by employees will, in turn,
lead to fewer errors overall.
Proposition 3c. The careful level of attention by employees will lead to a
greater understanding of the need for high-quality data throughout the
organization.
4.3. Analytics cycle improvements
In our case study, participant responses suggested support for For-
gionne's (1999) note that decision makers must dedicate time and re-
sources to clean data vital for analysis. For instance, one manager noted,
“I used to spend a lot of time trying to figure out if the information was
correct. This added several minutes—if not hours—when trying to
determine the right maintenance actions to take.” Another manager said,
“I used to have to go through the shop's logbooks to see what really
happened, because you know, the [information system] was never right.”
A supervisor mentioned previously having to validate certain tasks with
actual individuals that did the work to determine if the data were correct.
Indeed, some research suggests that spending the time to find more ac-
curate data is not always of benefit, due to the added time that it takes
and the fact that one cannot always track down the correct data (Frank,
2008). In fact, current literature suggests that approximately 80% of the
analytic cycle time is often consumed by cleaning the data and ensuring
they are correct and properly prepared for analysis (Dasu and Johnson,
2003; Wickham, 2014). One finding of note is that some managers have
to balance the anticipated benefit of verifying data with the need to make
a timely decision. As such, tradeoffs would have to be made. However,
after the DIP was instituted, managers spend much less time verifying
data and, if they still needed to verify data (often due to having to make
an important, high-dollar decision), they had better insight into these
tradeoffs. Upon implementing data quality improvement methods, the
focal organization realized higher quality data and information, which
evoked further use and satisfaction and ultimately improved
decision-making, again supporting the Delone and McLean (2003) model
for information systems success due to aspects of quality. This cycle
continued over time, as witnessed by the embedded researcher,
providing support for the feedback loops and interrelationships described
in the Information Systems Success Model.
Proposition 4. Organizations that implement data quality improvement
methods will experience reduced costs for verifying data quality. Organizations
will also be better positioned to successfully balance costs of verification with
costs of reduced quality to maximize resources, decision quality, and
net benefits.
In our case study interviews, one manager noted, “… we are doing so
much more with our data now. We've implemented new predictive an-
alytics tools that we never would have been able to use before.” This
finding supports recent literature where employees reported that the
improved quality data makes the analytics methods and tools available to
B.T. Hazen et al.
them more useful (Venkatesh and Bala, 2008; Venkatesh et al., 2012) and
increased feelings of empowerment on the part of the decision-making
employees (Kim and Gupta, 2014). Through purposeful data improve-
ment processes, managers may be able to identify and control problems
with data quality by using data quality improvement methods before
poor data quality becomes a pandemic and discourages use. Our case
study results and the literature lead us to propose the following:
Propositions 5a. Organizations that implement data quality improvement
methods will experience an increase in the use of business analytics tools
because the underlying data is perceived to be of high quality (5a).
Propositions 5b. An increase in the use of business analytics tools due to
the underlying perception that the data are of high quality will, in turn, lead to
higher quality decisions.
If a situation exists in which data quality is already perceived as being
poor, and many potential users are dissuaded from employing business
analytics, demonstrating improvement in data quality may be one way in
which management can energize their analytics efforts. Assuming that
data quality improvement initiatives have commenced, quantifiable ev-
idence of data quality improvement can be shared (as seen in the case
study), which demonstrates improvements in data quality to users. This
sense of increased reliability of data was shown to instill confidence in
managers who wish to use the data for decision-making, but have been
hesitant because of low confidence in the data, and subsequently, the
decisions derived from such data. This finding complements Expectation-
Value theories (Wigfield and Eccles, 2000) by suggesting that enhanced
levels of confidence in data quality lead to increased confidence in the
decisions made in consideration of such data.
Proposition 6a. Members of organizations that implement data quality
improvement methods will experience an increase in confidence related to
data quality.
Proposition 6b. An increase in confidence related to data quality will in-
crease confidence in decision-making.
Proposition 6c. : Confidence in decision-making can enhance decision
processes, and lead to more timely decisions.
Proposition 6d. Confidence in decision-making can enhance decision
processes, and lead to higher quality decisions.
Additionally, one manager remarked, “we're simply better at doing
analytics now.” Another stated: “I wasn't a big proponent of using ana-
lytics before. This was due somewhat to not understanding the value. It
was also due to the fact that I knew our data were [expletive].” Most in
management agreed that the DIP was not a panacea, but they generally
agreed that it was an antecedent to their employment of data analytics.
This study found that higher confidence in the quality of data spurs
company-wide discussions regarding the right questions that need to be
asked supporting Kahn et al. (2002) and Redman (2008). The informa-
tion technology department can coordinate efforts to modify the infra-
structure to alter data collection, storage, modeling, and analysis
processes based on the refined set of questions being asked by the
management team. Since organization-wide attention is already focused
on issues with data and decision-making, communication of results from
this refinement and reanalysis process is already occurring across divi-
sional boundaries beyond the level pre-DIP in the case study organiza-
tion. As found from the case study interviews, management saw little
value in investing resources in a full analytics process until they
perceived a level of quality to their data justifying the effort. This has
been a common theme found in industry research (Marsh, 2005).
Because any DIP is a company-wide effort, aligning everyone's view of
“reality,” all parts of the analytics cycle have the potential to be posi-
tively impacted.
Proposition 7. Increases in data quality driven by data quality improve-
ment will yield better analytics cycle outcomes.
743
International Journal of Production Economics 193 (2017) 737–747
This study found that the DIP gave uniform attention to all data.
Although some data were more critical to decision-making than other
data, this distinction was never made clear. One maintenance technician
noted, “why does anyone care about this [general maintenance task]?”
Yet, a manager mentioned how he needs to make certain decisions about
which most technicians are not familiar. Indeed, different data were used
to support different decisions at different levels, and although most in-
dividuals understood the decisions that are important to subordinates,
most were not familiar with the requirements of levels above them. Thus,
when instituting data quality improvement initiatives, it is important to
communicate analytics needs across the organization. This helps orga-
nizations understanding how information flows through, and is lever-
aged by, the organizations. Extant literature suggests that by
organizations understanding the overall information flow, they can focus
on the aspects of data quality that are most important to the various data
consumers within the firm (Wang and Strong, 1996; Bitterer and New-
man, 2007).
At the current time, such communication is largely absent at the case
study organization. However, this communication need not be difficult.
One suggestion is to develop documentation and guidance summarizing
what data are required for which decisions at what levels of the orga-
nization; the generated documentation and guidance should then be
disseminated across an organization. When DIPs are first implemented,
organizational resources can be scarce. By delineating which data are
most important for decision-making, the firm can limit resources to those
data sources and systems that produce the most valuable data. Of course,
the quality of all data in the organization can be improved given enough
time and resources; but, a focus on the most valuable and initially
important data can provide a quick demonstration of the worth of
improvement efforts. Although data-documenting materials would
require periodic updates, such materials could go a long way to routin-
izing and improving the business analytics cycle, because those at all
levels would have a better understanding of how data are used. Addi-
tionally, data could be ranked not only by its use and value, but also by
which data are of the poorest quality. Overall, firms should focus their
initial data quality improvement efforts on the most compelling set of
data used for decision-making within the firm. This supports the litera-
ture that suggests total data quality management should not be viewed as
an end-state but, rather, a process of continual improvement and treat-
ment of data as an organizational resource (Eppler and Helfert, 2004;
Haug et al., 2011; Marsh, 2005).
Proposition 8a. As the clarity and precision of the decision-making data
requirement increases, data quality improves.
Propositios 8b. As communication of the pertinent data requirements in-
creases vertically through the organization, data quality improves. Commu-
nication of data requirements can support data quality improvement efforts, as
well as business analytics performance.
4.4. Potential drawbacks
Consistent with the Hawthorne Effect, we recognized that as
personnel became more familiar with analytics—and with the accom-
panying extra attention from management—those using DIP also gained
proficiency in data analysis. As maintenance personnel recognized im-
provements in efficiency and effectiveness based on the value of data
analytics, they became more focused toward mastering their interaction
with the DIP and the analysis process.
Conversely, we found that some managers focused too much on DIP
results, to the point where DIP metrics were treated as equal to actual
performance indicators. Although important, once any metric is on
management's radar, there is a chance that it will be overemphasized.
Managers seemed to address the “lack of” DIP compliance, even when it
was in the 90þ percentile (vice the 100% completion as required in the
organization) in the same manner as low aircraft availability metrics
(primary organizational performance measures). Subordinates were
B.T. Hazen et al.
sometimes more apt to focus on 100% completion of the DIP to the peril
of other tasks, which may be have been more important. Data quality
improvement initiatives represent another process and suite of metrics
that must be managed. Managers responsible for such programs might
focus too much attention on the program vice the outcome by failing to
consider cost, time, and quality levels appropriate for a project's objec-
tives (Liberatore and Pollack-Johnson, 2013). Outcomes and achieve-
ment goals overall can be harmed when an over-emphasis on achieving
metric or performance levels comes at the expense of helping organiza-
tional members resolve underlying organizational issues (Choo, 2011).
This is often referred to as Goodhart's law, which suggests when a mea-
sure becomes a target, it ceases to be a good measure because “any
observed statistical regularity will tend to collapse once pressure is
placed upon it for control purposes” (Goodhart, 1975a, 1975b). The more
any quantitative indicator is used for decision-making, the more subject
it will be to corruption pressures, and the more apt it will be to distort and
corrupt the processes it is intended to monitor (Campbell, 1979). Man-
agers are cautioned to prioritize the need for data quality improvement in
consideration of other operational processes.
Proposition 9. Over-emphasis on data quality metrics by firm leadership
tend to accompany a related decrease in other organizational perfor-
mance metrics.
This study found that initiating the DIP within the organization
required a substantial level of additional work. No additional manpower
was provided, and every new function was added to existing workloads.
Management was required to initialize and track new metrics related to
DIP; maintenance supervisors were required to ensure completion; ana-
lysts had to establish and then manage the program, and maintenance
workers were required to spend additional time validating data. Many
individuals, especially maintenance workers and supervisors, were con-
cerned about the additional workload and commented that if man-hours
were used to improve data quality, then those were man-hours not being
used to conduct other work and achieve existing operational re-
quirements. One maintenance supervisor remarked, “it's one more thing
to do. Our plates are full … if we add this to our plate, something else is
going to need to come off.” Another mentioned, “the [DIP] is a good idea
… but who's going to do it? I don't have the people to do that.”
To date, the organization investigated in the study has yet to find the
optimal level of quality that should be achieved to properly balance
program costs with benefits to prevent a diminishing returns effect
(Cohen and Levinthal, 1990; Fichman, 2004a; Hollander, 1997). The
questions of “how many resources will be needed” and “what level of
quality is needed to achieve the decision-making benefits sought” are
questions stakeholders within the firm will need to address as DIPs are
planned and put into place. The improvement of data quality is an
organization-wide event with end-to-end impacts for the firm, and with
firm-wide increases in workload to handle the monitoring and
improvement of data.
Proposition 10. Once major and impactful data quality issues in the firm
have been resolved, successive improvements will require additional workload,
effort, and resources yielding diminishing returns for the costs involved.
Managers will need to find ways to weigh these costs against anticipated
benefits to achieve optimal ROI.
5. Summary
When considered through the information processing view of the firm
(Galbraith, 1974; Premkumar et al., 2005), ever larger volumes of data
that must be consumed by the individual decision-maker will inevitably
lead to information overload (Eppler and Mengis, 2004). Eppler and
Mengis (2004), state that information overload can be caused by, among
other reasons, “the information itself (its quantity, frequency, intensity,
and quality), … organizational design … and the information technology
that is used” in the firm (p. 330). Ultimately, the true quality and value of
744
International Journal of Production Economics 193 (2017) 737–747
data are its ability to provide decision-makers with the effective insights
to choose from among options for potential organizational actions, a view
of data quality set firmly in the decision theory view (Berger, 1985; Karr
et al., 2006). When taken together, the above Propositions suggest the
alteration of data-generating organizational processes and inclusion of
sophisticated information systems designed with a focus on improving
data quality.
To this end, Premkumar et al. (2005) specifically mention that one
strategy organizations facing competitive uncertainty and increased
dependence on information for decision-making follow is to “redesign
business processes” and “[implement] integrated information systems
that improve information flow and reduce uncertainty” (p. 260) (p. 260)
(p. 260) (p. 260) (p. 260) (p. 260). Further, Maier et al. (2012), note the
importance of a systematic assessment of organizational capabilities and
practices to provide a realistic view of the company's abilities and how
they compare against major competitors before making process changes.
Attempts to deliberately improve data quality through redesign of data
generation and management processes such that the DIP examined in this
research, can help to reduce uncertainty surrounding data used to inform
decision-making (Jones-Farmer et al., 2014; Karr et al., 2006). Methods
that are integrated into a data-generation and monitoring process can
equip the organization with the tools it needs to handle ever-increasing
volumes of data. Organizations of all sizes, and particularly their deci-
sion makers, can benefit from the reduced cognitive load realized by
improving data generation and quality monitoring processes, and pre-
venting information overload (Eppler and Mengis, 2004); thereby
improving data-grounded evidence-based decision-making.
The relationships uncovered in this research inform the development
of Fig. 2, the primary model. Fig. 2 illustrates a conceptual model of
outcomes associated with data quality improvement initiatives as
explained in Propositions 1 through 7 and Proposition 10 above. The
solid arrows indicate the primary effects. The single-dashed lines show
the secondary effects, and the double-dashed lines illustrate ter-
tiary effects.
Additionally, Fig. 3 illustrates the proposed relationship between
management emphasis on data quality and the impact of this emphasis
on the non-data quality, more general organizational outcome metrics as
described in Proposition 9. Organizational leaders must balance appro-
priate attention to organizational changes needed to improve the quality
of data used in analytics with a focus on preventing neglect with regards
to other organizational outcomes due to resource-drain. Finally, Fig. 4
illustrates the challenge for organizational leadership: as data quality
improves, the effort required to make successive improvements in-
creases, as explained in Proposition 10.
6. Implications of findings
6.1. Theoretical contributions
This study makes several important theoretical contributions. This is
the first study to comprehensively assess direct outcomes of employing
data quality improvement methods. Literature in this area is typically
focused on data quality improvement and assumes that positive out-
comes will naturally emerge, without any supporting evidence. Second,
most of this literature is applied in nature and often contributes to
mathematical theories related to quality improvement. By using an
inductive, single case study approach and integrating the findings with
extant management theory, this is one of the first studies in this area that
contributes to the theoretical implications of data quality improvement.
Third, the proposed framework and testable Propositions presented in
this article lay the groundwork for a new area of research regarding the
benefits and drawbacks of data quality improvement in support of
business analytics.
B.T. Hazen et al. International Journal of Production Economics 193 (2017) 737–747

Clarity/Precision of
Data Requirements
P8a + P1c + Data Quality of intra- and
inter-organiza onal elements

Communica on of
P8b + P7 +
Data Analy cs Cycle
Data Requirements Quality Outcomes
P2c +
Decision
P1a + P1b + Quality
P2b +
P2a + Stakeholder Perspec ve of
Commitment to Quality

Implemen ng Data P3a + P3b – Data Errors

Posi ve Employee
Quality Improvements Behavior Changes P3c +
Understanding of
P4 – Need for High Quality
Data Quality
Verifica on Costs
P5a +
P6a + Business Analy cs P5b +
Tool Use
Confidence in P6d +
Data Quality
P6b +
Confidence in
Decision-Making P6c +
Decision
Timeliness

Fig. 2. Conceptual Model of Data Quality Improvement Outcomes.

X = Point at which Leadership Emphasis maximizes

non-Data Quality Performance Metrics
High
High
Leadership Emphasis on Data Quality metrics

Effort Required
Low
Low

Low High
Low X High
Impact on non-DQ Performance Metrics
Data Quality Improvements

Fig. 3. Relationship between Leadership Emphasis on Data Quality (DQ) and the Impact Fig. 4. Relationship between Effort Required and Data Quality Improvement (Proposi-
on non-DQ Performance Metrics (Proposition 9). tion 10).

6.2. Managerial contributions increase analytic applications and evidence-based decision-making. In

other words, data improvement efforts are a fundamental requirement
This research also provides relevant implications for managers, who for organizations looking to increase their usage of analytics for better
can glean actionable information as to how they might improve their decision making.
business analytics programs. First, managers can rest assured that active Third, this research highlights potential drawbacks that managers
data quality improvement efforts can result in enhanced data quality and should acknowledge regarding data improvement efforts. These draw-
decision-making outcomes within their organization. Data improvement backs include the potential for biased emphasis regarding performance
efforts also have the potential to have positive effects on decisions made metrics and the uncertainty of optimally allocating resources to these
by firms horizontally and vertically aligned to the focal organization efforts. Thus, organizational leaders must weigh the deserved attention
along with improving the trust and commitment towards analytically required for data quality improvement with that on preventing negative
focused behavior. Thus, managers may use data improvement efforts to organizational outcomes that may accompany the effort.
not only change behaviors and improve decisions within their organi-
zation but also to drive these same attributes across their partner firms. 7. Conclusions and future research
Second, data improvement efforts have shown a long-term reduction
in data quality verification costs. Moreover, these efforts promote greater There are some limitations to this study that may inhibit the gener-
confidence in data quality and, therefore, the analytic and decision alizability of the results. First, the case study was focused on one orga-
outcomes from these data. Consequently, managers can use data nization type and size, and one data quality improvement program.
improvement efforts to promote greater confidence in their efforts to Future research can expand its focus across organizational types and DIP

745
B.T. Hazen et al.
sizes/implementations. This will ensure generalizability of results and
further examination of contextual factors that impact the relationship
between data quality improvement and outcomes from the decisions
made upon analysis of high-quality data in the organization. A second
limitation is that the study used an exploratory, inductive case study
approach. Future studies can use these results as the basis to engage in
both qualitative and quantitative confirmatory techniques to present a
more well-rounded view of the relationship between data quality and
organizational outcomes. Finally, we did not examine the analytic tech-
niques that would use the improved-quality data. The quality and so-
phistication of analytic techniques used by the organization would both
be impacted by the perception of the quality of the firm's data, and would
also serve as a mediating factor between the data and the impact of the
decisions made upon them. Future studies should take the analytic
technique/decision-making process itself into account when investi-
gating the nature of the data quality/analytics outcome relationship.
Given that organizations are beginning to work more deeply with
their large sources of data, the quality of that data is becoming increas-
ingly more important. The business analytics cycle—collecting, storing,
and retrieving the data, and then summarizing, analyzing, and commu-
nicating the results—critically relies on data that is of high quality.
Lacking high-quality data, the business decisions made are detrimentally
affected and increasingly ineffectual. Decisions based on outputs from
analytic techniques cannot be of high quality unless both the data
analyzed and the methods themselves are also of a high quality. In this
research, we have directly observed and assessed the outcomes generated
for both data and decision quality after the implementation of data
quality improvement efforts. Additionally, we synthesized the study
findings with extant management literature and theory, showing the
organizational outcomes that can be expected from data quality program
implementations. The findings suggest that subjecting data to contem-
porary quality control measures will help focus organizational attention
on data quality enhancement and improving analytics processes, which
invariably lead to better decision-making.
Also, the resultant proposed framework lays a useful groundwork for
the research community to further explore the relationships between
data quality improvement and organizational outcomes. Such relation-
ships have been assumed in the literature to be of a positive nature, and
this study provides evidence to confirm these assertions. However, our
analysis posits that these relationships are much more complex and
contextualized in nature. The improvement program analyzed in this
study can suggest a starting point for practitioners seeking a way to
improve their organization's data quality. Additionally, the Propositions
can help the practitioner gain top management and stakeholder support
for data quality improvement initiatives, as outcomes and the organiza-
tional costs associated with them are composed.
References
Ajzen, I., 1991. The theory of planned behavior. Organ. Behav. Hum. Decis. Process. 50
(2), 179–211.
Ajzen, I., 2011. The theory of planned behaviour: reactions and reflections. Psychol.
Health 26 (9), 1113–1127.
Akter, S., Wamba, S.F., Gunasekaran, A., Dubey, R., Childe, S.J., 2016. How to improve
firm performance using big data analytics capability and business strategy alignment?
Int. J. Prod. Econ. 182, 113–131.
Argyris, C., Sch€
on, D.A., 1999. Organizational Learning II: Theory, Method, and Practice.
Addison-Wesley Publishing Company, Reading, MA.
Ballou, D.P., Pazer, H.L., 1985. Modeling data and process quality in multi-input, multi-
output information systems. Manag. Sci. 31 (2), 150–162.
Ballou, D.P., Tayi, G.K., 1999. Enhancing data quality in data warehouse environments.
Commun. ACM 42 (1), 73–78.
Ballou, D.P., Wang, R., Pazer, H., Tayi, G.K., 1998. Modeling information manufacturing
systems to determine information product quality. Manag. Sci. 44 (4), 462–484.
Bartlett, R., 2013. A Practitioner's Guide to Business Analytics. McGraw Hill, New York,
NY.
Batini, C., Cappiello, C., Francalanci, C., Maurino, A., 2009. Methodologies for data
quality assessment and improvement. Assoc. Comput. Mach. Comput. Surv. 41 (3),
1–52.
Benbasat, I., Goldstein, D.K., Mead, M., 1987. The case research strategy in studies of
information systems. MIS Q. 369–386.
746
International Journal of Production Economics 193 (2017) 737–747
Berger, J.O., 1985. Statistical Decision Theory and Bayesian Analysis. Springer-Verlag,
New York, NY.
Bertalanffy, L.V., 1968. General Systems Theory: Foundations, Development, Applications
Braziller. George Braziller, Inc, New York.
Bitterer, A., Newman, D., 2007. Organizing for Data Quality. Gartner Research, Stamford,
CT.
Bose, R., 2009. Advanced analytics: opportunities and challenges. Ind. Manag. Data Syst.
109 (2), 155.
Campbell, D.T., 1979. Assessing the impact of planned social change. Eval. Program Plan.
2 (1), 67–90.
Castiaux, A., 2007. Radical innovation in established organizations: being a knowledge
predator. J. Eng. Technol. Manag. 24 (1–2), 36–52.
Chen, H., Chiang, R.H., Storey, V.C., 2012. Business intelligence and analytics: from big
data to big impact. MIS Q. 36 (4), 1165–1188.
Choo, A.S., 2011. Impact of a stretch strategy on knowledge creation in quality
improvement projects. Eng. Manag. IEEE Trans. 58 (1), 87–96.
Cohen, W.M., Levinthal, D.A., 1990. Absorptive capacity: a new perspective on learning
and innovation. Adm. Sci. Q. 35 (1), 128–152.
Cook, S.D.N., Yanow, D., 1993. Culture and organizational learning. J. Manag. Inq. 2 (4),
373–390.
Cooper, B.L., Watson, H.J., Wixom, B.H., Goodhue, D.L., 2000. Data warehousing
supports corporate strategy at First American Corporation. MIS Q. 547–567.
Creswell, J., 2013. Research Design: Qualitative, Quantitative, and Mixed Methods
Approaches, fourth ed. Sage Publications, Inc, Thousand Oaks, California.
Crossan, M.M., Lane, H.W., White, R.E., 1999. An organizational learning framework:
from intuition to institution. Acad. Manag. Rev. 24 (3), 522–537.
Curtis, B., Kellner, M.I., Over, J., 1992. Process modeling. Commun. ACM 35 (9), 75–90.
Dasu, T., Johnson, T., 2003. Exploratory Data Mining and Data Cleaning, vol. 479. John
Wiley & Sons.
Davenport, T.H., 2006, Jan. Competing on analytics. Harv. Bus. Rev. 84, 98–107.
Davenport, T.H., Harris, J.G., 2007. Competing on Analytics: the New Science of Winning.
Harvard Business School Publishing Corporation, Boston, MA.
Davis, F., 1989. Perceived usefulness, perceived ease of use, and user acceptance of
information technology. MIS Q. 13 (3), 319–340.
Davis, F.D., Bagozzi, R.P., Warshaw, P.R., 1989. User acceptance of computer technology:
a comparison of two theoretical models. Manag. Sci. 35 (8), 982–1002.
DeLone, W., McLean, E., 1992. Information systems success: the quest for the dependent
variable. Inf. Syst. Res. 3 (1), 60–95.
Delone, W., McLean, E., 2003. The DeLone and McLean model of information systems
success: a ten-year update. J. Manag. Inf. Syst. 19 (4), 9–30.
Drucker, P.F., 1991. The new productivity challenge. (cover story). Harv. Bus. Rev. 69 (6),
69–79.
Dub
e, L., Par
e, G., 2003. Rigor in information systems positivist case research: current
practices, trends, and recommendations. MIS Q. 597–636.
Dutta, D., Bose, I., 2015. Managing a big data project: the case of ramco cements limited.
Int. J. Prod. Econ. 165, 293–306.
Dyson, R.G., Foster, M.J., 1982. The relationship of participation and effectiveness in
strategic planning. Strat. Manag. J. 3 (1), 77.
Eppler, J.M., Helfert, M., 2004. A classification and analysis of data quality costs. In:
Paper Presented at the International Conference on Information Quality, Cambridge,
MA.
Eppler, J.M., Mengis, J., 2004. The concept of information overload: a review of literature
from organization science, accounting, marketing, MIS, and related disciplines. Inf.
Soc. 20 (5), 325–344.
Even, A., Shankaranarayanan, G., Berger, P.D., 2010a. Evaluating a model for cost-
effective data quality management in a real-world CRM setting. Decis. Support Syst.
50 (1), 152–163.
Even, A., Shankaranarayanan, G., Berger, P.D., 2010b. Managing the quality of marketing
data: cost/benefit tradeoffs and optimal configuration. J. Interact. Mark. 24 (3),
209–221.
Ezell, J.D., Hazen, B.T., Hall, D.J., Jones-Farmer, L.A., 2016. Enhancing data and decision
quality with statistical process control. In: Warkentin, M. (Ed.), The Best Thinking in
Business Analytics from the Decision Sciences Institute. Pearson FT Press, Upper
Saddle River, NJ, pp. 17–33.
Fenton, N.E., Pfleeger, S.L., 1998. Software Metrics: a Rigorous and Practical Approach,
second ed. PWS Publishing Co, Boston, MA.
Fichman, R.G., 2004a. Going beyond the dominant paradigm for information technology
innovation research: emerging concepts and methods. J. Assoc. Inf. Syst. 5 (8),
314–355.
Fichman, R.G., 2004b. Real options and IT platform adoption: implications for theory and
practice. Inf. Syst. Res. 15 (2), 132–154. http://dx.doi.org/10.1287/isre.1040.0021.
Fishbein, M., Ajzen, I., 1975. Belief, Attitude, Intention and Behavior: an Introduction to
Theory and Research. Addison-Wesley, Reading, MA.
Fisher, C.W., Chengalur-Smith, I.S., 2003. Impact of experience and time on the use of
data quality information in decision making. Inf. Syst. Res. 14 (2), 170–188.
Fisher, C.W., Kingma, B.R., 2001. Criticality of data quality as exemplified in two
disasters. Inf. Manag. 39 (2), 109–116.
Forgionne, G.A., 1999. An AHP model of DSS effectiveness. Eur. J. Inf. Syst. 8 (2),
95–106.
Frank, A.U., 2008. Analysis of dependence of decision quality on data quality. J. Geogr.
Syst. 10 (1), 71–88.
Galbraith, J.R., 1974. Organization design: an information processing view. Interfaces 4
(3), 28–36.
Goodhart, C., 1975a. Monetary Relationships: a View from Threadneedle Street Papers in
Monetary Economics, vol. I. Reserve Bank of Australia.
B.T. Hazen et al.
Goodhart, C., 1975b. Problems of Monetary Management: the UK Experience Papers in
Monetary Economics, vol. I. Reserve Bank of Australia.
Grant, R.M., 1996a. Prospering in dynamically-competitive environments: organizational
capability as knowledge integration. Organ. Sci. 7 (4), 375–387.
Grant, R.M., 1996b. Toward a knowledge-based theory of the firm. Strat. Manag. J. 17
(S2), 109–122.
Haug, A., Arlbjørn, J.S., 2011. Barriers to master data quality. J. Enterp. Inf. Manag. 24
(3), 288–303.
Haug, A., Zachariassen, F., Van Liempd, D., 2011. The costs of poor data quality. J. Ind.
Eng. Manag. 4 (2), 168–193.
Hazen, B.T., Boone, C.A., Ezell, J.D., Jones-Farmer, L.A., 2014. Data quality for data
science, predictive analytics, and big data in supply chain management: an
introduction to the problem and suggestions for research and applications. Int. J.
Prod. Econ. 154, 72–80.
Hey, T., 2010, Nov. The big idea: the next scientific revolution. Harv. Bus. Rev. 88, 56–63.
Ho, L.L., Quinino, R.C., 2013. An attribute control chart for monitoring the variability of a
process. Int. J. Prod. Econ. 145 (1), 263–267.
Hollander, S., 1997. The Economics of Thomas Robert Malthus, vol. 4. University of
Toronto Press.
Huang, K., Lee, Y., Wang, R.Y., 1999. Quality Information and Knowledge. Prentice Hall,
Saddle River, NJ.
Hwang, M.I., Schmidt, F.L., 2011. Assessing moderating effect in meta-analysis: a re-
analysis of top management support studies and suggestions for researchers. Eur. J.
Inf. Syst. 20 (6), 693–702.
Isson, J.P., Harriott, J., 2013. Win with Advanced Business Analytics: Creating Business
Value from Your Data. John Wiley & Sons, Hoboken, New Jersey.
Jones-Farmer, L.A., Ezell, J.D., Hazen, B.T., 2014. Applying control chart methods to
enhance data quality. Technometrics 56 (1), 29–41.
Kahn, B.K., Strong, D.M., Wang, R.Y., 2002. Information quality benchmarks: product and
service performance. Commun. Assoc. Comput. Mach. 45 (4), 184–192.
Karr, A.F., Sanil, A.P., Banks, D.L., 2006. Data quality: a statistical perspective. Stat.
Methodol. 3 (2), 137–173. http://dx.doi.org/10.1016/j.stamet.2005.08.005.
Kim, H.-W., Gupta, S., 2014. A user empowerment approach to information systems
infusion. Eng. Manag. IEEE Trans. 61 (4), 656–668.
Krippendorff, K., 2004. Content Analysis an Introduction to its Methodology, second ed.
Sage Publications, London.
LaValle, S., Lesser, E., Shockley, R., Hopkins, M.S., Kruschwitz, N., 2010, Winter. Big data,
analytics, and the path from insights to value. MIT Sloan Manag. Rev. 52, 21–31.
Lenz, H.J., Borowski, E., 2010, Aug. Business data quality control - a step by step
procedure. In: Paper Presented at the 10th International Workshop on Intelligent
Statistical Process Control, Seattle, WA.
Liberatore, M.J., Pollack-Johnson, B., 2013. Improving project management decision
making by modeling quality, time, and cost continuously. Eng. Manag. IEEE Trans. 60
(3), 518–528.
Lin, H.-E., McDonough III, E.F., 2011. Investigating the role of leadership and
organizational culture in fostering innovation ambidexterity. Eng. Manag. IEEE
Trans. 58 (3), 497–509.
Maier, A.M., Moultrie, J., Clarkson, P.J., 2012. Assessing organizational capabilities:
reviewing and guiding the development of maturity grids. Eng. Manag. IEEE Trans.
59 (1), 138–159.
Marsh, R., 2005. Drowning in dirty data? It's time to sink or swim: a four-stage
methodology for total data quality management. J. Database Mark. Cust. Strategy
Manag. 12 (2), 105–112.
Mayer-Sch€ onberger, V., Cukier, K., 2013. Big Data: a Revolution that Will Transform How
We Live, Work, and Think. Houghton Mifflin Harcourt, Boston.
McAfee, A., Brynjolfsson, E., 2012. Big data: the management revolution. Harv. Bus. Rev.
90 (10), 60–69.
McIntosh, P., 2010. Action Research and Reflective Practice. Creative and Visual Methods
to Facilitate Reflection and Learning. Routledge, Oxon.
Merriam, S.B., 1998. Qualitative Research and Case Study Applications in Education.
Revised and Expanded from“ Case Study Research in Education.”: ERIC.
Merriam, S.B., Tisdell, E.J., 2016. Qualitative Research: a Guide to Design and
Implementation, fourth ed. Jossey-Bass.
Mitra, A., 2008. Fundamentals of Quality Control and Improvement, third ed. John Wiley
& Sons, Hoboken, NJ.
Ou, Y., Wu, Z., Tsung, F., 2012. A comparison study of effectiveness and robustness of
control charts for monitoring process mean. Int. J. Prod. Econ. 135 (1), 479–490.
Patton, M.Q., 2015. Qualitative Evaluation and Research Methods, fourth ed. SAGE
Publications, Inc, Thousand Oaks, California, USA.
Pavlou, P.A., El Sawy, O.A., 2006. From IT leveraging competence to competitive
advantage in turbulent environments: the case of new product development. Inf. Syst.
Res. 17 (3), 198–227.
Pierchala, C.E., Surti, J., Peytcheva, E., Groves, R.M., Kreuter, F., Kohler, U., Young, J.,
2009. Control charts as a tool for data quality control. J. Off. Stat. 25 (2), 167–191.
Pipino, L.L., Lee, Y.W., Wang, R.Y., 2002. Data quality assessment. Commun. Assoc.
Comput. Mach. 45 (4), 211–218.
Port, D., Bui, T., 2009. Simulating mixed agile and plan-based requirements prioritization
strategies: proof of concept and practical implications. Eur. J. Inf. Syst. 18 (4),
317–331.
747
View publication stats
International Journal of Production Economics 193 (2017) 737–747
Porter, L.J., Rayner, P., 1992. Quality costing for total quality management. Int. J. Prod.
Econ. 27 (1), 69–81.
Powell, P., Woerndl, M., 2008. Time to stop researching the important things. Eur. J. Inf.
Syst. 17 (2), 174–178.
Premkumar, G., Ramamurthy, K., Saunders, C.S., 2005. Information processing view of
organizations: an exploratory examination of fit in the context of interorganizational
relationships. J. Manag. Inf. Syst. 22 (1), 257–294.
Price, R., Shanks, G., 2011. The impact of data quality tags on decision-making outcomes
and process. J. Assoc. Inf. Syst. 12 (4).
Raghunathan, S., 1999. Impact of information quality and decision-maker quality on
decision quality: a theoretical model and simulation analysis. Decis. Support Syst. 26
(4), 275–286.
Redman, T.C., 1992. Data Quality: Management and Technology. Bantam Books, New
York, NY.
Redman, T.C., 1996. Data Quality for the Information Age. Artech House Publishers,
Norwood, MA.
Redman, T.C., 2001. Data Quality: the Field Guide. Digital Press, Boston, MA.
Redman, T.C., 2008. To solve this data-driven crises, we need better data. Retrieved from.
http://blogs.hbr.org/cs/2008/09/we_need_better_data_to_solve_t.html.
Roberts, N., Galluch, P.S., Dinger, M., Grover, V., 2012. Absorptive capacity and
information systems research: review, synthesis, and directions for future research.
MIS Q. 36 (2), 625–648.
Roberts, S.W., 1959. Control chart tests based on geometric moving averages.
Technometrics 1 (3), 239–250.
Rungtusanatham, M., 2001. Beyond improved quality: the motivational effects of
statistical process control. J. Oper. Manag. 19, 653–673.
Shadish, W., Cook, T., Campbell, D., 2002. Experimental and Quasi-experimental Designs
for Generalized Causal Inference. Houghton Mifflin Co., Boston, MA.
Shin, B., 2003. An exploratory investigation of system success factors in data
warehousing. J. Assoc. Inf. Syst. 4 (1), 141–170.
Singh, R., Keil, M., Kasi, V., 2009. Identifying and overcoming the challenges of
implementing a project management office. Eur. J. Inf. Syst. 18 (5), 409–427.
Sparks, R., OkuGami, C., 2010a. Data quality: algorithims for automatic detection of
unusual measurements. In: Lenz, H.-J., Schmid, W., Wilrich, P.-T. (Eds.), Frontiers in
Statistical Quality Control, vol. 10. Springer, Heidelberg.
Sparks, R., OkuGami, C., 2010b, Aug.. Data quality: algorithims for automatic detection of
unusual measurements. In: Paper Presented at the 10th International Workshop on
Intelligent Statistical Process Control, Seattle, WA.
Tan, K.H., Zhan, Y., Ji, G., Ye, F., Chang, C., 2015. Harvesting big data to enhance supply
chain innovation capabilities: an analytic infrastructure based on deduction graph.
Int. J. Prod. Econ. 165, 223–233.
Venkatesh, V., Bala, H., 2008. Technology acceptance model 3 and a research agenda on
interventions. Decis. Sci. 39 (2), 273–315.
Venkatesh, V., Thong, J.Y.L., Xu, X., 2012. Consumer acceptance and use of information
technology: extending the unified theory of acceptance and use of technology. MIS Q.
36 (1), 157–178.
Voss, C., Tsikriktsis, N., Frohlich, M., 2002. Case research in operations management. Int.
J. Oper. Prod. Manag. 22 (2), 195–219.
Wamba, S.F., Akter, S., Edwards, A., Chopin, G., Gnanzou, D., 2015. How ‘big data’can
make big impact: findings from a systematic review and a longitudinal case study. Int.
J. Prod. Econ. 165, 234–246.
Wang, G., Gunasekaran, A., Ngai, E.W., Papadopoulos, T., 2016. Big data analytics in
logistics and supply chain management: certain investigations for research and
applications. Int. J. Prod. Econ. 176, 98–110.
Wang, R.Y., Kon, H.B., 1993. Towards total data quality management (TDQM). In:
Wang, R.Y. (Ed.), Information Technology in Action: Trends and Perspectives.
Prentice-Hall, Englewood Cliffs, NJ.
Wang, R.Y., Strong, D.M., 1996. Beyond accuracy: what data quality means to data
consumers. J. Manag. Inf. Syst. 12 (4), 5–33.
Warth, J., Kaiser, G., Kugler, M., 2011. The impact of data quality and analytical
capabilities on planning performance: insights from the automotive industry. In:
Paper Presented at the 10th International Conference on Wirtschaftsinformatik,
Zurich, Switzerland.
Weigel, F.K., Hazen, B.T., 2014. Technical proficiency for IS success. Comput. Hum.
Behav. 31, 27–36.
Weigel, F.K., Landrum, W.H., Hall, D.J., 2009. Human-technology adaptation fit theory
for healthcare. In: Paper Presented at the Twelfth Annual Conference of the Southern
Association for Information Systems (SAIS), Charleston, SC.
Weller, E.F., 2000. Practical applications of statistical process control [in software
development projects]. IEEE Softw. 17 (3), 48–55.
Wickham, H., 2014. Tidy data. J. Stat. Softw. 59 (10), 1–23.
Wigfield, A., Eccles, J.S., 2000. Expectancy-value theory of achievement motivation.
Contemp. Educ. Psychol. 25, 68–81.
Wixom, B.H., Watson, H.J., 2001. An empirical investigation of the factors affecting data
warehousing success. MIS Q. 25 (1), 17–41.
Wu, Z., Jiao, J., He, Z., 2009. A single control chart for monitoring the frequency and
magnitude of an event. Int. J. Prod. Econ. 119 (1), 24–33.
Yin, R.K., 2014. Case Study Research: Design and Methods. 2013. Sage Publications.