Professional Documents
Culture Documents
The DAMA Guide To The Data Management Body of Knowledge (DAMA-DMBOK Guide)
The DAMA Guide To The Data Management Body of Knowledge (DAMA-DMBOK Guide)
First Edition
Tables ................................................................................................................................. xv
Preface...............................................................................................................................xix
Acknowledgements ...........................................................................................................xxi
1 Introduction ..................................................................................................................... 1
1.1 Data: An Enterprise Asset ........................................................................................ 1
1.2 Data, Information, Knowledge ................................................................................. 2
1.3 The Data Lifecycle .................................................................................................... 3
1.4 The Data Management Function.............................................................................. 4
1.5 A Shared Responsibility ........................................................................................... 5
1.6 A Broad Scope ........................................................................................................... 6
1.7 An Emerging Profession ........................................................................................... 7
1.8 A Growing Body of Knowledge ................................................................................. 8
1.9 DAMA–The Data Management Association ............................................................ 8
1.10 Purpose of the DAMA-DMBOK Guide ................................................................... 9
1.11 Goals of the DAMA-DMBOK Guide ....................................................................... 9
1.12 Audiences of the DAMA-DMBOK Guide ...............................................................10
1.13 Using the DAMA-DMBOK Guide ..........................................................................10
1.14 Other BOK Guides .................................................................................................11
1.15 The DAMA Dictionary of Data Management ........................................................11
1.16 The DAMA-DMBOK Functional Framework ........................................................11
1.17 Structure of the DAMA-DMBOK Guide ................................................................15
1.18 Recurring Themes ..................................................................................................16
Bibliography.....................................................................................................................379
Index ................................................................................................................................393
The book is an exhaustive compilation of every possible subject and issue that warrants
consideration in initiating and operating a Data Management responsibility in a
modern Enterprise. It is impressive in its comprehensiveness. It not only identifies the
goals and objectives of every Data Management issue and responsibility but it also
suggests the natural organizational participants and end results that should be
expected.
Having said that, I would not call this a “how to” book, although there is considerable
advice about what to do and what not to do relative to many practices surrounding Data
Management and particularly, Data Development. Still, it is not a technical treatise. It
does have a plethora of references to technical, “how to” books that would literally fill a
library for those who are interested in the technical details of Data Management.
I have been associated with DAMA from its very inception and have seen the evolution
of this Body of Knowledge, both in its practice as well as in this document over the
many years, which now approximates nearly 50!! The publication began as a non-trivial,
sorely needed compilation of articles and substantive facts about the little understood
subject of data management orchestrated by some folks from the DAMA Chicago
Chapter. It was unique at the time, as there was little substantive reference material
on the subject. It has grown to become this pragmatic practitioner’s handbook that
deserves a place on every Data Management professional’s bookshelf. There is a wealth
of information for the novice data beginner, but it is also invaluable to the old timer as a
check-list and validation of their understanding and responsibilities to ensure that
nothing “falls through the cracks”! It is impressive in its breadth and completeness.
1. To build a consensus …
2. To provide standard definitions …
3. To identify guiding principles …
4. To overview commonly accepted good practices …
5. To briefly identify common issues …
6. To clarify the scope and boundaries …
7. To guide readers to additional resources for further understanding.
Thank you to all the contributors and especially to the editors of this monumental
undertaking.
Thank you to all the present and future Data Managers who are blazing trails into the
complexities of the Information Age. This piece of work will be for you, a valuable
Guide.
John A. Zachman
Glendale, California
November, 2008
The DAMA-DMBOK Guide in its form has been in development for over four years and
is a complete overhaul of the earlier Guidelines mentioned above. Starting in a winter
storm in 2004, Deborah Henderson traveled to the Chicago DAMA Chapter meeting and
presented the first structured framework for a ‘body of knowledge’ for data
management. She asked for input and volunteers to bring this big vision into reality.
Mark Mosley stepped up as Editor—Development and began with a white paper
framework published as a free download on our DAMA website. That white paper went
through three major revisions to date. Updates on progress have been given at regular
intervals to the DAMA Board and the membership at meetings and conferences. The
interest and input in the revisions of the initial framework is truly global with over
3500 downloads in three languages from over 78 countries, and still climbing.
We quickly discovered that understanding our own language was a very important pre-
requisite. Development of a Glossary for the Guide started in 2007 and it soon became a
substantial work in itself; indeed it could stand by itself. DAMA responds strongly to
limited and dated definitions of data-centric terms that have been propagated since the
1960s and 1970s. Data Management is not ‘using an application’, or a synonym for
‘database administration’. The DAMA-DMBOK Dictionary, published separately in
2008, is the glossary for the Guide and is now a companion to the DAMA-DMBOK
Guide. The Dictionary has been received with enthusiasm. We heard back from many
users of the Dictionary, some of whom have decided to leverage it across their
corporations for its completeness and authority.
The DAMA-DMBOK Guide was written based on the Framework paper, and was
developed in a collaborative manner involving many primary and secondary
contributors and many draft peer review sessions as well as its own development
website. Over 40 reviewers participated in the draft review sessions. Assistant Editor
Susan Earley doggedly followed up, incorporating the comments into the second level
drafts. These drafts proceeded, chapter by chapter, into copy editing and manuscript
development.
should expect that it covers data management functions at a certain depth augmented
with chapter focused bibliographies of significant further reading.
Our work at DAMA International parallels the development of the data management
profession itself. The maturity of the profession is reflected in the emerged DAMA
certification program and the DAMA continuing education program. It is also reflected
in the DAMA involvement with other organizations and government bodies to influence
and partner their activities such as curriculum development for data management
professional education and international data management standards. The DAMA-
DMBOK is part of this overall integrated push to represent Data Management
profession world-wide.
Publication of the DAMA-DMBOK Guide has been the most pressing issue from our
data community. We hope it doesn’t disappoint that community. We will correct any
errors by omission or commission in future editions. Looking ahead, DAMA intends to
update the DAMA-DMBOK Guide by publishing regularly scheduled revisions. As it
evolves we will be more tightly coupling our certification, education and research and
industry programs.
The DAMA-DMBOK Guide is truly a journey not to be represented in just one edition.
As new perspectives develop in data management we will be there, updating and
challenging the best practices in our profession. Your comments, concerns and
contributions are welcome, as we are already planning our next edition. Please contact
the editors at info@dama.org.
Thanks, also, to the primary contributors who took the Framework vision and, within
the tightly defined format and on a volunteer basis, were able to deliver the wonderful
chapter material in-time and on-schedule, for which we are truly grateful.
We particularly wish to thank Mark Mosley for his sound theory, personal fortitude,
and endless hours spent, and Michael Brackett for his sound advice, production, and
manuscript miracle. Special thanks to John Zachman, Len Silverston and Ben Hu, our
DAMA Advisors, for their enthusiasm.
Finally, we want to recognize the families of all the volunteers on this project, who
sacrificed personal time with loved ones involved in this second non-paying job.
----------
The DAMA-DMBOK Guide resulted from the contributions of many DAMA members.
Without the contribution of these people, the DAMA-DMBOK Guide would not have
been possible. The profession owes a great deal of gratitude to these DAMA members
for their participation in a monumental piece of work.
DAMA International, the DAMA International Foundation, and the DAMA Chapter
Presidents’ Council sponsored the DAMA-DMBOK Guide project. Their vision, insight,
patience, and continued support lead to the establishment and continuation of this
project.
Four people contributed substantial time and effort pulling all aspects of development,
review, and production of the DAMA-DMBOK Guide together.
The DAMA-DMBOK Guide Editorial Board provided comments on the direction of the
DAMA-DMBOK Guide, reviewed chapters, and provided valuable insights, edits, and
enhancements to the manuscript. They represented the front line of professionals
contributing to the development of a data management profession. The Editorial Board
members are listed below in alphabetical order with their role and affiliation.
The DAMA-DMBOK Guide Planning Committee handled the vast multitude of details
necessary to bring the manuscript to publication. Many of these details were behind the
scenes, but were critical for production of the DAMA-DMBOK Guide. Without their
constant, daily, participation, the DAMA-DMBOK Guide would not exist today.
The contributing authors wrote the initial drafts for each chapter. These draft chapters
were circulated for review and returned to the author and the Assistant Editor for
enhancement. The contributing authors are the professionals contributing to the
development of a data management profession.
Many DAMA Members reviewed the draft chapters and provided significant comment
that led to improvement of those chapters. These reviewers are another wave of
professionals contributing to the development of a data management profession.
Many DAMA Members logged on to the DAMA-DMBOK Guide web site but did not
submit any comments as part of the review process.
The coeditors sincerely thank all of those DAMA members involved in the DAMA-
DMBOK Guide project. Their contributions were invaluable in creating the DAMA-
DMBOK Guide and for furthering the development of a data management profession.
We sincerely apologize for the unintentional omission of any person who provided
support for the DAMA-DMBOK Guide.
Money and people have long been considered to be enterprise assets. Assets are
resources with recognized value under the control of an individual or organization.
Enterprise assets help achieve the goals of the enterprise, and therefore need to be
thoughtfully managed. The capture and use of such assets are carefully controlled, and
investments in these assets are effectively leveraged to achieve enterprise objectives.
Data, and the information created from data, are now widely recognized as enterprise
assets.
No enterprise can be effective without high quality data. Today’s organizations rely on
their data assets to make more informed and more effective decisions. Market leaders
are leveraging their data assets by creating competitive advantages through greater
knowledge of their customers, innovative uses of information, and operational
efficiencies. Businesses are using data to provide better products and services, cut costs,
and control risks. Government agencies, educational institutions, and not-for-profit
organizations also need high quality data to guide their operational, tactical, and
strategic activities. As organizations need and increasingly depend on data, the
business value of data assets can be more clearly established.
Yet for many important decisions, we experience information gaps – the difference
between what we know and what we need to know to make an effective decision.
Information gaps represent enterprise liabilities with potentially profound impacts on
operational effectiveness and profitability.
Every enterprise needs to effectively manage its increasingly important data and
information resources. Through a partnership of business leadership and technical
expertise, the data management function can effectively provide and control data and
information assets.
The official or widely accepted meanings of commonly used terms also represent a
valuable enterprise resource, contributing to a shared understanding of meaningful
information. Data definitions are just some of the many different kinds of “data about
data” known as meta-data. Meta-data, including business data definitions, helps
establish the context of data, and so managing meta-data contributes directly to
improved information quality. Managing information assets includes the management
of data and its meta-data.
Knowledge management is the discipline that fosters organizational learning and the
management of intellectual capital as an enterprise resource. Both knowledge
management and data management are dependent on high quality data and
information. Knowledge management is a closely related discipline, although in this
document, knowledge management is considered beyond the scope of data management.
Data is the foundation of information, knowledge, and ultimately, wisdom and informed
action. Is data truth? Not necessarily! Data can be inaccurate, incomplete, out of date,
and misunderstood. For centuries, philosophers have asked, “What is truth?”, and the
answer remains elusive. On a practical level, truth is, to some extent, information of the
highest quality – data that is available, relevant, complete, accurate, consistent, timely,
usable, meaningful, and understood. Organizations that recognize the value of data can
take concrete, proactive steps to increase the quality of data and information.
Data is fluid. Data flows in and out of data stores, and is packaged for delivery in
information products. It is stored in structured formats–in databases, flat files, and
tagged electronic documents–and in many less structured formats–e-mail and other
electronic documents, paper documents, spreadsheets, reports, graphics, electronic
image files, and audio and video recordings. Typically, 80% of an organization’s data
assets reside in relatively unstructured formats.
Data has value only when it is actually used, or can be useful in the future. All data
lifecycle stages have associated costs and risks, but only the "use" stage adds business
value.
When effectively managed, the data lifecycle begins even before data acquisition, with
enterprise planning for data, specification of data, and enablement of data capture,
delivery, storage, and controls.
Projects accomplish the specification and enablement of data, and some of the planning
for data. The System Development Lifecycle (SDLC), shown in Figure 1.2, is not the
same as the data lifecycle. The SDLC describes the stages of a project, while the data
lifecycle describes the processes performed to manage data assets.
However, the two lifecycles are closely related because data planning, specification and
enablement activities are integral parts of the SDLC. Other SDLC activities are
operational or supervisory in nature.
All these terms are generally synonymous, but this document consistently refers to Data
Management.
Often the word “enterprise” is included in the function name to emphasize the
enterprise-wide focus of data management efforts, i.e., Enterprise Information
Management or Enterprise Data Management. Enterprise-wide data management is a
recommended best practice. However, data management may also be performed
effectively in a local context without an enterprise-wide mandate, although with less
business benefit.
The scope of the data management function and the scale of its implementation vary
widely with the size, means, and experience of organizations. The nature of data
management remains the same across organizations, even as implementation details
widely differ.
resources entrusted to them. Data stewards ensure data resources meet business needs
by ensuring the quality of data and its meta-data. Data stewards collaborate in
partnership with data management professionals to execute data stewardship activities
and responsibilities.
Data is the content moving through the information technology infrastructure and
application systems. Information technology captures, stores, processes, and provides
data. The IT infrastructure and application systems are the “pipes” through which data
flows. As technological change has exploded over the past fifty years, IT organizations
have traditionally focused primarily on maintaining a modern, effective hardware and
software infrastructure, and a robust application system portfolio based on that
infrastructure. Most IT organizations have been less focused on the structure, meaning,
and the quality of the data content flowing through the infrastructure and systems.
However, a growing number of IT executives and business leaders today recognize the
importance of data management and the need for effective Data Management Services
organizations.
· Data Governance: Planning, supervision and control over data management and
use.
Data
Architecture
Data Management
Data
Quality
Development
Management
Database
Meta-data Operations
Management Management
Data
Governance
Data
Warehousing Reference &
& Business Master Data
Intelligence Management
Management
Standards and recognized best practices can improve the effectiveness of data stewards
and data management professionals. Moreover, standards help us communicate with
our teammates, managers, and executives. Executives especially need to fully
understand and embrace fundamental data management concepts in order to effectively
fund, staff and support the data management function.
The DAMA Foundation is the research and education affiliate of DAMA International,
dedicated to developing the data management profession and promoting advancement of
concepts and practices to manage data and information as enterprise assets.
The joint mission of DAMA International and the DAMA Foundation, collectively
known as DAMA, is to Lead the data management profession toward maturity. DAMA
promotes the understanding, development, and practice of managing data, information,
and knowledge as key enterprise assets, independent of any specific vendor, technology,
and method.
DAMA International seeks to mature the data management profession in several ways.
A few of these efforts include:
No single book can describe the entire body of knowledge. The DAMA-DMBOK Guide
does not attempt to be an encyclopedia of data management or the full-fledged discourse
on all things related to data management. Instead, this guide briefly introduces
concepts and identifies data management goals, functions and activities, primary
deliverables, roles, principles, technology and organizational / cultural issues. It briefly
describes commonly accepted good practices along with significant alternative
approaches.
· Helping standardize terms and their meanings within the data management
community.
· Helping data management professionals prepare for CDMP and CBIP exams.
The primary model for the DAMA-DMBOK Guide is A Guide to the Project
Management Body of Knowledge (PMBOK® Guide), published by the Project
Management Institute (PMI®). PMI® is a professional organization for project managers.
Among its many services, PMI® conducts the Project Management Professional (PMP)
certification program.
· Goals and Principles: The directional business goals of each function and the
fundamental principles that guide performance of each function.
• Architecture • Acquisition
• Integration Data • Recovery
Governance Database • Tuning
• Control Meta-data
Operations • Retention
• Delivery Management • Strategy
Management • Purging
• Organization & Roles
• Policies & Standards
• Projects & Services
Document & Content • Issues Data Security
Management • Valuation Management
• Standards
• Acquisition & Storage
• Classification
• Backup & Recovery Data • Administration
• Content Mgmt. Warehousing Reference & • Authentication
• Retrieval & Business Master Data • Auditing
• Retention Intelligence Management
Management
• Architecture • External Codes
• Implementation • Internal Codes
• Training & Support • Customer Data
• Monitoring & Tuning • Product Data
• Dimension Mgmt
· Activities: Each function is composed of lower level activities. Some activities are
grouped into sub-activities. Activities are further decomposed into tasks and
steps.
· Roles and Responsibilities: The business and IT roles involved in performing and
supervising the function, and the specific responsibilities of each role in that
function. Many roles will participate in multiple functions.
· Practices and Techniques: Common and popular methods and procedures used to
perform the processes and produce the deliverables. Practices and Techniques
may also include common conventions, best practice recommendations, and
alternative approaches without elaboration.
Chapters 3 through 12 each address one of the ten data management functions. One or
more subject matter experts contributed to each chapter. Each chapter includes: