Adelman Datastrateg

Data Strategy in Practice
Sid Adelman &

Associates
sidadelman@aol.com
818.783.9634
Data Strategy
 Module 1 – Introduction to Data Strategy
 Module 2 – Data Quality
 Module 3 – Metadata
 Module 4 – Organization, Roles &Responsibilities
 Module 5 – Security & Privacy
 Module 6 – Business Intelligence
 Module 7 – Information Integration
 Module 8 – Software/Products
 Module 9 – Performance & Measurement
Copyright Sid Adelman, 2007 2

Module 1 – Introduction to Data
Strategy
 Components of a data strategy
 Why have a data strategy
 Do these problems exist in your organization?
 Gain control
 Support the IT strategy
 Data in the Dark Ages
 Enlightened data strategy
 Critical success factors
 How to implement a data strategy
 Best Practices

Components of a Data Strategy +
 RDBMS - Relational Database Management
System
 Data Quality
 Metadata
 Performance
 Data Distribution
 Organization
 Data Ownership

 Security and Privacy
 Total Cost of Ownership
 Subject area databases
 Data modeling
 Data sharing
 Business Intelligence
 Information integration

 Legacy/operational data
 Standards
 Data migration
 Application packages
 Software/products
 Personal/departmental databases

Components of a Data Strategy
 Categorization of data
 Communicating and selling the data
strategy
 Measurement

Why Have a Data Strategy
 Capitalize on the data asset
 Support the IT Strategy
 Gain control

Do these problems exist in your
organization? +
 Uncontrolled redundant data
 Data not easily accessible by the user
 Lack of knowledge of available data
 Poor data quality
 Each new application designs, builds and
populates it own data base
 Inconsistent reports

Do these problems exist in your
organization?
 Private databases
 No central meta data repository
 Management unclear on the importance of
data
 No responsibility for data
 Data standards non existent, not understood
or not followed

Gain Control
 Consistent security implementation
 Understand, define and assign ownership
 Understand, define and assign stewardship
 Minimize redundancy
 Inventory data
 Develop consistent terminology

Support the IT Strategy
 Provide departments, projects and personnel with
guidelines for storing and accessing data
 Minimize the number of RDBMSs
 Establish, disseminate and maintain standards for
shared data resources
 Deliver a high level of service
– performance
– Availability
– response time
– responsiveness to user requests

Data in the Dark Ages
 Data is kept locked by each application or
department
 Users do not trust the data
 Data is not well understood either by users
or by IT
 Data is difficult to access
 Senior Management does not understand
the value of data

Enlightened Organization
 Data is shared
 Users trust the accuracy of the data
 Data is inventoried and terminology is clear
 Data is easily accessed by IT and by the users
 Senior Management view data as an asset that
is critical to the organization and to decision
making

Critical Success Factors
 Data Strategy supports IT plans
 Quality data
 Support of legacy data
 Support of development efforts
 Infrastructure
– Organization
– Skills
– Tools
 Achieve short-term successes

How to Implement a Data
Strategy
 Data environment assessment
 Establish a target data environment
 Develop an implementation plan
 Sell Data Strategy within the organization
 Evaluate progress and justify your
existence
 Revisit the plan

Best Practices
 Don’t get into the details too soon
 Don’t be seen as a theorist -- your actions
must be pragmatic
 Don’t lead with long-term deliverables
 Don’t commit more than you can deliver
 Avoid unproven technology

Module 1 Workshop
Assessment of Existing Organization

Module 2– Data Quality
 Management Support
 Evaluation/Diagnosis
 Timeliness
 ETL Validation
 Prioritization - Which Data to Clean First
 Cost of Cleansing
 Responsibility for Data Quality

Management Support
 Management awareness of importance of
data quality
 Cost justification of data quality initiative
 Ongoing commitment
 Finding a business management sponsor

Evaluation/Diagnosis
 Which source data is most correct

 Valid values (domains)
 Business rules
 Data types (e.g., hex, packed decimal)
 Completeness
 Inappropriate defaults
 Fields used for multiple purposes
 Accuracy
 Quality of historical data

Data Timeliness
 Currency of data, e.g., last Friday
 Frequency of update, e.g., daily, weekly,
monthly, quarterly
 User awareness – how will the users know?

ETL Validation
 Validation of ETL process
 Tie-outs
– Number of records
– Dollar matching
– Quantitative matching
 Automatic versus manual checking
 Referential integrity?

Triage - Prioritization
 Which data to clean
 Justification for cleansing
 Ease of cleansing
 Possibility of cleansing
 Political support for cleansing

Cost of Cleansing
 Automatic versus manual
– Tools to perform automatic cleansing
– Effort to support use of tools
 Use of defaults
 Knowledge/experience of those performing
manual cleansing

Responsibility for Data Quality
 “It’s not enough to say that data quality is
everyone’s responsibility.”
 Data Quality Administrator
 Ongoing commitment
 Data ownership responsibility
 Operational versus data warehouse
responsibility

Data Quality – Best Practices
 Inventory the quality of your data
 Sell the importance of data quality to
management
 Assign data quality responsibility
 Triage the cleansing process

Module 2 Workshop
Data Quality

Module 3– Metadata
 Management Support
 Meta Data as the Keystone
 Which Metadata to Capture
 Responsibility for Capture
 Responsibility for Maintenance
 Business Metadata
 Technical Metadata
 How will Metadata be Used
 Data Inventory

Metadata – Management Support
 IT and the Business
 Management understanding of the
importance of metadata
 Impact on project schedules
 Long term benefit of metadata
 Importance for operational and data
warehouse

Metadata as the Keystone
 Single version of the truth
 It’s the inventory of information
 Tears down dysfunctional information
fiefdoms
 Opportunities to reduce redundancy
 Opportunities for integration

Which Metadata to Capture
 Don’t boil the ocean
 What meta data is valuable
 Ease and cost of capture
 Political issues relating to capture

Responsibility for Capturing
Metadata
 Incentive for capturing
 Management direction
 Automatic and manual

Responsibility for Metadata
Maintenance
 Where does Metadata Repository
maintenance report?
 Why is maintenance important?
 Long-term commitment

Business Metadata
 Business definitions
 Source of data
 How data was derived (algorithms)
 Lineage (data genealogy)
 Timeliness
 Security
 Ownership
 Quality

Technical Metadata
 Field name
 Database
 Data type
 Source
 Length

How Will Metadata be Captured
 Data modeling tools
 ETL tool
 Access and analysis tool
 Metadata Repository tool
 Data dictionary
 Copybooks
 Home grown application

How Will Metadata be Used
 Business
– Understanding the data
– Understanding the meaning of results
– Avoiding incorrect conclusions
 IT
– Research
– Impact analysis
– Tool interchange

Inventory
 Where is the data?
 How and where is it used?
 Quality of data
 Redundancy
 Ownership
 Documentation

Metadata – Best Practices
 Determine which meta data to capture and
use
 Determine how the tools will capture and
use metadata
 Sell management on the importance
 Assign metadata responsibility

Module 3 Workshop
Metadata

Module 4 Organization – Data-
related Roles & Responsibilities
 Database Administrator
 Data Administrator
 Data Quality Administrator
 Security
 Architect
 Data ownership

Database Administrator
 Database design
 Backup and recovery
 Reorganization
 Monitoring
 Tuning
 Index creation

Data Administrator
 Data modeling
 Source data evaluation
 Enterprise data integration
 Data quality analysis
 Metadata responsibility

Data Quality Administrator
 Uncovering data quality problems
 Communicating data quality problems
 ETL verification
 Responsibility for some cleansing

Security
 Responsibility for who can do what to the

data
– Data access
– Data create/update/delete
 Working with those administering the tools
that have security capabilities

Architect
 Knowing what the enterprise needs
 Evaluating technical options
 Developing an appropriate architecture
 Selling the architecture

Data Ownership +
 Creation
 Access
 Determine requirements for performance
 Determine requirements for availability
 Determine historical requirements

Creation
 Data Entry process
– Training
– Incentives for quality
 Quality of data
 Data edits

Access
 Need to know
 Opt in/Opt out
 Level of granularity
 By department
 By role
 External access by people outside the
organization

Performance Requirements
 Response time
 What is excellent response time worth?
 Timeliness

Availability Requirements
 How many hours and days does the system
need to be available?
 What is the availability requirement during
scheduled hours?

Historical Requirements
 How far back to keep the data
 How detailed does old data need to be?
 Impact of code changes and organizational
changes over time

Organization – Best Practices
 Establish the appropriate organization for
your enterprise
 Enumerate roles and responsibilities
 Gain concurrence for roles and
responsibilities
– Management
– Those performing the functions

Module 4 Workshop
Organization

Module 5 Security & Privacy
 Categorization for security
 Responsibility for determining
 Mechanism for establishing procedures
 Security audit
 Regulatory issues
 Data sharing

Categorization for
Security/Privacy
 Does all data have the same
security/privacy requirements?
 Who determines security/privacy
requirements of data?
 What are the regulatory requirements for
security and privacy?
 Does your organization have a Security
Office? What authority do they have?

Responsibility
 Security Office
 Internal auditors?
 Data Owners
 Responsibility for administering
 Testing security and privacy

Mechanism for Establishing
Procedures
 Security requirements
– Internal
– Regulatory
 Tools that implement security
 Communicating security requirements to
those who implement

Security Audit
 Validating procedures
 Validating training
 Testing and probing
 Recommending mitigation
 Frequency of audits

Regulatory Issues
 Health Care – HIPPA
 Finance
 Brokerage - SEC
 Insurance
 Media – FCC

Data Sharing
 Inhibitors
 Motivation/incentives to share
 Management directives on sharing

Inhibitors
 Power
 Fear of others
 Fear of boss micromanaging

Motivation/incentives to share
 Are there any?

Management Direction on
Sharing
 Direction to share must come from the
CEO
– Need to know
– Reason for withholding access must be
documented
– Access only given when directed

Security & Privacy – Best
Practices
 Raise the consciousness of security and
privacy requirements
 Connect with your Security Office
 Determine security capabilities of tools
 Assign responsibilities
 Test and validate

Module 5 Workshop
Security & Privacy

Module 6 Business Intelligence
 Goals and Objectives
 Architecture
 Data Mining
 Tools
 Methodology

Goals and Objectives
 Why have a data warehouse?
 Have goals and objectives been identified
 Have they been communicated?
 Are they measured post-implementation

Architecture
 Platform
 Tools/products
 How the data flows

Data Mining
 Discovery versus hypothesis testing
 Different tools
 Different people mining the data

Tools
 RDBMS
 Data Modeling
 ETL
 Access and Analysis
 Data quality (Cleansing)
 Measurement

Methodology
 Spiral versus waterfall
 Phasing more appropriate
 Tasks more difficult to estimate

Business Intelligence – Best
Practices
 Set goals and objectives
 Set expectations early and often
 Establish cost justification
 Find a terrific sponsor

Module 6 Workshop
Business Intelligence

Module 7 Information Integration
 Integrating business data
 Data redundancy
 Different RDBMSs and their impact
 Data migration

Integrating Business Data
 Understanding the customer
 ERPs
 Supply chain

Data Redundancy
 Goal to reduce data redundancy?
 Inconsistent data
 Single version of the truth
 Cost of data redundancy

Different RDBMSs & Their
Impact
 More interface programs
 Less depth in DBA pool
 More product expense
 Integration problems
 Less optimizer capability

Data Migration +
 Should data be dropped?
 Should data be converted?
 Should data be integrated/consolidated?

Should Data be Dropped?
 Is it even being used?
 What’s the cost of maintaining this data?
 Could another database be used in its
place?
 Any political issues?
 Any regulatory issues?

Should Data be Migrated?
 Can we consolidate RDBMSs?
 What is the cost of migration?
 What is the impact on other systems?

Should Data be
Integrated/Consolidated?
 Why do we want to integrate/consolidate?
 Costs of integration/consolidation
 Savings of integration/consolidation
 Political issues
 Regulatory issues

Information Integration – Best
Practices
 Determine information integration benefits
and costs
 Sell information integration to management
 Establish and execute priorities

Module 7 Workshop
Information Integration

Module 8 Software/Products
 RDBMS
 Tools/utilities
 Organization standards for products
 Criteria for selection
 Responsibility for Selection
 Single vendor/best of breed
 Deals/Negotiation
 Relationship with vendors
 Application packages

RDBMS
 Which RDBMS is the standard
 Relation to platform
 What applications is it being used for

RDBMS Choices
 IBM (DB2, IMS, Informix)
 Microsoft (SQL Server)
 Oracle
 Sybase
 Teradata

Why standardize the RDBMS?
 Minimize the number of RDBMSs
 Less training required
 More leverage on RDBMS vendor
 Flexible assignments
 Fewer interface problems
 Fewer interface programs

Relation to platform
 RDBMS performance impacted by
platform
 Platform may dictate (or strongly
recommend) RDBMS choice
 Which decision comes first?

What application is RDBMS
being used for
 Operational/OLTP
 Data Warehouse/Business Intelligence

Tools/Utilities
 Platform dependent
 RDBMS dependent
 Expensive
 33% on the shelf
 Lots of product duplication
 Necessary?

Organization Standards for
Products
 Who sets standards?
 Are the standards known?
 Are they standards or guidelines?
 Who can give dispensation?

Criteria for Selection
 Need
 Cost
 Vendor
– Support
– Reputation
– Financial stability

Responsibility for Selection
 Technical evaluators
 Strategic architect
 Management

Single Vendor vs Best of Breed
 Single vendor
– Possibly a better relationship
– Leverage
– Not always the best products
– Products should all work together
 Best-of-breed
– Need to integrate yourself
– Finger pointing when problems
– Potential incompatibilities

Deals/Negotiations
 Have someone else negotiate
 Don’t let vendor know you have chosen
them before you negotiate
 www.dobetterdeals.com (Joe Auer –
ComputerWorld)

Relationship with Vendors
 Partnerships
 Money Issues
 Support
 Conferences
 Being a reference

Databases Required by the
Application Packages
 Packages do not support all RDBMSs
 Packages do not support all RDBMSs
equally well
 Does preferred RDBMS violate
organization standard
 Are support personnel (DBAs) available?

Impact of Package
 Machine Requirements
 Performance
 Availability

Software – Best Practices
 Determine real requirements
 Establish software standards
 Make use of existing software whenever
possible
 Talk to organizations who are using the
products

Module 8 Workshop
Software/Products

Module 9 – Performance and
Measurement
 Categorization for performance
 Capacity Planning
 Monitoring/Measuring
 Service Level Agreements
 Tuning
 Roles and Responsibilities
 Reporting performance

Categorization for Performance
 How good does response time need to be?
 How does it differ from application to
application?
 What is the cost-benefit of excellent
response time?
 Were performance considerations included
in the architecture?

Categorization for Availability
 Scheduled hours (24 X 7, 18 X 6,…)
 Availability during scheduled hours
 How does it differ from system to system?
 Is excellent availability cost justified?
 Was availability included in the
architecture?

Capacity Planning
 Database size
 Number of users
 Number of transactions
 Number of queries/reports
 Time and day of usage
 Complexity of transactions/queries/reports
 Proactive response to capacity increase

Monitoring/Measuring
 Response time
 Resource utilization (CPU, disk access,
network)
 Who is using the system
 When is the system being used
 Chargebacks

Service Level Agreements
 Response time
 Availability
– Schedule hours (hours/day, days/week)
– Availability during scheduled hours
 Timeliness of data
 Response to problems
 Response to new requests

Tuning
 Awareness of problems – measurement
tools and responsibilities
 Tuning capability of platform, RDBMS,
tools
 Responsibility for tuning

Roles and Responsibilities
 DBA - RDBMS
 Application performance
 Systems programmer – operating system
 System Architect
 Capacity Planner
 Performance testing

Reporting performance
 IT
– Who needs to take action
– Who needs to see reports/alerts
 Business
– Matching project agreements
– Expectations

Measurement Tools
 Performance
 Usage
 Resource utilization
 Network

Measurement Usage
 What do you do with the performance
measurement information?

Reporting to Management
 High level (not detailed)
 Problems, aberrations
 Frequency
 Form (tables, charts, graphs)

Service Level Agreements
 Response time
 Availability
 Who establishes agreements?
 What’s realistic?
 Incentives to meet SLAs

Performance & Measurement –
Best Practices
 Determine what is advantageous to measure
 Assign responsibilities
 Designate tools for measurement
 Report metrics to management

Module 9 Workshop
Performance & Measurement

Overall Data Strategy Best Practices
 Don’t get into the details too soon
 Don’t be seen as a theorist -- your actions
must be pragmatic
 Don’t lead with long-term deliverables
 Don’t commit more than you can deliver
 Avoid unproven technology

How to Implement a Data
Strategy
 Conduct a data environment assessment
 Establish a target data environment
 Develop an implementation plan
 Sell Data Strategy within the organization
 Evaluate progress and justify your
existence
 Revisit the plan

Summary
 Pitch the importance of a data strategy to
your CIO and CTO
 Ask to either lead the effort or to be a
permanent member of the team

Adelman Datastrateg

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Adelman Datastrateg

Uploaded by

Copyright:

Available Formats

Data Strategy in Practice

Sid Adelman &

Copyright Sid Adelman, 2007 2

Copyright Sid Adelman, 2007 3

Copyright Sid Adelman, 2007 4

Copyright Sid Adelman, 2007 5

Copyright Sid Adelman, 2007 6

Copyright Sid Adelman, 2007 7

Copyright Sid Adelman, 2007 8

Copyright Sid Adelman, 2007 9

Copyright Sid Adelman, 2007 10

Copyright Sid Adelman, 2007 11

Copyright Sid Adelman, 2007 12

Copyright Sid Adelman, 2007 13

Copyright Sid Adelman, 2007 14

Copyright Sid Adelman, 2007 15

Copyright Sid Adelman, 2007 16

Copyright Sid Adelman, 2007 17

Copyright Sid Adelman, 2007 18

Copyright Sid Adelman, 2007 19

Copyright Sid Adelman, 2007 20

 Which source data is most correct

Copyright Sid Adelman, 2007 21

Copyright Sid Adelman, 2007 22

Copyright Sid Adelman, 2007 23

Copyright Sid Adelman, 2007 24

Copyright Sid Adelman, 2007 25

Copyright Sid Adelman, 2007 26

Copyright Sid Adelman, 2007 27

Copyright Sid Adelman, 2007 28

Copyright Sid Adelman, 2007 29

Copyright Sid Adelman, 2007 30

Copyright Sid Adelman, 2007 31

Copyright Sid Adelman, 2007 32

Copyright Sid Adelman, 2007 33

Copyright Sid Adelman, 2007 34

Copyright Sid Adelman, 2007 35

Copyright Sid Adelman, 2007 36

Copyright Sid Adelman, 2007 37

Copyright Sid Adelman, 2007 38

Copyright Sid Adelman, 2007 39

Copyright Sid Adelman, 2007 40

Copyright Sid Adelman, 2007 41

Copyright Sid Adelman, 2007 42

Copyright Sid Adelman, 2007 43

Copyright Sid Adelman, 2007 44

Copyright Sid Adelman, 2007 45

 Responsibility for who can do what to the

Copyright Sid Adelman, 2007 46

Copyright Sid Adelman, 2007 47

Copyright Sid Adelman, 2007 48

Copyright Sid Adelman, 2007 49

Copyright Sid Adelman, 2007 50

Copyright Sid Adelman, 2007 51

Copyright Sid Adelman, 2007 52

Copyright Sid Adelman, 2007 53

Copyright Sid Adelman, 2007 54

Copyright Sid Adelman, 2007 55

Copyright Sid Adelman, 2007 56

Copyright Sid Adelman, 2007 57

Copyright Sid Adelman, 2007 58

Copyright Sid Adelman, 2007 59

Copyright Sid Adelman, 2007 60