OC - Module 1 - Intro To BDA 021312

Module 1 – Introduction to Big Data Analytics
2
EMC PROVEN PROFESSIONAL
Copyright © 2012 EMC Corporation. All Rights Reserved. Module 1: Introduction to BDA 1
Module 1: Introduction to Big Data Analytics
Upon completion of this module, you should be able to:

• Define big data
• Identify four business drivers for advanced analytics
• Distinguish the techniques for Business Intelligence from Data
Science
• Describe the role of the Data Scientist within the new big data
ecosystem
• Cite at least three illustrative examples of big data opportunities
2
Lesson 1: Big Data Overview

During this lesson the following topics are covered:
• Definition of big data
• Big data characteristics and considerations
• Unstructured data fueling big data analytics
• Analyst perspective on Data Repositories
2
Introduction to Big Data Analytics
Your Thoughts?
What is Big Data?
What makes data, “Big” Data?
2
Big Data Defined
• “Big Data” is data whose scale, distribution, diversity,
and/or timeliness require the use of new technical
architectures and analytics to enable insights that unlock
new sources of business value.
 Requires new data architectures, analytic sandboxes
 New tools
 New analytical methods
 Integrating multiple skills into new role of data scientist
• Organizations are deriving business benefit from analyzing

ever larger and more complex data sets that increasingly
require real-time or near-real time capabilities
Source: McKinsey May 2011 article Big Data: The next frontier for innovation, competition, and productivity
2
Copyright © 2011
2012 EMC Corporation. All Rights Reserved. Module 1: Introduction to BDA
Key Characteristics of Big Data
1. Data Volume
 44x increase from 2010 to 2020
(1.2zettabytes to 35.2zb)
2. Processing Complexity
 Changing data structures
 Use cases warranting additional transformations and
analytical techniques
3. Data Structure
 Greater variety of data structures to mine and analyze
2
Copyright © 2011
2012 EMC Corporation. All Rights Reserved. Module 1: Introduction to BDA
Big Data Characteristics: Data Structures
Data Growth is Increasingly Unstructured
Str • Data containing a defined data type, format, structure
uct • Example: Transaction data and OLAP
ur
ed • Textual data files with a discernable pattern,
More Structured
Semi- enabling parsing
Structured • Example: XML data files that are self

describing and defined by an xml schema
• Textual data with erratic data formats, can

be formatted with effort, tools, and time
“Quasi” Structured • Example: Web clickstream data that

may contain some inconsistencies in data
values and formats
• Data that has no inherent
structure and is usually stored
as different types of files.
Unstructured
• Example: Text documents,
PDFs, images and video
2
Four Main Types of Data Structures
Structured Data Quasi-Structured Data
Semi-Structured Data
View  Source
http://www.google.com/
#hl=en&sugexp=kjrmc&cp=8&gs_id=2m&xhr=t&q=data+scientist&pq=big+data&pf=p&sclien
t=psyb&source=hp&pbx=1&oq=data+sci&aq=0&aqi=g4&aql=f&gs_sm=&gs_upl=&bav=on.2,
or.r_gc.r_pw.,cf.osb&fp=d566e0fbd09c8604&biw=1382&bih=651
Unstructured Data
The Red Wheelbarrow, by
William Carlos Williams
2
For each data type shown, answer these questions:
1)What type of analytics are performed on these data?

2)Who analyzes this kind of data?
3)What types of data repositories are suited for each, or
requirements you may have for storing and cataloguing this kind
of data?
4)Who consumes the data?
5)Who manages and owns the data?
2
Copyright © 2012 EMC Corporation. All Rights Reserved. Module #: Module Name 9
Data Repositories, An Analyst Perspective
Data Islands Data Warehouses Analytic Sandbox
“Spreadmarts”
Centralized data containers Data assets gathered from multiple
Isolated data marts in a purpose-built space sources and technologies for analysis
• Spreadsheets and low- • Supports BI and reporting, but • Enables high performance analytics
volume DB‘s for restricts robust analyses using in-db processing
recordkeeping • Analyst dependent on IT & • Reduces costs associated with data
• Analyst dependent on DBAs for data access and replication into "shadow" file
data extracts schema changes systems
• Analysts must spend significant • “Analyst-owned” rather than “DBA
time to get extracts from owned”
multiple sources
2
Introduction to Big Data Analytics: Mini-Case Study
Yoyodyne Bank Scenario
• Evolving from small community bank to a global bank
• Needs to move away from its legacy mainframes to an environment that
supports more robust analytics
• Growing through mergers and acquisitions
• Subject to many new regulatory requirements
• Increasing customer base and increased product offerings
Your Thoughts?
Discussion Questions
1. Discuss how the bank’s data would change under these circumstances.
2. How are their needs changing with these business changes?
3. What do you need to consider from an analyst point of view? What are
some things to consider implementing as the bank grows?
2
Lesson 1: Summary
During this lesson the following topics were covered:
• Definition of big data
• Big data characteristics and considerations
• Unstructured data fueling big data analytics
• Analyst perspective on Data Repositories
2
Lesson 2: State of the Practice in Analytics

• Business drivers for analytics
• Current analytical architecture
• Business intelligence vs. data science
• Drivers of big data and new big data ecosystem
2
Business Drivers for Analytics
Current Business Problems Provide Opportunities for Organizations to
Become More Analytical & Data Driven
Driver Examples
1
Desire to optimize business
Sales, pricing, profitability, efficiency
operations
2
Desire to identify business risk Customer churn, fraud, default
3
Predict new business opportunities Upsell, cross-sell, best new customer prospects
4
Comply with laws or regulatory
Anti-Money Laundering, Fair Lending, Basel II
requirements
2
Copyright © 2011
2012 EMC Corporation. All Rights Reserved. Module 1: Introduction to BDA 15
Analytical Approaches for Meeting Business Drivers
Business Intelligence vs. Data Science
Predictive Analytics & Data Mining
(Data Science)
Typical • Optimization, predictive modeling,
Techniques & forecasting, statistical analysis
Data Types • Structured/unstructured data, many types
of sources, very large data sets
High Common • What if…..?

Questions • What’s the optimal scenario for our
business ?
• What will happen next? What if these
trends continue? Why is this happening?
Data
Science Business Intelligence
Typical • Standard and ad hoc reporting,
BUSINESS Techniques & dashboards, alerts, queries, details on
VALUE Data Types demand
Business • Structured data, traditional sources,
Intelligence manageable data sets
Common • What happened last quarter?
Questions • How many did we sell?
• Where is the problem? In which
situations?
Low
Past TIME Future
2
A Typical Analytical Architecture
1 Data
Sources
Non-Agile Models
2 Departmental
“Spread
Marts”
Warehouse
Enterprise 4
Departmental Applications
Warehouse
3 Prioritized
Operational
Processes
Static schemas
accrete over time Reporting Siloed
Analytics
Non-Prioritized Data Provisioning
Errant data & marts
2
Copyright © 2011
Implications of Typical Architecture for Data Science
• High-value data is hard to reach and leverage

• Predictive analytics & data mining activities are last
in line for data
 Queued after prioritized operational processes
• Data is moving in batches from EDW to local Slow
“time-to-insight”
analytical tools
&
 In-memory analytics (such as R, SAS, SPSS, Excel)
reduced
 Sampling can skew model accuracy business impact
• Isolated, ad hoc analytic projects, rather than
centrally-managed harnessing of analytics
 Non-standardized initiatives
 Frequently, not aligned with corporate business goals
2
Opportunities for a New Approach to Analytics
New Applications Driving Data Volume
MEASURED IN MEASURED IN WILL BE MEASURED IN

LARGE TERABYTES PETABYTES EXABYTES
1TB = 1,000GB 1PB = 1,000TB 1EB = 1,000PB
VOLUME OF INFORMATION
SMALL
1990’s 2000’s 2010’s

(RDBMS & DATA (CONTENT & DIGITAL ASSET (NO-SQL & KEY/VALUE)
WAREHOUSE) MANAGEMENT)
2
Opportunities for a New Approach to Analytics
Big Data Ecosystem
1
Data
Devices
Individual
Analytic Medical Information

Services Brokers Advertising Marketers Employers
Law
Enforcement
Government Internet
2
Data
Websites
3
Collectors Data
Aggregators
Data
Users/Buyers
Catalog
4 Co-Ops
Phone/TV Retail
Media
Private
Media Credit List Investigators
Archives Bureaus Financial Brokers Delivery /Lawyers
Banks Service
Government
2
Copyright © 2011
Considerations for Big Data Analytics
Criteria for Big Data Projects New Analytic Architecture
Analytic Sandbox
Data assets gathered from multiple sources
1. Speed of decision making and technologies for analysis
2. Throughput
• Enables high performance analytics
using in-db processing
3. Analysis flexibility • Reduces costs associated with data
replication into "shadow" file
systems
• “Analyst-owned” rather than “DBA
owned”
2
State of the Practice in Analytics: Mini-Case Study
Big Data Enabled Loan Processing at Yoyodyne
Traditional Big Data Enabled

Underwriting Underwriting Your Thoughts?
Risk Level Risk Level
Underwriting Risk
e t al
om on en y ing ais
c
In ati ym o r c or tory pr
ic plo is
t
t S is Ap
e rif Em H edi d H
V Cr An
TRADITIONAL DATA LEVERAGED BIG DATA LEVERAGED
2
Copyright © 2011
Lesson 2: Summary
• Business drivers for analytics
• Current analytical architecture
• Business intelligence vs. data science
• Drivers of big data and new big data ecosystem
2
Lesson 3: The Data Scientist
• Key Roles of the New Big Data Ecosystem
• Profile of a Data Scientist
2
Skills Needed In the New Data Ecosystem
Your Thoughts?
• What new skill sets do you need to take advantage of

the big data sets in the loan processing improvement
case study?
• Do most large organizations have people with these

skill sets?
• If so, who are they?
2
Three Key Roles of the New Data Ecosystem
Data Scientists Role Role Description

Projected U.S. People with advanced training in
talent gap: Deep Analytical quantitative disciplines, such as
140,000 to Talent mathematics, statistics, and machine
190,000 learning.
People with a basic knowledge of statistics
Data Savvy
and/or machine learning, who can define
Professionals
Analysts & Data key questions that can be answered using
Savvy Managers advanced analytics
Projected U.S. People providing technical expertise to
talent gap: 1.5 Technology & Data support analytical projects. Skills sets
million Enablers including computer programming and
database administration
Note: Figures above reflect a projected talent gap in US in 2018, as shown in McKinsey May 2011 article Big Data: The next frontier for innovation,
competition, and productivity
2
Copyright © 2011
Roles Needed for Analytical Projects
Data Scientist Key Activities
Data Scientists
Key Activities Data Data Bl LOB
Engineers Analyst Analyst
• Reframe business User
challenges as analytics
challenges Analytic Productivity Platform
• Design, implement and

Tools & Services
deploy statistical models
and data mining Data
Data Access & Query Platform
techniques on big data Admin
• Create insights that lead

to actionable
recommendations Cloud Infrastructure
2
Copyright © 2011
2012 EMC Corporation. All Rights Reserved. Module 1: Intro duction to BDA 30
Profile of a Data Scientist
Quantitative
Curious &
Technical
Creative
Skeptical Communicative
& Collaborative
2
Lesson 3: Summary
• Key Roles of the New Big Data Ecosystem
• Profile of a Data Scientist
2
Lesson 4: Big Data Analytics in Industry Verticals
During this lesson we cover the following representative examples:
• Health Care
• Public Services
• Life Sciences
• IT Infrastructure
• Online Services
2
Big Data Analytics: Industry Examples
1
Health Care
• Reducing Cost of Care Medical
2 Public Services Government Internet
• Preventing Pandemics
3 Life Sciences Data
Collectors
• Genomic Mapping
4 IT Infrastructure
• Unstructured Data Analysis
Phone/TV Retail
5 Online Services
Financial
• Social Media for Professionals
2
1
Big Data Analytics: Healthcare
• Poor police response and problems with medical care, triggered

Situation by shooting of a Rutgers student
• The event drove local doctor to map crime data and examine
local health care
• Dr. Jeffrey Brenner generated his own crime maps from medical
Use of Big Data billing records of 3 hospitals
• City hospitals & ER’s provided expensive care, low quality care
• Reduced hospital costs by 56% by realizing that 80% of city’s
Key medical costs came from 13% of its residents, mainly low-
Outcomes income or elderly
• Now offers preventative care over the phone or through home
visits
2
2
Big Data Analytics: Public Services
• Threat of global pandemics has increased exponentially

Situation
• Pandemics spreads at faster rates, more resistant to antibiotics
• Created a network of viral listening posts

• Combines data from viral discovery in the field, research in
Use of Big Data disease hotspots, and social media trends
• Using Big Data to make accurate predications on spread of new
pandemics
• Identified a fifth form of human malaria, including its origin
Key • Identified why efforts failed to control swine flu

Outcomes
• Proposing more proactive approaches to preventing outbreaks
2
3
Big Data Analytics: Life Sciences
Situation • Broad Institute (MIT & Harvard) mapping the Human Genome
• In 13 yrs, mapped 3 billion genetic base pairs; 8 petabytes
Use of Big Data

• Developed 30+ software packages, now shared publicly, along
with the genomic data
• Using genetic mappings to identify cellular mutations causing

Key cancer and other serious diseases
Outcomes
• Innovating how genomic research informs new pharmaceutical
drugs
2
4
Big Data Analytics: IT Infrastructure
Situation
• Explosion of unstructured data required new technology to
analyze quickly, and efficiently
• Doug Cutting created Hadoop to divide large processing tasks

into smaller tasks across many computers
Use of Big Data
• Analyzes social media data generated by hundreds of
thousands of users
• New York Times used Hadoop to transform its entire public

Key archive, from 1851 to 1922, into 11 million PDF files in 24 hrs
Outcomes
• Applications range from social media, sentiment analysis,
wartime chatter, natural language processing
2
5
Big Data Analytics: Online Services
Situation • Opportunity to create social media space for professionals
• Collects and analyzes data from over 100 million users

Use of Big Data
• Adding 1 million new users per week
• LinkedIn Skills, InMaps, Job Recommendations, Recruiting

Key
Outcomes • Established a diverse data scientist group, as founder believes
this is the start of Big Data revolution
2
Lesson 4: Summary
During this lesson the following representative examples were
covered:
• Health Care
• Public Services
• Life Sciences
• IT Infrastructure
• Online Services
2
Check Your Knowledge
1. What are the 3 characteristics of Big Data, and the Your Thoughts?
main considerations in processing Big Data?
2. What is an analytic sandbox?
3. Explain the difference between Business Intelligence
and Data Science.
4. Describe the challenges of the current analytical
architecture for Data Scientists.
5. What are the key skill sets and behavioral characteristics
of a Data Scientist?
2
Module 1: Summary
Key points covered in this module:

• Big data was defined
• Four business drivers for advanced analytics were identified
• The techniques for Business Intelligence were distinguished from
those of Data Science
• The role of the Data Scientist within the new big data ecosystem
was described
• Multiple illustrative examples of big data opportunities were cited
2

OC - Module 1 - Intro To BDA 021312

Uploaded by

Copyright:

Available Formats

You might also like

OC - Module 1 - Intro To BDA 021312

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

OC - Module 1 - Intro To BDA 021312

Uploaded by

Copyright:

Available Formats

Module 1 – Introduction to Big Data Analytics

Upon completion of this module, you should be able to:

Lesson 1: Big Data Overview

What is Big Data?

What makes data, “Big” Data?

• Organizations are deriving business benefit from analyzing

Str • Data containing a defined data type, format, structure

uct • Example: Transaction data and OLAP

Semi- enabling parsing

Structured • Example: XML data files that are self

• Textual data with erratic data formats, can

“Quasi” Structured • Example: Web clickstream data that

1)What type of analytics are performed on these data?

Lesson 2: State of the Practice in Analytics

During this lesson the following topics are covered:

High Common • What if…..?

Past TIME Future

Non-Prioritized Data Provisioning

Errant data & marts

• High-value data is hard to reach and leverage

MEASURED IN MEASURED IN WILL BE MEASURED IN

1990’s 2000’s 2010’s

Analytic Medical Information

Traditional Big Data Enabled

TRADITIONAL DATA LEVERAGED BIG DATA LEVERAGED

• What new skill sets do you need to take advantage of

• Do most large organizations have people with these

• If so, who are they?

Data Scientists Role Role Description

• Design, implement and

• Create insights that lead

2 Public Services Government Internet

• Poor police response and problems with medical care, triggered

• Threat of global pandemics has increased exponentially

• Created a network of viral listening posts

Key • Identified why efforts failed to control swine flu

• In 13 yrs, mapped 3 billion genetic base pairs; 8 petabytes

Use of Big Data

• Using genetic mappings to identify cellular mutations causing

• Doug Cutting created Hadoop to divide large processing tasks

• New York Times used Hadoop to transform its entire public

Situation • Opportunity to create social media space for professionals

• Collects and analyzes data from over 100 million users

• LinkedIn Skills, InMaps, Job Recommendations, Recruiting

Key points covered in this module:

You might also like