Big Data Project Management

Managing Big Data Project

Danairat T.
Line ID: Danairat
FB: Danairat Thanabodithammachari
+668-1559-1446

1 Danairat T., 2015, danairat@gmail.com

Agenda

• Introduction to Big Data

• Big Data Discovery Worksheet
• Big Data Project Life Cycle
• Big Data Team Structure
• Key Activities, People and Deliverables
• Key Success Factors
• Summary

2 Danairat T., 2015, danairat@gmail.com

Big Data Introduction
Volume

ZB

XB

PB

TB

GB

DB Table Batch
Delimited Text
XML, HTML Near real time
Free Form Text
Image, Music, VDO, Binary Real time

Variety Velocity
3 Danairat T., 2015, danairat@gmail.com
 Big Data Project Goals Worksheet
Sample
# Domain Increasing of Resource Reduce Risk
Value or Revenue Optimization

1 Finance Increase Revenue Create Services Compliance with laws

Portfolio regulations

2 Customer New Products, Reuse Business Service Continuity and

Service, Promotion Channels Availability, Customer
Innovation Complaint Management

3 Internal Create New Eliminate Production Standardize Change

Business Process Cost Control

4 Learning Seek more Talent Standardize Skill Enterprise Knowledge

and Growth People Required Repository

4 Danairat T., 2015, danairat@gmail.com

 Big Data Project Life Cycle
Big Data
Planning
• Identify Targeted Users
• Identify Target
Opportunities
• Identify Team Structure
• Identify Data
Sources/Types
• Identify Data Capturing
Approaches
• Identify Data Processing
and Visualization Planning
• Identify Big Data Platform
• Identify Security
• Identify Governance and
Change Control for
Operation
• Identify Phasing, Budget
and Cost
Big Data
Development
• Develop Use Cases
• Develop Requirements
Definition
• Develop Big Data
Solution Framework
• Develop the
Development and Test
Environment
• Develop Data Capture
• Develop Analytics
• Integrate Visualization
• Manage Assets and
Configurations
5 Danairat T., 2015, danairat@gmail.com
Operation and
Evaluation
Support
• Monitor Big Data • Adoption Rates for each
Platform Availability, analytics results
Utilization and Capacity
Planning • No. of Missing Analytic
Results
• Manage Operation
Tasks (Admin. Scripts, • No. of Missing Data
Commands), Data • Lost hours per month
Capturing System,
Upgrading or Patching • Avg. of each Analytic
Result Response Time
• Manage Service
Requests and Incidents • No. of Technology System
Failure per month
• System admin. Training
• Evaluate Activity
• Helpdesk Training Conformance with Policies
• End-User Training
(Analytic Results)
Big Data Discovery Worksheet
Identify Big Data Opportunity - Sample Customer Complaint

Who What Why When Data Sources

CEO แนวโน้มจํานวนเรื องร้องเรี ยน และ รักษาภาพลักษณ์ทีดีขององค์กร Monthly/Ad-hoc Call Center
แนวทางการตอบสนองความ ผูใ้ ช้บริ การมี Loyalty GIS/Map
ต้องการของลูกค้า รายได้เติบโต Pantip.com
Products/Services Master Data
Billing System –
Revenue/Month

COO ประเภทของการร้องเรี ยน และ พัฒนาบริ การให้ดีข. ึน Monthly/Ad-hoc Call Center

ความถีของเรื องร้องเรี ยน เรื องร้องเรี ยนลดลง GIS/Map
Pantip.com
Products/Services Master Data
CIO ประเภท จํานวน และแนวโน้มของ เพิมสื อ และช่องทางการให้ขอ้ มูล Monthly/Ad-hoc Call Center
การร้องเรี ยน ระดับการตัดสิ นใจแก่ผบู ้ ริ หาร GIS/Map
ระดับสู ง และทีมงานด้านธุรกิจที Pantip.com
เกียวข้อง Products/Services Master Data
CMO วิเคราะห์ประเภทและความรุ นแรง ปรับปรุ ้งบริ การในเขตความ Monthly/Ad-hoc Call Center
ความถีของเรื องร้องเรี ยน รับผิดชอบให้มีประสิ ทธิ ภาพสู ง GIS/Map
Products/Services Master Data
Business Line วิเคราะห์ประเภทและความรุ นแรง เพือส่ งต่อข้อมูลร้องเรี ยนให้กบั Monthly/Daily Call Center
Manager ความถีของเรื องร้องเรี ยน หน่วยงานทีเกียวข้อง และติดตาม GIS/Map
ปั ญหาจนสิ. นสุ ด Products/Services Master Data

6 Danairat T., 2015, danairat@gmail.com

 Big Data People and Team Structure

Big Data Program

Committee

Big Data
Evangelist

Data
Project Big Data Business
Data Scientist Integration Developer
Manager Architect Analyst
Specialist

7 Danairat T., 2015, danairat@gmail.com

 Big Data Team Structure
No. Roles Description
1 Big Data The Team to develop Big Data initiative and deliver solution
Program results
Committee
2 Big Data The business evangelist must have a combination of good
Evangelist communication and presentation skills and deep contextual
business knowledge, as well as a clear understanding of
technology in general and big data techniques.

3 Project The project manager “owns” the development schedule and

Manager is expected to ensure that the right architects, designers, and
developers are brought into the project at the right times.
4 Big Data The person who has background in parallel and distributed
Architect computing architecture. This person is knowledgeable about
fundamental performance “gotchas” that will impede the
speed, scalability, and extensibility of any application
requiring massive data volumes.

8 Danairat T., 2015, danairat@gmail.com

 Big Data Team Structure

No. Roles Description

5 Data The data scientist combines knowledge of computer science,
Scientist the use of high-performance applications, and statistics,
economics, mathematics, and probabilistic analysis skills.
6 Business The person who engages with the business process owners
Analyst and solicits their needs and expectations. Business analysts
who are able to effectively translate business expectations
into specific data analysis results.
7 Data The person who has experience in data extraction,
Integration transformation, loading, and data transformations in
Specialist preparation for cleansing and delivery to target systems.
Seek people with experience with data federation and
virtualization, data quality, and metadata analysis.
8 Application The technical resources with the right set of skills for
Developer programming and testing parallel and distributed
applications.

9 Danairat T., 2015, danairat@gmail.com

 Key Activities, People and Deliverables
No. Phases Activities People Deliverables
1 Planning Identify Targeted Users Big Data Program Committee Big Data Discovery
Worksheet
2 Planning Identify Target Big Data Program Committee Big Data Discovery
Opportunities Worksheet
3 Planning Identify Team Structure Big Data Program Committee Team Organization Chart

4 Planning Identify Data Big Data Architect, Data Scientist, Data Sources Types
Sources/Types Data Integration Specialist Information
5 Planning Identify Data Capturing Data Integration Specialist, Data Data Capturing
Approaches Scientist Information
6 Planning Identify Data Processing Business Analyst, Big Data Architect, Data Processing
and Visualization Planning Data Scientist, Developer Framework and User
Interface Summary
7 Planning Identify Big Data Platform Big Data Architect, Project Manager Big Data Platform
Summary
8 Planning Identify Security Corporate Information Security, Big Security Scope Summary
Data Architect, Project Manager
9 Planning Identify Governance and Internal Control Team, Corporate Governance, RACI,
Change Control for Information Security, Big Data Change Procedures
Operation Architect, Project Manager Summary

10 Planning Identify Phasing Budget CIO, CFO, Project Manager, Business Project Investment
and Cost Analyst Summary

10 Danairat T., 2015, danairat@gmail.com

 Key Activities, People and Deliverables
No. Phases Activities
1 Development Develop Use Cases
2 Development Develop Requirements
Definition
3 Development Develop Big Data Solution
Framework
4 Development Develop the Development and
Test Environment
5 Development Develop Data Capture
6 Development Develop Analytics
7 Development Integrate Visualization
8 Development Manage Assets and
Configurations
Agile Methodology may apply in Big Data Development Phase.
People Deliverables
Business Users, Business Analyst, Big Use Cases Summary
Data Architect, Big Data Evangelist
Business Users, Business Analyst, Big Requirements Summary
Data Architect
Big Data Architect Solution Component
Diagram
Big Data Architect, Development and Test
Data Integration Specialist, Developer Environment
Data Integration Specialist, Developer Data Capturing Module
Data Integration Specialist, Developer Data Analytic Module
Data Integration Specialist, Developer User Interface and
Visualization Results
Project Manager, Big Data Architect, Assets Inventory and
Corporate Information Security, Head Configurations Change
of IT Operation Procedure
11 Danairat T., 2015, danairat@gmail.com
 Key Activities, People and Deliverables
No. Phases Activities People Deliverables

1 Operation and Monitor Big Data Platform Availability, IT Operation Team Availability, Utilization
Support Utilization and Capacity Planning and Capacity Planning
Report

2 Operation and Manage Operation Tasks (Admin. IT Operation Team, Big Schedule or Ad-Hoc
Support Scripts, Commands), Data Capturing Data Architect Operation Activities
System, Upgrading or Patching
3 Operation and Manage Service Requests and IT Operation Team Service Requests and
Support Incidents Incidents Procedures

4 Operation and System Administration Training Big Data Architect, System Administration
Support Data Integration and Operation Training
Specialist, Developer, IT Activity
Administration, IT
Operation
5 Operation and Helpdesk Training IT Administration, IT Helpdesk Training Activity
Support Operation, IT Helpdesk

6 Operation and End-User Training (Analytic Results) Business Analyst, End-User Training
Support Business Users Activity

12 Danairat T., 2015, danairat@gmail.com

 Key Activities, People and Deliverables

No. Phases Activities People Deliverables

1 Evaluation Create Adoption Rates for IT Operation Percent of user adoption
each analytics Results

2 Evaluation Create No. of Missing Analytic Big Data Program Committee No. of Missing Analytics
Results Report
3 Evaluation Create No. of Missing Data Big Data Program Committee No. of Missing Data
Results Report
4 Evaluation Create Lost hours per month Big Data Architect, Data Scientist, Lost hours per month
Results Data Integration Specialist Report
5 Evaluation Create Avg. of each Analytic Data Integration Specialist, Data Analytic Processing and
Processing and Response Scientist Response Time Report
Time Results
6 Evaluation Create No. of Technology Business Analyst, Big Data Architect, Technology System
System Failure per month Data Scientist, Developer Failure per month Report
Results

7 Evaluation Evaluate Activity Conformance Big Data Architect, Project Manager Change Control Log
with Policies Report

13 Danairat T., 2015, danairat@gmail.com

Key Activities, People and Deliverables
Big Data Platform
Big Data Applications
Next Best Fraud Cyber Talent
BI/Report
Action Analysis Security Search

Big Data Platform

Descriptive Predictive Prescriptive
Analytics Analytics Analytics

Monitoring Distributed Data Processing

and Security
Management Framework
Framework Distributed Data Store and DWH

Integration and Metadata Framework

Big Data Infrastructure

Hardware, Storage, Network

14 Danairat T., 2015, danairat@gmail.com

Big Data Platform & Big Data Analytics
Hadoop Technology

15 Danairat T., 2015, danairat@gmail.com

 Key Activities, People and Deliverables
Analytic Types

Descriptive analytics answers the question, "What happened in

the business?" It looks at data and information to describe the
current business situation in a way that trends, patterns and
exceptions become apparent. This takes the form of reports,
dashboards, MIS, etc.

mu-sigma.com

16 Danairat T., 2015, danairat@gmail.com

 Key Activities, People and Deliverables
Analytic Types

Predictive analytics answers the question, "What is likely to

happen in the future?" Here data modeling and forecasting are used
to determine future possibilities

mu-sigma.com

17 Danairat T., 2015, danairat@gmail.com

 Key Activities, People and Deliverables
Analytic Types

Prescriptive analytics is the combination of the above to provide

answers to the "So what?" and the "Now what?" For example, what
should a business do to retain key customers? How can businesses
improve their supply chain to enhance service levels while reducing
costs?

mu-sigma.com

18 Danairat T., 2015, danairat@gmail.com

Key Success Factors

1. Support from Business Sponsor

2. Start with Outcome Answer First
3. Involve Real Users and Create Effective Use Cases
4. Define Quick-Win and Phasing
5. Sufficient Data Source
6. Choose the Open Technology Platform
7. Identify SLA for Service Operation
8. Project Review

19 Danairat T., 2015, danairat@gmail.com

Thank you very much.

20 Danairat T., 2015, danairat@gmail.com