DECS 43A - Big Data Analysis

Big Data Analytics
Government Arts and Science College

Tittagudi-606106
Department of Computer Science
DECS 43A – Big Data Analysis

II Year IV Semester
Unit -1
Introduction to Big Data
Dr. S. P. Ponnusamy
Assistant Professor and Head
1
Big Data Analytics
Tittagudi-606106
Unit -1
Introduction to Big Data
 Data
 Characteristics of data
 Types of digital data: Unstructured, Semi-structured and Structured,
 Sources of data
 Working with unstructured data
 Evolution and Definition of big data
 Characteristics and Need of big data
 Challenges of big data
 Data environment versus big data environment
2
Big Data Analytics
Tittagudi-606106
Data
• The quantities, characters, or symbols on which operations are performed by a
computer, which may be stored and transmitted in the form of electrical signals and
recorded on magnetic, optical, or mechanical recording media
Big Data
• Big Data is a collection of data that is huge in volume, yet growing exponentially
with time.
• It is a data with so large size and complexity that none of traditional data
management tools can store it or process it efficiently.
• It includes data mining, data storage, data analysis, data sharing, and data
visualization.
3
Big Data Analytics
Tittagudi-606106
Data vs Big Data
4
Big Data Analytics
Tittagudi-606106
Data Growth
• 1,024 bytes = 1 kilobyte (KB).

• 1,024 kilobytes (KB) = 1 MB.
• 1,024 MB = 1 GB.
• 1,024 GB = 1 TB
• 1,024 TB = 1 petabyte (PB).
• 1,024 PB = an exabyte (EB).
• 1,024 EB = a zettabyte (ZB)
• 1,024 ZB = 1 YB (Yottabyte).
5
Big Data Analytics
Tittagudi-606106
Data Growth
6
Big Data Analytics
Tittagudi-606106
Types of Data
7
Big Data Analytics
Tittagudi-606106
Structured Data
• This is the data which is in an organized form (e.g., in rows and columns) and can be
easily used by a computer program.
• Relationships exist between entities of data, such as classes and their objects.
• Data stored in databases is an example of structured data.
• Structured data is also called relational data.
• It is split into multiple tables to enhance the integrity of the data by creating a single
record to depict an entity.
• A Structured Query Language (SQL) is needed to bring the data together.
• Structured data is easy to enter, query, and analyze.
8
Big Data Analytics
Tittagudi-606106
Structured Data
9
Big Data Analytics
Tittagudi-606106
Structured Data - Sources
10
Big Data Analytics
Tittagudi-606106
Ease with Structured Data
11
Big Data Analytics
Tittagudi-606106
Semi-Structured Data
• This is the data which does not conform to a data model but has some structure.
• However, it is not in a form which can be used easily by a computer program.
• Example, emails, XML, markup languages like HTML, JSON document, etc.
• Metadata for this data is available but is not sufficient.
• It is commonly called NoSQL data
12
Big Data Analytics
Tittagudi-606106
Semi-Structured Data - Sources
13
Big Data Analytics
Tittagudi-606106
Semi-Structured Data –XML Example
<ProgrammerDetails>
<FirstName>Jane</FirstName>
<LastName>Doe</LastName>
<CodingPlatforms>
<CodingPlatform Type="Fav">GeeksforGeeks</CodingPlatform>
<CodingPlatform Type="2ndFav">Code4Eva!</CodingPlatform>
<CodingPlatform Type="3rdFav">CodeisLife</CodingPlatform>
</CodingPlatforms>
</ProgrammerDetails>

14
Big Data Analytics
Tittagudi-606106
Semi Structured Data – JSON Example
15
Big Data Analytics
Tittagudi-606106
Characteristics of Semi-Structured Data
16
Big Data Analytics
Tittagudi-606106
Unstructured Data
• This is the data which does not conform to a data model or is not in a form which can be
used easily by a computer program.
• Data can not be stored in the form of rows and columns as in Databases
• Data does not follows any semantic or rules
• Data lacks any particular format or sequence
• Data has no easily identifiable structure
• Due to lack of identifiable structure, it can not used by computer programs easily
• About 80–90% data of an organization is in this format.
• Example: memos, chat rooms, PowerPoint presentations, images, videos, letters, researches,
white papers, body of an email, etc.
17
Big Data Analytics
Tittagudi-606106
Unstructured Data – Example
18
Big Data Analytics
Tittagudi-606106
Unstructured Data – Sources
19
Big Data Analytics
Tittagudi-606106
Unstructured Data – issues
20
Big Data Analytics
Tittagudi-606106
Dealing with Unstructured Data
21
Big Data Analytics
Tittagudi-606106
Definition of Big Data

Big Data is high-volume, high-
High-volume
velocity, and high-variety High-velocity
information assets that demand High-variety
cost effective, innovative forms

of information processing for
enhanced insight and decision
making. Cost-effective, innovative forms of
information processing
Source: Gartner IT Glossary

Enhanced insight & decision
making
22
Big Data Analytics
Tittagudi-606106
Characteristics of Data
1. Composition: The composition of data deals with the structure of data, that is,
the sources of data, the granularity, the types, and the nature of data as to
whether it is static or real-time streaming.
2. Condition: The condition of data deals with the state of data, that is, "Can one
use this data as is for analysis?" or "Does it require cleansing for further
enhancement and enrichment?"
3. Context: The context of data deals with "Where has this data been generated?"
"Why was this data generated?" How sensitive is this data?" "What are the
events associated with this data?" and so on.
23
Big Data Analytics
Tittagudi-606106
Evolution of Big Data
24
Big Data Analytics
Tittagudi-606106
25
Big Data Analytics
Tittagudi-606106
26
Big Data Analytics
Tittagudi-606106
27
Big Data Analytics
Tittagudi-606106
Why of Big Data?
28
Big Data Analytics
Tittagudi-606106
Need of Big Data
29
Big Data Analytics
Tittagudi-606106
Characteristics of Big Data/What is Big Data?

• Volume: the size and amounts of big data that companies manage and
analyse.
• Variety: the diversity and range of different data types, including
unstructured data, semi-structured data and structured data.
• Velocity: the speed at which companies receive, store and manage data
– e.g., the specific number of social media posts or search queries
received within a day, hour or other unit of time.
30
Big Data Analytics
Tittagudi-606106
Characteristics of Big Data
31
Big Data Analytics
Tittagudi-606106
Characteristics of Big Data
32
Big Data Analytics
Tittagudi-606106
Characteristics of Big Data – other V’s
• Value: refers to the value that big data can

provide, and it relates directly to what
organizations can do with that collected
data.
• Veracity: the “truth” or accuracy of data
and information assets, which often
determines executive-level confidence
33
Big Data Analytics
Tittagudi-606106
Challenges of Big Data Capture
Storage
Curation
Challenges with Big Data

Search
Analysis
Transfer
Visualization
Privacy
Violations
34
Big Data Analytics
Tittagudi-606106
Sources of Big Data
35
Big Data Analytics
Tittagudi-606106
Traditional Business Intelligence (BI) versus Big Data
• In traditional BI environment, data resides in a central server whereas

in big data environment, data resides in a distributed file system.
• Traditional BI  Move data to code
• Big Data Environment  Move code to data
• In traditional BI environment, data is analyzed in offline mode

whereas in big data environment data is analyzed in both real time as
well as offline mode.
36
Big Data Analytics
Tittagudi-606106
A Typical Data Warehouse Environment
• In a typical DW environment, data is collected from multiple disparate sources,

integrated, cleansed and transformed before loading it to a data warehouse.
• A host of market leading BI tools can then be used on top of the data warehouse for
reporting/dashboarding, ad hoc querying and modelling.
37
Big Data Analytics
Tittagudi-606106
A Typical Hadoop Environment
Hadoop takes care of storage and processing using the following:
a)HDFS (Hadoop Distributed File System) (distributed storage)

b)MapReduce (distributed processing)
ODS-operational Data store

38
Big Data Analytics
Tittagudi-606106
Co-existence of Big Data and Data Warehouse
39
Big Data Analytics
Tittagudi-606106
End
40

DECS 43A - Big Data Analysis

Uploaded by

Copyright:

Available Formats

You might also like

DECS 43A - Big Data Analysis

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

DECS 43A - Big Data Analysis

Uploaded by

Copyright:

Available Formats

Big Data Analytics

Government Arts and Science College

DECS 43A – Big Data Analysis

Data vs Big Data

• 1,024 bytes = 1 kilobyte (KB).

Structured Data - Sources

Ease with Structured Data

Semi-Structured Data - Sources

Semi-Structured Data –XML Example

Semi Structured Data – JSON Example

Characteristics of Semi-Structured Data

Unstructured Data – Example

Unstructured Data – Sources

Unstructured Data – issues

Dealing with Unstructured Data

Definition of Big Data

cost effective, innovative forms

Source: Gartner IT Glossary

Evolution of Big Data

Evolution of Big Data

Evolution of Big Data

Evolution of Big Data

Why of Big Data?

Need of Big Data

Characteristics of Big Data/What is Big Data?

Characteristics of Big Data

Characteristics of Big Data

Characteristics of Big Data – other V’s

• Value: refers to the value that big data can

Challenges of Big Data Capture

Challenges with Big Data

Sources of Big Data

Traditional Business Intelligence (BI) versus Big Data

• In traditional BI environment, data resides in a central server whereas

• Traditional BI  Move data to code

• Big Data Environment  Move code to data

• In traditional BI environment, data is analyzed in offline mode

A Typical Data Warehouse Environment

• In a typical DW environment, data is collected from multiple disparate sources,

A Typical Hadoop Environment

Hadoop takes care of storage and processing using the following:

a)HDFS (Hadoop Distributed File System) (distributed storage)

ODS-operational Data store

Co-existence of Big Data and Data Warehouse

You might also like