Sghapter 02

Machine
Learning
Chapter 2
Understanding of Data
© Oxford University Press 2021. All rights reserved

What is Data?
• DATA ARE FACTS
• FACTS ARE IN THE FORM OF NUMBERS, AUDIO, VIDEO,
IMAGE
• NEED TO ANALYZE DATA FOR TAKING DECISIONS

Characteristics of Big Data

Characteristic of Data

Data Sources
A DATA SOURCE CAN BE ANYTHING
–
• STRUCTURED DATA
• SEMI-STRUCTURED DATA
• UNSTRUCTURED DATA

Structured Data
A STRUCTURED DATA CAN BE ANY ONE OF THE FOLLOWING –
• RECORD DATA
• GRAPHICS DATA
• DATA MATRIX
• ORDERED DATA – SEQUENCE DATA, TIME SERIES DATA, TEMPORAL
DATA

Unstructured Data
AN UNSTRUCTURED DATA CAN BE ANY ONE OF THE FOLLOWING
–
• VIDEO, IMAGE, PROGRAMS

• BLOG DATA
• 80% OF ORGANIZATION DATA

SEMI-Structured Data
A SEMI-STRUCTURED DATA CAN BE ANY ONE OF THE FOLLOWING
–
• XML/JSON OBJECTS
• RSS FEEDS
• HIERARCHICAL RECORDS

Data Storage

Data Storage
• DATABASE SYSTEMS
• TYPES ARE
1. TRANSACTIONAL DATABASE
2. TIME SERIES DATABASE
3. TEMPORAL DATABASE

Data Storage
• OTHER
TYPES

Descriptive Analytics

Diagnostic Analytics

Predictive Analytics

Prescriptive Analytics
use of data and models to suggest actions/outcomes to achieve goal

eg: a) Personalized product recommendations on e-commerce websites,
b) Fraud detection in financial transactions,
c)Predicting whether an article on a particular topic will be popular with
readers based on data about searches and social shares for related topics
d)Adjusting a worker training program in real-time based on how the worker
is responding to each lesson
Data Analysis Framework
• FRAMEWORK
raw data imported appropriate data structures – ETL
Preprocessing, parallel execution of queries, warehouse (data

in place)
statistical tests, build ml models dashboards,
display results

Types of Processing
•
CLOUD COMPUTINGpay per use, on demand, sharing cpu, applications, storage &services over network. IAAS, SAAS, PAAS
• GRID Grid computing is a distributed architecture of multiple computers connected by networks to accomplish a joint task. The computers work
together as a virtual supercomputer to perform large and complex tasks, such as
COMPUTING huge sets of data or weather analysis
•
H-COMPUTING Performs complex tasks at high speed.
PAAS: SAP Cloud, microsoft azure
SAAS: Google docs
IAAS: Amazon Web Services (AWS), Digital Ocean,

Microsoft Azure

Data collection
Good Data Characteristics

• GOD DATA SHOULD HAVE THESE
CHARACTERISTICS
Open-Source Data
1. DIGITAL LIBRARIES
2. EXPERIMENTAL DATA LIKE GENOMIC AND BIOLOGICAL
DATA
3. HEALTHCARE SYSTEMS LIKE PATIENT INSURANCE DATA
Social-Media Data
1. TWITTER DATA
2. FACEBOOK DATA
3. YOUTUBE VIDEOS
4. INSTAGRAM DATA
Multimodal Data
• IMAGE ARCHIVES WITH TEXT AND NUMERIC
DATA
• WWW
Data Preprocessing
DATA THAT CAN CAUSE
PROBLEMS
• INCOMPLETE DATA
• OUTLIER DATA
• INCONSISTENT DATA
• INACCURATE DATA
• MISSING VALUES
• DUPLICATE DATA

Missing Data

Noisy Data
BINNING
TECHNIQUE

Data Normalization
MIN-MAX PROCEDURE
TRANSFORMS DATA TO THE RANGE 0-
1
Data Normalization
Z-
SCORE

Types of Data
Patient id Name age fever disease

1 john 21 Low no
2 jack 36 high yes

Nominal Data

Ordinal Data

Numerical Data

Types of Data
BASED ON
VARIABLES

Univariate data analysis and visualization
Data Visualization

Data Visualization

Data Visualization

Data Visualization

Data Visualization

Central Tendency
MEAN OF
DATA

Central Tendency
MEDIAN OF
DATA

Central Tendency
MODE OF
DATA

DISPERSION
RANGE AND STANDARD
DEVIATION

DISPERSION
QUARTILES AND
IQR

Five-point summary
5-POINT
SUMMARY
Shape of Data
SKEWNESS AND
KURTOSIS

Shape of Data
KURTOSI
S

Shape of Data
MEAN ABSOLUTE DEVIATION AND COEFFICIENT OF
VARIATION

Stem-Leaf Plot

Q-Q Plot
QQ PLOT IS NORMALITY TEST. IF DATA CLOSER TO STRAIGHT LINE, THEN
THE DISTRIBUTION IS NORMAL.

Bivariate Data
INVOLVES TWO
VARIABLES

Bivariate Data Visualization

Bivariate Data – Covariance

Bivariate Data – Correlation

Bivariate Data – Correlation

Multivariate Data Visualization

Multivariate Data Visualization

Multivariate Essential Mathematics
1. GAUSSIAN
ELIMINATION

1. GAUSSIAN
ELIMINATION

1.MATRIX
DECOMPOSITION

1.MATRIX
DECOMPOSITION

1.
DISTRIBUTIONS

EXPONENTIAL
DISTRIBUTION

BINOMIAL
DISTRIBUTION

POSSON AND BERNOULLI
DISTRIBUTION

Density Estimation

Hypothesis Testing
Z-
TEST

Hypothesis Testing
PAIRED T-
TEST

Hypothesis Testing
PAIRED T-
TEST

Hypothesis Testing
PAIRED T-
TEST

Hypothesis Testing
CHI-SQUARE
TEST

Hypothesis Testing
CHI-SQUARE
TEST

Hypothesis Testing
CHI-SQUARE
TEST

Feature Engineering

Feature Engineering
• FEATURE TRANSFORMATION
• FEATURE SELECTIONS

Characteristics of Good Features
• FEATURES ARE REMOVED USING RELEVANCY
• FEATURES ARE REMOVED BASED ON
REDUNDANCY

FEATURE SELECTION
FORWARD
SELECTION

FEATURE SELECTION
BACKWARD
SELECTION

Principal Component Analysis

Compute
Covariance
matrix as
Compute Eigen
values and
Eigen vectors
and matrix A as
a set of eigen
vectors

Compute PCA as
The original
Data can be
recovered as

PCA Algorithm

PCA Example

PCA Example

PCA Example

PCA Example

PCA Example

Verification

LDA Algorithm

LDA Algorithm

SVD Algorithm

SVD Algorithm

SVD Example

SVD Example

SVD Example

SVD Example

Summary

Summary

Sghapter 02

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Sghapter 02

Uploaded by

Copyright:

Available Formats

Machine

© Oxford University Press 2021. All rights reserved

© Oxford University Press 2021. All rights reserved

© Oxford University Press 2021. All rights reserved

© Oxford University Press 2021. All rights reserved

© Oxford University Press 2021. All rights reserved

© Oxford University Press 2021. All rights reserved

• VIDEO, IMAGE, PROGRAMS

© Oxford University Press 2021. All rights reserved

© Oxford University Press 2021. All rights reserved

© Oxford University Press 2021. All rights reserved

© Oxford University Press 2021. All rights reserved

© Oxford University Press 2021. All rights reserved

© Oxford University Press 2021. All rights reserved

© Oxford University Press 2021. All rights reserved

© Oxford University Press 2021. All rights reserved

use of data and models to suggest actions/outcomes to achieve goal

raw data imported appropriate data structures – ETL

Preprocessing, parallel execution of queries, warehouse (data

statistical tests, build ml models dashboards,

© Oxford University Press 2021. All rights reserved

H-COMPUTING Performs complex tasks at high speed.

PAAS: SAP Cloud, microsoft azure

SAAS: Google docs

IAAS: Amazon Web Services (AWS), Digital Ocean,

© Oxford University Press 2021. All rights reserved

Good Data Characteristics

© Oxford University Press 2021. All rights reserved

© Oxford University Press 2021. All rights reserved

© Oxford University Press 2021. All rights reserved

© Oxford University Press 2021. All rights reserved

Patient id Name age fever disease

© Oxford University Press 2021. All rights reserved

© Oxford University Press 2021. All rights reserved

© Oxford University Press 2021. All rights reserved

© Oxford University Press 2021. All rights reserved

© Oxford University Press 2021. All rights reserved

© Oxford University Press 2021. All rights reserved

© Oxford University Press 2021. All rights reserved

© Oxford University Press 2021. All rights reserved

© Oxford University Press 2021. All rights reserved

© Oxford University Press 2021. All rights reserved

© Oxford University Press 2021. All rights reserved

© Oxford University Press 2021. All rights reserved

© Oxford University Press 2021. All rights reserved

© Oxford University Press 2021. All rights reserved

© Oxford University Press 2021. All rights reserved

© Oxford University Press 2021. All rights reserved

© Oxford University Press 2021. All rights reserved

© Oxford University Press 2021. All rights reserved

© Oxford University Press 2021. All rights reserved

© Oxford University Press 2021. All rights reserved

© Oxford University Press 2021. All rights reserved

© Oxford University Press 2021. All rights reserved

© Oxford University Press 2021. All rights reserved

© Oxford University Press 2021. All rights reserved

© Oxford University Press 2021. All rights reserved

© Oxford University Press 2021. All rights reserved

© Oxford University Press 2021. All rights reserved

© Oxford University Press 2021. All rights reserved

© Oxford University Press 2021. All rights reserved

© Oxford University Press 2021. All rights reserved