Session 11 12

1. Introduction to Business Analytics

2. Predictive Analytics / Data Mining
Process and Methodology
Prof Sanjiva Shankar Dubey (VF)
Former IBM Asia Pacific Executive

Source IT Strategy and Management Sanjiva Dubey 1

Indian Institute of Management Lucknow, Noida campus

Agenda

• MIS Textbook – Chapter 7 Information Knowldege and

Busienss Intelligence , Chapter 19 Data Warehouse
and Chapter 15 Busienss Intelligence for MIS
• Analytics 3.0, Davenport, Harvard Business Review
(R1312C-PDF-ENG)

Source IT Strategy and Management Sanjiva Dubey 2

Indian Institute of Management Lucknow, Noida campus

1
 Business Intelligence and Business Analytics

Business Business
Intelligence Analytics

Predictive analytics, which uses

Descriptive analytics, which statistical methods and models to
provides a summary of forecast future outcomes and scenarios
historical and present data based on data.

Source IT Strategy and Management Sanjiva Dubey 3

Indian Institute of Management Lucknow, Noida campus

Information Fallacy
• “The information you have is not the information you
want.
• The information you want is not the information you
need.
• The information you need is not the information you
can obtain.
• The information you can obtain costs more than you
want to pay”
• ― Peter L. Bernstein, Against the Gods: The Remarkable Story of Risk

Source IT Strategy and Management Sanjiva Dubey 4

7/25/2023 4
Indian Institute of Management Lucknow, Noida campus

2
 What is Data, Information, Knowledge ,
Intelligence , Wisdom
•

Source IT Strategy and Management Sanjiva Dubey 5

Indian Institute of Management Lucknow, Noida campus

Data, Information, Knowledge

• Data
– Items that are the most elementary descriptions of things, events,
activities, and transactions
– May be internal or external
• Information
– Organized data that has meaning and value
• Knowledge
– Processed data or information that conveys understanding or
learning applicable to a problem or activity
• Intelligence
– the ability to understand, learn and think derive hidden pattern that
does not seem apparent through normal analysis
• Wisdom
– the ability to use your knowledge and experience to make good
decisions and judgments

Source IT Strategy and Management Sanjiva Dubey 6

7/25/2023 6
Indian Institute of Management Lucknow, Noida campus

3
 Corpus ID: 7393553

DIKIW: Data, Information,

Knowledge, Intelligence,
Wisdom and their
Interrelationships
•A. Liew
•Published 2013
•Psychology

Source IT Strategy and Management Sanjiva Dubey 7

Indian Institute of Management Lucknow, Noida campus

Another context

Source IT Strategy and Management Sanjiva Dubey 8

Indian Institute of Management Lucknow, Noida campus

4
 Why Data has become more important
now than ever before
Need for accuracy
• Data helps make better decisions
in decision making

Solving Complex
• Data helps solve problems
Problem

Managing • Data helps understand performance

and improve processes
performance

Data helps you understand consumers

• Read this https://www.forbes.com/sites/ciocentral/2020/04/17/now-more-

than-ever--the-necessity-of-data-analytics-and-expertise/?sh=7199a81b20f4

Source IT Strategy and Management Sanjiva Dubey 9

Indian Institute of Management Lucknow, Noida campus

Data is new Oil

• With flood of data comes a tidal wave of

opportunity.
• Increasingly standardized corporate data, and
access to rich, third-party data sets—all leveraged
by cheap, fast computing and easier-to-use
software—are collectively enabling a new age of
data-driven, fact-based decision making.

• You agree ?

Source IT Strategy and Management Sanjiva Dubey 10

Indian Institute of Management Lucknow, Noida campus

5
 Let us clear some terminology

• Old-school terms like “decision support systems”

• Business intelligence (BI), a catchall term
combining aspects of reporting, data exploration
and ad hoc queries, and sophisticated data
modeling and analysis.
• New managerial lexicon is the phrase Analytics, a
term describing the extensive use of data,
statistical and quantitative analysis, explanatory
and predictive models, and fact-based management
to drive decisions and actions

Source IT Strategy and Management Sanjiva Dubey 11

Indian Institute of Management Lucknow, Noida campus

Enterprise Data Management Strategies ; Why?

• Data is the most important asset within
Information systems and for the enterprise.
• Enterprise data cannot be purchased from
anywhere and is difficult if not impossible to
recreate if it is lost
• Increased competition forcing firms to protect
their customer base and use data for focused
marketing by understanding customer behaviours
and preferences
•

Source IT Strategy and Management Sanjiva Dubey 12

7/25/2023 12
Indian Institute of Management Lucknow, Noida campus

6
 3 Data Dichotomies

• “When collecting and analyzing business intelligence data, analysts most

often focus on their organization’s internal data (70 ”
(Source: The Silicon Review)
• 1. INTERNAL VS. EXTERNAL DATA Internal data is information generated from
within the business, covering areas such as operations, maintenance, personnel, and
finance. External data comes from the market, including customers and competitors.
It’s things like statistics from surveys, social media post, questionnaires, research,
and customer feedback.
• 2. STRUCTURED VS. UNSTRUCTURED DATA Structured data is considered more
traditionally as BI, because it’s quantifiable. It’s easier to put in a database, search, and
analyze. Conversely, unstructured data is considered a newer type of data. It’s not pre-
defined and is typically text-heavy information, such as that from social networks or
customer comments.
• 3. HISTORICAL VS. REAL-TIME DATA Do a mixture of both. Companies often
spend significant time using historical data to identify and predict trends. But
without real-time data to compare it to, the value of that historical data is
limited.

Source IT Strategy and Management Sanjiva Dubey 13

Indian Institute of Management Lucknow, Noida campus

TYPES OF DATABASE within Enterprise

• Operational database for TPS and MIS systems.

usually (RDBMS)

• Data Ware house for future analysis

• Now Big data

Source IT Strategy and Management Sanjiva Dubey 14

7/25/2023 14
Indian Institute of Management Lucknow, Noida campus

7
 What is Big Data ?
 Big data is a popular term used to describe the exponential
growth and availability of data, both structured and unstructured.

 Extremely large data sets that may be analysed computationally

to reveal patterns, trends, and associations, especially relating to
human behaviour and interactions
 Not only VOLUME but Variety and Velocity

 Multi-structured Data
 Web logs
 Social media
 Sensor generated

 Analytics on data is called Big data analytics

Source IT Strategy and Management Sanjiva Dubey 15

7/25/2023 15
Indian Institute of Management Lucknow, Noida campus

11.3 CHARACTERISTICS OF BIG DATA

• Enormous volume: Variety:

• Velocity: Veracity Value:

Source IT Strategy and Management Sanjiva Dubey 16

7/25/2023 16
Indian Institute of Management Lucknow, Noida campus

8
 The Big Data Value Chain

Source IT Strategy and Management Sanjiva Dubey 17

Indian Institute of Management Lucknow, Noida campus

What is a Relational Database Management

System ?
Database Management System = DBMS
Relational DBMS = RDBMS

• A collection of files that store the data

• A big C program written by someone else that

accesses and updates those files for you

• Examples: Oracle, DB2, SQL Server

Source IT Strategy and Management Sanjiva Dubey 18

Indian Institute of Management Lucknow, Noida campus

9
 Where are RDBMS used ?

• Backend for traditional “database” applications

• Backend for large Websites
• Backend for Web services

Source IT Strategy and Management Sanjiva Dubey 19

Indian Institute of Management Lucknow, Noida campus

Functionality of a DBMS

The programmer sees SQL, which has two components:

• Data Definition Language - DDL
• Data Manipulation Language - DML
– query language

Behind the scenes the DBMS has:

• Query engine
• Query optimizer
• Storage management
• Transaction Management (concurrency, recovery)

Source IT Strategy and Management Sanjiva Dubey 20

Indian Institute of Management Lucknow, Noida campus

10
 How the Programmer Sees the DBMS

• Start with DDL to create tables:

CREATE TABLE Students (
Name CHAR(30)
SSN CHAR(9) PRIMARY KEY NOT NULL,
Category CHAR(20)
) ...

• Continue with DML to populate tables:

INSERT INTO Students
VALUES(‘Charles’, ‘123456789’, ‘undergraduate’)
. . . .

Source IT Strategy and Management Sanjiva Dubey 21

Indian Institute of Management Lucknow, Noida campus

How the Programmer Sees the DBMS

• Tables:
Students: Takes:

SSN Name Category SSN CID

123-45-6789 Charles undergrad 123-45-6789 CSE444
234-56-7890 Dan grad 123-45-6789 CSE444
… … 234-56-7890 CSE142
…
Courses:
CID Name Quarter
CSE444 Databases fall
CSE541 Operating systems winter

• Still implemented as files, but behind the scenes

can be quite complex

Source IT Strategy and Management Sanjiva Dubey 22

Indian Institute of Management Lucknow, Noida campus

11
 Database Systems

• The big commercial database vendors:

– Oracle
– IBM (with DB2)
– Microsoft (SQL Server)
– Sybase
• Some free database systems (Unix) :
– Postgres
– MySQL
– Predator
•
Source IT Strategy and Management Sanjiva Dubey 23
Indian Institute of Management Lucknow, Noida campus

Data bases types

• Operational Data base : discussed so far

• DSS data base for decision making ,, we will now

explain

Source IT Strategy and Management Sanjiva Dubey 24

7/25/2023 24
Indian Institute of Management Lucknow, Noida campus

12
 Decision Support Systems

• Extract Information from data to use as the basis

for decision making
• Used at all levels of the Organization
• Tailored to specific business areas
• Interactive
• Ad Hoc queries to retrieve and display information
• Combines historical operation data with business
activities

Source IT Strategy and Management Sanjiva Dubey 25

7/25/2023 25
Indian Institute of Management Lucknow, Noida campus

Decision Support Systems

• Created to facilitate the decision making process

• So much information that it is difficult to extract it
all from a traditional database
• Need for a more comprehensive data storage
facility

–Data Warehouse

Source IT Strategy and Management Sanjiva Dubey 26

7/25/2023 26
Indian Institute of Management Lucknow, Noida campus

13
 Learning Pause 1
• Identify two data bases which you are familiar with.
• What are the data elements, identify 5 with their
characteristics,
• How will you ensure its data quality? for being
– Accurate
– Consistent
– Complete
– Unique
– Relevant
– Timely
– Portable
• Write in your learning diary
Source IT Strategy and Management Sanjiva Dubey 27
7/25/2023 27
Indian Institute of Management Lucknow, Noida campus

Four Components of DSS

1. Data Store – The DSS Database

– Business Data
– Business Model Data
– Internal and External Data
2. Data Extraction and Filtering
– Extract and validate data from the operational
database and the external data sources
3. End-User Query Tool
– Create Queries that access either the Operational or
the DSS database
4. End User Presentation Tools
– Organize and Present the Data
Source IT Strategy and Management Sanjiva Dubey 28
7/25/2023 28
Indian Institute of Management Lucknow, Noida campus

14
Source IT Strategy and Management Sanjiva Dubey 29
7/25/2023 29
Indian Institute of Management Lucknow, Noida campus

Data Marts

• Small Data Stores

• More manageable data sets
• Targeted to meet the needs of small groups within
the organization

• Small, Single-Subject data warehouse subset that

provides decision support to a small group of
people

Source IT Strategy and Management Sanjiva Dubey 30

7/25/2023 30
Indian Institute of Management Lucknow, Noida campus

15
 Data Marts vs Data Warehouse

• Dependent
– Created from warehouse
– Replicated
• Functional subset of warehouse
• Independent
– Scaled down, less expensive version of data warehouse
– Designed for a department or SBU
– Organization may have multiple data marts
• Difficult to integrate

Source IT Strategy and Management Sanjiva Dubey 31

7/25/2023 31
Indian Institute of Management Lucknow, Noida campus

Data Warehouse : properties

• Subject oriented
• Scrubbed so that data from heterogeneous sources are
standardized
• Time series; no current status
• Nonvolatile
– Read only
• Summarized
• Not normalized; may be redundant
• Data from both internal and external sources is present
• Metadata included
– Data about data
• Business metadata
• Semantic metadata
• Process Metadata

Source IT Strategy and Management Sanjiva Dubey 32

7/25/2023 32
Indian Institute of Management Lucknow, Noida campus

16
 End Day 1

Source IT Strategy and Management Sanjiva Dubey 33

Indian Institute of Management Lucknow, Noida campus

Introduction to Business Analytics

• Business analytics is the process of using data, statistical
methods, and analytical tools to understand and improve
business performance.
• It helps managers and decision-makers to identify problems,
opportunities, and solutions in various domains such as
marketing, finance, operations, human resources, etc.
• Data Science and Business Intelligence: data collection,
cleaning, manipulation, visualization, and exploration
• Statistical Analysis and Quantitative Techniques: statistical
methods and models for analyzing data,
• Data Mining and Predictive Analytics: such as classification,
clustering, association rules, anomaly detection, etc
• Big Data Analytics: concepts and challenges of big data
analytics, such as volume, velocity, variety, veracity, value, etc
Source IT Strategy and Management Sanjiva Dubey 34
Indian Institute of Management Lucknow, Noida campus

17
 Business Intelligence and Analytics
• Business intelligence (capability )
– Acquisition of data and information for use in decision-
making activities
• Business analytics (Process and tools)
– Models and solution methods
• Data mining (Application )
– Applying models and methods to data to identify
patterns and trends
Intelligence : The ability to acquire and apply knowledge and skills.
Deriving New Meaning is Intelligence

Source IT Strategy and Management Sanjiva Dubey 35

35
Indian Institute of Management Lucknow, Noida campus

Source IT Strategy and Management Sanjiva Dubey 36

Indian Institute of Management Lucknow, Noida campus

18
 Business Analytics

• Descriptive analytics: the use of data to understand

past and current business performance and make
informed decisions
• Predictive analytics: predict the future by
examining historical data, detecting patterns or
relationships in these data, and then extrapolating
these relationships forward in time.
• Prescriptive analytics: identify the best
alternatives to minimize or maximize some
objective

Source IT Strategy and Management Sanjiva Dubey 37

Indian Institute of Management Lucknow, Noida campus

OLAP

• OLAP is an acronym for Online Analytical

Processing. OLAP performs multidimensional
analysis of business data and provides the
capability for complex calculations, trend analysis,
and sophisticated data modeling.
• A DSS tools that has
– Support for a DSS data store
– Data extraction and integration filter
– Specialized presentation interface

Source IT Strategy and Management Sanjiva Dubey 38

7/25/2023 38
Indian Institute of Management Lucknow, Noida campus

19
 Business Analytics and Business Intelligence

Multidimensional Data Model

The view that is showing is product versus region. If you rotate the cube 90
degrees, the face that will show is product versus actual and projected sales.
If you rotate the cube 90 degrees again, you will see region versus actual and
projected sales. Other views are possible.
Source IT Strategy and Management Sanjiva Dubey 39
Indian Institute of Management Lucknow, Noida campus

Watch this video

• https://www.youtube.com/watch?v=rnQDuz1ZkIo

Source IT Strategy and Management Sanjiva Dubey 40

Indian Institute of Management Lucknow, Noida campus

20
 OLAP
• Basic analytical operations of OLAP
• Four types of analytical operations in OLAP are:
– Roll-up
– Drill-down
– Slice and dice
– Pivot (rotate)

Source IT Strategy and Management Sanjiva Dubey 41

Indian Institute of Management Lucknow, Noida campus

OLAP
• Basic analytical operations of OLAP
• Four types of analytical operations in OLAP are:
– Roll-up
– Drill-down
– Slice and dice
– Pivot (rotate)

Source IT Strategy and Management Sanjiva Dubey 42

Indian Institute of Management Lucknow, Noida campus

21
 OLAP
• Basic analytical operations of OLAP
• Four types of analytical operations in OLAP are:
– Roll-up
– Drill-down
– Slice and dice
– Pivot (rotate)

Source IT Strategy and Management Sanjiva Dubey 43

Indian Institute of Management Lucknow, Noida campus

OLAP
• Basic analytical operations of OLAP
• Four types of analytical operations in OLAP are:
– Roll-up
– Drill-down
– Slice and dice
– Pivot (rotate)

Source IT Strategy and Management Sanjiva Dubey 44

Indian Institute of Management Lucknow, Noida campus

22
 OLAP
• Basic analytical operations of OLAP
• Four types of analytical operations in OLAP are:
– Roll-up
– Drill-down
– Slice and dice
– Pivot (rotate)

Source IT Strategy and Management Sanjiva Dubey 45

Indian Institute of Management Lucknow, Noida campus

Data Mining
Data mining is to identify hidden patterns and to build models from large
data sets.
• Customer segmentation—figuring out most valuable customers
• Marketing and promotion targeting—identifying customers will respond
to which offers at which price at what time.
• Market basket analysis—determining which products customers buy
together, used to cross-sell more products or services.
• Collaborative filtering—personalizing an individual customer’s
experience based on the trends and preferences across similar
customers.
• Customer churn—determining which customers are likely to leave
• Fraud detection—uncovering patterns consistent with criminal activity.
• Financial modeling—building trading systems to capitalize on historical
trends.
• Hiring and promotion—identifying characteristics consistent with
employee success in the firm’s various roles

Source IT Strategy and Management Sanjiva Dubey 46

Indian Institute of Management Lucknow, Noida campus

23
 Business Analytics and Business Intelligence

• Data mining:
• More discovery driven than OLAP
• Finds hidden patterns, relationships in large databases
• Infers rules to predict future behavior
• The patterns and rules are used to guide decision making
and forecast the effect of those decisions
• Popularly used to provide detailed analyses of patterns in
customer data for one-to-one marketing campaigns or to
identify profitable customers.
• Less well known: used to trace calls from specific
neighborhoods that use stolen cell phones and phone
accounts

Source IT Strategy and Management Sanjiva Dubey 47

Indian Institute of Management Lucknow, Noida campus

Business Analytics and Business

Intelligence
Using Databases to Improve Business Performance and Decision
Making

• Types of information obtainable from data mining

• Associations: Occurrences linked to single event
• Sequences: Events linked over time
• Classification: Recognizes patterns that describe group to
which item belongs
• Clustering: Similar to classification when no groups have
been defined; finds groupings within data
• Forecasting: Uses series of existing values to forecast what
other values will be

Source IT Strategy and Management Sanjiva Dubey 48

Indian Institute of Management Lucknow, Noida campus

24
 Business Analytics and Business
Intelligence
Using Databases to Improve Business Performance and
Decision Making
• Predictive analysis
• Uses data mining techniques, historical data, and assumptions
about future conditions to predict outcomes of events
• E.g. Probability a customer will respond to an offer or purchase a
specific product.
• https://www.youtube.com/watch?v=GO8Cd2eUTVE
• https://www.youtube.com/watch?v=cVibCHRSxB0
• Data mining seen as challenge to individual privacy
• Used to combine information from many diverse sources to create
detailed “data image” about each of us—income, driving habits,
hobbies, families, and political interests

Source IT Strategy and Management Sanjiva Dubey 49

Indian Institute of Management Lucknow, Noida campus

Predictive analytics is the use of data, statistical algorithms and machine

learning techniques to identify the likelihood of future outcomes based on
historical data. The goal is to go beyond knowing what has happened to
providing a best assessment of what will happen in the future.

Source IT Strategy and Management Sanjiva Dubey 50

Predictive analytics
Indian Institute of Management Lucknow, Noida campus

25
 Sisense

• https://www.youtube.com/watch?v=U6MAfa2KRFo

Source IT Strategy and Management Sanjiva Dubey 51

Indian Institute of Management Lucknow, Noida campus

Jam board exercise

• Visit this site and select one industry and

summarize one use case

• https://www.sas.com/en_in/insights/analytics/pred
ictive-analytics.html

Source IT Strategy and Management Sanjiva Dubey 52

Indian Institute of Management Lucknow, Noida campus

26
  Data visualization Data Visualization
A graphical, animation, or video presentation of data and the results of data
analysis
 The ability to quickly identify important trends in corporate and market data can provide
competitive advantage
 Check their magnitude of trends by using predictive models that provide significant
business advantages in applications that drive content, transactions, or processes

Source IT Strategy and Management Sanjiva Dubey 53

Indian Institute of Management Lucknow, Noida campus

Some popular data mining tools

• SAS Enterprise Minor . IBM Intelligent Minor
• Unica PRW . SPSS Clementine
• SGI Mineset .Oracle Darwin
• Angoss KnowledgeSeeker

The application-specific tools segment is rapidly gaining

momentum. Among these tools are the following:
• KDI (focuses on retail)
• Options & Choices (focuses on insurance industries)
• HNC (focuses on fraud detection)
• Unica Model 1 (focuses on marketing

Source IT Strategy and Management Sanjiva Dubey 54

7/25/2023 54
Indian Institute of Management Lucknow, Noida campus

27
 Examples 1
• Financial Data Analysis: neural networks have been used in
forecasting stock prices, option trading, rating bonds,
portfolio management, commodity-price prediction, and
mergers and acquisition analysis.
• Using IBM Intelligent minor, Mellon Bank developed a credit
card-attrition model to predict which customers will stop
using Mellon’s credit card in the next few months.

• Telecommunications Industry: The hyper-competitive nature

of the industry has created a need to understand customers,
to keep them, to model effective ways to market new
products.

Source IT Strategy and Management Sanjiva Dubey 55

55
Indian Institute of Management Lucknow, Noida campus

Example 2
• Retail Industry: Retail data mining can help identify
customer-buying behaviors, discover consumer-shopping
patterns and trends.

• Healthcare and biomedical research: The analysis of large

quantities of time-stamped data will provide doctors with
important information regarding the progress of the
decease. For ex., NeuroMedicalSystems used neural
networks to perform a pap smear diagnostic aid.

• Science and engineering: To improve its manufacturing

process. Boeing has successfully applied machine-learning
algorithms to the discovery of informative and useful rules
from its plant data.
Source IT Strategy and Management Sanjiva Dubey 56
7/25/2023 56
Indian Institute of Management Lucknow, Noida campus

28
 What is Big Data ?
 Big data is a popular term used to describe the exponential
growth and availability of data, both structured and unstructured.

 Extremely large data sets that may be analysed computationally

to reveal patterns, trends, and associations, especially relating to
human behaviour and interactions
 Not only VOLUME but Variety and Velocity

 Multi-structured Data
 Web logs
 Social media
 Sensor generated

 Analytics on data is called Big data analytics

Source IT Strategy and Management Sanjiva Dubey 57

7/25/2023 57
Indian Institute of Management Lucknow, Noida campus

How BIG is Big Data?

Statistics Canada, a government agency, last year tried to estimate the value of the
country’s data. The result was between$118bn and $164bn. the value of all the
data in America, whose GDP is 12 times that of Canada, could amount to $1.4trn-
$2trn, which would be nearly 5% of America’s stock of private physical capital.
Unstructured Data
The proportion of firms IDC
according to Forbes that 90 The market research firm
estimates the world will create
don’t have the expertise to 95% Zettabytes this much amount of data
deal with big data. (19 trn DVDs) alone in 2019 & 2020. To put
Hence, majority of the that into perspective, that is
big data solutions are more than the data produced
outsourced. since the advent of computers.

Data warehouse Apache Hadoop

Expected worth of the
optimisation data mining program in
The proportion of companies $99.31
70% 2020.
that feel optimising data billion Imagine the cumulative
warehouse is critical, worth of such programs.
according to Forbes.
Indian Institute of Management Lucknow, Noida campus

29
 11.3 CHARACTERISTICS OF BIG DATA

• Enormous volume: Variety:

• Velocity: Veracity Value:

Source IT Strategy and Management Sanjiva Dubey 59

7/25/2023 59
Indian Institute of Management Lucknow, Noida campus

Learning Pause

• What data we should maintain to prevent the

migrant workers crisis.
• How to collect this data .
• Create one consolidated learning diary
– Index page number

Use databox https://databox.com/

– https://www.youtube.com/watch?v=9uLTv_ALdlU

Source IT Strategy and Management Sanjiva Dubey 60

7/25/2023 60
Indian Institute of Management Lucknow, Noida campus

30