Professional Documents
Culture Documents
INS2061 Introductions
INS2061 Introductions
INS2061 Introductions
& BUSINESS
ANALYTICS
INS2061
Course info
• Module code: INS2061
• Credit: 3
• Grades:
• Attendances: 10%
• Midterm: 30% - Projects
• Final: 60% - Written
• Textbooks:
• Introduction to Data Mining, Pang-Ning Tan, Michael Steinbach, and Vipin Kumar, Pearson
Publisher, 2ndedition, 2018, ISBN: 0133128903
• DATA MINING FOR BUSINESS ANALYTICS. Concepts, Techniques, and Applications in Python.
Galit Shmueli. Peter C. Bruce. Inbal Yahav. Nitin R. Patel. Kenneth C. Lichtendahl Jr., 1st edition,
2020, Wiley publisher, ISBN-10: 1118879368
• Programming language: Python
• Tools: Jupyter notebook, Google Colab, Pycharm, ..
• Topic 1: Introduction to data mining
• Topic 2: Data mining with the Python language
• Topic 3: Classification: Nearest Neighbor
• Topic 4: Classification: Logistic Regression
• Topic 5: Association rules mining
• Topic 6: Unsupervised Learning (clustering with K-
Course means, HAC)
• Topic 7: Classification: Advanced Predictive Model
outlines (ANN: Perceptron, MLP, FFNN, CNN)
• Topic 8: Model evaluation and Fine-tuning
• Topic 9: Advanced topics on data mining:
Computer vision
• Topic 10: Results communication and Model
Deployment (Fast API)
Introduction to data mining
Data is Everywhere!
§ There has been enormous data
growth in both commercial and
scientific databases due to
advances in data generation
and collection technologies
E-Commerce
§ New mantra Cyber Security
• Description Methods
• Find human-interpretable patterns that describe the data.
Data mining tasks
Clu
ste Data
ring ng
Tid Refund Marital Taxable
Status Income Cheat
li
1 Yes Single 125K No
ode
2 No Married 100K No
e M
tiv
3 No Single 70K No
c
4 Yes Married 120K No
i
ed
5 No Divorced 95K Yes
Pr
6 No Married 60K No
7 Yes Divorced 220K No
8 No Single 85K Yes
9 No Married 75K No
An
10 No Single 90K Yes
De oma
11 No Married 60K No
i tec ly
soc
13 No Single 85K Yes
As 14 No Married 75K No
tio
n
les
15 No Single 90K Yes
u
10
Milk
Predictive Modeling: Classification
Model for predicting credit
worthiness
Class Employed
# years at
Level of Credit Yes
Tid Employed present No
Education Worthy
address
1 Yes Graduate 5 Yes
2 Yes High School 2 No No Education
3 No Undergrad 1 No
{ High school,
4 Yes High School 10 Yes Graduate
Undergrad }
… … … … …
10
Number of Number of
years years
Yes No Yes No
Classifications: Example
l l ive
ir ca ir ca t # years at
go go t ita Tid Employed
Level of
present
Credit
ate ate uan lass Education
address
Worthy
c c q c 1 Yes Undergrad 7 ?
# years at 2 No Graduate 3 ?
Level of Credit
Tid Employed present 3 Yes High School 2 ?
Education Worthy
address
… … … … …
1 Yes Graduate 5 Yes 10
Set
Learn
Training Model
Set Classifier
Classfications
Use of K-means to
partition Sea Surface
60
Land Cluster 2
Ice or No NPP
(NPP) into clusters that
-30
reflect the Northern and
Sea Cluster 2 Southern Hemispheres.
-60
Sea Cluster 1
-90
-180 -150 -120 -90 -60 -30 0 30 60 90 120 150 180
Cluster
longitude
Clustering applications
• Market Segmentation:
• Goal: subdivide a market into distinct subsets of customers where
any subset may conceivably be selected as a market target to be
reached with a distinct marketing mix.
• Approach:
• Collect different attributes of customers based on their geographical and
lifestyle related information.
• Find clusters of similar customers.
• Measure the clustering quality by observing buying patterns of customers in
same cluster vs. those from different clusters.
Clustering applications
• Document Clustering:
• Goal: To find groups of documents that are similar to each other
based on the important terms appearing in them.
• Approach: To identify frequently occurring terms in each document.
Form a similarity measure based on the frequencies of different
terms. Use it to cluster.
• Medical Informatics
• Rules are used to find combination of patient symptoms and test results
associated with certain diseases
Deviation/anomaly/change detection
• Detect significant deviations from normal behavior
• Applications:
• Credit Card Fraud Detection
• Network Intrusion
Detection
• Identify anomalous behavior from sensor networks for monitoring and surveillance.
• Detecting changes in the global forest cover.
What is data Attributes
Objects
• Attribute is also known as variable, field, 4 Yes Married 120K No
characteristic, dimension, or feature
5 No Divorced 95K Yes
• A collection of attributes describe an
6 No Married 60K No
object
• Object is also known as record, point, 7 Yes Divorced 220K No
case, sample, entity, or instance 8 No Single 85K Yes
9 No Married 75K No
10 No Single 90K Yes
10
Attribute values
• Attribute values are numbers or symbols assigned to an
attribute for a particular object
• Sparsity
• Only presence counts
• Resolution
• Patterns depend on the scale
• Size
• Type of analysis may depend on size of data
Type of datasets
• Record
• Data Matrix
• Document Data
• Transaction Data
• Graph
• World Wide Web
• Molecular Structures
• Ordered
• Spatial Data
• Temporal Data
• Sequential Data
• Genetic Sequence Data
Record data
• Data that consists of a collection of records, each of which
consists of a fixed set of attributes
Tid Refund Marital Taxable
Status Income Cheat
timeout
season
coach
game
score
play
team
win
ball
lost
Document 1 3 0 5 0 2 6 0 2 0 2
Document 2 0 7 0 2 1 0 0 3 0 0
Document 3 0 1 0 0 1 2 2 0 3 0
Transaction data
• A special type of data, where
• Each transaction involves a set of items.
• For example, consider a grocery store. The set of products purchased by a
customer during one shopping trip constitute a transaction, while the individual
products that were purchased are the items.
• Can represent transaction data as record data
TID Items
1 Bread, Coke, Milk
2 Beer, Bread
3 Beer, Coke, Diaper, Milk
4 Beer, Bread, Diaper, Milk
5 Coke, Diaper, Milk
Graph data
2
5 1
2
5
An element of
the sequence
Ordered data
• Genomic sequence data
GGTTCCGCCTTCAGCCCCGCGCC
CGCAGGGCCCGCCCCGCGCCGTC
GAGAAGGGCCCGCCTGGCGGGCG
GGGGGAGGCGGGGCCGCCCGAGC
CCAACCGAGTCCGACCAGGTGCC
CCCTCTGCTCGGCCTAGACCTGA
GCTCATTAGGCGGCAGCGGACAG
GCCAAGTAGAACACGCGAAGCGC
TGGGCTGCCTGCTGCGACCAGGG
Ordered data
• Spatio-Temporal Data
Average Monthly
Temperature of
land and ocean
Data Quality
• Poor data quality negatively affects many data processing efforts
• Data mining example: a classification model for detecting people who are loan
risks is built using poor data
• Some credit-worthy candidates are denied loans
• More loans are given to individuals that default
• Examples of data quality problems:
• Noise and outliers
• Wrong data
• Fake data
• Missing values
• Duplicate data
Noise
Outliers
Missing values
• Reasons for missing values
• Information is not collected
(e.g., people decline to give their age and weight)
• Attributes may not be applicable to all cases
(e.g., annual income is not applicable to children)
• Examples:
• Same person with multiple email addresses
• Data cleaning
• Process of dealing with duplicate data issues
• r = 2. Euclidean distance
• Do not confuse r with n, i.e., all these distances are defined for all numbers of
dimensions.
Properties of distances
• Distances, such as the Euclidean distance, have some well
known properties.
1. d(x, y) ³ 0 for all x and y and d(x, y) = 0 if and only if x = y.
2. d(x, y) = d(y, x) for all x and y. (Symmetry)
3. d(x, z) £ d(x, y) + d(y, z) for all points x, y, and z.
(Triangle Inequality)
where d(x, y) is the distance (dissimilarity) between points (data
objects), x and y.
yi = xi2
• mean(x) = 0, mean(y) = 4
• std(x) = 2.16, std(y) = 3.74
𝐻 𝑋, 𝑌 = − + + 𝑝𝑖𝑗log # 𝑝𝑖𝑗
! "
Where pij is the probability that the ith value of X and the jth value of Y occur together
Mutual information of Student Status and Grade = 0.9928 + 1.4406 - 2.2710 = 0.1624
• Applies mutual information to two continuous variables
• Consider the possible binnings of the variables into discrete
categories
• nX × nY ≤ N0.6 where
• nX is the number of values of X
• nY is the number of values of Y
• N is the number of samples (observations, data objects)
• Compute the mutual information
• Normalized by log2(min( nX, nY )
• Take the highest value
• Reshef, David N., Yakir A. Reshef, Hilary K. Finucane, Sharon R. Grossman, Gilean McVean, Peter J. Turnbaugh, Eric S. Lander, Michael
Mitzenmacher, and Pardis C. Sabeti. "Detecting novel associations in large data sets." science 334, no. 6062 (2011): 1518-1524.
Data preprocessing
• Aggregation
• Sampling
• Discretization and Binarization
• Attribute Transformation
• Dimensionality Reduction
• Feature subset selection
• Feature creation
Aggregation
• Combining two or more attributes (or objects) into a single attribute (or object)
• Purpose
• Data reduction - reduce the number of attributes or objects
• Change of scale
• Cities aggregated into regions, states, countries, etc.
• Days aggregated into weeks, months, or years
• More “stable” data - aggregated data tends to have less variability
Sampling
• Sampling is the main technique employed for data reduction.
• It is often used for both the preliminary investigation of the data and
the final data analysis.
• Using a sample will work almost as well as using the entire data set, if
the sample is representative
• Definitions of density
and distance between
points, which are
critical for clustering
and outlier detection,
become less
meaningful • Randomly generate 500 points
• Compute difference between max and
min distance between any pair of points
Dimension reduction
• Purpose:
• Avoid curse of dimensionality
• Reduce amount of time and memory required by data mining algorithms
• Allow data to be more easily visualized
• May help to eliminate irrelevant features or reduce noise
• Techniques
• Principal Components Analysis (PCA)
• Singular Value Decomposition
• Others: supervised and non-linear techniques
Dimension reduction: PCA
• Goal is to find a projection that captures the largest amount
of variation in data x2
x1
Feature subset selection
• Another way to reduce dimensionality of data
• Redundant features
• Duplicate much or all of the information contained in one or more other
attributes
• Example: purchase price of a product and the amount of sales tax paid
• Irrelevant features
• Contain no information that is useful for the data mining task at hand
• Example: students' ID is often irrelevant to the task of predicting students'
GPA
• Many techniques developed, especially for classification