CHAPTER 4 • Variable Representation

DESCRIPTIVE DATA MINING • The data in a data set are often said to
be “dirty” and “raw” before they have
INTRODUCTION
been preprocessed
The increase in the use of data-mining
• We need to put them into a form that is
techniques in business has been caused largely
best suited for a data-mining algorithm
by three events
• Data preparation makes heavy use of
• The explosion in the amount of
the descriptive statistics and data
data being produced and
visualization methods
electronically tracked
TREATMENT OF MISSING DATA
• The ability to electronically
warehouse these data • The primary options for
addressing missing data are:
• The affordability of computer
power to analyze the data • To discard observations
with any missing values
Observation: set of recorded values of variables
associated with a single entity • To discard any variable
with missing values
Data-mining approaches can be separated into
two categories: • To fill in missing entries
with estimated values
• Supervised learning—For
prediction and classification • To apply a data-mining
algorithm (such as
• Unsupervised learning—To
classification and
detect patterns and
regression trees) that
relationships in the data
can handle missing
• Thought of as high- values
dimensional descriptive
• Dealing with missing data requires
analytics
understanding of why the data are
• Designed to describe missing and the impact of the missing
patterns and data
relationships in large
• If the missing value is a random
data sets with many
occurrence, it is called a data value
observations of many
missing completely at random (MCAR)
variables
• If the missing values are not completely
DATA PREPARATION
random (i.e., correlated with the values
• Treatment of Missing Data of some other variables), these are
called missing at random (MAR)
• Identification of Outliers and
Erroneous Data
 • Data is missing not at random (MNAR)
if the reason that the value is missing is
related to the value of the variable
IDENTIFICATION OF OUTLIERS AND ERRONEOUS
DATA
• Examining the variables in the data set
by means of summary statistics,
histograms, PivotTables, scatter plots,
and other tools can uncover data-
quality issues and outliers
• Closer examination of outliers may
reveal an error or a need for further
investigation to determine whether the
observation is relevant to the current
analysis
• A conservative approach is to create
two data sets, one with and one
without outliers, and then construct a
model on both data sets
• If a model’s implications depend on the
inclusion or exclusion of outliers, then
one should spend additional time to
track down the cause of the outliers
VARIABLE REPRESENTATION
• In many data-mining applications, the
number of variables for which data is
recorded may be prohibitive to analyze
• Dimension reduction: Process of
removing variables from the analysis
without losing any crucial information
• One way is to examine pairwise
correlations to detect variables or
groups of variables that may supply
similar information
• Such variables can be aggregated or
removed to allow more parsimonious
model development
• A critical part of data mining is
determining how to represent the
measurements of the variables and
which variables to consider
• The treatment of categorical variables is
particularly important
• Often data sets contain variables that,
considered separately, are not
particularly insightful but that, when
combined as ratios, may represent
important relationships
CLUSTER ANALYSIS
• Measuring Similarity Between
Observations
• Hierarchical Clustering
• k-Means Clustering
• Hierarchical Clustering Versus k-Means
Clustering
• Goal of clustering is to segment
observations into similar groups based
on observed variable
• Can be employed during the data-
preparation step to identify variables or
observations that can be aggregated or
removed from consideration
• Commonly used in marketing to divide
customers into different homogenous
groups; known as market segmentation
• Used to identify outliers
• Clustering methods:
• Bottom- up hierarchical
clustering starts with each
observation belonging to its own
cluster and then sequentially
merges the most similar clusters
to create a series of nested
clusters
• k-means clustering assigns each
observation to one of k clusters in
 a manner such that the customers within a group are
observations assigned to the similar and dissimilar with
same cluster are as similar as respect to key characteristics
possible
• For each customer, KTC has
• Both methods depend on how two corresponding to a vector of
observations are similar—hence, we measurements on seven
have to measure similarity between customer variables, that is,
observations (Age, Female, Income, Married,
Children, Car Loan, Mortgage)
MEASURING SIMILARITY BETWEEN
OBSERVATIONS Example: The observation u = (61, 0, 57881, 1,
2, 0, 0) corresponds to a 61-year-old male with
• When observations include
an annual income of $57,881, married with two
numeric variables, Euclidean
children, but no car loan and no mortgage
distance is the most common
method to measure Figure 4.1: Euclidean Distance
dissimilarity between
Euclidean distance becomes smaller as a pair of
observations
observations become more similar with respect
• Euclidean distance: Most to their variable values
common method to measure
dissimilarity between
observations, when
observations include
continuous variables

• Let observations u = (u1, u2, . . . ,

uq) and v = (v1, v2, . . . , vq) each
comprise measurements of q
variables

• The Euclidean distance between

observations u and v is:
• Euclidean distance is highly influenced
2 2 2
√
d u , v = ( u1−v 1 ) + ( u2−v 2 ) +∙ ∙ ∙+ ( uq −v q ) by the scale on which variables are
measured

• Common to standardize the

ILLUSTRATION:
• KTC is a financial advising
company that provides
personalized financial advice to
its clients
• KTC would like to segment its
customers into several groups
(or clusters) so that the
units of each variable j of each
observation u
• Example: uj, the value of
variable j in observation u, is
replaced with its z-score, zj
• The conversion to z-scores also makes it
easier to identify outlier measurements,
 which can distort the Euclidean distance
between observations
• When clustering observations solely on
the basis of categorical variables
encoded as 0–1, a better measure of
similarity between two observations
can be achieved by counting the
number of variables with matching
values
• The simplest overlap measure is called
the matching coefficient and is
computed by:
• A weakness of the matching coefficient
is that if two observations both have a 0
entry for a categorical variable, this is
counted as a sign of similarity between
the two observations
• Determines the similarity of
two clusters by considering the
similarity between the
observations composing either
cluster
• Starts with each observation in
its own cluster and then
iteratively combines the two
clusters that are the most
similar into a single cluster
• Given a way to measure
similarity between
observations, there are several
clustering method alternatives
for comparing observations in
two clusters to obtain a cluster
similarity measure
• Single linkage
• Complete linkage

• To avoid misstating similarity due to the • Group average linkage

absence of a feature, a similarity
• Median linkage
measure called Jaccard’s coefficient
does not count matching zero entries
and is computer by:

Table 4.1: Comparison of Similarity Matrixes for

Observations with Binary Variables

Figure 4.2:
Measuring Similarity Between Clusters

HIERARCHICAL CLUSTERING
 K-MEANS CLUSTERING

• Given a value of k, the k-means

algorithm randomly partitions
the observations into k clusters

• After all observations have been

assigned to a cluster, the
resulting cluster centroids are
calculated

• Using the updated cluster

• Centroid linkage uses the averaging
concept of cluster centroids to define
between-cluster similarity
centroids, all observations are
reassigned to the cluster with
the closest centroid
Figure 4.4: Clustering Observations by Age and
Income Using k-Means Clustering with k = 3

• Ward’s method merges two clusters

such that the dissimilarity of the
observations with the resulting single
cluster increases as little as possible

• When McQuitty’s method considers

merging two clusters A and B, the
dissimilarity of the resulting cluster AB
to any other cluster C is calculate as:
((dissimilarity between A and C) + Table 4.2: Average Distances within Clusters
(dissimilarity between B and C))/2) Table 4.3: Distances Between Cluster Centroids
• A dendrogram is a chart that depicts
the set of nested clusters resulting at
each step of aggregation

Figure 4.3: Dendrogram for KTC

HIERARCHICAL CLUSTERING VERSUS K-MEANS

CLUSTERING
ASSOCIATION RULES EVALUATING ASSOCIATION RULES

• Evaluating Association Rules • An association rule is ultimately judged

on how actionable it is and how well it
• Association rules: if-then statements
explains the relationship between item
which convey the likelihood of certain
sets
items being purchased together
• For example, Wal-Mart mined its
• Antecedent: the collection of items (or
transactional data to uncover strong
item set) corresponding to the if portion
evidence of the association rule, “If a
of the rule
customer purchases a Barbie doll, then
• Although association rules are an a customer also purchases a candy bar”
important tool in market basket
• An association rule is useful if it is well
analysis
supported and explain an important
• Consequent: the item set previously unknown relationship
corresponding to the then portion of
Table 4.5: Association Rules for Hy-Vee
the rule

• Support count of an item: number of

transactions in the data that include
that item set

Table 4.4: Shopping-Cart Transactions

• Confidence: Helps identify reliable

association rules

• Lift ratio: Measure to evaluate the

efficiency of a rule

• For the data in Table 4.4, the rule “if

{bread, jelly}, then {peanut butter}” has
confidence = 2/4 = 0.5 and a lift ratio =
0.5/(4/10) = 1.25