Professional Documents
Culture Documents
Discriminant Analysis For Classification and Prediction
Discriminant Analysis For Classification and Prediction
Similarities
Number of One One One
dependent
variables
Number of
independent Multiple Multiple Multiple
variables
Differences
Nature of the
dependent Metric Metric Categorical
variables
Nature of the
independent Categorical Metric Metric
variables
The objective of discriminant Analysi
s
Development of discriminant functions, or linear c
ombinations of the independent variables, that will
best discriminate between the categories of the de
pendent variable( group ).
Examination of whether significant differences exi
Scatter plot of data that appears to be from two
groups
.
Cutting
score=ZCE
Group A Group
B
ZA ZB
Classify as
Classify as B
A (Purchaser
(Nonpurch )ZA
aser)
- - 0 N=3 4
4 2 8
Plot of Group Centroids (Z)
The overlap between A and B
is smaller than any other line
drawn through the scatter
plot. Z axis represents the two
variable profiles of A and B as
single numbers. By finding a
linear composites of the
original profile scores we can
potray each profile as a point
in the line. Thus Z axis
“condenses” the information
about group separability into a
set of points on the Z axis.Z-
A
axis is the discriminant AXIS.
V
B 1
Discriminant Function
Z
Two-Group Discriminant
Analysis
XXOXOOO
XXXOXXOOOO
Price
XXXXOOOXOOO
Sensitivity XXOXXOXOOOO
XXOXOOOOOOO
O-segment
x = high propensity to buy
o = low propensity to buy
Evaluation of new product
*(1=very poor, 10=excellent)
Group based on purchase X1 X2 X3
intention Durability Performance Style
Group I: Would purchase
Subject 1 8 9 6
Subject 2 6 7 5
Subject 3 10 6 3
Subject 4 9 4 4
Subject 5 4 8 2
Group Mean 7.4 6.8 4.0
Group II: Would not
purchase
Subject 6 5 4 7
Subject 7 3 7 2
Subject 8 4 5 5
Subject 9 2 4 3
Subject 10 2 2 2
Group mean 3.2 4.4 3.8
Difference between 4.2 2.4 0.2
group means
1 8
0
X1 9 7 5 6 2 1 4 3
Durabili
ty
6 7
X2 5 1
Durabili 1 4 8 3 2
ty 0
1
0
7 9 8
X3
5 3 4 2 1 6
Style
Consumer Evaluations (Like Versus Dislike) of Ten Cereals Varying in Nutritional Content
Evaluation X1 X2
Protein Vitamin D
Group I: Dislike
Person 1 2 4
Person 2 3 2
Person 3 4 5
Person 4 5 4
Person 5 6 7
Group Mean 4.0 4.4
Group II: Dislike
Person 6 7 6
Person 7 8 4
Person 8 9 7
Person 9 10 6
Person10 11 9
Group mean 9.0 6.4
12 Disliker
s
Likers
10
8
ta
m
Vi
in
X
2
6
4
2
0
4 8 10 12
2 6
Protein, X2
Centroid(Liker)
10
Total sample
centroid
Centroid
(Liker)
*
1
0
.824
1.596 2.368
Discriminant Analysis Model
The Discriminant Analysis Model involves linear combinati
ons of the following form :
D = b 0 + b 1 X 1 + b 2 X 2 + b 3 X3 + . . . + b k X k
D = discriminant score
b = discriminant coefficient or weight
X = independent variable.
Statistics Associated with Discriminant Analys
is
Canonical correlation : measures the extent of associati
on between the discriminant scores and the groups. It is a
measure of association between the single discriminant fu
nction and the set of dummy variables that define the gro
up membership.
Centroid : is the mean values for the discriminant scores
for a particular group. There are as many centroids as ther
e are groups, as there is one for each group. The means fo
r a group on all the functions are the group centroids.
Classification matrix : sometimes also called confusion
matrix, the classification matrix contains the number of co
rrectly classified and mis-classified cases.
Discriminant function coefficients( unstandardized ) :
are the multipliers of variables, when the variables are in t
he original units of measurement.
Discriminant scores : The unstandardized coefficients ar
e multiplied by the values of the variables. These products
are summed and added to the content term to obtain the
discriminant scores.
Eigenvalues : For each discriminant function, the eigenva
lue is the ratio of between-group to within-group sums of
squares.
F values and their significance : These are calculated fr
om a one-way ANOVA, with the grouping variable serving
as the categorical independent variable. Each predictor, in
turn, serves as the metric dependent variable in the ANOV
A.
Group means and group stand deviations : These are
computed for each predictor for each group.
Pooled within-group correlation matrix : is computed
by averaging the separate covariance matrices for all the
groups.
Standardized discriminant function coefficients : use
d as the multipliers when the variables have been standar
dized to a mean of 0 and a variance of 1.
Structure correlations : Also referred to as discriminant
loadings, the structure correlation represent the simple co
rrelations between the predictors and the discriminant fun
ction.
Total correlation matrix : If the cases are treated as if
they were from a single sample and the correlations comp
uted, a total correlation matrix is obtained.
Analysis Sample : part of the total sample that is u
sed for estimation of the discriminant function.
Validation Sample : That part of the total sample
used to check the results of the estimation sample.
Direct Method : An approach to discriminant analy
sis that involves estimating the discriminant function
so that all the predictors are included simultaneousl
y.
D = w1Z1 + w2Z2 + w3Z3 + .... wiZi
D = discriminant score;
w i = weighting (coefficient) for variable i;
Z i = standardised score for variable i ( a standardised
variable has a mean of 0 and a standard deviation of 1 ).
Stepwise Discriminant Analysis : Discriminant analysis
in which the predictors are entered sequentially based on
their ability to discriminate between the groups.
Territorial Map : A tool for assessing discriminant analys
is results by plotting the graph membership of each case
on a graph.
Conducting Discriminant Analysis
There are 5 steps procedure as following :
1.Formulating the discriminant problem:
Indentification of the dependent and independent variable
s.
The sample is divided into two parts. One part, the analysis
sample, is used to estimate the discriminant function.
The other part, the holdout sample, is used for validation.
2. Estimation:
developing a linear combination of theindependent
variables, called discriminant functions, so that the groups
differ as much as possible on the values of independent
variables.
3. Determination of Statistical significance
testing the null hypothesis that, in the population, the mea
ns of all discriminant functions in all groups are equal.
If the null hypothesis is rejected, it is meaningful to interpr
et the results.
4. The interpretation of discriminant weights or coefficients
similar to that in multiple regression analysis.
The relative importance of the variables may be
understood by examining the absolute magnitude of the st
andardized discriminant function coefficients
also by examining the structure correlations or discriminant
loading.
The simple correlations between each independent and the
discriminant function represent the variance that the
independent shares with the function.
5. Validation:
developing the classification matrix.
The discriminant weights estimated by using the ana
lysis sample are multiplied by the values of the
independent variables in the hold-out sample.
The cases are then assigned to groups based on thei
r discriminant scores and an appropriate decision rul
e.
The percentage of cases correctly classified is deter
mined and compared to the rate that would be expe
cted by chance classification.
Steps for Statistical Interpretation:
Step1:How Good is this model?
Interpretation from the output table “Classification Matrix”
Step2:
Statistical significance of the discriminant functions
This can be answered by looking Wilks’ Lambda (proportio
n of discriminant score not explained by difference amo
ng group)and Probability value of F-test.The value of Wi
lk’s Lambda ranges between 0 and 1 with a lower value
indicating better discriminating power of the model.
Step3:Which Variables are relatively better in discrimin
ating between consumers for national and local brand?
From the output table for “Standardised coefficients for ca
nonical variables’ compare the absolute value of standa
rdised coefficients of each independent variable
Steps for Statistical Interpretation:
1 1 50.2 5 8 3 43 M (2)
2 1 70.3 6 7 4 61 H (3)
3 1 62.9 7 5 6 52 H (3)
4 1 48.5 7 5 5 36 L (1)
5 1 52.7 6 6 4 55 H (3)
6 1 75.0 8 7 5 68 H (3)
7 1 46.2 5 3 3 62 M (2)
8 1 57.0 2 4 6 51 M (2)
9 1 64.1 7 5 4 57 H (3)
10 1 68.1 7 6 5 45 H (3)
11 1 73.4 6 7 5 44 H (3)
12 1 71.9 5 8 4 64 H (3)
13 1 56.2 1 8 6 54 M (2)
14 1 49.3 4 2 3 56 H (3)
15 1 62.0 5 6 2 58 H (3)
Information on Resort Visits: Analysis
Sample
Table 18.2 cont.
16 2 32.1 5 4 3 58 L (1)
17 2 36.2 4 3 2 55 L (1)
18 2 43.2 2 5 2 57 M (2)
19 2 50.4 5 2 4 37 M (2)
20 2 44.1 6 6 3 42 M (2)
21 2 38.3 6 6 2 45 L (1)
22 2 55.0 1 2 2 57 M (2)
23 2 46.1 3 5 3 51 L (1)
24 2 35.0 6 4 5 64 L (1)
25 2 37.3 2 7 4 54 L (1)
26 2 41.8 5 1 3 56 M (2)
27 2 57.0 8 3 2 36 M (2)
28 2 33.4 6 8 2 50 L (1)
29 2 37.5 3 2 3 48 L (1)
30 2 41.3 3 3 2 42 L (1)
Results of Two-Group Discriminant
Analysis
Table 18.4
GROUP MEANS
VISIT INCOME TRAVEL VACATION HSIZE AGE
1 60.52000 5.40000 5.80000 4.33333 53.73333
2 41.91333 4.33333 4.06667 2.80000 50.13333
Total 51.21667 4.86667 4.9333 3.56667 51.93333
Group Standard Deviations
1 9.83065 1.91982 1.82052 1.23443 8.77062
2 7.55115 1.95180 2.05171 .94112 8.27101
Total 12.79523 1.97804 2.09981 1.33089 8.57395
Pooled Within-Groups Correlation Matrix
INCOME TRAVEL VACATION HSIZE AGE
INCOME 1.00000
TRAVEL 0.19745 1.00000
VACATION 0.09148 0.08434 1.00000
HSIZE 0.08887 -0.01681 0.07046 1.00000
AGE - 0.01431 -0.19709 0.01742 -0.04301 1.00000
Wilks' (U-statistic) and univariate F ratio with 1 and 28 degrees of freedom
Variable Wilks' F Significance
INCOME 0.45310 33.800 0.0000
TRAVEL 0.92479 2.277 0.1425
VACATION 0.82377 5.990 0.0209
HSIZE 0.65672 14.640 0.0007
AGE 0.95441 1.338 0.2572
Contd.
Results of Two-Group Discriminant
Analysis
Table 18.4 cont.
CANONICAL DISCRIMINANT FUNCTIONS
% of Cum Canonical After Wilks'
Function Eigenvalue Variance % Correlation Function Chi-square df Significance
: 0 0 .3589 26.130 5 0.0001
1* 1.7862 100.00 100.00 0.8007 :
* marks the 1 canonical discriminant functions remaining in the analysis.
Standard Canonical Discriminant Function Coefficients
FUNC 1
INCOME 0.74301
TRAVEL 0.09611
VACATION 0.23329
HSIZE 0.46911
AGE 0.20922
Structure Matrix:
Pooled within-groups correlations between discriminating variables & canonical discriminant functions (variables ordered by size of
correlation within function)
FUNC 1
INCOME 0.82202
HSIZE 0.54096
VACATION 0.34607
TRAVEL 0.21337
AGE 0.16354
Contd.
Results of Two-Group Discriminant
Analysis
Table 18.4 cont.
Contd.
Results of Two-Group Discriminant
Analysis
Table 18.4 cont.
Classification Results for cases not selected for use in the analysis (holdout sample)
Predicted Group Membership
Actual Group No. of Cases 1 2
Group 1 6 4 2
66.7% 33.3%
Group 2 6 0 6
0.0% 100.0%
Percent of grouped cases correctly classified: 83.33%.
Models with More than one Group
For two group model above, Discriminant
analysis found best projection of plot
points (cases) for separating the groups
and introduced Linear Discriminant
Function.
When there are more than two groups,
Discriminant Function become the focus
of Analysis.
Models with More than one Group
The first Discriminant Function is the Linear
combination of the variables that maximizes the
differences between the means of k groups in
one dimension.
The second Discriminant Function represents the
maximum dispersion of the means in a direction
orthogonal (perpendicular) to the first function.
Third represents the disperson in a dimension
independent of the first two dimensions.
DF’s are factors that discriminate optimally
among group centroids relative to disperson
within Group.
Group based on Switching Intention X1 X2 Service Level
Price Competitiveness
Group I: Definitely switch
Subject 1 2 2
Subject 2 1 2
Subject 3 3 2
Subject 4 2 1
Subject 5 2 3
Group Mean 2.0 2.0
Group II: undecided
Subject 6 4 2
Subject 7 4 3
Subject 8 5 1
Subject 9 5 2
Subject 10 5 3
Group mean 4.6 2.2
Group III: Definitely not switch
Subject 11 2 6
Subject 12 3 6
Subject 13 4 6
Subject 14 5 6
Subject 115 5 7
Group mean 3.8 6.2
1
5
1
1
1 1
1 4
1 1
1 1 1
5
5 3 11
X2 1
1 00
1 7 19
4 2 1
1 00
1
2
1 3 6 8
1
4
19
1
15
4
1 1
1
7 10
3
1 2 10
1
1 1
1 1
X1 1 1 1 1
2 11 2 6
3 8
4
5
|_______|_______|_______|______|_______|_______|
1 1 1 1
_______| 1 1 1 1
0 1 2 3 4 5 6
7
Discriminant Function 2
1
7 5
1
1
6 1
1
1
2
1
3
1
4
1 1 1 1
1 1 1 1
5
3 5 7 1
0
2
2 1 3 6 9
1 8
4
Discrimiant Function 1
Group Means
AMOUNT INCOME TRAVEL VACATION HSIZE AGE
INCOME 1.00000
TRAVEL 0.05120 1.00000
VACATION 0.30681 0.03588 1.00000
HSIZE 0.38050 0.00474 0.22080 1.00000
AGE -0.20939 -0.34022 -0.01326 -0.02512 1.00000
Contd.
Results of Three-Group Discriminant
Analysis
Table 18.5 cont.
FUNC 1 FUNC 2
INCOME 1.04740 -0.42076
TRAVEL 0.33991 0.76851
VACATION -0.14198 0.53354
HSIZE -0.16317 0.12932
AGE 0.49474 0.52447 Contd.
Results of Three-Group Discriminant
Analysis
Table 18.5 cont.
Structure Matrix:
Pooled within-groups correlations between discriminating variables and canonical discriminant
functions (variables ordered by size of correlation within function)
FUNC 1 FUNC 2
INCOME 0.85556* -0.27833
HSIZE 0.19319* 0.07749
VACATION 0.21935 0.58829*
TRAVEL 0.14899 0.45362*
AGE 0.16576 0.34079*
Group 1 10 9 1 0
90.0% 10.0% 0.0%
Group 2 10 1 9 0
10.0% 90.0% 0.0%
Group 3 10 0 2 8
0.0% 20.0% 80.0%
Percent of grouped cases correctly classified: 86.67%
Classification results for cases not selected for use in the analysis
Predicted Group Membership
Actual Group No. of Cases 1 2 3
Group 1 4 3 1 0
75.0% 25.0% 0.0%
Group 2 4 0 3 1
0.0% 75.0% 25.0%
Group 3 4 1 0 3
25.0% 0.0% 75.0%
Percent of grouped cases correctly classified: 75.00%
All-Groups
Fig. 18.2 Scattergram
Across: Function 1
Down: Function 2
4.0
1 1
1 *1 3
23 3 *3 3
1 1 12 * 3 3
0.0 1 1 2 2
3
1 2 2
2
-4.0
* indicates a group
centroid
Taste
Cadbury’s
Price
Nestle
0
-1 3
-2 1 2
0
Packaging Availability
-0.5
Quality
1
Amul -1.5
Interpretation
Internet
0.5
Fuzzy
Colour
Weight
Remote
0.5
Audio
Sansug Price
Perceptual Map of Lipstick Brands and Attributes
2.00
Lakme
1.50
Packaging Maybelline
1.00
Finish Price
0.50
0.00
-0.50
Long lasting
colour
-1.00
-1.50
-2.00
Revion
-2.50
Dimension 1
1.5
B4
1
A1
A3 A6
0.5
A4 B2
0
A5
-4 -3 B1 -1 0 1
-2 2
A2
-0.5
-1
B3
-1.5
Stepwise Discriminant Analysis
Stepwise discriminant analysis is analogous to stepwise
multiple regression (see Chapter 17) in that the
predictors are entered sequentially based on their ability
to discriminate between the groups.
An F ratio is calculated for each predictor by conducting
a univariate analysis of variance in which the groups are
treated as the categorical variable and the predictor as
the criterion variable.
The predictor with the highest F ratio is the first to be
selected for inclusion in the discriminant function, if it
meets certain significance and tolerance criteria.
A second predictor is added based on the highest
adjusted or partial F ratio, taking into account the
predictor already selected.
Stepwise Discriminant Analysis
Each predictor selected is tested for retention based on
its association with other predictors selected.
The process of selection and retention is continued until
all predictors meeting the significance criteria for
inclusion and retention have been entered in the
discriminant function.
The selection of the stepwise procedure is based on the
optimizing criterion adopted. The Mahalanobis
procedure is based on maximizing a generalized
measure of the distance between the two closest
groups.
The order in which the variables were selected also
indicates their importance in discriminating between the
groups.
Export Data set
Y1= Willingness to Export
X1=Employee Size
x2= Firm Revenue
X3= Years of Operation in the Domestic
Market
X4=Number of Products
X5=Training of Employees
X6=Management Experience in
International Operations