591
CHAPTER 20 Clueter Analyis
hierarchical clustering
A
clustering procedure character
ized by the development of a hierar
chy or tree-like structure.
agglomerative clustering
Hierarchical clustering procedure
where each object starts out in a
separate cluster. Clusters are formed
by grouping objects into bigger and
bigger clusters.
divisive chusteringR
Hierarchical clustering procedure
where all objects start out in one
giant cluster. Clusters are formed by
dividing this cluster into smaller and
smaller clusters.
linkage methods
Agglomerative methods of hierar
chical clustering that cluster objects
based on a computation of the dis
tance between them.
single linkage
Linkage method that is based on
minimum distance or the nearest
neighbor rule.
Figure 20.4
A
Classification of Clustering
Procedures
Hierarchical
Agglomerative
Linkage
Methods
Single
Linkage
he variables are measured in vastly different units, the clustering solution wi e
nTiuenced by the units of measurement. In a supermarket shopping study, attituainal
variables may be measured on a nine-point Likert-type scale: patronage, in ters
frequency of visits per month and the dollar amount spent; and brand loyalty. In teis
Pereentage of grocery shopping expenditure allocated to the favorite supermarket. In these
Cases, before clustering respondents. we must standardize the data by rescaling eacn
Vanable to have a mean of zero and a standard deviation of unity. Although standardization
Can remove the influence of the unit of measurement. it can also reduce the differences
oetween groups on variables that may best discriminate groups or clusters. It 1s also
desirable to eliminate outliers (cases with atypical values).'o
results. Hence, it is
OSe of different distance measures may lead to different clustering
advisable to use different measures and compare the results. Having selected a distance or
similarity measure, we can next select a clustering procedure.
Select a Clustering Procedure
Figure 20.4 is a classification of clustering procedures. Clustering procedures can be
hierarchical or nonhierarchical. Hierarchical clustering is characterized by the develop
or
ment of a hierarchy or tree-like structure. Hierarchical methods can be agglomerative
divisive. Agglomerative clustering starts with each obiect in a separate cluster. Clusters
are formed by grouping objects into bigger and bigger clusters. This process is continued
until all objects are members of a single cluster. Divisive clustering starts with all the
is in a
objects grouped in a single cluster. Clusters are divided or split until each object
separate cluster.
Agglomerative methods are commonly used in marketing research. They consist of
linkage methods, error sums of squares or variance methods, and centroid methods.
Linkage methods include single linkage, complete linkage, and average linkage. The
single linkage method is based on minimum distance or the nearest neighbor rule. The first
Chst
Nonhierarchical
Divisive
Sequential Parallel Optimizing
Threshold Threshold Partitioning
Variance Centroid
Methods Methods
Ward's
Method
Complete Average
Linkage Linkage
 592
Figure 20.5
Linkage Methods of Clustering
complete linkage
Linkage method that is based on
maximum distance or the furthest
neighbor approach.
arerage linkage
A
linkage method based on the
average distance between all pairs
of objects, where one member of the
pair is from each of the clusters.
variance methods
An agglomerative method of hierar
chicalclustering in which clusters
are generated to minimize the
within-cluster variance.
Ward's procedure
Variance method in which the
squared euclidean distance to the
cluster means is minimized.
centroid methods
A variance method of hierarchical
clustering in which the distance
between two clusters is the distance
between their centroids (means for
all the variables).
nonhierarchical custering
A
procedure that first assigns or
determines a cluster center and then
groups all objects within a prespeci
fied threshold value from the center.
sequential threshold method
A nonhierarchical clustering method
in which acluster center is selected
and all objects within a prespecified
threshold value from the center are
grouped together.
parallel threshold method
Nonhierarchical clustering method
that specifies several cluster centers
at once. All objects that are within a
prespecified threshold value from
the center are grouped together.
PART II Dala Collection, Prparutim, Analuis, and Reporting
Single Linkage
Minimunm
Distance
Cluster| Cluster 2
Complete Linkage
Maximum
Distance
Cluster l Cluster 2
Average Linkage
Average
Distance
Cluster! Cluster2
twoobjects clustered are those that have the smallest distance between them. The next
shortest distance is identified, and either the third object is clustered with the first two, or a
new two-object cluster is formed. At every stage, the distance between two clusters is the
distance between their two closest points (see Figure 20.5). Two clusters are merged at any
stage by the single shortest link between them. This process is continued until allobjects
are in one cluster. The single linkage method does not work well when the clusters ae
poorly defined. The complete linkage method is similar to single linkage. except that it is
based on the maximum distance or the furthest neighbor approach. In complete inkage.
the distance between two clusters is calculated as the distance between their two furthest
points. The average linkage method works similarly. However, in this method. the
distance between two clusters is defined as the average of the distances between all pairs
of objects, where one member of the pair is from each of the clusters (Figure 20.5). As can
be seen, the average linkage method uses information on allpairs of distances, not merely
the minimum or maximum distances. For this reason, it is usually preferred to the singe
and complete linkage methods.
The variance methods attempt to generate clustersto minimize the within-cluster
variance. Acommonly used variance method is the Ward's procedure. For each cluster, he
means for all the variables are computed. Then, for each object, the squared euclidean
distance to the cluster means is calculated (Figure 20.6). These distances are summed tor
all the objects. At each stage, the two clusters with the smallest increase in the overall sunt
of squares within cluster distances are combined. In the centroid methods, the distanct
between two clusters is the distance between their centroids (means for all the variables.
as shown in Figure 20.6. Every time objects are grouped, a new centroid is computed.
the hierarchical methods, average linkage and Ward's methods have been shown "
perform better than the other procedures. l
The second type of clustering procedures, the nonhierarchical clustering methou
frequently referred to as k-means clustering. These methods include sequential thresno
parallel threshold, and optimizing partitioning. In the sequential threshold metho,
cluster center is selected and all objects within a prespecified threshold value from u
center are grouped together. Then a newcluster center or seed is selected, and the pro
is repeated for the unclustered points. Once an object is clustered with a seed. t
longer considered for clustering with subsequent seeds. The parallel threshold mele
 593
CHAPTER 20 Cluster Analui
Figure 20.6
Other Agglomerative Clustering Ward's Method
Methods

Centroid Method

simultaneously. and
operates similarly, except that several cluster centers are selected
optimizing partitioning method objects within the threshold level are grouped with the nearest center. The optimnizing
Nonhierarchical clustering method partitioning method differs from the two threshold procedures in that objects can later be
that allows for later reassignment of overall criterion, such as average within-cluster
reassigned to clusters to optimize an
obiects to clusters to optimize an distance for a given number of clusters.
overall criterion.
the number of
Two major disadvantages of the nonhierarchical procedures are that Furthermore.
clustersmust be prespecified and the selection of cluster centers is arbitrary.
selected. Many nonhierarchical
the clustering results may depend on how the centers are
missing values as initial
programs select the first k (k= number of clusters) cases without
order of observations in the
cluster centers. Thus, the clustering results may depend on the
methods and has merit when
data. Yet nonhierarchical clustering is faster than hierarchical
suggested that the hierarchical and
the number of objects or observations is large. It has been is obtained
nonhierarchical methods be used in tandem. First, an initial clustering solution
Ward's. The number of clusters
using a hierarchical procedure, such as average linkage oroptimizing
as inputs to the partitioning method.2
and cluster centroids so obtained are used
Choice of a clustering method and choice of a distance measure are interrelated. For
should be used with the Ward's and centroid
example, squared euclidean distances
distances.
methods. Several nonhierarchical procedures also use squared euclidean clustering. The output
We will use the Ward's procedure to illustrate hierarchical
Table 20.2. Useful information is
obtained by clustering the data of Table 20.1 is given in cases or clusters
the number of
contained in the agglomeration schedule, which shows
stage 1, with 19 clusters.
being combined at each stage. The first line represents shown in the columns labeled
Respondents 14 and 16 are combined at this stage, as
these two respondents is
"Clusters Combined." The squared euclidean distance between "Stage Cluster First
entitled
oiven under the column labeled Coefficients." The column illustrate, an entry oflat
To
Appears" indicates the stage at which a cluster is tirst formed. last column, "Next
at stage 1. The
stage 6 indicates that respondent 14 was first grouped
(respondent) or cluster is combined with
Stage." indicates the stage at which another case
column is 6, we see that at stage 6.
this one Because the number in the first line of the last Similarly, the second Jine
cluster.
respondent 10 is combined with 14 and l6 to fornm a single and7 are grouped together
respondents 6
represents stage 2 with l8 clusters. In stage 2, plot given in Figure 20.7.
Another important part of the output is contained in the iciclecase respondents labeled
in this
The columns correspond to the objects being clustered, read from bottom to
through 20.The rows correspond to the number of clusters. This figure is
considered
cases are as individual clusters. Because there are 20 respondents.
top. At first, all objects are combined, resulting in
there are 20 initial clusters. At the first slep, the two closest
PART III Data Csllatim, Prepartin, Analyi, and Reportng
TABLE 20.2
Results of Hierarchical Clustering
CASE PROCESSING SUMMARY®,b
CASES
VALID MissING
TOTAL
Percent Percent zN

20 100,0 0.0 20
Percent
100.0
"Squared Euclidean Distance used
bWard Linkage

WARD LINKAGE
AGGLOMERATION SCHEDULE
STAGE CLUSTER FIRST
CLUSTER COMBINED APPEARS
STAGE CLUSTER 1 CLUSTER 2 COEFFICIENTS CLUSTER 1 CLUSTER 2 NEXT STAGE
14 16 1.000 6
6 7 2.000 7
13 3.500 15
5.000
3 6.500 16
10 |4 8.167 9
7 6 12 10.500
9 20 13.000
9 10 15.583 (0 6 12
6 18.500 7 13
23.000 4 8
12 19 27.750
13 1 17 33.100 14
14 41.333 13 16
2 S1.833 3 11
16 3 64.500 14
17 79.667 12
18 4 172.667 15 17 19
19 2 328.600 16 18

CLUSTER MEMBERSHIP
CASE 4CLUSTERS 3 CLUSTERS 2 CLUSTERS
1
2 2
3
4 3 2
2 2
6
7
8
9
3 2
2 2
12
13 2 2
14 3

16 3 3
17
18 4 3
19 3
20
 1 X X X X X X X X X
X X X X X X X X X
X X X X X X X X X
X
6 X X X X X X X X X X X X
X X X X X X X
X X X X X X X X X X X X X
XX X
X X
7 X X X X X X X X X X X X X X X
X X X X X X X X X X X X
12 X X X X X X X X X
X X X X
X X
X X X X X X X
17 X X X X X X
X X X X X X X X X X X
X
X X X X X
X
15 X X XI X X
X X X X
X X X X X X X X X
X X X
3 X X X X X X X X X X X X X
X X X X X X
X X X X
X X X X X X X X X
X X X X X X X X X X X X X X X
X X
X

X X X X X X X X
2 X X X X X X X X X X X

X X X X X
X X X X X X X X X
13 X X X X X X
X X X X X X
X X X X X X X

X X X X
X XX
5
X X X X X X X X X X X X X X X

X X X X X X X X X
X X
X X X X X X X
11 X X X X X X X X X X X X

X X X X X X
X X XX
X
X X X X X X X X X X X X X
X X X X

X X X X X X
20 X X X X X X X X X X X X X X X X X X

X XX
X
X X X X X
X X X X X X X XX X X X X
4
X X X X X X X

X X X X X X X X X X
10 X XXX X X X X X

X X X X X X
X X X
X
X X X X X X X X X X X X
14 X X X
X X X X
X X X
X X X X X X X X X
Procedure
X X X X X
X X
X
X X X X
X X X X X X X X X
16 X X X
X X X X X X X
X X X X X X X X X X X X X X Ward's
19 X X
X X X X X X X X X X Using
X X X X
18 X X X X X X

CLUSTERS Plot
lcicle
20.7
Figure
OF Vertical
NUMBER
12 3 14 15 16 17 18 19
CASE 8 9 10
2 3 4

595
 PART II Data Cllacton, Prpantiwm, Analut, and Reporting
14
Using Ward's
4

Number
Case
Label

12

10 15 20 2
Rescaled Distance Cluster Combine

19 clusters. The last line of Figure 20.7shows these 19

clusters. The two cases, respondents
14 and 16, that have been combined at this stage have between them all
Xs in rows Ithrough
19. Row number 18 corresponds to the next stage, with 18 clusters. At
this stage, respondents
6 and 7 are grouped together. The column of Xs between respondents 6 and 7 has a
blankin
row 19.Thus, at this stage there are 18 clusters; 16 of them consist of individual
respondents.
and two contain two respondents each. Each subsequent step leads to the formation of a new
cluster in one of three ways: (1) two individual cases are grouped together, (2) a case is joined
to an already existing cluster, or (3) two clusters are grouped together.
Another graphic device that is useful in displaying clustering results is the
dendrogram (see Figure 20.8). The dendrogram is read from left to right. Vertical lines
represent clusters that are joined together. The position of the line on the scale indicates
the distances at which clusters were joined. Because many of the distances in the early
stages are of similar magnitude, it is difficult to tell the sequence in which some of the
earlyclusters are formed. However, it is clear that in the last two stages, the distances at
which the clusters are being combined are large. This information is useful in deciding
on the number of clusters.
It is also possible to obtain information on cluster membership of cases if the number of
clusters is specified. Although this information can be discerned from the icicle plot, a
tabular display is helpful. Table 20.2 contains the cluster membership for the cases,
depending on whether the final solution contains two, three, or four clusters. Information of
this type can be obtained for any number of clusters and is useful for deciding on the number
of clusters.

Decide on the Number of Clusters

Amajor issue in cluster analysis is deciding on the number of clusters. Although there are
no hard and fast rules, some guidelines are available:
1. Theoretical, conceptual, or practical considerations may suggest a certain number ot
clusters. For example, if the purpose of clustering is to identify market segmets.
management may want a particular number of clusters.
2. In hierarchical clustering, the distances at which clusters are combined can be used as
criteria. This information can be obtained from the agglomeration schedule or from
the dendrogram. In our case, we see from the agglomeration schedule in Table 20.
that the value in the "Coefficients" column suddenly more than doubles between
stages 17 (3 clusterS) and 18 (2 clusters). Likewise, at the last two stages of the de