A Machine Learning approach to segment the
ϮϬϮϯ/ŶƚĞƌŶĂƚŝŽŶĂůŽŶĨĞƌĞŶĐĞŽŶƌƚŝĨŝĐŝĂů/ŶƚĞůůŝŐĞŶĐĞĂŶĚ<ŶŽǁůĞĚŐĞŝƐĐŽǀĞƌLJŝŶŽŶĐƵƌƌĞŶƚŶŐŝŶĞĞƌŝŶŐ;/KE&ͿͮϵϳϵͲϴͲϯϱϬϯͲϯϰϯϲͲϰͬϮϯͬΨϯϭ͘ϬϬΞϮϬϮϯ/ͮK/͗ϭϬ͘ϭϭϬϵͬ/KE&ϱϳϭϮϵ͘ϮϬϮϯ͘ϭϬϬϴϰϯϯϵ
customers of online sales data for better and
efficient marketing purposes
Mathesh T1, Sumathy G2 and Maheshwari A2
1
Department of Information Technology Jeppiaar Engineering College Chennai, India
E-mail : mati02official@gmail.com sumathyg@srmist.edu.in maheshwa1@srmist.edu.in
Abstract- The Internet is becoming huge and is used
by a more diverse audience every day. The amount of
data gathered from the platform through different
online lead companies are gargantuan so it needs to be
maintained and segregated in order to extract
meaningful data from it. A lot of companies have
started to gather customer data through their own
platform or through various vendors who sell it to
sales companies/organizations/individuals for some
profit. Sometimes these data are large and scattered
enough to even confuse big sales organizations. In
order for better and more effective marketing of these
sales data, We propose to use four machine learning
clustering algorithms( K-Means, Agglomerative,
Mean-Shift and DBSCAN) in order to find customer
segments based on the data provided. Based on this
segmented customer group, we can be able to find a
pattern and decide which customer group is better for
which business.
Keywords— Machine Learning, Customer
Segmentation, K-Means, Agglomerative, DBSCAN,
Marketing
I.INTRODUCTION
A major objective of the research and study of these
online leads data is to find meaningful customer
segments and use them for better and more efficient
marketing. There are various problems gathered
from the online leads which are not structured in a
proper way. Some of the problems include wastage
of time and resources in trying to send promotional
emails or telecalling the customers who won’t fit the
target audience of the sales company/ organization/
individual, also from a huge amount of lead data but
only less than one per cent of sales conversion data
is achieved due to improper and generic approach in
marketing. There is no meaning in trying to sell a
youth product to a customer lead who is above 50
years of age.
1
Authorized licensed use limited to: SRM Institute of Science and Technology. Downloaded on September 04,2023 at 08:04:00 UTC from IEEE Xplore. Restrictions apply.
Only a small percentage of companies who buys
sales lead has some team setup to segregate and
segment the customer leads, others fall into trap of
using a common approach for all the leads which
leads to poor sales conversion rate of leads data.
Individual salesperson is the most affected because
they only have domain knowledge of their respective
sales area but fail to understand the clusters or
segments present in the sales data. Basic or common
filters provided by various software won’t work very
well for multi-dimensional columns. So, it is
important for each sales executive to understand the
customer segments in the data and customise their
marketing technique for each group.
In this paper, for research purposes, we bought a set
of sales leads from a company named Shrine. There
are three important sources of this data:- Internal
customers of the company, data gathered from
various online networking tools and job profiles
from a hiring company. The data gathered has been
diverse and contain columns related to their personal
and work data. A lot of analysis and cleaning was
required in order for the data to be used by the
model which is covered in the Methodology section.
Clustering is a machine-learning approach for
finding the patterns in data and grouping them as
clusters. Unsupervised Clustering is a method to find
clusters without a target or a previously clustered
column. We are going to use four unsupervised
clustering algorithms in this paper namely K-Means,
Agglomerative, Mean Shift and DBSCAN. The
same data but in a different format based on their
algorithm is going to be trained by these models and
their output is studied and analysed.
 II. RELATED WORK
Every market is unique and heterogenous and
finding the right consumers is a big and difficult task
for the sales and marketing team. Gillian has
stressed the importance of Market Segmentation by
suggesting how segmentation allows an organization
to achieve high ROI (Rate Of Investment) [1].
Online Leads have opened a completely new path
for the sales organization to gather customers, a lot
of data are available to them unlike in the old days
where companies have to use alternative methods
like campaigns, and forms to gather data. There are
different ways to gather leads like social media,
online ads etc. Simon, Kilian, Christian and
Alexander have revealed how a typical Online Lead
Generation Funnel Works [2].
Clustering Algorithms use machine learning
techniques and help us in identifying the clusters in
the data. The Distance between the points is one of
the key factors for deciding the clusters and there are
several ways to calculate the distance like Euclidean,
Minkowski etc [3]. Danuta and Jan suggested how
using different clustering algorithms like DBSCAN
and K-Means helps in the segmentation of Bank
Customers [4]. The comparative study using the
local grocery store data helped in identifying clusters
and predicting potential and not-potential customers
[5].
Segmentation of Customers for various domains is
discussed in the literature (see [6], [7], [8], [9]).
These techniques stress the importance of the quality
of data which is fed to the model for prediction. The
original data needs to be undergone several steps of
tuning and pre-processing in order to be used by the
model. Not every data point is in a cluster, real data
Fig 1. Partial view of online sales data before pre-processing
2
Authorized licensed use limited to: SRM Institute of Science and Technology. Downloaded on September 04,2023 at 08:04:00 UTC from IEEE Xplore. Restrictions apply.
has some noise. Nika and Ben emphasizes how can
we proceed with clustering keeping noise present in
the data as one of the deciding parameters [10].
Labelling each cluster after the model prediction
requires some domain knowledge about the data.
Sulekha suggests that various factors like
Demographic, Psychographic and Behavioural
which can be used to determine the naming of the
clusters, once the model prediction is over [11].
III. RESEARCH METHODOLOGY
The following section covers how the entire process
of research methodology from data collection to
model training is done.
A. Data Collection
The data used in the paper was brought from the
company named Shrine and the three important
sources for this data collection are Internal
customers of the company, data gathered from
various online networking tools and job profiles
from a hiring company. The data received was
diverse and had a lot of different features like Name,
Date Of Birth, Profession, Designation, Current
Company etc. Figure 1 shows the partial view of the
initial data which was received. There were five
different datasets given to us and EDA (Exploratory
Data Analysis) has to be done before proceeding as
there were a lot of anomalies present in the data. The
Initial dataset consists of 9108 rows and not all the
datasets have all the features present. We had to
study and select the features which could result in
meaningful analysis. From the 15 columns
presented, we narrowed it down to four columns
(DOB, Education, Experience, Salary) as these were
the common and meaningful data present in
common in all the datasets.
 Fig 2. Final view of online sales of data after pre-processing
B. Data Pre-processing
The final dataset which was used in the model had a
dimension of (3107 rows x 4 Columns). The
elimination of the rest of the rows is due to
duplicates in data, a lot of missing values, unrealistic
and improper values present in the data, which had
to figure out separately in the Data Pre-processing
process.
Since we require the data to be of the same
magnitude the same data had to undergo another set
of pre-processing for the 4 columns. The DOB is not
in the right format to be used by the model, so we set
a common reference date for all the DOB data and
computed age from it. The education columns had
different ordinal categorical variables like Bachelor's
and Master's degrees which needed to be converted
into a numerical format to be used by the model.
Since this Education follows some orders, a
particular weight has been assigned to each degree,
for example, PhD being the highest degree present in
the data mapped to the highest value while 10th
Grade being the lowest educational value mapped to
a lower value. The salary columns were in different
scales(thousands, lakhs, crores) and different
currency formats (INR, USD, SGD). So, all the data
in the column are converted into INR format and the
values are brought in per ten thousand scales by
dividing all the values. The experience columns have
been converted to months from years in order to
match the magnitude of data present in other
columns. We can see the final data after
preprocessing in figure 2.
C. Models
Instead of using a single model for analysing the
data, we chose to study the significance of the
impact of different models of data by comparatively
3
Authorized licensed use limited to: SRM Institute of Science and Technology. Downloaded on September 04,2023 at 08:04:00 UTC from IEEE Xplore. Restrictions apply.
studying their respective outputs. The four models
discussed in this paper are:- K-Means,
Agglomerative, Mean-Shift and DBSCAN as these
models have a completely different approaches to
identifying clusters.
K-Means Clustering:-
K-Means is one of the most used unsupervised
clustering algorithms where clusters are identified
through a partitioning principle using the centroids.
The K-Means algorithm work as follows:-
1. Initialize the K-clusters which are needed to be
found from the data. The K-value can be found
through an elbow mechanism which is discussed in
the experiment section. [12]
2. Once the K value is decided, the algorithm picks a
k number of random centroids and from that
centroid, nearby data points are picked as clusters.
3. These nearby data points are calculated by the
distance metrics. The one which we are using in the
model is named Euclidean Distance where the
distance is calculated based on the square root of the
sum of squared differences between the two points.
Let the two points are P(x1, x2) and Q (y1, y2) the
Euclidean distance is given by :-
4. The cluster’s centroid keeps on shifting for a
given no. of iterations or until the centroid allocation
no longer changes.
Agglomerative Clustering:-
Agglomerative is one of the famous Hierarchical
based clustering algorithms where clustering starts
 by taking each data point as a single cluster and
starts merging these clusters based on the similarity
between them. The procedure is repeated until all the
data points are merged under a single cluster.
Linkage helps in merging two clusters into one in
Agglomerative clustering. There are various types of
linkages and we are using Ward’s method in this
paper as it helps in creating compact and even-sized
clusters.
In ward’s method, the distance between two clusters
is based on the similarity calculated as the sum of
the square of the distances between the points x and
y divided by the product of their centroids C1 and
C2.
Mean – Shift Clustering:-
Mean-Shift Clustering identifies clusters by
iteratively shifting their centroids to a higher
probability density region. Once the centroid cannot
move further the algorithm stops. The advantage of
the Mean-Shift algorithm is unlike the K-Means we
don’t need to specify the number of clusters to be
found. Mean shift automatically finds the number of
clusters based on the density. Kernel(K) is a
weighting function used in the calculation of the
density function. Bandwidth(h) is one of the
important parameters needed for the Mean-Shift. It
can be considered the radius of the circle which
denotes how many data points should be present in
the cluster. Below is the formula for calculating
Fig 3:- Distribution of Age, Salary, Experience respectively through histogram.
4
Authorized licensed use limited to: SRM Institute of Science and Technology. Downloaded on September 04,2023 at 08:04:00 UTC from IEEE Xplore. Restrictions apply.
PDF(Probability Density Function) where n, d and x
correspond to length, dimensions and datapoint
value respectively.
DBSCAN Clustering:-
DBSCAN refers to the Density-Based Spatial
Clustering of Applications with Noise. As
mentioned in the Literature review, not all data
points can be clusters, the real data has some noise.
DBSCAN helps us in identifying those clusters and
the algorithm does not have any impact on outliers,
unlike K-Means which is very sensitive to outliers.
There are two parameters (Epsilon(eps) and
Minimum Points(minpoints) which are needed for
the DBSCAN to perform. The eps describe the
radius around the cluster and the minpoints establish
a condition for the minimum number of points
needed to be in the clusters. The idea of how to
derive these values is explained in the Features and
Parameter section.
D. Features and Parameter Selection:-
Selecting the right parameters for each clustering
algorithm with the right data has a huge significance
and impact in successful model creation. For training
the model and selecting the respective features the
famous python library scikit-learn is used [13]. This
section describes how the features are scaled and
how parameters are selected for each algorithm.
 Scaling and Outlier Removal:-
The data we use in research is gathered in such a
way the whole data points have been scattered with a
very high and low magnitude of values to match the
real-world scenario of data gathering. Some of the
algorithms we use are more sensitive to outliers [14]
and need to be taken care of before using them in the
model, to achieve quality results. Figure 3 shows the
distributions of data points in our data through a
histogram.
We can clearly see some of the data lying in the
outlier zone. To remove the outlier, we are going to
replace high and low values with the upper and
lower limit of IQR(Interquartile Region). The IQR
describes the middle 50% of values when ordered
from lowest to highest. For dealing with the scaling
of data, we are going to Z-Score mechanism, since
the variance is more or less between the columns. Z-
Score helps in understanding how much data differs
from its standard deviation. Following is the formula
of Z-Score where x, ߤ, ߪ corresponds to observed
value, mean and standard deviation respectively.
Determination of K value:
For selecting the right K value, the elbow method
mechanism is used. It works by finding the Sum of
Squares Error (SSE) values of each data point with
its respective centroids with different values of K.
The Formula for SSE is given below where x, ߤ݅ and
k correspond to datapoint, centroid and cluster size
respectively.
Fig 4: Plot for determination of elbow method with
respective to WSS (Within SSE Score)
Since K Means is highly sensitive to outliers, the
5
Authorized licensed use limited to: SRM Institute of Science and Technology. Downloaded on September 04,2023 at 08:04:00 UTC from IEEE Xplore. Restrictions apply.
scaled data is used in finding the elbow value. When
K values increase the SSE will decrease. The point
where the elbow is found is chosen as the K-Cluster
value. From figure 4, using the elbow method, the K
value for the data is chosen as 2 and the model is
trained accordingly.
Training the Agglomerative model:-
Agglomerative clustering considers all the data
points present in the plane as a separate cluster at
first and then proceed to group them together as one
final cluster. Since Agglomerative data is also
sensitive to outliers the same scaled data is being
used for the model-building process. As explained in
the model section we are going to use the ward's
linkage in merging clusters.
In order to visualize and determine the output of the
clustering the process Dendrogram is used. A
dendrogram shows the hierarchical relationship
between each cluster. Figure 5 shows the
dendrogram for the scaled data. From the above
dendrogram, the clusters are chosen as 3 based on
the longest vertical line which is not being cut by
any of the horizontal lines. The Agglomerative
model is then trained with 3 clusters.
Fig 5:- Dendrogram for the scaled data.
Bandwidth for Mean Shift Algorithm:-
Mean shift works by assuming all data points in the
feature space as an empirical probability density
function. The determination of the correct bandwidth
is crucial for the clustering process in the Mean Shift
algorithm. To determine the right bandwidth, we are
going to calculate it based on the K-Nearest
neighbours mechanism defined by Dorin and Peter
[15].
The bandwidth was found to be 197.59 using the
above approach. With the same bandwidth value, the
 Mean Shift algorithm was trained and 14 different unique clusters were formed. Some clusters were
clusters were found in the output. These outputs are identified as noise by the algorithm and marked as -
analysed in the result and analysis section. 1, which could be useful during analysis

Determining the epsilon and min points:-

There are two parameters needed to be estimated for
the DBSCAN algorithm. One is the epsilon value
which describes the radius of the circle and the other
one is the minimum points which specifies the
conditions on how many minimum numbers of
points should be present in the cluster. A rule of
thumb says to determine the minimum number of
points based on the (k+1) rule where k is the number
of features present. But we also took some domain
knowledge into account by looking at the data.
Keeping both the conditions in mind, the min points Fig 6:- Plot to determine the EPS value.
value was chosen as 5.
IV. EXPERIMENT RESULT AND ANALYSIS
The determination of epsilon value is much more
challenging one and determine statistical proof in The number of clusters formed from the K-Means,
order for the model to be accurate enough. Agglomerative, Mean-Shift and DBSCAN models is
Following is the approach followed for listed in figures 6, 7, 8 and 9 respectively. In the
determination of epsilon value:- figures, we have listed out the mean values of each
1. Calculate the nearest neighbours for all feature, from which we can create label based on the
datapoints. domain knowledge of the product to be sold. The
2. Find the K-neighbour distance and sort the Scale for Degree Level is as follows 5- PhD, 4 –
distance and plot them in the graph. Masters, 3 – Bachelors, 2.5 – Diploma, 2 - 12th
3. The knee point(bend point) in the plot is Grade, 1 - 10th Grade. The number of clusters found
determined as the epsilon value. for K-Means, Agglomerative, Mean-Shift and
DBSCAN algorithms are 2, 3, 14 and 17
Following the above approach, the plot shown in respectively with DBSCAN having a special column
figure 6 was created. By zooming in on the plot, the for outliers. The salary is mentioned in 10k scale per
epsilon value was found to be 22. These parameters annum in INR.
are used to train the DBSCAN algorithm and the 17

Fig 7:- K-Means Cluster Results (Mean Values)

Fig 8:- Agglomerative Cluster Results (Mean Values)

6
Authorized licensed use limited to: SRM Institute of Science and Technology. Downloaded on September 04,2023 at 08:04:00 UTC from IEEE Xplore. Restrictions apply.
 Fig 9:- Mean-Shift Cluster Results (Mean Values)
From the above experiment, we found that both K-
Means and Agglomerative models struggled to
identify and find more clusters from the data point.
The reason for the issue is generally because of the
working procedure and algorithm implemented by
each model which got affected by differences in
magnitude present in data, even after scaling was
done. The identified clusters for the K-Means and
Agglomerative Models can be named based on their
salary as listed in figures 7 and 8. For K-means, we
can name the labels above higher middle-class
income and below higher middle-class income and
for Agglomerative the three labels can be high class,
middle class and low class.
Both the K-Means and Agglomerative models were
able to find some meaningful clusters but they were
not meaningful enough for a marketing company to
maximize their efficiency in categorizing potential
customers for each product. This is where the Mean
Shift and DBSCAN were able to succeed mainly due
to their density-based approach. The total number of
clusters formed from the Mean Shift and DBSCAN
were 14 and 17 respectively. The respective labels
for both Mean Shift and DBSCAN can be
determined on a different basis based on the domain
knowledge and the mean values present in each
feature of the clusters. For example, if we look at
cluster 4 from figure 9 in Mean-Shift Cluster Results
we can see people in that customer segment are
earning above 3 crores INR per annum on average
7
Authorized licensed use limited to: SRM Institute of Science and Technology. Downloaded on September 04,2023 at 08:04:00 UTC from IEEE Xplore. Restrictions apply.
but have been under age 40. This customer segment
can be named as Customers with high potential for
luxury products.
Unlike the K-Means and Agglomerative models
where the labels are only based on salary, much
more meaningful information has been able to be
obtained from Mean Shift and DBSCAN models. In
DBSCAN models we were able to significantly
identify the noise present in the data and segregate
them as a separate cluster. These noises are
extraordinary cases where it only represents a
minority portion of a customer segmentation
problem and can be used only in very few cases.
When analyzing all the models, bringing all the
column scales to an equal margin was one of the key
factors and helped in identifying the right clusters
even though the experience and salary column had a
little large magnitude than the other columns. To
measure and analyze how good the clustering is
done in all algorithms, Silhouette Score is used.
Silhouette score is the measure to understand how
well each data point has been clustered into the
correct cluster by the below formula where ‘a’
corresponds to average distance between the
centroid of a cluster and the datapoints and ‘b’
corresponds to average distance between the data
point and closest cluster data points.

Fig 10:- DBSCAN Cluster Results (Mean Values)
Fig 11:- Silhouette score of 4 models used.
Figure 11 shows the silhouette score of each model
used. For DBSCAN the silhouette score is calculated
by removing the -1 labels to make sure the silhouette
score is calculated meaningfully. We can see a high
score for both Agglomerative and Mean Shift
models denoting how well and efficiently clustering
was done for each data point. But the analysis can’t
be only determined based on silhouette score.
Silhouette score is one of the metrics for analyzing
unsupervised clustering model results. The real-
world scenario may be different. From the usage
perspective, we found DBSCAN to be much more
effective than Agglomerative despite having a low
score, mainly because of how well it was able to
differentiate and segments customers based on
8
Authorized licensed use limited to: SRM Institute of Science and Technology. Downloaded on September 04,2023 at 08:04:00 UTC from IEEE Xplore. Restrictions apply.
different principles or dimensions.
People with domain knowledge will prefer
DBSCAN or Mean Shift as having more customer
segments helps them to market and reach the target
audience for each product more effectively. Thus,
the algorithms which use some density-based
approach like DBSCAN or Mean shift tend to be
more useful for any marketing with the aim to
identify different customer segments from a large
pile of different and scattered data.
V. CONCLUSION
We were able to identify some meaningful customer
segments which could be used in marketing
effectively from an online lead source which had a
huge number of different features present in
different scales. A huge amount of time had to be
allocated for data pre-processing and tuning in order
to arrive at the correct result. Having many
Customer segments segregated based on various
factors from online lead data can easily help any
marketing team or individual in targeting,
customizing or personalizing their market campaign
for a specific target audience which could in turn
maximize their revenue and their sales conversion
rate.
REFERENCES
[1] G. Martin, "The importance of marketing segmentation,"
American journal of business education (AJBE), vol. 4, no. 6,
pp. 15-18, 2011.
[2] S. Stolz, K. Wisskirchen, C. Schlereth and A. Hoffmann,
"Online Lead Generation: An Emerging Industry," Marketing
Review St Gallen, June 2021.
[3] K. Bindra and A. Mishra, "A detailed study of clustering
algorithms," 6th International Conference on Reliability,
Infocom Technologies and Optimization (Trends and Future
Directions)(ICRITO) (IEEE), pp. 371 - 376, 2017.
[4] D. Zakrzewska and J. Murlewski, "Clustering algorithms for
bank customer segmentation," International Conference on
Intelligent Systems Design and Applications (ISDA'05) (IEEE),
pp. 197 - 202, 2005.
[5] A. Choudhury and K. Nur, "A machine learning approach to
identify potential customer based on purchase behavior,"
International Conference on Robotics, Electrical and Signal
Processing Techniques (ICREST) IEEE, pp. 242 -247, 2019.
[6] T. Kansal, S. Bahuguna, V. Singh and T. Choudhury,
"Customer segmentation using K-means clustering,"
International conference on computational techniques,
electronics and mechanical systems (CTEMS) (IEEE), pp. 135 -
139, 2018.
[7] P. Balakrishnan, M. Cooper, V. Jacob and P. Lewis,
"Comparative performance of the FSCL neural net and K-means
algorithm for market segmentation," European journal of
operational research,, vol. 93, no. 2, pp. 346 - 357, 1996.
[8] M. Zait and H. Messatfa, "A comparative study of clustering
methods," Future Generation Computer Systems, vol. 12, no.
(2-3), pp. 149 - 159, 1997.
 [9] Y. Rani and H. Rohil, "A study of hierarchical clustering
algorithm," ter S & on Te SIT, vol. 2, no. 113, 2013.
[10] S. Ben-David and N. Haghtalab, "Clustering in the presence of
background noise," International Conference on Machine
Learning PMLR, pp. 280 - 288, 2014.
[11] S. Goyat, "The basis of market segmentation: a critical review
of literature," European Journal of Business and Management,
vol. 3, no. 9, pp. 45 - 54, 2011.
[12] T. Kodinariya and P. Makwana, "Review on determining
number of Cluster in K-Means Clustering," International
Journal,, vol. 1, no. 6, pp. 90 - 95, 2013.
[13] "Scikit-Learn," scikit-learn, [Online]. Available: https://scikit-
learn.org/stable/.
[14] V. Peter and R. Mehta, "Impact of outlier removal and
normalization approach in modified k-means clustering
algorithm," International Journal of Computer Science Issues
(IJCSI), vol. 8, no. 5, p. 331, 2011.
[15] D. Comaniciu and P. Meer, "Mean shift: A robust approach
toward feature space analysis.," IEEE Transactions on pattern
analysis and machine intelligence, vol. 24, no. 5, pp. 603 - 619,
2002.

9
Authorized licensed use limited to: SRM Institute of Science and Technology. Downloaded on September 04,2023 at 08:04:00 UTC from IEEE Xplore. Restrictions apply.