An efficient framework for classifying the Clothing
items based on fashion and fabric of the images
Suvarna Nandyal
Department of Computer Science and Engineering
PDA College of Engineering
Kalaburagi, India
suvarna_nandyal@yahoo.com
Abstract— In recent days classification of images over
fashion domain has become fundamental research problem
with lot of computer vision based applications. In most of the
existing research image classification is done based on the
labels on the objects, however in the real world scenarios the
images needs to classified into the labels based on the domain
specific with proper guidelines, therefore the existing research
works fails in achieving the evaluation measures like
classification accuracy, precision, recall and time take taken
for classification, Hence there is need of an efficient
frameworks that classifies the images based on fashion and
fabric by addressing existing research problems. This paper
presents an efficient framework for segmenting using grab cut
techniques with the integrating the histograms of oriented
gradients (HOG) and the speeded-up robust features (SURF)
techniques. Whereas HOG technique is used to retrieve the
global features and SURF technique is used for the local
features and then segmentation is done based on Scale-
Invariant Feature Transform(SIFT) method that segments
the line, solve the incident scaling, changes in lighting
condition and rotation between the two images, Later
classification of the clothing images is done by using feature
matching classification techniques, with proper text analysis
the classification of images is done based on color and fabric
of the image. The proposed technique is compared with
existing multi-level classification technique to prove the
proposed framework is more efficient than existing works.
With the proposed technique, we could able to achieve the
accuracy of 93% by varying the dataset of the images which
is 10-15% more accurate than existing multi-level
classification.
Keywords— Feature matching Classification, Speeded-up
robust features (SURF), histograms of oriented gradients
(HOG), Scale-Invariant Feature Transform(SIFT), Fashion
domain
I. INTRODUCTION
Image retrieval for e-commerce is an emerging field of
research, which because of a huge commercial potential has
gradually attracted attention. Image retrieval technology
combined with commerce can better play its inherent
advantages. Technically, commodity images generally have
obvious image features and clear background, making it
easy to calculate feature operators and make similarity
comparison, thereby bypassing such core technical
problems as image semantic, greatly improving retrieval
accuracy. From the market’s perspective, the user demand
is huge, and image retrieval can improve retrieval accuracy
and service diversification, enhance the core
competitiveness of enterprises and user shopping
experience. Image retrieval can provide more accurate
results to enhance user experience and improve search
efficiency.
Recently, there has been an increase in the number of
multimedia applications including text, images, video and
978-0-7381-4272-2/20/$31.00 ©2020 IEEE
Nikhil S Tengli
Research Scholar, Dept. of CSE, PDACE, Kalaburagi
Assistant Professor, C&IT, REVA University, Bengaluru
nikhil.tengli039@gmail.com
audio data in every field with equal number of processing
requirements. The coming up of huge sized image datasets
calls for optimal techniques for storing, searching, indexing
and retrieving desired data at a faster speed. Target object
recognition is one of the most popular and widely
researched topics in computer vision field that has gained a
lot of momentum. The goal of this topic is to recognize the
objects belonging to the target object class based on its
features.
Shape of an object is a visually distinguishable feature
that can be easily utilized for recognition task but it has its
own challenges like pose variations, occlusions in the scene,
lighting conditions, resolution etc. A number of methods are
developed for achieving recognition of target based on
shape cues. The basic means of object recognition involves
matching the extracted features of objects in the image with
a sample target object feature.
This feature matching step is carried on all the objects in
every test image to find potential target object present.
Search technique is included in object recognition
methodology where the feature set extracted will be put to
comparison with the already extracted features of the target
object for easy and accurate recognition. To ensure good
recognition rate, the features extracted must be
distinguishable well and relevant for the target object to be
recognized. A variety of techniques are available for the
purpose of target recognition. Some of them are briefly
described here. The most important points to be considered
in object recognition tasks are the extraction of features that
are independent of rotation and translation [3]. The shape
invariant feature-based recognition of target objects
involves following three steps-
• Data Pre-processing
• Feature extraction
• Classification
In the first step, the input image is filtered using noise
removal methods and processed to improve its quality for
recognition purpose. Image processing aims to enhance the
image for obtaining good measures of its features. In the
next step, feature extraction from pre-processed image is
performed where the number and type of features extracted
depends on the end goal of the research. The extracted
features of all input images in the dataset are stored in the
database to be matched further for recognition of similar
target. Thus, feature extraction plays an important role in the
entire object recognition task that decides the performance
of the system developed
This paper presents an efficient framework that
classifies the images using feature matching classification
technique and the viable procedure in view of visual words
combination of SURF and HOG include descriptors with a
specific end goal to remove both neighborhood and
worldwide highlights from the images. Worldwide
highlights are utilized to remove semantic data of the entire
image, speak to image visual data in a smaller shape, and
perform better for those images that have the nearby visual
closeness between visual substances. Worldwide highlights
additionally perform better for different low-level
applications, for example, characterization and protest
identification, though neighborhood highlights are more
vigorous because of image portrayal as nearby fixes and
perform better in question acknowledgment based larger
amount applications. The execution of the proposed
procedure is likewise contrasted and the highlights
combination of SURF-HOG descriptors, independent
SURF and HOG include systems in view of the BoVW
philosophy.
To enhance the strength and effectiveness, like the bits
of writing [19, 20], we set our strategy as takes after. It right
off the bat utilizes Harris corner location administrator to
separate key focuses and after that develops coordinated
diagram of removed focuses. Also, it portrays coordinated
line sections with SIFT highlight and matches them to
achieve unpleasant coordinating of focuses. At long last, it
alters coordinating focuses and disposes of wrong matches
through Ransac technique to achieve montage of images.
The entire structure of our strategy can be seen from Figure
1. The technique proposed here has the accompanying
significant preferences: (1) the depiction in light of
highlights of work has solid heartiness for image turn,
twisting, and scaling; (2) the portrayal of coordinated line
section with SIFT highlights has certain strength for image
lighting impacts and revolution; (3) harsh point
coordinating by factual strategy could enhance coordinating
exactness.
II. LITERATURE SURVEY
This section discusses the related work in various papers
with respect to image retrieval in order to analyze the
previous approaches and come up with a better system for
designing a shopping guide with image retrieval using a
mobile interface.
[5] This paper introduced an effective visual word fusion
technique based on the SURF and the HOG feature
descriptor. Here only partial segmentation is done for the
images based on the line and edge of image beams, this
process is carried out with following features:
1. HOG method is applied for the better performance
than features fusion of SURF
2. The word fusion gives the accuracy around 84%
using the support vector machine
3. Partial segmentation done based on the fusion of
words.
[6] The authors have proposed a mobile app that works
in sync with a Content-Based Image Retrieval system for
online shopping of fashion products. The application can
retrieve most matched products of a textile image given as
an input. The proposed method works by manual selection
of the product name by the end user framed by the camera
image which is then sent to the server. Next, a search for
image retrieval is initiated that automatically recognizes and
maps the object like the requested one reducing the effort of
a prolonged manual search. To measure the performance of
the image retrieval system designed, the authors have
validated against three datasets- First set involves products
of clothing category from various online shopping websites
and the other two datasets are the images as well as video
frames of clothing obtaining from Internet users. The
system is found to be feasible for real time application.
The application works on client-server architecture. The
client’s role is to present the end user with the graphical user
interface asking query image dynamically. The client then
sends the final query formed to the server. Initially, an XML
file is downloaded that includes the different product names,
their underlying category, and the focus masks.
[7] In this work, the authors has developed a computer
vision system for efficient detection and classification of
clothing images for e-shopping images. A combined
architecture involving convolution neural networks that
cover Residual networks, Squeeze Net and Single Shot
MultiBox Detector (SSD) is considered. The training
experiments for detecting the clothing images was done on
Deep Fashion dataset containing cloth location annotated by
rectangular box. The system was evaluated against various
cloth images available online on e-commerce sites. Ground
truth for the image dataset were obtained using the online
shopping catalog containing metadata for five properties of
color, style, sleeve-type, neck design and hemline.
Automatic annotation collection showed up mean rate of
83% rate accuracy. In the analysis part, the authors have
sought the effects of several improvements attempted like
data augmentation under varying backgrounds, network size
increase and ensemble usage on the classification rate. Also,
the classification rate versus processing ease is computed in
this work. The most optimal network organizations for
accurately classifying the input clothing images are also
presented.
The authors have carried out experiments by
incrementing the network model size gradually to test and
record the best possible classification results, but the
computational complexity is compromised. Apart from a
single network, ensembles of 5 networks are considered.
The popular pre-trained Google Net classifier is used for
fashion textile classification by eliminating the last fully
connected layer and pre-training on the Image Net dataset.
Image Net dataset is a huge sized image collection
containing around 1.2 million images with 1000 categories.
Fine tuning the last fully connected layer with the fashion
image dataset collected is finally done. The system classifies
a training image into one of the classes from a possible 24
fine grained classes.
[8] In this work, the authors have carried out a study on
mobile phone-based image retrieval system for the design
of shopping guide for online shopping website. This guide
helps in various items to be retrieved from the online
repository of products using mobile interface. The design is
aimed at providing easy to use interactive methods and
innovative layout for image retrieval so that the drawbacks
of existing image retrieval systems are resolved. The
developed system has a decent practical value and
simplicity.
[9] Here, the authors have developed a mobile interface-
based image search system producing the relevant web
pages on the Internet containing matching objects similar to
the query image. Image matching produces very accurate
results when matching whole image or the scene is carried
out like landmarks etc. or also in cases where the object to
be matched has a distinguishable outline. The authors have
devised an interactive interface to obtain a segmented object
boundary with shape matching algorithm to spot the desired
objects of interest on Web pages.
3. Classification of the clothing images is done by using
feature matching classification techniques, with proper text
analysis the classification of images is done based on color
and fabric of the image.
5. Finally we retrieve the similar images based on color
and fabric of the cloths.

[10] In this paper, a novel Multiple-Instance Learning 6. Determine the accuracy of proposed method by
(MIL) based technique for image classification is described. Comparing with existing Multilevel Classification
The authors have extended the usual MIL method by Techniques.
incorporating DD-SVM, where a bag is labelled based on a IV. PROPOSED FLOW CHART DIAGRAM
few instances satisfying stated rules. Diverse Density (DD)
function is used for learning instance prototype categories The Flow Chart Diagram depicted in figure 1 describes
in a DD-SVM. Since every instance prototype is how the fashion items database is considered from various
representative of the class of instances to appear in bags e-commerce based applications and then applied with data
with a fixed label than in the unlabeled bags, a nonlinear pre-processing on the data set taken, feature selection of the
mapping to map each bag to a point in bag feature space is attributes for the input image and then classifying them
done. Support vector machines are used for training in the based on colour and fabric of the images, Finally accuracy
bag feature space. of proposed method will be measured by comparing with
existing multilevel classification technique.
[11] In this work, the authors concentrate on the task of
automatic woven fabric classification using a novel feature
extraction technique. The local binary pattern and the gray
level co-occurrence matrix features are initially extracted
for the fabric images in the data collection. To reduce high
dimensional feature information, principal component
analysis is performed. Support vector machine classifier is
employed for recognition of the fabric image. The method
is tested against three different types of woven fabric
categories namely plain, twill and satin weave fabrics.
The survey provides comprehensive analysis of research
work done so far in image retrieval techniques. Since the
accuracy of the methods used in the existing work are more
comparative and effective in detecting the object from the
images, these techniques can be used for our research work
in improving the accuracy for the detection of the images.
III. PROPOSED METHODOLOGY
The proposed methodology of this work is presented as
follows:
1. Apply Pre-processing on the dataset using resizing
and cropping the input images.
2. Apply segmentation on fashion-based images using
Grab-Cut techniques which is an image segmentation Figure 1: Proposed Flow Chart for Fashion Item
method based on graph cuts. The SURF and HOG features Classification
are extracted for each image in the dataset. That extracts the
global and local features of the image, then SIFT feature is
applied to extract the line, solve the incident scaling, V. RESULTS AND DISCUSSIONS
changes in lighting condition and rotation between the two
images.
The results are drawn in Matlab simulation toolkit-2016 thresholds (to detect strong and weak edges) and includes
for 750 images, using Feature matching based classification the weak edges in the output only if they are connected to
of images based on color and fabric of the input image. The strong edges described in figure 2. This method is therefore
segmentation is done based on Edge detection is an image less likely than the others to be affected by noise, and more
processing technique for finding the boundaries of objects likely to detect true weak edges.
within images. It works by detecting discontinuities in
brightness. Edge detection is used for image segmentation Then extracting the shape features using SURF
and data extraction in areas such as image processing, technique is used for the extraction of local features and
computer vision, and machine vision. HOG feature descriptors is carried out for better
performance than features fusion of SURF. This paper
The most powerful edge-detection method that edge presents a novel image mosaic method based on SIFT (Scale
provides is the canny method, Canny method differs from Invariant Feature Transform) feature of line segment
the other edge-detection methods in that it uses two different described in figure 3, aiming to resolve incident scaling,
rotation, changes in lighting condition, and so on between
two images.
Geometric Feature Extraction is done by using
Multimarked Point Process, a new stochastic marked point
process for describing images in terms of a finite library of
geometric objects described in figure 4.
Figure 2: Segmentation of image by Extracting the Edge
features of an image
Figure 3: Segmentation of image by Extracting the Shape
features of Image
Figure 4: Segmentation of image by Extracting the Line
features of Image
Figure 5: Segmentation of an image based on fabric of cloth
100
80
Accuracy Proposed
Feature
60
matching
40 Classification
Existing
20
Multilevel
0 Classification
250 500 750
Number of Images
Figure 6 describes different of images considered vs.
classification accuracy for the proposed feature matching
classification and comparing with existing multilevel
classification
VI. CONCLUSION AND FUTURE WORK
This paper presents an efficient framework for
classifying the images based on the color and fabric of the
images using both the segmentation and the classification
technique. First segmentation of the images is done using
speeded-up robust features (SURF) and histograms of
oriented gradients (HOG) feature descriptors, then apply an
image mosaic method based on Scale-Invariant Feature
Transform(SIFT) feature for line segment, aiming to resolve
incident scaling, rotation, changes in lighting condition,
Finally classification of the clothing images is done by using
feature matching classification techniques, with proper text
analysis the classification of images is done based on color
and fabric of the image. With the proposed technique, we
could able to achieve the accuracy of 93% by varying the
dataset of the images which is 10-15% more accurate than
existing multi-level classification. In the future, this
classification can be directly applied over real time e-
commerce databases by developing the mobile application
with the proposed techniques. A mobile application can be
developed by integrating the proposed framework that is
capable of classifying the results based on fashion and fabric
of clothing items.
REFERENCES
[1] Basavaraj S Anami, Suvarna S Nandyal and A Govardhan, “A
Combined Color, Texture and Edge Features Based Approach for
Identification and Classification of Indian Medicinal Plants”,
International Journal of Computer Applications, vol. 6 (no. 12),
2010, pp. 45-51
[2] NirupamaDevangaon, Suvarna Nandyal, "Verifiable Attribute-
Based Through Text and Image Search with Fine-Grained Owner-
Enforced Search Authorization in the Cloud", International Journal
of Science and Research (IJSR), vol. 6 Issue 6, 2017, pp. 1495-1500
[3] Suvarna Nandyal and Sandhya Koti, “Annotated Image search:
Annotated Image Search using Text and Image Features”,
International Journal of Scientific and Research Publications
(IJSRP), vol. 3, Issue 10, 2013
[4] Megha Gupta, Charul Bhatnagar, A.S. Jalal, “Clothing Image
Retrieval Based On Multiple Features for Smarter Shopping”,
Procedia Computer Science,vol. 125,2018, pp. 143-148
[5] Mehmood et al., “Content-Based Image Retrieval Based on Visual
Words Fusion Versus Features Fusion of Local and Global
Features”, Arabian Journal for Science and Engineering, 2018, pp.
1-20
[6] A. Nodari et al., "A mobile visual search application for content
based image retrieval in the fashion domain," 2012 10th
International Workshop on Content-Based Multimedia Indexing
(CBMI), Annecy, 2012, pp. 1-6
[7] J. Cychnerski, A. Brzeski, A. Boguszewski, M. Marmolowski and
M. Trojanowicz, "Clothes detection and classification using
convolutional neural networks", 22nd IEEE International
Conference on Emerging Technologies and Factory Automation
(ETFA), Limassol, 2017, pp. 1-8
[8] Dibin Zhou et al., “Design of Shopping Guide System with Image
Retrieval Based on Mobile Platform”, 2nd International Symposium
on Computer, Communication, Control and Automation, 2013, pp.
147-150
[9] Tom Yeh, Kristen Grauman, Konrad Tollmar, Trevor Darrell, “A
Picture is Worth a Thousand Keywords: Image-Based Object Search
on a Mobile Platform”, CHI, ACM, 2005, pp. 2025-2028
[10] Yixin Chen, Jinbo Bi and J. Z. Wang, "MILES: Multiple-Instance
Learning via Embedded Instance Selection", IEEE Transactions on
Pattern Analysis and Machine Intelligence, vol. 28, no. 12, 2006, pp.
1931-1947
[11] P. F. Li, J. Wang, H. H. Zhang and J. F. Jing, "Automatic woven
fabric classification based on support vector machine", International
Conference on Automatic Control and Artificial Intelligence, 2012,
pp. 581-584
[12] Min et al.,”A survey on context-aware mobile visual recognition",
Multimedia Systems, vol. 23(no. 6), 2017, pp. 647-665
[13] Y. H. Kuo and W. H. Hsu, "Dehashing: Server-Side Context-Aware
Feature Reconstruction for Mobile Visual Search“, IEEE
Transactions on Circuits and Systems for Video Technology, vol.
27(no. 1), 2017, pp. 139-148
[14] L. Zheng, S. Wang, and Q. Tian, “Coupled binary embedding for
large-scale image retrieval”, IEEE Transactions on Image
Processing (TIP), vol. 23, no. 8, 2014, pp. 3368–3380
[15] J. Tang, Z. Li, M. Wang, and R. Zhao, “Neighborhood discriminant
hashing for large-scale image retrieval,” IEEE Transactions on
Image Processing (TPI), vol. 24, no. 9, 2015, pp. 2827–2840
[16] G. Tolias, Y. Kalantidis, Y. Avrithis, and S. Kollias, “Towards large-
scale geometry indexing by feature selection,” Computer Vision and
Image Understanding, vol. 120, 2014, pp. 31–45
[17] Angadi, Sanjeevkumar, and Suvarna Nandyal. “Human
Identification System Based on Spatial and Temporal Features in the
Video Surveillance System.” International Journal of Ambient
Computing and Intelligence (IJACI), vol. 11, no. 3, 2020, pp. 1–21