2020 IEEE International Conference for Innovation in Technology (INOCON)
Bengaluru, India. Nov 6-8, 2020
Sorting of Objects Using Image Processing
1st Prashast Yadav 2nd Meenakshi Uikey
Electronics And Telecommunications Electronics And Telecommunications
Vidyalankar Institute of Technology Vidyalankar Institute of Technology
Mumbai, India Mumbai, India
prashast.yadav@vit.edu.in meenakshi.uikey@vit.edu.in
4th Sachin Kayande
Electronics And Telecommunications
Vidyalankar Institute of Technology
Mumbai, India
sachin.kayande@vit.edu.in
Abstract—Sorting is a regular process for many industries
and small businesses. The paper proposes a low-cost automated
system using Deep Learning, Raspberry Pi 3 module B, and a
conveyor belt system for sorting of various day to day products.
Object sorting is done based on their features. The physical
implementation includes a conveyor belt and a pneumatic piston,
which is controlled by the Raspberry Pi for the sorting of the
object. TensorFlow, with some libraries and methods, is used for
the detection of the object.
Index Terms—Conveyor belt system, Pneumatic Cylinder,
Raspberry Pi 3, TensorFlow, SSD Mobilenet, USB Camera
I. I NTRODUCTION
Automation improves the overall productivity in any indus-
try. The paper suggests a new approach to develop a system
that can segregate the products placed on the conveyor belt
based on their features by building mechanical structures to
sort the same.
This automated system does not require any special human
vigilance and thus reduces the probability of human errors.
The proposed Automated Object Sorting System is devel-
oped using Single Shot Detector Model (SSD) with Mobilenet
feature extractor. A Deep Learning based image classification
model is used to classify objects as per requirement, based
on the object recognition accuracy in real-time. Deep learning
classifies objects into different categories. It used multiple lay-
ers of data for feature extraction and representation, and once
the system is trained with initial datasets, the identification of
objects can be achieved with almost accuracy.
A. Problem Statement
Sorting of products has been reported to be a complex
problem in various small scale industries and businesses.
These businesses rely on the old mechanism for sorting these
products, to reduce costs which can be utilized in various
other productive activities. Thus, there is a need to design an
automated sorting system that separates various products and,
at the same time, move objects automatically to the basket.
The system must be feasible, reliable, and affordable.
978-1-7281-9744-9/20/$31.00 ©2020 IEEE 1
Authorized licensed use limited to: San Francisco State Univ. Downloaded on June 18,2021 at 16:08:20 UTC from IEEE Xplore. Restrictions apply.
3rd Prajakta Lonkar
Electronics And Telecommunications
Vidyalankar Institute of Technology
Mumbai, India
prajakta.lonkar@vit.edu.in
5th Amit Maurya
Electronics And Telecommunications
Vidyalankar Institute of Technology
Mumbai, India
amit.maurya@vit.edu.in
B. Literature Survey
For several years considerable effort has been taken to
improve the sorting process in various industries. Over the
years, this sorting was done with the help of human strug-
gles that brought up many disadvantages like increased time
consumption, and human errors. To overcome these problems,
many researchers have proposed various methods for the
sorting process, which includes object detection and sorting
on multiple bases, like sorting based on color or shape or
size. In one of the previous papers, the sorting of objects was
done based on colors. Euclidean distance-based approach was
used to find the distance between average color in a region to
that of the desired color. The major drawback of this approach
is when the colors are mixtures between two colors, making
it difficult for the system to recognize the object [1,2]. The
proposed system in this paper uses a different approach to
overcome such issues that arise when sorting is done only
based on colors or shapes.
The basic idea for the construction of the conveyor belt and
the sorting system was obtained from the IEEE paper entitled
“Object Detection and Robotic Sorting System in Complex
Industrial Environment”. [3]This system has designed and
developed a sorting machine with the help of deep learning.[4]
In some previous papers, it was documented that object
detection can be done using the SIFT approach[7]. However,
there are some drawbacks in using the SIFT approach as it
uses intensive computational power due to which it cannot be
used for real-time application. Also, the SIFT approach cannot
detect objects that do not show regions of texture. [5]-[7].
Real-time object detection is performed using TensorFlow’s
Object Detection API and OpenCV. According to the research
mentioned in ‘Detection of Real-Time Objects Using Tensor-
Flow and OpenCV’ it is concluded that the image processing
done using Tensorflow is more efficient and reliable than the
Pytorch deep learning library [8]. The paper uses the YOLO
object detection system. SSD (single-shot detection) Mobilenet
is a convolutional neural network that has the best detection
rate of 87.7 percent with a training to test data ratio of 80:20.
It has to do about nine times less work than other neural
networks, providing the same accuracy. Thus, the suggested
system in this paper papers uses SSD MobileNet.
II. M ETHODOLOGY
A. TensorFlow
Tensorflow is the open-source library that is being used
in the proposed system for object detection. The tensors are
multidimensional arrays that consist of elements of the same
data type. These are a standard way of representing data in
machine learning. For object detection, we provide some input
data which consists of a set of images. Using TensorFlow,
the model is trained using any machine learning algorithm.
The main objective of the model is to extract the main visual
features based on edge detection and visual recognition. For
testing the model, test data is provided, which is again a set of
images, and the final output is obtained. The output image is
then converted to Numpy array so that computation becomes
easy. TF records are also being used, which consists of images
along with the tags. TF records stores the data as a sequence
of binary strings.[16]
D. Client-Server Communication
There are two communication protocols in the client-server
system; the UDP protocol and the TCP protocol [13]. In
UDP (User Datagram Protocol), each time the client sends
a datagram to the server, it also has to send the local socket
descriptor and the receiving socket’s address. In TCP (Transfer
Control Protocol), a connection is established between the two
sockets of client and server. The client’s socket will request
a connection while the other socket, i.e., the server’s socket,
will listen to the connection request and respond to it. Once
the connection is established, the transmission of data can be
done in both the direction.
In the proposed system, the TCP communication protocol
is used to be more efficient than UDP[16]. The PC, where the
object detection is done, acts as a server, and Raspberry Pi 3,
acts as a client. To establish a connection between the devices,
they are connected to the same local network through Router.

B. COCO Dataset
The suggested system uses a pre-trained object detection
model ’ssd mobilenet v1 coco’ on the COCO dataset, which
refers to ”Common Objects in Context.” [9],[12]. It is a
dataset used for large scale object detection, segmentation,
and captioning. This package provides Matlab, Python, and Fig. 1. Client Server Communication
Lua APIs. The COCO dataset has 330000 images, out of
which more than 200000 are labeled. It has 1.5 million object
instances, 80 object categories, 91 stuff categories, and five III. S YSTEM D ESIGN
captions per image. It also has 250000 people with key points. A. Block Diagram
[17,18]

C. SSD MobileNet
SSD is one of the popular methods used for the detection of
objects in an image. It provides scores for each object category
during screening and makes changes and adjustments to the
box if needed to match the object’s shape. This method is
easy to train as SSD has eliminated proposal generation and
feature resampling and encapsulation of all computation in a
single network. SSD is a faster method compared to other
single-stage methods.[10] Previous research has demonstrated
that SSD with Mobilenet features extractor is one of the most
accurate models out of all fastest models.[11] A significant
advantage of using MobileNet feature extractor is that it can Fig. 2. Block Diagram.
also be used in Low Powered Devices without GPU. The
efficiency of object detection using the MobileNet model is 1) Image Capturing and Processing Block:
well explained in the paper ”Real Time Object Detection Can a) USB Webcam: A USB camera captures a real-time
be Embedded on Low Powered Devices”[12]. The MobileNet image. We have used Logitech camera for this purpose. It
model makes the Object Detection process more efficient, has 720p HD video- capturing capacity, a lens of 5MP and
as well explained in the paper ”Object Sorting using Image automatic light correction features that rectifies the brightness-
Processing.[5]” related errors to give an improved viewing experience [14].

Authorized licensed use limited to: San Francisco State Univ. Downloaded on June 18,2021 at 16:08:20 UTC from IEEE Xplore. Restrictions apply.
 Fig. 3. USB Webcam

b) PC: PC is the processing unit that takes input from

Fig. 5. Relay Module
Webcam and sends the processed results to the Raspberry Pi.
The real-time images from the Webcam are sent to the PC
3) Mechanical Block:
using the OpenCV library and processed in this unit using
TensorFlow Object Detection API. It then recognizes and
sends the data to Raspberry Pi.
2) Decision and Command Block: This block consists of
a Raspberry Pi and a relay module. Raspberry Pi acts as a
client, and it receives the data from the PC. It analyzes the
data and commands the relay module accordingly.
a) Raspberry Pi 3: “Fig 4.” shows the Raspberry Pi 3 B
module. It has 40 GPIO pins that are used to connect the relay.
HDMI cable used to connect the monitor and 5v micro USB
port it uses to give the supply to Raspberry Pi 3 B module. Fig. 6. Top View AutoCAD Model
Ethernet port is used to provide the network to the module.
The four built-in USB ports can provide output up to 1.2A,
which will enable us to connect more USB devices [15],[20].

Fig. 7. Front View AutoCAD Model

Fig. 4. Raspberrypi 3

b) Relay Module: The solenoid used to control the piston

Fig. 8. Final Conveyor Belt Setup
requires a power supply of 12V. Since the output of GPIO
pins of Raspberry Pi is 3.3V, we cannot control the solenoid a) Conveyor Belt Frame:
directly through GPIO pins. Thus, a relay module comes into
• Main conveyor frame: The material used for making the
the picture. We connect the 12V external power supply and
outer frame of the conveyor belt was 304 Stainless Steel.
the solenoid across the open circuit of the relay module and
The following two cuttings were done for the same- Two
then control the relay’s switching using the GPIO pins. The
plates of size 860x65mm One plate of size 740x200mm
relay module provides an inbuilt driving circuit that amplifies
the voltage to 5V and isolates the two circuits [14].

Authorized licensed use limited to: San Francisco State Univ. Downloaded on June 18,2021 at 16:08:20 UTC from IEEE Xplore. Restrictions apply.
 Fig. 9. Conveyor Belt Frame
• Bearing Blocks: Bearing blocks of iron were used to hold
the rods on both ends of conveyor belt. Four bearing
blocks of size 50x50x20mm were made.
• Rods and Rollers: Two iron rods of diameter 20mm were
used on both ends of the frame to support the rollers and
belt. Nylon rollers were further fitted on these rods.
• Conveyor Stands: Hollow aluminium square pipes of
dimension 130x20x10mm were used.
b) DC geared motor: DC geared motor was used to meet
the high torque requirement of the conveyor belt system. Gears
provide a balance between torque and speed.[14] Motor with
the following specifications was used: Torque: 120kgcm Input
Voltage: 12V-16V RPM: 40rpm
Fig. 10. DC Geared Motor
c) Pneumatic solenoid valve: The pneumatic solenoid
valve converts electricals signals into pneumatic functions.
Air is directed into the solenoid as soon as the electricity is
applied. The solenoid valve used in the proposed system is a
5/2-way directional valve that has two output ports, one inlet
port, and two exhaust ports. Solenoid valve with the following
specifications is used: Model: J4V110-06, Working Pressure:
0. 15-0.8MPa.
Authorized licensed use limited to: San Francisco State Univ. Downloaded on June 18,2021 at 16:08:20 UTC from IEEE Xplore. Restrictions apply.
Fig. 11. 5/2 Solenoid Valve
d) Pneumatic Cylinder: A Double-acting pneumatic
cylinder is used in the system to sort the objects in two
directions. A rectangular extension is attached to its piston,
which ensures that force applies to a larger portion of the
object.
Fig. 12. Pneumatic Cylinder
B. System Workflow
The system has a conveyor belt that rotates at a constant
speed with 12V dc geared motor. The objects are fed at the
feed side of the conveyor belt, where the camera captures the
object’s real-time image. The image captured is then identified
and recognized by the PC using TensorFlow Object Detection
API. Then a Client-Server network is established between PC
and Raspberry Pi to send the object label as ’string’ data type.
Raspberry Pi then compares this data with the object labels
that we stored in the program. If the detected label is the
same as the stored label, then Raspberry pi makes GPIO pin
HIGH or LOW accordingly. The pins of the Raspberry Pi are
connected to the relay module, which controls the solenoid.
The Raspberry Pi controls the switching of the solenoid via the
relay module. The HIGH input to the relay module will turn on
the solenoid. A pressure between 0.15MPa-0.8MPa is applied
to the Pneumatic cylinder when the solenoid is switched on.
The piston then pushes the object to its respective baskets.
 Fig. 15. Bottle detection in sufficient light

• CASE-II: Under moderate lighting conditions-

Fig. 16. Cup detection in moderate light

Fig. 13. System Workflow

IV. R ESULT AND A NALYSIS

The pre-trained model used in this application shows an
accurate result of object identification. Two objects, a cup,
and a bottle, are used to test the system. The testing was done
under three different conditions of light intensity, which are
as follows:
• CASE-I: When a sufficient amount of light was available-

Fig. 17. Bottle detection in moderate light

• CASE-III: Under poor lighting conditions-

The results show that the object was detected in all the above
cases, provided the accuracy score reduced ad the amount of
light decreased. The time taken for the object to reach the
Pneumatic cylinder from the feed position is found to be 1
second. Thus a 1-second delay is taken before commanding
the GPIO pins.

Fig. 14. Cup detection in sufficient light

Authorized licensed use limited to: San Francisco State Univ. Downloaded on June 18,2021 at 16:08:20 UTC from IEEE Xplore. Restrictions apply.

Fig. 18. Cup detection in poor light
Fig. 19. Bottle detection in poor light
V. C ONCLUSION
Real-time Object detection is successfully implemented us-
ing a CPU with an i5 6th generation processor. The Raspberry
Pi controls the pneumatic solenoid and piston for sorting
purposes. The accuracy of the recognition of objects depends
on various factors like the lighting conditions and orientation.
After object recognition, it is ensured that the pneumatic
pistons work synchronously with proper delay for the sorting
of objects. For the improvement in accuracy, the environment
consisting of constant light and monochromatic background is
used. Efficiency can also be enhanced by training our model
for a vast dataset.
[6] S. Sumahasan, Udaya Kumar Addanki, Navya Irlapati, Amulya Jonnala,
“Object Detection using Deep Learning Algorithm CNN”, International
Journal for Research in Applied Science and Engineering Technology
(IJRASET), Volume 8 Issue VII July 2020
[7] Raghu Raj Prasanna Kumar, Suresh Muknahallipatna, John McInroy,
“An Approach to Parallelization of SIFT Algorithm on GPUs for Real-
Time Applications”, Journal of Computer and Communications(JCC),
2016
[8] Ajay Talele, Aseem Patil, Bhushan Barse, “Detection of Real Time
Objects Using TensorFlow and OpenCV”, Asian Journal of Convergence
in Technology, Volume-V, Issue-I
[9] R.S.Sandhya Devi, Vijaykumar VR , M.Muthumeena, “Waste Seg-
regation using Deep Learning Algorithm”, International Journal of
Innovative Technology and Exploring Engineering (IJITEE), Volume-
8 Issue-2S December, 2018
[10] Wei Liu, Dragomir Anguelov, Dumitru Erhan, Christian Szegedy, Scott
Reed, Cheng-Yang Fu, Alexander C. Berg, “SSD: Single Shot MultiBox
Detector”, UNC Chapel Hill Zoox Inc. Google Inc. University of
Michigan, Ann-Arbor
[11] Huang J, Rathod V, Sun C, Zhu M, Korattikara A, Fathi A, Fischer I,
Wojna Z, Song Y, Guadarrama S, Murphy K, ”Speed/accuracy trade-offs
for modern convolutional object detectors”, CVPR 2017
[12] Jimut Bahan Pal, Shalabh Agarwal, “Real Time Object Detection Can be
Embedded on Low Powered Devices”, International Journal of Computer
Sciences and Engineering (JCSE), Vol.-7, Issue-3, March 2019
[13] Ming Xue, Changjun Zhu, “The Socket Programming and Software
Design for Communication Based on Client/Server”, 2009 Pacific-Asia
Conference on Circuits,Communications and System.
[14] Ranjitha S Rai, Madan G, K.R Prakash, G. Ravi Krishna, “Automation
of Object Segregation”, International Research Journal of Engineering
and Technology (IRJET), Volume: 06, Issue: 06 June 2019
[15] Sushrut Nagesh Kulkarni, Sanjay Kumar Singh, “Object Sorting Au-
tomated System using Raspberry Pi”, Proceedings of the International
Conference on Communication and Electronics Systems (ICCES 2018)
[16] Ming Xue, Changjun Zhu, “The Socket Programming and Software
Design for Communication Based on Client/Server”, 2009 Pacific-Asia
Conference on Circuits,Communications and System.
[17] https://blog.tensorflow.org/2020/07/tensorflow-2-meets-object-detection-
api.html
[18] https://cocodataset.org/home
[19] https://github.com/cocodataset/cocoapi
[20] https://www.raspberrypi.org/

R EFERENCES
[1] Sheela. S, Shivaram. K. R, Meghashree. S, Monica. L, Prathima. A,
Shriya.M.Kumar, “Low Cost Automation for Sorting of Objects on
Conveyor Belt”, International Journal of Innovative Research in Science,
Engineering and Technology (IJERSET), Vol. 5, Special Issue 10, May
2016
[2] Vishnu R. Kale, V. A. Kulkarni, “Object Sorting System Using Robotic
Arm”, International Journal of Advanced Research in Electrical, Elec-
tronics and Instrumentation Engineering (IJAREEIE), Vol. 2, Issue 7,
July 2013
[3] Liang BinYan, Wang YanBo, Chen ZhiHong, LiuJiaYu, Lin JunQin,
“Object Detection and Robotic Sorting System in Complex Industrial
Environment”, 2017 Chinese Automation Congress (CAC), Jinan, 2017
[4] Navinkumar Patle, Shweta Dhake, Devayani Ausekar, “Automatic Ob-
ject Sorting using Deep Learning”, International Research Journal of
Engineering and Technology (IRJET), Volume: 05, Issue: 08 Aug 2018
[5] Rahul Vijay Soans, Pradyumna G.R., Yohei Fukumizu, “Object Sorting
using Image Processing”, 2018 3rd IEEE International Conference on
Recent Trends in Electronics, Information and Communication Technol-
ogy (RTEICT-2018), MAY 18th and 19th 2018

Authorized licensed use limited to: San Francisco State Univ. Downloaded on June 18,2021 at 16:08:20 UTC from IEEE Xplore. Restrictions apply.