The Human Following Shopping Cart Assistance Robot

“THE HUMAN FOLLOWING SHOPPING CART
ASSISTANCE ROBOT”
PID-BS IT-F17M03
MUHAMMAD SAQIB ASLAM
HAMZA KHAN
SHAFIQ UR REHMAN
MUHAMMAD FARHAD NAZIR
DEPARTMENT OF INFORMATION SCIENCES

UNIVERSITY OF EDUCTION LAHORE MULTAN
CAMPUS
2017-2021
The Human Following Shopping Cart Assistance Robot ii
THE HUMAN FOLLOWING SHOPPING

CART ASSISTANCE ROBOT
MUHAMMAD SAQIB ASLAM HAMZA KHAN

17-UE-06419 17-UE-06427
BSF1703969 BSF1703977
2017-2021 2017-2021
SHAFIQ UR REHMAN M. FARHAD NAZIR

17-UE-05842 17-UE-05840
BSF1703600 BSF1703598
2017-2021 2017-2021
A thesis submitted in partial fulfillment of the

requirements for the award of the degree of
INFORMATION TECHNOLOGY
DEPARTMENT OF INFORMATION
SCIENCES MULTAN CAMPUS
UNIVERISTY OF EDUCATION
LAHORE
2017-2021
Department of Information Sciences, University of Education Lahore (Multan Campus)

The Human Following Shopping Cart Assistance Robot iii
CERTIFICATE OF APPROVAL
This is to certify that the research work presented in this thesis, entitled “THE
HUMAN FOLLOWING SHOPPING CART ASSISTANCE ROBOT” was
completed under the supervision of Dr.Sajid Ali at Department of Information
Sciences, University of Education Lahore, Multan Campus in partial fulfillment of the
requirement for the degree of Bachelor of Science in Information Technology under
my guidance and supervision.
In our opinion, it is satisfactory and up to the mark and therefore fulfills the
requirements of Bachelor of Science in Information Technology Degree.
Signature: Signature:
Student Name: Student Name:
Date: Date:
Date: Date:
Project Supervisor Convener (Project Evaluation Committee)
Name: Dr. Sajid Ali Name: Dr. Muhammad AsadMeraj
Designation: Associate Professor Designation: Principal
University of Education, Multan campus University of Education, Multan campus
Member (Project Evaluation Committee) Member (Project Evaluation Committee)
Name: Dr. Sajid Ali Name: Designation: Associate

Designation: Professor Designation:

The Human Following Shopping Cart Assistance Robot iv
―We hereby declare that I have read this thesis and in my opinion this thesis is
sufficient in terms of scope and quality for the award of the degree of
Bachelors in Information Technology.‖
Project SupervisorName: Dr. Sajid Ali
Designation: Associate Professor
Signature:
Date:
University of Education, Multan Campus

The Human Following Shopping Cart Assistance Robot v
DECLARATION
We declare that this thesis entitled ―THE HUMAN FOLLOWING SHOPPING
CART ASSISTANCE ROBOT‖ is the result of our own research except as
cited in the references. The thesis has not been accepted for any degree and
is not concurrently submitted in candidature for any other degree. At any time
if my statement is found to be incorrect even after award of Bachelors in
Information Technology degree, the university has the right to withdraw my
degree of Information Technology.
Date: Date:
Date: Date:

The Human Following Shopping Cart Assistance Robot vi
Acknowledgement
This work was somewhat upheld by BS IT 8th semester group No: 3, Group
Id‖ PID-BS IT-F17M03‖. We are thankful to ALLAH ALMIGHTY, who gave us
the strength and ability to understand this knowledge and stand where we are,
and to His beloved Prophet Muhammad (S.A.W) who is the Holy Prophet
(PEACE BE UPON HIM) of Allah Almighty and symbol of guidance to all
mankind. We extend special thanks to our PARENTS and my teachers;
without their support we were unable to stand where we are now.
We express my heartiest respect and gratitude to my internship supervisor

who made me able to complete this Project to fulfill all the requirements Also
my special expression of gratitude to all working staff of saving center who
introduced me with required information to earn the professional work with
open heart.

The Human Following Shopping Cart Assistance Robot vii
TABLE OF CONTENTS
CHAPTER 1 INTRODUCTION ........................................................................ 1
CHAPTER 2 REVIEW OF DEVELOPED SYSTEMS: ..................................... 4
2.1 Information and Help Systems: ...................................................... 5
2.2 Shopping Cart Robots: ................................................................... 5
2.3 Robot Person following Behavior: .................................................. 7
CHAPTER 3 SYSTEM DESCRIPTION: THE SHOPPING CART

ASSISTANCE ROBOT ................................................................................. 10
3.1 The Mobile Platform: .................................................................... 11
3.2 Arduino UNO ................................................................................ 12
3.3 Motor Driver:................................................................................. 13
3.4 Gear Motor: .................................................................................. 14
3.5 Servo Motor.................................................................................. 16
3.6 Ultrasound Localization System: .................................................. 16
3.7 TDoA with RF and Ultrasound: ..................................................... 18
3.8 Trilateration for Position Determination: ....................................... 19
3.9 Trilateration Geometrical Error Study: .......................................... 21
3.10 Infrared Sensor:............................................................................ 22
3.11 Infrared Radiation Theory ............................................................. 22
CHAPTER 4 PERSON FOLLOWING FUNCTIONALITY ............................. 25
4.1 User Interface:.............................................................................. 26
4.2 The 4 states are portrayed as follows: .......................................... 27
CHAPTER 5 SIMULATION RESULTS ......................................................... 29
5.1 Small Grocery Store Simulation ................................................... 30
5.2 Large Grocery Store Simulation: .................................................. 31

The Human Following Shopping Cart Assistance Robot viii
CHAPTER 6 FIELD EXPERIMENTS (I) ........................................................ 33
6.1 Person following Functionality in a Small Grocery Store .............. 34
CHAPTER 7 FIELD EXPERIMENTS (II) ....................................................... 36
7.1 Usability Tests with Two Robots in a Large Grocery Store ........... 37
CHAPTER 8 CONCLUSION AND FUTURE WORK..................................... 39
REFERENCES .............................................................................................. 42

The Human Following Shopping Cart Assistance Robot ix
LIST OF FIGURE
Figure 3.1 Development Board Microcontroller for Project ....................................... 13
Figure 3.2 This shield is driven by 2 L293D chips, which are very strong 4-channel
drivers. Because of the power of these chips, this shield can control 4 DC Motors .. 14
Figure 3.3 This is a DC motor and wheel set for making robots! These motors are
light weight, high torque and low RPM. .................................................................... 15
Figure 3.4 MICRO SERVO SG90 9G ELETRONIC MODULE RC DIY KIT ROBOT
MOTOR TOWER ..................................................................................................... 16
Figure 3.5 Ultrasonic Distance Sensor ..................................................................... 17
Figure 3.6 Infrared Sensor Module has built-in IR transmitter and IR receiver that
sends out IR rays and looks for reflected IR rays to detect presence of any object .. 23
Figure 3.7 Working Principle of IR sensor ................................................................ 23
Figure 4.1 Person following behaviour 4-state machine ........................................... 26
Figure 4.2 Person following behaviour: the person can be in one of the 3 shown
regions .................................................................................................................... 27
Figure 5.1 Simulation of five persons and robots on a small grocery, consisting of
three corridors with shelves on each side. ............................................................... 30
Figure 5.2 Simulation of five people shopping in a larger environment..................... 32
Figure 8.1 Future Vision Robotic Shopping cart ....................................................... 40

The Human Following Shopping Cart Assistance Robot 1
CHAPTER 1
INTRODUCTION

As of late, the expansion of enormous grocery stores and malls, added
to the fast improvement of robot innovation, has created automated
frameworks for aiding individuals in these specific conditions. Some particular
kinds of utilizations, for example, upgrading the actual capacities of the client,
helping in the vehicle of items by giving a versatile shopping bin, and
improving the data given to the client in a shrewder way, have been portrayed
in logical writing.
Helping senior individuals in their surroundings is a method of
fundamentally improving their personal satisfaction. For instance,
organizations of sensors and actuators inserted in structures (i.e., Ambient
Intelligence) can give valuable capacities. Indeed, a few European Union
ventures have zeroed in on this specific subject. Be that as it may, assisting
older with peopling outside their standard climate is likewise essential, to
assist them with doing day by day errands like shopping.
Populace maturing is a worry for most created nations and it is
conjecture that it will speed up in the years to come. The principle objective in
the field of help advanced mechanics is to help clients in their regular day to
day existences, in assignments, for example, moving around, taking care of
articles, or collaborating, conceivably distantly, with others, either companions
or family members, or experts. In this unique circumstance, robot gadgets are
pointed not just at helping individuals, rather than supplanting them, yet
additionally at embracing the part of associates. In this way, they should be
intended to cooperate with undeveloped individuals, and an incredible
exertion should be made to empower clients to acknowledge them without
any problem.

Additionally, these help robots may need to work in unstructured
conditions that can't be redone, like homes, shops, or stops. This infers a
requirement for the robot that is aware of its environmental factors, which
requires the utilization of an assortment of sensors. Also, a large number of
the things that should be taken care of, like food, don't come in standard sizes
and shapes, as they regularly do in a modern setting.
In this paper we present an individual after shopping basket and help
robot named Advance Robo, the buddy robot prepared to do effectively
assisting old individuals with conveying items in a store. As a matter of first
importance, the paper presents an overview of related frameworks intended to
help individuals in grocery stores and business shopping centers and to help
individuals shopping and conveying their articles in a general store utilizing
various methodologies, like append able modules and PC vision. From that
point onward, the paper depicts exhaustively the proposed framework: a
versatile stage that carries out individual after conduct that utilizes ultrasound
and radio transmissions innovation to give strength and usability according to
the perspective of the client. At last, the paper dissects the versatility of this
framework for expanding the quantity of robots in a reenacted climate and its
ease of use in a little supermarket.

CHAPTER 2
REVIEW OF DEVELOPED SYSTEMS:

2.1 Information and Help Systems:
With regards to data and help frameworks, the administrations that
those frameworks give to the clients in shopping centers and huge stores are
chiefly focused on showing the area of specific shops and offering vague
notice data dependent on client distinguishing proof.
In a system is described that has the ability to accompany the user in
the search of a list of articles previously specified. The system does not carry
the food. Moreover, the robot performs navigation and obstacle detection by
using a camera and a laser range finder.
Using a Robot IIF platform with humanoid aspect, in a system is
described that interacts with the persons in a mall and is able to give them
directions about the localization of the products or even recognizing them
through RFID tags. The user input is provided by means of a speech
recognition system.
An intelligent multirobot system for big stores is proposed in , where
the robots may develop cooperative tasks in order to detect available space in
the shelves, checking the real location of the products by looking at their
barcodes, knowing the number of articles in a shelf by using RFID tags,
making an inventory of the available products, and helping customers to
localize specific articles.
2.2 Shopping Cart Robots:
As of late, a few examinations have tended to fostering a self-ruling
shopping basket. For instance, the chance of fostering a porting framework for
huge measured offices like a shopping center or an air terminal is

concentrated in. To accomplish this reason, they have connected a drive
framework to a shopping basket, to communicate capacity to the truck
wheels. To play out the ideal individual after capacity, they utilize a sound
system camera introduced before the truck. One benefit of this framework is
that the individual doesn't need to wear any connected component.
Conversely, as they are utilizing acknowledgment by shading division, the
individual needs to wear
garments of a particular tone. Also, they need to deal with every one of
the issues identified with PC vision: high computational expense, light
fluctuation issues, foundation shading obstruction issues, etc.
A more complete model is proposed in, made out of two robots, a
friend and a transporter. To restrict the situation of the friend, a bunch of
cameras is utilized, which permits the identification of robots and people
present in the situation. The transporter robot can follow the buddy one, which
can control the client to the ideal situation, just as follow him while trusting that
the following activity will be performed. A framework for following shopping
baskets in indoor conditions is created in. This framework comprises of the
establishment of a PC and a camcorder on the shopping baskets, for them to
have the option to self-limit and send their situation to an incorporated
framework. We discover this framework fascinating, as it could likewise be an
initial step to having an independent truck with individual after capacities, yet
as a weakness, it requires an unpredictable framework to be introduced on
the truck, and a structure with prior foundation.
The examination completed in infers that senior individual’s interface in
a superior manner with robots conveying the shopping bin when they have a

humanoid viewpoint and give conversational abilities, albeit this reality infers a
significant expense and complex framework. In a shopping help framework is
created, ready to get the shopping list from a cell phone through a QR code,
convey the shopping crate and show at every second which articles are on it,
and speak with the store PC framework to educate about the area of articles.
It utilizes a laser range locater, sonar, and contact sensors (guards) to
explore. Then again, some older individuals lean toward being available in the
general store to contact the items and make a more itemized choice.
2.3 Robot Person following Behavior:
Individual after performed by a robothas generally been a significant
exploration theme in the machine vision region. It is normal to envision that
portable robot of things to come, particularly those working openly puts, will
be required to have this ability. For instance, in two distinctive effective
individual following calculations for a dream based versatile robot utilizing two
freely moving cameras are introduced. With this methodology it is feasible to
complete ongoing individual continuing in indoor conditions. In the creators
portray a robot with following capacity and returning capacity utilizing a
monocular camera.
An alternate human following technique utilizing layout coordinating in
range information got from a laser range locater (LRF) is portrayed in. The
square coordinating is performed utilizing put away formats, which compare to
appearances of human legs. The creators use laser examines and separating
procedures for following moving items and people. In the creators utilize a
laser-based individual following technique and two distinct ways to deal with

individual after, bearing after and way following, and a mix of the two
strategies is proposed, a mixture approach, with the robot consequently
choosing which strategy to utilize. In the laser-based leg indicator is joined
with a Kinect framework to distinguish the client position, anticipate its
direction, and cause the vehicle to follow it. Inthe laser range locater is joined
with two cameras to distinguish the client position and the presence of
deterrents, while the primary objective of the paper is to furnish the robot with
various conduct techniques (e.g., Go To User, Follow User, Local Search, and
Global Search) and movement ways (Free Space and Cluttered Space)
choosing the suitable one in each circumstance.
A robot global positioning framework dependent on a camera and a
light-radiating gadget is utilized. Their motivation is to foster a self-ruling
versatile robot which can follow a person. The robot with a camera takes a
gander at a human having a light-transmitting gadget. The gadget comprises
of two LEDs. The robot can gauge the situation of the human from the
distance between two LEDs, and the heading, from the situation of the LED
on the caught picture.
In a twofold sensor framework is utilized to perform individual after by a
robot. The robot can go with an individual utilizing vision-based objective
location and keep away from obstructions with ultrasonic sensors while
following the individual. Ina camera is utilized to identify both the individual to
be followed and the natural hindrances, while RFID labels are utilized to get
the robot's restriction.
A shopping help robot is proposed for outwardly debilitated individuals,
utilizing RFID labels for item recognizable proof and a laser range locater for

limitation and route, while, in, RFID is blended in with vision methods to
distinguish the client in a group and track their situation to make it conceivable
to follow him.
The framework proposed in this paper is a shopping basket robot that
executes an individual after conduct. The goal is to have a straightforward and
hearty framework, simple to utilize, uniquely for more established individuals,
and that addresses a generally minimal expense arrangement contrasted with
different methodologies. The proposed individual after framework depends on
a mix of radio and ultrasound signs to perform estimations between the
individual and the robot. When the framework takes a couple of estimations, a
straightforward activity gauges the situation of the human to be followed. The
execution two or three estimation gadgets introduced on the robot and a
ultrasound ring connected to the individual's leg. This location strategy, as
various to different executions, maintains a strategic distance from the
utilization of cameras and the huge measure of calculation important to
picture preparing.
This framework has effectively demonstrated its effectiveness with
regards to the EU GUARDIANS project, where the hazardous and low
perceivability conditions required a vigorous and solid framework. Its solid
focuses are the effortlessness, power, and its autonomy to changes in
perceivability conditions.
Contrasted with other proposed techniques, computational necessities
are low, so it is feasible to carry out the entire framework in a practical
programmable board.

CHAPTER 3
SYSTEM DESCRIPTION: THE SHOPPING
CART ASSISTANCE ROBOT

In this part, the proposed shopping basket help robot named Advance
Robo is introduced. The worldly improvement of the framework and the
accomplished achievements are addressed. To the best of the creators'
information, it is the main shopping help robot that consolidates seclusion,
straightforwardness, and usability at a sensible expense that is particularly
focused to help the old individuals in a particularly regular action like doing
shopping. As it will be portrayed beneath, it depends on a business
mechanical stage and a few frill like a RFID label peruse that gives extra
control of the items that are being added to the truck. Also, when the
framework is associated with a cell phone like a tablet PC or cell phone, it
gives extra data to the client and permits the change of various individual after
boundaries.
3.1 The Mobile Platform:
The Advance Robo depends on the Erratic-Videre portable robot
stage, a stage which has been furnished with a shopping crate. It is a
conservative and amazing stage and is fit for conveying a full heap of
mechanical technology hardware, including an incorporated PC, laser range
locater, and different sensors. The choice was persuaded in light of the fact
that it is a straightforward stage, sensibly valued, profoundly expandable, and
powerful. Also, this stage is truly adaptable, as it has been intended to work
either with an on-board PC, utilizing the force supply from the robot batteries,
or with a PC, with its own battery. Furthermore, the equipment is viable with
Player/Stage and ROS (Robot Operating System), both open-source and
generally utilized in the mechanical technology local area. The stage is
furnished with three 12V 7AH lead-corrosive batteries, giving a sensible self-

governance to the depicted reason (around 1 hour for an ordinary shopping
use). The stage weighs 2 Kg including the base, batteries, and frill. The
upheld payload is 5 Kg.
3.2 Arduino UNO
Arduino is an open-source gadgets stage dependent on simple to-
utilize equipment and programming. Arduino sheets can understand inputs -
light on a sensor, a finger on a catch, or a Twitter message - and transform it
into a yield - initiating an engine, turning on a LED, distributing something on
the web. You can guide your board by sending a bunch of guidelines to the
microcontroller on the board. To do so you utilize the Arduino programming
language (in view of Wiring), and the Arduino Software (IDE), in light of
Processing.
Throughout the long term Arduino has been the cerebrum of thousands
of ventures, from regular items to complex logical instruments. An overall local
area of creators - understudies, specialists, craftsmen, software engineers,
and experts - has assembled around this open-source stage, their
commitments have amounted to an amazing measure of available information
that can be of incredible assistance to fledglings and specialiststhe same.
Arduino was brought into the world at the Ivrea Interaction Design
Institute as a simple apparatus for quick prototyping, focused on understudies
without a foundation in gadgets and programming. When it arrived at a more
extensive local area, the Arduino board began changing to adjust to new
requirements and difficulties, separating its proposal from straightforward 8-bit
sheets to items for IoT applications, wearable, 3D printing, and installed

conditions. All Arduino sheets are totally open-source, engaging clients to
construct them freely and in the long run adjust them to their specific
requirements. The product, as well, is open-source, and it is developing
through the commitments of clients around the world.
Figure 3.1 Development Board Microcontroller for Project
3.3 Motor Driver:
The most commonly used actuator in any electronic device/machine
will be motors next to solenoids, pneumatics and hydraulics. From a simple
vibration motor inside a mobile phone to complex stepper motors in CNC
machines, these DC machines can be found everywhere. To control a motor
using a Microcontroller or processors we need something called a Motor
Driver or Motor Controller. Depending upon the type of motor and type of
control required the type of Motor Drivers will also change. In this article we
will focus only of DC motors and how to control a DC motor using a Motor
Driver with the most popular H-bridge Topology. This technique will help us
drive small or large DC Motors and also control its direction.

Figure 3.2 This shield is driven by 2 L293D chips, which are very strong 4-channel drivers.
Because of the power of these chips, this shield can control 4 DC Motors
The motor driver IC is an integrated circuit chip used as a motor
controlling device in autonomous robots and embedded circuits. L293D and
ULN2003 are the most commonly used motor Driver IC that is used in simple
robots and RC cars. A motor driver is undoubtedly something that makes the
motor move as per the given instructions or the inputs (high and low). It listens
to the low voltage from the controller/processor and control an actual motor
which needs high input voltage. In simple words, a motor driver IC controls
the direction of the motor based on the commands or instructions it receives
from the controller. Many motor drivers follow different topology, in this article
we will focus on the popular H-bridge topology which is used in the L293D
motor driver IC.
3.4 Gear Motor:
Essentially, a gear motor is a pairing of gear reducer and ac or dc

electrical motor. The gear and the motors are combined into one unit.
A gear motor delivers high torque at low horsepower or low speed. The
speed specifications for these motors are normal speed and stall-speed
torque. These motors use gears, typically assembled as a gearbox, to reduce
speed, which makes more torque available. Gear motors are most often used
in applications that need a lot of force to move heavy objects.
Figure 3.3 This is a DC motor and wheel set for making robots! These motors are light weight,
high torque and low RPM.

3.5 Servo Motor
A servo motor is a self-contained electrical device, that rotate parts of a
machine with high efficiency and with great precision. The output shaft of this
motor can be moved to a particular angle, position and velocity that a regular
motor does not have.The Servo Motor utilizes a regular motor and couples it
with a sensor for positional feedback. The controller is the most important part
of the Servo Motor designed and used specifically for this purpose.
Figure 3.4 MICRO SERVO SG90 9G ELETRONIC MODULE RC DIY KIT ROBOT MOTOR TOWER
3.6 Ultrasound Localization System:
The limitation framework utilized in the shopping basket robot has been
adjusted from the past outcomes got in the field of individual after under crisis

circumstances and low perceivability conditions. This framework utilizes
ultrasound-based signs to make estimations between two focuses. It
additionally utilizes radio signs for synchronization purposes. The created
ultrasonic limitation framework includes the accompanying gadgets:
Hagisonic sonar’s (transmitters/beneficiaries): two unique models of
9ultrasonic sonar have been read and utilized for executing time contrast of
appearance (TDoA) to infer position data. Those sensors (HG-M40DAI and
HG-M40DAII) are ultrasonic item identifiers and reach locaters that offer
extremely short-to-medium-range recognition. The AII model offers an
uncommon restricted directional reaction, degrees in vertical heading to limit
the impression of undesirable sound waves.
Radio modules, standard radio transmitter/beneficiary working at
433 MHz: those modules permit the regulation of computerized flags up to
1 KHz at greatest distances of around 20–30 inch. Those remote modules are
planned to give a synchronization instrument, by conveying constant
messages between the various robots that could be available.
Figure 3.5 Ultrasonic Distance Sensor

3.7 TDoA with RF and Ultrasound:
With this estimation, we attempt to decide the 2D situation of an actual
objective (i.e., an individual) concerning a portable robot by utilizing the time
distinction of appearance of two changed signals every one with a realized
proliferation speed (i.e., radio and ultrasound)
The initial step was to decide the attainability of the utilization of
ultrasound signs to get distances. To do that, few research center trials have
been done to assess some accessible ultrasound transmitters/collectors
utilizing wires to associate both transmitter and beneficiary. Those sensors
(HG-M40DAI and HG-M40DAII) are ultrasonic item locators and reach
locaters that offer extremely short-to-medium-range identification. The AII
model offers an exceptional restricted directional reaction, 25–30
degrees vertical way to limit the impression of undesirable sound waves. We
acquire bigger transmission separates by eliminating the front facing front of
the sensors that goes about as an anisotropic channel.
In the wake of playing out those experimentations, we presumed that
the estimation of the hour of trip of ultrasound waves could be a plausible
method of assessing distances. The assessment of distances by estimating
the hour of spread of ultrasonic waves can be helpful for a few limitation
strategies dependent on the information on certain distances. Further
experimentation showed that this strategy offered a decent execution of the
sensors for distances up to 7-8 meters.
The following stage was to present synchronization radio signs to
gauge distances between two free hubs (i.e., a robot and the individual or two

robots). To accomplish this reason, the hypothetical thought of the time
contrast of appearance (TDoA) was executed. The thought is to quantify the
hour of trip of the ultrasound signals utilizing the radio sign as synchronization
technique. The hour of trip of the radio sign is viewed as consistent for the
scope of distances that we will quantify (from 0 to a limit of 8 meters).
The most difficult errand for executing the TDoA was to discover an
equipment stage that could take those estimations, having the option to
control both, radio and ultrasound sensors. After thorough examination about
its capacities, the Handy Board, broadly utilized for showing purposes in
Jaume I University, was utilized. Some extra gadgets were vital for coupling
those segments to the board. The programming of the load up was finished
with a combination of C programming language for the worldwide daily
practice and low level computing construct for the continuous perusing and
composing of signs.
The estimation abilities of the sheets were tried, and the outcome is
that the got estimations were very exact, having the option to compare 8
meters with a most extreme blunder of around 3 cm
3.8 Trilateration for Position Determination:
The model introduced in this paper a few estimation sheets to decide
the 2D situation of the individual. The two radio/ultrasound estimation
frameworks are isolated by a known distance.
With this setup and utilizing a mathematical trilateration technique for a
no algebraic approach dependent on valuable mathematical contentions), we
can gauge the good ways from a producer situated at P point. The P point is

the situation of the individual that we need to find.
To get the (x,y)(x,y) directions of the P point we continue as follows:
the P point is in the convergence of the two circuits C 1C1 and C2C2. Leave the
condition of the circuit alone (x−a)2+(y−b)2=r2(x-a)2+(y-b)2=r2, where (a,b)(a,b)
is the middle, r is the span, and (x,y)(x,y) are the marks of the outline.
Knowing both focuses (−dr/2,0)(- dr/2,0) and (dr/2,0)(dr/2,0), and the
distances d1d1, d2d2 from the item situated at P(x,y)P(x,y), we can compose
the accompanying arrangement of conditions:
Equation 3.1
Solving this system of equations, we obtain the point P(x,y)P(x,y)
where the emitter is located:
Equation 3.2
The condition for the y arranges gives two arrangements (positive and
negative), however we just utilize the positive arrangement since ultrasound
sensors are just accepting signs from both y positive quadrants, as they are
mounted in sure y heading and not in regrettable y bearing. When we know
the general situation of the P point concerning the robot, a control calculation
registers the resultant straight and rakish velocities for controlling the robot.

3.9 Trilateration Geometrical Error Study:
To evaluate the blunder delivered by singular estimations when utilizing
the above trilateration technique, a mathematical report was made for the
most widely recognized arrangements.
When utilizing the carried out trilateration procedure, two autonomous
estimations d1d1 and d2d2 are taken by two estimating gadgetsfrom the
getting focuses (−dr/2,0)(- dr/2,0) and (dr/2,0)(dr/2,0) to the item situated at
P(x,y)P(x,y), outfitted with a transmission gadget. Singular estimation
mistakes in d1d1 and d2d2 produce a vulnerability region while computing
P(x,y)P(x,y) by trilateration.
For instance, in, we present a mathematical portrayal of the
vulnerability regions in which the point P(x,y)P(x,y) could be on the off chance
that we consider an individual mistake for every estimation of ±5 cm (an upper
bound, regular blunder is around ±1 cm). The vulnerability region is addressed
for expanding good ways from the producer to the collectors (from 1 to 8
meters).the producer is arranged at 40° from the opposite to the section that
associates the two beneficiaries. While expanding the distance, the
vulnerability region increases for the level facilitate (gets more extensive)
however stays little for the upward organize. The flat vulnerability is hilter kilter
because of the rakish removal of the producer. The mistake area increments
with the separation from the transmitter to the beneficiaries however it very
well may be decreased by expanding the partition distance (dr) between
recipients that, practically speaking, is restricted by the robot measurements.
Indeed, even this hypothetical blunder study uncovers that the position

dictated by the strategy can be influenced by a significant level mistake; by
and by, the individual measures acquired from the gadgets are a lot more
modest than the hypothetical greatest, so the situation of the individual can be
precisely decided and followed by utilizing a basic sifting procedure.
3.10 Infrared Sensor:
An infrared sensor is an electronic instrument that is used to sense
certain characteristics of its surroundings. It does this by either emitting or
detecting infrared radiation. Infrared sensors are also capable of measuring
the heat being emitted by an object and detecting motion.
An IR sensor can measure the heat of an object as well as detects the
motion. These types of sensors measure only infrared radiation, rather than
emitting it that is called a passive IR Sensor.
3.11 Infrared Radiation Theory
Infrared waves are not visible to the human eye. In the electromagnetic
spectrum, infrared radiation can be found between the visible and microwave
regions. The infrared waves typically have wavelengths between 0.75 and
1000µm.

Figure 3.6 Infrared Sensor Module has built-in IR transmitter and IR receiver that sends out IR
rays and looks for reflected IR rays to detect presence of any object
The infrared spectrum can be split into near IR, mid IR and far IR. The
wavelength region from 0.75 to 3µm is known as the near infrared region. The
region between 3 and 6µm is known as the mid-infrared region, and infrared
radiation which has a wavelength greater higher than 6µm is known as far
infrared.
Figure 3.7 Working Principle of IR sensor

The product engineering incorporates a C++ worker running in the
robot's ready PC, which gives a TCP/IP interface to an Android gadget. Right
now of composing the worker is running on a Windows Virtual Machine,
because of certain drivers issues, which made it difficult to run the worker
straightforwardly on the Linux working framework and ROS stage.

CHAPTER 4
PERSON FOLLOWING FUNCTIONALITY

4.1 User Interface:
In this unique circumstance, the client can, for instance, change the
accompanying distance, just as adjusting the accompanying velocity and the
robot turn delay. This will permit the client to tune the way he/she needs the
robot to perform. Also, by utilizing this interface, the client can know other
robot boundaries, similar to the battery status, etc.
Figure 4.1 Person following behaviour 4-state machine
The individual after usefulness is executed in a PC program designated
on the on-board PC of the robot. The PC runs a Ubuntu 12.04 LTS dispersion,
and the robot is controlled utilizing ROS, because of the erratic_player hub
that wraps the Erratic driver found in Player project. The framework
usefulness is halfway executed in C++ to carry out the individual after
usefulness and to interface with ROS and the equipment stage regulator.

The individual after usefulness is executed utilizing a 4-state machine.
Each state addresses one diverse situation of the followed individual as for
the robot .
4.2 The 4 states are portrayed as follows:
Follow: the current position P(x,y)P(x,y) of the followed individual is into
the functioning region labeled as Follow. The control program registers the
robot speed such that the situation of the individual P(x,y)P(x,y) merges to the
ideal position P(x′,y′)P(x′,y′). To accomplish this objective, the turn paces of
both robot wheels are processed utilizing a corresponding regulator.
Figure 4.2 Person following behaviour: the person can be in one of the 3 shown regions
Wait_1: the current position P(x,y)P(x,y) of the followed individual is
into the closest region, labeled as Wait_1, a ways off from the robot that is not

exactly that ideal. The present circumstance occurs, for instance, when the
individual returns to take something different from the racks, or when the
individual places something into the truck. In this express, the robot can act in
two unique manners relying upon the client setup: to quit moving, with the
goal that the individual can move toward the robot to put something on the
container or to somewhat move in reverse, giving more opportunity to the
client when moving advances and in reverse. In this last case, the client can
leave things in the crate by drawing nearer to the robot from the side (Wait_2
state).
Wait_2: in this express, the individual has shifted aside from the robot
and leaves the inclusion space of ultrasound collectors. The present
circumstance occurs, for instance, when the individual needs to leave
something on the crate, or when the individual shifts course. For this situation
the robot sits tight for a configurable timeframe (turn delay), simply on
the off chance that the individual returns to the inclusion region. In the event
that the individual returns to the inclusion region, the new state will be Wait_1
or Follow. Something else, in the wake of holding up a measure of time, the
program will progress to the Search state.
Search: the robot begins turning gradually to look for the individual.
The present circumstance occurs, for instance, after the individual has
adjusted course. When the individual has been found, the conceivable next
states will be Follow or Wait_1.

CHAPTER 5
SIMULATION RESULTS

A few 2D reproductions of the robots utilizing Player/Stage and ROS
have been completed to test the practicality and versatility of the introduced
framework and are portrayed beneath. The recordings of the reproductions
are accessible online.
5.1 Small Grocery Store Simulation
The principal reenactment reproduces a little supermarket in which
there are five individuals doing shopping and five robots every one after one
of the five people. The climate depends on a little basic food item, comprising
of three passageways with racks on each side. The clients move along the
halls and stop before the racks to pick an item.
We were especially keen on the versatility of our framework; that is,

how might the framework proceed as the quantity of robots increments?
Because of the impedances between the sonar sensors of various robots, the
recurrence of the beats is diminished. In this recreation the situating
framework gives new estimations at a pace of only 2 Hz. Such low recurrence
permits the gathering of robots to make estimations of every individual thusly,
without covering.
Figure 5.1 Simulation of five persons and robots on a small grocery, consisting of three
corridors with shelves on each side.

Robots are customized to follow the people at an objective distance of
1 m. Every robot follows the individual portrayed in a similar shading. Figure
16 presents the distances between every individual and the relating supporter
robot, during a time frame seconds. In such a stretch, a typical individual
performs somewhere in the range of three and five stops and movements.
At the point when an individual begins to move, the distance to the
robot increments immediately, with tops at about 2.2–2.4 m, since the speeds
of robots are restricted, and the input control circle is absolutely relative. Be
that as it may, the situating framework directs the robot towards the individual,
following the fitting way until the individual stops again before a rack.
Upon the individual moving toward the robot under a given limit, it will
move gradually in reverse to keep the preselected distance (e.g., see the
distance of the robot r_5 in the plot between seconds 60 and 70), as indicated
by the individual after usefulness portrayed in Section 4.
5.2 Large Grocery Store Simulation:
The past devotee calculation turns out just for little distances. We have
tried a more perplexing adherent conduct in a bigger shopping climate.
Portrays the new recreation climate, comprising of a bigger shop with a higher
number of hallways and racks. The robots are customized to compute a
direction towards the situation of the individual, given the guide of the climate,
and a legitimate self-situating framework. We have run a reproduction of the
five individuals doing shopping over a time of six minutes. Distances between
every individual and the relating supporter robot are appeared. Every robot
refreshes the situation of the followed individual at 1 Hz. The prearranged

distance is 1 m and more often than not the robots stay inside a distance to
the individual more diminutive than 2.5 m. Yet, the distance may increment
because of the postponement in the update, and the moving obstructions
(different people or potentially robots). With our accessible sensors, the
situating of the individual can be dependably decided up to a distance of 8 m
(limit appeared in the figure). Two robots (r_1 and r_4) approach such
distance in two moments of the reproduction. In such cases, the robot
should pause and trust that the individual will go in close vicinity to the
locationdistance again.
Figure 5.2 Simulation of five people shopping in a larger environment

CHAPTER 6
FIELD EXPERIMENTS (I)

6.1 Person following Functionality in a Small Grocery Store
To assess the convenience, possibility, and most helpful robot
following boundaries in a genuine climate, a few tests were done in a little
supermarket. The framework was tried with a little pilot gathering of old
individuals, with the plan to get some input identified with ease of use and
sufficient robot boundaries. Every one of them were needed to do their typical
shopping, following around the direction appeared.
Two diverse direct after speeds were chosen, first at 0.2 m/s and
afterward at 0.3 m/s, following the inclinations of the clients. For each chose
speed, the separation from the robot to the individual was recorded during a
time frame seconds, as in the principal recreation try. The point presently is to
test the vigor of the proposed following technique at low estimating rates (i.e.,
2 Hz) coming from the situating framework. Likewise, unique after distances
were tried d={0.2,0.5,1.0}d={0.2,0.5,1.0}, to notice the conduct at various
straight rates. As can be found in (direct speed 0.2 m/s), when the individual
begins to move, the distance to the robot increments and the robot attempts
to limit it (utilizing a straightforward relative regulator) to the preselected
following distances portrayed in level lines. The little wave design that can be
seen in every one of the diagrams compares to the strolling design, as the
ultrasonic ring was appended to the individual's leg. This example doesn't
influence the devotee usefulness in any capacity.
As can likewise be noticed, when the individual does a stop to get
something, the distance diminishes as the robot moves toward the individual.
This happens quicker in as the direct speed is 0.3 m/s, marginally more

prominent than in the tests portrayed in (0.2 m/s). At the point when the
individual methodologies the robot, for instance, to place something into the
bushel, the robot stays static if the client comes closer from the side, as
indicated by the state machine clarified in Section 4. Oppositely, if the client
comes closer from the front, the robot starts to move gradually in reverse to
keep the predefined distance, giving more opportunity to the client while
moving along the hall.
For the tried after distances, the robot acts satisfactorily, yet on the off
chance that more noteworthy distances were required (i.e., a client that
inclines toward a more prominent robot distance), an impediment aversion
framework ought to be utilized to stay away from crashes. To handle this
issue, the stage is likewise outfitted with a Hokuyo URG-04LX Scanning laser
range locater (LRF) and utilizes a nearby organizer for snag evasion, in light
of the Dynamic Window Approach to neighborhood route on a plane. Given
the reference position of the individual to follow and a costmap got by a laser
range locater, the nearby organizer produces speed orders to ship off the
portable base. The data for the sonar should be joined with the laser
estimations all together for the distinguished individual not to be treated as a
hindrance. By the by, with the utilization of a LRF this gadget increments
essentially the expense of the stage, so as another option, a lot less
expensive optical triangulation-based distance sensor (like the Sharp GP2D12
distance estimating sensor) could be utilized all things being equal.
Further subtleties, similar to a video with the individual after framework
in real life and some fundamental ease of use results can likewise be seen
online.

CHAPTER 7
FIELD EXPERIMENTS (II)

7.1 Usability Tests with Two Robots in a Large Grocery
Store
In light of the main client investigations and ideas, an improved
rendition of the robot has been planned, and a first model was tried in a bigger
supermarket. In the advanced adaptation, a standard shopping basket pack
was fitted to the robot through a PVC structure, improving the ergonomics and
the feel. The past model had a 22 × 25 × 40 cm (H×L×WH×L×W) sack, while
the improved one is 65 × 29 × 44 cm. The upper piece of the PVC design can
be utilized as a handle both to somewhat incline toward it and to move the
truck. Likewise, the tallness of the sack has been expanded, making it simpler
to put articles inside and expanding also the limit of the truck. Beneath the
sack, a particular ensured space has been empowered to hold the PC and
sensors.
In this subsequent examination, the new stage was teleoperated,
utilizing a distant PC associated remotely to the PC on board the robot, and
utilizing the ROS teleop bundle. The target at this prototyping stage was to get
however much input as could reasonably be expected from clients before the
plan and advancement of an improved individual after usefulness. Five unique
individuals were approached to do typical shopping in a supermarket, going
about as regular as could really be expected and disregarding pushing the
shopping basket.
The client encounters were gathered in a progression of meetings.
Every one of them showed that this new form is more agreeable than the past
one, basically with the new plan of the sack. A rundown of the primary

thoughts brought by the clients to improve the framework is the
accompanying.
Make the robot considerably lighter, to permit the client to move it
physically in the event of need.
Plan a device or lift that assists senior with peopling to remove the
merchandise in a basic and simple way.
The utilization of a clever arm watch rather than an appendable belt to
the leg.
Also, a bunch of four inquiries was posed to the four individuals that
tried the robots. Their reactions to a Likert 1–5 scale are summed up in a
graph. As can be seen, every one of them are exceptionally happy with the
new plan, and they principally propose the utilization of an insightful arm
watch as a collaboration gadget with the robot.

CHAPTER 8
CONCLUSION AND FUTURE WORK

In this paper we have introduced an individual after framework for a
portable help robot that depends on a blend of radio and ultrasound signs to
perform estimations between the individual and the robot. One of the
destinations of the work was to foster a straightforward, dependable, and
simple to utilize framework that could give opportunity of developments to the
older individuals. To do as such, the versatile robot follows the individual by
perusing the signs from a ultrasound ring connected to the individual's leg.
A second significant goal was the client agreeableness. In the wake of
testing the framework for certain expected clients, we had the option to
presume that this shopping basket robot is very much acknowledged among
old individuals, since they comprehend that it is an improvement in their lives,
a mechanical development, rather than something that replaces their actual
abilities.
Figure 8.1 Future Vision Robotic Shopping cart

As future work, we are wanting to incorporate the as of late proposed
indoor radio-based confinement framework into the shopping basket. This
framework depends on fingerprinting methods and can be applied to robot
and individual confinement to know their situation in a specific climate, simply
utilizing just the WiFi/ZigBee handset data, and giving a 1.5 m limitation
mistake. This could improve the robot route inside a mall, and the robot could
likewise show the individual where to go to track down a specific item.
Consolidating the radio/sonar restriction for route and the fingerprinting
worldwide methodology, more modern applications will be conceivable.
Indeed, further work can, for instance, give a more natural UI by giving the
chance of looking to the accessible items in the store, making a list of things
to get, and allow the robot to direct the client to the appropriate spot in a
proficient and independent way. This would require a superior mix of the robot
interface with the store information base. Then again, if the client needs to
lead the route, the cell phone can give directions to discover the items
identified with the real situation of the robot and the client.
In addition, a few trials are being performed to improve the worldwide
confinement arrangement of the individual by utilizing the eZ430-Chronos
watch, a gadget that empowers the incorporation of a complex radio-based
limitation framework and gives a bidirectional correspondence connect among
client and the robot.

REFERENCES

Ahmad, M.F., Alhady, S.S.N., Kaharuddin, S., Othman, W.A.F.W.: Visual
based sensor cart follower for wheelchair by using microcontroller. In:
ICCSCE (2015)
Gross, H.-M., Boehme, H., Schroeter, C., Mueller, S., Koenig, A., Einhorn, E.,
Martin, C., Merten, M., Bley, A.TOOMAS: interactive shopping guide
robots in everyday use—final implementation and experiences from
long-term field trials Proceedings of the IEEE/RSJ International
Conference on Intelligent Robots and Systems (IROS ′09) October
2009.
Hung-Chi Chu, ―Design and implementation of an auto-following robot-cart
system for the elderly‖, 2016, IEEE.
Ilias, B., Nagarajan, R., Murugappan, M., Helmy, K., Awang Omar, A.S.,
Abdul Rahman, M.A.: Hospital nurse following robot: hardware
development and sensor integration. Int. J. Med. Eng. Inf. 6(1), 1–13
(2014)
Kanda, T., Shiomi, M., Miyashita, Z., Ishiguro, H., Hagita, N.A communication
robot in a shopping mallIEEE Transactions on
Robotics201026589791310.1109/TRO.2010.20625502-s2.0-
77957700115
Katic, D., Rodic, A.Intelligent multi robot systems for contemporary shopping
mallsProceedings of the 8th International Symposium on Intelligent
Systems and Informatics (SISY ′10)September 2010Subotica,
Serbia109113
Kobayashi Y, Yamazaki S, Takahashi H, Fukuda H, Kuno Y (2018) Robotic

shopping trolley for supporting the elderly. In: International conference
on applied human factors and ergonomics. Springer, Cham, pp 344–
353
Kobilarov, M., Sukhatme, G., Hyams, J., Batavia, P.: People tracking and
following with mobile robot using an omnidirectional camera and a
laser. In: International Conference on Robotics and Automation (2006)
Kohtsuka T, Onozato T, Tamura H, Katayama S, Kambayashi Y (2011)
Design of a control system for robot shopping carts. In: International
conference on knowledge-based and intelligent information and
engineering systems, Springer, Berlin, Heidelberg, pp 280–288
Leo Louis, ―Working Principle of Arduino and using it as a tool for Study and
Research‖, 2018, International Journal of Control, Automation,
Communication and Systems (IJCACS).
Marin-Hernandez A, de Jesus Hoyos-Rivera G, Garcia-Arroyo M, Marin-Urias
LF (2012) Conception and implementation of a supermarket shopping
assistant system. In: 2012 11th Mexican international conference on
artificial intelligence, IEEE, New York, pp 26–31
Ng, Y.L., Lim, C.S., Danapalasingam, K.A., Tan, M.L.P., Tan, C.W.:
Automatic human guided shopping trolley with smart shopping
system. Jurnal Teknologi 73(3) (2015)
Nishimura S, Takemura H, Mizoguchi H (2007) Development of attachable
modules for robotizing daily items-Person following shopping cart
robot. In: 2007 IEEE international conference on robotics and
biomimetics (ROBIO), IEEE, New York, pp 1506–1511

Rawashdeh, N.A., Haddad, R.M., Jadallah, O.A., To’ma, A.E.: A person-
following robotic cart controlled via a smartphone application: design
and evaluation. In: REM (2017)
Wikipedia contributors. (2021, May 30). Arduino. Wikipedia. ACCESSED ON
MARCH 14.2021 https://en.wikipedia.org/wiki/Arduino

The Human Following Shopping Cart Assistance Robot

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

The Human Following Shopping Cart Assistance Robot

Uploaded by

Copyright:

Available Formats

“THE HUMAN FOLLOWING SHOPPING CART

MUHAMMAD SAQIB ASLAM

MUHAMMAD FARHAD NAZIR

DEPARTMENT OF INFORMATION SCIENCES

THE HUMAN FOLLOWING SHOPPING

MUHAMMAD SAQIB ASLAM HAMZA KHAN

SHAFIQ UR REHMAN M. FARHAD NAZIR

A thesis submitted in partial fulfillment of the

Department of Information Sciences, University of Education Lahore (Multan Campus)

Student Name: Student Name:

Student Name: Student Name:

Project Supervisor Convener (Project Evaluation Committee)

Name: Dr. Sajid Ali Name: Dr. Muhammad AsadMeraj

Designation: Associate Professor Designation: Principal

University of Education, Multan campus University of Education, Multan campus

Member (Project Evaluation Committee) Member (Project Evaluation Committee)

Name: Dr. Sajid Ali Name: Designation: Associate

Department of Information Sciences, University of Education Lahore (Multan Campus)

Bachelors in Information Technology.‖

Project SupervisorName: Dr. Sajid Ali

Designation: Associate Professor

University of Education, Multan Campus

Department of Information Sciences, University of Education Lahore (Multan Campus)

Student Name: Student Name:

Student Name: Student Name:

Department of Information Sciences, University of Education Lahore (Multan Campus)

We express my heartiest respect and gratitude to my internship supervisor

Department of Information Sciences, University of Education Lahore (Multan Campus)

CHAPTER 1 INTRODUCTION ........................................................................ 1

CHAPTER 2 REVIEW OF DEVELOPED SYSTEMS: ..................................... 4

2.1 Information and Help Systems: ...................................................... 5

2.2 Shopping Cart Robots: ................................................................... 5

2.3 Robot Person following Behavior: .................................................. 7

CHAPTER 3 SYSTEM DESCRIPTION: THE SHOPPING CART

3.1 The Mobile Platform: .................................................................... 11

3.2 Arduino UNO ................................................................................ 12

3.3 Motor Driver:................................................................................. 13

3.4 Gear Motor: .................................................................................. 14

3.5 Servo Motor.................................................................................. 16

3.6 Ultrasound Localization System: .................................................. 16

3.7 TDoA with RF and Ultrasound: ..................................................... 18

3.8 Trilateration for Position Determination: ....................................... 19

3.9 Trilateration Geometrical Error Study: .......................................... 21

3.10 Infrared Sensor:............................................................................ 22

3.11 Infrared Radiation Theory ............................................................. 22

CHAPTER 4 PERSON FOLLOWING FUNCTIONALITY ............................. 25

4.1 User Interface:.............................................................................. 26

4.2 The 4 states are portrayed as follows: .......................................... 27

CHAPTER 5 SIMULATION RESULTS ......................................................... 29

5.1 Small Grocery Store Simulation ................................................... 30

5.2 Large Grocery Store Simulation: .................................................. 31

Department of Information Sciences, University of Education Lahore (Multan Campus)

CHAPTER 6 FIELD EXPERIMENTS (I) ........................................................ 33

6.1 Person following Functionality in a Small Grocery Store .............. 34

CHAPTER 7 FIELD EXPERIMENTS (II) ....................................................... 36

CHAPTER 8 CONCLUSION AND FUTURE WORK..................................... 39

Department of Information Sciences, University of Education Lahore (Multan Campus)

light weight, high torque and low RPM. .................................................................... 15

MOTOR TOWER ..................................................................................................... 16

Figure 3.5 Ultrasonic Distance Sensor ..................................................................... 17

Figure 3.7 Working Principle of IR sensor ................................................................ 23

Figure 4.1 Person following behaviour 4-state machine ........................................... 26

three corridors with shelves on each side. ............................................................... 30