FACULTY OF ELECTRICAL

AND ELECTRONIC ENGINEERING

2020 – 2021
-------*-------

CAPSTONE PROJECT I
TOPIC: HUMAN FOLLOWING ROBOT

Instructor: Assoc. Prof Ph.D Huynh Thai Hoang

Students: Le Ngoc Thinh – 1751094

Chau Nguyen Thai Son – 1751079

Ho Chi Minh City, 1/2021

Capstone project 1
Instructor: Huynh Thai Hoang

CONTENTS
1. INTRODUCTION..............................................................................................................................1

1.1 Abstract..................................................................................................................................1

1.2 Objective................................................................................................................................1

2. THEORY...........................................................................................................................................2

3. IMPLEMENT..................................................................................................................................14

4. RESULT..........................................................................................................................................23

5. CONCLUSION, APPLICATION AND FUTURE WORK........................................................................23

5.1 Conclusion............................................................................................................................23

5.2 Application............................................................................................................................23

5.3 Future work...........................................................................................................................23

6. REFERENCE...................................................................................................................................24

2
Capstone project 1
Instructor: Huynh Thai Hoang

1. INTRODUCTION

1.1 Abstract

For a robot that operates autonomously, the interaction between the person and
the robot is the major factor. In order for a robot to communicate and interact with the
person, it should also be capable of following that particular person. Keeping this in
mind, there should be a capacity in the robot to get information from the surroundings
while pursuing the required object. The primary goal of our work was to design and
fabricate a robot that not only tracks the target but also moves towards it while doing
the tracking. The main hindrance in this kind of work is that the detection of the target
is a sensitive thing to carry out. The object has to be unique for the robot to recognize it
and carry out the objective. An ultrasonic sensor is used to protect the robot from
collision with the object and keep a constant distance between human and robot. All the
processing is carried out by the microprocessor while the control of the motors is
carried out by the controller.

1.2Objective

Robotic technology has increased appreciably in the past couple of years. Such
innovations were only a dream for some people a couple of years back. But in this rapid
moving world, now there is a need for robots such as “A Human Following Robot” that
can interact and co-exist with them.

Robot requires a mechanism to perform this task accurately. This helps the
individual to envision and act accordingly. The robot has to be intelligent enough to
obey a human in crowded places, a vibrant setting, indoors and outdoors places.

Typically, human following robots are equipped with several different diverse
combination of sensors i.e... light detection and ranging sensor, radio frequency
identification module (RFID), laser range finder (LFR), infrared (IR) sensing modules,
thermal imaging sensors, camera, wireless transmitter/receiver etc. for recognition and
locating the target. All the sensors and modules work in unison to detect and follow the
target.

1
Capstone project 1
Instructor: Huynh Thai Hoang
The capability of a robot to track and follow a moving object can be used for
several purposes.

● To help humans.
● To create ease for people.
● Can be used for defense purposes.

In this paper, we presented a method of a human following robot based on

identification and detection by using an ultrasonic sensor and IR sensor which is
processed by STM32 microcontroller.

2. THEORY

2.1 Sensor

2.1.1 HC-SR04 (Ultrasonic)

What is Ultrasonic?

Ultrasonic is high-pitched sound waves with frequencies higher than the

audible limit of human hearing.

Human ears can hear sound waves that vibrate in the range from about
20 times a second (a deep rumbling noise) to about 20,000 times a second (a
high-pitched whistling). However, ultrasound has a frequency of over 20,000 Hz
and is therefore inaudible to humans.

HC-SR04 Ultrasonic distance sensor consists of two ultrasonic

transducers. The one acts as a transmitter which converts electrical signal into
40 KHz ultrasonic sound pulses. The receiver listens for the transmitted pulses. If
it receives them it produces an output pulse whose width can be used to
determine the distance the pulse travelled. As simple as pie!

The sensor is small, easy to use in any robotics project and offers excellent
non-contact range detection between 2 cm to 400 cm (that’s about an inch to 13
feet) with an accuracy of 3mm.

2
Capstone project 1
Instructor: Huynh Thai Hoang

Operating Voltage DC 5V

Operating Current 15 mA

Operating Frequency 40kHZ

Max Range 4m

Min Range 2 cm

Ranging Accuracy 3mm

Measuring Angle 15 degree

Trigger Input Signal 10 uS TTL pulse

Dimension 45 x 20 x 15mm

HC-SR04 Pinout

VCC: is the power supply for HC-SR04 Ultrasonic distance sensor which we
connect the 5V pin on the STM32F103C8T6

Trig (Trigger): pin is used to trigger the ultrasonic sound pulses.

Echo: pin produces a pulse when the reflected signal is received. The length of
the pulse is proportional to the time it took for the transmitted signal to be detected.

GND: should be connected to the ground of stm32f103c8t6.

When a pulse of at least 10 µS (10 microseconds) in duration is applied to the

Trigger pin. In response to that the sensor transmits a sonic burst of eight pulses at 40
KHz. This 8-pulse pattern makes the “ultrasonic signature” from the device unique,

3
Capstone project 1
Instructor: Huynh Thai Hoang
allowing the receiver to differentiate the transmitted pattern from the ambient
ultrasonic noise.

The eight ultrasonic pulses travel through the air away from the transmitter.
Meanwhile the Echo pin goes HIGH to start forming the beginning of the echo-back
signal.

In case, if those pulses are not reflected back then Echo signal will timeout after
38ms and return low. Thus, a 38mS pulse indicates no obstruction within the range of
the sensor.

If those pulses are reflected back the Echo pin goes low as soon as the signal is
received. This produces a pulse whose width varies between 150 µS to 25 mS,
depending upon the time it took for the signal to be received.

The width of the received pulse is then used to calculate the distance to the
reflected object.

Formula:

Distance = Speed x Time

2.1.2 Infrared (IR)

An infrared sensor is an electronic device that emits in order to sense some

aspects of the surroundings. An IR sensor can measure the heat of an object as well as
detects the motion.

These types of radiations are invisible to our eyes, which can be detected by an
infrared sensor. The emitter is simply an IR LED (Light Emitting Diode) and the detector
is simply an IR photodiode that is sensitive to IR light of the same wavelength as that
emitted by the IR LED. When IR light falls on the photodiode, the resistances and the
output voltages will change in proportion to the magnitude of the IR light received.

Working Principle

4
Capstone project 1
Instructor: Huynh Thai Hoang
This sensor includes an IR LED & an IR Photodiode, so by combining these two
can be formed as a photo-coupler otherwise optocoupler.

IR LED is one kind of transmitter that emits IR radiations. This LED looks similar
to a standard LED and the radiation which is generated by this is not visible to the
human eye. Infrared receivers mainly detect the radiation using an infrared transmitter.
These infrared receivers are available in photodiodes form. IR Photodiodes are
dissimilar as compared with usual photodiodes because they detect simply IR radiation.
Different kinds of infrared receivers mainly exist depending on the voltage, wavelength,
package, etc.

Once it is used as the combination of an IR transmitter & receiver, then the

receiver’s wavelength must equal the transmitter. Here, the transmitter is IR LED
whereas the receiver is IR photodiode. The infrared photodiode is responsive to the
infrared light that is generated through an infrared LED. The resistance of photo-diode
& the change in output voltage is in proportion to the infrared light obtained. This is the
IR sensor’s fundamental working principle.

Types of Infrared Sensor

Active IR Sensor

This active infrared sensor includes both the transmitter as well as the receiver.
In most of the applications, the light-emitting diode is used as a source. LED is used as a
non-imaging infrared sensor whereas the laser diode is used as an imaging infrared
sensor.

These sensors work through energy radiation, received & detected through
radiation. Further, it can be processed by using the signal processor to fetch the
necessary information. The best examples of this active infrared sensor are reflectance
and break beam sensor.

Passive IR Sensor

The passive infrared sensor includes detectors only but they don’t include a
transmitter. These sensors use an object like a transmitter or IR source. This object

5
Capstone project 1
Instructor: Huynh Thai Hoang
emits energy and detects through infrared receivers. After that, a signal processor is
used to understand the signal to obtain the required information.

The best examples of this sensor are pyroelectric detector, bolometer,

thermocouple-thermopile, etc. These sensors are classified into two types like thermal
IR sensor and quantum IR sensor. The thermal IR sensor doesn’t depend on wavelength.
The energy source used by these sensors is heated. Thermal detectors are slow with
their response and detection time. The quantum IR sensor depends on the wavelength
and these sensors include high response and detection time. These sensors need regular
cooling for specific measurements.

2.2 H - Bridge (L298N)

Controlling a DC Motor

In order to have a complete control over DC motor, we have to control its speed
and rotation direction. This can be achieved by combining these two techniques.

● PWM – For controlling speed

● H-Bridge – For controlling rotation direction

PWM – For controlling speed

The speed of a DC motor can be controlled by varying its input voltage. A

common technique for doing this is to use PWM (Pulse Width Modulation)

PWM is a technique where the average value of the input voltage is adjusted by
sending a series of ON-OFF pulses.

The average voltage is proportional to the width of the pulses known as Duty
Cycle.

6
Capstone project 1
Instructor: Huynh Thai Hoang
The higher the duty cycle, the greater the average voltage being applied to the DC
motor (High Speed) and the lower the duty cycle, the less the average voltage being
applied to the DC motor (Low Speed).

H-Bridge – For controlling rotation direction

The DC motor’s spinning direction can be controlled by changing polarity of its

input voltage. A common technique for doing this is to use an H-Bridge.

An H-Bridge circuit contains four switches with the motor at the center forming
an H-like arrangement.

Closing two particular switches at the same time reverses the polarity of the
voltage applied to the motor. This causes a change in spinning direction of the motor.

Control Pins

7
Capstone project 1
Instructor: Huynh Thai Hoang
For each of the L298N’s channels, there are two types of control pins which allow
us to control speed and spinning direction of the DC motors at the same time viz.
Direction control pins & Speed control pins.

Direction Control Pins

Using the direction control pins, we can control whether the motor spins
forward or backward. These pins actually control the switches of the H-Bridge circuit
inside L298N IC.

The module has two direction control pins for each channel. The IN1 and IN2
pins control the spinning direction of the motor A while IN3 and IN4 control motor B.

The spinning direction of a motor can be controlled by applying either a logic

HIGH (5 Volts) or logic LOW (Ground) to these inputs.

Input 1 Input 2 Spinning Direction

Low (0) Low (0) Motor OFF

High (1) Low (0) Forward

Low (0) High (1) Backward

High (1) High (1) Motor OFF

Speed Control Pins

The speed control pins viz. ENA and ENB are used to turn the motors ON, OFF
and control its speed.

Pulling these pins HIGH will make the motors spin, pulling it LOW will make
them stop. But, with Pulse Width Modulation (PWM), we can actually control the speed
of the motors.

The module usually comes with a jumper on these pins. When this jumper is in
place, the motor is enabled and spins at maximum speed. If you want to control the

8
Capstone project 1
Instructor: Huynh Thai Hoang
speed of motors programmatically, you need to remove the jumpers and connect them
to PWM-enabled pins on the stm32f103c8t6.

L298N Motor Driver Module Pinout

VCC: pin supplies power for the motor

GND: is a common ground pin

5V: pin supplies power for the switching logic circuitry inside L298N IC.

ENA: pins are used to control speed of Motor A. Pulling this pin HIGH will make the
Motor A spin, pulling it LOW will make the motor stop.

IN1 & IN2: pins are used to control spinning direction of Motor A. When one of them is
HIGH and other is LOW, the Motor A will spin. If both the inputs are either HIGH or LOW
the Motor A will stop.

IN3 & IN4: pins are used to control spinning direction of Motor B. When one of them is
HIGH and other is LOW, the Motor B will spin. If both the inputs are either HIGH or LOW
the Motor B will stop.

9
Capstone project 1
Instructor: Huynh Thai Hoang
ENB: pins are used to control speed of Motor B. Pulling this pin HIGH will make the
Motor B spin, pulling it LOW will make the motor stop.

OUT 1 & OUT 2: pins are connected to Motor A.

OUT 3 & OUT 4: pins are connected to Motor B.

2.3 Microcontroller (STM32F103C8T6)

2.3.1 Features

■ Core: ARM 32-bit Cortex™-M3 CPU – 72 MHz, 90 DMIPS with 1.25 DMIPS/MHz –
Single-cycle multiplication and hardware division – Nested interrupt controller with 43
maskable interrupt channels – Interrupt processing (down to 6 CPU cycles) with tail
chaining.

■ Memories – 32-to-128 Kbytes of Flash memory – 6-to-20 Kbytes of SRAM.

■ Clock, reset and supply management – 2.0 to 3.6 V application supply and I/Os – POR,
PDR, and programmable voltage detector (PVD) – 4-to-16 MHz quartz oscillator –
Internal 8 MHz factory-trimmed RC – Internal 32 kHz RC – PLL for CPU clock –
Dedicated 32 kHz oscillator for RTC with calibration.

■ Low power – Sleep, Stop and Standby modes – VBAT supply for RTC and backup
registers.

■ 2 x 12-bit, 1 µs A/D converters (16-channel) – Conversion range: 0 to 3.6 V – Dual-

sample and hold capability – Synchronizable with advanced control timer –
Temperature sensor.

■ DMA – 7-channel DMA controller – Peripherals supported: timers, ADC, SPIs, I 2Cs and
USARTs.

■ Debug mode – Serial wire debug (SWD) & JTAG interfaces.

■ Up to 80 fast I/O ports – 32/49/80 5 V-tolerant I/Os – All mappable on 16 external

interrupt vectors – Atomic read/modify/write operations.

10
Capstone project 1
Instructor: Huynh Thai Hoang
■ Up to 7 timers – Up to three 16-bit timers, each with up to 4 IC/OC/PWM or pulse
counter – 16-bit, 6-channel advanced control timer: up to 6 channels for PWM output
Dead time generation and emergency stop – 2 x 16-bit watchdog timers (Independent
and Window) – SysTick timer: a 24-bit down counter.

■ Up to 9 communication interfaces – Up to 2 x I2C interfaces (SMBus/PMBus) – Up to 3

USARTs (ISO 7816 interface, LIN, IrDA capability, modem control) – Up to 2 SPIs (18
Mbit/s) – CAN interface (2.0B Active) – USB 2.0 full speed interface.

2.3.2 Pin descriptions

11
Capstone project 1
Instructor: Huynh Thai Hoang

STM32F103xx performance line LQFP48 pinout

12
Capstone project 1
Instructor: Huynh Thai Hoang

2.3.3 Block diagram

13
Capstone project 1
Instructor: Huynh Thai Hoang

14
Capstone project 1
Instructor: Huynh Thai Hoang

Flow chart

3. IMPLEMENT

3.1. HC-SR04 ultrasonic sensor interface with stm32

Ultrasonic ranging module HC – SR04 provides 2cm – 400cm non-contact

measurement function, the ranging accuracy can reach to 3mm. The modules
include ultrasonic transmitters, receiver and control circuit. The module emits an
ultrasound at 40 KHz which, after reflecting from the obstacle, bounces back to the
module.

15
Capstone project 1
Instructor: Huynh Thai Hoang
By using the travel time and the speed of the sound, we can calculate the
distance between the sensor and the obstacle.

According to the datasheet of hc-sr04, the following is required to be done :

● Keep the Trig pin HIGH for at least 10us

● The Module will now send 8 cycle burst of ultrasound at 40 kHz and
detect whether there is a pulse signal back
● IF the signal returns, the module will output a HIGH PULSE whose width
will be proportional to the range of the object.
● Distance can be calculated by using the following formula :- range = high
level time * velocity (340m/s) / 2
● It is recommended to wait for at least 60ms before starting the operation
again.

As mentioned above, the module sends a HIGH signal proportional to the

distance measured. This signal remains high for a few microseconds. To measure the
width of the signal in microseconds, I will use the Input Capture to measure this
Pulse Width.

16
Capstone project 1
Instructor: Huynh Thai Hoang

As shown above, I have selected the Input Capture Direct Mode.

Prescaler is set to 72 -> This will divide the APB clock by 72, and bring the timer
clock to 1 MHz. It is necessary because the HCSR04 sends the signal in microseconds.

ARR is set to 0xffff -> I have selected is as maximum as possible. This basically
sets the limit, up to which we can count in microseconds.

Also make sure the TIMx Capture Compare interrupt is enabled. You can enable
the Global interrupt also, if you don’t see this option

17
Capstone project 1
Instructor: Huynh Thai Hoang

I2C 1 is selected -> To connect the LCD1602

Pin PA8 -> is automatically selected as TIM1_CH1 pin. I will use it as the ECHO
Pin

Pin PA9 -> will be used as TRIG Pin, so select it as the output.

3.2. Control DC motor with stm32 and H-Bridge L298N

3.2.1 Formula derivation and calculations to generate desired/specific

PWM frequency?

Formula for calculating values for pwm generating is not very complex, it only
requires in depth knowledge of the stm32f103 clock, timer structure and timer

18
Capstone project 1
Instructor: Huynh Thai Hoang
registers. I divided the formula and method in steps and it is very easy to calculate the
values. Below formula for calculating the value’s for generating specific pwm signals.

Stm32f103 Pwm(Pulse Width Modulation) frequency calculation formula

Timer Tick Frequency: is the frequency at which the timer is completing its one
instruction cycle.

Counter Frequency: is the frequency at which we want our timer tick counter
to increment.

Pwm resolution is the number of Pwm steps required to generate a required

counter frequency. Usually we use an arbitrary number here and then adjust it
according to our need. Pwm resolution must be in between 0-65535 since this value is
stored in a 16-bit register and maximum 16-bit count is 65535.

19
Capstone project 1
Instructor: Huynh Thai Hoang

Stm32f103 pwm formula for Prescaler value

Note: Pwm Resolution and Timer Prescaler are 16-bit registers their value must not exceed
65535.

Now looking at the final formula we need two things in it. Timer Prescaler Value and Pwm
Resolution(Steps). You can assign Timer Prescaler Value or Pwm Resolution(Steps) a fixed
value and calculate the other perimeter.

3.2.2 Setup parameter on Stmcube-Mx

20
Capstone project 1
Instructor: Huynh Thai Hoang

Stm32f103c8t6 has 4, 16 bit timers, Timer 1,2,3 and 4. We can use all four
timers for generating pwm(pulse width modulation) signal. Each timer has fixed
pins on which the pwm signal can be output called channels. Timers 1 and 2 are
multiplexed with other peripherals to use their pwm function. We have to make
sure that the timer is not colliding with any other peripheral function.

Timer 3 and 4 are stand alone and they do not collide with any other
peripheral function. So it's good to use them. I am using Timer 4 in the
project/tutorial.

I am using an Internal clock source so check this check box in the timer
settings. I am using timer 4, channel 1 for pwm output so i selected channel 1.
Channel 1 corresponds to PB6 of stm32f103 microcontroller. The settings
diagram is given on the right side.

21
Capstone project 1
Instructor: Huynh Thai Hoang

I am using an internal HIS (High Speed Internal) oscillator/clock store. After

dividing the clock with prescalers at different stages the final clock that is supplied to
timer 3 is 72 MHz, translating 72 Mhz to hz is 72000000 Hz. So, the final timer input
frequency is 72000000 Hz.

3.2.3 Calculation PWM value and counter setting

From the PWM calculation formula in section 3.2.1, now we will calculate the
values pwm signal output for 50 Hz frequency of motor DC.

Timer Input Clock = 72 Mhz or 72000000 Hz

Required Frequency = 50 Hz (In Time Domain 1

second).

Counter Period/Pwm Resolution (Steps) = 200000 (I picked a

random value)

Counter Frequency = frequency required * Counter Period

Counter Frequency = 50Hz * 20000 = 1000000 Hz

Timer Prescaler value = (Timer Input Clock/Counter Frequency)-1

Timer Prescaler value = (72000000 Hz / 1000000 Hz) – 1 = 71

22
Capstone project 1
Instructor: Huynh Thai Hoang

3.2.4 GPIO pin

o Device use and connection schematic in test

STM32f103C8T6
HC-SR04

23
Capstone project 1
Instructor: Huynh Thai Hoang
IR Sensor
H- Bridge (L298N)
16x2 LCD Display and I2C

o The software used in the work and program execution

STM32 CubeMX
Keil uVision5
STM32 ST-link Utility

4. RESULT

5. CONCLUSION, APPLICATION AND FUTURE WORK

5.1Conclusion

A successful implementation of a person follower robot is illustrated in this

research. This robot does not only have the detection capability but also the tracking
and following ability as well. The tracking is basically performed on the tag and the
human is followed on the basis of that detection. It was also kept in mind that the
following capability of the robot should be as efficient as possible. The tests were
performed on the different conditions to pinpoint the mistakes in the algorithm and
correct them. The different sensors that were integrated with the robot added an
additional advantage.

5.2Application

Looking deeply into the environment or our surroundings, we will be able to

interpret that “YES” there is a need for such robots that can assist humans and can serve
them. Such a robot can be used for many purposes. With a few changings, the robot can
act as a human companion as well. Some other applications of this robot are:

● Can assist in carrying loads for people working in hospitals, libraries, airports,
etc.
● Can service people at shopping centers or public areas.
● Can assist elderly people, special children and babies.
● Can follow a particular Vehicle.

24
Capstone project 1
Instructor: Huynh Thai Hoang
5.3 Future work

There are many interesting applications of this research in different fields

whether military or medical. A wireless communication functionality can be added in
the robot to make it more versatile and control it from a large distance. This capability
of a robot could also be used for military purposes. By mounting a real time video
recorder on top of the camera, we can monitor the surroundings by just sitting in our
rooms. We can also add some modifications in the algorithm and the structure as well to
fit it for any other purpose. E-g a vehicle follower. Similarly it can assist the public in
shopping malls. So there it can act as a luggage carrier, hence no need to carry up the
weights or to pull that. Using this algorithm the robot will automatically follow that
person.

6. REFERENCE
1. Linux Embedded, “Hướ ng dẫ n lậ p trình PWM vớ i STM32”, Hướ ng dẫ n lậ p
trình PWM vớ i STM32 | Embedded System (hethongnhung.com).
2. Last Minute Engineers, “L298N with DC Motor”, In-Depth: Interface
L298N DC Motor Driver Module with Arduino (lastminuteengineers.com).
3. Last Minute Engineers, “SR04 Ultrasonic Sensor”, How HC-SR04
Ultrasonic Sensor Works & How to Interface It With Arduino
(lastminuteengineers.com).
4. Last Minute Engineers, “
5. Electronics | Project| Focus, “ What is an IR Sensor : Circuit Diagram & Its
Working”, IR Sensor : Circuit Diagram, Types Working with Applications
(elprocus.com).

25