FORK AGV : ESTIMATION OF COMBINED
MAGNETIC SENSOR WITH VISION GUIDE
APPLIED ON AUTOMATIC MOBILE
TRANSPORTER FOR NAVIGATION AND
ACCURATE POSITIONING
Mohamad Nasyir Tamara[1], Anhar Risnumawan[2], Ni’am Tamami[3], Basith Abdurrohman A.[4]
[1,2,4]
Department of Mechanical and Energy Engineerings
[3]
Department of Electrical Engineering
Politeknik Elektronika Negeri Surabaya(PENS)
[1,2,3]
nasir_meka,anhar,niam@pens.ac.id, [4]basabasy@gmail.com,
Abstract—The project creates a Fork AGV robotic navigation
system that aims to increase the speed of goods mobilization and
effectiveness at operational costs and reduce the high level of error
in the world of warehousing and industry then replaceit with
the creation of an automatic integrated navigation and control
system of AGV robots. This AGV robot prototype using
navigation with a combination of lIDAR sensor for localization
and environmental mapping processes, cameras to detect objects
that will be targeted by robots and magnetic sensor pins that will
be used by Fork AGV robot to estimate and accurate position
of robots that will become a guide to the operational motion of
the Fork AGV robot. The use and development of the sensor can
make a Fork AGV robot has better and more stable capabilities
and performs an expansion and improvement of the system from
an existing AGV robot.
Keywords—Fork AGV robot, Visual guidance, Navigation and
Estimation control
I. INTRODUCTION
(Automatic Guided Vehicle) or called AGV is a form of
robotics vehicle that has the intelligence to control the work
of arrangement, distribution, and transportation in the industry
that can be operated automatically by utilizing technological
advances that continue to improve the system it has Activ-
ities in the industry that are related to the production and
distribution process, surely a system is needed that can do the
placement of various materials as well as inventory needs and
the results of the production processes carried out by industry
[4].
However, in its implementation there were also many weak-
nesses in this system such as not being able to quickly change
positions, requiring more operational costs, inflexible,
requiring lots of space, complicated maintenance process, and
many other weaknesses. So, the main idea of creating a vehicle
that has intelligent navigation emerged as a solution to the
problem of developing the AGV robot that happened.
Seen from the development of the navigation system that
is owned by the robot transporter today, the majority use a
mapping navigation system that can recognize the position and
drawing the environmental conditions directly which is used
to be the main idea in making the AGV Fork Robot navigation
system [8].
Another features added on the navigation of the Fork AGV
Robot is the existence of a camera device that has a function
as the eye for detecting the pallet as an object target. The
camera act as an image capturer and object detector, because of
the lIDAR only sensing the environment’s condition by the
laser in 2-dimensional image visualized from the results of laser
reflection points engineered to form a flat map using SLAM.
The camera controlled by the OpenCV platform based
reference object detection feature that has been assigned by the
predefined parameters. The selection of object detection is the
most effective method for applying the AGV vehicle [3].
In addition to the visualization and guidance sensor used in
the process of recognizing the environment, it also added with
a magnetic sensor that is useful for detecting magnetic pins
scattered in the room as an estimation reference for positioning
motion of the Fork AGV robot in recognizing the surrounding
environment. The sensor will calibrate the position when the
robot moves from the initial position to the specified location
for picking or drop the pallet [10].
II. MODELLING
The division of work is divided into several subdivisions
because the title of the project is the experiment combination of
processes of various methods used as a reinforcement and
equipment process of the formation of the main purpose for
navigation of the Fork AGV robot creation. These divisions
such as localization and mapping, pallet detection objects,
magnetic pins and Graphical User Interface on the AGV Fork
robot
A. Localization and Mapping
SLAM (Simultaneous Localization and Mapping) SLAM The approximated gradient can be done using the four
is a combination of localization and mapping processes from integer coordinates P00 , P01 , P10 and P11 whi c h can be
Fork AGV robots. SLAM uses a package that has been written as:
integrating into one section based on ROS (Robot Operating
System) which is also very helpful in the management process
of the Fork AGV robot system.

B. Pin Magnetic Detection

The Magnetic Pin added for estimating the movement of
the Fork AGV robot when being operated by calibrating the
robot’s position using magnetic pulling force and will be added And then, for the derivative at points is formulated by:
to the robot using a magnetometer as a guide in estimating the
location of the robot when starting from the initial position to
set point position to be addressed.
With only IMU for navigation to compensate error from
encoders will keep AGV moving out from the planned route.
Due to the cumulative errors of the two used sensors the
position error even larger and larger. Thus it is needed external
sistem to correct the generated cumulative errors. Several
magnetic pins need to tbe installed on the surface of the floor
which usually concrete base material. The pin should be flatten
to the floor surface. Then the magnetic sensor mounted to the
bottrm of the vehicle to detect those pins. For each pin detected For 2D environment, a pose i s expressed as ξ=(px, py, ψ)T.
then the cumulative error can be corrected with precise
To seek the minimum error of the grid occupancy M(Si(ξ)) and
position.
at the specified point, the target function is:
C. Pallet Detection
Detection of a pallet object is a process used in detecting a
pallet object to be taken by the Fork AGV robot. The Pallet
Object Detection will be used by the OpenCV platform using
a single camera with the Color Object Tracking Detection
method which is recognized by looking at the color reference Where Si(ξ) is the universal coordinates of the point
owned by the pallet then the thresholding will be done so that scanned si(si, x, si, y)T . The poses of the robot is:
it selects the color rather than other objects around it.

D. Graphical User Interface

GUI Navigation used as an interactive screen of operators
with robots that displays various information about all things
that happen to robots that can facilitate operators in evaluating
and controlling the performance of Fork AGV robots.
III. METHOD
A. Hector SLAM
Hector SLAM method was chosen to estimate the real-time
motion o f t h e robot based on the high updated frequency
from laser sensor beam and the measurement of low-distance
noise to complete 2D SLAM [11]. The benefit of using Hector
SLAM is that it’s doesn’t utilize odometry, which is usually
used for UGV or USV. The Hector SLAM algorithm estimate
the probability by using the bilinear function of a grid map.
From point Pm in a continuous map, the probability of M(Pm),
and the gradient is:
Using the priority value of ξ to estimate the next pose
ξ
change, minimizing laser point errors can be expressed as:
B. Hall Effect Magnet Detection
From installed magnet pole on the floor and hall-effect
sensor module mounted beneath the robot, the navigation is
obtained based on the central axis of the magnetic flux density
distribution. The magnetic flux density can be approached by a
mathematical model to predict the magnet spot position and of
course to identify the maximum magnetic flux density which
will show the central axis. It needs to note that the magnet spot
is assumed to be a magnetic dipole to predict position next
target destination. for this calculation, the components (Bx, By,
and Bz) of the three axis magnetic flux density can be derived
from the magnet spot in three dimensional space which makes
it possible to predict the position and control the steering
angle The magnetic dipole potential for the magnet spot is
given in Eq.(8).
Here, to compute the magnetic flux density of the magnet
spots in three axis space from the position of the Hall-effect
sensor away from the magnet spot[12]:
(8)
U is the potential of a magnetic dipole, M is the Moment of
a magnetic dipole, and µ0 is the permeability of free space 4π
x 10 7 H/m:
(9)
The magnitudes of the three axis components of the
magneticflux density (Bx,By,and Bz) is expressed in Eq.(9) (11),
respectively, from the relationship between the magnetic dipole
potential and magnetic field [12]:
(10)
(11)
(12)
Here k: Ma/4π, a: unit coefficient, Bx and By are magnetic
flux density measured by Hall-effect sensor from the front
or rear side and left and right sides respectively of the
magnet spots. While Bz is the magnetic flux density obtained
from the height of the magnet spot and Hall-effect sensor Fig.
1. As shown in Fig. 2 and Fig. 3, the distribution of the
magnetic flux density of the magnet spot in three axis (x, y, z)
directions is continuously measured by the Hall-effect sensor
when passes through the magnet spots. The current position of
the AGV can be located by analyzing the distribution of
magnetic flux density which shows the relationship between
the magnet spot and Hall-effect sensor dependently. Thus, The
AGV’s navigation steering angle can be controlled effectively.
Fig. 1. Magnetic Dipole Hall Effect Sensor Detection in
Cartesian.
Fig. 2. Generation of magnetic spot and magnetic field
Principle.
Fig. 3. Relation between Hall-effect sensor and magnet spot
installation.
The x-axis is driving direction for forward or backward
movement, y-axis is left and right directions of AGV central
line and z-axis is upper or lower directions between AGV
central line and installed magnetic sensor. In this research, the
Hall-effect sensor module installed in the AGV measures the
magnetic flux density distribution from the central axis of the
magnet spot. Thus the AGV’s path error correction can be
performed by compensating cumulative error from other
sensors as informed above.
C. Pallet Object Reference detection
There are many methods for detecting the objects using
vision system. Some of theme are color detection, edge
detection, pattern detection, Neural network, or deep learning.
For this research we use object detection algorithm based on
color identification which has 3D coordinates system used in
order to identify unique specific colors. From Fig. 9, the pallet
detection algorithm relies on the essential image processing
operation, called the segmentation operation. It is performed in
HSV to make optimal use of the color space data channels,
and improves object detection accuracy. As the base color is
represented only by the hue value, which starts at red and wraps
around a cylinder through orange, yellow, green, blue, indigo
and violet that make it possible which then compared with the
RGB (Red, Green, and Blue) color, where all the three values
are primary colors. Then the segmentation process reduces the
image to regions that correspond to some specific property
practically. The rule for binary image is expressed as:
(13)
Where T1 and T2 is user-specified thresholds, a is the lower
bound of V tone of YUV color space, b is the upper bound, and
T1 > b. From the image processing process, the pallet boundary
is extended for filling holes and eroded for reducing noise.
After that, we need to calculated the 8-connected region, and
then the pallet is decided as the area that had more extensive
than the thresholding process.
 IV. SIMULATION AND RESULT EXPERIMENT
The making of the navigation system from the AGV robot
is divided into several parts that are tested and simulated
separately and then collaborate between one system with
another. For more details on testing the system used separately
is explained in the following chapter as follows:
A. SLAM Test
The SLAM test produces a map image of the environment
around the robot that is carried out directly using the Hector
SLAM method based on the Robot Operating System platform.
• Location
The hardware position used for SLAM is placed at thetop
of the lifter system of the AGV robot, shown in the
following figure:
Fig. 4. Location of LIDAR installation
• Design
From the running process on the ROS platform, different
drawing results are obtained on mapping using LIDAR
sensor in indoor and outdoor condition shown in the
following figure:
Fig. 5. Indoor mapping with Hector SLAM
The map result at the Fig. 5 is the indoor mapping from the
Vehicle Robotics Laboratory environment in real-time
condition the Hector SLAM package.
Fig. 6. Outdoor mapping and Localization with HectorSLAM
The map result at the Fig. 6 is the outdoor mappingfrom
the D3 building coridor environment in real-time condition
using the Hector SLAM package.
• Result
The results of the data from the making of SLAM canbe
observed by output with the .bag file displayed on the Ubuntu
terminal. The sensitivity factors of the LIDAR sensor
summarized with the following table:
TABLE 1. RESULT LIDAR’S ANGLE DETECTION ON
MAPPING
The results of the LIDAR measurement on TABLE 1 are
done by comparing the LIDAR data obtained viathe terminal
and by measuring directly on the sampleof selected objects
to be detected. The error detection results obtained are
approximately equal to less than 10 cm. Then the results of
localization from LIDAR were also obtained from the testing
room, namely in the east corridor of the D3 building as shown
in the following table :
TABLE 2. RESULT LIDAR’S DETECTION ON LO-
CALIZATION
From the results on TABLE 2 also found an error ofless
than 10 percent. this is because the test environment is still
within the scope of the detection capabilities of RPLIDAR
A1M8 that used. From the two result of mapping, we can get
better results if the mapping isassisted by using IMU and rotary
encoder and combined with filter for map visualization such as
using a KalmannFilter. IMU sensor supports by measuring the
inertial motion carried out by the robot by looking at the
directionof the pole to estimate the heading direction of the
 robot using a magnetometer, the stability of the robot’s
movement with the accelerometer and balance motion with
the gyroscope and also reinforcement through a rotary
encoder to see the actual movement of the robotin the field.
B. Pin Magnetic Detection Test
The magnetic pin using neodymium magnets are mounted
on a 3x3 combination magnet in-plane box to be easily
detected by transistor magnetic hall effect sensor that installed.
• Design
by six magnetic sensors at once. The result means that the
maximum positioning estimation can be made up to six
magnetic pins when located in the middle of a row of
magnetic sensors. Ifthe maximum value that can be detected is
directly on sixmagnetic sensors, the bit detection value of the
magnetic pin that obtained entirely shown in the following
table:
TABLE 4. CLASSIFICATION FOR HEADING TEST

The design of the magnetic pin sensor is shown in the Fig.7

below :

Fig. 7. Board PCB Design of Pin Magnetic Sensor

The magnetic sensor is categorized as low budget sensor
module with the code KY-026 based on Arduino, which is
installed 16 pieces in a row. In addition, the accuracy of the
heading direction of the robot is also carried out usingthe
Magnetometer robot so that it can make a rotational motion
towards the direction of the robot to be addressed.
• Result
The results of magnetic detection measurements will be
tested as the test sketch form below:

Fig. 8. Test Pin Magnetic Sensor

From the magnet detection using the sensor on Fig.8result
will be calibrated using magnetometer for the reference of the
robot’s heading direction in single and multiple sensor
condition that the result shown as below:

TABLE 3. RESULT ON CALIBRATION DIREC-

TION USING MAGNETOMETER
The results of the magnetic measurement at TABLE 3 are
done by comparing the detection data obtained via serial
monitors and by measuring directly on selected sample objects
to be detected. The results of single detection are 16
degrees, and the better results shown by multi- detection are
only 4 degrees. From the direct test result can be seen that one
magnetic guidance pin installed, can be detected immediately
From TABLE 4, the results of the detection value bits from
the sensor will be taken into consideration by ordering the
degree of actualization of the Fork AGV Robot. If the detection
value is perfect (6 sensors detect), it will give a command to
handle from 0o-45o following which bit that had value ”1” was
detected. If more centralized detection occurs, the command
angle givento the motor will also be smaller. On the other
hand,the edge will be given to the edge where the magnet is
detected.
Likewise, the magnet sensor does not detect 6 magnets
directly. So the fewer detected, the greater the handling angle
given to move the robot to follow the existing magnetic
guidelines. The degree given to edge detection ranges from
50o-90o following the edge where the magnet is detected.
C. Pallet Detection Test
Pallet Detection Test using a camera to detect and measure
the position of the robot’s centerline to the pallet reference that
placed in the middle of the pallet. This pallet detection will
test the accuracy of taking or putting a pallet by detecting the
reference point object from the pallet with pixel values. After
the camera gets its pixel values, the camera will also see the
magnification value obtained from the object reference and
finally actualize the value to reference the motor movement
for the heading repositioning of the robot by sending a
classification of pixel values with the PWM motor which will
be actualized via serial communication on the serial
communication on Arduino.
• Location
The location of the hardware placement used for SLAMis
placed at the top of the lifter system of the AGV robot shown
in the following figure:
Fig. 9. Location of camera installation
• Design
The final display design is processed using OpenCV from
the camera that used to detect reference object shown in the
following figure:
Fig. 10. Pallet Detection Reference Detection Interface On
Fig. 10 results obtained measurements on the carte- sian XY
axis of the detection of the reference objectand also the
magnification transformation of the reference object, and
also the magnification transformation of the reference
object, this could be a benchmark for the robot to move
towards the object later (Forward, backward and turning).
• Result
Explained in the visualization above, this detection Xaxis
will look at the position angle of the robot towards the
pallet. The Y axis is to see the height position ofthe pallet
and the reference transformation is to estimate the pallet
distance against the AGV robot Fork. The zoning detection
numbering from 1-9 is made the resultof a combination of
the X, Y axis and the referencetransformation is an additional
feature to support the results of the detection position by
providing a speed reference that must be set on the AGV
robot. The values of the X, Y and transformation axes are
obtained fromthe retrieval and processing of pixel values of
the images displayed at 640 x 480 resolution. The following
is a complete description of the zoning shown in the following
table:
TABLE 5. CLASSIFICATION ON REPRESENTA- TION
DATA
The data above is then encoded and sent serial commu-
nication to Arduino to provide a movement of the robot’s
actulisation process. The format for encrypting the data itself
is: (X-axis, Y-axis, Transformation Value)
D. GUI Navigation
The navigation GUI is made up of AGV robotic access
command buttons, navigation screens and also the status of the
operating AGV Fork robot. The GUI is the result of the
development of the main navigation system which completed
and for supporting the automation process of the Fork AGV
Robot.
• Design
Design from the GUI for navigate the entire Fork AGV
Robot that is made as shown in the following figure:
Fig. 11. GUI Fork AGV Robot Design
V. COMBINATION OF NAVIGATION
Combination navigation is a concept of combining overall
navigation in the Fork AGV Robot. This process will make the
navigation subdivision integrated so that it will be ableto
carry out operations simultaneously. The concepts used in
combining sensor of navigation shown in the following figure:
Fig. 12. Design Combination of Navigation
Then, the results obtained from the simulation of the naviga-
tion concept above using MATLAB software to the reference
process for the motion of the Fork AGV Robot shown in the
following figure:
Fig. 13. Result on Navigation Simulation System
The data obtained is the result of a trial running test of
the Fork AGV Robot at a distance that is not too far away,
because stability has not been wholly obtained and must
regularly test to get the optimal result. The test realization is
shown in the following figure:
Fig. 14. Running Test Of Fork AGV Robot
CONCLUSION
The research discusses the process of combining one sensor
with another is done by maintaining the function of eachsensor
that supports the reference motion for the AGV Fork robot.
The fusion sensor managed using the ROS (Robot Operating
System), which is an open-source platform and create a
package system for instantly used by many people.
In mapping and localization using LIDAR found varied
errors with an average error of 0%-5% of the actual distance by
looking at the indoor or outdoor testing environment. Then, on
a magnetic sensor when done on a single sensor, the deflection
that occurs is 16o from the desired angle and 4o deflection on
the use of multi-sensors. Detection of pallet reference position
has been categorized according to the region displayed on the
GUI interface.
The final conclusion results for this experimental tests
obtained have found a fundamental theory in the application of
combining sensors used for Fork AGV Robot. But, the results
obtained are not optimal yet and requires testing directly in
the field even more intensive to achieve stable and maximum
results.
ACKNOWLEDGEMENT
Authors must express our gratitude to Politeknik Elek-
tronika Negeri Surabaya (PENS), especially Vehicle Robotics
Laboratory of Mechatronics Engineering on Mechanical En-
gineering and Energy Department, for arranging the necessary
equipment and providing the laboratory facilities to conduct the
experiment.
REFERENCES
[1] Fu, G.,Zhang, J.,et.al, Precise Localization of Mobile Robots via Odom-
etry and Wireless Sensor Network. International Journal of Advanced
Robotic Systems,2015.
[2] Bui, T.L.,Doan, P.T.,et.al,AGV Trajectory Control Based on Laser Sen-
sor Navigation,IJSE Journal,2013.
[3] Varga, R.,Nedevschi, S.,Robust pallet detection for automated logistics
operations, International Conference on Computer Vision Theory and
Applications,2016.
[4] Feledy, C.,Luttenberger, M.S.,A State of the Art Map of the AGVs
Technology and a Guideline for How and Where to Use It, Lund
University, Swedia,2017,p.20-29.
[5] Pizarro, D.,Mazo, M.,et.al,Localization of Mobile Robots Using Odom-
etry and an External Vision Sensor, Sensors 2010,2010,p.3655-3678.
[6] Wei, P.,Cagle, L.,et.al,LIDAR and Camera Detection Fusion in a Real-
Time Industrial Multi-Sensor Collision Avoidance System,Electronics
2018 book,MDPI,2018.
[7] Renaudin, V.,Afza, M.H.,et.al,New Method for Magnetometer Based
Orientation Estimation, IEEE/ION PLANS 2010, Palm
Springs,CA,2010.
[8] L.Bini, M.Chiaberge,Architecture and Methods for Autonomous In-
door Localization and Mapping, Masters Degree Thesis:Politecnico Di
Torino,2018.
[9] Prayoga, S., Budianto, A.,et al, Sistem Pemetaan Ruangan 2D Menggu-
nakan Lidar, Jurnal Integrasi,Vol.9,2017,p.73-79.
[10] Park, J.,Jung, K.,et.al,Accuracy Improvement of Magnetic Guidance
Sensor using Fuzzy Inference System,SCIS-ISIS 2012,Kobe,Japan,2012.
[11] Kohlbrecher S., Stryk O V, Meyer J, et al. A flexible and scalable SLAM
system with full 3D motion estimation[C], IEEE International Sympo-
sium on Safety, Security, and Rescue Robotics. IEEE, 2011,p.155-160.
[12] Cole, S.B., Magnetic field-based navigation of a mobile robot. Ann
Arbor,Michigan: ProQuest Co,2005.
[13] Ramsden, E., Hall-effect sensors: theory and application,Elsevier-
Newnes,Oxford,1998.
[14] Brauml T. Embedded robotics: mobile robot design and applications with
embedded systems. 2nd ed. Berlin Heidelberg: Springer,2006,p.9899.
[15] Siegwart R, Nourbakhsh I. Introduction to autonomous mobile
robots,MIT Press,2004,p.186.
[16] Chen G.,Cui G.,Accurate Intrinsic and Extrinsic Calibration of RGB-
D Cameras with GP-based Depth Correction,IEEE SENSORS
JOURNAL,2018,p.1558-1748.
[17] Tamara, Mohamad Nasyir., Darmawan, A., Tamami, N. et al.Electronics
System Design for Low Cost AGV Type Forklift, International
Conference on Applied Science and Technology (iCAST),Manado,2018.