sensors

Letter
Improved Pose Estimation of Aruco Tags Using a
Novel 3D Placement Strategy
Petr Oščádal 1 , Dominik Heczko 1 , Aleš Vysocký 1 , Jakub Mlotek 1 , Petr Novák 1 ,
Ivan Virgala 2 , Marek Sukop 3 and Zdenko Bobovský 1, *
1 Department of Robotics, Faculty of Mechanical Engineering, VSB-TU Ostrava, 70833 Ostrava,
Czech Republic; petr.oscadal@vsb.cz (P.O.); dominik.heczko@vsb.cz (D.H.); ales.vysocky@vsb.cz (A.V.);
jakub.mlotek@vsb.cz (J.M.); petr.novak@vsb.cz (P.N.)
2 Department of Mechatronics, Faculty of Mechanical Engineering, Technical University of Košice,
04200 Košice, Slovakia; Ivan.Virgala@tuke.sk
3 Department of Robotics, Faculty of Mechanical Engineering, Technical University of Košice, 04200 Košice,
Slovakia; marek.sukop@tuke.sk
* Correspondence: zdenko.bobovsky@vsb.cz




Received: 14 July 2020; Accepted: 23 August 2020; Published: 26 August 2020


Abstract: This paper extends the topic of monocular pose estimation of an object using Aruco tags
imaged by RGB cameras. The accuracy of the Open CV Camera calibration and Aruco pose estimation
pipelines is tested in detail by performing standardized tests with multiple Intel Realsense D435
Cameras. Analyzing the results led to a way to significantly improve the performance of Aruco tag
localization which involved designing a 3D Aruco board, which is a set of Aruco tags placed at an
angle to each other, and developing a library to combine the pose data from the individual tags for
both higher accuracy and stability.

Keywords: Aruco; pose estimation; 3D grid board; precision

1. Introduction
In robotics, engineers attempt to get the maximum usage out of the available sensors, often
outside the recommended use cases. One popular example would be the use of monocular vision to
get a rough estimate of the 3D world being imaged, which is done by using mathematical models
like the pinhole camera model [1–5]. However, to make use of the pinhole camera model, we have
to experimentally determine the characteristics of the camera—the distortion coefficients (radial and
tangential distortion), as well as the camera matrix, the accuracy of which greatly relates to the accuracy
of the 3D world reconstruction that we get. This paper starts with this calibration process, which
was found out to be improvable using a straightforward approach. Some of these approaches to
calibrate the cameras have been discussed in the literature. In [6], the authors used a single image
to calibrate the camera online while accepting a low calibration accuracy. The Camera Calibration
Toolbox integrated in MATLAB was used and demonstrated in [7]. The methods in [6,7] provide
calibration to be obtained quickly, but with lower accuracy. Another study [8] presented a calibration
method that uses the transformation of the shape of a circle from the world plane to the image
plane to determine the internal parameters of the camera. Based on a comparison of four methods,
this approach proposes good effectiveness but assumes a high-resolution image in order to keep the
properties of the circle. Grids at different depths (3D points) were used to calibrate the stereo-vision
system in [9]. Article [10] dealt with camera calibration and reuses this calibration for different camera
modes. The study in [11] was based on marker detection, where the detected position of the robot
was compared between the position estimates from internal and external cameras. It is also possible

Sensors 2020, 20, 4825; doi:10.3390/s20174825 www.mdpi.com/journal/sensors

Sensors 2020,
Sensors 2020, 20,
20, 4825
x FOR PEER REVIEW 15
2 of 16

to utilize genetic algorithms [12] for the calibration. The use of Aruco tags, made available by the
to utilize
Open CV genetic algorithms
library [13], was used [12]
to for
helpthe calibration.
estimate the poseThe ofuse of Aruco
objects tags,taken
in images made byavailable by the
a single camera
Open CV library [13], was used to help estimate the pose of objects in images
from a single perspective. The OpenCV Aruco module provides the users with a simple interface totaken by a single camera
from
plug ainsingle perspective. The
the characteristics OpenCV
of the tag they Aruco
desire module
to detectprovides
the pose.the users
It takeswith
as ainput
simplea 2Dinterface
image
to plug in the characteristics of the tag they desire to detect the pose. It takes
containing the tag along with the camera parameters to give out an estimate of the pose of the tag inas input a 2D image
containing
a 3D coordinatethe tagspace
alongrelative
with thetocamera parameters
the camera. to give outtoanAruco
An alternative estimate of the pose
provided of the tag
by OpenCV is
in a 3D coordinate space relative to the camera. An alternative to Aruco
Charuco, which embeds Aruco-like tags in a chessboard. Charuco is more accurate than the Aruco provided by OpenCV is
Charuco, which
grid, thanks to embeds Aruco-like
the included tags in and
chessboard a chessboard.
takes overCharuco is more
advantages of accurate
tag-based than the Aruco
calibration grid,
which
thanks
supportsto the included
occluded chessboard
or partially and takes
visible over advantages
calibration of tag-based
patterns. However, therecalibration
are manywhichothersupports
types of
occluded or partially visible calibration patterns. However, there are many other
calibration patterns, such as circle-based patterns, Halcon, Kalibr [14], CALTag [15] and others types of calibration
patterns,
mentioned such as circle-based
in [16]. patterns,
Another method Halcon, Kalibr
mentioned [14],
in [17] usesCALTag
only a [15] and others
modified mentioned
chessboard in [16].
to calibrate
Another
cameras.method mentioned
This system works inby[17] uses only the
recalibrating a modified
camera chessboard
parameters to calibrate
when cameras. to
it is necessary This system
detect the
works by recalibrating the camera parameters when it is necessary to detect
camera position and orientation, which has both advantages and disadvantages if only the camera the camera position and
orientation, which has both advantages and disadvantages if only the camera position changes.
position changes.

2. Calibrating the Camera

2. Calibrating the Camera
In order to localize the Aruco tags precisely, we need to know the calibration matrix of the camera
In order to localize the Aruco tags precisely, we need to know the calibration matrix of the
in use [18]. OpenCV provides a systematic pipeline to do exactly so [19]; however, getting the camera’s
camera in use [18]. OpenCV provides a systematic pipeline to do exactly so [19]; however, getting the
intrinsic characteristics is a process prone to approximation errors, and it generally requires several
camera’s intrinsic characteristics is a process prone to approximation errors, and it generally requires
datapoints to get right. A bad calibration results in an incorrect stereographic projection, which in
several datapoints to get right. A bad calibration results in an incorrect stereographic projection,
this case would lead to bad pose estimates of the tag. Through this paper, a set of Intel RealSense
which in this case would lead to bad pose estimates of the tag. Through this paper, a set of Intel
D435i cameras were used to eliminate errors coming from any particular device. Our attempts to
RealSense D435i cameras were used to eliminate errors coming from any particular device. Our
calibrate the camera using the standard procedure were initially amiss during several attempts to do
attempts to calibrate the camera using the standard procedure were initially amiss during several
so. Constant illumination was maintained throughout the process, and various orientations of the
attempts to do so. Constant illumination was maintained throughout the process, and various
calibration board were imaged to maximize the number of points available for more precise calibration.
orientations of the calibration board were imaged to maximize the number of points available for
The
moreexperimental setup isThe
precise calibration. shown in Figure 1.
experimental setup is shown in Figure 1.

Figure 1.
Figure 1. Calibration
Calibration process.
process.

calibration process
This calibration process yielded
yielded the
the parameters
parameters shown
shown in
in Equations
Equations (1)
(1)and
and(2).
(2).
′
𝑐𝑎𝑚𝑒𝑟𝑎
camera0 s𝑠distortion
𝑑𝑖𝑠𝑡𝑜𝑟𝑡𝑖𝑜𝑛 𝐶𝑜𝑒𝑓𝑓𝑖𝑐𝑖𝑒𝑛𝑡𝑠
f f icients = =2.1479
(2.147946.0661
46.06610.0708
0.0708 −0.1847 0.8319) (1)
 
Coe −0.1847 0.8319 (1)
 5577.3914 0 362.06

 5577.3914 0 362.06 
𝑐𝑎𝑚𝑒𝑟𝑎0′ 𝑠 𝑖𝑛𝑡𝑟𝑖𝑛𝑠𝑖𝑐 𝑝𝑎𝑟𝑎𝑚𝑒𝑡𝑒𝑟𝑠 = ( 0 5884.5847 649.0561  ) (2)
camera s intrinsic parameters =  0 0 5884.5847
0 649.05611 (2)
 
0 0 1
The intrinsic parameters of the camera and its distortion coefficients describe the nature of the
projection of the imaged objects from the world to the image plane [19]. To test the parameters, we
Sensors 2020, 20, 4825 3 of 16

The intrinsic parameters of the camera and its distortion coefficients describe the nature of the
Sensors 2020, 20, x FOR PEER REVIEW 3 of 15
projection of the imaged objects from the world to the image plane [19]. To test the parameters, we used
it to detect
used the pose
it to detect the of an of
pose Aruco tag while
an Aruco rotating
tag while the tagthe
rotating slightly along its
tag slightly X-axis
along its (Roll).
X-axis The camera
(Roll). The
resolution for this test was set to 1280 × 720, and the frames were recorded at 30 frames
camera resolution for this test was set to 1280 × 720, and the frames were recorded at 30 frames per second
per
(FPS). Figure 2 shows the results of this test. Notice the jumps in the detected angle.
second (FPS). Figure 2 shows the results of this test. Notice the jumps in the detected angle.

Figure
Figure2.2.Roll
Rollangle
angleduring
duringmanual
manualmeasurement.
measurement.

The
Theresults
resultsindicate
indicatethat
thatthe
thecalibration
calibrationmatrix
matrixwaswasnotnotaccurate;
accurate;however,
however,the the experiment
experimentwas was
repeated
repeatedseveral
severaltimes,
times,and
andthe
thereprojection
reprojectionloss
losswith
withthese
theseparameters
parameterswaswascalculated
calculatedtotobebeclose
closeto
to
−14
22××1010 . While
−14 calibrating,
. While thethe
calibrating, OpenCV
OpenCV guideline
guidelinefor
forcalibration
calibrationwas
wasstrictly
strictlyfollowed
followedwith
with3030images
images
being
beinginput
inputinto
intothe
thecalibration
calibrationfunction
functionwith
withdifferent
differentpositions
positionsand
andorientations
orientationsin inthe
thefield
fieldof
ofview
view
(FOV) of the camera.
(FOV) of the camera.

3.3.Benchmark
Benchmarkfor
forPose
PoseEstimates
Estimates
To know how accurate
To accurate the
theAruco
ArucoTag Tagdetection
detectionwaswas with
withthethe
currently
currentlyacquired parameters,
acquired parameters,a test
a
waswas
test devised which
devised involved
which moving
involved the Aruco
moving tags intags
the Aruco the camera FOV along
in the camera FOV precisely defined positions
along precisely defined
in the 3Din
positions space
the 3D (Figure
space3a(Figure
shows two such positions).
3a shows A workplace
two such positions). was created
A workplace forcreated
was this evaluation
for this
evaluation which allowed us to perform automated and precise measurements using a robot.
which allowed us to perform automated and precise measurements using a UR3 collaborative UR3
A flat grid with
collaborative robot.4 Aruco tags with
A flat grid was fixed
4 Arucoat the
tagsTCP
was(tool
fixedcentre point)
at the TCP of the
(tool robot,
centre andofthe
point) thecamera
robot,
usedthe
and in the benchmark
camera used in(Intel RealSense D435i)
the benchmark was fixed D435i)
(Intel RealSense in the robot coordinate
was fixed in thesystem. RealSense
robot coordinate
D435i cameras
system. RealSense come withcameras
D435i an inbuilt
comeinertial
with measurement unit measurement
an inbuilt inertial (IMU) which we used
unit (IMU)to ensure
which that
we
the camera
used to ensurewas pointing
that in the
the camera right
was direction
pointing parallel
in the to the TCP
right direction Z-axis.toTo
parallel theensure that the
TCP Z-axis. Todistance
ensure
between
that the the camera
distance and the the
between tag remained
camera and constant while
the tag performing
remained the benchmark,
constant we used data
while performing the
from the depth
benchmark, we stream
used datato align
fromthem before stream
the depth the benchmark. The camera
to align them beforewasthe configured
benchmark.toThe grabcamera
frames
at a configured
was resolution of grab ×frames
to 1280 720 and framerate of 30
at a resolution FPS;
1280 theand
× 720 dataa from this camera
framerate were
of 30 FPS; thepassed onto
data from
a Jetson Nano, which performed the pose estimation and passed the
this camera were passed onto a Jetson Nano, which performed the pose estimation and passed thedata to the server (Figure 3b).
The server
data to the compared
server (Figurethese3b).
dataThe
with the expected
server compared data to detect
these data the
withamount of errordata
the expected in thetodetection
detect theof
the Aruco tag at that particular pose in the frame. The library to control a robot
amount of error in the detection of the Aruco tag at that particular pose in the frame. The library to from Universal Robots
named aURX
control robot[20] wasUniversal
from used for communication
Robots named URX with the
[20]real
wasworld
used robot.
for communication with the real
worldTable
robot.1 lists information about the setup of the Aruco grid.

Table 1. Parameters of the 2D grid board.

Number of Tags on the Board 2×2

Tag edge length 70 mm
Spacing between tags 10 mm
Tag type 6 × 6 × 1000

(a) (b)
Figure 3. Workplace: (a) two measurements at the edge of the measurement range; (b) general block
diagram.
 was configured to grab frames at a resolution of 1280 × 720 and a framerate of 30 FPS; the data from
this camera were passed onto a Jetson Nano, which performed the pose estimation and passed the
data to the server (Figure 3b). The server compared these data with the expected data to detect the
amount of error in the detection of the Aruco tag at that particular pose in the frame. The library to
control
Sensors a robot
2020, 20, 4825 from Universal Robots named URX [20] was used for communication with 4the real
of 16
world robot.

(a) (b)
Figure
Figure 3. 3. Workplace:(a)
Workplace: (a)two
two measurements
measurements at
at the
the edge
edgeofofthe
themeasurement
measurementrange; (b)(b)
range; general block
general
diagram.
block diagram.

All of the coordinate frames (of the individual tags in the camera coordinate frame) were
transformed relative to the robot coordinate system. In order to cover the entire FOV of the camera
in the experiment, a grid of 30 × 30 positions (for the Aruco tags) was defined with constant spacing
between the positions for the detected tag pose in the frame. The benchmark also involved rotating the
Aruco board along its X, Y, and Z axes (Roll, Pitch, Yaw) within the range from −30◦ to 30◦ with a step
size of 15◦ for each axis to check the impact of angle on pose estimation. Algorithm 1 describes the
measurement process.

Algorithm 1. The Measurement Process.

for i = 0 to m // m[1:30] represents the number horizontal positions for the effector
set_y_position_of_end_effector (i)
for j = 0 to n // n[1:30] represents the number of measurements in a row
set_x_position_of_end_effector (j)
for k = 0 to o // o[1:10] represents the number of frames in the measured position
Grab image
Process image
Save data after processing
end for
end for
end for

The tagged effector was illuminated to ensure constant lighting of the grid board (Figure 4).
The robot gradually moved the tag through the positions according to the defined grid.
The measurements were made only when the manipulator stabilized in the desired position for which
we added a small delay between reaching the position and the pose being detected. Subsequently,
the pose estimation was run ten times for each pose of the tag, and the data were stored to disk.
All measurement data were then stored in files for further processing. One measurement took
approximately 30 min. All measurements were repeated ten times to collect more data so we could
avoid systematic errors.
The range of the effector of the robot limited the range of experimental positions, as it had to be
ensured that all the positions were imaged consistently for all the orientations. The orientations that
we could perform the measurements for are mentioned in Table 2.
 Save data after processing
end for
end for
end for
Sensors 2020, 20, 4825 5 of 16
The tagged effector was illuminated to ensure constant lighting of the grid board (Figure 4). The
robot gradually moved the tag through the positions according to the defined grid. The
Table 2. Measurement configurations.
measurements were made only when the manipulator stabilized in the desired position for which we
added a small delay between
Axis/reaching
◦ −30◦ the −15
position
◦ and
0◦ the pose
15◦ being30◦detected. Subsequently, the
pose estimation was run ten timesYES
Roll
for eachYESpose of the tag,YES
and theNOdata were stored to disk. All
measurement data were Pitch
then stored
NO in files
NO for further
YES processing.
YES NO One measurement took
approximately 30 min. All Yaw
measurements
NO were
NO repeated ten YEStimes to NO
collect more data so we could
avoid systematic errors.

Sensors 2020, 20, x FOR PEER REVIEW 5 of 15

The range of the effector of the robot limited the range of experimental positions, as it had to be
ensured that all the positions were imaged consistently for all the orientations. The orientations that
we could perform the measurements for are mentioned in Table 2.

Table 2. Measurement configurations.

Axis/° −30° −15° 0° 15° 30°

Roll (a) YES YES (b) YES NO
Pitch NO NO YES YES NO
Yaw Figure
Figure 4. Measuring workplace
4.NO
Measuring workplace
NO (a)(a) back
back view;
view; (b)
(b) front
front view.
view. YES NO
4. Benchmark Results for the 2D Aruco Board
4. Benchmark Results for the 2D Aruco Board
During the experiments, one of the more prevalent errors came in the form of unexpected jumps
During the experiments, one of the more prevalent errors came in the form of unexpected jumps
in the approximation of pose in which the detected axes would change by several degrees with a minor
in the approximation of pose in which the detected axes would change by several degrees with a
change in the actual position of the tag. This problem is recorded and shown in Figure 5, in which
minor change in the actual position of the tag. This problem is recorded and shown in Figure 5, in
a,b,c,d,e are measurements with a minuscule change in the position of the tag, but the detected pose of
which a,b,c,d,e are measurements with a minuscule change in the position of the tag, but the detected
the blue Z-axis (Yaw) shifts noticeably.
pose of the blue Z-axis (Yaw) shifts noticeably.

(b) (c)

(a)

(d) (e)
Figure
Figure 5. Images
5. Images from
from measurements:
measurements: (a)(a) full-frame;
full-frame; (b–e)
(b–e) observation
observation in four
in four frames
frames consecutively.
consecutively.

TheThe experimental
experimental results
results areare visualized
visualized in the
in the form
form of aof30a ×
3030× matrix
30 matrix of deviations,
of deviations, where
where thethe
deviations
deviations areare represented
represented withwith colors
colors determined
determined by the
by the colour
colour bar bar in Figure
in Figure 6. 6.

Figure 6. The colour bar represents the range of deviations and the colour to represent it throughout
 (d) (e)
Figure 5. Images from measurements: (a) full-frame; (b–e) observation in four frames consecutively.

The experimental results are visualized in the form of a 30 × 30 matrix of deviations, where the
deviations are4825
Sensors 2020, 20, represented with colors determined by the colour bar in Figure 6. 6 of 16

Figure 6. The colour bar represents the range of deviations and the colour to represent it throughout
Figure 6. The colour bar represents the range of deviations and the colour to represent it throughout
this paper. For angle estimates, the range of deviation is ± 0.278 rad whereas, for the position, the range
this paper. For angle estimates, the range of deviation is ± 0.278 rad whereas, for the position, the
of deviation is ±0.002 m.
range of deviation is ± 0.002 m.
Each little square in Figure 7 represents a relative position of the Aruco tag in the frame with its
Each little
neighboring square
pixels; for in Figure 7the
example, represents
deviationafor
relative position of the
the measurement Aruco 5a
in Figure tagwould
in the be
frame with its
represented
neighboring
by the highlightedpixels; forsquare
little example, the deviation
approximately in the for theofmeasurement
center in 7a.
the grid in Figure Figure 5a wouldmap
The deviation be
represented by the highlighted little square approximately in the center of the grid
remained almost consistent across several iterations of the same measurement (for the same angle and in Figure 7a. The
Sensors 2020, 20, x FOR PEER REVIEW 6 of 15
deviation
distance); map remained
an example of almost
this canconsistent across
be observed several7,iterations
in Figure where a,of the same
b and c are measurement (for
different iterations the
of
same angle and
error measurements distance); an example of this can be observed in Figure 7, where a, b and c are
different iterations ofinerror Rollmeasurements
value of the pose estimate,
in Roll valuewhile
of thethe
posenormal of the
estimate, tag was
while parallelofwith
the normal the
the camera Z-axis.
tag was parallel with the camera Z-axis.

(a) (b) (c)

Figure
Figure 7.
7. Deviation
Deviation map
map observed
observed for
for Roll
Roll estimate
estimate of of the
the 2D
2D Aruco
Aruco grid
grid board:
board: (a)
(a) measurement
measurement 1,
1,
observe
observe the
the highlighted
highlighted square,
square,this
thissquare
squareis isforforthethe
frame in in
frame Figure 5a; 5a;
Figure (b) (b)
measurement 5; (c)
measurement 5;
measurement
(c) measurement10. 10.

The data from the benchmark with a systematic set of variations of the pose of the tag within the
frame are shown
shown inin Table 3; it represents data from 10 iterations
iterations of
of the
the same
same measurement
measurement process.
process.
Each row is for a particular orientation of the tag, while the columns depict the
the benchmark
benchmark results.
results.

Table 3. Results from the benchmark for the 2D Aruco grid board, deviations along each detected
parameter of the tags for various positions of the grid board in the camera field of view (FOV).

Roll Pitch Yaw X Y Z

+15°
Roll

−15°
Roll

−30°
Roll

+15°
 Figure
Figure
Figure
Figure
observe 7.7.
Figure
Figure7.
observe
observe 7.7. Deviation
Deviation
7. Deviation
Deviation
Deviation
Deviation
the
the the map
map
highlighted
highlighted map
map
mapmap
highlighted observed
observed
observed
observed
square,
square, for
observed
observed for
this
square,
this for
for
for Roll
for
Roll
this Roll
Roll
Roll
square
square Roll estimate
estimate
estimate
estimate
estimate
estimate
square isis for
for
isis ofof
for
the of
of
the of
the
theof
the
the the
the 2D
the
2D
frame
frame 2D2D
2D2D
frame Aruco
Aruco
Aruco
Aruco
Aruco
Aruco
in
in grid
grid
grid
grid
grid
grid
Figure
Figure
in Figure board:
board:
board:
board:
board:
5a;
5a; 5a;(b)
(b) (a)
board:(a)
(a) (a)
(a) measurement
measurement
measurement
(a) measurement
measurement
measurement
measurement
(b)
measurement
measurement 5;5;5;1,1, 1,1,
1,1,
1,(c)
(c)
(c)
observe
Figure
Figure
observe
observe 7.7.the
observe
observe
Figure
Figure
Figure
Figure
observe
observe 7.
observe
observe
Figure
Figure
Figure the
the the
7.
the
the
7.7.
highlighted
the highlighted
Deviation
Deviation
7. 7. highlighted
Deviation
Deviation
Deviation
Deviation
highlighted
highlighted
the
the map
map
map
highlighted
highlighted
highlighted
highlighted
Deviation
Deviation
square,
map
map
map
mapsquare,
square,
map
this
square,
square,
observed
observed
observed
observed
observed
observed
square,
square, this
this
square,
square,
observed
square
this
this
for
for
for
forfor
this
this
observed
square
square
square
square
this
this
for
Roll
Roll
Roll
Roll
forRoll
square
square
for square
square
Roll
Roll
is
is
is isfor
is
estimate
estimate
estimate
estimate
Roll estimate
estimate
for
for
is
is
the
for
for
isfor
for
estimate
estimate of
the
the
for
for
the
of
of
the
the frame
the
of
the
of the
the
of
the
the
of
frame
the
frame
frame
of
frame
the
2D 2D
2D
the
frame
frame
the 2Din
2Din
in
frame
frame
the 2D2D
Figure
in
in
Aruco
Aruco
Aruco
2D Aruco
in Figure
Aruco
Figure
Figure
inin
in
Aruco
Aruco
5a;
Figure
grid
grid
grid
grid
Aruco grid
Figure
Figure 5a;
5a;
Figure
Figure
grid
grid
(b)
5a;
5a;
board:
board:
board:
gridboard:
(b)
(b)
5a;
5a;
5a; (b)
5a;
measurement
(b)
(b)
board:
(b)
board:
board:
measurement
measurement
(a)
(a)
(a)
(a)
board: (a) measurement
measurement
measurement
(a) 5;
measurement
measurement
measurement
measurement
measurement
(b) measurement
measurement
(b) measurement
measurement
(a)
(a) 5;5;
measurement
measurement 1,5;
(c)
5;
(c)
(c)
5;5;5; 1,(c)
1,(c)
1,(c)
(c)
(c)
5; 1,
(c)
1,1,
Figure
Figure7.7.
Figure
observe
observe
observe
observe
observe
Figure
Figure7.
Figure
Figure
Figure7.
observe
the
7.
Deviation
7.
the
the
7.7. Deviation
Deviation
Deviation
the
the
7.the map
map
highlighted
highlighted
Deviation
Deviation
Deviation
Deviation
Deviation map
map
highlighted
highlighted
highlighted
highlighted
map
map
map
map
observed
map observed
observed
observed
square,
square,
square,
square,
square,
square,
observed
observed
for
forfor
for
this
this
this
this
this
observed
observed for
observed for
Roll
Roll
this
for
for Roll
Roll
square
square
Roll
forRoll
Roll
Roll
estimate
square
square
square
square
Rollestimate
estimate
estimate
isisis
estimate is
for
is
for
estimate is
for
estimate
estimate
estimatefor
for of
of
the
for
the of
of
the
of of
of
the
the
the
the
ofofthe
the
frame
the 2D
2D
frame
frame
the the
the 2D
the 2D
2D
frame
frame
2D
2D
the
Aruco
Aruco
frame
in
2D in
2D Aruco
Aruco
inin
inin
Figure
Aruco
Aruco
Aruco
Aruco
Aruco
grid
grid
grid
grid
Figure
Figure
Figure
Figure
Figure
grid
grid
board:
5a;board:
board:
board:
5a;
5a;
5a;
5a;
grid
grid
grid (b)
5a;
(b)
board:(b)
board:
(a)
(a)
(b)
(b)
(b)
board:
board:
board: (a)
(a) measurement
measurement
measurement
measurement
measurement
measurement
measurement
measurement
measurement
measurement
(a)
(a)measurement
(a)
(a) 5;
5;
measurement
(a)measurement
measurement
measurement5; 1,1,
5;
(c)
5;
(c) 1,
5;
(c)
1,
1,1,
(c)
(c)
(c)
1,
1,1,
Figure
observe 7.
measurement
measurement
observe
observe
observe Deviation
measurement
measurement
measurement
measurement
observethe
observe the
the the
the 10.
10.
10.10. map
10.
10.
highlighted
highlighted
highlighted
highlighted
the highlighted
highlighted observed
square,
square,
square,
square, for
this
this
square,
square, Roll
this
this square
square
this
this estimate
square
square is
square
square isis isfor
for
for
is is of
for
for the
the
the
for the
the
the 2D
frame
frame
the Aruco
frame
frame in
frame
frame in
in in
in grid
Figure
Figure
Figure
Figure
in board:
5a;
5a;
Figure
Figure 5a;
5a;
5a;(b)
(b)
5a; (a)
(b)
(b) measurement
measurement
measurement
measurement
measurement
(b)
(b) measurement
measurement 5;
5; 5;5;1,
5;(c)
(c)
(c)5; (c)
(c)(c)
measurement
measurement
measurement
measurement
measurement
measurement
observe
observe
observe
observe the the
thethe 10.
10.10.
10. 10.
10.
highlighted
highlighted
highlighted
highlighted square,
square,
square,
square, this
thisthis
this square
square square
square isfor
isisisis isfor
for for
the thethe
frameframe
frame
in inin Figure
Figure
Figure 5a;5a;
5a;
(b) (b)
(b) measurement
measurement
measurement 5;
5;5;5;5; 5;
(c) (c)
(c)
observe
observe
observe
observe the
measurement
measurement
measurement
observe
observe the
the
measurement
measurement
measurement
observe
observe thethe
the
measurement
measurement
measurement
measurement
measurement
measurement
highlighted
the
thehighlighted
10.
10. 10.
10.
10. 10.
highlighted
highlighted
highlighted
highlighted
highlighted
highlighted
10.
10.
10.
10.10.10.
square,
square,
square, this
square,
square,
square, this
this
square,
square, this
square
this
this
this
thissquare
square
square
squaresquare
square
square for
isisisis isfor
for
isfor
for
the
for
for the
the
the
the
the
for
the
frame
thethe
frame
frame
frame
frame
frame
frame
frame in
frame
ininin
in
in
inin
in Figure
Figure
Figure
Figure
Figure5a;
Figure
Figure
FigureFigure
5a;
5a;
5a;
5a; (b)
5a;(b)
5a;
5a; 5a;
(b)
(b)
(b)
(b)
(b)
(b)(b)measurement
measurement
measurement
measurement
measurement
measurement
measurement
measurement
measurement
(c)
5;5;5;5;
(c)
5;
(c)
(c)
(c)
(c)
(c)
5; (c)
(c)
measurement
measurement
measurement
measurement
measurement
measurement 10.
10. 10.
10.
10. 10.
measurement
Themeasurement
Themeasurement
measurement
Themeasurement
measurement
The data
The
data
data
The data
data
data from
from
from from10.
10.10.
from10.
10.
the
the
from 10.
the benchmark
the
benchmark
benchmark
benchmark
the
the benchmark
benchmark with
with
with
withaaasystematic
with
with asystematic
asystematic
asystematic
systematic
systematic setset
setset of
set of
set
of variations
variations
of variations
variations
ofof variations
variations ofof
of the
the
of
the
ofofthe pose
the
pose
pose
the pose
pose of
of
pose of
ofthe
the
of
the
of tag
the
thetag
tag
the within
tag
tagwithin
within
tag within
within
within the
the
the the
the
the
The
TheThe
The data
datadata
data from
from from
from the
the the
the benchmark
benchmark
benchmark
benchmark with
with
with
withaasystematic
asystematic
aasystematic set
setsetof
of variations
variations
of variations of
of ofthe
the pose
pose
the pose of
of the
the
of tag
tag
the within
within
tag within the
thethe
frame
frame The
frameThe
The
frame
frame
frameare
The
The
TheThe
The
The
The
The data
The
are
are
The
The data
data
are are
shown
are
data
data
data
data
data
data
data
data
shown
shown
shown
data
data
from
from
shown
from
data from
shown
from
fromfrom
from
from
from
from
from
in
fromin
from
the
the
in the
Table
inin
the
the
the
the
the
the
the
the
Table
the
theTable
benchmark
benchmark
benchmark
benchmark
benchmark
benchmark
benchmark
benchmark
Table
Table
in Table3;
3; 3;
benchmark
benchmark
benchmark
the it
3;
benchmarkit
3;
benchmark
benchmark it
3; with
represents
ititit aaaaaaasystematic
with
with
with
with
with
with
with
represents
represents
represents
represents
with
represents
with
with
withwith
with data
a
aasystematic
data
a
systematic
systematic
systematic
systematic
asystematic
systematic
data systematic
systematic
data
data
data
systematic
systematica from
from
from from
from
systematicfrom
systematic
systematic
set
set
set
10
10 10
set
set
set
set
set
set
set
set
10 of
set
of of
of
of variations
of
of
of variations
variations
variations
variations
variations
of variations
variations
variations
iterations
iterations
10
set
10
setof
of of
set iterations
iterations
iterations
of
iterations of
of
of
variations
variations
variations
ofof of
variations
variations ofof
of
of
the
the
of
the
of
of
variations of
the
of
of
the
of
of
of
the
the
thethe
the
of
the
same
same
of
thethe
the
the
of of
pose
the
the
the
same
same
the
same
pose
pose
pose
pose
pose
pose
same pose
pose ofof
of of
of
of the
of
of
the
of
the the
the
the
measurement
pose
pose
pose
the
the pose
tag
the
the
tag
the
tag
measurement
measurement
measurement
measurement
pose of the
measurement
of
poseof
ofthe
of the
the
of
tag
tag
tag
tag
tag
tag
tag
tag
the
the
within
tag
tag within
within
within
within
within
within
within
within
process.
within
within
tag
tag
the
the
the
the
the
process.
the
the
the
the
process.
process.
process.
within the
process.
within the
the
the
within
within thethe
frame
frame
frame
frame
frame
frame
The
The
frame are
are
The
Sensors
frame
frame
frame
frameThe are
are
The
TheThe
are shown
shown
are
are
data
data
are
are
The data
2020,
are shown
shown
data
data
shown
are
data shown
data
shown
from
from
shown
shown
data from
20,
shown
shown
fromfrom in
from
from in
from
4825
inin
the
ininTable
Table
inin
the
the
in
theinTable
Table
in Table
the
thethe 3;
3; 3;
benchmark
benchmark
Table
Table
Table
the Table3;
benchmark
Table 3; it
3;
benchmark
Table 3;
benchmark it
benchmark
benchmark
benchmark 3;
3;
itit
3;represents
represents
itit represents
represents
it represents
with
withwith
represents
with
with
with
itorientation
3;
it it a
itrepresents a
represents
represents
represents
represents
with
with aof adatadata
a
systematic
data data
data
a datafrom
from
from
from
systematic
systematic
a systematic
data
systematic
systematic
data
data
asystematic
data datafromfrom
from
from 10
10
from
set
from10
from 10
10
set iterations
iterations
set
10
set
set
10
set
10 of
10 iterations
iterations
10
set
of of
of
of of of
of
iterations of
of
variations
variations
variations
variations
iterations
iterations
10
ofiterations of
iterations
variations
iterations of
of the
the
variations
iterations
variations of
of
ofthe
the
of
of
theof
of same
same
of
theof
the
the
the
of
of
the same
same
the
the
the
same
the same
the
the
same
same measurement
measurement
pose
pose
pose
same
same
pose
same measurement
measurement
pose
posepose
ofmeasurement
of of
of
of of
the
the the
the
measurement
the
the
measurement tag
tag
measurement tag
tag
tag
tag
measurement
measurement
of the
measurementtag process.
process.
process.
process.
process.
within
within
within
within
within
within the
process. the
process.
the
the
the
7
process.
within the
of
process.
process.
the
process. 16
frame
Each
Each
Each
Each
frame
frame The
The
Each
row
frame
Each
frame
frame
frame The
are
row
row The
row
are
are data
row
is
are
row
are
are is
is data
data
shown data
for
for
isis
shown
shown
shown
are from
for
is
shownfor
shownfrom
a
for
shownfor
a from
a from
in the the
particular
a
in a
in the
Table
particular
a the
particular
ininin benchmark
benchmark
Table
particular
Table
Table
Table
in benchmark
3;
benchmark
particular
particular
Table
Table 3;
3; it represents
orientation
orientation
orientation
3; it
3; orientation
3;
itit
3;
itit with
with
with
represents
represents
it with
aof
of
represents
orientation
represents aof
represents
represents data
the
dataof
data asystematic
asystematic
systematic
thethe
of systematic
the
data systematic
thefrom
tag,
tag,
tag,
data
the
data
data tag,
tag, 10
set
while
from
from
from set
while
while set
set
while
from
tag, 10
from
from 10
1010set
the
10
while
10 of
ofofvariations
ofvariations
variations
iterations
of
while the
the variations
columns
the of
variations
columns
the
columns
columns
iterations
the columns
iterations
iterations
iterations
iterations
iterations of
of
of the
columnsof of
of of
depict
of the
ofthe
the
same
the
depict
depict
depict
the
the
the
the
of of
the thepose
pose
depict
the
same
depict
same
same
same
thesame pose
pose
pose of
ofof
ofthe
measurement
the
the of the
benchmark
the
the the
thethe
tag
benchmark
the
benchmark
benchmark
measurement
benchmark
measurement
measurement
measurement
same tag
benchmark
measurement
measurement tag
tagtagwithin
within
within
within
process.
within
results.the
results.
results.
results.the
thethe
results.
process.
results.
process.
process.
process. the
process.
process.
Each
Each
Each
Each
frame
frame
frame
frame
Eachrow
row
Each
frame row
row
frame
Each are row
row
are
are
are
row is
is
areis
are is
shown
shown for
for
is
shown
shown
is for
for
is
shown a
shown
for
fora
fora aparticular
particular
ina
in particular
particular
a
inin particular
in
in Table
TableTable
particular
Table
Table
Table 3;
3; orientation
orientation
3;3;
it orientation
orientation
3;
it3;orientation
it
it itit of
of of
represents
represents
orientation
represents
represents
represents
represents the
of
datathe
of the
of
data the tag,
tag,
the
data
data
the
data
data tag,
tag,
tag,
from while
while
tag, while
while
from
from
from
from
from 10
10 10
10 the
the
while
10
10
while the columns
columns
the columns
columns
the columns
iterations
iterations
the
iterationscolumns
iterations
iterations
iterations of
of of
of
of depict
depict
of
the
the depict
depict
the
the
the the
depict
same the
the
depictthe
same
same
same
same
same benchmark
benchmark
the
the benchmark
benchmark
the benchmark
measurement
measurement
benchmark
measurement
measurement
measurement
measurement results.
results.
results.
results.
results.
process.
process.
results.
process.
process.
process.
process.
Each
Each
frame
Each
frame
Each
Each
Each
Each
Each
Each
row
Each
frame
frame
frameframe
row
Each
Each
Each
Each
Each
Each
Each
row
are
row
are
row
row
rowrow
row
rowrow
row
row
are
are
row
row
row
is
is
are
roware
isis
is
isis for
is
shown
isfor
is
is
is is
is
for
shown
shown
for
shown isfor
for
for
for is
is
for
shownaafor
shown
for
for
for
for
for a for
for
a a
in
a aaparticular
in
a
in
a
particular
aaaaparticular
aparticular
particular
aaparticular
in particular
Table
in
inTable
TableTable
Table
Table
particular
particular
particular
particular
particular
aparticular
particular
particular
particular
particular
particular
3;3; 3; orientation
orientation
orientation
itorientation
3;3;orientation
itorientation
it
3;represents
itorientation
orientation
orientation
orientation
orientation
orientation
orientation
orientation
orientation
orientation
orientation
orientation
of
itrepresents
itrepresentsof
of
of
of
of
represents
represents
represents of of
of
of
the
data
the
of
of
the
of
the
of
the
of
the
of
data
the
the
of
of
the
the
data
data tag,
the
data
data
tag,
the
thetag,
the
the
the
the the
tag,
tag,
tag,
tag,
tag,
from
tag,
from
from from
from
tag,
tag,
tag,
tag,
tag,
tag,
tag,
tag,
while
while
while
from
while
10 10
1010
while
while
while
while
tag,while
while
the
while
10
while 10
the
the
the
while
while
while
while
while
while the
the
the
the
the
iterations
the
the
the
the
the
the
the
columns
columns
columns
columns
iterations
iterations
iterations of
columns
iterations
columns
iterations of of
of
columns
columns
columns
columns
the
the columns
columns
columns
columns
columns
columns
columns
columns
the
of the
of depict
depict
depict
the
the the
depict
the depict
depict
depict
depict
the
depict
same
depictsame
samesame
same
same
the
the
depict
depict
depict
depict
depict
depict
depict
depict the
the
the
the
the
the
the
thethe
the
the
the
benchmark
benchmark
benchmark
benchmark
measurement
the benchmark
measurement
measurement
measurement
measurement
benchmark
measurement
benchmark
benchmark
benchmark
benchmark
the
the benchmark
benchmark
benchmark
benchmark
benchmark
benchmark
benchmark
benchmark
results.
results.
results.
results.
process.
results.
process.
process.
process.
process.
results.
process.
results.
results.
results.
results.
results.
results.
results.
results.
results.
results.
results.
results.
Each
Each
EachEach
Each row
Each
rowrow
row
Table
Table
Table isis
row
row3.is
Table
Table
Table
Table 3.for
isis
3.for
3. for
for
is for
Results aResults
afor
Results
3.Results
Results
3.3. aparticular
Results aparticular
aparticular
Results particular
aparticularparticular
from
from
from from
from
from the
the
the
from theorientation
the
the orientation
orientation
orientation
orientation
orientation
benchmark
benchmark
benchmark
the benchmark
benchmark
benchmark
benchmark
of
of
forof
for
for of
the
for the
of
forthe
of the
for
the
the
forthetag,
the
the
tag,
the2D
thetag,
tag,
tag,
2D
the
2D2D
the
while
tag,
2D
2D while
while
while
while
Aruco
Aruco
2D Aruco
Aruco
Aruco
Arucothe
while
the
Aruco the
grid
grid
grid columns
the
thethecolumns
columns
columns
columns
columns
board,
grid
board,
board,
board,
grid
grid
grid
depict
depict
depict
depict
depict
deviations
deviations the
depict
the
deviations
deviations
board,
board,
board, deviations
deviations thebenchmark
the
thethebenchmark
along
along
along
deviations benchmark
benchmark
benchmark
benchmark
along
along each
each
each
along
along each
each results.
results.
results.
results.
detected
detected results.
results.
detected
detected
each
eachdetected
detected
detected
Table
Table 3.3.3.3.
Table
Table Results
Results
3.Results from
fromfrom the
thethe benchmark
benchmark
benchmark for
forforthe
thethe2D2D2D Aruco
Aruco
Aruco grid
grid board,
board,
grid board, deviations
deviations
deviations along
along
along each
eacheachdetected
detected
detected
Table
Table
Table
Table
Table
Table
3.
parameter
parameter
parameter
parameter
parameter
parameter 3.Results
3.
3.3.Results
Table
Table of
of
Results
Results
Results
3.Results
Results
3. Results
Results
of the
the
of
the
of
of
from
from
from
from
from
from
from
from
from
tags
the
tags
tags
the
the the
tags
for
tags
tags
the
the
for
for
the
the
the
the
the
benchmark
benchmark
benchmark
benchmark
benchmark
the benchmark
benchmark
benchmark
benchmark
various
for
various
various
various
for
for
for
for
for
for
for
for
for
positions
positions
positions
various
various
the
for
the
for
the
positions
positionsof
positions of
of
the
the
the
the
the 2D
the
ofof
2D
2D
the 2D
2D
the
the
of
2D
2D
2D Aruco
Aruco
grid
the
grid
grid
the
the
Aruco
Aruco
Aruco
Aruco
2D Aruco
Aruco
Aruco
grid
grid
grid
board
board
board
grid
grid
grid
grid
grid
grid
board
boardin
board
board,
grid
board,
in
inthe
in
in
board,
board,
board,
board,
grid
grid board,
the
the
in the
deviations
deviations
deviations
deviations
deviations
board,
board, deviations
deviations
deviations
deviations
camera
camera
camera
camera
the
the camera
camera field
field
field
along
along
along
of
field
fieldof
of
field of
along
along
along
along
along
view
view
of
view
of
each
each
along
each
view
view
view
each
each
each
each
detected
detected
detected
detected
detected
each
each detected
detected
detected
detected
(FOV).
(FOV).
(FOV).(FOV).
(FOV).
(FOV).
Table
Table
Table
Table 3.
3.
Table
Table
parameter
parameter 3.
parameter
parameter 3.
parameter
parameter
parameter 3.3.
of
of Results
Results
Results
Results Results
Results
of of
the
the
of the
the
of
of the
tags
tags
the from
from
from
from from
from
tags
tags
tags
the the
the
for
for
tags
tags forthe
the
thethe
for benchmark
benchmark
benchmark
benchmark
various
various
for benchmark
benchmark
various
various
for
for for
for
positions
positions
various
various for
for
for
positions
positions
various the
the
the
for
positionsthe
of
of
positions
positions of 2D
the
the
of
of 2D
2D
the
of
the
of 2D
2Dgrid
grid
the Aruco
Aruco
Aruco
Aruco
2D
the Aruco
grid
grid
the
the grid
Aruco
board
board
grid board
board
grid
grid grid
grid
gridin
in
board
board inthe
the
in
in board,
board,
board,
board,
grid
grid
board board,
in
the
in the
camera
camera
the deviations
deviations
deviations
deviations
board, deviations
camera
camera
camera
the
the field
field
camera
camera field along
deviations
field
of
of
field of
field
field along
along
along
along
of
view
view
of of
of each
along
view
view
viewview
view each
each
each
each detected
detected
detected
detected
each
(FOV).
(FOV).
(FOV).
(FOV).
(FOV).
(FOV).
(FOV). detected
detected
Table
TableTable
Table
Table3.3.
Table
parameter3.3.
parameter
parameter
parameter
parameter 3.Results
3.
Results
parameter of
of Results
Results
Results
Results
ofof
the
ofofthe
the
the from
from
the
tags
the from
from
from
from
tags
tags
tags
tags the
the
for
tags
forforthe
the
the
forthe benchmark
various
for
for benchmark
benchmark
benchmark
benchmark
benchmark
various
various for
for
positions
various
various
various for
for
for
positionsfor
the
the
positions the
the
of
positions
positions
positions ofof the
2D
of2D
the
of
theof 2D
2D
2D 2D
the
the
thethe
grid Aruco
Aruco
Aruco
Aruco
Aruco
Aruco
grid
grid
grid
grid
grid grid
grid
board
board
board
board grid
grid
grid
grid
in
board
board ininin
the
in
the board,
inboard,
board,
board,
board,
board,
the
the
thethe deviations
deviations
deviations
deviations
deviations
deviations
camera
camera
camera
camera
camera
camera field
field
fieldof
field
field
field of of
of along
along
of along
along
along
along
view
of each
each
view
view
view
view view each
each
each
each
(FOV).
(FOV).
(FOV). detected
(FOV).
(FOV).
(FOV). detected
detected
detected
detected
detected
Table
Table
Table 3.
Table
Table
Table
3. 3.
parameter
parameter
parameter 3.
parameter
parameter Results
3. Results
3.
Results
parameter of
of Results
Results
Results
ofof
ofthe
the
the
of from
from
the from
from
from
from
tags
tags
tags
thethe for
tags
tags the
the
tags the
for
for the
the benchmark
thebenchmark
benchmark
benchmark
benchmark
benchmark
for various
various
various
various
for
for for
forforthe
for
for
positions
positions
positions
positions
various
various positionsof
positionsthe
the
for
theof
of of2D
the
2D
the
the
of of2D
2D2D
the
the Aruco
2D
grid
the
the Aruco
Aruco
Aruco
ArucoAruco
grid
grid
grid grid
gridgrid
board
board
board
board
grid
grid in
board
board board,
grid
grid
grid
in
in board,
in
the
the
in board,
board,
board,
inthe
the deviations
board, deviations
deviations
deviations
deviations
deviations
camera
camera
camera
camera
thethe field
field
field
field
camera
camera ofalong
of
of
field
field along
alongalong
along
of
view
of of each
along
each
view
view
view each
each
each detected
each detected
(FOV).
(FOV).
(FOV).
(FOV).
view
view detected
detected
(FOV).
(FOV). detected
detected
parameter
parameter
parameter
parameter
parameter
parameter of
of ofof
the
of
of
the the
the
the the
tags
tags tags
tags
tags
tags for
for for
for
for
for various
various
various
various
various
various positions
positions
positions
positions
positions
positions of
ofof ofthe
of
theof
the the
the
the grid
gridgrid
grid
grid
grid board
board
board
board
board
board in
in inthe
in
in
thein
the the
thethecamera
camera
camera
camera
camera
camera field
field
field
field
field
field of
of of
of ofview
of
view
view view
view
view (FOV).
(FOV).
(FOV).
(FOV).
(FOV).
(FOV).
parameter
parameter
parameter
parameter
parameter
parameter of
of ofthe
of
of
of
the the
the tags
the
the
tagstagsfor
tags
tags
tags
Roll
for
Roll
Roll
Roll various
for
for
forfor
Roll
Roll
Roll variouspositions
various
various
various
various positions
positions
positions of
positions
Pitch
positions
Pitch
Pitch
PitchPitch
Pitch
Pitch ofof the
of
of
the the
of grid
the
thethegrid
grid
Yaw
Yaw board
grid
grid
grid
Yaw board
board
board
board
Yaw
YawYaw
Yaw in
board
ininthe
in
in
the the
in camera
the
thethecamera
Xcamera
camera field
camera
camera
XXXXX field
X field of
field
field of
field
of Yview
ofofYview
of
view Yview
Yview
Y (FOV).
Yview (FOV).
(FOV).
(FOV).
Y(FOV).(FOV).ZZZZZ
ZZ
Roll
Roll
Roll
Roll
Roll
Roll
Roll
Roll
Roll
Roll
Roll Pitch
Pitch
Pitch
Pitch
Pitch
Pitch
Pitch
Pitch
Pitch
Pitch
Pitch Yaw
Yaw
Yaw
Yaw
Yaw Yaw
Yaw
Yaw
Yaw
YawYaw XX
XX
XXXX
XXX YY
YY
YYYY
YYY ZZ
ZZ
ZZZZ
ZZZ
Roll
Roll
Roll
Roll
Roll
Roll
Roll Pitch
Pitch
Pitch
Pitch
PitchPitch
Pitch Yaw
YawYaw
Yaw
Yaw Yaw
Yaw X
XXXXXX Y
YYYYYY Z
ZZZZZZ
Roll
Roll
Roll
Roll
Roll
Roll
Roll
Roll
Roll
Roll
Roll
Roll Pitch
Pitch
PitchPitch
Pitch
Pitch
Pitch
Pitch
Pitch
Pitch
PitchPitch Yaw
YawYaw
Yaw
Yaw Yaw
Yaw
Yaw
YawYaw
YawYaw XXXXXX
XXXXXX YYYYYY
YYYYYY ZZZZZZ
ZZZZZZ
+15°
+15°
+15°
+15°
+15° +15°
+15°
+15°
+15°
+15°
+15°
+15°
+15 ◦
+15° Roll
+15°
+15°
+15°
+15°
+15°
Roll
Roll
Roll
Roll
Roll Roll
Roll
+15°
Roll
+15°
+15°
+15° +15°
+15°
Roll
Roll
Roll
+15°
Roll
+15°+15°
+15°
+15°
+15°
Roll
Roll
Roll
+15°Roll
Roll
+15°
+15°
Roll
+15°+15°
+15°
RollRoll
Roll
RollRoll
Roll
Roll
RollRoll
Roll
Roll
Roll
Roll
Roll
Roll
RollRoll
Roll

−15°
−15°
−15°
−15°
−15° −15°
−15°
−15°
−15°
−15°
−15°
−15°
◦−15°
−15°
−15°
−15°
−15°
−15 −15°
Roll
Roll
Roll
Roll
Roll
Roll Roll
Roll
−15°
Roll
−15°
−15°
−15° −15°
−15°
Roll
Roll
Roll
−15°
Roll
−15°−15°
−15°
−15°
−15°
Roll
Roll
Roll
−15°Roll
Roll
−15°
−15°
Roll
−15°−15°
−15°
RollRoll
Roll
RollRoll
Roll
Roll
RollRoll
Roll
Roll
Roll
Roll
Roll
Roll
RollRoll
Roll

−30°
−30°
−30°
−30°
−30° −30°
−30°
−30°
−30°
−30°
−30°
−30°
−30°−30°
−30°
−30°
Roll
Roll −30°
−30°
RollRoll
Roll
−30°
Roll
−30°
−30°
−30°
−30 ◦
Roll
Roll −30°
−30°
Roll
Roll
−30° Roll
Roll
−30°
−30°
Roll
−30°
−30°
−30°
Roll
−30°Roll
Roll
Roll
−30°
RollRoll
−30°
−30°
−30°
−30°
Roll
Roll
Roll
Roll
Roll Roll
Roll
Roll
Roll
Roll
Roll
Roll
Roll
Roll
Roll
RollRoll
Roll

+15°
+15°
+15°
+15°
+15° +15°
+15°
+15°
+15°
+15°
+15°
+15°
+15°+15°
+15°
+15°
Pitch
Pitch +15°
+15°
Pitch
Pitch
Pitch
+15°
Pitch
+15°
+15°
+15°
Pitch
Pitch
+15°+15°
+15°
Pitch
Pitch
Pitch
+15°
+15°
Pitch
+15°
+15°
+15°
+15 ◦
+15°Pitch
Pitch
Pitch
Pitch
Pitch
Pitch
+15°
+15°
Pitch
+15°+15°
+15°
Pitch
Pitch
Pitch
Pitch Pitch
Pitch
Pitch
PitchPitch
Pitch
Pitch
Pitch
Pitch
Pitch
Pitch
Pitch
Pitch Pitch

+15°
+15°
+15°
+15°
+15° +15°
+15°
+15°
+15°
+15°+15°
+15°
Yaw
Yaw +15°
+15°
+15°
+15°
+15°
Yaw +15°
Yaw
Yaw
+15°
Yaw
+15°
+15°
+15°
Yaw
YawYaw
Yaw
+15° +15°
+15°
Yaw
+15°
+15°
Yaw
+15°
+15°
+15°
Yaw
Yaw
+15°
+15 Yaw
Yaw
+15°
YawYaw
+15°
+15°
+15°
◦ +15°
Yaw
Yaw Yaw
Yaw
YawYaw
Yaw
YawYaw
Yaw Yaw
Yaw
Yaw
Yaw
YawYaw
Yaw
YawYaw

0°0°
0°
0°
0° 0°
0°
0°
0°0°0°
0°
(Parallel
(Parallel
(Parallel
(Parallel0°
0°0°
(Parallel
0°0°
0°
(Parallel
to
0°0°
to
toto the
the
to
the
to the
the
the
(Parallel
(Parallel
(Parallel
(Parallel 0°
0° 0°
to
(Parallel
(Parallel
◦ 0°
0° 0°
0°
0° 0°
to
to
0° the
the
to the
the
to
to the
the
0 0°
(Parallel
(Parallel
(Parallel (Parallel
(Parallel
(Parallel
(Parallel
0°0°
0° to
0°
to to
to
0° to
the
to
the
the the
the
the
imaging
imaging
imaging imaging
imaging
(Parallel
imaging
(Parallel
(Parallel
(Parallel
(Parallel
imaging
imaging
imaging
imaging
imaging
(Parallel
(Parallel
imaging
(Parallel to theplane)
plane)
plane)
to
to
to
(Parallel
toto
plane)
plane) plane)
plane)
to
plane)
plane) the
plane)
the
the
the
to the
the
plane)
to
to
the the
plane)the
(Parallel
(Parallel
(Parallel
imaging
imaging
imaging
imaging
(Parallel
imaging
(Parallel
imaging(Parallel
(Parallel
(Parallel to
(Paralleltoto
tothe
plane)
to
to
plane)
to the
to the
the
plane)
plane)
plane)
plane)
the
the
to
the the
the
imagingimaging
imaging
imaging
imaging imaging
imaging
imaging
imaging
imaging imaging
imaging
imaging plane)
plane)
plane)
plane) plane)
plane)
plane)
plane)plane)
plane)
plane)
plane)
imaging
imaging
imagingimaging
imaging
imaging plane)
plane)
plane)
plane)
plane)
plane) plane)

Table 3 leads us to the inference that the deviation map greatly varies with the rotation of the tags.
To make the results clear, the deviations observed while detecting the pose of the tag are presented in
the form of histograms in Table 4.
 Sensors
Sensors
Sensors
Sensors 2020,
2020,
2020,
2020,20,20,
20,
20,xxFOR xxFOR
FOR PEER
PEER
PEER REVIEW
REVIEW
REVIEW 77of 77ofof
15
1515
Sensors
Sensors
Sensors
Sensors
Sensors
Sensors
Sensors
Sensors
Sensors
Sensors
Sensors
Sensors
2020,
2020,
2020,
2020,
2020,
Sensors 2020,
2020,
2020,
2020,
2020,
2020,
2020,
20,
20,
20,
2020, 20,
20,
20,
20, xxxxxxxxxFOR
20,
20,
20,
20,
20,
20,
FOR
FOR
FOR
xFOR
xFOR
xxFORFOR
FOR
FOR
FOR
FOR
FOR
FOR
PEER
PEER
PEER
PEER
PEER
PEER
PEER
PEER
PEER
PEER
PEER
PEER
PEER
PEER
REVIEW
REVIEW
REVIEW
REVIEW
REVIEW
REVIEW
REVIEW
REVIEW
REVIEW
REVIEW
REVIEW
REVIEW
REVIEW
REVIEW 777777777of
7of
77of
of
of
of
of
7ofof
of
of
of15
of
of
15
15
15
15
15
1515
15
15
15
15
15
15

Table
Table
Table
Table
Table
Table
Table
Table
Table
Table
Table
Table 33333333333333leads
leads
leads
leads
leadsleads
leads
3leads
leads
leads
leads
leads
leads us
us
us
usus
us
usus
us
us
us to
usto
to
to to
to
toto
to
to
to the
tothe
the
thethe
the
the
the
the
the
the inference
theinference
inference
inference
inference
inference
inference
inference
inference
inference
inference
inference that
that
that
that
that
that
that
that
that
that the
that
that the
thethe
the
the
the deviation
the
the
the
the
the deviation
deviation
deviation
deviation map
deviation
deviation
deviation
deviation
deviation
deviation
deviation map
map
map
map
map
map
map
map
map
map
map greatly
greatly
greatlyvaries
greatly
greatly
greatly
greatly
greatly
greatly
greatly
greatly
greatly varies
varieswith
varies
varies
varies
varies
varies
varies
varies
varies
varies with
with
withwith
with
with
with
with
with
with
with the
the
the
the
the
the
the rotation
the
the
the
the
the rotation
rotation
rotation
rotation
rotation
rotation
rotation
rotation
rotation
rotation
rotation of
of
of of
of
of
of
of the
of
of
of
of the
the
the
the
the
the the
the
the
the
the
Table
Table
Table leads
leads us
usus to
to to the
thethe inference
inference
inference that
that
that the
thethe deviation
deviation
deviation map
map
map greatly
greatly
greatly varies
varies
varies with
with
with the
thethe rotation
rotation
rotation of
ofof the
thethe
tags.
tags.
tags.
tags.
tags.
tags.
tags.
tags.
tags.
tags. To
To
tags.
tags. To
ToTo
To
To
To
To
To
To make
Tomake
make
make
make
make
make make
make
make
make
make the
the
the
thethe
the
the
thethe
the
the
the results
results
results
results
results
results
results
results
results
results
results
results clear,
clear,
clear,
clear,
clear,
clear,
clear,
clear,
clear,
clear, the
clear,
clear, the
the
the
the the
the
thethe
the
the
the deviations
deviations
deviations
deviations
deviations
deviations
deviations
deviations
deviations
deviations
deviations
deviations observed
observed
observed
observed
observed
observed
observed
observed
observed
observed
observed
observed while
while
while
while
while
while
while
while
while
while
while
while detecting
detecting
detecting
detecting
detecting
detecting
detecting
detecting
detecting
detecting
detecting
detecting the
the
the
the
the
the
the pose
pose
the
the
the
the
the pose
pose
pose
pose
pose
pose
pose
pose
pose
pose of
of
of of
of
of
of the
of
of
of
of
of the
the
the
the
the
the tag
the
the
the
the
the tag
tag
tag
tag
tag
tag tag
tag
tag
tag
tag are
are
are
are
are
are
are are
are
are
are
are
tags.
tags.
tags.
Sensors To
ToTo make
make
make
2020, 20, 4825 the
thethe results
results
results clear,
clear,
clear, the
thethe deviations
deviations
deviations observed
observed
observed while
while
while detecting
detecting
detecting the
thethe pose
pose
pose of
ofof the
thethe tag
tagtag are
8 are
of are
16
presented
presented
presented
presented
presented
presented
presented
presented
presented in
in
in inin inin
inin the
thethe
the
the the
the
the
the formform
form
form
form
formform
form
form of
of
of of
of
ofof
of
of histograms
histograms
histograms
histograms
histograms
histograms
histograms
histograms
histograms in
in
in in
in
inin
in
in Table
Table
Table
Table
Table
Table
Table
Table
Table 4.4.
4.4.
4.
4.
4. 4.
4.
presented
presented
presented
presented
presentedin
presented in in
in in inthe the
the
the
thetheformform
form
form
formof
form ofof
of
of histograms
histograms
histograms
ofhistograms
histogramsin
histograms in
inin
in Table
Table
Table
inTable
Table4.
Table 4.
4.
4.
4.
4.
Table
Table
Table
Table
Table 4.4.
Table
Table
Table
Table 4.
4.
4. Histogram
4.
4. Histogram
4.
4. Histogram
Histogram
Histogram
Histogram
Histogram
Histogram
Histogram of
ofof
of
of deviations
of
of
of deviations
deviations
deviations
deviations
deviations
deviations
deviations
deviations for
forfor
for
forfor
for the
for
for
thethe
the
the
the
the 2D
the
the
2D 2D
2D
2D2D
2D
2D
2Dgrid
grid
gridgridboard
grid
grid
grid
grid
gridboard
board
board
board pose
board
board
board
boardpose
posepose
pose
pose
pose
pose
poseestimates.
estimates.
estimates.
estimates.
estimates.
estimates.
estimates.
estimates.
estimates. The
TheTheThe
The
Thebars
The
The
The
The
bars bars
bars
barsfurther
bars
bars
bars
barsfurther
furtherfurther
further from
furtherfrom
further
further
further from
from
from
fromfrom
the the
from
from
from the
the
the
the
thethe
the
center
Table
Table
Table
Table
Table
Table
Table 4.
4.
4.4.
4.4.
4. Histogram
Histogram
Histogram
Histogram
Histogram
Histogram
Histogram of
ofof
of
ofof
of deviations
deviations
deviations
deviations
deviations
deviations
deviations for
forfor
for
for for
for thethe
the
the
the
the
the 2D
2D2D
2D2D
2D
2D grid
grid
grid
grid
grid
grid
grid board
board
board
board
board
board
board pose
pose
pose
pose
pose
pose
pose estimates.
estimates.
estimates.
estimates.
estimates.
estimates.
estimates. The
The
The
The
The
The bars
bars
bars
bars
bars
bars
bars further
further
further
further
further
further
further from
from
from
from
from
from thethe
the
the
the
the
the
center
center
center
of of
center
center of
center
center
center of
theof
center of
of
bar the
of
of the
of
of the
the
the bar
the
the bar
thebar
graph
the graph
bar
bar
bar
bar
bar graph
graph
graph
graph
graph
graph
graph
bar represent
represent
representthe
represent
represent
represent
represent
represent
represent
graph the
the
the
the
the count
represent the
thethe
the count
count
count
count
count
count
count
count
of of
of
of
of
of measurements
of
of
ofmeasurements
measurements
measurements
measurements
measurements
measurements
measurements
measurements
count of with
with
with
with
with
with
with
with
with more
measurements with more
more
more
more
more significant
more
more
more significant
significant
significant deviations.
significant
significant
significant
significant
significant
more deviations.
deviations.
deviations.
deviations.
deviations.
deviations.
deviations.
deviations.
significant deviations.
center
center
center
center
center
center
center of
ofof
of the
of
of the
the
the
thethe
the bar
bar
bar
bar
bar
bar
bar graph
graph
graph
graph
graph
graph
graph represent
represent
represent
represent the
represent
represent
represent the
the
the
the the
the count
count
count
count
count
count
count ofof
of
ofof
of
of measurements
measurements
measurements
measurements
measurements
measurements
measurements with
with
with
with
with
with
with more
more
more
more
more
more
more significant
significant
significant
significant
significant
significant
significant deviations.
deviations.
deviations.
deviations.
deviations.
deviations.
deviations.
Roll
Roll
Roll
Roll
Roll
Roll
Roll
Roll
Roll Pitch
Pitch
Pitch
Pitch
Pitch
Pitch
Pitch Yaw
Yaw
Yaw
Yaw
Yaw
Yaw
Yaw X
XXX
X
XX Y
YYY
YY Z
ZZZ
ZZ
Roll
Roll
Roll
Roll
Roll
Roll
Roll
Roll
Roll Pitch
Pitch
Pitch
Pitch
Pitch
Pitch
Pitch
Pitch Yaw
Yaw
Yaw
Yaw
Yaw
Yaw
Yaw
Yaw X
X
X
XX
XXX Y
Y
YY
Y
YYY Z
Z
ZZ
Z
ZZZ

+15°
+15°
+15°
+15°
+15°
+15°
+15°
+15°
+15°
+15°
+15°
+15°
+15°
+15°
+15°
Roll
Roll
+15Roll
◦Roll
Roll
Roll
Roll
RollRoll
Roll
Roll
Roll
Roll
Roll
Roll
Roll

−15°
−15°
−15°
−15°
−15°
−15°
−15°
−15°
−15°
−15°
−15°
−15°
−15°
−15°
−15°
Roll
Roll
−15 Roll
◦Roll
Roll
Roll
Roll
RollRoll
Roll
Roll
Roll
Roll
Roll
Roll
Roll

−30°
−30°
−30°
−30°
−30°
−30°
−30°
−30°
−30°
−30°
−30°
−30°
−30°
−30°
◦ Roll
Roll
Roll
−30Roll
Roll
Roll
Roll
Roll
Roll
Roll
Roll
Roll
Roll
Roll
Roll
Roll
Roll
Roll

+15°
+15°
+15°
+15°
+15°
+15°
+15°
+15°
+15°
+15°
+15°
+15°
+15°
+15°
+15°
+15 ◦ Pitch
Pitch
Pitch
Pitch
Pitch
Pitch
Pitch
Pitch
Pitch
Pitch
Pitch
Pitch
Pitch
Pitch
Pitch
Pitch

+15°
+15°
+15°
+15°
+15°
+15°
+15°
+15°
+15°
+15°
+15°
+15°
+15°
+15°
+15°
+15◦ Yaw
Yaw
Yaw
Yaw
Yaw
YawYaw
Yaw
Yaw
Yaw
Yaw
Yaw
Yaw
Yaw
Yaw
Yaw

0°
0°
0° 0°
0°
0°
0°0°
0°
0°
0°
0°
0◦ (Parallel
0°
0°
(Parallel
(Parallel
(Parallel
(Parallel
(Parallel
(Parallel
(Parallel
(Parallel
(Parallel
(Parallel
(Parallel
(Parallel
to the toto
toto
to
to
to to
to
to to
to
(Parallel
(Parallel
(Parallel to
to to
the
thethe
thethe
the
thethe
the
the
the
the imaging
imaging
imaging
imaging
imaging
imaging
imagingimaging
imaging
imaging
imaging
imaging
imaging
the
thethe imaging
imaging
imaging
plane)
plane)
plane)
plane)
plane)
plane)
plane)
plane)
plane)
plane)
plane)
plane)
plane)
plane)
plane)
plane)
The
The
The
The
The
The
TheThe
The
The
The
The error
error
error
error
error
error
error
error
error
error
error
error is
is
is
is is
is
isis
is
is
is large
islarge
large
large
large
largelarge
large
large
large
large
large while
while
while
while
while
whilewhile
while
while
while
while
while moving
moving
moving
moving
moving
moving
moving
moving
moving
moving
moving
moving along
along
along
along
along
along
along
along
along
along
along
along the
the
the
thethe
the
the
thethe
the
the
the +Z-axis
+Z-axis
+Z-axis
+Z-axis
+Z-axis
+Z-axis
+Z-axis
+Z-axis
+Z-axis
+Z-axis
+Z-axis
+Z-axis (Yaw);
(Yaw);
(Yaw);
(Yaw);
(Yaw);
(Yaw);
(Yaw);
(Yaw);
(Yaw);
(Yaw);
(Yaw);
(Yaw); however,
however,
however,
however,
however,
however,
however,
however,
however,
however,
however,
however, changes
changes
changes
changes
changes
changes
changes
changes
changes
changes
changes
changes in
in
in
in in
in
inin
in
in the
in
in the
the
the
the
the the
the
the
thethe
the Yaw
Yaw
Yaw
Yaw
Yaw
Yaw
Yaw Yaw
Yaw
Yaw
Yaw
Yaw angle
angle
angle
angle
angle
angle
angle
angle
angle
angle
angle
angle
The
The
The error
error
error is
is is large
large
large while
while
while moving
moving
moving along
along
along the
thethe +Z-axis
+Z-axis
+Z-axis (Yaw);
(Yaw);
(Yaw); however,
however,
however, changes
changes
changes in
in in the
thethe Yaw
YawYaw angle
angle
angle
have
have
have have
have
have
have have
have
have
havesmall
have
small
small
small
The
small
small
small
small
small
small
small
small
effects
effects
effects
error
effects is
effects
effects
effects
effects
effects
effects
smalleffects
effectson
effects
on
on
large
onon
on
on
on
on
on
on
on
onthethe
the
the the
the
thewhile
the
the
the
the
the
deviations.
deviations.
deviations.
moving
deviations.
deviations.
deviations.
deviations.
deviations.
deviations.
deviations.
thedeviations.
deviations.This
deviations.
This
This This
along
This
This This
This This
This
This
Thisis
This
is
is
is is
is
isexpected
is
is expected
the
is +Z-axis
expected
expected
expected
expected
expected
expected
is expected
expected
isexpected
is expectedas
expected
as
asas
as
as
as as
as
as
when
when
when
(Yaw);
when
when
when
when
when
as when
when
aswhen
as whenthe
when
thethe
the
the
the the
the
the
board
board
board
however,
the
the board
board
board
board
board
board
board
theboard
the boardmoves
board
moves
moves
moves
changes
moves
moves
moves
moves
moves
moves
moves
movesaway
moves
away
awayaway
awayin
away
away
away
away
away
away
awayfrom
away
from
the from
from
from
from
from Yaw
from
from
from
from
fromthe
from
thethe
theangle
the
the
the
the the
the
the
thecamera,
the
camera,
camera,
camera, have
camera,
camera,
camera,
camera,
camera,
camera,
camera,
camera,
camera,
have
have small effects on the deviations. This is expected as when the board moves away from the camera,
the
the
the
the the
small
the
the the
the
the
the
the pose
pose
pose
pose
pose
pose
pose
pose
pose
pose
pose
pose estimate
estimate
estimate
effectsestimate
estimate
estimate
estimate
estimate
estimate
estimate
estimate
estimate becomes
on thebecomesbecomes
becomes
deviations.
becomes
becomes
becomes
becomes
becomes
becomes
becomes
becomes more
more
more
This
more
more
more
more
more
more
more
more
more sensitive
sensitive
sensitive
is expected
sensitive
sensitive
sensitive
sensitive
sensitive
sensitive
sensitive
sensitive
sensitive to
to
to
to to
to
toto
to
to noise
to
to noise
noise
as
noise
noise
noise
noise
noise
noise
noise
noise and
when
noise and
and
and
and
andand
and
and
and
and
and errors
theerrors
errors
board
errors
errors
errors
errors
errors
errors
errors
errors
errors in
in
inin
in in
in
in
in in
in calibration.
calibration.
moves
in calibration.
away Rotation
calibration.
calibration.
calibration.
calibration.
calibration.
calibration.
calibration.
calibration.
calibration. Rotation
Rotation
Rotation
from
Rotation
Rotation
Rotation
Rotation
Rotation
Rotation
Rotation
Rotation of
of
the camera,
of
of
of of
of
ofof
of
of
of the
the
the
the
the
the
the
the the
the
the
the tag
tag
tag
the
tag
tag
tag
tag tag
tag
tag
tag
tag
the
the the pose
pose
pose estimate
estimate
estimate becomes
becomes
becomes more
more
more sensitive
sensitive
sensitive to
to to noise
noise
noise and
and
and errors
errors
errors in
in in calibration.
calibration.
calibration. Rotation
Rotation
Rotation of
of of the
thethe tag
tagtag
sometimes
sometimes
sometimes
pose
sometimes
sometimes estimate
sometimes
sometimes
sometimes
sometimes
sometimes
sometimes
sometimes leads
leads
leads
leads
leads
leads
leads
leads to
becomes
leads
leads
leads
leads to
to
to to
to
toto
to
to the
to
to the
thethe
the
the
the the
the
the
the
the formation
formation
formation
more
formation
formation
formation
formation
formation
formation
formation of
of
of
of
of of
sensitive
formation
formation of
of
of
of a
of
of a aaboundary
a boundary
boundary
to noise
boundary
boundary
aboundary
boundary
aaaaaaaaboundary
boundary
aboundary
boundary
boundary
boundary at
at
and at
at
at
at which
at
at
at
at
at which
which
errors
at which
which
which
which
which
which
which
which
which moving
in moving
moving
moving
moving
moving
moving
moving
moving
moving the
calibration.
moving
moving the
thethe
the
the
the the
the
the
the
the tagtag
tag
tag
tag by
by
tag
tag
tag
tag
tag by
by
by
by
by by
by
by a
Rotation
tag
tag byby a aasmall
a small
smallof
small
small
asmall
aaaaaaaasmallsmall
small
asmall
small
small
small amount
the amount
amounttag
amount
amount
amount
amount
amount
amount
amount
amount
amount leads
leads
leads
sometimes
leads
leads
leads
leads
leads
leads
leads
leads
leads toto
to
to to
to
to
toto
to
to
to
sometimes
sometimes
sometimes leads
leads
leads to
to to the
thethe formation
formation
formation of
ofof boundary
boundary at
atat which
which
which moving
moving
moving the
thethe tag
tagtag by
byby small
small amount
amount
amount leads
leads
leads to
to to
asignificant
aaaaaaleads
aa significant
significant
to the
significant
significant
significant
significant
significant
a significant change
change
change
formation
change
change
change
change
change
change in
in
in in
in
ininthe
in
in the
the
ofthe
the
the
the the
the a detected
detected
detected
boundary
detected
detected
detected
detected
detected
detected orientation.
orientation.
orientation.
at
orientation. which
orientation.
orientation.
orientation.
orientation.
orientation. This
This
This
moving
This
This
This
This
This
This step
step
step
step
step the
step
step
step
step boundary
boundary
boundary
tag
boundary by
boundary
boundary
boundary
boundary
boundary a is
is
is is
small
is
isis
is
is shown
shown
shown
shown
shown
shown
shown
shown in
amount
shown in
in in
in in
in
inin Figure
Figure
Figure
leads
Figure
Figure
Figure
Figure
Figure
Figure 8
888
88
and
8 and
and
to
8 and
and
8 a
and
and
and
and is
is
is is
is
is is similar
similar
similar
significant
is similar
similar
similar
similar
similar
is similar
aaaaasignificant
significant
significant
significant
significant
significant change
change
change
change
changein
change in
inin
ininthethe
the
the
thethe detected
detected
detected
detected orientation.
orientation.
orientation.
detectedorientation.
detected orientation.This
orientation. This
This
This step
step
step
Thisstep
This step boundary
boundary
boundary
stepboundary
boundaryis
boundary is
isis
is shown
shown
shown
isshown
shown
shown inin
in in
in Figure
Figure888888and
Figure
Figure
inFigure
Figure and
and
and
andandis is
isis
is similar
similar
similar
similar
is similar
similar
to
toto
toto to
to what
to
to what
change
to to what
what
what
what
what
what
what
what
what was
was
was
in
wasthe
was
was
was
was
was
was
was observed
observed
observed
detected
observed
observed
observed
observed
observed
observed
observed
observed in
in
in
in in
in
in Figure
in
in Figure
Figure
orientation.
inin Figure
Figure 5.
Figure
Figure
Figure
Figure
Figure
Figure 5.
5.
5.
5. 5.This step boundary is shown in Figure 8 and is similar to what was
5.
5.
5.
5.
5.
to
to to what
what
what was
was
was observed
observed
observed in
in in Figure
Figure
Figure 5.
5.
5.
observed in Figure 5.
Sensors 2020, 20, 4825 9 of 16
Sensors 2020, 20, x FOR PEER REVIEW 8 of 15

(a) (b) (c)

(d) (e) (f)

Figure 8.
Figure 8. The
The black
black lines
lines depict
depict step
step boundaries;
boundaries; when
when the
the tag
tag moves
moves across
across this
this line,
line, aa sudden
sudden change
change
in the detected tag orientation is observed. Observations for (a) Roll; (b) Pitch; (c) Yaw; (d) X;
in the detected tag orientation is observed. Observations for (a) Roll; (b) Pitch; (c) Yaw; (d) X; (e)
(e) Y; (f) Z.
Y; (f) Z.

5. Recalibration with
5. Recalibration with aa Liquid-Crystal
Liquid-CrystalDisplay
Display
The
The observations
observations fromfrom the
the benchmark
benchmark process
process mentioned
mentioned in in Table
Table 33 justified
justified that
that the
the standard
standard
calibration process yielded a set of parameters that left a lot of instability in the Aruco
calibration process yielded a set of parameters that left a lot of instability in the Aruco tag detection tag detection
and
and the
the estimation
estimation of of its
its pose.
pose. Upon
Upon observing
observing thethe fact
fact that
that the
the benchmark
benchmark results
results were
were different
different for
for
+15 ◦ Roll and −15◦ Roll, although a symmetry was expected, it was decided that a recalibration was
+15° Roll and −15° Roll, although a symmetry was expected, it was decided that a recalibration was
required. On the
required. On the second
second attempt,
attempt, instead
instead ofof printing
printing thethe calibration
calibration board
board anew,
anew, wewe decided
decided to to use
use aa
Liquid-crystal
Liquid-crystal display
display toto display
display the board, as
the board, as it
it is
is never
never warped
warped (stays
(stays completely
completely flat)
flat) and
and is
is backlit,
backlit,
which should help to detect the board better (see Figure 9). The dimensions of the chessboard
which should help to detect the board better (see Figure 9). The dimensions of the chessboard squares squares
were
were measured
measured and
and were
were input
input into
into the
the calibration
calibration algorithm.
algorithm.

Figure 9. The calibration with a Liquid-crystal display.

and the estimation of its pose. Upon observing the fact that the benchmark results were different for
+15° Roll and −15° Roll, although a symmetry was expected, it was decided that a recalibration was
required. On the second attempt, instead of printing the calibration board anew, we decided to use a
Liquid-crystal display to display the board, as it is never warped (stays completely flat) and is backlit,
which should
Sensors 2020, help to detect the board better (see Figure 9). The dimensions of the chessboard squares
20, 4825 10 of 16
were measured and were input into the calibration algorithm.

Figure
Figure 9.
9. The
The calibration
calibration with
with aa Liquid-crystal
Liquid-crystal display.
display.

The results from the new calibration are mentioned in Equations (3) and (4).
 
camera0 s distortion coe f f icients = 0.1679 −0.4901 −0.0032 −0.0011 0.4101 (3)
 
 902.9293 0 634.7668 
camera0 s intrinsic parameters = 
 
0 902.3267 354.3479  (4)
 
0 0 1
The effects of the improved calibration are mentioned further in this paper, this calibration will be
referred to as “new calibration” and the previous calibration will be called “old calibration”.

6. Three-Dimensional Aruco Grid Board

The observations made on the 2D Aruco grid-board led to these inferences:

1. Pose estimates are unstable when the orientation of the markers relative to the camera is
perpendicular, i.e., the camera Z-axis is overlapping with the marker Z-axis.
2. An error detection metric with redundant tags being detected at the same time can be used to
improve pose estimation, i.e., if we have five tags and we know one of them has a pose estimate
which is considerably different from the others, we could discard it and use the other tags for
pose estimation.

Given the first inference, it was decided that a 3D organization of the tags should ensure that
no two tags are normal to the camera at any point; therefore, a 3D grid board was proposed that
consisted of five Aruco tags on five different planes angled to the center plane at 15◦ (see Figure 10a).
This allowed having multiple tags in one image with a known rigid transformation from any tag to the
center tag. It also allows the development of a metric to detect the error in the net detected pose (the
second inference). The characteristics of the grid are mentioned in Table 5. The grid board was mounted
on a UR3 robot (Figure 10d), and the robot coordinate system was used for all the measurements.
 Sensors 2020, 20, 4825 11 of 16
Sensors 2020, 20, x FOR PEER REVIEW 10 of 15

(a) (b)

(c) (d)
Figure
Figure10.
10.Three-dimensional
Three-dimensionalgrid gridboard:
board: (a)
(a) schema;
schema; (b)
(b) position and orientation
position and orientationofofindividual
individualtag
tagon
onthe
the3D
3Dgrid
gridboard;
board;(c)(c)the
thereal
real3D
3Dgrid
grid board;
board; (d)(d) the
the UR3
UR3 robot
robot with
with the
the 3D3D grid
grid board.
board.

An example of the rigid Table 5. Three-dimensional

transform grid board
between a non-central parameters.
tag (Aruco 1, Figure 10b) to the central
tag (Local Coordinate System of the grid board) is depicted
Number of Tags on the Grid Board in Equation
5 (5).
Tag edge0.9659
length 0 −0.2588 70−0.1
mm
Spacing between
0 tags1 0 30 mm
0
𝑀𝑇1→𝐿𝐶𝑆 =
Tag(type 6 × 1000) (5)
0.2588 0 0.9659 6 ×−0.0129
0 0 0 1
The first step of the process was to detect the tags and transform their detected pose from the
7.camera
Error Metric for the
coordinate 3D to
frame Grid Boardcoordinate system. A Python module was written for this purpose
the robot
that enables modelling the transforms for such 3D grid boards. For the grid itself, the local coordinate
To ensure that the net pose calculated from the 3D grid board benefits from the redundancy of
frame is identical with the coordinate system of the center tag.
having five tags with the rigid transformations known between the individual tags, an error metric
An example of the rigid transform between a non-central tag (Aruco 1, Figure 10b) to the central
was devised to detect if any subset of the tags was detected incorrectly. First, all the individual tags
tag (Local Coordinate System of the grid board) is depicted in Equation (5).
were detected, and their pose was estimated, then the detected poses of the non-central tags were
transformed to the coordinate frame of the  central tag (Figure 11a). Once this was done, the tag with
 0.9659 0 −0.2588 −0.1 
the highest distance (Figure 11b) from the  expected position was eliminated  from the net pose
 0 1 0 0 
MT1→LCS
estimate to improve the accuracy. =  to measure the error in the final pose estimate, we defined
In order (5)
 0.2588 0 0.9659 −0.0129 
an error metric that took as its input the transformed origins of all the non-central tags (and the
0 0 0 1
distance from the real center for the central tag) and computed a scalar that described the error in the
net
7. pose
Errorestimate. This
Metric for theerror metric
3D Grid is described in Algorithm 2.
Board
To ensure that the net pose calculated from the 3D grid board benefits from the redundancy of
having five tags with the rigid transformations known between the individual tags, an error metric
was devised to detect if any subset of the tags was detected incorrectly. First, all the individual tags
Sensors 2020, 20, 4825 12 of 16

were detected, and their pose was estimated, then the detected poses of the non-central tags were
transformed to the coordinate frame of the central tag (Figure 11a). Once this was done, the tag with
the highest distance (Figure 11b) from the expected position was eliminated from the net pose estimate
to improve the accuracy. In order to measure the error in the final pose estimate, we defined an error
metric that took as its input the transformed origins of all the non-central tags (and the distance from
the real center for the central tag) and computed a scalar that described the error in the net pose
estimate. This error metric is described in Algorithm 2.

Algorithm 2. Calculating the Error in the Net Pose Estimation of the 3D Grid Board.
Detect tvecs of Tags //tvecs represent array of translations of the origin of detected tags in
for i = 0 to length(tvecs) // camera’s coordinate frame
for j = 0 to length(tvecs)
deviation_distance[i] += (distance(tvecs[i] – tvecs[j])) //calculate distance between transformed
//tag centers
end for
end for
filtered_tvecs = remove_detections_with_high_deviation (tvecs, deviation_distance)

volume = (get_max_x(filtered_tvecs)–get_min_x(filtered_tvecs)) * // get min and get max functions

(get_max_y(filtered_tvecs)–get_min_y(filtered_tvecs)) * // return the min and max tvecs
(get_max_z(filtered_tvecs)–get_min_z(filtered_tvecs)) // along each axis
Sensors 2020, 20, x FOR PEER REVIEW 11 of 15

(a) (b)
Figure 11. An illustration (not to scale) to describe the error measurements and correction. (a)
Figure 11. An illustration (not to scale) to describe the error measurements and correction. (a) transforms
transforms the tag poses from their coordinate frame to the center tag frame. (b) shows the error in
the tag poses from their coordinate frame to the center tag frame. (b) shows the error in deviation
deviationduring
observed observed during thethe
the experiment, experiment, the marked
detected pose detectedwith
pose
themarked with
yellow box the
has theyellow
highestbox has the
deviation.
highest deviation.
8. Results
Algorithm 2 Calculating the Error in the Net Pose Estimation of the 3D Grid Board
This section discusses the results from experiments across this paper and compares the results
Detect tvecs of Tags // tvecs represent array of translations of the origin of detected tags in
between different benchmarks to identify how the different grid boards performed. Table 6 lists the
for i = 0 to length(tvecs) //camera’s coordinate frame
indices used across this section to label the different experiments done.
for j = 0 to length(tvecs)
deviation_distance[i] += (distance(tvecs[i] – tvecs[j])) //calculate distance between transformed
//tag centers
end for
end for
filtered_tvecs = remove_detections_with_high_deviation (tvecs, deviation_distance)

volume = (get_max_x(filtered_tvecs)–get_min_x(filtered_tvecs)) * // get min and get max functions

(get_max_y(filtered_tvecs)–get_min_y(filtered_tvecs)) * // return the min and max tvecs
Sensors 2020, 20, 4825 13 of 16

Table 6. Labels for individual measurement results.

Old Calibration New Calibration

2D grid board Measurement 1 (M1) Measurement 3 (M3)
3D grid board Measurement 2 (M2) Measurement 4 (M4)

The benchmark results for all the cases are shown and compared in Table 7, the error metric was
used for the 2D grid board as well, and the worst detected tag was filtered from the net pose estimate
for all cases mentioned except for M0, which is given for reference (Table 3). M0 is the observation
when directly
Sensors 2020,
Sensors
Sensors
Sensors
Sensors
Sensors 2020, using
20,
2020,
2020,
2020,
2020, 20,
20, x20,
20,xFOR
x20,
FOR xOpenCV’s
xFOR
xFOR PEER
FOR
FORPEER
PEER
PEER
PEER
PEER grid board pose estimation.
REVIEW
REVIEW
REVIEW
REVIEW
REVIEW
REVIEW 12
1212of
12
12
of 12
of
1515
of
of15
1515
of 15
Sensors
Sensors
Sensors2020,
2020,
Sensors
Sensors
Sensors
Sensors
Sensors
Sensors 20,
2020,
2020,
Sensors 20,
2020,
2020,
2020,
Sensors 20,
2020,
2020,
2020, 20,
20,
2020,
20,
20, xxFOR
xxFOR
x20,
20,
xx20,
FOR
20,xxFOR
FOR
FOR
FOR PEER
xxFOR
FORPEER
FOR
PEER
FORPEER
PEER
PEER REVIEW
PEER
PEER REVIEW
PEER
PEER REVIEW
REVIEW
REVIEW
REVIEW
REVIEW
PEER REVIEW
REVIEW
REVIEW
REVIEW 12
12
12
12 ofof
1212
12
1212
of
of of
1215
of
of
12 15
of
of
15
of
15
15 15
15
of 15
15
15
15

Table
Table 7.
Table
TablePerformance
Table
Table
Table 7.7.7.Performance
7.7.
7. comparison
Performance
Performance
Performance
Performance
Performance comparisonbetween
comparison
comparison
comparison
comparison
comparison thethe
between
betweenAruco
betweenthe
between
between
between the
the
thethegrid
Aruco
Aruco
Arucoboards
grid
Aruco
Aruco
Aruco grid
grid
grid
grid
grid described
boards
boards
boards
boards
boards
boards in this
described
describedin
described
described
described
described in in paper.
this
in
ininthis
this
this this The
paper.
this paper.normal
The
paper.
paper.
paper.
paper. The Thenormal
The
TheThenormal
normal
normal
normal
normal
Table
Table
Table
Table
TableTable
Table 7.7.
Table
Table
Table
Table
Table 7.7.7.Performance
7. Performance
7.Performance
7.
7. Performance
Performance
7. Performance
Performance
Performance
Performance
Performance
7.7. Performance
Performance comparison
comparison
comparison
comparison
comparison
comparison
comparison
comparison
comparison
comparison
comparison
comparison between
between
between
betweenthe
between
between
between
the
between
between
between
between the
the
the
between Aruco
theAruco
theAruco
the
the
Aruco
the grid
Aruco
Aruco
the
the grid
Aruco
Aruco
grid
Aruco
Aruco
Arucogrid
Aruco boards
gridboards
grid
grid boards
grid
grid
grid described
boardsdescribed
boards
boards
boards
boards
boards
boards
grid
gridboards describedin
inin
described
described
described
in
described
described
described
described
boards describedin
described this
inthis
in
this
in
in
in
this
this
in
in paper.
thispaper.
this
this paper.
this
this
this The
paper.The
paper.
paper.
paper.
The
paper.
paper.
paper.
this The
The
The
paper.
paper. normal
The
The normal
The
The
Thenormal
normal
normal
normal
normal
normal
normal
normal
The normal
normal
to the
totototag
the
to
to
tothe
the
the is
thetag
the
tag tagparallel
tagis
tag
tag
is is
is is
is with
parallel
parallel
parallelthe
with
parallel
parallel
parallel camera
the
with
with
with
with
with the
the
the
the the Z-axis.
camera
camera
cameraZ-axis.
camera
camera
camera Z-axis.
Z-axis.
Z-axis.
Z-axis.
Z-axis.
toto
to
to
to the
toto
the
to
to
to
the
the
to the
the tag
the
the
tag
the
thethe
tag
tag tag
tag isis
tag
tag
is
tag
tagtag
is
is parallel
is
parallel
is
is
is
parallel
parallel
is with
parallel
is parallel
parallel
with
parallel with
parallel
parallel the
with
with
with
the
with
with
with
parallel the
the
with
the
the
with camera
the
the
the camera
camera
thecameraZ-axis.
camera
camera
camera
camera
camera
camera
the camera Z-axis.
Z-axis.
Z-axis.
Z-axis.
Z-axis.
Z-axis.
Z-axis.
Z-axis.
Z-axis.
Z-axis.
Roll Pitch Yaw XX Y
Roll
Roll
RollRoll
Roll
Roll
Roll
Roll
Roll
Roll
Roll
Roll
Roll
Roll
Pitch
Pitch
Pitch
PitchPitch
Pitch
Pitch
Pitch
Pitch
Pitch
Pitch
Pitch
Pitch
Pitch
Yaw
Yaw
YawYaw
Yaw
Yaw Yaw
Yaw
Yaw
Yaw
Yaw Yaw
Yaw
Yaw
XX
XXXXXX
XX
XX X YYY
Y
YY
YYY
YY
YY Y ZZZ
Z
ZZ ZZ
Z
ZZZZ
ZZ

M0
M0
M0 M0
M0M0
M0
M0
M0M0
M0
M0
M0M0
M0

M1
M1
M1M1
M1
M1 M1
M1
M1
M1 M1
M1
M1
M1
M1
M1

M2
M2
M2
M2
M2M2
M2
M2M2
M2
M2
M2
M2 M2M2

M3
M3
M3 M3
M3M3
M3
M3
M3M3
M3
M3
M3M3
M3

M4
M4
M4M4 M4
M4
M4M4
M4M4
M4
M4 M4M4
M4

The
The
The
The The
The
TheThe
The The
The
The observations
observations
observations
observations
observations
observations
observations
observations
observations
observations
observations
observations confirm
confirm
confirm
confirm
confirm
confirm
confirm
confirm
confirm
confirm
confirm
confirm that
that
that
that
that
that
that
that
that there
that
there there
there
that
that there
there
there
there
there there
there
there was
was
was
was
was
was
was was
was was
an
was
wasanan
an an
an
an
an increase
an
an increase
increase
increaseincrease
increase
increase
an
an increase
increase
increaseincrease
increase in
inin
inin
inin
the
in the
inin
the
the
in the
the
the
in the
the accuracy
theaccuracy
accuracy
accuracy
accuracy
accuracy
accuracy
the
the accuracy
accuracy
accuracy
accuracy
accuracy of
ofof
of
of
ofthe
of
of
of
the
the
of the
the
the
the
of
the
of the detected
the detected
detected
detected
detected
detected
detected
the
the detected
detected
detected
detected
detected pose
pose
pose
pose pose
pose
pose
pose
pose
pose
pose
pose when
when when
when
when when
when
when
whenwhen
when
when
The
The The
The observations
observations
observations
observations confirm
confirm
confirm
confirm that
that
that
that there
there
there there was
was
waswas ananan
an increase
increase
increase
increase ininin the
the
in the
the accuracy
accuracy
accuracy
accuracy ofof
ofthe
the
of the detected
detected
detected
the detected pose
pose pose
pose when
whenwhen
when
the The
the
the
the
thethe observations
proposed
proposed
proposed
proposed
proposed
proposed 3D
3D 3D
3D 3D 3D
grid confirm
grid
grid
grid grid board
grid board
boardthat
board
board
board was
waswasthere
was
wasused,
was used,was
used,
used,
used, used,and
and an
and
and
andand
theincrease
thethe
the theresults
the results
resultsinare
results
results
results the
areare
are
are accuracy
consistent
are consistent
consistent
consistent
consistent
consistent of the
across
across detected
across different
across
across
across different
differentpose
different
different
different when
measurements.
measurements.
measurements.
measurements.
measurements.
measurements. the
the
the
the
the
the proposed
the
the proposed
proposed
proposed
proposed
proposed
proposed
the proposed 3D
3D
3D 3D
3D
3D3D grid
3D
grid
grid grid
gridgrid
grid board
grid board
board
board
board
board
board
board was
was
was was
waswas
was used,
was used,
used,
used,
used, used,
used,
used, and
and
and and
andand
and
and the
the
the the
the the
the results
the results
results
results
results
results
results
results are
are
are are
are
areconsistent
are consistent
consistent
consistent
consistent
consistent
consistent
are consistent across
across
acrossacross
across
across different
across
across different
different
different
different
different
different
different measurements.
measurements.
measurements.
measurements.
measurements.
measurements.
measurements.
measurements.
proposed
There
There
There
There
There 3D
isisis
is grid
still
still
is an
still
still board
still
anananincrease
an wasinin
increase
increase
increase
increase used,
in
in error
inin and
error
erroras
error
error asthe
as
asthe
as
the results
the
thetag
thetag
tag
tag was
tagwasare
was
was
was consistent
moved
moved
moved
movedmoved away
away
away
away across
away from
fromdifferent
from
fromfromthethe
thethecamera
the measurements.
camera
camera
camera
camera (+Z-axis),
(+Z-axis),
(+Z-axis),
(+Z-axis),
(+Z-axis), There
but
but
but
but that
but that
thatthat
thatisisisisis
There
There
There
There
There
There
There
There is
is
is isstill
still
is
Thereisisstill still
is
is
still anan
still
still
an
still
stillan an
anincrease
anincrease
anincrease
increase increase
increase
increase
anincrease
increasein in
in in error
in error
in error
in
error
inerror as
error
error as
error
as
erroras as the
as the
as
as
the
asthe thetag
the
the tag
thetag
tag
thetag was
tag
tag was
tag
was
tagwas was was
was moved
wasmoved
moved
moved
wasmoved moved
moved
moved
movedaway away
away
away
away away
away
away
awayfrom from
from
from
from from
from the
fromthe
the
fromthe the camera
the
the camera
the camera
camera
thecamera camera
camera
camera (+Z-axis),
(+Z-axis),but
(+Z-axis),
(+Z-axis),
(+Z-axis),
(+Z-axis),
(+Z-axis),
(+Z-axis),
camera(+Z-axis), butbut
butthat
butthat
but
that
but
but
but that
that
that
that
that isisis
isis
that isisisis
is still an
expected increase
expected
expected
expected because
becausein
because
because error
the the
the as
tags
the the
tags
tags
tagsare tag
are
are
are was
more
more
more
more moved
distant
distant
distant
distant away
from
from
from
from from
thethe
the
the the
camera.
camera.
camera.
camera.cameraThe
The
The
The (+Z-axis),
histograms
histograms
histograms
histograms but
ininin that
Table
inTableis
Table
Table expected
88888888confirm
8confirm
confirm
8confirm
confirm thethethe
expected
expected
expected
expected
expected
expected
expected
expected
expected
expected
expected
because
because
because
because
because
because
because
because
because
because
because
the
the the
the
the
the the
the
thethe tags
tags
tags
tags
the
tags are
tags
tags
tags
tags
tags
tags
are
are
are
are are
are
are are
are
are
more
more
more
more
moremore
more
more
more
more
more
distant
distant
distant
distant
distant
distant
distant
distant
distant
distant
from
distant from
from
from
from
from from
from
from
the
from
from the
the
the
the
the the
the
the
camera.
camera.
camera.
camera.
the
the camera.
camera.camera.
camera.
camera.
camera.
camera.
The The
The
The
The
The The
The
TheThe
The histograms
histograms
histograms
histograms
histograms
histograms
histograms
histograms
histograms
histograms
histograms
inin
in
in
in
ininin
inTable
Table
Table
Table
in
Table
in Table
Table
Table
Table
Table
Table 88confirm
88confirm
confirm
confirm
8confirm
confirm
confirm
confirm
confirm
confirm the the
the
the
the
the the
the
the
the
the
the
because
fact
factfact
factthe
fact
fact that
that
that
that tags
that
that the
thethe are
the
the
the more
large
large
large
large
large large distant
deviations
deviations
deviations
deviations
deviations
deviations from the
from
from
from
from from camera.
from real
real
real
real
real
real pose The
pose
pose
pose
pose pose histograms
estimates
estimates
estimates
estimates
estimates
estimates havein
have Table
have
have
havehave been 8been
been
been
been
been confirm the factreduced
significantly
significantly
significantly
significantly
significantly
significantly that
reduced the inin
reduced
reduced
reduced
reduced in ininin
fact
fact fact
fact
fact
fact
fact that
that
that
that that
that
that the
the
the
the thethe
the large
large
large
large large
large
large deviations
deviations
deviations
deviations
deviations
deviations
deviations from
from
from
from from
from
from realreal
real
real
real
real
real pose
posepose
pose
pose
pose pose estimates
estimates
estimates
estimates
estimates
estimates
estimates have
havehave
have
have
have have been
beenbeen
been
been
been been significantly
significantly
significantly
significantly
significantly
significantly
significantly reducedreduced
reduced
reduced
reduced
reduced
reduced inin
ininininin
large deviations
measurements
measurements
measurements
measurements
measurements
measurements
measurements from done
done done
done
done real
done
done with pose
with
with
withwith
with
with the
the estimates
the
the
the 3D
the
the
3D
3D 3D
3D3Dgrid
3D grid
grid
grid
grid have
grid board
grid
board
board been
board
board
board
board (i.e.,
(i.e.,
(i.e.,significantly
(i.e., most
(i.e.,
(i.e.,
(i.e., most most
most
most
most mostof
of of
of
of the
of
theof
the
the reduced
the
the deviations
the deviationsin
deviations
deviations
deviations
deviations
deviations measurements
are
are are
are
are small).
are
are small).
small).
small).small).
small).
small). ForFor
For
For
For done
For For example,
example, with
example,
example,
example,
example,
example, inin
ininininin
measurements
measurements
measurements
measurements
measurements
measurements
measurements
measurements done
done
donedone
done
donedone
done withwith
with
with with
with
with
withthe the
the the
the
thethe
the
3D
3D3D
3D
3D
3D3D
3D grid
gridgrid
grid
grid
gridgrid
grid board
board
board
boardboard
board
board
board (i.e.,
(i.e.,
(i.e.,
(i.e.,(i.e.,
(i.e.,
(i.e.,
(i.e., most
most
mostmost
most most
most
most ofof
ofof
of of
of
the
of the
the the
the
the the
the deviations
deviations
deviations
deviations
deviations
deviations
deviations
deviations areare
are
are
areare
are small).
small).
small).
are small).
small).
small).
small).
small). For
For
ForFor
For
For For
For example,
example,example,
example,
example,
example,
example,
example, ininin
inin
ininin
theM13D
M1M1
M1
M1 M1grid
(the
M1 (the
(the
(the
(the (the board
first
(the first
first
first
first
first row
first
row (i.e.,
row
row
row row
row
in in
inin most
in Table
inin
Table
Table of
Table
Table
Table
Table 8),
8), the
8),
8),
8), we
8),
8),
we
we deviations
we
weweobserve
we observe
observe
observe
observeobserve
observe are
that
that
that
that
that
that small).
there
that
therethere
therethere
there
there are For
are
are
are
are are
areexample,
more
more
more
more
more
more more data
data data
data
data
data in
data M1
points
points
points
pointspoints
points
points (the
with
with
with
with
with first
with highly
withhighly
highly
highly row
highly
highly
highlyin
positive Table
positive
positive
positive
positive
positive positive 8),
(blue)
(blue)
(blue)
(blue)
(blue)
(blue) (blue)
M1M1M1
M1
M1M1
M1 (the
(the
(the
(the (the
(the
(the first
first
first
first first
first
first row
row row
row
rowrow
row ininin in
inin Table
Table
Table
in Table
Table
Table
Table 8),
8), 8),
8), 8),
8),
we
we
8), we
we
wewe
we observe
observe
observe
observe
observe
observe
observe that
that
that
that that
that
that there
there
there there
there
there
there are
are
are
are are
are
are more
more
more
more more
more
more data
data
data
data data
data
data points
points
points
points
points
points
points with
with with
with
with
with
with highly
highly
highly
highly
highly
highly
highly positive
positive
positive
positive positive
positive
positive (blue)
(blue) (blue)
(blue)
(blue)
(blue)
(blue)
weandobserve
and
and
and
and
and negative
and
negative that
negative
negative
negative
negative
negative there
(red)
(red)
(red)
(red)
(red)
(red) (red)are more
deviations
deviations
deviations
deviations
deviations
deviations
deviations data points
compared
compared
compared
compared
compared
compared
compared with
to
to to
to
toM4M4
toto
M4M4highly
M4
M4 (the
M4 (the
(the
(the
(the
(the (thepositive
last
last
last
last
last row),
last row),
last row),
row),
row),
row), row),(blue)
where
wherewhere
wherewhereand
where
where there
there
there
there negative
there
there are
there are
are
are
are more
are
are
more
more
more (red)
more
more
moredatadeviations
data
data
data
data
data data points
points points
pointspoints
points
points with
with with
with
with with
with
and
and
and and
and
and negative
negative
negative
negative
negative
negative (red)
(red)
(red) (red)
(red)
(red) deviations
deviations
deviations
deviations
deviations
deviations compared
compared
compared
compared
compared
compared to
to
to to
to
M4 to
M4M4M4
M4M4 (the
(the
(the (the
(the
(the last
last
last last
last
lastrow),row),
row),
row),
row),
row), where
where where
where
where
where there
there
there there
there
there are
are
are
are are
aremore
more
more
more more
more data
data
data
data data
data points
points points
points
points
points with
with
with
with with
with
compared
small
small
small
small
small
small
small to M4
deviations
deviations
deviations
deviations
deviations
deviations
deviations (the last
(green).
(green).
(green). row),
(green).
(green).(green).
(green). where there are more data points with small deviations (green).
small
small
smallsmall
small
small
small
small
small deviations
deviations
deviations
deviations
deviations
deviations
deviations
deviations
deviations (green).
(green).
(green). (green).
(green).
(green).
(green).
(green).
(green).
Table
Table
Table
Table
Table
Table 8.8.Histogram
Table
Table
Table8.
8.
Table
Table
8.8. Histogram
8.
8. Histogram
8.Histogram
Histogram
Histogram
Histogram
8. Histogram
8. Histogram
Histogram
8. Histogram of
ofof
of
of deviations
of
of deviations
ofdeviations
deviations
deviations
deviations
deviations
of deviations
of deviations
deviations
of deviations for
for
for
for for
for
forthe
for
for
the
the
for
for
the the
the the
the measured
measured
themeasured
measured
measured
measured
measured
the
the measured
measured
measured
measured data.
data.
data.
data. The
data.The
data.
data.
data. The
The
The
data.
data.
data.
The normal
The
Thenormal
normal
normal
Thenormal
The
The to
normal to
normal
normal to
to
normal
normal
normal
to the
to
to
thethe
to
the
toto
the
to the
the tag
the
tag
the tag
the
the
tag tag
tag was
tag
waswas
tag
was
tag
tag
tag
waswas
was parallel
was
parallel
wasparallel
was
was with
parallel
parallelwith
parallel
parallel
with
with
parallel
parallel
parallel
parallel
with the
with
with
with
the
with the
the
with
with
the the
thethe
the
the
the
Table
Table
Table
Table
Table 8.
Table 8.Histogram
8. Histogram
8.8.Histogram
Histogram
Histogram
Histogram of
of
of ofdeviations
ofdeviations
deviations
deviations
deviations
of deviations for
for
for for
for the
for
the
the the
the measured
themeasured
measured
measured
measured
measured data.
data.
data. The
data.
data.
The
The
data. Thenormal
The normal
normal
normal
normal
The to
normal to
to the
to
to
the
the
to the
thetag
tag
tag
the was
tag
tag
was
was
tag was
wasparallel
parallel
parallel
was with
parallel
parallelwith
with
with
parallel the
with
the
the
with the
the
the
camera
camera
camera
camera
camera
camera
cameraZ-axis.
camera
camera
camera Z-axis.
Z-axis.
Z-axis.
camera Z-axis.
Z-axis.
Z-axis.The
Z-axis.
Z-axis.
Z-axis. The
The
The
The
Z-axis.
TheThebars
The
Thebars
bars
The
The bars
bars further
bars
bars
bars
bars further
further
bars
bars furtherfrom
further
further
further
further from
from
further
further
further
from from
from
from fromfrom
from
from the
the
the
the the
the
the center
the
the
center
the
the
center of
center
center
center
center
center ofof
of
center the
of
of
the
center
center
of the
of
the
the
ofof
the
of bar
the
the
bar
the bar
bar
the
the
bar graph
bar
bargraph
bar
graph
graph
bar
bar
bar
graph represent
graph
graph
graph represent
represent
represent
graphrepresent
graph
graph represent the
represent
represent the
the
represent
represent
represent
the the
the count
the
the
count
the count
count
the
the
count of
count
count
countof
count of
of
count
count
of measurements
of
ofmeasurements
ofmeasurements
measurements
measurements
measurements
measurements
of measurements
of measurements
measurements
of measurements
camera
camera
camera
camera
camera
camera Z-axis.
Z-axis.
Z-axis.
Z-axis.Z-axis.
Z-axis. The
The
The
The Thebars
Thebars
bars
bars
bars further
bars further
furtherfrom
further
further
further from
from
from from
from the
the
the the
the center
thecenter
center of
centerof
center
center of
of the
ofthe
of
the
the thebar
thebar
bar
bar graph
bargraph
bargraphrepresent
graph
graph
graph represent
represent
represent
represent
represent the
the
the the
the count
thecount
count
count
count of
count
of
of ofmeasurements
ofmeasurements
ofmeasurements
measurements
measurements
measurements
with
with
with
with
with more
with
with
with
with more
more
more
more
more
more
with
with
more moresignificant
significant
significant
more
more significant
significant deviations.
significant
significant
significant deviations.
deviations.
deviations.
deviations.
deviations.
deviations.
deviations.
significant
significant
significant deviations.
deviations.
deviations.
with
with
withwith
with more
more
more
more more significant
significant
significant
significant
significant deviations.
deviations.
deviations.
deviations.
deviations.
Roll
Roll
Roll
Roll
Roll Pitch
Pitch
Pitch
Pitch
Pitch Yaw
Yaw
Yaw
Yaw Yaw XXX
XXX YYY
YYY ZZZ
ZZZ
Roll
Roll
Roll
Roll
Roll
Roll
Roll
Roll
Roll
Roll
Pitch
Pitch
Pitch
Pitch
Pitch
Pitch
Pitch
Pitch
Pitch
Pitch
Yaw
Yaw
Yaw
YawYaw
YawYaw
Yaw
Yaw
Yaw
XXX
XXXXXX YYY
YYYYYY ZZZ
ZZZZZZ
 expected
expected
expected because
becausebecause the
the tags
the
tags are
tags
are more
are
more distant
more from
distant
distant from the
from
the camera.
the
camera. The
camera.The histograms
The histograms
histograms inin Table 8Table
inTable confirm
inTable 8confirm the
8confirm the
expected
expected
expected
expected
expected
expected
expected because
because
expected
expected because
because
because
because
because
the
because
because the
the
the the
tags
the
the
tags
tags
the
the
tags tags
are
tags
tags
areare
are
tags are
more
tags
are
are
more
more
are
are
more moremore
more
moredistant
more
distant
distant
distant
distant
distant
distant from
distant
distant from
from
from from
from
from
from
the the
the
from
the the
the thethe
camera.
camera.
the
camera.
camera.
camera.
camera.
camera. The
camera.
camera. The
The
The The
TheTheThe
histograms
Thehistograms
histograms
histograms
histograms inin
histograms
histograms
histograms
histograms inTable
inTable
in
Table
in
inTable
8confirm
inTable
confirm
888confirm
8Table
Table
Table 8confirm
8confirm
8confirm
8confirm
the
confirm
the
confirmthe
the
the the
the thethe
the
fact
fact
factthat
fact
that
fact
fact fact
that the
that
that
that the
that
the
the large
the
thelarge
the
large
large
large deviations
large deviations
deviations
large deviations
deviations
deviations
deviations from
from
from
from
from real
from
real
from
real
real pose
real
realpose
real
pose
posepose estimates
pose
pose estimates
estimates have
have
estimates
estimates
estimates
estimates have
have been
have
been
have
have been
been
been significantly
been
been significantly
significantly reduced
reduced
significantly
significantly
significantly
significantly reduced
reduced in
in
reduced
reduced
reduced in inin in
fact
fact fact
that
factfact
that that
the
that that
the the
large
the the
large large
largedeviations
large deviations
deviations from
deviations
deviations from from
fromreal
from real
pose
real
real real
pose posepose
estimates
pose estimates
estimates have
estimates
estimates have have
been
havehave been
been
been been significantly
significantly reduced
significantly
significantly
significantly reduced reduced inininininin
reduced
reduced
measurements
measurements
measurements
measurements
measurements
measurements
measurements done
done
done
done donewith
done
with
done
with the
with
with
with the
with
the
the 3D
the
3D
the3D
3D grid
the
3D3D
grid3D
gridboard
grid
board
grid
grid grid
board
board
board (i.e.,
board
(i.e.,
board(i.e.,most
(i.e.,
most
(i.e.,
(i.e.,
(i.e., most
mostmostof
most
of the
most
oftheof
the
of deviations
ofthe
the thedeviations
deviations are
are
deviations
deviations
deviations small).
are are
are are For
small).
small). For
small).
small).
small). Forexample,
For
example,
For For example,
example, in
in
example,
example, in in in
ininin
measurements
measurements
measurements
measurements
measurements done
done donedone
with
done
with with
the
with with
the 3D
thethe
3D grid
the
3D 3D3D
grid grid
board
grid grid board
board
board (i.e.,
board (i.e.,
most
(i.e.,
(i.e., (i.e.,
most most ofof
most
ofmost
the
ofthe
of
the of
the deviations
the
the deviations
deviations are
deviations
deviations
deviations are
are are small).
are
small).
are small).
small). For For
small).
small).ForFor example,
For
example,
For
example,example,ininin
example,
example, in
M1
M1M1
M1 (the
M1M1
(the
M1 M1
(thefirst
(the
first
(the
(the (the
firstrow
first
row
first
first
(the first
row
row
first rowin
row
inrow
row in
inTable
in inTable
Table
Table
Table
in 8),
inTable
8),
Tablewe
8),
8), we8),
we
8),
we observe
wewe
observe
8), we
observe
observe
observe that
observe
that
observethatthere
that
that
that there
that
there are
there
there
there are
there
are
are more
are
more
are are
more data
more
more
more data
more
data
data points
data
points
data
data points with
points
with
points
points
points withhighly
with
highly
with
with with
highly positive
highly
highly
highly
highly positive
positive
positive (blue)
(blue)
positive
positive
positive (blue)
(blue) (blue)
(blue)
(blue)
M1M1 (the
M1 M1
(the first
(the (the
firstrow
firstfirst
row in
row in Table
rowin
TableinTable
8),
TableTable
8),we 8),
8),
we 8),
we we
observe
we
observeobserve
that
observe
observe thatthatthat
therethat
there there
are
there there
are are
more
are are
more more
data
moremore data
points
data
data data points
points with
points
points with with
highly
with with highly
highly positive
highly
highly positive
positive (blue)
positive
positive (blue)
(blue)
(blue) (blue)
and
and
and
and negative
and
and negative
negative
and
negative
negative
negative (red)
(red)
negative (red)
(red)
(red) deviations
(red)
(red) deviations
deviations compared
deviations
deviations
deviations
deviations compared
compared
compared to
to
compared
compared
compared M4
toM4
toto
M4 (the
to
M4 M4
(theM4
(the last
(the
last
(the (therow),
lastlast
row),
last last
row), where
row),
where
row),
row), where there
where
there
where
where there are
there
are
there more
there
are are
more
are are
more data
more
moredata
more
datapoints
data
points
data data
points
points with
points
with
points
with with
with with
and
and and
negative
and
Sensors and negative
negative
2020, 20, (red)
negative
negative (red)
4825 (red)
deviations
(red) (red) deviations
deviations compared
deviations
deviations compared
compared tototo
compared
compared M4to
M4M4
toM4
(theM4(the
to(the
M4 (the
last
(the last
(the
last last row),
last
row),
lastrow),
where
row),
row), where
row), where
where
where there
there
where there
are
there
there are
more
there
areare more
are
are
more more
data
more data
more data
data
data points
points
data points
points
points with
with
points
14 with
of with
with
16 with
small
small
small deviations
small
small
small
small deviations
deviations
small
deviations
deviations
deviations
deviations (green).
deviations (green). (green).
(green). (green).
(green).
(green).
(green).
small
small deviations
small
small deviations
deviations
deviations (green).
(green). (green).
(green).
Table
Table 8.8.
Table Histogram
8.Histogram
8.Histogram
8.Histogram ofof
8.Histogram
Histogram of deviations
ofdeviations for
ofdeviations
deviations for the
the formeasured
the
measured data.
measured
data. The
data.
The normal
The
normal normal the
to to
the totag
tag the was
was tag parallel
was
parallel with
parallel
with the
with
the the
ofof deviations for the measured data. The normal toto the tag was parallel with the
Table
Table Table
Table Histogram
Histogram of deviations
deviations for for
the the measured
measured data. data.
The The
normalnormal
to the to the
tag tag
was was parallel
parallel with with
the the
Table
Table Table
8.Table
Table
Table 8.8.
8.8.Histogram
Histogram
8.8.Histogram
Histogram
Histogram of
deviations
of ofofdeviations
ofdeviations for
deviations
deviations
deviations for
forthefor
for the
for
the the
for
thethe measured
measured
measured
themeasured
measured data.
measured
measured data.data.
data.
data.
The
data.The The
The The
normal
data.
The normal
normal
The
normalnormal
tothe
normal
normal to
tothe
theto the
tagto
the tag
the
tag tag
was
the
tag was
tag
was was parallel
parallel
was
parallel
tag
was was
parallel with with
with
parallel the
parallel
parallel with
with the
with the
with
the the
the the
camera
camera
camera
camera Z-axis.
Z-axis.
camera
camera
camera Z-axis.
camera
Z-axis. The
Z-axis.
The
Z-axis.
Z-axis. Thebars
Z-axis.The
bars
The
The Thefurther
barsbars
further
bars
bars bars
further from
further
further
further from
further
from the
from
from
from the center
from
the
the the
center
the the
centerof
center
of the
the
center
center
center of bar
of
the
of
ofofthe the
bar
of
the graph
bar
graph
the
bar
barbarbar
graph represent
graph
graph
graph graph the
represent
represent the
represent
represent
represent
represent the
thecount
thethe
countthe
count of measurements
measurements
count
of
count
count
count of of
ofof of measurements
measurements
of measurements
measurements
measurements
measurements
camera camera
camera Z-axis.
camera
camera Z-axis.Z-axis.
The
The The
bars
Z-axis.
Z-axis. The The bars
further
bars
bars barsfurther
further from
further
further from from
the
from from
the thethe
centerthe
centercenter
ofofthe
center
center theof
bar
of
thethe
barthe
barbar
graphbargraph
graph
graph graph represent
represent the
represent
represent
represent the thethe
count the count
ofcount
count
count ofofmeasurements
ofmeasurements
measurements
measurements
measurements
with
with
with more
more
with
withmore
with
with with
more significant
significant
more
more
more more deviations.
deviations.
significant
significant
significant
significant
significant
significant deviations.
deviations.
deviations.
deviations.
deviations.
deviations.
with
with with
more
withwith
more more
more more significant
significantsignificant
significant
significant deviations.
deviations. deviations.
deviations.
deviations.
Roll
RollRoll Pitch
PitchPitch Yaw
Yaw Yaw XXX XX YY YY ZZZZZ ZZ
Roll
Roll
Roll
Roll
Roll
RollRoll
Roll
Roll
Roll Pitch
Pitch
Pitch
Pitch
Pitch
Pitch
Pitch
Pitch
Pitch
Pitch Yaw
Yaw
Yaw
Yaw
Yaw Yaw
Yaw
Yaw
Yaw Yaw XX
XXXXXX YYY
YYYYYY ZZZ
ZZ Z

M0
M0
M0M0
M0
M0M0M0
M0
M0M0
M0M0

M1
M1M1
M1 M1M1
M1 M1
M1
Sensors
M1 M1 M120, x FOR PEER REVIEW
2020,
M1 13 of 15
Sensors
Sensors
Sensors
Sensors
Sensors
Sensors 2020,
2020,
2020,
Sensors
Sensors 2020,
2020,
2020,
20,
20,20,
2020,
2020, 20,
xFOR
FOR 20,
20,
xx20,FOR
xx20,
FOR xx
xxxPEER
FOR
xxFOR FOR
FOR
PEER
PEER
FORPEER PEER
PEER
PEER REVIEW
REVIEW
REVIEW
REVIEW
REVIEW
REVIEW
PEER REVIEW 13
131313
of 13
of13
of
15
13 13
of
15
15
of of
of15
15 15
1515
SensorsSensors
SensorsSensors
2020,
Sensors
Sensors
Sensors 2020,
Sensors 2020,
20,
2020,
2020,
2020,
Sensors 20,
2020,
20,
20,
2020, x20,
2020,
x20,
20, 20,
xFOR
FOR
FOR
20,
FOR
x FOR
xFOR PEER
FOR
PEER
FORPEER
PEER
PEER REVIEW
PEER
REVIEW
PEER REVIEW
REVIEW
PEER REVIEW
REVIEW
REVIEW
REVIEW
REVIEW
PEER REVIEW 13 of
13
13 13
15
13of
of
13 13
of
of of
13
15
13
15 of
15
15
of 15
of
15 15
of 15

M2
M2
M2
M2
M2 M2
M2
M2
M2
M2
M2 M2
M2
M2
M2M2
M2
M2

M3
M3
M3
M3
M3 M3
M3
M3
M3
M3
M3 M3
M3
M3
M3
M3
M3
M3

M4
M4
M4
M4
M4 M4
M4
M4
M4
M4
M4 M4
M4
M4
M4
M4
M4
M4

Figure 12 shows the percentage of small deviations ±5% of the full range (the two center bars
fromFigure Figure
theFigure
Figure barFigure
Figure
Figure
Figure
Figure
Figure
Figure
Figure
Figure 12
12
Figure
12
Figure12
12
1212
12graphs12
12
shows 12
shows
shows
shows
12
12 12
shows
shows
12
shows shows
shows
shows
shows
shows
shown
shows the
the
shows
the
shows thethe
the
the
the the
thethe
the
the percentage
percentage
percentage
percentage
percentage
percentage
percentage
percentage
inpercentage
the
percentage Table
percentage 8).of
of
percentage
percentage
percentage
the of
percentage
percentage of
ofof
ofofof
The
of
small of
small
small
small
of
of of small
small
small
small
small
histograms
small
small
small
of
small small deviations
deviations
deviations
deviations
deviations
deviations
deviations
deviations
small ±5%
deviations
deviationsshow
deviations
deviations
deviations ±5%
deviations
deviations ±5%
±5%
±5%
±5%
±5% ±5%
±5%
of
±5%
±5%
of ±5%
of
an
±5% of
±5%
the
of
of
±5%
of of
the of
the
the
of
of of
the
full
the
the
increase
ofof
the
the
theof the
the
full
the
full full
full
range
thefull
full
range
full
full
the
full full
full
range
range
in
full
full range
range
range
(the
range
range
small
range
range
range
full
range (the
range
(the
range (the
(the
(the
(the
(the two
(the
(the(the
(the
two
two two
deviations
(the
two (the
two
two
(the
two two
two
twotwo
center center
center
center
center
center
center
two
two center
center
center
center
two center
bars bars
asbars
bars
bars
center bars
bars
abars
bars
bars
center
center bars bars
bars
bars
bars
bars
Figure 12 shows the of small deviations ±5% of the full range (the two center
fromfrom
from
from from
from
thethe
the
Figure
from the
bar
the the
the
bar
bar
12 barbar
bar
graphs
graphs
graphs
shows
bar graphs
graphs
graphs
the
graphs shown
shown
shown shown
shown
shown
in
percentage
shown in
in in
Table in
in
Table
Table
in ofTable
Table
Table
8).
8).8).
small
Table 8).
The 8).
8).
The
The
8). The The
The histograms
histograms
histograms
histograms
histograms
histograms
deviations
The ±5%
histogramsshowshow
show
of show
theshow
show
an
show an
an
fullananan
increase
increase
an range increase
increase
increase
increase
increase inin
in
(the in
smallinin
smallsmall
small
two
in small
small
center
small deviations
deviations
deviations
deviations
deviations
deviations bars
deviations asas
fromas
a as
a
aas
as as
a aa
larger
from from
from
from
from from
the
from from
fromthe
the
the the
fraction
bar
thethe
the
bar
the
bar
bar bar
of
bar
bar
graphs graphs
all the
graphs
graphs
bar
graphs
graphs
bar graphs
shown
graphs
graphs shown
deviations,
shown
shown
shown shown
shown
inin
shown
shown in
Table
inin
in Table
which
ininTable
Table
in
Table
Table Table
8).
Table
Table 8).
The
8).
8).
8). 8).
The
8).
The
The
8). The
corresponds
8).The
The Thehistograms
to ashow
histograms
histograms
The decrease
histograms
histograms
histograms
histograms
histograms
histograms show
show
show show
show
show show
an
show
an an
an an
in
an an
increase increase
the
an number
increase
increase
an increase
inin
increase
increase
increase
increase in
small
inin
in of
in small
in
small
in
small
small large
small
small deviations
deviations
small
small deviations
deviations
deviations
deviations
deviations
deviations
deviations asas
deviations as
aas
aasas
as aaaaaa
as
aaas
larger
thelarger
larger
largerbar larger
larger
larger
larger
larger
observed.larger
larger
larger fraction
fraction
graphs
larger
fraction fraction
fraction
fraction
fraction of
fraction
fraction
fraction
fraction
larger of
of
fraction
of allof
all
shown of
all
all
of
of
ofof
fraction
fraction of of
all
the
in
the
all
all all
all
ofof
of all
the
allthe
all
the the
thethe
the
the
all deviations,
deviations,
deviations,
deviations,
deviations,
deviations,
Table 8). The
deviations,
deviations,
thedeviations,
deviations,
the
all
the deviations,
deviations,
the which
which
which
which
deviations,
deviations, which
which
which
which
which
which
which
which which corresponds
corresponds
corresponds
corresponds
corresponds
corresponds
histograms show an
corresponds
corresponds to
corresponds
corresponds
which to
corresponds
corresponds toto
corresponds
corresponds to
a
atoto
a
a
to
ato to
aatoto
a a a
decrease
decrease
decrease
increase decrease
decrease
decrease
in
aaadecrease
to
todecrease decrease
decrease in
small
aadecrease
decrease
to decrease
inin
decrease in
in
the
the in
inin in
the
the
in
ininin
the
the the
the
in the
the
number
number
number
deviations
the
number
in number
number
number
number
number
thenumber
number
the of
ofas
number
number of
of of
large
of of
of of
of
largelarge
large
large
aoflarger
of
large large
large
large
large
ofof large deviations
deviations
deviations
deviations
deviations
deviations
fraction
largedeviations
deviations
deviations
deviations deviations
deviations
large deviations
larger
larger fraction
fraction of all the
all deviations,
the deviations, which
which corresponds
corresponds to adecrease
decrease in the
the the number
number
number ofof large
largelarge deviations
deviations
deviations
observed.
all
observed. observed.
observed.
observed.
observed.
observed.
ofobserved.
the
observed.
observed. deviations,
observed.
observed.
observed.
observed.
observed. which corresponds to a decrease in the number of large deviations observed.
observed.

Figure 12.
Figure Improvement in
12. Improvement in pose
pose estimation
estimation across
across different
different grid
grid boards
boards and
and camera
camera calibrations.
calibrations.
Figure
Figure
Figure
Figure
Figure
Figure 12.
12.
Figure
Figure
FigureFigure
Figure 12.
Figure
12.
Figure 12.
12. 12.
12.
12. Improvement
Improvement
Improvement
Improvement
Improvement
Improvement
12. Improvement
12.
Improvement
12.
12. Improvementin
inin
Improvement in
Improvement
Improvement
Improvement in pose
in in
pose
posepose
in
inin
inpose in
pose
inin pose
pose
pose
pose
in estimation
estimation
estimation
estimation
estimation
estimation
pose estimation
pose across
estimation
estimation
pose
pose estimation
estimation
estimation across across
across
across
across
across
estimation across
across
across
across different
different
different
different
different
different grid
grid
grid
different
different
different
different
across
across different
grid
different
different grid
grid
grid
boards
grid
grid
grid
boards
grid
grid boards
boards
boards
boards
boards and
and
boards
boards
boards
grid boards
and
boards
boards and
and
and
and and
and and
and
camera
and
and
camera camera
camera
camera
camera
camera camera
camera
camera
camera
camera camera calibrations.
calibrations.
calibrations.
calibrations.
calibrations.
calibrations.
calibrations.
calibrations.
calibrations.
calibrations.
calibrations.
calibrations.
calibrations.
FigureFigure
Figure 12.
12.
Figure 12. Improvement
Improvement
Improvement
12. Improvement in inpose
posepose estimation
estimation
estimation
estimationacrossacross
across different
different
different
across grid
grid
different grid
boards
grid boards
boards and
and
boards and
camera
and camera
camera
camera calibrations.
calibrations.
calibrations.
calibrations.
When recording the frequency of deviations, Figure 12 confirms that compared to M0, the
accuracy
WhenWhen
WhenWhen
When
When
When When
When
When
When
When
of the
When
When
recording
When poserecording
recording
recording
recording
recording
recording
recording
recording
recording the
thethe
estimate
recording
recording
the
recording
recording
recording the
the
the
the the
the the
the
the of
the
frequency frequency
frequency
frequency
frequency
frequency
frequency
frequency
frequency
M4 of
frequency
frequency
the of
of
ofof of
increased
frequency
frequency
frequency
frequency of
of of
ofof
of
deviations,deviations,
deviations,
deviations,
deviations,
deviations,
deviations,
of
of during Figure
Figure
deviations,
deviations,
deviations,
deviations,
of
deviations,
deviations, Roll
deviations,
Figure Figure
Figure
Figure
Figure
Figure
deviations,
Figure
Figure by12
Figure
Figure
Figure12
1212 12
77%,
Figure
12 12
Figure
12 1212
1212
confirms confirms
confirms
confirms
confirms
confirms
confirms
12
12 confirms
confirms
Pitch
confirms
confirms
12 that
by
confirms
that
confirms
confirms
confirms that
that
that
that
that that
that
that
that
3%,
that Yaw
that
compared
that compared
compared
compared
compared
compared
compared
compared
compared
compared
that by
compared
to
compared
compared
compared to
to
toto
42%,
compared to
M0,
M0,
to
to to
toto
M0,
M0,
to
to M0,
M0,
to
M0,
M0, and
to M0,
M0,
the
the
M0,
M0,
M0,
the the
M0, the
the
M0,
the
the the
thethe
the
thethe
the
When recording the frequency of deviations, Figure 12 confirms that compared to M0, the
along
accuracy accuracy
accuracy
accuracy
accuracy
accuracy
accuracyaccuracy
accuracy
X
accuracy by
accuracyof
of
accuracy
accuracy
accuracy of
of
50%,
accuracy
oftheof
the
ofof
of of
the
ofof
the
of
of the
pose
the
of
the
the the
the
Y
of
pose the
the
by
the pose
pose
pose
the
pose
the
pose
pose pose
pose
pose
pose
45%
pose
pose
estimate andestimate
estimate
estimate
estimate
estimate
estimateestimate
estimate of
estimate
estimate
pose Z of
estimate
of
estimate
estimate
estimate of
M4
by
M4 of
M4
of
of
ofof
of
of of
of
M4 M4
9%.
ofof
M4
M4
M4 ofM4
M4
M4
M4M4
M4
increased increased
increased
increased
increased
increased
increased
increased
increased
increased
increased
increased
M4 during
increased
increased
increased during
during
during
during
during
during
during
duringRoll
duringRoll
Roll
during
Roll
during
during
during
during byRoll
Roll
Roll
by
by
Roll
Roll
Roll
by
Roll
Roll
Roll by
Roll
by
Roll
by
by by
77%,by
by
77%,
77%,
77%,
by
by by 77%,
77%,
77%,
Pitch
77%,
77%,
77%,
77%,
by
77%,
77%, Pitch
Pitch
77%,
Pitch Pitch
Pitch
Pitch
by
Pitch
Pitch by
by
Pitch
by
Pitch
77%,
Pitch
Pitch Pitch
by by
3%,
3%,
by
Pitch
by by
3%,
byby
3%,
by
by 3%,
by
3%,
by
3%,
3%, 3%,
3%,
3%,
Yaw Yaw
Yaw
Yaw
Yaw
3%,
3%,3%, Yaw
Yaw
by
Yaw
Yaw
Yaw
3%,
Yaw
Yaw by
Yaw
by by
Yaw
byby
Yaw
bybyby
by
by
42%, by
42%,
42%,
by 42%,
42%,
42%,
by
42%,
by
42%,
42%, 42%,
42%,
and
and
42%,
42%,and
42%,
and and
42%,
and
and and
and
and and
and
and
and
and
accuracy the pose estimate M4 increased during Roll by 77%, Pitch by 3%, Yaw by 42%, and
along
along
along along
along
along
X
along
along XX
byX
byX
by
X X
by
50%,byby
50%,
50%,
by 50%,50%,
50%,
YY
Y50%,
50%, by Y
byY
by
Y Y
by
45% by
45%by
45%
by 45%
45%
and
45% 45%
and
andand
Z and
and
Z
andZ
by Z
byZ
by
Z Z
by by
by
9%.
9%.9%.
by 9%.
9%.9%.
9%.
along along
along
along
along Xalong
by
along
along XX XXX
by
by by
50%,
Xby X
byby
X 50%,
by
50%,
by
50%,
50%, 50%,
Y
50%, 50%,
by
Y YY
Yby
Y Y
45%
by
by Yby
Yby 45%
by
45%
by
45%
45%
by 45%
and
45% 45%
45% and
Z
and
and
andandand
Zand
by ZZ
Zby
and
Z Z
by
byby
9%.
ZZby
9%.
by
9%.
9%.
by 9%.
by9%.
9%. 9%.
9%.
9. Conclusions
9. 9.
9.9.9. 9.
9.
9.9. Conclusions
Conclusions
9.9.
Conclusions
Conclusions
Conclusions
Conclusions
9. Conclusions
Conclusions
Conclusions
9.
Conclusions Conclusions
9.Conclusions
9. Conclusions
Conclusions
Conclusions
9. Conclusions
This paper presented a new way of using Aruco tags to detect the pose of objects in 3D space.
Sensors 2020, 20, 4825 15 of 16

When recording the frequency of deviations, Figure 12 confirms that compared to M0, the accuracy
of the pose estimate of M4 increased during Roll by 77%, Pitch by 3%, Yaw by 42%, and along X by
50%, Y by 45% and Z by 9%.

9. Conclusions
This paper presented a new way of using Aruco tags to detect the pose of objects in 3D space.
To test the pose estimation accuracy and compare different tags’ performance, a standard benchmark
was created that tested the error in pose estimates across all parameters. This benchmark was
applied over a 2D grid placement of tags, and the observations were used to determine that a 3D
organization of multiple tags performs better than a 2D placement of tags. Different visualizations
were created to demonstrate the observations appropriately. All experiments were done using Python,
where they were first simulated in a simulation environment and then replicated in the real world.
UR3 collaborative robot was used to ensure that the measurements were consistent and multiple Intel
Realsense D435 cameras were used interchangeably to avoid errors coming from any particular camera.
A Python library [21] was written that allowed the implementation and benchmarking of arbitrary 3D
organization of Aruco tags and an error metric was defined that allowed for the estimation of the error
in the pose estimates. The result is a substantial improvement in pose estimates when compared with
a combination of Aruco tags on a 2D surface. From the benchmark process, it was observed that the
3D grid board led to an average improvement of 40% across all detected parameters related to the
pose. We are fully aware that the better calibration of the cameras could lead to better results, and this
was also proved in this paper. Regardless of the type of calibration, we acquire better pose estimation
with the proposed 3D placement strategy than with the traditional 2D grid. Further work on this topic
could be done to determine the best relative orientation between tags themselves for an even better
estimate of the pose. More positions and orientations can be added to the benchmark process to make
it more robust.

Author Contributions: Conceptualization, Z.B. and P.O.; methodology, P.O. and Z.B.; software, P.O., Z.B. and
D.H.; validation, A.V. and I.V.; formal analysis, M.S. and D.H.; investigation, J.M., M.S. and I.V.; resources, M.S.,
D.H. and P.N.; data curation, P.O., J.M. and A.V.; writing—original draft preparation, A.V.; writing—review
and editing, J.M. and I.V.; visualization, P.O. and J.M.; supervision, Z.B.; project administration, P.N.; funding
acquisition, P.N. All authors have read and agreed to the published version of the manuscript.
Funding: This work was supported by the Research Platform focused on Industry 4.0 and Robotics in Ostrava
Agglomeration project, project number CZ.02.1.01/0.0/0.0/17_049/0008425 within the Operational Programme
Research, Development and Education and VEGA 1/0389/18 Research on kinematically redundant mechanisms.
This article has been also supported by specific research project SP2020/141 and financed by the state budget of the
Czech Republic.
Conflicts of Interest: The authors declare no conflict of interest.

References
1. Garrido-Jurado, S.; Muñoz-Salinas, R.; Madrid-Cuevas, F.J.; Marín-Jiménez, M.J. Automatic generation and
detection of highly reliable fiducial markers under occlusion. Pattern Recogn. 2014. [CrossRef]
2. Zhong, X.; Zhou, Y.; Liu, H. Design and recognition of artificial landmarks for reliable indoor self-localization
of mobile robots. Int. J. Adv. Robot. Syst. 2017. [CrossRef]
3. Chavez, A.G.; Mueller, C.A.; Doernbach, T.; Birk, A. Underwater navigation using visual markers in the
context of intervention missions. Int. J. Adv. Robot. Syst. 2019. [CrossRef]
4. Marut, A.; Wojtowicz, K.; Falkowski, K. ArUco markers pose estimation in UAV landing aid system.
In Proceedings of the 2019 IEEE 5th International Workshop on Metrology for AeroSpace (MetroAeroSpace),
Torino, Italy, 19 June 2019; pp. 261–266. [CrossRef]
5. Zheng, J.; Bi, S.; Cao, B.; Yang, D. Visual localization of inspection robot using extended kalman filter and
aruco markers. In Proceedings of the 2018 IEEE International Conference on Robotics and Biomimetics
(ROBIO), Kuala Lumpur, Malaysia, 12 December 2018; pp. 742–747. [CrossRef]
Sensors 2020, 20, 4825 16 of 16

6. Zhu, Q.; Li, Y. A practical estimating method of camera’s focal length and extrinsic parameters from a planar
calibration image. In Proceedings of the 2012 Second International Conference on Intelligent System Design
and Engineering Application, Sanya, Hainan, 6 January 2012; pp. 138–142. [CrossRef]
7. Fetić, A.; Jurić, D.; Osmanković, D. The procedure of a camera calibration using Camera Calibration
Toolbox for MATLAB. In Proceedings of the 2012 35th International Convention MIPRO, Opatija, Croatia,
21 May 2012; pp. 1752–1757.
8. Wang, X.; Zhao, Y.; Yang, F. Camera calibration method based on Pascal’s theorem. Int. J. Adv. Robot. Syst. 2019.
[CrossRef]
9. Ramírez-Hernández, L.R.; Rodríguez-Quiñonez, J.C.; Castro-Toscano, M.J.; Hernández-Balbuena, D.;
Flores-Fuentes, W.; Rascón-Carmona, R.; Lindner, L.; Sergiyenko, O. Improve three-dimensional point
localization accuracy in stereo vision systems using a novel camera calibration method. Int. J.
Adv. Robot. Syst. 2020. [CrossRef]
10. Bushnevskiy, A.; Sorgi, L.; Rosenhahn, B. Multimode camera calibration. In Proceedings of the 2016 IEEE
International Conference on Image Processing (ICIP), Phoenix, AZ, USA, 25 September 2016; pp. 1165–1169.
[CrossRef]
11. Schmidt, A.; Kasiński, A.; Kraft, M.; Fularz, M.; Domagała, Z. Calibration of the multi-camera registration
system for visual navigation benchmarking. Int. J. Adv. Robot. Syst. 2014. [CrossRef]
12. Dong, Y.; Ye, X.; He, X. A novel camera calibration method combined with calibration toolbox and genetic
algorithm. In Proceedings of the 2016 IEEE 11th Conference on Industrial Electronics and Applications
(ICIEA), Hefei, China, 5 June 2016; pp. 1416–1420. [CrossRef]
13. Bradski, G. The OpenCV library. Dr. Dobbs J. Softw. Tools 2000, 120, 122–125.
14. Calibration Targets. Available online: https://calib.io (accessed on 20 August 2020).
15. Atcheson, B.; Heide, F.; Heidrich, W. CALTag: High precision fiducial markers for camera calibration.
In Proceedings of the Vision, Modeling, and Visualization Workshop 2010, Siegen, Germany,
15–17 November 2010; pp. 41–48.
16. Xing, Z.; Yu, J.; Ma, Y. A new calibration technique for multi-camera systems of limited overlapping
field-of-views. In Proceedings of the 2017 IEEE/RSJ International Conference on Intelligent Robots and
Systems (IROS), Vancouver, BC, Canada, 24 September 2017. [CrossRef]
17. Miseikis, J.; Glette, K.; Elle, O.J.; Torresen, J. Automatic calibration of a robot manipulator and multi 3D
camera system. In Proceedings of the 2016 IEEE/SICE International Symposium on System Integration (SII),
Sapporo, Japan, 13 December 2016; pp. 735–741. [CrossRef]
18. Sergio, G.; Nicholson, S. Detection of ArUco Markers. OpenCV: Open Source Computer Vision.
2020. Available online: https://docs.opencv.org/master/d5/dae/tutorial_aruco_detection.html (accessed
on 1 February 2020).
19. Camera Calibration with OpenCV. OpenCV: Open Source Computer Vision. 2020. Available online:
https://docs.opencv.org/2.4/doc/tutorials/calib3d/camera_calibration/camera_calibration.html (accessed on
1 February 2020).
20. Dubonnet, Olivier Roulet. “python-urx”. GitHub Repository. 2020. Available online: https://github.com/
SintefManufacturing/python-urx (accessed on 1 February 2020).
21. Oščádal, P.; Heczko, D.; Bobovský, Z.; Mlotek, J. “3D-Gridboard-Pose-Estimation”. GitHub Repository.
2020. Available online: https://github.com/robot-vsb-cz/3D-gridboard-pose-estimation (accessed on
1 February 2020).

© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access
article distributed under the terms and conditions of the Creative Commons Attribution
(CC BY) license (http://creativecommons.org/licenses/by/4.0/).