PROCEEDINGS OF SPIE

SPIEDigitalLibrary.org/conference-proceedings-of-spie

Automatic target tracking with time-

delayed measurements for
unmanned surface vehicles

Han, Jungwook, Son, Nam-sun, Kim, Jinwhan

Jungwook Han, Nam-sun Son, Jinwhan Kim, "Automatic target tracking with
time-delayed measurements for unmanned surface vehicles," Proc. SPIE
11197, SPIE Future Sensing Technologies, 111971B (12 November 2019);
doi: 10.1117/12.2547429

Event: SPIE Future Sensing Technologies, 2019, Tokyo, Japan

Downloaded From: https://www.spiedigitallibrary.org/conference-proceedings-of-spie on 07 May 2022 Terms of Use: https://www.spiedigitallibrary.org/terms-of-use

 Automatic target tracking with time-delayed measurements
for unmanned surface vehicles
Jungwook Hana , Nam-sun Sona , and Jinwhan Kimb
a
Korea Research Institute of Ships and Ocean Engineering, Daejeon, Korea
b
Korea Advanced Institute of Science and Technology, Daejeon, Korea

ABSTRACT
This paper presents the automatic target detection and tracking of marine obstacles for unmanned surface
vehicles (USVs). For practical applications with a USV, the automatic detection of surrounding obstacles is a
crucial capability, and marine radars have been commonly used to detect and estimate the motion of obstacles.
However, their tracking performance degrades when a target is approaching at a high relative velocity due to
their relatively low sampling rate. This study addresses the automatic target tracking of marine obstacles by
considering time-delayed measurements provided by a marine radar. The relative position information between
a USV and nearby obstacles is obtained using the radar sensor, and the obstacles’ motion including position,
course, and speed is estimated using an extended Kalman filter (EKF)-based tracking filter by compensating
the measurement delay. To validate the feasibility of the proposed method, a real-sea experiment was conducted
using a USV and the results are presented.
Keywords: Unmanned surface vehicles, marine radar, target tracking, time-delayed measurements

1. INTRODUCTION
During several decades, unmanned surface vehicles (USVs) have drawn a lot of attention for their practical abil-
ities to perform labor-intensive and risky missions such as ocean sampling and reconnaissance and surveillance.
To perform these missions using a USV, autonomous detection and motion estimation of surrounding obstacles
are crucial capabilities.

In marine environments, radars are one of the standard navigational sensors for surface ships. The sensor
provides relative position information of surrounding obstacles, and thus they have been commonly used for
target ship tracking and collision avoidance. However, their tracking performance degrades when a target is
approaching at a high relative velocity due to their relatively low sampling rate. As the demand for marine
defense applications using a USV increases, a compensation approach is required to enhance the tracking capa-
bility by properly handling the time-delayed measurements in an encountering situation at a high relative velocity.

In this study, we address an automatic target tracking method considering time-delayed measurements. The
proposed tracking approach consists of three steps. First, target obstacles are detected using a pulse radar
mounted on a USV by applying an automatic target detection algorithm on radar measurements. Second, the
relative position information between a USV and targets is extracted from the detected targets. Finally, target
motion analysis is implemented to estimate the detected target motion including position, course, and speed.
For this, an extended Kalman filter (EKF)-based tracking filter is employed, and the detected target information
is applied in the framework of the tracker considering time-delayed measurements.
Jungwook Han: Maritime Safety and Environmental Research Division, Korea Research Institute of Ships and Ocean
Engineering (KRISO), E-mail: jungwook@kriso.re.kr
Nam-sun Son: Maritime Safety and Environmental Research Division, Korea Research Institute of Ships and Ocean
Engineering (KRISO), E-mail: nsson@kriso.re.kr
Jinwhan Kim: Department of Mechanical Engineering, Korea Advanced Institute of Science and Technology (KAIST),
E-mail: jinwhan@kaist.ac.kr

SPIE Future Sensing Technologies, edited by Masafumi Kimata, Christopher R. Valenta, Proc. of SPIE
Vol. 11197, 111971B · © 2019 SPIE · CCC code: 0277-786X/19/$21 · doi: 10.1117/12.2547429

Proc. of SPIE Vol. 11197 111971B-1

Downloaded From: https://www.spiedigitallibrary.org/conference-proceedings-of-spie on 07 May 2022
Terms of Use: https://www.spiedigitallibrary.org/terms-of-use
 2. AUTOMATIC TARGET DETECTION
To extract obstacle information from radar sensor measurements including noise inherent in ocean environments,
a set of preprocessing steps are required. Well-known fast time constant (FTC) and sensitivity time control (STC)
are applied to minimize rain and sea clutters, respectively. To extract target-like shape on the radar image with
minimized noise, a dynamic thresholding method is applied, which is one of cell averaging constant false alarm
rate (CA-CFAR) detector.1, 2 For this, a window mask is designated by considering the size of interested targets
and a threshold level is generated by a moving window average and sufficiently strong signals compared to the
threshold level are extracted. The extracted signals are grouped into an identical target, and then the weighted
center of a target is obtained considering the intensity of each signal.2

3. TARGET MOTION ANALYSIS

In this study, a constant velocity model is applied to describe the motions of the observer (i.e., USV) and the
detected target in the EKF-based tracking filter.3 The state vector to represent the motion of the observer is
described as xo = [ xo yo ψo Vo ]T , where the xo and yo are the observer’s position, ψo is the observer’s course and
Vo is the observer’s speed in the longitudinal direction and they are defined in the global frame. The equation
associated with the motion in the filter’s system kinematics is described as follows:

ẋo = [ Vo cos ψo Vo sin ψo 0 0 ]T . (1)

The state vector of the tracking filter structure consists of the observer state vector xo and the target state vector
xT = [ xT T
T1 x T2 · · · ]
T
where xTi = [ xTi yTi ψTi VTi ]T represents the state vector of the ith detected target. The
system dynamics can be expressed as the state vector equation consisting of the motion models of the observer
and the target, which can be written as ẋ = [ ẋT T T
o ẋT ] + w, where w represents the zero-mean Gaussian process
noise to describe the uncertainty of the motion model.

The motion of the observing vehicle is updated from an onboard sensor measurements and the associated equation
can be expressed as follows:
zo = [ zx zy zψ zV ]T = [ xo yo ψo Vo ]T + vo (2)
where zx , zy , zψ and zV denote the position, course and speed measurements from the onboard motion sensor.
The vo is the zero-mean Gaussian measurement noise to reflect the uncertainty of onboard motion sensor (e.g.,
GPS).

The motion of the target is updated from the radar sensor measurements and the associated equation can
be expressed as follows:    
−yo

zβ
 arctan xyTT −x o
− ψ o
zT = = q  + vT (3)
zρ 2
(x − x ) + (y − y )
2
T o T o

where zβ and zρ denote relative bearing and range measurements obtained from the radar sensor. The vT is the
zero-mean Gaussian measurement noise to describe the uncertainty of radar sensor.

4. TIME-DELAYED MEASUREMENTS
In a typical radar-based tracking filter, the measurement update is implemented after collecting all the target
measurements acquired in one rotation of radar antenna as depicted in Fig. 1. The one rotation of a marine
radar takes approximately 2 seconds, and detected targets have different times of arrival, which results in time
delay (τD ) when they are used for target motion update in tracking filter as depicted in Fig. 2. If the sampling
rate to provide sensor measurements is relatively high compared to that of filter update, the error driven by
the time delay is negligible. However, a marine radar has a relatively low sampling rate, and thus the sensor
requires an additional technique to achieve satisfactory target tracking results by compensating the time delay
τD between the point of measurement generation (tk-1 ) and the point of its motion update( tk ) by using the

Proc. of SPIE Vol. 11197 111971B-2

Downloaded From: https://www.spiedigitallibrary.org/conference-proceedings-of-spie on 07 May 2022
Terms of Use: https://www.spiedigitallibrary.org/terms-of-use
 measurement in the tracking filter as described in Fig. 2. For this, a target state which is estimated in the
tracking filter is predicted to the time of measurement generation and the residual (i.e., innovation) is formed,
which represents the difference between the retrodicted target state and the measurement.4 Then, the residual
term is used to update the target motion state in the filtering framework, and the associated covariance matrix
is updated considering the time delay.5 The associated equation for target motion update with time-delayed
measurements can be expressed as follows:

He = HΦ(−τD ) (4)
T T −1
K(k) = P (−)He [ He P (−)He + R ]
xT (+) = xT (−) + K(k)(zT − He xT (−))
P (+) = [ I − K(k)He ]P (−)

where, H represents the measurement model, Φ represents the state-transition model, and K represents the
kalman gain. P describes the associated covariance matrix of the estimated target state and R describes the
covariance of the measurement noise. Note that the minus and plus signs depict the state before and after
including the time-delayed measurement.

Figure 1. Out-of-sequence measurements produced by rotation of radar antenna

Figure 2. Schematic illustration of time-delayed measurement in the radar-based target tracking.4 The time delay τD
degrades the accuracy of the estimated target motion.

Proc. of SPIE Vol. 11197 111971B-3

Downloaded From: https://www.spiedigitallibrary.org/conference-proceedings-of-spie on 07 May 2022
Terms of Use: https://www.spiedigitallibrary.org/terms-of-use
 5. FIELD EXPERIMENT
A field experiment was carried out in a real-sea environment using a USV developed by Korea Research Insti-
tute of Ships and Ocean Engineering (KRISO) as shown in Fig. 3. A pulse radar (Furuno FAR-2117) with an
approximately 0.5 Hz sampling rate and the Real-Time Kinematic Global Positioning System (RTK-GPS) were
installed on the platform. To obtain the ground-truth trajectory of the target ship, an automatic identification
system (AIS) was installed on the target ship. The speed of the ships was set to be ∼ 15 knots for the USV and
∼ 30 knots for the target ship in a head-on encountering scenario.

(a) Experimental site

(b) Experimental platforms

Figure 3. Experimental setup. (a) Experimental site and scenario. (b) Experimental platforms (left: ARAGON USV,
right : target ship).

Figure 4 describes the image sequences of the estimated target motion from the proposed method. The tra-
jectory of the USV is denoted by the solid line in blue and the estimated trajectory of the target ship is denoted
by the solid line in red. The σ envelope of the target position estimated in the filter is depicted with an ellipse
in the black line.

Figure 5 describes the error of the estimated target position. In the figure, the top image shows the USV
speed, the middle image shows the target speed, and the bottom image shows the comparison of the position
error before and after considering the time-delayed effect for the motion update in the tracking filter. The target

Proc. of SPIE Vol. 11197 111971B-4

Downloaded From: https://www.spiedigitallibrary.org/conference-proceedings-of-spie on 07 May 2022
Terms of Use: https://www.spiedigitallibrary.org/terms-of-use
 Figure 4. Image sequence of the target tracking results. The trajectories of the USV and the target are described in blue
and red lines, respectively. The error bound of the target position estimated in the tracking filter is represented by the
black line, and the estimated course of the target is represented by the direction of the red line.

was initially stationary, and then it speeded up rapidly as shown in the middle image of Fig. 5. The error of
estimated target position was reduced by handling the time-delayed measurements by the approach introduced
in this study, and the gap of the position errors before and after considering the time-delayed effect became
larger as the relative velocity between the two vehicles increased.

6. CONCLUSION
In this paper, a radar-based target tracking was introduced for USV operation. For this, a surrounding target
ship was detected and its motion was estimated in an EKF-based tracking filter. To reduce the tracking per-
formance degradation by the low sampling rate of a marine radar which induces time-delayed measurements, a
compensation scheme was introduced and implemented in the tracking filter. To demonstrate the feasibility of
the introduced approach, the target tracking algorithm considering the time-delayed measurements was applied
to a developed USV platform and a field experiment was performed at a real-sea environment in this study.

Proc. of SPIE Vol. 11197 111971B-5

Downloaded From: https://www.spiedigitallibrary.org/conference-proceedings-of-spie on 07 May 2022
Terms of Use: https://www.spiedigitallibrary.org/terms-of-use
 Figure 5. Analysis of the target tracking results. The bottom figure represents the comparison of the position error
before and after considering the time-delayed effect for the motion update in the tracking filter. The gap of the position
errors before and after considering the time-delayed effect became larger as the relative velocity between the two vehicles
increases.

ACKNOWLEDGMENTS
This research was a part of the project entitled “Development of situation awareness and autonomous navigation
technology of unmanned surface vehicle based on the artificial intelligence (PES3050)” funded by the Korea
Research Institute of Ships and Ocean Engineering.

REFERENCES
[1] Rohling, H., “Radar cfar thresholding in clutter and multiple target situations,” IEEE transactions on
aerospace and electronic systems (4), 608–621 (1983).
[2] Cambridge Pixel Ltd., “[online]. available: http://www.cambridgepixel.com,” (2018).
[3] Han, J., Kim, J., and Son, N.-s., “Persistent automatic tracking of multiple surface vessels by fusing radar
and lidar,” 1–5, IEEE OCEANS 2017-Aberdeen (2017).
[4] Blackman, S. and Popoli, R., [Design and analysis of modern tracking systems], Norwood, MA: Artech House
(1999).
[5] Bar-Shalom, Y. and Li, X.-R., [Multitarget-multisensor tracking: principles and techniques], vol. 19, YBs
Storrs, CT (1995).

Proc. of SPIE Vol. 11197 111971B-6

Downloaded From: https://www.spiedigitallibrary.org/conference-proceedings-of-spie on 07 May 2022
Terms of Use: https://www.spiedigitallibrary.org/terms-of-use