Proceedings of the Nineteenth (2009) International Offshore and Polar Engineering Conference

Osaka, Japan, June 21-26, 2009

Copyright © 2009 by The International Society of Offshore and Polar Engineers (ISOPE)
ISBN 978-1-880653-53-1 (Set); ISSN 1098-618

Development of an Underwater Vehicle that Has Functions of Acquisition

and Tracking Target Objects Using Images Acquired by Itself
Toshinari Tanaka, Muneo Yoshie
Construction and Control Systems Department, Port and Airport Research Institute
Yokosuka city, Kanagawa prefecture, Japan

Mutsuo Oosato
Sendai Research and Engineering Office for Port and Airport, Tohoku Regional Development Bureau
Sendai city, Miyagi prefecture, Japan

ABSTRACT.
ROVs are often used to underwater observation and research mission.
However, the operation of conventional ROVs under disturbances such
as wave and current conditions are often problem. If the target object is
moving by disturbances, the operation will not be able to continue.
Therefore, we are aiming at realization of smart and secure in situ
operations onboard. Especially, objective of our project is maintenance
examination for swinging mooring facility of GPS buoys by our
developed underwater vehicle. The buoys were deployed for survey
hydrographic phenomena. Our key methods for semi-autonomous
underwater vehicle are optical navigation technology and smart
operational interface.
KEY WORDS: Semi-autonomous underwater vehicle; Optical
navigation; Maintenance examination, GPS wave height meter.
INTRODUCTION
that combines remote operation and autonomous control. The main
autonomous functions are based on underwater real-time image which
obtained by itself. As a result of the use of these methods, troublesome
operations for acquisition and tracking the target objects are relieved.
Objective of our project is maintenance examination for mooring
facility of GPS buoy by our developed underwater vehicle. In the past,
the research group, including our institute, had tried active utilization of
underwater images for ROV operation (Ohata, 2005; Sakai, 2004a;
Sakai 2004b). In this paper, we report an outline of new project to
utilize underwater images actively and our experimental semi-
autonomous underwater vehicle applied the system combines the
remote operation and autonomous control. We describe our large tank
examination of our underwater vehicle, and show our key technology
for realizing smart and secure in situ operations offshore. Our research
work has realized one function of acquisition and tracking the swinging
target object in the large tank test. We are planning several offshore
trial missions such as next picture in Kamaishi, Iwate Prefecture.

The Japanese Government is advancing the deployment of large

offshore floating equipment such as GPS buoys which survey hydraulic
phenomena throughout sea area around Japan. To operate the
equipment securely, the system needs the maintenance examinations
regularly. Especially, the mooring facility needs several checks to keep
safety of system operations. It is necessary that the wear damage of the
chain part are measured precisely. However, it is difficult to raise
settled large offshore floating structures and to investigate the mooring
facility. Therefore, not to raise the settled floating structures, it is
preferred that measuring methods in situ survey is developed.
Generally, ROVs are often used to underwater observation and research
mission. However, the operation of conventional ROVs in disturbances
such as wave and current conditions are often problem. If the target
object is swinging by waves, the operation will not be able to continue.
For these reasons, it is not easy to survey the mooring facility by using
only conventional ROV’s technology.
Therefore, we are aiming at realization of smart and secure in situ
operations onboard. Our key methods for semi-autonomous underwater
vehicle are optical navigation technology and smart operational
interface. Our developing equipment is installed our original system
Fig. 1 Expected Mission of Maintenance Examination in Situ.

651
EXISTING CONDITION OF UNDERWATER PART OF GPS
WAVE HEIGHT METER

We executed the survey of existing conditions of an underwater part of

the GPS waves height meter in the offing of Kamaishi which passed
from the settling about one and a-half years. In the survey, the wave
height was about 0.8m, and the current condition was about 1.5kt from
north-northeast. Therefore, it was difficult for conventional small ROV
to take images, and we were not able to obtain clear images. However,
the condition of underwater part had never been confirmed after the
settlement of itself. Therefore, the obtained image included very
interesting results.
We saw large amount of attached organisms at the body part of the
GPS wave height meter from the obtained images, and confirm the
decrease in attached organisms with increase in depth. Therefore, it
would appear that the image measurement for the mooring facility is
possible.
However, it is assumed that attached organisms will increase as years
go by. Moreover, we found a flaked scar of attached organisms. It
seems that they flaked off under their own weight and by severe Fig. 4 Flaking of Attached Organisms.
hydrographic phenomena.
DEGREE OF SOUNDNESS CHECK METHODS OF THE
CHAIN PART

Ordinary Degree of Soundness Check Method of the Chain Part

A case example of the degree soundness check for lighted buoy’s

mooring facility is shown in this subsection. In this case, divers
measure the wear damage of the predefined the chain part using slide
gauge. In other words, they measure sum of two wire diameters of
connected pieces, and can grasp the degree of soundness of the chain
part as remaining amount of chain. In particular, they measure amount
of C (from C-1 to C-5) in the next figure.

Fig. 2 Body Part of the GPS Wave Height Meter

with Large Amount of Attached Organisms.

Fig. 5 Mooring Facility of Lighted Buoy in Tokyo Bay.

Fig. 3 Chain Part of the GPS Wave Height Meter. Fig. 6 Measured Points of Lighted buoy’s Chain (from C-1 to C-5).

652
Proposed Method Verification Test of Image Measurement System in Water

In the other hands, it is difficult to check the mooring facilities by diver We executed an image measurement experiment of the chain as the
because of severe hydrographic phenomena. Utilization of slide gage in system verification test in small tank. The used image measurement
situ is almost impossible even more. In addition, it is also difficult to system was estimated parameter of optical system. In this experiment,
raise settled large offshore floating structures and to investigate the roughly half the size of the real thing (wire diameter;φ 50) was used.
mooring facility.
Therefore, we adopt a method of measuring forgoing sum of two wire
diameters of connected pieces using obtained images in situ. The link
chain consists of several size chains and shackles. These link points can
be favorable certain parts as inspection points. In image measurement
system, we can grasp “real dimension D [mm]” using “shooting
distance L [mm]” and “number of pixels N [pixel]” in general if the
optical system is given. As an example, the identification result of the
optical system is shown as follows since we calibrate an optical system
of image measurement system and identified the parameters.
In the optical system of image measurement system, inverse of N is
proportional to L. For instance, relations between N and L in case of
D=1mm is shown in next equation.

1
= aL + b [mm pixel]. (1)
N D =1mm

In addition, “number of pixels N” is proportional to “real dimension D”,

and is in inverse proportion to “shooting distance L”. These relations
are shown in next equation. Fig. 8 Chain which was used the Image Measurement Experiment
(L300xB175xφ 50, 10kg/piece).
1
N = D× [pixel]. (2)
The developed measurement application software has only simple
aL + b
composition. The major algorithm is based on blob analysis of edges in
In the next place, parameters of optical system a and b are estimated by the longitudinal direction of the chain. The appearance of the image
the method of least squares, and refractive index correction factor c was measurement experiment and the verification test result are shown as
calculated by the ratio of actual measurement values. In the result, D is follows.
estimated by next equation. In this experiment, we were able to check the behavior of developed
measurement application. In addition, we were able to verify the
estimated error of the image measurement system.
⎧a = 1.044 × 10 −3
⎪
D = cN ⋅ (aL + b ) [mm], −3
⎨b = −4.483 × 10 .
(3)
⎪ c = 8.204 × 10 −1
⎩

8 1
In air (actual values) In air (actual values)
7 In air (estimated a and b) In air (estimated a and b)
Number of pixels ND =1mm [pixel/mm]

0.8 In water (estimated a, b, and c)

6
1 / ND =1mm [mm/pixel]

5
0.6

3 0.4

2
0.2
1

0 0
0 100 200 300 400 500 600 700 800 900 1000 0 100 200 300 400 500 600 700 800 900 1000
Shooting distance L [mm] Shooting distance L [mm]
(a) Number of pixels ND=1mm (b) Inverse of the number of pixels 1/ ND=1mm
Fig. 7 One Characteristic Graphs Example of the Optical System of Image Measurement System
(Specifications; 1/3” 0.8Mpixels (XGA) CCD, f=4mm (V/H; 61.9deg / 48.5deg) C mount LENS).

653
 Control PC

The control PC is key component of hybrid operation system that

combines remote operation and autonomous control, and the roles are
acquisition of sensor information and vehicle control.

Fig. 9 Situation of Image Measurement Experiment in Small Tank. Fig. 11 Console Application of Control PC.

Table 1. Verification Test Result The control PC indicates sensor information from vehicle, and it
Specification of chain L300xB175x φ 50 processes operation command and send them to the vehicle. Moreover,
Actual value by slide gauge 99.05[mm] operator is able to direct onsets of three autonomous functions which
Result of image measurement 100.465[mm] are direction hold, depth hold and target tracking.
Shooting distance 500 [mm] The system is able to respond to the interrupt from joystick while the
Error 1.415[mm] autonomous functions are in place. Namely, direction to the vehicle is
combination of remote operation and autonomous control. In other
TANK TEST MODEL words, operator makes a decision about basic operations such as
approaching target object or bearing off, and entrust partial low-order
Tank test model for mooring facility maintenance examination machine operation such as postural hold or target acquisition to the vehicle.
was constructed by enhancement of the I/O interface for cooperation Therefore, the operation is simplified. We are aiming at safe and secure
with PC and rigging a conventional small ROV. The equipped devices operation by the simplification, and we can expect it to produce the
are able to unrig easily, and the test model is able to be restored to the effect in severe hydrographic phenomenal conditions.
original ROV if necessary. The major features of the tank test model
are to have a camera for image measurement and a hybrid operation
system that combines remote operation and autonomous control (semi-
autonomous underwater vehicle). The main autonomous functions are
based on underwater real-time image which obtained by itself.

Consists of the System

Tank test model consists of three main components. There are clear
assignments of roles between two PC, and they fulfill a specialized
function respectively. Composition of the model is shown as follows.
Fig. 12 Yawing Control System for Target Tracking.

Image processing PC

This PC processes images for measurement and tracking target object

Fig. 10 Composition of the Tank Test Model.
using images acquired by itself. All the image processing application
was developed by us on the Microsoft VC++ using an image processing
library of MVTec HALCON. We are able to select two methods for
acquisition of target object using the color and edge information on the
application. In the acquisition method using color information, after the
RGB information convert into HSV, the hue information is used mainly.
As a result, the illuminance dependency of the target recognition rate is
decreased. However, the color and hue change due to water layer
thickness, and the target recognition rate is decreased. In water, the
target recognition rate of acquisition method using edge information is
higher than the former method in most cases. Therefore, tank test
results using edge information is shown in next section.

654
 TANK TEST OF ACQUISITION AND TRACKING TARGET
OBJECT

In this test, a long chain which connected 10 pieces of chain A

(10kg/piece) to 2 pieces of chain B (5.25kg/piece) through two pieces
of shackles (6kg/piece) was prepared as a target object. The long chain
was hooked with crane. After it was submerged in water, the test of
acquisition and tracking target object was executed. The scales of used
chains are roughly half the size of the real thing. In the test, the vehicle
was operated by using joystick with two autonomous functions which
are depth hold and target tracking.

Shackle

175.0

50.0

Chain A

Fig. 13 Image Processing Application

(Acquisition of Target Object by using Hue Value).
Shackle
140.0

40.0

5.25ｋｇ
Chain B

Fig.16 Long Chain which is Target Object.

Acquisition of Target Object

The vehicle started acquisition of target object from the distance of

about 3.0m. After that, the vehicle was able to navigate easily with less
operation of joystick.
As a result of recognizing the target object, chain was yellow-edged,
and was enclosed in rectangle. Situation of acquisition of target object
is shown as follows.

Fig. 14 Control PC, Image Processing PC and Control Box of Vehicle.

Underwater Vehicle Part

2 horizontal thrusters, a gyro compass, and an image processing-only

camera for optical navigation were installed to conventional small ROV
(DELTA150). As a result, the vehicle has 2 cameras and 6 thrusters in
total. Moreover, by enhancement of the I/O interface for cooperation
with PC, the control of the vehicle and utilization of sensor information
through the control PC was actualized. The vehicle of tank test model is
shown as follows.

Fig. 17 Tank Test of Acquisition of Target Object.

(a) Side view (b) Front view. (Acquisition of Target Object by using Edge Information).
Fig. 15 Underwater Vehicle Part of Tank Test Model
(Semi-autonomous Underwater Vehicle).

655
Tracking Target Object
In the case of swinging captured target object, we executed an
acquisition and tracking target object test. We swung the hooked long
chain with a crane in the water. As a result, the vehicle was able to
capture and track the target object to the distance of about 3.0m.
Situation of tracking target object is shown as follows.
Fig. 18 Submerged Tank Test Model while Tracking Target Object.
Next picture shows trajectory of on-screen lateral location of target
object and the vehicle direction by yawing for target tracking. In this
test, onset of target tracking function was at the elapsed time of 550sec.
It would appear that the amplitude of target position decrease after
about 550sec. The result shows a part of advantageous effect of the
target tracking function although it remains a matter of the optimization.
CONCLUSION
We had started new project for maintenance examination of mooring
facility of GPS wave height meter settled in the offing of Japan.
In this paper, we described the outline of the project and our aims at
first. Our aims and end were measurement of wear damage of the
chains in situ and the unmanned underwater work.
Secondly, we report a survey of existing condition of GPS wave height
meter in the offing of Kamaishi which passed from the settling about
one and a-half years. Because the condition had never been confirmed,
we could exhibit significant current of attached organisms.
Thirdly, we constructed an experimental image measurement system. In
this system, we identified the optical system, and confirmed the
characteristics and the error.
Forth, we constructed a semi-autonomous underwater vehicle as tank
test model for mooring facility maintenance examination machine. It
consists from control PC, image processing PC, and underwater vehicle.
The test model included three autonomous functions which are target
tracking using obtained image by itself, direction hold, and depth hold.
Fifth, we executed tank tests of acquisition and tracking target using the
tank test model. In a result, we could show a part of advantageous
effect of the target tracking function.
One of the remaining issues is performance-improved of controller for
tracking target object. We will optimize gains and configuration of the
controller for performance-improved. Moreover, another one is that
acquisition function of distance information from target object is
unpopulated in our system. We will install the methods for acquisition
of it and keeping the distance to the tank test model.
In response to the result of our activities, we started development of a
part of practical trial model. Alongside the development of practical
trial model, we are developing various applications using tank test
model. Therefore, we are planning offshore trial missions for
maintenance examination of mooring facility after the completion of
practical trial model in the offing of Tohoku district which include of
Kamaishi.

320 ACKNOWLEDGEMENTS
Vehicle direction [deg]
Target position [pixel]
240 In case of primary data gathering, we were supported by the Tohoku
Regional Development Bureau and the Kanto Regional Development
160 Bureau, Ministry of Land, Infrastructure, Transport and Tourism, Japan.
Vehicle direction [deg]
Target position [pixel]

80
REFERENCES
0 Ohata, S, Ishii, K, Sakai, H, Tanaka, T, and Ura, T (2005).
“Development of an autonomous underwater vehicle for observation of
-80
underwater structures,” Proceeding of Oceans 2005, Vol 3, pp 1928-
1933.
-160 Sakai, H, Tanaka, T, Ohata, S, Ishitzuka, M, Ishii, K, and Ura, T
(2004a). “Applicability and Improvement of Underwater Video Mosaic
-240
System Using AUV,” Proceeding of Oceans 2004, Vol 2, pp 659-664.
Sakai, H, Tanaka, T, Ohata, S, Mori, T, Ishii, K, and Ura, T (2004b).
-320 “Underwater Video Mosaicing Using AUV and Its Application to
400 450 500 550 600 650
Vehicle Navigation,” Proceeding of Underwater Technology 2004, pp
Elapsed time [sec]
405-410.
Fig. 19 Trajectory of On-screen Lateral Location of Target Object and
the Vehicle Direction
(Onset of Target Tracking; 550sec).

656