See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/224354870

Position measurement of container crane spreader using an image sensor

system for anti-sway controllers

Conference Paper · November 2008

DOI: 10.1109/ICCAS.2008.4694588 · Source: IEEE Xplore

CITATIONS READS

14 144

4 authors, including:

Young-Bok Kim
Pukyong National University
191 PUBLICATIONS   631 CITATIONS   

SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Funded by Korea Gov. View project

Funded by Korea Gov. View project

All content following this page was uploaded by Young-Bok Kim on 03 May 2022.

The user has requested enhancement of the downloaded file.

International Conference on Control, Automation and Systems 2008
Oct. 14-17, 2008 in COEX, Seoul, Korea
Position Measurement of Container Crane Spreader Using an Image Sensor
System for Anti-sway Controllers
H. Kawai1, Y. Choi1, Y. B. Kim2 and Y. Kubota1
1
Department of Information Systems Science, Soka University, Tokyo, Japan
(Tel : +81-42-691-8197; E-mail: hkawai@soka.ac.jp)
2
Department of Control & Mechanical Engineering, Pukyong National University, Busan, Korea
(Tel : +82-51-620-1584; E-mail: kpjiwoo@pknu.ac.kr)

Abstract: Speedy operation efficiency is constantly required of the container crane operators who work for cargo
handling in the port. They are experts in suppressing the sway motion which is generated in the process of the handling.
This undesirable motion may deteriorate the operation efficiency due to strained nerves of the operator. Therefore, an
anti-sway controller has been required on practical job-site. To accomplish an adequate anti-sway control, sensors are
also required to be able to measure the position of a crane spreader moving on. In this paper, we propose a position
measurement system with an image sensor. The image sensor detects two devised landmarks attached to the upside
surface of a spreader to measure the displacement and height to one. The image processing technique used for the
sensor is a kind of robust template matching method, named Vector Code Correlation (VCC), devised to consider the
conditions of real environments. We employ the simple mechanism, so that the system can be easily applicable to
existing container cranes. The details and performance of the measurement system are introduced together with the
experimental results performed with the pilot system. We will also show that the system has the sufficient efficiency
and accuracy for the use on practical job-site.

Keywords: Container crane, Image sensor, Position measurement, Robust template matching.

1. INTRODUCTION is a kind of robust template matching method, named

The Container cargo is the central means in
transportation, and a lot of containers transferred are
handled in a container port. The Gantry cranes and
transfer cranes are widely used to handle containers
from the container ship to the piling them up and down
or the trucks in the container yard of a port. In recent
years, the operating efficiency in size and operation
speed of these cranes has been required to get larger and
faster according to the increasing of container amount to
be treated. However, in the surroundings for crane
operators, it is the current state to be unable to follow
the force for the mechanism improvement. Therefore,
those studies such as anti-sway control system and
auto-control system have been required to treat a
number of containers as fast as possible.
To accomplish an adequate anti-sway control, sensors
are required to be able to measure the position of a
crane spreader including the displacements from the
static position and the height from the trolley. Many
measurement systems using image sensor have been
studied for the anti-sway controller[1,2]. However,
these systems have some issues as follows:
・Some systems can’t simultaneously measure the
displacement and height of the spreader.
・Some systems seem that it is difficult to install on
existing cranes.
In this paper, we propose a position measurement
system with an image sensor. The image sensor
simultaneously measures the displacement to sway and
the height of the spreader, by using the ratio of that in
the captured image to the real distance between two
landmarks mounted on the spreader. The image
processing technique used to detect the landmarks here
Vector Code Correlation (VCC), devised to consider the
conditions of real environments [3]. The details and
performance of the measurement system is introduced
together with the experimental results performed with
the pilot system.
2. SYSTEM DESCRIPTION
Fig.1 shows the proposed system that consists of an
image sensor and devised landmarks. The image sensor
is placed on the trolley of a crane, containing a CCD
camera and an image processor. And, two landmarks are
attached to the upside surface of the spreader. The shape
of the landmarks is shown on the right side of Fig. 1.
The landmark is designed to be able to similarly see at
different height so that the image sensor can detect it at
all height.
The processing flow of the position measurement is
explained as the following. The CCD camera on the
image sensor detects the position of two landmarks in
the captured image. The processor on it calculates the
Trolley
CCD Camera &
Image processor
Spreader
Container Landmark
Landmarks
Fig. 1 Schematic of the position measurement system.

683
Authorized licensed use limited to: Pukyong National Univ. Downloaded on May 03,2022 at 00:43:37 UTC from IEEE Xplore. Restrictions apply.
distance varied from a certain standard location and the by using the ratio of the distance D in the real world
height of the spreader with the distance relationship of and the distance d on the image. By using the
two landmarks in a captured image. positions of two landmark, the distance d and the
displacement of the spreader in the image defined as
3. POSITION MEASUREMENT ( xs , ys ) are calculated as follows:
USING THE IMAGE SENSOR
d = ( x1 − x2 ) 2 + ( y1 − y 2 ) 2
(1)
3.1 Principle of position measurement
Fig.2 shows the schematic diagram of position (xs , ys ) =  x1 + x2 , y1 + y2 
 2 2  (2)
measurement of the spreader using the image sensor
Then, the real displacement ( X s , Ys ) of the spreader is
system.
The measurement of sway motion is to measure the calculated as the following by using ( xs , ys ) found
displacement of the container spreader from a static above.
position. Fig.2 (a) illustrates the diagram of
measurement of sway motion. Where, the origin Os on ( X s , Ys ) =  D xs , D ys  (3)
d d 
the trolley is through the center of a spreader in the Fig.2 (b) illustrates the diagram for height
static state and oc is the center of the spreader in an measurement of the spreader. The spreader height Z s
input image captured at that time, and the position is divided into two parts, Z m and Z c . Where, Z m and
( X s , Ys ) illustrates the displacement of the container
Z c are the distance from the origin Os to the focal
spreader that is moved from the point Os . This
point of the camera on the image sensor and the distance
displacement is measured by the image sensor, which is from the focal point to the upside surface of the spreader,
installed on the trolley side. The center oc captured by respectively. They are calculated by the ratio of D and
the image sensor corresponds to the center Os of the d . Thus, the spreader height Z s is found as follows:
spreader in the static state. The image sensor detects the Z s = Zm + Zc (4)
positions ( x1 , y1 ) and ( x2 , y2 ) of two landmarks on the D
Zc = f (5)
captured image as shown in the right side of Fig.2 (a). d
Then, the measurement of sway motion can be achieved
3.2 Detection of the landmarks using the VCC
In general, one of the techniques that have been
widely used in finding a target in a given image is the
Spreader
L1 template matching method. The method is to discover
( x1 , y1 )
( X s , Ys )
the target by scanning the entire image while comparing
(xs , ys )
D
Z
Y
d y
x
a partial image to the target template. However, the
X ( x2 , y 2 ) oc
Os common method has the disadvantage to need a long
L2 time in finding the target template in a captured image.
To overcome this, we have developed a kind of template
matching method, named vector code correlation (VCC),
Displacement of the spreader
in an input image
promoted in the image processing time and illumination
condition. The VCC method is especially useful to the
Static position
environmental conditions such as the light variation in
Displacement of the spreader
in the real world
outdoor and the processing speed for detecting the
(a) Diagram of sway measurement. landmarks. Thus, we employ the VCC method on the
Y
image sensor for detecting the landmarks. The VCC is a
Os X
Trolley
method to calculate the similarity by comparing vector
Z codes instead of the intensity of pixels commonly used
Zm d Image plane
f
in image processing. The vector code to be said here is
Focal point codes (01, 00, 10), divided into 3 directions to x, y-axis
Zs = Zc + Zm respectively, for the gradient of an approximate plane of
Zc pixel intensity. The gradient of an approximate plane is
found by the intensity of 8 neighbor pixels for a
Spreader D one-pixel, as shown in the Fig.3.
The gradient of the intensity variation for a one-pixel
to x, y-axis are respectively encoded by a vector code of
each 2 bits. For example, the vector code of the gradient
in the ‘Template’ of Fig.3 is 0101; it is because there is
(b) Diagram of height measurement 01 of plus direction to x, y-axis, respectively.
Fig. 2 Schematic diagram of position measurement of Likewise, the vector code of the gradient in the ‘Partial
the spreader using the image sensor system. image’ of Fig.3 is encoded by 0000, because there is 00

684
Authorized licensed use limited to: Pukyong National Univ. Downloaded on May 03,2022 at 00:43:37 UTC from IEEE Xplore. Restrictions apply.
Intensity y Table 1 Specifications of the measurement system
0101 Camera PX-VGA120-LM(Mono)
encode Resolution 640×480[pixel]
x
Template vector Pixel size 7.4×7.4[μm]
Gradients codes
Exclusive OR 0101 2
Intensity y count Focal length: 12[mm]
degree of Landmark distance: 70[mm]
0000 dissimilarity
Processing speed 110[fps]
encode
x 100
Partial image
80
Fig. 3 Scheme of the vector code correlation 60
of horizontal direction to x, y-axis, respectively. These

displacement [mm]
40
two vector codes encoded are calculated by the 20
‘exclusive OR’, to be resulted in 0101. By counting the 0
number of ‘1’ for this result 0101, we can obtain the -20
degree of dissimilarity, that is, 2 between two vector -40
codes encoded. -60
By adopting the VCC method for detecting the -80
landmarks, the image sensor can obtain the robustness -100
0 10 20 30 40 50
and cost reduction in the image processing under time [sec]
dynamical outdoor conditions. This can also expect high (a) Displacement of the spreader
accuracy for the image sensor when considering the
850
practical application on job site like container yard in a
port.
750

4. EXPERIMENTS
height [mm]

650

To prove the usefulness of our image sensor system,

550
we performed the experiments for displacement and
height measurement of the spreader with a container
450
crane model as shown in Fig. 4. The model is made up
of a driven trolley at the top and a spreader rolled
350
up/down till 800 mm from the trolley. The measurement 0 10 20 30 40 50
system consists of a CCD camera at the center on the time [sec]

trolley, two landmarks in the level surface on the
spreader shown in the extended image of Fig.4 (b) and a
PC executing image processing. Specifications of the
measurement system are shown in Table 1.
4.1 Accuracy of the image sensor system
To evaluate the position accuracy of our image sensor
system, we made measurement experiments for the
displacement and height of the spreader under indoor
luminescent lamps as shown in Fig.6 (a). Fig.5 (a)
shows the displacement of the spreader obtained while it
is swayed at the height of 600mm. We can see that the
displacement of sway motions gradually gets small with
the elapse of time in the figure. From the experiment for
CCD Camera
Landmarks
(b) Height of the spreader
Fig. 5 Experimental results under indoor luminescent
lamps illumination
the height of the spreader, Fig.5 (b) shows the
measurement results obtained while the spreader rolls
up and down between 300mm. Here, we can see that the
spreader is rolled up from 750mm to 450mm between
about 3-8 second, and begun to roll down at about 33
second.
In addition, for the details in the measurement
accuracy, we investigated the frequency in measurement
errors. As the standard to compute measurement errors,
we used the values measured by a laser sensor named
ODS96[5] for the sway measurement and by an encoder
for the height measurement, which has widely been
utilized in common. From the results compared on the
basis of each other sensor, Fig.8 (a) shows the
frequency in measurement errors for the sway motion,
and Fig.8 (b) shows the frequency in measurement
errors for the height. Their maximum errors shown in
Fig.8 (a)(b) are 4.3mm and 8.0mm, respectively.

4.2 Robustness of the image sensor system

To evaluate the robustness to illumination changes
(b) Measurement for our image sensor system, we made measurement
(a) Overall view of the Crane model system experiments under changeable conditions in lighting.
Fig. 4 Crane model with the measurement system.

685
Authorized licensed use limited to: Pukyong National Univ. Downloaded on May 03,2022 at 00:43:37 UTC from IEEE Xplore. Restrictions apply.
 600
(1) Routine illumination
(2) Changed illumination
500

400

Frequencey
300

200

100 Minimum error: Maximum error:

(1)-0.2[mm] (1)3.8[mm]
(2)-0.2[mm] (2)4.3[mm]
0
(a) Image in a routine illumination (b) image in a changed illumination -4.5 -4.0 -3.5 -3.0 -2.5 -2.0 -1.5 -1.0 -0.5 0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 4.0 4.5
Fig. 6 Examples of illumination change to the landmarks Error [mm]

100 (a) Frequency in measurement error for the sway motion

80 600
measurement error
60
displacement [mm]

500
40
20 400
0

Frequency
300
-20
-40 200
-60
-80 100 Minimum Maximum
error: -5.7 [mm] error: 8.0 [mm]
-100
0 10 20 30 40 50 0
time [sec] -8.0 -7.0 -6.0 -5.0 -4.0 -3.0 -2.0 -1.0 0.0 1.0 2.0 3.0 4.0 5.0 6.0 7.0 8.0
Error [mm]
Fig. 7 Result measured in the condition of Fig.6(b) (b) Frequency in measurement error for the height
Fig.7 shows the result measuring the displacement of Fig. 8 Frequency in measurement errors compared on the
the spreader to change the illumination condition from a basis of the values of each other sensor
usual indoor illumination Fig.6 (a) to the state as shown
in Fig.6 (b), subject to the condition that Fig.5 (a) is
REFERENCES
obtained. We can see that Fig.7 is almost the same as
Fig.5 (a) without regard to the changeable illumination [1] H. Yoshikawa, “Development of Position Sensor
condition. This means that the sensor system has the for Quayside Cranes”, Mitsubishi Heavy
robustness to the change of illumination conditions. It Industries Technical Review, Vol. 37, No. 5, pp.
can also be confirmed from the details for measurement 290-293, 2000. (in Japanese)
accuracy, as shown in (1) routine and (2) changed [2] Y. Yoshida, “Visual Feedback Control of Traveling
illumination of Fig.8 (a). Crane”, Trans. SICE, Vol. 41, No. 6, pp.527-532,
2005. (in Japanese)
5. CONCLUSION [3] H. Kawai, “The Vector Code Correlation: A
template Matching Method Adequate for Outdoor
A Position measurement system of container crane Use”, Proc of ISSS’07, pp.7-9, 2007.
spreader using an image sensor has been proposed and [4] S. Kunimitsu, “Measurement of Relative Position
designed so that it can simply be mounted on container of Container with Image Processing for Automatic
cranes having been used. From the experimental results Container Cranes”, Trans. IEE Japan, Vol. 121-C,
performed with a model crane mounting the sensor No. 5, pp. 882-891, 2001. (in Japanese)
system, we proved the effective characteristics of the [5] http://www.leuze.com/products/los/mess/ods96/od
system as follows: (1) The measurement system can s96a_en.html
measure the displacement and height of a spreader at
high accuracy. (2) The system can be used in real
environments handling containers at day and night, by
using the VCC method robust to changeable
illumination conditions. Hereafter, we will verify the
usefulness and efficiency of the position measurement
system by applying it to real cranes used on a practical
job-site.

ACKNOWLEDGEMENT
This work was supported by grant No. R01-2005-0
00-10418-0 from Korea Science & Engineering Fou
ndation in Ministry of Science & Technology.

686
Authorized licensed use limited to: Pukyong National Univ. Downloaded on May 03,2022 at 00:43:37 UTC from IEEE Xplore. Restrictions apply.
View publication stats