Object Tracking in Deformed Patterns
Ashraf Aboshosha, M. Hassan1, M. Ashour1
(1) Radiation Engineering Department, NCRRT, AEA, P. O. Box. 29, 8th District,
Nasr City, Cairo Egypt
Abstract- The geometrical modeling and tracking of visual
targets is a key subject in several advanced technologies. The
precision of the visual tracking is still an open area for research
due to the presence of several serious problems. This paper
presents a successful technique to benefit from the camera as a
real ranging sensor by which the serious errors due to
deformations will be eliminated. The visual object tracking is an
important subject for precise identification and pursuing of
targets (human and objects) in deformed video stream. A three
dimensional (3D) geometrical model has been developed to
determine the current pose of an object and predict its future
location based on FIR model learned by the OLS while the visual
extraction relies on the YCbCr model. We propose a robust
ranging technique to track a visual target to replace the
traditional expensive ranging sensors. The presented research
work is applied to real video stream and achieved high
precession results
Index Terms-Visual Interferometery, modeling, motion
estimation, visual ranging, digital crime, Forensic science,
I. Introduction
V isual object tracking become a key subject of several
advanced applications such as, satellite and space
systems, traffic systems, medical applications,
surveillance, security, autonomous vehicles and Visual
ranging which defined as estimation of the distance between
the camera and the tracked object. Camera distance estimation
from objects in images has been a adaptable area for computer
vision and image processing researchers. There are different
algorithms to calculate the distance between the camera and
the object from digital images and videos. These methods
depend on the knowledge of the angle between the cameras
and the object, therefore limiting the application of such
algorithms. These algorithms utilize methods such as pan and
tilt or laser rangers to measure the angles which limits the
ability of these algorithms.The images and sequences coming
from video systems need to be digitalized in order to be
processed by dedicated software to enhance features useful
for visual tracking analysis. Generally, this is done either to
reduce the different kinds of corruptions that have been
introduced in the acquisition, conversion, and storage
processes of data or to overcome the limits of the overall
system. The characteristic problems in this area are; to deal
with low resolution, lack of contrast, different types of noise
and change of illumination. The precision of the visual
detection is still an open area for research due to the presence
of several serious deformation sources such as deformations
caused by the motion.
A..I mage Deformations, a Review,
S. Harasse et al. [1] presented a method for the detection and
tracking of multiple faces in a video by using a model of first
and second order local moments. An iterative process is used
978-1-4244-5844-8/09/$26.00 ©2009 IEEE
to estimate the position and shape of multiple faces in images,
and to track them. X. Liu and T. Chen [2] proposed a face
mosaicing approach to model both the facial appearance and
geometry from pose-varying videos, and apply it in face
tracking and recognition. Payman Haqiqat [3] presented a
region-moment based method for tracking moving objects.
Region-based methods that compute a similarity measure, like
cross-correlation, between the two segmented regions in two
successive frames of the input sequence work well when the
moving objects have only a translational motion. These
methods get into trouble in the presence of rotational motion
of the moving objects. They used image moments to define a
distance function between two circular regions. In [4] the
authors presented a parametric skin color classifier that can be
adapted to the conditions of each image or image sequence.
Its high speed and high accuracy makes it appropriate for real
time applications such as face tracking and mimic
recognition.
B. Visual Tracking, a Review,
In object tracking several researchers focused on thresholding,
region based segmentation, region merging and splitting, edge
based segmentation, clustering and on hybrid techniques to
track a visual guide.
Parker proposed a thresholding method capable of
thresholding images that have been produced in the context of
variable illumination [5]. Thresholding is the oldest
segmentation method and is a very popular technique in
image segmentation since it is computationally inexpensive
and fast [6]. Cheriet et al. proposed a more sophisticated
approach, the “recursive thresholding technique”, in order to
segment bank cheques [7]. Ohlander proposed a thresholding
technique for outdoor colored images. He adopted an
approach using nine one-dimensional histograms of features
such as color intensity (for red green and blue), and hue to
segment natural scenes [8].
The purpose of this paper is to present a different approach to
camera distance estimation from objects. The method that we
are proposing does not depend on the knowledge of the angle
between the camera and the object, therefore this method can
be applied to images taken with generic digital cameras and
eliminate the need for expensive laser rangers, moreover, A
three dimensional (3D) geometrical model has been
developed to determine the current pose of an object and
predict its future location based on FIR model learned by the
OLS.
The rest of the paper is organized as follows; section 2
presents image denosing technique used to improve the
quality of an image corrupted by a lot of noise. Section 3
illustrates color table thresholding segmentation technique to
extract the visual target. Section 4 shows the enhancement of
applying the morphological operations on the segmented
51
visual target. Section 5 deuces the center of gravity of the
tracked object. Section 6 explains the experimental results.
Finally section 7 summarizes the presented research work.
II. Image Preprocessing
Image denoising is a key issue in all image processing
techniques. It could be considered as the first preprocessing
step in dealing with image processing. It is used to improve
the quality of an image corrupted by a lot of noise due to the
undesired conditions of image acquisition phase. The great
challenge of image denoising is how to preserve the edges and
details of an image when reducing the noise. Noise can occur
during image capture, transmission or processing, and may be
dependent on or independent of image content. Noise is
usually described by its pattern and by its probabilistic
image with salt and pepper image with Gaussian noise
False tracking bad tracking
restored image from salt and restored image from
pepper nose Gaussian noise
Fig. 1. The effected of the denoising on the visual tracking
III. Segmentation
To track a visual target we have to relay on a segmentation
technique. Segmentation involves partitioning an image into a
set of homogeneous and meaningful regions, such that the
pixels in each partitioned region possess an identical set of
properties or attributes. These sets of properties of the image
may include gray levels, contrast, spectral values, or textural
properties. The result of segmentation is a number of
homogeneous regions, each having a unique label. An image
is thus defined by a set of regions that are connected and
nonoverlapping, so that each pixel in the image acquires a
unique region label that indicates the region it belongs to. The
set of objects of interest in an image, which are segmented,
undergoes subsequent processing, such as object classification
and scene description
There may exist a number of possible partitions, but the
selection of an appropriate set of regions depends on the
52
characteristics. There is a wide variety of noise types while
we will focus on the following types; Gaussian noise, speckle
noise, poison noise, impulse noise, salt and pepper noise. A
robust image Improvement technique has to suppress the
noise while preserving natural information in the images. A
large number of linear and non linear filtering algorithms [9]
have been proposed to remove noise from corrupted image to
enhance image quality. At this frame work we applied
adaptive to eliminate the noise to improve the visual tracking.
This is nonlinear filter where his response is based on
ordering (ranking). Its edge-preserving nature makes it useful
in cases where edge blurring is undesirable. As shown in
figure (1) there is false and bad tracking due to the presence
of the noise.
image with poison noise image with speckle noise
False tracking False tracking
restored image from restored image from
poison noise speckle noise
choice of the property association in the region [10-15]. In
object tracking several researchers focused on thresholding,
region based segmentation, region merging and splitting, edge
based segmentation, clustering and on hybrid techniques to
track a visual guide.
During this framework we employ a color table thresholding
segmentation technique to extract the visual target. Because
it's fast and need less computational time so it is suitable in
real time tracking. At this technique there are six threshold
values. Most images as two-dimensional arrays (i.e.,
matrices), in which each element of the matrix corresponds to
a single pixel in the displayed image. (Pixel is derived from
picture element and usually denotes a single dot on a
computer display.) Some images, such as true color images,
require a three-dimensional array, where the first plane in the
third dimension represents the red pixel intensities, the second
plane represents the green pixel intensities, and the third plane
represents the blue pixel intensities. According to the previous
knowledge we have six thresholds; max _red, max_green,
max_blue, min_red, min_green, min_blue. During motion the
color model deformation may happen due to the change in
illumination. It is beneficial to segment the image precisely to
track the object. The main goal of the vision system is to
extract the object (guide) and to estimate its range
geometrically. In this paper tracking homogenous/
inhomogeneous colored objects is studied. The purpose is to
select the appropriate color model for these two types of
objects.
Visual guide extraction is achieved by applying the following
steps 1) select the appropriate color model, 2) define the color
of the guide, 3) apply a Gaussian filter to eliminate the noise
of the image 4) extract the object based on thresholding, 5)
deduce the x and y projections of the guide’s center of gravity
COG [16, 17, 18].
A.. Selection of an Appropriate Color Model
Although many of defined color spaces exist, RGB and HSV
are the most commonly used models. The RGB color space
represents all colors as a mixture of red, green and blue which
constitute the primary colors used by video cameras,
televisions and PC monitors. When combined, these colors
can create any color on the spectrum. The RGB is too
sensitive with respect to change of illumination while the
HSV is unstable with the gray scale spectrum. To avoid these
problems we decided to rely on the YCbCr color model. The Y
RGB color table thresholding (homogenous)
RGB color table thresholding (inhomogeneous)
Fig.2. RGB and YCbCr based tracking under illumination changes
IV. Deduction of Guide’s COG
To track a guide dynamically its center of gravity is calculated
and the results are delivered to the control system to steer the
robot adaptively, see figure (4). Compared with contour
detection presented in [19, 20 and 21] the COG is more
accurate because the object to be tracked is represented with a
single point in 3D dimensions. Moreover, the elapsed time
required to extract the geometrical LUT of the object is
shorter. This representation is easy to be manipulated in
control system as a definite feedback, see figure (4). The
COG is calculated according to the following:
n
¦x i 1
i
(4)
xc
n N
n
¦y
i 1
n
(5)
yc
n and N is the projection size. To reject the noise a threshold
53
in the YCbCr denotes the luminance component, and Cb and
Cr represent the chrominance factors.
To analyze and process images in color, computer vision
systems typically use data from either the RGB or YCbCr
color spaces, depending on a given task’s complexity. For
example, in simple applications such as tracking highly
saturated homogeneous colored guides it is enough to use the
RGB model. With more complex applications, however, such
as persons tracking based on the skin color under changing
illumination, a vision system may require YCbCr or
normalized RGB information to perform the operation.
In short, the RGB model is suited for image color generation,
whereas the YCbCr model is suited for image color
description. The conversion from RGB to YCbCr can be
calculated by the following;
ªY º ª0.257 0.504 0.098 º ª R º ª16 º
«C » « 0.148  0.291 0.439 » «G »  «128»
« b» « »« » « »
«¬Cr »¼ «¬0.439  0.368  0.071»¼ «¬ B »¼ «¬128»¼
As shown in figure (2) that both RGB and YCbCr are
successful in dealing with highly saturated homogenous
objects even under change of illumination and there is no
need to use the YCbCr color model with more computational
burden. For tracking of inhomogeneous colored targets, face
skin color, the YCbCr model is successful in under change of
illumination while the RGB fails to achieve this target.
YCbCr color table thresholding (homogenous)
YCbCr color table thresholding (inhomogeneous)
Original image X=191.897, Y=108.563
Fig. 4. Center of gravity (COG) of a target
V. Geometrical Calibration
According to figures (5, 6) which show the tracked object at
different distance. We can deduce the distance between the
camera and the guide mathematically using the following
form:
ae bD (6)
Where, D is the distance between the robot and the guide a, b
is a constant depending on the geometry of the camera’s lens
number is used to delimit the projection size. From the size of
the object N we estimate the distance D to the guide. Then by ªy t  n  1 º
using the least square approximation (LS) we obtained the « »
coefficient data, a = 30606.621, b=-0.03410108. «y t  n  2  »
To calculate the pose of the guide in 3D dimensions y  «« y t  n  3 »» (11)
geometrically, first we use the COG rule to get x and y axes «# »
parameters COG (x), COG (y). Then, we use the range « »
calibration method to calculate the z axis parameter, range D ¬«
y t  ¼»
as shown in Figure (7).
ªu t  n  1 º
VI. Motion Estimation and Prediction based on FIR-OLS « »
technique «u t  n  2  »
)  ««u t  n  3 »»
In real time the computation complexity is a vital issue to
overcome this problem we use the FIR structure. For a single- (12)
input/single-output system (SISO), the FIR model structure is «# »
« »
«¬u t 
presented by the discrete equation, see figure (8)
n »¼
y t  ¦ au t  i   e t  aT u t   e t  (7)
i 1
ªe t  n  1 º
« »
Where, y (t) represents the output at time t, u (t) represents the «e t  n  2  »
input at time t, and e (t) is output additive noise that is i.i.d. e  ««e t  n  3 »» (13)
(identically, independently with 0 mean and variance V2;
«# »
>a1 a3 "an @
T
a a2 (8) « »
«¬e t  »¼
Is the FIR model kernel; and
u t  ¬ªu t  1 u t  2  u t  3 " u t  n  ¼º
T
(9) The OLS solution to the above problem is found by solving
Is the vector of lagged inputs. the standard least-squares
Problem
 y  )a   y  )a 
T
VII. FIR Identification using OLS min (14)
a
With N observations, input-output data can be written in
Whose solution is the estimate of the FIR model kernel:
matrix form as
) )
1
y )u  e (10) aˆ T
)T y  R 1)T y (15)
Where
Where, R  )T ) The bias of the OLS estimator â

Dist=30 Dist=40 Dist=50 Dist=60 Dist=70

Dist=90 Dist=100 Dist=80 Dist=110 Dist=110

Fig. 5. object tracking at different distance

54
 Fig. 6. The relation between range (D) and projection Fig.7. The relation between the range and location of the
size (N) object in 3D domain

Where, a is the true FIR model coefficient vector, and E is the

expectation operator.
Similarly it can be easily shown that the variance of the OLS
estimator â is
E « a  aˆ  a  aˆ T » V 2 R 1 (17)
¬ ¼
Fig. 8. FIR model structures

and the mean square error for the OLS estimator is

E >a  aˆ @ E « a  aˆ   a  aˆ  » V 2trace  R 1 
T
b ¬ ¼ (18)

a  E >aˆ @
Figure (9) shows the different models w.r.t. the actual output.
a  E ª¬ R 1)T y º¼ Figure (10) shows the auto correlation of all model . Figure
(11) shows the output of the FIR model w.r.t. the actual
a  R 1)T E [)a  e ] (16)
output , Figure (12) shows the the capability of the model to
a  R 1)T )  E >a @  E e   predict the output if the system input is known.

a  R 1Ra
0

Fig. 9. The different models w.r.t. the Fig. 10. The auto correlation of all model
actual output

Fig.11 .Model output w.r.t. system output Fig.12 Model output w.r.t. system output

55
7. Conclusion
Throughout this framework we present an idea to convert the
normal visual cameras to a quasi-precise ranging sensor. We
proposed a new robust computationally efficient approach to
build a 3D geometrical model of the visual tracking process.
A three dimensional (3D) geometrical model has been
developed to determine the current pose of an object and
predict its future location based on FIR model learned by the
OLS. The applied visual tracking algorithm exhibits higher
reliability and achieves the goaled robustness. We could
eliminate the effects of noise and change of illumination
significantly. In contrary to the dominant idea about the RGB/
YCbCr the RGB color model is a stable and fast technique in
recognition of homogeneous saturated color targets even in
presence of change of illumination while the YCbCr is
efficient with inhomogeneous visual targets such as face
tracking on an embedded system to develop a visual. As a
future work, we are going to implement the applied algorithm
on An embedded system to develop a visual RADAR System
8. References
[1] S. Harasse, L. Bonnaud, M. Desvignes, "Finding
People in Video Streams by Statistical Modeling", 3rd
ICAPR Int'l International Conference on Advances in
Pattern Recognition, pp 608-617, Bath, August 2005.
[2] X. Liu and T. Chen, "Online Modeling and Tracking of
Pose-Varying Faces in Video," Video Proceedings of
IEEE International Conference on Computer Vision
and Pattern Recognition 2005.
[3] Payman Haqiqat, "Using Image Moments for Tracking
Rotating Objects", ICGST international Conference on
Automation, Robotics and Autonomous Systems,
ARA05, 19-21 December 2005.
[4] M. Wimmer, B. Radig, "Adaptive Skin Color
Classificator", ICGST international conference on
Graphics, Vision and Image processing, CICC, Cairo,
Egypt, 19-21 December 2005.
[5] Parker, J.R. (1991) Grey Level Thresholding in Badly
Illuminated Images, Transactions on Pattern Analysis
and Machine Intelligence, Vol. 13 No. 8 813-819.
[6] M.Sonka, Vaclav Hlavac & Roger Boyle (1993).
Image Processing,Analysis and Machine Vision.
Chapman & Hall Computing 1st Edition pp 112-178.
[7] Cheriet M., Said J. N. and Suen C. Y. (1998) A
recursive thresholding technique for image
segmentation, IEEE Transactions on Image Processing,
vol. 7, no. 6, pp. 918-920.
56
[8] Ohlander R. B, (1975). Analysis of natural scenes,
Ph.D. dissertation, Department of Computer Science,
Carnegie-Mellon University, Pittsburgh, PA.
[9] Zack, G.W., W.E. Rogers, and S.A. Latt (1977)
Automatic Measurement of Sister Chromatid Exchange
Frequency.. 25(7): p. 741-753.
[10] P. Rosin (2001). Unimodal Thresholding. Pattern
Recognition 34 (11), 2083-2096.
[11] Pavlidis, T. (1977). Structural Pattern Recognition.
Springer Verlag, Berlin, Heidelberg, New York.
[12] Prager J.M. (1980) Extracting and labeling boundary
segments in natural scenes, IEEE Transactions on
Pattern Analysis and Machine Intelligence, vol. 2, no.
1, pp. 16-27.
[13] Pal N. R. and Pal S. K. (1993). A review on image
segmentation techniques,” Pattern Recognit., vol. 26,
pp. 1227–1249.
[14] Rafael Gonzalez Richard Woods, Digital Image
Processing, Pearson Publications.
[15] Hanzi Wang and David Suter, "A Model-Based Range
Image Segmentation Algorithm Using a Novel Robust
Estimator", 3rd Int'l Workshop on Statistical and
Computational Theories of Vision (in conjunction with
ICCV'03), Nice, France, October 2003.
[16] Ashraf Aboshosha, "Adaptive Navigation and Motion
Planning for Autonomous Mobile Robots", Ph.D.
thesis, University of Tuebingen, 2004.
[17] R. Aufrere, C. Mertz, and C. Thorpe. "Multiple Sensor
Fusion for Detecting Location of Curbs, Walls, and
Barriers" , In Proceedings of the IEEE Intelligent
Vehicles Symposium (IV2003), 2003
[18] T. Belker, M. Beetz, and A. B. Cremers, "Learning of
Plan Execution Policies for Indoor Navigation" , AI
Communications, 15(1):3–16, 2002.
[19] Alexandre Bernardino, Jose’ Santos-Victor, and Giulio
Sandini. Foveated active tracking with redundant 2d
motion parameters. In Robotics and Autonomous
Systems, volume Vol. 39. Elsevier Science, June 2002.
[20] Darius Burschka, Jeremy Geiman, and Gregory D.
Hager. Optimal Landmark Con.guration for Vision-
Based Control of Mobile Robots. In Proc. of
International Conference on Robotics and Automation
(ICRA), 2003.
[21] Darius Burschka and Gregory D. Hager. Principles and
practice of real time tracking on consumer hardware .
In Tutorial 1 at IEEE VR2003: Recent Methods for
Image-Based Modelling and Rendering, March 2003