2014 5th International Conference on Information and Communication Systems (ICICS)

A Compact Portable Object Tracking System

Karam Abughalieh,Waleed Qadi,Karam Melkon, Boulos Fakes, Belal Sababha, Amjed Al-mousa
King Abdullah II Faculty for Engineering, Princess Sumaya University for Technology
Amman 11941, Jordan
k.ghalieh@psut.edu.jo

Abstract - Computer vision and object tracking are systems, while the second gives a brief background of the
becoming increasingly more important with a wide variety of object tracking algorithms that were used in this work.
applications in our daily life. Most of the available tracking
systems are not compact enough to be mounted on small A. Commercial off The Shelf (COTS) Tracking Systems
ground or aerial robots. Also, most of these systems are Moog quickset has a video tracking system (Model 7-
relatively expensive. The tracking system presented in this 08625)[1] that is designed to track boats, aircrafts, vehicles
work is a compact low cost system which makes it suitable for
and people. Its algorithm has been modified for the tracking
weight sensitive applications. The proposed system is a
commercial off the shelf android-based mobile device that is
of these objects in the sky or on water. This comes at the
mounted on a pan/tilt gimbal. The system utilizes the camera expense of extra complexity, weight and cost.
and the processor of the mobile device to capture and process In order to achieve this, two high end cameras positioned
video frames. The tracking algorithm is a newly modified on a quickset pan/tilt base were used. One of which is a
algorithm that combines three well known algorithms: SURF, thermal camera and the other is a visible camera, both are
CAMShift, Lucas-Kanade. Each of these algorithms is connected to a PC through a video tracking server/power
deployed at a different stage of the tracking process which
and controlled via a rugged joystick connected to the PC.
yields a reliable real-time tracking system. The newly modified
Fig.1, illustrates the system [1].
tracking algorithm was developed using OpenCV within an
Android environment. An indoor lab experimental test showed
that the system was able to track a (3cm x 5cm) object moving
at a speed of 133cm/sec and placed 50 cm away from the
system.

Keywords—Computer Vision, object tracking, tracking

algorithms.

Fig. 1. Moog Quickset Video Tracking System

I. INTRODUCTION
In this paper we present a compact tracking system that
uses an android-based smart phone to capture and analyze
images. The system also uses a hybrid of multiple image
processing algorithms to enhance the capture time and
tracking.
Section II of the paper discusses existing tracking
systems and tracking algorithms. In section III we discuss
the design of our system showing the multiple components
and both the hardware and software designs. Section IV
shows the experimental setup used to test the algorithm. It
also discusses the results measured. Finally, the paper is
concluded in section V.
II. BACKGROUND
Video tracking technology has been around for a while
now and it is developing at a fast pace. This section includes
two sub-sections. The first reviews commercial off the shelf
Another system is the RV2 video capture and tracking
system. Fig.2, shows the system. This system provides
digital video recording and real-time tracking [2]. It also
includes features like auto exposure, auto gain and auto
white balance. Such features are considered to be of high
cost features to include in a tracking system. Even though it
has a relatively smaller spatial consumption, it is still not
compact enough to be mounted on an aerial robot.
To solve the compactness, portability and cost
constraints we aimed to use a compact, flexible, highly
modifiable and low cost system. Our video tracking system
consists of small and cheap components such as a smart
phone, for the processing, a cheap microcontroller and an
open source library (openCV [9]) for implementing the
computer vision algorithms. Combined together these
provide the desired high performance under the
aforementioned constraints.

978-1-4799-3023-4/14/$31.00 ©2014 IEEE

 2014 5th International Conference on Information and Communication Systems (ICICS)

GalaxyS2, GT-I9100). The 8 mega pixel camera of the

phone is the main input device of the system. The phone
camera captures live video of the object that is being
tracked. A camera with that resolution will provide enough
key points for our algorithms to keep the object locked
without losing it.

Fig. 2. RV2 Video Capture and Tracking System

B. Tracking Algorithms
To perform video tracking we need to get sequential
video frames and process them using a tracking algorithm
which analyzes the movement of the object of interest
between the sequential frames. There are two major
requirements that a visual tracking system has to satisfy:
identifying the object and its location, as well as filtering
any undesired elements. Several tracking algorithms have
been developed, each one has strengths and weaknesses,
depending on its use case.
Fig. 3. High Level Hardware Design
SURF [3] (Speeded Up Robust Features) is a tracking
algorithm that is categorized as a feature matching
algorithm. It is one of the most reliable algorithms in this
category [4]. SURF selects interest points of an image, and
then builds local features based on histograms of gradient- With a frame rate of 30 frames per second, the system
like local operators. However SURF consumes a lot of can track a mid-speed moving object. The phone processor
system resources, which makes it inapplicable for real time will handle the entire image processing needed for the
tracking with limited resources. tracking system. The phone processor is an Arm Cortex A9
CAMShift (Continuously Adaptive Mean Shift) [5] is a Dual Core Processor clocked at 1.2 GHz.
tracking algorithm that is categorized as a Kernel-based The actuation function of the system is implemented via
tracking algorithm [6]. This matching algorithm works on an Arduino Uno [8] microcontroller communicating serially
similarities between frames based on color histograms. The with the phone. The microcontroller will control a
algorithm does not consume system resources, which makes mechanical gimbal composed of two servo motors that will
it suitable for real time tracking. On the other hand, be responsible for moving the entire tracking system setup.
CAMShift cannot be used for shape detection. Also due to Fig.4, illustrate the pan/tilt gimbal setup, while Fig.5
changes in lighting CAMShift may lose track of the desired demonstrates the electrical circuit design.
object.
Another known algorithm is Lucas Kanade. It is an
optical flow algorithm [7]. This algorithm keeps tracking of
object edges between frames. Lucas Kanade algorithm does
not consume a lot of resources for low detailed
backgrounds. However, in order to keep locking the object
in a high detailed background, it will require additional error
computations and filtering.

III. SYSTEM DESIGN

A. Hardware Design
Fig.3, illustrates the high level hardware design. The
system functions integrated all together, achieve the overall Fig. 4. Mechanical Gimbal Design
tracking system. These three functions are: sensing,
processing and actuation. In the presented design, the
sensing and processing functions are performed through
using a commercial android-based smartphone (Samsung
 2014 5th International Conference on Information and Communication Systems (ICICS)

Fig. 5. Electrical Circuit Design

As shown in Fig.4, one of the servo motors of the gimbal

is responsible for moving the entire setup radially along a
fixed base, while the other servo handled rotating the
smartphone in tilt position with the gimbal.
B. Software Design
The algorithm is designed to accurately detect a
specified object and track it in real time with minimum
resources. The algorithm is a combination of three
algorithms to overcome the disadvantages of each algorithm
by its own. The flowchart shown in Fig.6 illustrates the
algorithms.
The algorithm uses SURF specifically for acquiring the
lock on the target. SURF compared to other algorithms is
fast at acquiring the lock but it does not perform well in
keeping track of an object. That's why Lucas Kanade and Fig. 6. Algorithm Flow Chart
CAMShift are used. The CAMShift algorithm will use the
mean value that is provided by Lucas Kanade to localize the
object, and provide the edges of the object. Lucas Kanade
will track the sharp edges that are provided by CAMShift in
order to update the object's color mean value.
The following steps explain the details of the system
workflow. It is shown how the three algorithms SURF,
CAMShift and Lucas Kanade are used to produce the new
tracking algorithm.
1) A specific object is provided to the algorithm as
an image. SURF will detect the features of the desired
object in the image and extract the object’s key points which
will be matched later with moving frames. Fig.7, shows
SURF extracting the key points from the object’s image.
Fig. 7. SURF extracting the object’s key points
2) SURF will start detecting features and extracting
key points from each frame until it acquires a good match
with respect to the key points extracted from the image
provided to the system earlier. Using the good matches from
SURF, the algorithm will localize the object in the frame
using homography to draw the bounding rectangle as seen in
Fig.8.
 2014 5th International Conference on Information and Communication Systems (ICICS)
Fig. 8. Match localizing the object in the frame
Using the dimensions and position of the bounding
rectangle, a ROI (Region of Interest) will be extracted from
the current frame. A mask is created based on the extracted
ROI which will isolate the object, create an image out of it
and cover all of the undesired elements in the frame. This
step is illustrated in Fig.9.
Fig. 9. Creating a mask based on the extracted ROI
The isolated image is converted into HSV color space,
which will be used to calculate the object’s histogram as
shown in Fig.10. The histogram is used to calculate the
mean value of the colors in the isolated image. The mean
value is then used by the CAMShift algorithm to maintain
lock on the object.
Fig. 10. Object’s Histogram
3) The CAMShift algorithm uses the mean value
and the position of the ROI to keep localizing the object in
each frame and find the sharp edges of the object to be used
by Lucas Kanade algorithm.
4) The Lucas Kanade algorithm works in parallel
with the CAMShift algorithm and uses the sharp edges
defined by CAMShift to update the object’s color mean
value in each frame so that they can be used by CAMShift
in the next frame.
Fig. 11. Lucas Kanade working in parallel with the CAMShift algorithm
5) In each frame the distance from the center of the
ROI and the center of the frame is calculated (X, Y), see
Fig.11. The X and Y values found in this step are sent to the
microcontroller through a serial link.
6) The Arduino microcontroller receives the X and
Y values and scales each of them to an angle between 0 and
180 degrees. These angle values will be later converted to
Pulse Width Modulation (PWM) signals to control the
rotation angles of the two servo motors of the gimbal.
IV. EXPERIMENTAL SETUP AND RESULTS
In order to compare the newly developed algorithm with
existing algorithms a test setup has been created in the
physics lab. A tilted bar with an air cushion is used to slide
the desired object on top of the tilted bar, see Fig.12. The
tracking system is placed 50 cm away from the setup. As the
object slides down, the algorithm’s ability to keep track of
the object is monitored. Multiple tests with different speeds
have been performed. The setup has a limitation on the
maximum speed, which was 4.7 Km/h.
Fig. 12. Tilted bar with an air cushion
A. Speed Experiment Results
Multiple tests were conducted to see how well the new
algorithm performed against CAMShift and Lucas Kanade
in terms of tracking objects with various velocities. In each
of these tests the speed of the object was increased gradually
by allowing the object to travel further down the bar. These
experiments were conducted after each algorithm acquired
the lock on the target. The object is allowed to travel longer
distance in each test. The results are shown in Table 4.1
below.
 2014 5th International Conference on Information and Communication Systems (ICICS)

TABLE 4.1: SPEED EXPERIMENT RESULTS

Distance (cm) 40 cm 90 cm 100 cm 130 cm REFERENCES

Avg Speed(cm/s) 67 81 105 132 [1] MOOG QuickSet, "Video Tracking System,"
http://www.quickset.com/literature/Space_Defense/QuickSet/
New Algorithm 9 9 9 9 Video_Tracking_System.pdf
CAMShift 9 9 9 9 [2] Tucker Davis Technologies, "Video Capture & Tracking
System," http://www.tdt.com/products/RV2.html
Lucas–Kanade 9 9 Fail Fail [3] Bay, Herbert, et al. "Speeded-up robust features
(SURF)." Computer vision and image understanding 110.3
(2008): 346-359.
It is clear from the table that the Lucas-Kanade [4] Juan, Luo, and Oubong Gwun. "A comparison of sift, pca-sift
algorithm lost track of the object in both the 100cm and 130 and surf."International Journal of Image Processing
cm tests. Meanwhile, since the new algorithm uses (IJIP) 3.4 (2009): 143-152.
CAMshift mostly to track the object after acquiring lock, it [5] Allen, John G., Richard YD Xu, and Jesse S. Jin. "Object
can be seen that both of them were able to successfully track tracking using camshift algorithm and multiple quantized
the object in all cases. However CAMshift cannot acquire feature spaces." Proceedings of the Pan-Sydney area
lock on object by its own. workshop on Visual information processing. Australian
Computer Society, Inc., 2004.
B. Lock Time Experiment Results [6] Prajna Parimita Dash, Dipti patra,Sudhansu Kumar
Mishra,Jagannath Sethi, " Kernel based Object Tracking using
In order to calculate each algorithm’s speed in acquiring Color Histogram Technique." International Journal of
the lock on the object, another experiment is conducted. The Electronics and Electrical Engineering, Vol. 2, Issue 4, April
experiment involves placing the desired object in a well lit 2012.
area and on a stable surface. Then the time needed for each [7] Baker, Simon, and Iain Matthews. "Lucas-kanade 20 years
algorithm from the moment it is fired up till the moment it on: A unifying framework." International Journal of
acquires the lock on the target is recorded. There are four Computer Vision 56.3 (2004): 221-255.
algorithms involved in acquiring lock: SURF, BRISK [10], [8] Arduino Uno official website, found at:
SIFT [11] and FAST [12]. arduino.cc/en/Main/arduinoBoardUno
[9] OpenCV official website, found at: http://opencv.org/
The results of the experiment are shown in Table 4.2. It [10] Leutenegger, Stefan, Margarita Chli, and Roland Y. Siegwart.
can be seen that even though SURF consumes a lot of "BRISK: Binary robust invariant scalable
system resources, the results shows that it is the fastest keypoints." Computer Vision (ICCV), 2011 IEEE
compared with other less resource intensive algorithms. International Conference on. IEEE, 2011.
[11] Lowe, David G. "Distinctive image features from scale-
TABLE 4.2: ACQUIRING LOCK TIME COMPARISON invariant keypoints."International journal of computer
vision 60.2 (2004): 91-110.
Algorithm Time to lock on (sec) [12] Rosten, Edward, and Tom Drummond. "Machine learning for
high-speed corner detection." Computer Vision–ECCV 2006.
SURF 0.52
Springer Berlin Heidelberg, 2006. 430-443.
BRISK 3.635
SIFT 4.761
FAST 2.492

V. CONCLUSION
In this paper we have presented a compact object
tracking system. The system utilized the image processing
and computing power of commercial smart phones along
with servo motors and an Arduino microcontroller unit. The
system uses a new algorithm to acquire lock and keep track
of the target object. The algorithm uses SURF to acquire
lock on the target, and then uses a hybrid of Lucas Kanade
and CAMShift to keep track of it. As can be seen the new
algorithm has the best time to acquire the lock on the target
because it uses SURF, and it efficiently keeps track of the
object as it accelerates as it uses CAMshift and Lucas–
Kanade for this purpose.