Pattern Recognition 41 (2008) 25122524

www.elsevier.com/locate/pr
Ahierarchical approach for fast and robust ellipse extraction
F. Mai

, Y.S. Hung, H. Zhong, W.F. Sze

Department of Electrical and Electronic Engineering, The University of Hong Kong, Pokfulam Road, Hong Kong
Received 16 June 2007; received in revised form 5 December 2007; accepted 28 January 2008
Abstract
This paper presents a hierarchical approach for fast and robust ellipse extraction from images. At the lowest level, the image is described as a
set of edge pixels, from which line segments are extracted. Then, line segments that are potential candidates of elliptic arcs are linked to form arc
segments according to connectivity and curvature conditions. Next, arc segments that belong to the same ellipse are grouped together. Finally,
a robust statistical method, namely RANSAC, is applied to t ellipses to groups of arc segments. Unlike Hough Transform based algorithms,
this method does not need a high dimensional parameter space, and so it reduces the computation and storage requirements. Experiments on
both synthetic and real images demonstrate that the proposed method has excellent performance in handling occlusion and overlapping ellipses.
2008 Elsevier Ltd. All rights reserved.
Keywords: Ellipse extraction; Elliptic arcs; Arc segments; RANSAC
1. Introduction
Circular and elliptical objects are frequently seen in the real
world. An ellipse or a circle in 3Dspace is projected to an ellipse
in an image. Therefore, the extraction of ellipses from images
is a key problem in computer vision and pattern recognition.
Applications include determination of the 3D position of the
object or the camera pose [1], and recognition of trafc signs
or manufactured objects in industrial environments [24].
All existing approaches for detecting ellipses in an image
start from extracting edge pixels from the image, and then
ellipses are tted to the edge pixels. Generally they can be
grouped into three classes: Hough Transform (HT) based algo-
rithms [513], GeneticAlgorithm(GA) based methods [1418],
and edge-following algorithms [1923].
HT is one of the most widely used approaches for detecting
parametric curves or other predened shapes from a planar set
of points. The Duda and Harts formulation of the HT [5] is
commonly referred to as the Standard Hough Transform (SHT),
in which each of the edge points votes for the parameters,
followed by a search for peaks in the quantized parameter space.

Corresponding author. Tel.: +852 2859 2711; fax: +852 2559 8738.
E-mail addresses: feimai@eee.hku.hk (F. Mai), yshung@eee.hku.hk
(Y.S. Hung), hzhong@eee.hku.hk (H. Zhong), wfsze@eee.hku.hk (W.F. Sze).
0031-3203/$30.00 2008 Elsevier Ltd. All rights reserved.
doi:10.1016/j.patcog.2008.01.027
In spite of its popularity owing to its simple theory of operation
and its robustness to occlusion, the SHT is largely impractical
for ellipse extraction due to its heavy computational burden and
large storage requirement.
In order to detect ellipses effectively and quickly using HT,
many variations have been suggested. One way is to use a sub-
set of the edge points for voting rather than all of them, such as
the Randomized Hough Transform (RHT) [6] and Probabilis-
tic Hough Transform (PHT) [7]. In Ref. [8], the RHT shows its
improvements over SHT on accuracy and computational com-
plexity, but it is easy to nd examples where the RHT cannot
detect all obvious ellipses in the scene, and as the number of
ellipses increases, the accuracy gets poorer. The PHT works
well with the assumption that the pixels of the edge points are
distributed evenly throughout the image, but this condition is
hard to be satised in real images. Furthermore, PHT needs a
context-dependent criterion to stop sampling.
Another way to overcome the excessive time and space
requirements for HT-based ellipse extraction methods is to
decompose the ve-dimensional parameter space into several
sub-spaces of lower dimensions. The decomposition is achieved
by exploiting geometric characteristics of the ellipses, such as
slope [9,10] and symmetry [11,12]. These properties help to
accelerate the process, but they constrain the application at
the same time. For example, it is quite often the case that the
F. Mai et al. / Pattern Recognition 41 (2008) 25122524 2513
ellipses in an image are occluded so that the symmetry charac-
teristic is no longer available.
The second group, namely GA-based methods, extracts el-
lipses by solving an optimization problem [1418]. Although
GA-based techniques might be effective for detecting ellipses
with partial occlusion, unfortunately, they consume excessive
computational time. Furthermore, an inherent weak point of GA
is that there is always a risk of nding a suboptimal solution.
The third group is the edge-following methods. They require
an input of lists of chained edge points produced from the orig-
inal edge image after edge thinning and linking operations. The
UpWrite method [19] belongs to this group, and is robust for
images containing perturbation noise and not computationally
demanding, but it is sensitive to the resolution radius. Further-
more, according to the experiments reported in Ref. [20], it
may have difculties for overlapping features that obscure the
ellipses. Kim et al. [20] proposed a fast and robust ellipse ex-
traction method. Although the method can extract ellipses from
complicated images, it depends on the tangent information,
which is quite sensitive to noise in digital images. In Ref. [21],
Rosin proposed a nonparametric technique which segments a
digital arc into short line segments and elliptical arc segments,
and Ji et al. [3] merged these short segments by a statistical
method. However, they limit the candidate segments for merg-
ing to those who are close together at least at one side. This
limitation causes difculties when there is occlusion separat-
ing two arc segments far apart at both sides. Furthermore, the
assumption underlying the statistical model may not always be
valid for real images. In Ref. [22], the edge-following technique
is combined with HT. They detect a circle osculating to an edge
segment by the HT, nd edge pixels tangent to it, ret an ellipse
to the detected pixels, and repeat this process. This approach
works well for an image containing a single ellipse, but it may
have difculties for scenes containing multiple ellipses.
Note that the edge-following based methods extract ellipses
in a bottom-up way. Contrary to the HT-based methods which
view the whole image globally, these bottom-up methods are
based on local features and pay little attention to global infor-
mation.
The work presented here is an extension and improvement
on that presented in Ref. [24]. It is based on the idea of edge
following, but the difculty due to the lack of global view in
existing methods is addressed by means of a new hierarchical
approach. In this framework, the image is described as a set
of edge pixels at the lowest level, from which line segments
are extracted. Then line segments that are potential candidates
of elliptic arcs are linked to form arc segments according to
connectivity and curvature conditions. Next, arc segments that
belong to the same ellipse are grouped together. Finally, a mod-
ied RANSAC (RANdom SAmple Consensus) approach [25]
is applied to t ellipses robustly. Experimental results demon-
strate that our method is capable of accurately extracting mul-
tiple ellipses from an image with complex background, even
when the ellipses are partially overlapped or occluded.
This paper is organized as follows. In Section 2, we de-
scribe the proposed hierarchical approach consisting of four
different stages. In Section 3, experimental results on both syn-
thetic data and real images are provided to illustrate the perfor-
mance of our method in comparison with ve existing methods.
Section 4 contains some concluding remarks.
2. The hierarchical approachfrom edge points to ellipses
In this paper, we adopt a hierarchical approach to ellipse
detection, generating progressively higher level descriptions of
the data. The hierarchical algorithm, as shown in Fig. 1, consists
of four stages:
(i) Line segment extractionat the lowest level, the image is
described as a set of edge pixels, fromwhich line segments
are extracted.
(ii) Line segment linkingline segments that are potential
candidates of elliptic arcs are linked to form arc segments.
(iii) Arc segment groupingarc segments that belong to the
same ellipse are grouped together.
(iv) Ellipse ttingmodied RANSAC [25] is used to ro-
bustly t ellipses to arc segment groups.
The detailed operations in each stage are described in the
following subsections.
2.1. Line segment extraction
The rst stage, line segment extraction, is to represent the
boundaries of the objects in the images by piecewise linear
segments. There are many methods for line extraction [2629].
Ellipse fitting
Input image
Edge points
Edge detection
Line segments
Line segment extraction
Elliptic arc segments
Line segment linking
Groups of arc segments belonging to the
same ellipse
Arc segment grouping
Extracted ellipses
Fig. 1. The four stages in the proposed hierarchical approach for ellipse
extraction method.
2514 F. Mai et al. / Pattern Recognition 41 (2008) 25122524
We will employ a pixel-connectivity edge-linking based method
[26]. First, Canny edge detector [30] is used to nd edges with
hysteresis thresholds of 0.1. Secondly, edge pixels are linked
together into lists of sequential edge points with one list for
each edge contour segment of at least 10 pixels long. A contour
segment starts (or stops) at an open end or a junction of more
than two contours. Thirdly, each contour segment is approxi-
mated by a series of piecewise linear segments by the iterative
end-point t algorithm [31]. Finally, a line segment cleaning
process is performed to remove line segments shorter than a
minimum length (of 10 pixels by default).
In our software implementation, we use Kovesis code
(available from http://www.csse.uwa.edu.au/pk/research/
matlabfns) for line segment extraction with a slight modica-
tion to output not only the extracted line segments, but also
their corresponding supporters. Here a supporter of a certain
feature (line segment, arc segment, or ellipse) is generally de-
ned as an edge point sufciently close to the feature where
closeness is measured by the Euclidean distance of the edge
point to the feature.
2.2. Line segments linking
2.2.1. Two conditions for linking
After line segment extraction, the boundaries of the objects
in the images are represented by line segments. The purpose
of the line segment linking stage is to link line segments into
arc segments, where an arc segment is a segment of an ellipse,
represented by a sequence of n(n3) connected line segments
satisfying the following connectivity and curvature conditions.
2.2.1.1. Connectivity condition. Given two line segments l
1
and l
2
each with two end points, the proximity d of the two line
segments is measured by the smallest of the four (Euclidean)
distances between their end points.
The connectivity condition is
d d
0
, (1)
where d
0
is a predened threshold for closeness. We will refer to
two line segments satisfying the above condition as connected
line segments. A sequence of n connected line segments is
dened to be an ordered n-tuple of line segments in which
each intermediate line segment is connected to its predecessor
in the n-tuple at one end and to its successor at the other end
and is otherwise not connected to any other line segment of
the n-tuple, whereas the rst and the last line segments may be
connected to each other at their free ends.
2.2.1.2. Curvature condition. If a sequence of connected line
segments traces an arc segment, the angle between two con-
nected line segments represents the curvature of the arc. This
angle should not be too small in order for the sequence of
line segments to represent a genuine curve, but yet not too
large so that the curve do not have a sharp turning point. Let
l
i
(i =1, . . . , n) be a sequence of connected line segments, and
let
i
(i =1, . . . , n 1) be the angle measured from l
i
to l
i+1
.
Then, the above curvature condition can be stated as

min
|
i
|
max
(i =1, . . . , n 1). (2a)
Furthermore, if the sequence of connected line segments rep-
resents a convex arc (which is the case for an elliptic arc), the
curvature should be of the same sign throughout the sequence.
Hence, a second condition for curvature is

1
, . . . ,
n1
are of the same sign. (2b)
The two curvature conditions mentioned above dene only a
loose criterion for a sequence of line segments to represent a
convex arc. To screen out convex arc that does not resemble
an elliptic arc, we t an ellipse to all the supporters of all the
line segments in the sequence l
i
(i =1, . . . , n), and impose an
additional supporter ratio criterion:
r >r
0
, (2c)
where
r =
m
M
(3)
is the ratio of the number of supporters supporting both the
tted ellipse and the line segments in the sequence (denoted
m) to the total number of supporters of all line segments in the
sequence (denoted M), and r
0
is a predened threshold.
If r attains its maximum value of 1, then all the line segments
in the sequence t an ellipse closely, and thus they should be
linked together as an elliptic arc segment. In practice, if r is
greater than a given threshold r
0
(=0.9 by default), we shall
accept the sequence l
i
(i =1, . . . , n) as a candidate elliptic arc
for further processing in the ellipse extraction process.
Note that by means of Eqs. (2a)(2c), we are applying in-
creasingly strict conditions on the curvature of the sequence of
line segments for it to represent an elliptic arc segment. This
strategy of progressively tightening the curvature condition en-
ables sequences that are unlikely to come from an ellipse to be
screened out in a computationally efcient manner.
2.2.2. The linking algorithm
The algorithm implementing the above process of linking a
sequence of line segments into an elliptical arc segment con-
sists of two steps: arc initialization and arc growing. In the arc
initialization step, a line segment pair satisfying the connectiv-
ity condition (1) and the curvature condition (2a) is identied.
Starting from this initial line segment pair, the arc segment is
grown in the second step by taking in line segments satisfy-
ing conditions (1) and (2a)(2c). This two-step process will be
repeated to get the arc segments one by one, as described in
Algorithm 1 below, with the following notations:
L the set of all line segments obtained in the line segment
extraction stage.
A
o
a set of line segments from which connected line segment
pairs are drawn for arc initialization.
F. Mai et al. / Pattern Recognition 41 (2008) 25122524 2515
A
e
a set of elliptic arc segments to be found satisfying
the connectivity condition (1) and curvature conditions
(2a)(2c).
Algorithm 1.
1. Set A
o
=L, A
e
=.
2. While A
o
= , repeat Steps 2.1 and 2.2:
(2.1) Arc initialization:
Choose any line segment l
1
A
o
. Find a line seg-
ment l
2
L connected to one end of l
1
satisfying (1) and (2a). If no such line segment exists,
remove l
1
from A
o
. Repeat this process
until s =(l
1
, l
2
) is found, or else stop if A
o
=.
(2.2) Arc growing:
(i) Given sequence s = (l
1
, . . . , l
i
) (i 2), identify
from L the set of line segments {l
i+1
}
connected to l
i
at the tail end (i.e., the end of l
i
not
connected to l
i1
) and satisfying
conditions (2a,2b,2c).
If {l
i+1
} = , select the segment with the highest
supporter ratio (3) as l
i+1
, and update
s =(l
1
, . . . , l
i
, l
i+1
).
Repeat this process until s cannot be grown any
further in length at the tail end.
(ii) Reverse the indices of s so that l
1
becomes the tail
end of the sequence, and repeat Step
2.2(i).
(iii) If the length of s 3, enter s into A
e
. In any case,
remove all line segments in s from
A
o
.
end while
3. Output the set of elliptic arc segments A
e
.
end of Algorithm 1
In step 2.2.(i), the highest supporter ratio condition is meant to
handle the case of a junction where more than one line segment
qualies for arc growing. Locking on the highest supporter ratio
may have the risk of nding suboptimal solution. An alternative
approach is to allow multiple solutions for arc growing when
we come to a junction point, but at the expense of a more
complex procedure for arc growing.
2.3. Arc segment grouping
Algorithm 1 detects all arc segments represented by se-
quences of line segments that are locally linked together hav-
ing the form of an elliptical arc. In this section, we look for
global grouping of arc segments that are not connected (e.g.
due to occlusion) but may come from the same ellipse. We
again apply the idea of supporter ratio for the purpose of arc
segment grouping. Specically, given a pair of arc segments,
s
1
= (l
1
1
, . . . , l
1
n
1
) and s
2
= (l
2
1
, . . . , l
2
n
2
), an ellipse is tted
to all the supporters of line segments of both s
1
and s
2
using
least-squares technique. Suppose the line segments of each of
the sequence s
i
(i =1, 2) has a total of M
i
supporters, among
which m
i
are supporters of the tted ellipse . Dene the
supporter ratio of each arc segment for the ellipse as
r
i
=
m
i
M
i
(i =1, 2). (4)
If both r
1
and r
2
are larger than the threshold r
0
, we consider
that the two arc segments belong to the same ellipse and merge
them into a group. Observe that the use of two supporter ratios
(one for each arc segment) is to avoid mistakenly merging a
short arc segment with a long one dominant in length even if
they do not belong to the same ellipse.
This process can be continued to merge further arc segments
into a group of arc segments as if the group is a single arc
segment. The extension is obvious.
2.4. Ellipse tting by modied RANSAC
Up to now we have grouped arc segments (with their sup-
porters) that presumably belong to the same ellipse. However,
due to noise and extraneous features in the image, some sup-
porters of the arc segments which do not belong to the ellipse
may have been included in the group. Hence, the supporters
of the arc segments can be classied as inliers that t the el-
lipse and outliers that do not. In order to extract the ellipse ac-
curately using only the inliers, we apply RANSAC [25] (with
some modications) to remove the outliers and recover the el-
lipse model.
Let D be the set of supporters of individual line segments
belonging to a group of arc segments presumably coming from
the same ellipse. Our problem is to t an ellipse robustly to
the points of D. Let D
I
be the set of inlier points that supports
the ellipse . In order to take into account line segments that
are part of but may have been missed out in our process
so far (e.g. due to occlusion, or the sequence of line segments
being too short to qualify as an arc segment), we expand our
search space and look for inliers for among the set E of all
supporters of all line segments in the image. Let E
I
be the
set of inlier points in E that supports the ellipse . Ideally, we
wish to maximize both E
I
and D
I
. However, when we search
for an E
I
with a larger size, it is possible that in practice the
size of D
I
may decrease slightly due to noise in the image. To
cater for this, we will use a modied RANSAC algorithm to
maximize the size of the inlier set E
I
, subject to the constraint
that the size of the inlier set D
I
is within a small margin
0
of
its maximum size. It is clear that this margin should be small,
so that RANSAC does not ignore points in D (supporting arc
segments presumably belonging to the same ellipse) in avour
of including more points from the global image. We will use
the following notations in Algorithm 2 below.

D
,
E
denote the estimated percentages of inliers in D and
E, respectively
n() denotes the number of points in a set ()
N
trial
denotes the number of trials required to obtain a set
of n inlier points from D with probability p (set to
0.95 in our experiments), given by
N
trial
=log(1 p)/ log(1
n
D
). (5)
2516 F. Mai et al. / Pattern Recognition 41 (2008) 25122524
l
2 l
3
l
4
l
5
l
6 l
7
l
1
Fig. 2. Example illustrating the reason for using two inlier percentages in
the modied RANSAC Algorithm 2.
Algorithm 2
Initialize N
trial
=, k =0, D
I
=, E
I
=,
D
=0,
E
=0.
While k N
trial
, repeat
(a) Select n(=5) edge points randomly from D and t an
ellipse to the chosen points.
(b) Determine the sets of supporters D
I
D and E
I
E
for the ellipse , and compute

D
=
n(D
I
)
n(D)
and

E
=
n(E
I
)
n(E)
(c) If

D

0
and

E
, then
if

D
, update
D
=

D
;
update
E
=

E
;
update N
trial
by (5).
(d) increase k by 1.
Re-estimate the ellipse using all the points in E
I
,
output and stop.
end of Algorithm 2
The main difference between Algorithm 2 and standard
RANSAC [25] is that we consider the inlier percentages (i.e.,

D
and
E
) in the local set D as well as the global image E,
whereas standard RANSAC would look for supporters of a
model only in the global image. Hence, condition (c) in Algo-
rithm 2 requires the global inlier percentage
E
to exceed the
current level of support, possibly at the expense of a slight re-
duction (by no more than
0
) in the local inlier percentage
D
in order for the new solution to be accepted as a better one. The
reasons for doing so can be illustrated by an example shown in
Fig. 2, where an ellipse represented by black dotted line exists
in the image. However, due to occlusion or deciencies of the
edge detector or line segment extractor, only line segments l
1
to l
4
are detected as an elliptic arc segment and line segments
l
5
to l
7
are detected as isolated line segments. As a result, l
5
to l
7
will not be linked to any group and (l
1
, l
2
, l
3
, l
4
) is the
only arc segment available for tting an ellipse. Therefore, an
ellipse will be tted to the arc segment in accordance with
Algorithm 2, where D includes supporters of l
1
to l
4
and E
contains supporters of all line segments in the image (including
l
5
to l
7
). If we consider maximizing only the inlier percentage

D
, then the green ellipse may be the optimal solution. How-
ever, if we take into account the inlier percentage
E
in the
global scene, then the dotted ellipse would be a better solution.
3. Experimental results
We perform experiments to evaluate the performance of the
proposed algorithm, which is implemented using Matlab. The
choice of the main parameters used in our method, their inu-
ence and the robustness of the method to these parameters are
explained below.
In the linking stage, the threshold for the connectivity condi-
tion is set at d
0
=5 pixels. If it is set too small, some line seg-
ments that should be linked together may be missed, whereas
if it is too large, there is a risk of including line segments that
are not really connected. Our experience is that the choice of
d
0
is not critical (e.g. any value in the range of 510 will do
for most of images we tested).
The thresholds for the lower and upper bounds of the cur-
vature condition (2a) are
min
= 0 and
max
= 0.5 rad. These
are reasonable choices as
min
should be small to link line seg-
ments representing elliptical arc with low curvature, and
max
should not be too large (say,
max
[0.5, 0.8]) to allow poly-
gons to be mistaken as ellipses. The threshold for detecting
supporters is 2 pixels and the threshold for supporting ratio in
curvature condition (2c) is r
0
=0.9.
The threshold
0
= 2% and p = 0.95 are used in Algorithm
2. The reason for choosing a small
0
is already explained in
Section 2.4 before Algorithm 2. A discussion of the inuence
of the choice of p can be found in Ref. [32]. The above values
are used as default parameter values in all the examples shown
in this paper, and they work well without the need of tuning,
which suggests that the proposed method is robust to the choice
of parameters.
To compare the accuracy and computation time of our
method with existing techniques, potential candidates that may
be used as reference include:
HT-based methods, such as SHT, RHT, PHT.
Edge-following methods, such as UpWrite and Fast Ellipse
Extraction (FEE) method of Ref. [20].
GA-based method (e.g. Ref. [17]), rule-based method (e.g.
Ref. [10]) and other methods (e.g. Ref. [4]).
Since HT-based methods are widely cited in the literature, we
will include SHT, RHT and PHT as reference methods for
comparison. As our method is based on edge-following and
it bears some similarity to FEE, we will also adopt UpWrite
and FEE as reference methods. Judging from the results given
in Ref. [17], the GA-based method is computationally much
more demanding requiring computation times of an order of
magnitude higher than our method. The rule-based technique
of Ref. [10] is computationally efcient aimed at real-time
applications, but the underlying four-points-two-directions-
one-intersection process of the algorithm tends to detect
extraneous ellipses. There are yet other methods (e.g. Ref.
[4]) that detect approximate elliptic shapes rather than exactly
F. Mai et al. / Pattern Recognition 41 (2008) 25122524 2517
Fig. 3. (color online) Testing results for synthetic image 1 (500 400, three
ellipses): The original image (denoted black ellipses) superimposed with
extraction results (denoted either as red or colored ellipses) obtained using
(a) SHT; (b) RHT; (c) PHT; (d) UpWrite; (e) FEE and (f) our method.
tted ellipses and therefore the target of the detection problem
is not the same as ours. Hence, we shall employ only SHT,
RHT, PHT, UpWrite and FEE as reference methods in our
experiments for synthetic and real images.
When testing these reference algorithms, we run the source
code for the HT-based algorithms and UpWrite (in Linux C)
kindly provided by the authors of Ref. [19], directly after
edge detection. We have tried to tune the parameters in the
source code but found that the default values perform the best,
and therefore the default values are adopted in the compar-
isons. For FEE, we adopt an open-source implementation form
LTI-lib (in C + +, available from http://ltilib.sourceforge.net/
doc/html/classlti_1_1fastEllipseExtraction.html). The experi-
ments are performed on a 2.4 GHz Pentium 4 PC.
When comparing the time cost of our method against the
reference algorithms, it should be noted that Matlab code runs
slower than the corresponding C/C++ code in iterative algo-
rithms (which is the case for our method). Hence, it is expected
that the computation times for our method would be signi-
cantly reduced if the algorithm is re-coded in C or C ++.
Fig. 4. (color online) Testing results for synthetic image 2 (500 400 four
incomplete ellipses): (a) original image and extraction results (denoted as red
ellipses) obtained using (b) SHT; (c) RHT; (d) PHT; (e) FEE and (f) our
method.
3.1. Synthetic images
Synthetic edge maps are generated and tested, four of which
are shown in Figs. 36. The rst image (Fig. 3) contains three
complete ellipses separated from each other, the second image
(Fig. 4) contains four incomplete ellipses to simulate occlusion
the third image (Fig. 5) is more complex with 12 ellipses, some
of which are overlapping and four are incomplete, and there
are overlapping lines in the image, and the fourth image (Fig.
6) is a corrupted version of Fig. 5 with salt and pepper noise
(at noise level = 5%) added to imitate extraneous features in
the image.
To assess the extraction accuracy, we dene an error mea-
sure in terms of the area ratio =S
n
/S
g
, where S
n
is the non-
overlapping area of the extracted ellipse and the ground truth
ellipse, and S
g
is the area of the ground truth ellipse. Note that
S
n
includes both the areas bounded by detected ellipse not in-
side the ground truth ellipse and vice versa, and therefore may
exceed a value of 1. The lower the error measure, the higher
the extraction accuracy, with =0 in the case of an exact t.
2518 F. Mai et al. / Pattern Recognition 41 (2008) 25122524
Fig. 5. (color online) Testing results for synthetic image 3 (800 700 complicated 12 ellipses, with four incomplete). The original image (denoted black
ellipses) superimposed with extraction results (denoted either as red or colored ellipses) obtained using (a) RHT; (b) UpWrite; (c) FEE and (d) our method.
Fig. 6. (color online) Testing results for synthetic image 3 with 5% salt and pepper noise (800 700 complicated 12 ellipses, with four incomplete). The
original image (denoted black ellipses) superimposed with extraction results (denoted either as red or colored ellipses) obtained using (a) RHT; (b) UpWrite;
(c) FEE and (d) our method.
F. Mai et al. / Pattern Recognition 41 (2008) 25122524 2519
Table 1
Test results for synthetic image 1
Algorithm SHT RHT PHT UpWrite FEE Ours
Time(s) 36.76 0.62 1.64 0.39 1.28 0.55
Accuracy
#Detected ellipses 2 3 1 3 3 3
F, F 0.12, 0.20, 0.16 F NA 0.05, 0.03, 0.07 0.01, 0.02, 0.01
Note: Accuracies of erroneously extracted ellipses with large (>0.8) will be denoted by F (for failed extraction). NA means that the accuracy cannot be
computed because the ellipse parameters are not available.
Table 2
Testing results for synthetic image 2
Algorithm SHT RHT PHT UpWrite FEE Ours
Time(s) 38.93 1.05 2.31 0.43 1.33 0.67
Accuracy
#Detected ellipses 3 1 3 0 6 4
0.04, 0.05, 0.14 0.07 0.04, 0.05, 0.09 0.38, 0.02, 0.69, 0.67, 0.13, 0.04 0.01, 0.03, 0.01, 0.01
Table 3
Testing results for synthetic image 3
Algorithm SHT RHT PHT UpWrite FEE Ours
Time(s) 1616.03 8.72 26.79 2.03 6.89 2.89
Accuracy
#Detected ellipses 0 2 0 0 12 12
0.3, F Mean =0.04 Mean =0.01
Note: For SHT and PHT, the accuracies for all extracted ellipses are poor ( >0.8), so we just use F to denote the extraction failure. For the two extracted
ellipses by RHT, one has a smaller value ( >0.3), the other one is quite poor ( >0.8), denoted by F. For the accuracy of our method, we show the mean
value of here for space considerations.
Table 4
Testing results for synthetic image 4
Algorithm SHT RHT PHT UpWrite FEE Ours
Time(s) 13521.49 44.60 201.94 29.00 6.89 4.64
Accuracy
#Detected ellipses 0 1 0 0 14 12
F Mean =0.11 Mean =0.01
The original images and ellipses extracted by the ve ref-
erence methods (i.e., SHT, RHT, PHT, UpWrite and FEE) in
comparison with the proposed method are shown in Figs. 36,
and the computation times (excluding that for edge detection
for all methods) and the accuracy assessments are given in
Tables 14. Note that for UpWrite, we are unable to recover
the ellipse parameters from the source code, and so cannot
be computed (marked as NA in the tables) and we can only
make visual judgements on its accuracy based on the displays
generated using the source code.
From these results, we observe that: (i) The computational
cost of SHT algorithm is the most expensive and its accuracy is
poor. The other two HT-based algorithms reduce the computa-
tion time signicantly, but the ellipse extraction accuracies re-
main poor. (ii) For a simple image such as Fig. 3(a), UpWrite,
FEE and our method extract all three ellipses accurately, but
UpWrite is a bit faster than ours. RHT also extracts all three
ellipses, but with poor accuracy. This is because RHT is in-
tended for single ellipse extraction and does not work well for
multiple ellipses [17,33]. (iii) For occluded ellipses shown in
Fig. 4, the proposed algorithm is capable of extracting all the
four incomplete ellipses accurately, whereas all the other meth-
ods fail with missing and/or erroneously extracted ellipses. (iv)
With increased scene complexity (Figs. 5 and 6), the HT-based
methods and UpWrite fail to detect any ellipse correctly, while
the proposed algorithm can still extract all the ellipses accu-
rately, even in the noisy image (Fig. 6). In this case, FEE is the
only competitor among the reference methods capable of de-
tecting most of the ellipses, but with some erroneously detected
ellipses (for Fig. 6) and more computational time.
2520 F. Mai et al. / Pattern Recognition 41 (2008) 25122524
Fig. 7. Testing results for real image 1 (no-feeding sign) with size 306 281: (a) original image; (b) the extracted line segments and extracted ellipses by (c)
RHT; (d) UpWrite; (e) FEE and (f) our method.
Fig. 8. Testing results for real image 2 (car) with size 640 480: (a) original image; (b) the extracted line segments and extracted ellipses by (c) FEE and
(d) our method.
F. Mai et al. / Pattern Recognition 41 (2008) 25122524 2521
Fig. 9. Testing results for real image 3 (spool) with size 640 480: (a) original image and the extracted ellipses by (b) UpWrite; (c) FEE and (d) our method.
Fig. 10. Testing results for real image 4 (ball) with size 640 480: (a) original image; (b) the extracted line segments and the extracted ellipses by (c) FEE
and (d) our method.
3.2. Real images
We also test the proposed method on six real images, as
shown in Figs. 712. To illustrate the effectiveness of the
method under different scenarios, the images selected include
concentric ellipses, ellipses with occlusion and overlapping
ellipses. The elliptical objects may also be embedded in com-
plex backgrounds such as leaves or a keyboard. Some ellipses
2522 F. Mai et al. / Pattern Recognition 41 (2008) 25122524
Fig. 11. Testing results for real image 5 (globe) with size 640 480: (a) original image; (b) the extracted line segments and the extracted ellipses by (c) FEE
and (d) our method.
Fig. 12. Testing results for real image 6 (trafc sign) with size 640 480: (a) original image; (b) the extracted line segments; (c) the extracted ellipses by
our method.
are intersected by linear features (e.g. edges of a keyboard or
monitor in Figs. 9 and 11, and supporting pole in Fig. 12). We
tested the images with the proposed algorithm and the refer-
ence methods as well. From the experimental results, we nd
that: (i) The proposed method can extract the ellipses success-
fully. (ii) The reference HT-based methods have difculty with
real images and fail in all our example images. (iii) UpWrite
can detect elliptical objects in simple images (for example, it
detects the complete ellipse in Fig. 7 with clean background),
but it fails to detect ellipses that are occluded or embedded
in a complex background (it nds one elliptical segment in
Fig. 9, but fails to detect any ellipse in Figs. 8, 1012). (iv) FEE
is capable of extracting some ellipses in the images shown in
Figs. 711, but it also misses and/or erroneously detects some
ellipses in the images. A comparison of the computation ef-
ciency given in Table 5 shows that our method outperforms the
reference algorithms not only in accuracy but also in speed. (v)
The detailed time cost of the proposed method in Table 6 lists
the time costs in every stages of the proposed algorithm, from
which we can see that the ellipse tting by modied RANSAC
is quite efcient. The reason is that after the grouping process,
the outlier ratio in each group is quite low (usually less than
F. Mai et al. / Pattern Recognition 41 (2008) 25122524 2523
Table 5
Comparison of time cost for real images (s)
Method Image
No-feeding sign ( Fig. 7) Car ( Fig. 8) Spool ( Fig. 9) Ball ( Fig. 10) Globe ( Fig. 11) Trafc sign ( Fig. 12)
SHT
RHT 3.9
PHT
UpWrite 1.34 6.27
FEE 2.03 7.03 5.70 6.58 5.97
Ours 1.00 7.05 4.02 3.65 3.46 7.23
Note: (1) means the method fails to detect any ellipse.
(2) The time costs for SHT, RHT, PHT and UpWrite do not include the time for edge detection, while the time costs for FEE and our method include it.
Table 6
Time cost of our method for real images (s)
Stage Image
No-feeding sign ( Fig. 7) Car ( Fig. 8) Spool ( Fig. 9) Ball ( Fig. 10) Globe ( Fig. 11) Trafc sign ( Fig. 12)
Edge detection 0.26 0.95 0.94 0.98 0.96 0.99
Line segment extraction 0.47 3.82 1.82 1.70 1.72 5.12
Line segment linking+
arc segment grouping+
ellipse tting by RANSAC
0.27 2.28 1.26 0.97 0.78 1.12
Total time 1.00 7.05 4.02 3.65 3.46 7.23
10%). Therefore, RANSAC performed in each group will not
take much time (less than six iterations) [32].
4. Conclusions
This paper presents a fast and robust method to detect ellip-
tical objects in images. We detect the ellipses using a hierarchi-
cal approach, whereby the features of an ellipse are detected
progressively from a local to the global level. The main contri-
butions of the proposed method are:
Although each of the four stages of our algorithm is well-
established method, choosing appropriate conditions to use
at different stages and tting them together into a robust
hierarchical procedure for ellipse extraction is by no means
obvious.
A distinctive characteristic of our method is that despite the
extraction of progressively higher levels of data description
in a hierarchical approach, we do not discard low-level in-
formation at later stages of ellipse extraction, but make use
of the supporters of high-level features throughout all stages
of the process
The modication to RANSAC in order to maximize the inlier
percentages for both the local and global sets is novel.
Furthermore, the proposed method does not need a high di-
mensional parameter space like HT based algorithms, and so
it reduces the computation and memory requirements. The
algorithm is easy to implement. Generally, it is capable of
extracting multiple ellipses with complex background in the im-
ages quickly and correctly, even when the ellipse is occluded by
other objects. Experiments on synthetic and real images show
that the proposed method is superior to the existing methods,
especially when the testing images are complicated.
However, our method may have difculties when the ellipse
to be extracted is relatively small in the image, in which case
the ellipse cannot be reasonably approximated by a sequence
of line segments. One way to overcome this problem is to take
a multi-scale approach.
Acknowledgement
The work described in this paper was supported by a grant
from the Research Grants Council of Hong Kong Special
Administrative Region, China (Project No. HKU7135/05E).
References
[1] C.A. Rothwell, A. Zisserman, C.I. Marinos, D.A. Forsyth, J.L. Mundy,
Relative motion and pose from arbitrary plane curves, Image Vision
Comput. 10 (4) (1992) 250262.
[2] A. Soetedjo, K. Yamada, Fast and robust trafc sign detection, in:
Proceedings of IEEE International Conference on Systems, Man and
Cybernetics, Waikoloa, Hawaii, October 2005, pp. 13411346.
[3] Q. Ji, R.M. Haralick, A statistically efcient method for ellipse detection,
in: Proceedings of IEEE International Conference on Image Processing,
Kobe, Japan, October 1999, pp. 730734.
[4] C. Teutsch, D. Berndt, E. Trostmann, M. Weber, Real-time detection
of elliptic shapes for automated object recognition and object tracking,
in: Proceedings of Machine Vision Applications in Industrial Inspection
XIV, San Jose, CA, USA, January 2006, pp. 171179.
2524 F. Mai et al. / Pattern Recognition 41 (2008) 25122524
[5] R.O. Duda, P.E. Hart, Use of the Hough transform to detect lines and
curves in pictures, Commun. ACM 15 (1) (1972) 1115.
[6] L. Xu, E. Oja, P. Kultanen, A new curve detection method: randomized
Hough transform (RHT), Pattern Recognition Lett. 11 (5) (1990)
331338.
[7] N. Kiryati, Y. Eldar, A.M. Bruckstein, A probabilistic Hough transform,
Pattern Recognition 24 (4) (1991) 303316.
[8] R.A. McLaughlin, Randomized Hough transform: improved ellipse
detection with comparison, Pattern Recognition Lett. 19 (3) (1998)
299305.
[9] N. Guil, E.L. Zapata, Lower order circle and ellipse Hough transform,
Pattern Recognition 30 (10) (1997) 17291744.
[10] S.C. Zhang, Z.Q. Liu, A robust, real-time ellipse detector, Pattern
Recognition 38 (2) (2005) 273287.
[11] C.T. Ho, K.H. Chen, A fast ellipse/circle detector using geometric
symmetry, Pattern Recognition 28 (1) (1995) 117124.
[12] Y. Lei, K.C. Wong, Ellipse detection based on symmetry, Pattern
Recognition Lett. 20 (1) (1999) 4147.
[13] Y.C. Cheng, The distinctiveness of a curve in a parameterized
neighborhood: extraction and applications, IEEE Trans. Pattern Anal.
Mach. Intell. 28 (8) (2006) 12151222.
[14] T. Mainzer, Genetic algorithm for shape detection, Technical Report no.
DCSE/TR-2002-06, University of West Bohemia, 2002.
[15] S. Procter, J. Illingworth, A comparison of the randomized Hough
transform and a genetic algorithm for ellipse detection, in: Pattern
Recognition in Practice IV: Multiple Paradigms, Comparative Studies
and Hybrid Systems, Elsevier, Amsterdam, 1994, pp. 449460.
[16] K.U. Kasemir, K. Betzler, Detecting ellipses of limited eccentricity in
images with high noise levels, Image Vision Comput. 21 (2) (2003)
221227.
[17] J. Yao, N. Kharma, P. Grogono, A multi-population genetic algorithm
for robust and fast ellipse detection, Pattern Anal. Appl. 8 (1) (2005)
149162.
[18] T. Kawaguchi, R.-I.Nagata, Ellipse detection using a genetic algorithm,
in: Proceedings of the 14th International Conference on Pattern
Recognition, USA, 1998, pp. 141145.
[19] R.A. McLaughlin, M.D. Alder, The Hough transform versus the UpWrite,
IEEE Trans. Pattern Anal. Mach. Intell. 20 (4) (1998) 396400.
[20] E. Kim, M. Haseyama, H. Kitajima, Fast and robust ellipse extraction
from complicated images, in: Proceedings of IEEE International
Conference on Information Technology & Applications (ICITA), 2002.
[21] P.L. Rosin, G.A.W. West, Nonparametric segmentation of curves into
various representations, IEEE Trans. Pattern Anal. Mach. Intell. 17 (12)
(1995) 11401153.
[22] K. Kanatani, N. Ohta, Automatic detection of circular objects by ellipse
growing, Int. J. Image Graphics 4 (1) (2004) 3550.
[23] E. Kim, M. Haseyama, H. Kitajima, The extraction of circles from arcs
represented by extended digital lines, IEICE Trans. 88-D (2) (2005)
252267.
[24] F. Mai, Y.S. Hung, H. Zhong, W.F. Sze, A hierarchical approach for
fast and robust ellipse extraction, in: Proceedings of IEEE International
Conference on Image Processing, San Antonio, TX, USA, September
2007.
[25] M.A. Fischler, R.C. Bolles, Random sample consensus: a paradigm
for model tting with applications to image analysis and automated
cartography, Commun. ACM 24 (6) (1981) 381395.
[26] R. Nevatia, K.R. Babu, Linear feature extraction and description,
Comput. Vision Graphics Image Process. 13 (2) (1980) 257269.
[27] J. Princen, J. Illingworth, J. Kittler, A hierarchical approach to line
extraction based on the Hough transform, Comput. Vision Graphics
Image Process. 52 (1) (1990) 5777.
[28] J. Chanussot, G. Mauris, P. Lambert, Fuzzy fusion techniques for linear
features detection in multi-temporal SAR images, IEEE Trans. Geosci.
Remote Sensing 37 (3) (1999) 12921305.
[29] J. Chanussot, P. Lambert, An application of mathematical morphology to
road network extraction on SAR images, in: ISMM984th International
Symposium on Mathematical Morphology and its Applications,
Amsterdam, The Netherlands, June 1998, pp. 399406.
[30] J. Canny, A computational approach to edge detection, IEEE Trans.
Pattern Anal. Mach. Intell. 8 (6) (1986) 679714.
[31] R.O. Duda, P.E. Hart, Pattern Classication and Scene Analysis, Wiley,
New York, 1973.
[32] R. Hartley, A. Zisserman, Multiple View Geometry in Computer Vision,
Cambridge University Press, Cambridge, 2000, pp. 103-104.
[33] R.A. McLaughlin, Intelligent algorithms for nding curves and surfaces
in real world data, Ph.D. Thesis, Department of Electrical and Electronic
Engineering, The University of Western Australia, 2000.
About the AuthorFEI MAI received her B.Eng. and M.Eng. degrees in Control Theory and Control Engineering in 2000 and 2003, respectively from
Tianjin University, China. She is currently a Ph.D. student in the Department of Electrical and Electronic Engineering at the University of Hong Kong. Her
research interests include feature extraction, feature matching and 3D reconstruction.
About the AuthorY.S. HUNG received his B.Sc. (Eng) degree in Electrical Engineering and B.Sc. degree in Mathematics, both from the University of
Hong Kong, and his M.Phil. and Ph.D. degrees from the University of Cambridge. He has worked as a research associate at the University of Cambridge,
and as a lecturer at the University of Surrey before he joined the University of Hong Kong in 1989, where he is now an associate professor and the head of
the department. Dr. Hung is a recipient of the Best Teaching Awards in 1991 from the Hong Kong University Students Union. He is a chartered engineer, a
fellow of IEE and a senior member of IEEE. He has authored and co-authored over 100 publications in books, international journals and conferences.
About the AuthorHUANG ZHONG received his B.Eng. and M.Eng. degrees in Control Theory and Control Engineering in 1996 and 1999, respectively
from South China University of Technology, China, and his Ph.D. degree from the University of Hong Kong in 2006. He is now a research associate in the
Department of Electrical and Electronic Engineering at the University of Hong Kong. His research interests include camera calibration and 3D reconstruction.
About the AuthorWUI-FUNG SZE received B.Eng. (1st class honour) and M.Phil. degrees in Electronic and Communications Engineering from the
University of Hong Kong in 2004 and 2006. His research interests are in the area of computer vision.