Signal Processing 74 (1999) 127—151

Adaptive fuzzy hybrid multichannel filters for removal of

impulsive noise from color images
Hung-Hsu Tsai, Pao-Ta Yu*
Department of Computer Science and Information Engineering, National Chung Cheng University, Chiayi 62107, Taiwan
Received 24 July 1998; received in revised form 19 October 1998

Abstract

On the design of multichannel filters, especially in color image restoration, it is not easy to simultaneously achieve
three objectives: noise attenuation, chromaticity retention, and edges or details preservation. In this paper, we propose
a new class of multichannel filters called adaptive fuzzy hybrid multichannel (AFHM) filters to achieve these three
objectives simultaneously. Our novel approach is mainly based on human concept (heuristic rules) and provides
a significant framework to take the merits of filtering behavior of three filters: a vector median (VM) filter, a vector
directional (VD) filter, and an identity filter. On the design of an AFHM filter, our approach is a powerful and flexible
scheme to achieve these three objectives because human concept can be efficiently expressed by fuzzy implicative rules for
improving the filtering performance. The AFHM filters are able to effectively inherit the merits of filtering behaviors of
these three filters in color image restoration applications. This is the first paper to include human concept to design
multichannel filters. Moreover, a faster convergence property of the learning algorithm is investigated to reduce the time
complexity of the AFHM filters. Extensive simulation results illustrate that AFHM filters not only achieve these three
objectives but also possess the robust and adaptive capabilities, and demonstrate that the performance of AFHM filters
outperforms that of other proposed filtering techniques.  1999 Elsevier Science B.V. All rights reserved.

Zusammenfassung
Beim Entwurf von mehrkanaligen Filtern ist es, zumal für die Farbbildrestaurierung, nicht leicht, folgende drei
Zielvorstellungen unter einen Hut zu bringen: Rauschunterdrückung, Erhaltung der Farbwerte und Bewahrung von
Kanten und Details. In dieser Arbeit schlagen wir eine neue Klasse von mehrkanaligen Filtern mit der Bezeichnung
adaptive fuzzy hybride Mehrkanalfilter (AFHM) vor, die die drei Zielvorstellungen zugleich erreichen sollen. Unser
neuartiger Ansatz gründet vor allem auf der menschlichen Intuition (heuristische Regeln) und bietet einen bedeutsamen
Rahmen zur Ausnutzung der Vorzüge des Filterverhaltens von dreierlei Filtern: Vektor-Medianfilter (VM), Vektor-
Richtungsfilter (VD) und Identitätsfilter. Beim Entwurf eines AFHM Filters erweist sich unser Ansatz als ein mächtiges
und flexibles Schema, um diese drei Zielvorstellungen zu verwirklichen, da fuzzy Folgerungsregeln zur Darstellung der
menschlichen Intuition bei der Verbesserung der Filterleistung taugen. Die AFHM Filter sind imstande, die Vorzüge des
Filterverhaltens dieser drei Filter in Anwendungen der Farbbildrestaurierung tatsächlich zu ererben. Dies ist die erste
Arbeit, die auch die menschliche Intuition für den Entwurf mehrkanaliger Filter einbezieht. Darüber hinaus wird die

* Corresponding author. Tel.: 886 5 2720411 ext. 6015; fax: 886 5 272 0859; e-mail: csipty@ccunix.ccu.edu.tw
 This work was supported by the National Science Council, R.O.C., under Grant NSC88-2213-E-194-011.

0165-1684/99/$ — see front matter  1999 Elsevier Science B.V. All rights reserved.
PII: S 0 1 6 5 - 1 6 8 4 ( 9 8 ) 0 0 2 0 7 - 2
128 H.-H. Tsai, P.-T. Yu / Signal Processing 74 (1999) 127—151

Beschleunigung der Konvergenz des Lernalgorithmus untersucht, um die zeitliche Komplexität der AFHM Filter zu
verringern. Umfassende Simulationsergebnisse belegen, da{ AFHM Filter nicht nur diese drei Zielvorstellugnen verwirk-
lichen, sondern auch robuste und adaptive Eigenschaften besitzen, und sie zeigen, da{ die Leistungsfähigkeit von AFHM
Filtern die anderer Filterverfahren übertrifft.  1999 Elsevier Science B.V. All rights reserved.

Résumé
Dans le domaine du filtrage multi-canaux, spécialement dans la restauration d’images couleur, il n’est pas facile
d’atteindre simultanément les trois objectifs suivants: une atténuation du bruit, une rétention des couleurs et une
préservation des contours et des détails. Dans cet article, nous proposons une nouvelle classe de filtres multicanaux,
appelés filtres multi-canaux hybrides flous adaptatifs (MHFA), afin d’atteindre ces objectifs simultanément. Notre
nouvelle approche repose principalement sur un concept humain (des règles heuristiques) et fournit un cadre de travail
significatif pour tirer parti des caractéristiques de filtrage de trois filtres: un filtre médian vectoriel (MV), un filtre
directionnel vectoriel (DV) et un filtre identité. Pour la conception du filtre MHFA, notre approche consiste en un
schéma puissant et flexible pour atteindre ces trois objectifs car un concept humain peut être efficacement exprimé par des
règles implicatives floues pour améliorer la performance de filtrage. Les filtres MHFA sont capables d’hériter efficace-
ment des mérites des caractéristiques de filtrage de ces trois filtres pour la restauration des images couleur. Ceci est le
premier article à inclure le concept humain pour concevoir des filtres multi-cannaux. De plus, la propriété de convergence
plus rapide de l’algorithme d’apprentissage est étudiée pour réduire la complexité en temps des filtres MHFA. Des
résultats de simulations extensives illustrent que les filtres MHFA non seulement atteignent les trois objectifs mais aussi
possèdent des capacités de robustesse et d’adaptation, et démontrent que les performances des filtres MHFA dépassent
celles d’autres techniques de filtrage proposées.  1999 Elsevier Science B.V. All rights reserved.

Keywords: Multichannel filter; Fuzzy rule-based system; Learning algorithm; Color image restoration; Impulsive noise

Notation motion, biomedical image processing, and the

high-definition television (HDTV) [10,13].
xJ the output of a VM filter A multitude of nonlinear filters based on order
4+ statistics (OS) have successfully been developed to
xJ the output of a BVD filter
4" restore monochrome images corrupted by impul-
xJ the central vector-valued pixel in the filter
K sive noises [21]. Those nonlinear techniques heav-
window
o distance function, +0,1,2,255, ily depend on order statistics of observed samples
;+0,1,2,255,PR which are obtained via a filter window. A total
dJ the aggregated distance corresponding to ordering exists on the observed samples since these
4+ samples are scalar. The order statistics of observed
xJ
4+ samples can be easily obtained by use of sorting
dJ the aggregated distance corresponding to
4" algorithms. However, in a multichannel image
xJ
4" each observed sample becomes a vector-valued
dJ the aggregated distance corresponding to
4+ K
xJ while o is defined by a VM filter sample instead of a scalar. The total ordering no
K longer exists on vector-valued samples, for in-
dJ the aggregated distance corresponding to
4" K
xJ while o is defined by a BVD filter stance, vector-valued samples in RGB color im-
K ages, so vector-valued samples are not necessary to
be comparable. Vector-valued samples are unable
1. Introduction to be directly ranked just due to their vector
structures. Four vector-valued sub-ordering
In the field of signal processing, the multichannel including marginal ordering (M-ordering),
image processing has recently taken on new im- reduced (aggregated) ordering (R-ordering), partial
portance for applications in remote sensing, robot ordering (P-ordering) and conditional ordering
 H.-H. Tsai, P.-T. Yu / Signal Processing 74 (1999) 127—151
(C-ordering) schemes have been proposed by Bar-
nett [4].
One class of OS multichannel filters is based on
M-ordering schemes by means of applying mono-
chrome filtering algorithms to each channel image
individually [16,20]. However, this kind of multi-
channel filters does not consider channel depend-
ence which often exists in real multichannel images
[25]. Another class of OS multichannel filters is
based on R-ordering schemes to obtain order stat-
istics on observed vector-valued samples. The R-
ordering scheme is a mapping from a vector-valued
sample with high dimensions to a scalar sample.
One version of R-ordering scheme is the distance
between one observed sample and the centralized
sample (e.g., sample mean vector). This distance cri-
terion (e.g., Mahalanobis distance) has been used on
the design of multichannel filters [11]. Another R-
ordering scheme is the aggregated distance which is
a summation of the distances between one sample
and the others. Distinct distance measures are in-
volved to develop different OS multichannel filtering
techniques [3,25]. For example, the Euclidean dis-
tance is used to design VM filters [3], and the vector
angle is employed to design VD filters [25].
The VM filters possess noise attenuation in con-
trast to the VD filters which possess chromaticity
retention. Therefore, many distance measures are
proposed to combine these two distance measures
mentioned above to devise another class of multi-
channel filters [14,19]. However, these conven-
tional filters have difficulty to achieve these three
objectives. Especially in details preservation, details
such as fine lines or spots are rich in lots of images.
This motivates us to propose hybrid multichannel
(HM) filters to achieve these three capabilities si-
multaneously for further improving the filtering
performance of these conventional filters. The out-
put of an HM filter is a convex combination of the
outputs of a VM, a VD, and an identity filter.
Moreover, HM filters are called AFHM filters if the
coefficients (or weights) associated with the outputs
of the VM, the VD, and the identity filter are
decided by adaptive fuzzy rules (or fuzzy im-
plicative rules). One of main contributions in this
paper is that AFHM filters are able to effectively
inherit the merits of behavior of the VM, the VD,
and the identity filter so that the filtering perfor-
129
mance of AFHM filters can be better than that of
those conventional filters.
Because of the underlying power of fuzzy set
theory and fuzzy logic, fuzzy rules are flexible and
powerful enough to model complex control systems
[15,22,29]. The behavior of an AFHM filter can be
viewed as a simple control system, since it depends
on two control values. Moreover, AFHM filters
also possess highly robust capability which is
examined in simulations. Recently, lots of filtering
techniques based on fuzzy set theory have been
proposed, and have gained promising results
[1,5,6,18,23,26,27]. However, many filtering tech-
niques are just applied to restore monochrome
images [1,5,6,23,26,27]. In [18], an approach with
fuzzy decision making is employed to design multi-
channel filters, but it is difficult to express human
concept. However, it is very important to express
human concept while developing complex systems
[6]. In general, this kind of human concept is highly
nonlinear and is difficult to be expressed by means
of conventional mathematical models. Therefore,
this is the first paper to include human concept
expressed by fuzzy rules to design multichannel
filters. Another contribution of this paper is to
propose a learning algorithm to reduce computa-
tional burden during the learning period. The de-
sign of our learning algorithm is based on the
classical least-mean-square (LMS) algorithm for
adaptive linear filtering [12]. Our learning algo-
rithm possesses a faster convergence property and
yields a lower ensemble-averaged error than the
LMS algorithm does.
In Section 2, some basic concepts are introduced.
In Section 3, HM filters are defined, and some of
their properties are investigated. In Section 4, the
design of AFHM filters is described in detail. In
Section 5, the simulation, in color image restora-
tion applications, are exhibited. Finally, con-
clusions are discussed in Section 6.
2. Basic concepts
2.1. Ranking vector-valued data
In recent years, the multichannel filter applica-
tions mainly remain in the field of color image
130 H.-H. Tsai, P.-T. Yu / Signal Processing 74 (1999) 127—151

restoration. The following subsection reviews some By use of the row-major method, X in Eq. (2)
GH
basic concepts and defines notations concerning can be represented by
color images. X "(x ,x , ,x , ,x ,
GH G\O H\O G\O H\O> 2 GH 2 G>O H>O\
x ). (3)
Definition 2.1. A digital multichannel image I with G>O H>O
size ¸;K is represented by a matrix defined by Let the sliding window cover over the image I in
a raster scan fashion, i.e., from top to bottom and
x x 2 x 2 x from left to right. For convenience, the subscript ij
  H ) in Eq. (3) can be substituted by a scalar running
x x 2 x 2 x
  H ) index, l"(i!1);¸#j. Hence, Eq. (3) can be re-
$ $ $ $ written as
I"
x x 2 x 2 x XJ"(xJ, xJ,2, xJ,2, xJ , xJ). (4)
G G GH G)   I ,\ ,
$ $ $ $ In the case of m"3, xJ"[xJ, xJ, xJ]2, xJ,
I PI EI @I PI
x x 2 x 2 x xJ, xJ3+0,1,2,255,, 1)k)N and T denotes
* * *H *) EI @I
the matrix transpose.
"[x ] , (1) Next, the M-ordering and R-ordering are re-
GH *")
viewed. First, xJ, xJ,2, and xJ are ranked based
where x 3+0,1,2,2,255,K, and denotes an m;1   ,
GH on the M-ordering approach to yield
column vector. In this paper, x is called the vec-
GH xJ )xJ )2)xJ ,
tor-valued pixel located at position (i, j) in I. Note P P P,
that m"3 when the digital multichannel image xJ )xJ )2)xJ , (5)
E E E,
is an RGB color image. Hereafter, m"3 in this
xJ )xJ )2)xJ .
paper, unless otherwise stated. @ @ @,
Obviously, this approach is to apply a mono-
chrome filtering algorithm to each component sep-
Definition 2.2. A filter window (or a sliding win-
arately. According to Eq. (5), the ranked order of
dow) with size N"(2q#1) (N is odd, in general)
XJ is denoted by
covers on the image I at position (i, j) to obtain an
observed sample matrix X (or filter window con- XI J"(xJ , xJ ,2, xJ )
GH   ,
tent) defined by
with an M-ordering, xJ )xJ )2)xJ ,
  ,
X " where each xJ "[xJ , xJ , xJ ]2. Chromaticity
GH I PI EI @I
shift may occur, because each xJ may be or may be
I
x 2 x 2 x not one of entries in XJ [25]. For example, accord-
G\O H\O G\O H G\O H>O
$ $ $ ing to Eq. (5), the output of marginal median (MM)
filters is xJ "[xJ , xJ , xJ ]2 where m"UN/2 V,
x x x K PK EK @K
G H\O
2
GH
2
G H>O , and UV ) stands for the ceiling function. In some
$ $ $ case, xJ is not one of entries in original observed
K
x 2 x 2 x O>" sample matrix XJ. Therefore, vector processing ap-
G>O H\O G\O H G>O H>O O> proach is advocated to design the multichannel
(2) filters in recent years [11,14,18,19,24,25].
Second, the R-ordering of xJ, xJ,2, and xJ is
where 1)i)¸ and 1)j)K. Each entry x in   ,
GYHY based on distance criteria or functions. An aggreg-
X is called an observed vector-valued sample (or ated distance corresponding to xJ can be defined
GH G
observed vector-valued pixel) where i!q)i) by
i#q and j!q)j)j#q. Moreover, x denotes
GH ,
the central vector-valued sample (or central vec- dJ" o(xJ, xJ), (6)
tor-valued pixel). G G H
H
 H.-H. Tsai, P.-T. Yu / Signal Processing 74 (1999) 127—151 131

where o : +0,1,2,255,;+0,1,2,255,PR is a where o is an angle function between two vectors,

distance function. For instance, o can be one of xJ and xJ. Let the aggregated angle distance cor-
G H
Minkowski metrics defined by responding to xJ be denoted as dJ
4" 4"
("dJ, k3+1,2,2,N,) which is the smallest ele-

 
N I
o(xJ, xJ)" "xJ!xJ"N , (7) ment, dJ , among dJ, dJ,2, and dJ. According to
   ,
G H IG IH
IZ+P E @, the definition of the R-ordering, we define F as
4"
or be the vector angle distance defined by Eq. (11). Note further that dJ differs from dJ due
4+ 4"
to the different o.


xJ2xJ
o(xJ, xJ)"arccos G H . (8) In the next section, we first define HM filters and
G H ""xJ"" ""xJ"" then develop some of their properties. The HM
G H
Note that Eq. (7) is called Block (¸ norm), Euclid- filters are called the AFHM filters when the fuzzy
 techniques and learning algorithms are applied to
ean (¸ norm), and Max (¸ norm) distances while
  them.
p"1, 2 and R, respectively. Thus, each xJ is
G
reduced to a scalar dJ through the mapping of
G
aggregated distances. The R-ordering of xJ, xJ,2,
  3. HM filters
and xJ is decided according to the order statistics
,
of dJ, dJ,2, and dJ. In other words, let
  , 3.1. Definition of HM filters
dJ, dJ,2, and dJ be sorted in ascending order to
  ,
yield
The structure of HM filters is depicted in Fig. 1.
dJ )dJ )2)dJ . (9) The structure of an HM filter comprises four basic
  ,
On the basis of the total ordering of real numbers, components: VMF, BVDF, IF and a summation
Eq. (9) defines an R-ordering of xJ, xJ,2, and xJ, combinator. VMF, BVDF, and IF stand for a VM,
  , a BVD, and an identity filter, respectively. The
that is, xJ )xJ )2)xJ .
  , summation combinator is represented by .
2.2. Review of VM and VD filters
Definition 3.1. Let F be the window operator
&+
The design of VM filters is based on the aggreg- associated with an HM filter. The output of the
ated Euclidean distance to select a minimal entry HM filter is defined by
from the observed sample matrix XJ [3]. Let F (XJ;aJ,bJ)"ŷJ
F be the window operator associated with a VM &+
4+ "(1!aJ)bJxJ #(1!bJ)xJ #aJbJxJ,
filter, which is defined by 4" 4+ K
(12)
xJ "F (XJ)"xJ , (10)
4+ 4+ 
where aJ, bJ3[0,1]. xJ denotes the central vec-
where the distance function o is ¸ norm. Let the K
 tor-valued pixel of a filter window when the filter
aggregated Euclidean distance corresponding to
window covers on an image at position l (i.e., at
xJ be denoted as dJ ("dJ, k3+1,2,2,N,)
4+ 4+ I (i, j)). xJ , xJ , and xJ are the outputs of a BVD,
which is the smallest element, dJ , among 4" 4+ K
 a VM, and an identity filter, respectively.
dJ,dJ,2, and dJ. According to the definition of
  ,
the R-ordering, we define F as Eq. (10).
4+ The output of the HM filter can be repre-
The design of VD filters is based on the aggreg-
sented as ŷJ"[ŷJ, ŷJ, ŷJ]2, and ŷJ, ŷJ, ŷJ3
ated vector angle distance. One class of VD filters P E @ P E @
+0,1,2,255,. Two control values, aJ and bJ, deci-
called basic vector directional (BVD) filters is to
de the coefficients within Eq. (12) to adapt the
select a minimal entry from the observed sample
window content, namely XJ. The HM filter is
matrix XJ [25]. Let F be the window operator
4" called the AFHM filter when aJ and bJ are deci-
associated with a BVD filter, which is defined by
ded by fuzzy techniques and learning algorithms. In
xJ "F (XJ)"xJ , (11) this paper, one of main contributions is to apply
4" 4" 
132 H.-H. Tsai, P.-T. Yu / Signal Processing 74 (1999) 127—151

Fig. 1. The structure of an HM filter.

a fuzzy rule-based system and a gradient-descent By Eqs. (6) and (14), we can obtain
learning algorithm to adjust these two control
, ,
values such that the AFHM filters can adapt the d" o(cxJ, cxJ)"c o(xJ, xJ)"cd
G G H G H G
distinct window contents [12,22]. The detailed de- H H
sign of AFHM filters is described in the next for each i. (15)
section.
Thus, the ranked order of d ,d ,2, and d is the
  ,
same as the ranked order of d ,d ,2, and d .
  ,
3.2. Properties of HM filters That is,
d )d )2)d "rd )rd )2)rd
Before we state the design of the fuzzy rule-based   ,   ,
system and the learning algorithm to decide aJ and 0 d )d )2)d . (16)
  ,
bJ, we need to analyze some deterministic proper- This implies that the ranked order of XJ, namely
ties of HM filters to understand the behavior of XI J, is the same as the ranked order of cXJ. Let the
HM filters. minimal entry of XI J be referred to as xJ . That is,
4+
xJ "xJ . Hence, we have F (XJ)"xJ and
Property 1. HM filters are invariant under scaling.  4+ 4+ 
F (cXJ)"cxJ , and
That is, 4+ 
F (cXJ)"cxJ "cF (XJ). (17)
4+ 4+ 4+
F (cXJ;aJ,bJ)"cF (XJ;aJ,bJ) where c'0. Second, we need to show that F (cXJ)"
&+ &+ 4"
(13) cF (XJ). Let o be the angle distance function
4"
used to design a BVD filter, and d and d be the
G G
Proof. Let F be a window operator associated aggregated distances corresponding to xJ and
&+ G
with an HM filter, and XJ be scaled by a positive cxJ, 1)i)N. We have
G
scalar c to yield cXJ"(cxJ, cxJ,2, cxJ , cxJ).
  ,\ , o(cxJ,cxJ)"o(xJ, xJ), 1)j)N, (18)
Now we want to show that F (cXJ;aJ, bJ)" G H G H
&+
cF (XJ;aJ, bJ). First, we need to show that because the angle between two vector-valued pixels
&+
F (cXJ)"cF (XJ). Let o be Euclidean distance remains unchanged when they are scaled by c. By
4+ 4+
used to design a VM filter, and d and d be the Eqs. (6) and (18), we obtain
G G
aggregated distances corresponding to xJ and
G , ,
cxJ, 1)i)N, respectively. We have d" o(cxJ,cxJ)" o(xJ, xJ)"d
G G G H G H G
H H
o(cxJ,cxJ)"co(xJ, xJ), 1)j)N. (14) for each i. (19)
G H G H
 H.-H. Tsai, P.-T. Yu / Signal Processing 74 (1999) 127—151 133

Obviously, the ranked order of XJ is the same as xJ and MxJ, 1)i)N, respectively. We have
G G
the ranked order of cXJ due to d"d for each i.
G G o(MxJ, MxJ)"o(xJ, xJ), 1)j)N, (24)
Let the minimal entry of XI J be referred to as G H G H
xJ . That is, xJ "xJ . Hence, we obtain since
4"  4"
F (XJ)"xJ and F (cXJ)"cxJ , and
4"  4"  o(MxJ, MxJ)
F (cXJ)"cxJ "cF (XJ). (20) G H
4" 4" 4"
"[(MxJ)2MxJ]"[xJ2M2MxJ]
Finally, by Definition 3.1, Eqs. (17) and (20), we G H G H
have "[xJ2xJ]"o(xJ, xJ). (25)
G H G H
F (cXJ; aJ, bJ) Hence, the distances measured by ¸ norm remains
&+ 
"(1!aJ)bJF (cXJ)#(1!bJ)F (cXJ) unchanged between two vector-valued pixels when
4" 4+ they are transformed by M. By Eqs. (6) and (24), we
#aJbJcxJ
K obtain
"cF (XJ;aJ,bJ). (21)
&+ , ,
Thus, this property holds. 䊐 d" o(MxJ,MxJ)" o(xJ, xJ)"d
G G H G H G
H H
Note that Property 1 also holds when on the for each i. (26)
design of VM filters o is one of other Minkowski
It is trivial to know that the ranked order of MXJ
metrics, e.g., ¸ norm.
 is the same as the ranked order of XJ due to d"d
G G
for each i. In addition, Eqs. (24) and (26) also hold
Property 2. HM filters are invariant under rotation if
based on Eq. (25) when o is the angle distance.
o is ¸ norm on the design of VM filters. That is,
 Accordingly, by the similar way in Property 1, we
F (MXJ; aJ, bJ)"MF (XJ; aJ, bJ), (22) have Eq. (22). Thus, this property holds. 䊐
&+ &+
where M denotes the rotation matrix to represent the
transformation which rotates a coordinate system to On the design of filters, root signal properties
be another coordinate system. should be considered and analyzed, because the
characteristics of window contents can be adapted
Proof. In R, suppose that a right-handed coordi- to preserve fine image structures, such as edges or
nate system +x,y,z, is transformed by a 3;3 rota- details. Root signal properties of monochrome me-
tion matrix M to be another right-handed dian and stack filters have been examined in
coordinate system +x,y,z,. The rotation matrix [2,8,9,28]. In [3,24], root signal properties of multi-
M is given channel filters have been investigated.

 
cos h sin h 0 Property 3. Suppose that aJO1 and bJO1. An
M" !sin h cos cos h cos sin input image I is a root of the HM filter of length N if
and only if the central vector-valued pixel of the
!sin h sin !cos h sin cos
window xJ is a convex combination of xJ and xJ ,
(23) K 4" 4+
that is,
if xy plane is rotated in a counterclockwise dir- xJ"j xJ #(1!j) xJ
ection through the angle h, and yz plane is rotated K 4" 4+
in counterclockwise direction through the angle (1!aJ)bJ
where j" . (27)
. Suppose that XJ is transformed by M to 1!aJbJ
yield MXJ"(MxJ, MxJ,2, MxJ , MxJ). We
  ,\ ,
want to show that F (MXJ; aJ, bJ)" Proof. (N) I is a root of the HM filter if, for all
&+
MF (XJ;aJ,bJ). Let o be ¸ norm, and d and l, F (XJ; aJ, bJ)"xJ. That is, I is invariant un-
&+  G &+ K
d be the aggregated distances corresponding to der a raster scan filtering by the HM filter. By
G
134 H.-H. Tsai, P.-T. Yu / Signal Processing 74 (1999) 127—151

Definition 3.1, we have 4. The design of AFHM filters

xJ"(1!aJ)bJxJ #(1!bJ) xJ #aJbJxJ

K 4" 4+ K The structure of AFHM filters is illustrated in
Fig. 2. The structure of an AFHM filter is com-


(1!aJ)bJ (1!aJ)bJ
N xJ" xJ # 1! xJ posed of three principal components: an HM filter,
K 1!aJbJ 4" 1!aJbJ 4+ a fuzzy rule-based system (FRS), and a learning
"j xJ #(1!j) xJ . (28) algorithm. An AFHM filter can be made adaptive
4" 4+ for further improving the filtering performance of
(=) We can rewrite Eq. (27) to be an HM filter, because the FRS and the learning
algorithm are employed to adjust aJ and bJ with



(1!aJ)bJ 1!bJ respect to the different window contents. Section
xJ" xJ # xJ . (29)
K 1!aJbJ 4" 1!aJbJ 4+ 4.1 describes the design motivation of AFHM fil-
ters based on human concept. Section 4.2 specifies
Two sides of Eq. (29) are multiplied by 1!aJbJ to
the detailed design of the FRS based on human
obtain
concept. Section 4.3 mentions the detailed des-
(1!aJbJ) xJ"(1!aJ)bJxJ #(1!bJ) xJ . cription of the learning algorithm based on the
K 4" 4+ gradient-descent method. Moreover, a faster
(30)
convergence property of the learning algorithm is
Accordingly, the output of the HM filter can be investigated.
simplified by replacing Eq. (30) with Eq. (12) to
yield 4.1. The design motivation of AFHM filters
based on human concept
F (XJ; aJ, bJ)
&+
According to human concept obtained from ob-
"(1!aJ)bJxJ #(1!bJ) xJ #aJbJxJ
4" 4+ K serving experimental results, four cases are con-
sidered on the design of AFHM filters. Let dJ and
"(1!aJbJ) xJ#aJbJxJ"xJ.
K K K
(31) 4+
dJ denote the aggregated distances correspond-
K 4+
Thus, this property holds. 䊐 ing to xJ and xJ, respectively. In addition, let
4+ K

Fig. 2. The structure of an AFHM filter.

 H.-H. Tsai, P.-T. Yu / Signal Processing 74 (1999) 127—151 135

dJ and dJ denote the aggregated distances
4" K 4"
corresponding to xJ and xJ, respectively. Recall
4" K
that, in Section 2.2, dJ is dJ ("dJ) and dJ is
4+  I K 4+
dJ where dJ(dJ, kOj, and k, j3+1,2,2,N,.
H I H
Similarly, dJ is dJ ("dJ) and dJ is dJ where
4"  I K 4" H K
dJ(dJ and kOj, and k, j3+1,2,2,N,. Note that
I H
dJ differs from dJ due to the different o. We
4+ 4"
further let uJ and vJ be, respectively,
uJ""dJ !dJ " (32)
4+ K 4+
and
vJ""dJ !dJ ". (33)
4" K 4"
Note that uJ and vJ are two measures for detecting
the possibility whether the central vector-valued
pixel is contaminated or not. Specifically, a large
value of uJ indicates that the vector median is more
dissimilar to the central vector-valued pixel. This
phenomenon conveys the possibility that the cen-
tral vector-valued pixel is an outlier. That is, the
possibility that the central vector-valued pixel is
contaminated is high. Similarly, a large value of
vJ signifies that the direction of the central vector-
valued pixel is more dissimilar to that of most of the
others in the filter window. This phenomenon ex-
hibits the possibility that the central vector-valued
pixel is a directional outlier [25]. That is, the possi-
bility that the central vector-valued pixel is corrup-
ted is high. These four cases are summarized in
Table 1, and are described in detail in the following
sections.
In the first case, when uJ is large and vJ is large,
this indicates that the possibility that xJ is corrup-
K 4+
ted should be very high. According to the property
of the R-ordering scheme, the aggregated distance
corresponding to a corrupted vector-valued pixel
should be far away from that of the median vector
[4]. The behavior of an AFHM filter is emphasized
to inherit the noise-attenuation and chromaticity-
retention capabilities from the VM and the BVD
filter, respectively. Because xJ is regarded as a cor-
rupted pixel, the possibility of contributing xJ to
K
the output of the AFHM filter should be very low.
In other words, the output of the AFHM filter can
be expressed by the combination of xJ and xJ .
4+ 4"
Moreover, from Figs. 1 and 2, this kind of behavior
can be decided by two control values, aJ and bJ, in
Eq. (12). That is, aJ and bJ should be low such that
the coefficient associated with xJ approaches to
K
zero.
Both the VM and the BVD filter belong to the
class of median-based filters. One of the character-
istics of this kind of filters is to blur edges or details.
Hence, the edges or details-preserving capability of
the AFHM filter is required to enhance it. This is
investigated in the following three cases.
In the second case, when uJ is small and vJ is
large, this indicates that xJ is regarded as an uncor-
K
rupt vector by the VM filter, whereas xJ is re-
K
garded as a corrupted vector by the BVD filter.
This creates a conflict whether xJ is corrupted or
K
not. In this case, for the sake of details preservation,
the possibility of contributing xJ to the output of
4"
the AFHM should be low. Therefore, the behavior
of the AFHM filter is emphasized to inherit the
noise-attenuation and details-preservation capabil-
ities from the VM and the identity filter, respective-
ly. In other words, the output of the AFHM filter
can be expressed by the combination of xJ and xJ.
K
Consequently, in Eq. (12), aJ should be high and
bJ should be low to resulting in the coefficient
associated with xJ approaching zero.
4"

Table 1
According to human concept, the summarization to four cases

Conditions Actions Filtering behavior

uJ vJ aJ bJ

Large Large Low Low Noise attenuation and chromaticity retention
Small Large High Low Noise attenuation and details preservation
Large Small Low High Chromaticity retention and details preservation
Small Small Very high Very high Details preservation
136 H.-H. Tsai, P.-T. Yu / Signal Processing 74 (1999) 127—151

In the third case, when uJ is large and vJ is 4.2. The design of the fuzzy rule-based system
small, this indicates that xJ is regarded as a corrup-
K
ted vector by the VM filter, whereas xJ is regarded A fuzzy rule-based system consists of a set of
K
as an uncorrupted vector by the BVD filter. This fuzzy rules represented in If—then format
also creates a conflict which is the converse case [15,22,29]. According to human concept stated in
with respect to the second case. For the sake of Section 4.1, the behavior of the AFHM filter can be
details preservation, the possibility of contributing described by means of four fuzzy rules:
xJ to the output of the AFHM should be low. The
4+ Rule 1: If uJ is large and vJ is large,
behavior of the AFHM filter is emphasized to in-
then aJ is low and bJ is low.
herit the chromaticity-retention and details-preser-
vation capabilities of the BVD and the identity Rule 2: If uJ is small and vJ is large,
filter, respectively. In other words, the output of the then aJ is high and bJ is low. (34)
AFHM filter can be expressed by the combination
of xJ and xJ. Consequently, in Eq. (12), Rule 3: If uJ is large and vJ is small,
4" K then aJ is low and bJ is high.
aJ should be low and bJ should be high to result in
the coefficient associated with xJ approaching Rule 4: If uJ is small and vJ is small,
4+
zero. then aJ is very high and bJ is very high.
In the fourth case, when uJ is small and vJ is
small, this indicates that the possibility that two In (34), two linguistic variables uJ and vJ are
median vectors based on Eqs. (10) and (11) are defined by Eqs. (32) and (33), respectively. In addi-
xJ should be very high. The output of the AFHM tion, five linguistic terms, small, large, high, low,
K very high can be expressed by fuzzy sets with 1-D
filter is identical to the output of the identity filter
since the possibility that xJ is not corrupted should shape membership functions [15]. In this paper, all
K of the membership functions of these linguistic
be very high. In this case, the AFHM filter achieves
these three objectives: noise attenuation, terms have the same support, [0,1]. Note that it is
chromaticity retention and details preservation. So, necessary to normalize uJ and vJ in order to sim-
in Eq. (12), aJ and bJ should be very high to result plify the implementation. In general, the member-
in these two coefficients associated with xJ and ship functions of these linguistic terms can be 1-D
4+ nonlinear functions. For example, Fig. 3 illustrates
xJ approaching zero.
4" the membership functions of small and large are
From the above scenario, the behavior of
AFHM filters is heavily controlled by aJ and bJ. represented by
However, the functions of aJ and bJ are highly




x!c 
nonlinear and are difficult to be precisely represent- exp !  if x3[0,1],
k (x)" d
ed by convectional mathematical models. In  
this paper, we propose a fuzzy rule-based system 0, else
and a learning algorithm to model the functions (35)
of aJ and bJ. Two characteristics of our approach
are: (i) the above scenario can be naturally
expressed by fuzzy rules; (ii) the highly non-
linear functions can be efficiently modelled
by a fuzzy rule-based system and a learning algo-
rithm. The AFHM filters can effectively take
the merits of the behavior of a VM, a BVD, and
an identity filter to achieve these three objec-
tives simultaneously, because they exploit fuzzy
techniques and learning algorithms to adjust
aJ and bJ with respect to the distinct window Fig. 3. Examples of two gaussian-type membership functions
contents. for linguistic terms small and large.
 H.-H. Tsai, P.-T. Yu / Signal Processing 74 (1999) 127—151 137

Fig. 4. (a) An example of the shape of linguistic variable aJ. (b) An example of the shape of linguistic variable bJ.

and



x!c 
exp !  if x3[0,1],
k (x)" d
 
0, else
(36)
where c "0, c "1, and d "d "0.3. Fig. 5. (a) An example of the form of 1-D step-type function for
    linguistic small. (b) An example of the form of 1-D step-type
Two 2-D continuous functions to decide aJ and
function for linguistic large.
bJ can be obtained if these four fuzzy rules are
merged [1]. For instance, these two 2-D continu-
ous functions are depicted in Fig. 4 . These two 2-D the output of AFHM filters becomes a complicated
functions can be viewed as the membership func- nonlinear function. In this case, the problem of the
tions of linguistic variables aJ and bJ. In most learning algorithm is either that the convergent
fuzzy rule-based systems with fuzzy inference tech- speed of the learning algorithm is slow or that the
niques, these two 2-D membership functions are learning algorithm often is stuck on a local min-
obtained by merging 1-D membership functions of imum. To circumvent this problem, the 1-D non-
linguistic terms, small and large, and their fuzzy linear membership function with continuous type
relations to aJ and bJ [15,22,29]. However, in can be approximated by a 1-D step-shape function.
some applications such as the field of signal pro- For example, the membership functions of linguis-
cessing, it is difficult to obtain these two member- tic terms small and large shown in Fig. 3 can be
ship functions accurately. The problem can be approximated as 1-D step-shape functions illus-
circumvented by approximated functions which trated in Fig. 5. By the similar method, the linguis-
can be obtained by a learning approach [1]. tic terms low and high can also be approximated as
1-D step-shape functions. In addition, very is a lin-
4.3. Learning algorithm guistic hedge (or a modifier), and it is associated
with one of mathematical operators, for instance,
The fuzzy rule-based system can be made adap- concentration [29]. Hence, the output of AFHM
tive by adding a learning algorithm to adjust the filters yields a more simple function. Moreover,
parameters of nonlinear membership functions, for the implementation can be simplified and the
instance, c and d in Eq. (35). However, training time can be reduced. Furthermore, the
 
a gradient-decent learning algorithm, such as the 2-D continuous membership functions can be ap-
LMS algorithm, is involved to optimize the para- proximated by 2-D step-shape functions. A 2-D
meters of nonlinear membership functions. Then, step-shape function comprises a finite number of
138 H.-H. Tsai, P.-T. Yu / Signal Processing 74 (1999) 127—151

piecewise regions, such as Fig. 6(a, b). The two 2-D output of an AFHM filter. Let 1)n)"S " where
NO
step-shape functions are the membership functions "S " denotes the cardinality of the index set S . In
NO NO
corresponding to linguistic variable aJ and bJ, Eq. (39), the weights a and b constitute the
NO NO
respectively. error surface e . On the basis of the gradient-
NO
This subsection describes the learning algorithm decent method, a and b can be adjusted in an
NO NO
based on the LMS algorithm, and investigates its iterative fashion along with the error surface to-
faster convergence property. First, we want to de- ward the optimum solution. The updated rules of
rive the updated rules (or delta rules) used in the a and b can be, respectively, obtained as
NO NO
learning algorithm. Suppose that the supports of a (n#1)
uJ and vJ can be divided into P and Q intervals, NO
respectively. Let uJ and vJ belong to the pth inter- "a (n)#g (n)b (n)(yJ!ŷJ)2(xJ!xJ ) (40)
NO NO NO K 4"
val and the qth interval, respectively. aJ and bJ are and
separately calculated by two 2-D step-shape func-
b (n#1)"b (n)#g (n)(yJ!ŷJ)2
tions, f ( ) , ) ) and f ( ) , ) ), which are defined by NO NO NO
? @
;[xJ !xJ #a (n)(xJ!xJ )],
aJ"a "f (h (uJ),h (vJ)) (37) 4" 4+ NO K 4"
NO ?   (41)
and
where g denotes the learning-rate parameter as-
NO
bJ"b "f (h (uJ),h (vJ)), (38) sociated with region (p,q). For detailed derivations
NO @  
where h ( ) ) and h ( ) ) are two index functions which of the updated rules refer to Appendix A. Conse-
  quently, the values of all piecewise regions can be
decide (uJ,vJ) belong to which region. Moreover,
h (uJ)"p and h (vJ)"q. Furthermore, two func- obtained via the training process according to Eqs.
  (40) and (41). For instance, the two membership
tion values of the piecewise region (p,q) of f ( ) , ) )
? functions of linguistic variables aJ and bJ repre-
and f ( ) , ) ) are equal to a and b , respectively.
@ NO NO sented by 2-D step-shape functions are depicted in
The values of a and b can be trained by means
NO NO Fig. 6(a, b), respectively. For example, p"4 de-
of the LMS algorithm which is subject to minimize
the cost function (or error surface) e with respect notes 0.3)uJ(0.4, and q"6 denotes
NO 0.5)vJ(0.6. Hence, aJ is a "0.978 and bJ is
to region (p,q) [12], 
b "0.295.

e "E[""e""]"E[""yJ!ŷJ""], (39) Next, on the basis of pattern-by-pattern fashion
NO  
where e represents the estimated-error vector, the learning algorithm is summarized as follows.
∀l3S "+j : (h (uH),h (vH))"(p,q),, and E denotes We call this learning algorithm as Algorithm I.
NO  
the statistical expectation operator. Moreover, Step 1. Set k"1 and l"1 where k and l denotes
yJ"[yJ,yJ,yJ]2 is the desired pixel and ŷJ is the the kth epoch and the lth training pattern,
P E @

Fig. 6. (a) An example of the 2-D step-shape function f . (b) An example of the 2-D step-shape function f .
? @
 H.-H. Tsai, P.-T. Yu / Signal Processing 74 (1999) 127—151 139

respectively. For example, the lth training Property 4. Let the lth training pattern (XJ, yJ) be-
pattern comprises the lth observation long to region (p,q), and l be the nth element of S .
NO
sample matrix XJ and the corresponding That is, on the basis of pattern-by-pattern fashion
desired sample yJ, i.e., (XJ, yJ). Next, set (XJ, yJ) is regarded as the nth training pattern asso-
n"1 and initial values to a (n), b (n) ciated with region (p,q). According to updated rules
NO NO
and g (n) for each p and q where (40) and (41), the learning algorithm converges with
NO
1)p)P and 1)q)Q. respect to region (p,q) if g (n) is chosen as
NO
Step 2. At the kth epoch, the lth training pattern
g (n)"
is trained. A VM, a BVD and an iden- NO
tity filter are invoked to produce g
 ,
(xJ , dJ , dJ ), (xJ , dJ , dJ ) and
4+ 4+ K 4+ 4" 4" K 4" ""xJ !xJ ""#[a (n)#b (n)]""xJ!xJ ""
xJ, respectively. Next, the values of uJ and 4" 4+ NO NO K 4"
K (42)
vJ are calculated by Eqs. (32) and (33),
respectively. Suppose that h (uJ)"p and where g is the initial value of learning-rate para-
 
h (vJ)"q. Thus, (uJ,vJ) belongs to region meter, and 0(g )1.
 
(p,q).
Step 3. By Eqs. (40) and (41), a (n#1) and Proof. Let
NO
b (n#1) are evaluated. If l is equal to
NO A"(yJ!ŷJ)2(xJ!xJ ) (43)
¸;K, go to Step 4. Otherwise, set K 4"
l"l#1, and go to Step 2. and
Step 4. If the learning algorithm achieves the satis-
B"(yJ!ŷJ)2[xJ !xJ #a (n)(xJ!xJ )].
factory performance or the maximum ep- 4" 4+ NO K 4"
och, i.e., k'k , then exit. Otherwise, set (44)

n"1, l"1 and k"k#1, and go to Let e and e be the estimated-error vector after and
Step 2. before updating a and b , respectively. We want
NO NO
to show that E[""e""]!E[""e""])0 when g (n) is
Finally, we investigate the convergence property of NO
chosen to satisfy Eq. (42). According to updated
the learning algorithm mentioned above. The conver- rules (40) and (41), e can be obtained as
gence property is necessary to take into account

  
(1!a (n#1))b (n#1) 2 xJ
NO NO 4"
e"yJ! 1!b (n#1) xJ
NO 4+
a (n#1)b (n#1) xJ
NO NO K

  
(1!(a (n)#g (n)b (n)A))(b (n)#g (n)B) 2 xJ
NO NO NO NO NO 4"
"yJ! 1!(b (n)#g (n)B) xJ
NO NO 4+
(a (n)#g (n)b (n)A)(b (n)#g (n)B) xJ
NO NO NO NO NO K

  
!B#a (n)B#b (n)A#g (n)b (n)AB 2 xJ
NO NO NO NO 4"
"e#g (n) B xJ . (45)
NO 4+
!a (n)B!b (n)A!g (n)b (n)AB xJ
NO NO NO NO K

Without loss of generality, let g (n) approach 0.

while devising learning algorithms [12,17]. The NO
Thus, Eq. (45) can be rewritten as
learning algorithm mentioned above possesses the
convergence property which is analyzed as follows. e"e#g (n)c, (46)
NO
140 H.-H. Tsai, P.-T. Yu / Signal Processing 74 (1999) 127—151

where during the learning process. Algorithm II possesses

a potentially faster convergence and obtains

  
!B#a (n)B#b (n)A 2 xJ a lower ensemble-averaged error than Algorithm
NO NO 4"
c" B xJ I does. Therefore, the time complexity of the design
4+ of the AFHM filters can be reduced greatly. Simu-
!a (n)B!b (n)A xJ
NO NO K lation results are illustrated in Fig. 7. In general,
"B(xJ !xJ )#a (n)B(xJ !xJ) g is determined empirically for different cases. We
4+ 4" NO 4" K 
set 0.02 to g in our simulations after many values
#b (n)A(xJ !xJ) 
NO 4" K of g are evaluated.

"!(C #C #C )e (47)
  
and C , C2 and C are as follows, respectively,
  5. Simulation results
C "[""xJ !xJ ""#a (n)(xJ!xJ )2
 4" 4+ NO K 4" In this simulation, three RGB color images,
;(xJ !xJ )],
4" 4+ Lenna, Baboon, and Peppers images, with size
C "a (n)[(xJ !xJ )2(xJ !xJ) 480;512 are examined and are shown in Figs.
 NO 4" 4+ 4" K 11(a) and 12(a), respectively. For convenience, let
#a (n)""xJ!xJ ""],
NO K 4" I and I be a whole original image and its corrup-
G G
C "b (n)""xJ!xJ "". ted version, respectively, where i3+B,¸,P,. For
 NO K 4" example, I and I , denote Baboon image and its
Accordingly, we can rewrite Eq. (46) to be corrupted version, respectively. The artificial addi-
e"[1!g (n)(C #C #C )]e. (48) tive noises are generated to simulate the corruption
NO    process. In addition, a kind of channel dependence
If g (n) is chosen as is simulated by correlation coefficients o , o ,
NO 0% %
g (n) and o which are 0.5 in this simulation. Three
NO 0
quantitative measures are used to assess the filter-
g ing performance. The first measure is the nor-
" 
(C #C2#C ) malized mean squared error (NMSE),
 
g *")""yJ!ŷJ""
"  , NMSE" J , (51)
""xJ !xJ ""#[a (n)#b (n)]""xJ!xJ "" *")""yJ""
4" 4+ NO NO K 4" J
(49) where yJ and ŷJ represent the original and the
then the statistical expectation operator E is added estimated image vector-valued pixels, respectively.
into two sides of Eq. (48) and g in Eq. (48) is The second measure is the mean chromaticity error
NO (MCRE) which is based on the distances on Max-
substituted by Eq. (49). We can obtain
well triangle plan
E[""e""]"E[""(1!g )e""]"(1!g )E[""e""]
 
*") "" pJ!p̂J""
E[""e""] MCRE" J , (52)
N "(1!g ))1 ¸;K
E[""e""] 
where pJ and p̂J are the intersection points of
N E[""e""]!E[""e""])0. (50)
yJ and ŷJ with Maxwell triangle plan, respectively
Therefore, the property holds. Note that using the [25]. The distance between two vector-valued
similar way we can prove that this learning algo- pixels on Maxwell triangle plan is to indicate the
rithm converges with respect to other regions. 䊐

The modified version of Algorithm I is called  Due to the limited space, Peppers image is not shown in this
Algorithm II, which calculates g (n) from Eq. (42) paper. It can be found in many related published papers.
NO
 H.-H. Tsai, P.-T. Yu / Signal Processing 74 (1999) 127—151
Fig. 7. Two examples of ensemble-averaged MAE learning curves on the design of AFHM filters using Algorithms I and II. MAE
denotes mean absolute error using ¸ norm to calculate the absolute error (distance) of two vectors.

chromaticity error of them. The third measure is
the mean color difference (MCD) which is based on
the ¸HaHbH space,
*") ""hJ!hK J""
MCD" J , (53)
¸;K
where hJ and hK J are the points in the ¸HaHbH space
corresponding to yJ and ŷJ, respectively
[13,19,25]. MCD indicates the error of color im-
ages in human perception. A 5;5 filter window is
employed in this simulation. For convenience, the
simulated filters are denoted as follows:
E AM: arithmetic mean filter
E GVD/ATM: generlized vector directional fil-
ter/a-trimmed mean filter [25]
E MM: marginal median filter [20]
E VM: vector median filter [3]
E BVD: basic vector directional filter [25]
E DD: distance and directional filter [14]
E ANNM: adaptive nearest neighbor multichannel
filter [19]
The comparisons of filtering performance results
with three distinct measures are shown in Fig. 8. It
is obvious to conclude that the performance of
AFHM filters is better than that of other simulated
filters under consideration. Although it is not easy
to evaluate the performance of details-preserving
capability of filters, one method for this evaluation
is to make a comparison among the filtering results
of the noise-free (0%) images. From Fig. 8, the
141
details-preserving capability of the family of
AFHM filters is superior to that of other simulated
filters. The reason is that details preserving capabil-
ity is emphasized by human concept on the design
of AFHM filters. One contribution of this paper is
to reflect human concept which is efficiently ex-
pressed by fuzzy rules on the design of AFHM
filters. Therefore, our approach provides a powerful
and flexible framework to design multichannel fil-
ters. On the design of AFHM filters, the training
and the testing images remains the same, so far.
Note that an AFHM filter is trained using a pair of
training images (I , I), and then is applied to
G G
restore I. In this paper, this capability is called
G
memorized capability of filters. In other words,
a filter is trained by a pair of training images in
advance to memorize prior knowledge in terms of
weights. In addition, another important robust ca-
pability is also considered in this simulation. In
order to assess the robust performance the training
and testing images should be different [1,7,12].
Therefore, the robust capability must be taken into
account on the design of AFHM filters.
In this simulation, two design approaches are
utilized to examine the robust capability of AFHM
filters. Let I be divided into two separated subim-
G
ages I1 and I0 such that I "I16I0 where I1 is
G G G G G G
a square portion of I from (160, 160) to (415, 415).
G
For examples, I1 and I1 encompassed by white
*
squares are shown in Figs. 11(a) and 12(a), res-
pectively. Then I0 is the portion of I excluding
G G
142 H.-H. Tsai, P.-T. Yu / Signal Processing 74 (1999) 127—151

Fig. 8. The comparisons of filtering performance results for the case of impulsive noise. The training images applied to AFHM filters are
the same as the testing images. But AFHM—L and AFHM—P denote the AFHM filters trained using Lenna and Peppers images,
respectively.
 H.-H. Tsai, P.-T. Yu / Signal Processing 74 (1999) 127—151 143

Fig. 9. The comparisons of filtering performance results for the case of impulsive noise. An AFHM filter is trained using (I0, I0Y), and
then is applied to filter I1Y. The filtering results are exhibited in (a), (c) and (e). (b), (d) and (f) show the results of another AFHM filter
which is trained using (I1 , I1Y), and then applied to restore I0Y.
144 H.-H. Tsai, P.-T. Yu / Signal Processing 74 (1999) 127—151

Fig. 10. The comparisons of filtering performance results for the case of impulsive noise. An AFHM filter is trained using (I0, I0Y), and
* *
then is applied to filter I1Y. The filtering results are exhibited in (a), (c) and (e). (b), (d) and (f) show the results of another AFHM filter
*
which is trained using (I1 , I1Y), and then applied to restore I0Y.
* * *
 H.-H. Tsai, P.-T. Yu / Signal Processing 74 (1999) 127—151 145

I1 (i.e., I !I1). We further let I1Y and I0Y repres- AFHM filter is applied to filter the testing image
G G G G G
ent the corrupted versions of I1 and I0, respective- I1Y, or vice versa. Figs. 9 and 10 illustrate that the
G G G
ly. In the first approach, (I0, I0Y) is exploited to filtering performance of this kind of AFHM filters
G G
train an AFHM filter in advance and then the is better than that of other simulated filters. In the

Fig. 11(a)—(f). (a) Original Baboon image (I ). I1 is the region encompassed by a while square on (a). (b) Baboon image corrupted by
6% impulsive noise. (c) Restored image of a 5;5 AM filter. (d) Restored image of a 5;5 GVD/ATM filter. (e) Restored image of a 5;5
MM filter. (f) Restored image of a 5;5 VM filter.
146 H.-H. Tsai, P.-T. Yu / Signal Processing 74 (1999) 127—151
Fig. 11(g)—(j). (g) Restored image of a 5;5 BVD filter. (h) Restored image of a 5;5 DD filter. (i) Restored image of a 5;5 ANNM filter.
(j) Restored image of a 5;5 AFHM filter trained using (I , I ).
second approach, (I , I) is taken to train an
G G
AFHM filter in advance and then the AFHM filter
is applied to filter the testing image I where iOj.
H
In this simulation, the pairs of training images are
(I , I ) and (I , I ), and the testing image is I .
* * . .
Let AFHM—L and AFHM—P stand for AFHM
filters trained using (I , I ) and (I , I ), respec-
* * . .
tively. Fig. 8(a, c, e) shows that the filtering perfor-
mance of this class of AFHM filters is superior to
that of other simulated filters. From Figs. 8—10, we
claim that AFHM filters possess highly robust ca-
pability. That is, AFHM filters are still able to
obtain a better filtering performance than other
simulated filters do, even though some of images
are not taken to train AFHM filters in advance.
Besides these quantitative measures for compari-
sons of filtering performance results, a qualitative
measure is also required for human visual percep-
tion. The smeared version of original Baboon im-
age is corrupted by 6% impulsive noise, and is
shown in Fig. 11(b). The restored images shown in
Fig. 11(c—i) are obtained by applying the simulated
filters to restore this smeared Baboon image. Fig.
11(j) exhibits the resulted image restored by ap-
plying a 5;5 AFHM filter trained using (I , I ).
To compare Fig. 11(j) with the others, Fig. 11(j) has
better visual quality than the others do. Especially,
the region with beard in Fig. 11(j) is more clear than
that in the others. This indicates that AFHM filters
possess the highly detailed-preserving capability.
Moreover, the chromaticity of Fig. 11(j) is closer to
Fig. 11(a) than that of the others. Next, Fig. 12
shows the comparisons of qualitative measures for
evaluating the robust capability of AFHM filters. It
is manifest to observe that Fig. 12(j, k) have better
visual quality than the others which are blurred.
 H.-H. Tsai, P.-T. Yu / Signal Processing 74 (1999) 127—151 147

Fig. 12. (a) Original Lenna image (I ). I1 is the region encompassed by a while square on (a). (b) Lenna image corrupted by 4%
* *
impulsive noise. (c) Restored image of a 5;5 AM filter. (d) Restored image of a 5;5 GVD/ATM filter. (e) Restored image of a 5;5 MM
filter. (f) Restored image of a 5;5 VM filter. (g) Restored image of a 5;5 BVD filter. (h) Restored image of a 5;5 DD filter. (i) Restored
image of a 5;5 ANNM filter. (j) Restored image of a 5;5 AFHM filter trained using (I1 ,I1Y). (k) Restored image of a 5;5 AFHM filter
* *
trained using (I0,I0Y).
* *
148 H.-H. Tsai, P.-T. Yu / Signal Processing 74 (1999) 127—151

Fig. 13. Testing image is Baboon image corrupted by 4% impulsive noise. (a) Restored image of a 5;5 AHMF—L filter trained using
(I , I ). (b) Restored image of a 5;5 AHMF—P filter trained using (I , I ).
* * . .

From Figs. 8 and 13, one observation is worthy to duced. Finally, the extensive simulation results il-
remark that the AFHM—L filter has better filtering lustrate that AFHM filters not only possess the
performance than that of the AFHM—P filter. As capabilities of noise attenuation, chromaticity re-
observed, details in Lenna image are more rich tention, and edges or details preservation but also
than that in Peppers image. The former possesses possess robust and adaptive capabilities.
better details-preserving capability than the latter
does, because the former is trained using Lenna
image which has more details. Therefore, the filter- Acknowledgements
ing performance of the former is superior to that of
the latter if the testing image has more detailed The authors would like to appreciate Prof. A.N.
image structures. Hence, we briefly conclude that Venetsanopoulos, Prof. P.E. Trahanias, and Dr. D.
the selection of appropriate training images is im- Karakos for providing correlated noise generation
portant to design AFHM filters for further improv- programs, and to thank Prof. I. Pitas, Prof. C. Ko-
ing the filtering performance. tropoulos, and Prof. K.N. Plataniotis for discussing
the aspects of filter design. Finally, authors also thank
the anonymous reviewers for valuable comments.
6. Conclusions
Appendix A.
A novel class of nonlinear multichannel filters
called AFHM filters has been proposed in this
In the following, we derive the updated rules of
paper. Some deterministic properties of AFHM
a and b , i.e., Eqs. (40) and (41). First,
filters have been developed. On the design of NO NO
AFHM filters, human concept has been efficiently e "E[""yJ!ŷJ""]
NO 
expressed by fuzzy rules to decide control values for "E[""yJ""]!E[ yJ2ŷJ]#E[""ŷJ""], (A.1)
adapting different window contents. This fuzzy  
where ∀l3S . Then we have
technique provides a flexible and powerful ap- NO
proach to efficiently model the filtering behaviors. E[ yJ2ŷJ]"(1!a )b E[ yJ2xJ ]
NO NO 4"
Moreover, a faster convergence property of our
#(1!b )E[ yJ2xJ ]
learning algorithm has been investigated such that NO 4+
the time complexity of AFHM filters can be re- #a b E[ yJ2xJ] (A.2)
NO NO K
 H.-H. Tsai, P.-T. Yu / Signal Processing 74 (1999) 127—151 149

and # (1!2b )a E[xJ2 xJ]

NO NO 4+ K
E[""ŷJ""]"E[""(1!a )b xJ #2(a !a )b E[xJ2xJ ]
N O N O 4" NO NO NO K 4"
#(1!b ) xJ #a b xJ""] "0. (A.5)
N O 4+ NO NO K
"(1!a )b E[""xJ ""] e and N Oe denote the gradients of error
NO NO 4" ?N O N O @ NO
surface e upon a and b . These two equations,
#(1!b )E[""xJ ""] NO NO NO
NO 4+ (A.4) and (A.5), are known as Wiener—Hopf equa-
tions [12]. However, for an ‘unknown’ environ-
# a b E[""xJ""]
NO NO K ment it is difficult to obtain the optimum solution
#2+(1!a )(b !b )E[xJ2 xJ ] by mean of solving the equations. Therefore, the
NO NO NO 4" 4+
2
LMS algorithm (an adaptive approach) is con-
# (b !b )a E[xJ xJ] sidered to adjust a and b for adapting the
NO NO NO 4+ K NO NO
#(a !a )b E[xJ2 xJ ],. (A.3) ‘unknown’ environment. In the random process,
NO NO NO K 4" the LMS algorithm is based on instantaneous esti-
We know that the weights a and b constitute mates of correlation functions which include the
NO NO
the error surface e of region (p,q). To obtain the cross-correlation and auto-correlation function,
NO
optimal solution of e , we can differentiate Eq. and can be estimated as follows:
NO
(A.1) with respect to a and b , then set the
NO NO E[ yJ2xJ ]"yJ2xJ ,
results to zero, that is, 4" 4"
E[ yJ2xJ ]"yJ2xJ ,
e 4+ 4+
?N O N O 2 2
E[ yJ xJ]"yJ xJ,
*e K K
" NO
*a E[""xJ ""]"""xJ "",
NO 4" 4"
"b +E[ yJ2xJ ]!E[ yJ2xJ], E[""xJ ""]"""xJ "",
NO 4" K 4+ 4+
!(1!a )b E[""xJ ""]a b E[""xJ""] E[""xJ""]"""xJ"",
NO NO 4" NO NO K K K
#(b !b )+E[xJ2 xJ]!E[xJ2 xJ ], E[xJ2 xJ ]"xJ2 xJ ,
NO NO 4+ K 4" 4+ 4" 4+ 4" 4+
2 2
E[xJ xJ]"xJ xJ,
# (1!2a )b E[xJ2xJ ] 4+ K 4+ K
NO NO K 4"
E[xJ2 xJ ]"xJ2 xJ . (A.6)
"0 (A.4) K 4" K 4"
The definitions in Eq. (A.6) can be generalized to
and include nonstationary environments [12]. As
e known, signals are highly nonstationary in image
@N O N O processing [17]. Therefore, the definitions in Eq.
*e (A.6) can be applied in the following derivations.
" NO
*b Thus, the gradients of the error surface e upon
NO NO
the weights a and b can be, respectively, rewrit-
"!+(1!a )E[ yJ2xJ ] NO NO
NO 4" ten as
!E[ yJ xJ ]#a E[ yJ2xJ],
2
e "b yJ2(xJ !xJ)
4+ NO K ?N O N O NO 4" K
# (1!a )b E[""xJ ""] !(1!a )b ""xJ ""#a b ""xJ""
NO NO 4" N O N O 4" NO NO K
!(1!b )E[""xJ ""] #(b !b )xJ2 (xJ!xJ )
NO 4+ NO N O 4+ K 4"
# a b E[""xJ""]
NO NO K #(1!2a )b xJ2xJ
NO NO K 4"
#(1!a )(1!2b )E[xJ2 xJ ] "! b (yJ!ŷJ)2(xJ!xJ ) (A.7)
NO NO 4" 4+ NO K 4"
150 H.-H. Tsai, P.-T. Yu / Signal Processing 74 (1999) 127—151

and [5] R.-C. Chen, P.-T. Yu, An optimal design of fuzzy (m, n)
2 rank order filtering with hard decision neural learning, J.
e "!yJ [(1!a )xJ !xJ #a xJ]
@N O N O N O 4" 4+ NO K Inform. Sci. Eng. 11 (1995) 567—593.
[6] Y.S. Choi, R. Krishnapuram, A robust approach to image
#(1!a )b ""xJ ""
N O N O 4" enhancement based on fuzzy logic, IEEE Trans. Image
! (1!b )""xJ ""#a b ""xJ"" Process. 6 (6) (June 1997) 808—825.
N O 4+ NO NO K [7] K.K. Delibasis, P.E. Undrill, G.G. Cameron, Genetic algo-
#(1!a )(1!2b ) xJ2 xJ rithm implementation of stack filter design for image resto-
NO N O 4" 4+
ration, IEE Proc.-Visual Image Signal Process. 143 (3)
# (1!2b )a xJ2 xJ
N O N O 4+ K (June 1996) 177—183.
#2(a !a )b xJ2xJ [8] J.P. Fitch, E.J. Coyel, N.C. Gallagher Jr., Root properties
NO NO NO K 4" of and convergence rates of median filters, IEEE Trans.
"!(yJ!ŷJ)2[x J !x J Acoust. Speech Signal Process. ASSP-33 (1) (February
4" 4+ 1985) 230—240.
#a ( xJ!xJ )]. (A.8)
NO K 4" [9] N.C. Gallagher Jr., G.L. Wise, A theoretical analysis
of the properties of median filters, IEEE Trans. Acoust.
By the method of steepest decent, a and b can
NO NO Speech Signal Process. ASSP-29 (6) (December 1981)
be adjusted in an iterative fashion along with the 1136—1141.
error surface toward the optimum solution. Given [10] R.M. Haralick, L.G. Shapiro, Computer and Robot Vi-
the old values of the weights a (n) and b (n) at sion, Vol. 1, Addison-Wesley, Reading, MA, 1992.
NO NO
iteration n, the updated values of a and b at [11] R.C. Hardie, G.R. Arce, Ranking in RN and its use in
NO NO multivariate image estimation, IEEE Trans. Circuits Sys-
next iteration n#1 are, respectively, calculated as
tems Video Technol. 1 (2) (June 1991) 197—209.
a (n#1)"a (n)#*a (n) [12] S. Haykin, Neural Networks, Macmillan College Publish-
NO NO NO ing Company, New York, 1995.
"a (n)!g e [13] A.K. Jain, Fundamentals of Digital Image Processing,
NO N O ?N O N O
Prentice-Hall, Englewood Cliffs, NJ, 1989.
"a (n)#g b ( yJ!ŷJ)2(xJ!xJ )
NO NO NO K 4" [14] D. Karakos, P.E. Trahanias, Generalized multichannel
(A.9) image-filtering structures, IEEE Trans. Image Process.
6 (7) (July 1997) 1038—1045.
and [15] G.J. Klir, B. Yuan, Fuzzy Sets and Fuzzy Logic Theory
and Applications, Prentice-Hall, Upper Saddle River, NJ,
b (n#1)"b (n)#*b (n)
NO NO NO 1995.
"b (n)!g e [16] C. Kotropoulos, I. Pitas, Multichannel ¸-filters based on
NO N O @N O N O marginal data ordering, IEEE Trans. Signal Process. 42
"b (n)#g (yJ!ŷJ)2 (10) (October 1994) 2581—2595.
NO NO [17] C. Kotropoulos, I. Pitas, Adaptive LMS ¸-filters for noise
;[xJ !xJ #a (xJ!xJ )]. suppression in images, IEEE Trans. Image Process. 5 (12)
4" 4+ NO K 4" (December 1996) 1596—1608.
(A.10)
[18] K.N. Plataniotis, D. Androutsos, A.N. Venetsanopoulos,
Thus, Eqs. (40) and (41) are derived. 䊐 Multichannel filters for image processing, Signal Process-
ing: Image Communication 9 (1997) 143—158.
[19] K.N. Plataniotis, V. Sri, D. Androutsos, A.N. Vene-
tsanopoulos, An adaptive nearest neighbor multichannel
References filters, IEEE Trans. Circuits Systems Video Technol. 6 (6)
(December 1996) 699—703.
[1] K. Arakawa, Median filters based on fuzzy rules and its [20] I. Pitas, P. Tsakalides, Multivariate ordering in color im-
application to image processing, Fuzzy Sets and Systems age filtering, IEEE Trans. Circuits Systems Video Technol.
77 (1996) 3—13. 1 (3) (September 1991) 247—259.
[2] J. Astola, P. Heinonen, Y. Neuvo, On the roots structures [21] I. Pitas, A.N. Venetsanopoulos, Nonlinear Digital Filters
of median and median type filters, IEEE Trans. Acoust. — Principles and Applications, Kluwer Academic Pub-
Speech Signal Process. ASSP-35 (8) (August 1987) lishers, Boston, 1990.
1199—1201. [22] T.J. Ross, Fuzzy Logic with Engineering Applications,
[3] J. Astola, P. Haavisto, Y. Neuvo, Vector median filtering, Mc-Graw-Hill, New York, 1995.
IEEE Proc. 78 (4) (April 1990) 678—689. [23] F. Russo, G. Ramponi, A fuzzy filter for images corrupted
[4] V. Barnett, The ordering of multivariate data, J. Roy. by impulse noise, IEEE Signal Process. Lett. 3 (6) (June
Statist. Soc. A 139 (3) (1976) 318—343. 1996) 168—170.
 H.-H. Tsai, P.-T. Yu / Signal Processing 74 (1999) 127—151
[24] K. Tang, J. Astola, Y. Neuvo, Nonlinear multivariate im-
ages filtering techniques, IEEE Trans. Image Process. 4 (6)
(June 1995) 788—798.
[25] P.E. Trahanias, D. Karakos, A.N. Venetsanopoulos, Di-
rectional processing of color images: theory and experi-
mental results, IEEE Trans. Image Process. 5 (6) (June
1996) 868—880.
[26] X. Yang, P.S. Toh, Adaptive fuzzy multilevel median filter,
IEEE Trans. Image Process. 4 (5) (May 1995) 680—682.
151
[27] P.-T. Yu, R.-C. Chen, Fuzzy stack filters — their defini-
tions, fundamental properties, and application in image
processing, IEEE Trans. Image Process. 5 (6) (June 1996)
838—854.
[28] P.-T. Yu, E.J. Coyel, Convergence behavior and N-roots of
stack filters, IEEE Trans. Acoust. Speech Signal Process.
ASSP-38 (9) (September 1990) 1529—1544.
[29] H.-J. Zimmermann, Fuzzy Set Theory and its Applica-
tions, Kluwer Academic Publishers, Boston, 1991.