Image and Vision Computing 21 (2003) 10191026 www.elsevier.

com/locate/imavis

Image analysis by bidimensional empirical mode decomposition

J.C. Nunes*, Y. Bouaoune, E. Delechelle, O. Niang, Ph. Bunel
` mes (LERISS), Universite Paris XII-Val de Marne, Bat P2-Piece 230 61 Avenue du Laboratoire dEtude et de Recherche en Instrumentation, Signaux et Syste General de Gaulle, 94010 Creteil Cedex, France Received 30 September 2002; received in revised form 18 April 2003; accepted 9 May 2003

Abstract Recent developments in analysis methods on the non-linear and non-stationary data have received large attention by the image analysts. In 1998, Huang introduced the empirical mode decomposition (EMD) in signal processing. The EMD approach, fully unsupervised, proved reliable monodimensional (seismic and biomedical) signals. The main contribution of our approach is to apply the EMD to texture extraction and image ltering, which are widely recognized as a difcult and challenging computer vision problem. We developed an algorithm based on bidimensional empirical mode decomposition (BEMD) to extract features at multiple scales or spatial frequencies. These features, called intrinsic mode functions, are extracted by a sifting process. The bidimensional sifting process is realized using morphological operators to detect regional maxima and thanks to radial basis function for surface interpolation. The performance of the texture extraction algorithms, using BEMD method, is demonstrated in the experiment with both synthetic and natural images. q 2003 Elsevier B.V. All rights reserved.
Keywords: Bidimensional empirical mode decomposition; Texture analysis; Unsupervised texture decomposition; Radial basis function; Surface interpolation

1. Introduction The joint space-spatial frequency representations have received special attention in the elds of image processing, vision and pattern recognition. Huang [15] introduces a multiresolution decomposition technique: the empirical mode decomposition (EMD), which is adaptive and appears to be suitable for non-linear, non-stationary data analysis. We propose a new analysis method of texture images based on bidimensional empirical mode decomposition (BEMD), rstly presented in Ref. [24]. This paper is organized as follow. Section 2 presents state of the art of texture analysis methods. Section 3 presents an introduction to the EMD and its extension on bidimensional data. It describes too the implementation details of sifting process, including the extrema detection by morphological reconstruction and the radial basis function (RBF) for surface interpolation. In Section 4, experimental results on different texture images indicate the interest of this new multiresolution decomposition with statistical descriptors. Finally, a conclusion is presented in Section 5.
* Corresponding author. Tel.: 33-1-45-171-492. E-mail address: nunes@univ-paris12.fr (J.C. Nunes). 0262-8856/03/$ - see front matter q 2003 Elsevier B.V. All rights reserved. doi:10.1016/S0262-8856(03)00094-5

2. Texture analysis methods Texture features are determined by the spatial relations between neighbouring pixels. The difculty of texture analysis is demonstrated by the number of different texture denitions attempted by vision researchers [12,17,28]. Approaches to texture feature extraction and recognition span a wide range of methods. Several books and articles give overviews of the available methodology [9, 33]. There are four major issues in texture analysis [22]: feature extraction (local texture properties), texture discrimination (image partition corresponding to different textures), texture classication (classes denition in nite number, normal and pathological) and shape from texture (reconstruction of 3D surface geometry from texture information). Four major method categories may be identied [20]: statistical with cooccurrence and autocorrelation features [12], geometrical with Voronoi tessellation [33] and structural features, model based with Markov random eld [5,11] and fractal parameters [6,26,32], and multiscale features with AM FM analysis [13], morphological operators [27], Wigner-Ville Distributions [14,31], Gabor lters [8,16], and wavelet transforms [21,23].

1020

J.C. Nunes et al. / Image and Vision Computing 21 (2003) 10191026

A few comparisons between texture feature extraction schemes have been presented in Refs. [22,29,30]. Comparative studies showed different conclusions. Different setups, test images, and ltering methods may be the reasons for the contradicting results. No single approach did perform best or very close to the best for all images. The multiresolution techniques, which based on human visual perception, intend to transform images into a representation in which both spatial and frequency information are present. The most commonly multiscale features used are Wigner distributions [14,31], Gabor functions [2,7,16] and wavelets transforms [18,19,21,34,36]. These multiscale features try to characterize textures by lter responses directly. However, one difculty of the multiresolution analysis is its non-adaptive nature since it uses ltering schemes. We have applied the EMD technique to texture images for two reasons. The rst, the advantage of this technique, the EMD is a fully data driven method [25], does not use any pre-determined lter [8] or wavelet functions [19]. The second, we can easily implement a bidimensional extension of EMD. This paper, which is based on multiscale decomposition, examines the issue of designing texture decomposition by BEMD.

non-stationary analysis, edge detection, image enhancement, recovery of 3D shapes from texture, computational stereopsis, texture segmentation and classication. The sifting algorithm for s [ l2 Z reads as follows: 1. Initialise: r0 s (the residual) and j 1 (index number of IMF), 2. Extract the jth IMF: 3. (a) Initialise h0 rj21 ; i 1; (b) Extract local minima/maxima of hi21 ; (c) Compute upper envelope and lower envelope functions xi21 and yi21 by interpolating, respectively, local minima and local maxima of hi21 ; (d) Compute mi21 xi21 yi21 =2 (mean envelope), (e) Update hi : hi21 2 mi21 and i : i 1; (f) Calculate stopping criterion (standard deviation SDji ) (g) Repeat steps (b) to (f) until SDji # SDMAX and put then sj hi (jth IMF) 4. Update residual rj rj21 2 sj ; 5. Repeat steps 2 4 with j : j 1 until the number of extrema in rj is less than 2. 3.1.2. Finding all the IMFs

3. Texture analysis based on the empirical mode decomposition 3.1. The 1D empirical mode decomposition In this paper, we use the EMD, rst introduced by Huang et al. [15]. Different applications as medical and seismic signals analysis have showed the effectiveness of this method. This method permits to analyse non-linear and nonstationary data. Its principle is to decompose adaptively a given signal into frequency components, called intrinsic mode functions (IMF). These components are obtained from the signal by means of an algorithm called sifting process. This algorithm extracts locally for each mode the highest frequency oscillations out of original signal. 3.1.1. Sifting procedure The sifting procedure decomposes a sampled signal sk by means of the EMD. The sifting procedure is based on two constraints: each IMF has the same number of zero crossings and extrema; each IMF is symmetric with respect to the local mean. Furthermore, it assumes that s has at least two extrema. The EMD represent adaptively non-stationary signals as sums of zero-mean AM FM components [10], i.e. an IMF is an AM FM component. AM FM analysis [13] has been used successfully in a variety of applications including

1. Once the rst set of siftings results in an IMF, dene c1 h1i : This rst component contains the nest spatial scale in the signal. 2. Generate the residue, r1 ; of the image by subtracting out c1 ; r1 I 2 c1 : The residue now contains information about larger scales. 3. Resift to nd additional components r2 r1 2 c2 ; ; rn rn21 2 cn The P superposition of all the IMF reconstructs the data: I n j1 cj rn : We have to determine a criterion for the sifting process to stop. This can be accomplished by limiting the size of the SD, computed from the two consecutive sifting results as: " # K X lhji21 k 2 hji kl2 2 SDji h2 ji21 k k1

3.2. The bidimensional empirical mode decomposition Texture analysis is considered as a challenging task. The ability to effectively classify and segment images based on textural features is of key importance in scene analysis, medical image analysis, remote sensing and many other application areas. Feature extraction is the rst stage of image texture analysis. To extract the 2D IMF during the sifting process, we have used morphological reconstruction

J.C. Nunes et al. / Image and Vision Computing 21 (2003) 10191026

1021

to detect the image extrema and RBF to compute the surface interpolation. A 2D IMF is a zero-mean 2D AM FM component. The image AM FM decomposition [13] is partially unsupervised feature based segmentation algorithm, whereas the EMD is fully unsupervised. In the two methods, the results of the decomposition are very efcient and competitive with the best analysis results [13,25]. We dene a bidimensional sifting process [24]: Identify the extrema (both maxima and minima) of the image I by morphological reconstruction based on geodesic operators; Generate the 2D envelope by connecting maxima points (respectively, minima points) with a RBF; Determine the local mean m1 ; by averaging the two envelopes; Since IMF should have zero local mean, subtract out the mean from the image: I 2 m1 h1 ; repeat as h1 is an IMF. As described above, the process is indeed like sifting to separate the nest local mode from the image rst based only on the characteristic multiscale. The sifting process, however, has two effects to eliminate riding waves and to smooth uneven amplitudes. 3.2.1. Extrema detection Morphological reconstruction is a very useful operator provided by mathematical morphology [1]. Its use in hierarchical segmentation proves its efciency in all the steps of the process, from extrema detection to hierarchical image construction. The image extrema have been detected by using morphological reconstruction based on geodesic operators. We dene the geodesic reconstruction as follows. The grayscale reconstruction IrecI J of I from J is n obtained by iterating grayscale geodesic dilations W 1 nof J under I until a stability is reached, i.e. IrecI n$1 1 J : A well-known use of the morphological reconstruction is the extraction of the extrema (minima and maxima). If we take J I 2 1 (subtract one gray level to every pixel of original image) and if we perform the reconstruction Irec (by geodesic dilation) of J by I ; the difference I 2 Irec corresponds to the indicator function of the maxima of I (Fig. 1). Conversely, the difference between Irecp (reconstruction by geodesic erosion) and I (original image) produces

the indicator function of the minima of I : This extrema detection method is described in Ref. [35]. 3.2.2. Surface interpolation by radial basis function In Ref. [15], Huang proposed to use cubic spline interpolation on non-equidistant sampled data. We have choice to use the RBF rather than the bicubic spline for different reasons developed in Ref. [4]. One of the technical problems is that the cubic spline tting creates distortions near the end points. RBFs are presented as a practical solution to the problem of interpolating incomplete surfaces derived from three-dimensional (3D) medical graphics. A RBF is a function of the form: s x pm x where s is the radial basis function, pm low degree polynomial, typically linear or quadratic, a member of mth degree polynomials in d variables, kk denotes the Euclidian norm, the li s are the RBF coefcients, -F is a real valued function called the basic function, the xi s are the RBF centres. The radial basis approximation method offers several advantages over piecewise polynomial interpolants. The geometry of the known points is not restricted to a regular grid. Also, the resulting system of linear equations is guaranteed to be invertible under very mild conditions. Finally, polyharmonic RBFs have variational characterizations, which make them eminently suited to interpolation of scattered data, even with large data-free regions. These applications include geodesy, geophysics, signal processing, and hydrology. RBFs have also been successful employed for medical imaging and morphing of surfaces in three dimensions. Experimental results demonstrate that high-delity reconstruction is possible from a selected set of sparse and irregular samples. RBF are introduced to compute a continuous surface through a set of irregularly spaced points (extrema) during the sifting process.
N X i1

li Fkx 2 xi k;

x [ Rd ; li [ R;

4. Results and discussion Real-world texture frame, as well synthetic texture frames, has been used to test and validate the proposed approach. The decomposition approach is applied to both synthetic and natural images, created by composing textures selected from Brodatz [3]. A study is performed to show the efciency and performance of the texture extraction. Then, a set of texture images is selected to illustrate the application procedure of the EMD method.

Fig. 1. Morphological reconstruction Irec (by geodesic dilation) of I by I 2 1:

1022

J.C. Nunes et al. / Image and Vision Computing 21 (2003) 10191026

Fig. 2. Photographic texture fabric.

4.1. Texture extraction The 2D decomposition by sifting process of an image provides a representation that is easy to interpret. Every mode (IMF) contains information of a specic scale, which is conveniently separated. Spatial information is retained within the mode. Our algorithm is able to decompose a broad range of texture elements. Examples using images (200 200 in grey scale) from the Brodatz texture album, or using synthetic and real images are shown in Figs. 3, 6 12. To stop the sifting process, we used the standard deviation (SD). We have used SDMAX between 0.05 and 0.75. In a rst step, we applied our algorithm on a synthetic image (Fig. 2a). It is an image of sinusoidal components, built with a sum of three horizontal (h1 40; h2 14; h3 0:2) and three vertical (v1 60; v2 17; v3 0:3) frequencies with respective amplitudes (ah1 40; ah2 30; ah3 190; av1 40; av2 50; av3 190). All stages of results are shown in Figs. 2 and 3. We observe the images of the minima (Fig. 2b), the maxima (Fig. 2c) detection, the mean envelope (Fig. 2d) and the subtraction from mean envelope (Fig. 2e). Fig. 3 shows

the performed decomposition in two modes (Fig. 3b and c) and the residue image (Fig. 3d). To prove the effectiveness of our method, we compare the density proles of the original image and the various frequencies composing it. In Fig. 4, we traced the prole of density of sinusoidal images (corner in top on the left towards the corner in bottom in right-hand side): the sum of the three frequencies (Fig. 4a), the high frequency (Fig. 4b), the medium frequency (Fig. 4c) and the low frequency (Fig. 4d). In Fig. 5, we can observe the density prole of decomposition, the sum of three frequencies (Fig. 5a), the rst mode (Fig. 5b), the second mode (Fig. 5c) and the image residue (Fig. 5d). We can compare these different proles. We observe that the rst mode corresponds to the sinusoidal image with the highest spatial frequency, the second to the medium and the residue to the lowest. These proles are very similar. However, we can watch in residue image (Fig. 5d) an irregularity, which is due to interpolation distortions near the end points. In Fig. 6, the decomposition is performed in two modes. This fabric texture is quite regular, mainly diagonal and horizontal structures, making it well suited to the proposed approach.

Fig. 3. Synthetic texture.

Fig. 4. Synthetic texture (frequency components).

J.C. Nunes et al. / Image and Vision Computing 21 (2003) 10191026

1023

Fig. 5. Synthetic texture (BEMD).

Fig. 6. Photographic fabric texture.

The rst mode corresponds to the woven structure, the second to the pattern and the residue to the horizontal stripes (black and white). In Fig. 7, the decomposition is performed in three modes. It is interesting to observe that the rst mode corresponds to the woven structure, the second to the nest pattern, the third to the medium pattern and the residue to largest pattern. In Fig. 8, the decomposition is performed in three modes. In Fig. 9, the decomposition of MRI is performed in three modes. We observe the three modes and the residue, which contain the pattern structures from nest to coarsest.

We can observe that the extraction quality of a mode depend on the quality of the previous modes. Therefore, the stop criteria (SD) of sifting process is important. 4.2. Image ltering Since sifting process extracts rstly the highest frequency, the rst modes correspond generally to the noise. The information of the noise is contained in the very rst modes and in the residual image. Conversely, the image tendency is contained only in the residual image.

Fig. 7. Photographic texture fabric.

Fig. 8. Photographic texture (D69 Brodatz).

1024

J.C. Nunes et al. / Image and Vision Computing 21 (2003) 10191026

Fig. 9. Brain MRI.

The tendency can be represented by a polynom of order relatively low 0; 1; 2: The results in Fig. 10 show the possibility of low level processing (ltering or denoising) with this technique. After having applied the decomposition, this ltering would be easily carried out by the subtraction with the original image of one or several modes. The residue represents the ltered image.

mode or more generally in residual image resulting to the sifting process (Fig. 11). After the decomposition, we subtract residue image from original image (Fig. 12c) to perform inhomogeneous illumination correction. 4.4. Perspectives In the bidimensional case, the regional extrema are not always well dened. The saddle points (or more generally, the pattern of ridges and valleys) should be taken into account. Thus, we propose the straightforward extension of the 1D EMD to the 2D case by using morphological operators. To detect lines peaks (respectively, the line

4.3. Extraction of inhomogeneous illumination Since sifting process extracts rstly the highest frequency, the rsts modes correspond generally to the noise. Conversely, the image tendency is contained in the latest

Fig. 10. Retinal image.

Fig. 11. Synthetic texture.

J.C. Nunes et al. / Image and Vision Computing 21 (2003) 10191026

1025

Fig. 12. Brain MRI.

connecting the lowest points) corresponding to the maxima (respectively, minima), we use the watershed, a powerful tool for image segmentation. More details of the BEMD from these lines (the saddle points) will be presented in future paper. We can observe in Fig. 12 the BEMD from these lines and can compare these results with BEMD from regional extrema in Fig. 9. For the BEMD from the regional extrema (Fig. 9), we used SDMAX 0:75: For the BEMD from the saddle points (Fig. 12), we used SDMAX 0:47: By comparing Figs. 9 and 12, we can observe that the details

corresponding to the modes are ner for the BEMD from the saddle points. Consequently, BEMD based on saddle points seem to produced more IMF than the preceding version.

5. Conclusion In this paper, we present a new modulation domain feature-based approach for discriminating textured images. We have applied the EMD to texture image analysis. The BEMD permits to extract spatial frequency components or

1026

J.C. Nunes et al. / Image and Vision Computing 21 (2003) 10191026 [16] A.K. Jain, F. Farrokhnia, Unsupervised texture segmentation using Gabor lters, Pattern Recognition 24 (12) (1991) 11671186. [17] B. Julesz, J.R. Bergen, Textons, the fundamental elements in preattentive vision and perception of textures, The Bell Syst. Technical J. 62 (6) (1983) 1619 1645. [18] A. Laine, J. Fan, An Adaptive Approach for Texture Segmentation by Multi-channel Wavelet Frames, Technical Report TR-93-025, Center for Computer Vision and Visualization, University of Florida, Gainesville, FL, August 1993. [19] S. Livens, G. Van de Wouwer, Wavelets for Texture Analysis: An Overview, In Proceedings of the Sixth International Conference on Image Processing and its Applications (IPA97), Dublin, Ireland, 1997, pp. 581 585. [20] X. Liu, D. Wang, Texture classication using spectral histograms, Electronic report 25 (OSU-CISRC-7/2000-TR17), citeseer.nj.nec. com/liu00texture.html [21] S. Mallat, Wavelets for a vision, Proc. IEEE 84 (4) (1996) 604 614. [22] A. Materka, M. Strzelecki, Texture Analysis Methods: A Review. Technical University of Lodz (1998), COST B11 Report. [23] Y. Meyer, Wavelets: Algorithms and Applications, SIAM, Philadelphia, 1993. chelle, S. Guyot, Ph. Bunel, Texture [24] J.C. Nunes, Y. Bouaoune, E. Dele analysis based on the bidimensional empirical mode decomposition, Mach. Vision Appl. (2003) in press. [25] P.J. Oonincx, Empirical mode decomposition: a new tool for S-wave detection, CWI Rep. Probab., Networks Algorithms (PNA) PNAR0203 (2002) ISSN 1386-3711. [26] A.P. Pentland, Fractal-based description of natural scenes, IEEE Trans. Pattern Anal. Mach. Intell. 6 (1984) 661674. [27] R.A. Peters II, Morphological pseudo bandpass image decompositions, J. Electron. Imaging 5 (2) (1996) 198213. [28] R.M. Pickett, Visual Analyses of Texture in the Detection and Recognition of Objects, Picture Processing and Psyeho Pietories, Academic Press, New York, 1970. [29] O. Pichler, A. Teuner, B.J. Hosticka, A comparison of texture feature extraction using adaptive Gabor ltering pyramidal and tree structured wavelet transforms, Pattern Recognition (29) (1996) 733 742. [30] T. Randen, J.H. Husoy, Filtering for texture classication: a comparative study, IEEE Trans. Pattern Anal. Mach. Intell. 21 (1999) 291 310. [31] T.R. Reed, H. Wechesler, Segmentation of textured images and Gestalt organization using spatial/spatial-frequency representations, IEEE Trans. Pattern Anal. Mach. Intell. 12 (1990) 112. [32] N. Sarkar, B.B. Chaudhuri, An efcient approach to estimate fractal dimension of textural images, Pattern Recognition 25 (9) (1992) 10351041. [33] M. Tuceryan, A.K. Jain, Texture analysis, in: C.H. Chen, L.F. Pau, P.S.P. Wang (Eds.), The Handbook of Pattern Recognition and Computer Vision, second ed., World Scientic, Singapore, 1998, pp. 207 248. [34] M. Unser, Texture classication and segmentation using wavelet frames, IEEE Trans. Image Process. 4 (11) (1995) 15491560. [35] L. Vincent, Morphological grayscale reconstruction in image analysis: applications and efcient algorithms, technical report 9116, Harvard Robotics Laboratory, November 1991, IEEE Trans. Image Process. 2 (2) (1993) 176 201. [36] M.V. Wickerhauser, Adapted Wavelet Analysis from Theory to Software, IEEE Press, New York, 1994.

different spatial scales, i.e. structures from nest to coarsest scales. This method, derived from the image data and fully unsupervised, permits to analyse non-linear and nonstationary data as texture images. We have shown experimental results for both natural and synthetic textures. By having access to these representations of scenes or objects, we can concentrate on only one or several modes (one individual or several spatial frequency components) rather than the image entirety. Clearly the 2D EMD offers a new and promising way to decompose and extract texture features without parameter.

References
[1] S. Beucher, Geodesic reconstruction, saddle zones and hierarchical segmentation, Image Anal. Stereol. 20 (2001) 137 141. [2] A.C. Bovik, M. Clark, W.S. Geisler, Multichannel texture analysis using localized spatial lters, IEEE Trans. Pattern Anal. Mach. Intell. 12 (1) (1990) 5573. [3] P. Brodatz, Textures: A Photographic Album for Artists and Designers, Dover publications, New York, 1966. [4] J.C. Carr, W.R. Fright, R.K. Beatson, Surface interpolation with radial basis functions for medical imaging, Comput. Graph. Proc., Annu. Conf. Ser. (SIGGRAPH 2001) (2001) 6776. [5] E. Cesmeli, D.L. Wang, Texture segmentation using Gaussian Markov random elds and LEGION, Proc. IEEE Int. Conf. Neural Networks (ICNN97) III (1997) 15291534. [6] C.C. Chen, J.S. Daponte, M.D. Fox, Fractal feature analysis and classication in medical imaging, IEEE Trans. Med. Imaging 8 (1989) 133142. [7] J. Daugman, Complete discrete 2D Gabor transform by neural networks for image analysis and compression, IEEE Trans. Acoust., Speech, Signal Process. 36 (1988) 11691179. [8] D. Dunn, W.E. Higgins, J. Wakeley, Texture segmentation using 2-D Gabor elementary functions, IEEE Trans. Pattern Anal. Mach. Intell. 16 (2) (1994) 130149. [9] K.B. Eom, Segmentation of monochrome and color textures using moving average modeling approach, Image Vision Comput. 17 (3) (1999) 231242. [10] P. Flandrin, G. Rilling, P. Gonc alves, Empirical mode decomposition as a lter bank, IEEE Signal Processing Letters (2003), accepted for publication. [11] G. Gimelfarb, Image Textures and Gibbs Random Fields, Kluwer Academic Publishers, Dordrecht, 1999, 250 p. [12] R. Haralick, Statistical and structural approaches to texture, IEEE Proc. 67 (5) (1979) 786 804. [13] J.P. Havlicek, D.S. Harding, A.C. Bovik, Multidimensional quasieigenfunction approximations and multicomponent AMFM models, IEEE Trans. Image Process. 9 (2) (2000) 227 242. bal, High resolution spectral analysis of images [14] J. Hormigo, G. Cristo using the pseudo-Wigner distribution, IEEE Trans. Signal Process. 46 (6) (1998) 17571763. [15] N. Huang, et al., The empirical mode decomposition and the Hilbert spectrum for non-linear and non-stationary time series analysis, Proc. R. Soc., Lond. A 454 (1998) 903 995.