ELEC533 Final Project

A Modest Overview of Image Denoising

Wan Pengfei 10800496
AbstractIn this project, I implement and evaluate a variety of most commonly used image denoising algorithms and propose noticeable experimental results by comprehensive comparisons among different denoising approaches. These algorithms include the canonical Isotropic Filtering [5], Anisotropic Filtering [6], Non-local Filtering [7], SUSAN Filtering [8][9], Total Variation Minimization [10], as well as the state-of-the-art BM3D [3][4] image denoising algorithm. Index Terms Image denoising.

Particularly, as for a gray-scale image with bit-depth of 8, PSNR is defined as follows: ( )

where MSE denotes the mean square error, i.e. ( ) ( )

I. INTRODUCTION

denoising is a classical and flourishing research topic in image processing. The demand for image denoisng approaches comes from the fact that during the process that a digital image is captured, quantized, recorded and transmitted, it will inevitably be contaminate by a variety of noises, which will result in annoying artifacts and decrease the visual quality. To date, a great deal of image denoising algorithms have been proposed [3][5][6][7][8][10], originating from various subjects ranging from probability and statistics to differential equations, from spatial-domain filtering to transform-domain spectrum manipulation. Among which many have achieved remarkable performance and been widely used in our social life. Generally speaking, image denoising is an important topic of image restoration, which aims at reconstructing an image with a certain degree of degradation. A generally degradation model of an image can be described as follows: ( ) ( ) ( ) Where x represents the two-dimensional location of a pixel, u denotes the original image we want to reconstruct and v is the noisy observation of the image. n denotes the noise, which can be arbitrary digital noise that could occur, such as pulsing noise, salt-andpepper noise as well as Gaussian valued noise. Usually, when we talk about image denoising, we refer the noise as additive white Gaussian noise (AWGN). The goal of image denoising is to find out , which is the estimate of original image u from its noisy observation v. Several criterions exist to evaluate the denoising performance of different denoisng algorithms. Among which the PSNR is probably the most widely used one. A higher PSNR indicates a better performance of the algorithm. PSNR is short for Peak Signal to Noise Ratio. Generally in signal processing, signal-to-noise-ratio is used to represent the proportion of power between the signal and the noise.
MAGE

The rest of the report is organized as follows. I will briefly introduce the six representational image denoising algorithms in Section II. Experimental results can be found in Section III, and finally I will conclude my work in Section IV. II. EXISTING DENOISING ALGORITHMS A. A brief overview Plenty of denoisng algorithms are proposed. Among these algorithms, some perform spatial convolution locally, such as the Gaussian smooth model (Gabor [5]); some apply spatial convolution in a global range, such as non-local means [7]; some achieve denoising by anisotropic filtering based on the texture and edge information of the image (Perona-Malik [6]). Some exploit the prior information of the image in both Euclidean space as well as the Intensity space (Smith, Brady [8]). Also there are some algorithms look into the denoisng problem in frequency domain, such as BM3D (Dabov [4]), as well as some approaches using wavelet thresholding. Another class are the optimizationbased algorithms, which typically achieve denoising by calculus of variations, such as Total Variation Minimization (Rudin-Osher-Fatemi [10]). Recently, in ICIP2010, a game theory based denoisng algorithm is proposed, which shows the abundant research enthusiasm in image denoisng. B. BM3D BM3D is the abbreviation of Block Matching and 3D filtering [3][4], which is a brilliant work of Dabov, et al from Tampere University of Technology, Finland. It is commonly regarded as the state-of-the-art real-time denoisng algorithm. The main idea is based on an enhanced sparse representation of image blocks in transform-domain. The enhancement of the sparsity is achieved by grouping similar 2D image fragments (e.g. blocks) into 3D data arrays. Then collaborative filtering is applied to deal with these 3D groups. The collaborative filtering is achieved by three successive steps: 3D transformation of 3D group, shrinkage of transform

Wan Pengfei, PhD student of ECE Department, HKUST

ELEC533 Final Project

Figure 2.1.1 BM3D flow chart spectrum, and inverse 3D transformation. The result is a 3D estimate that consists of the jointly filtered grouped image blocks. By attenuating the noise, the collaborative filtering reveals the details shared by grouped blocks. And the filtered blocks are then returned to their original positions. Because the blocks are overlapping, for each pixel we can obtain many different estimates which need to be combined. Aggregation scheme is then exploited to take advantage of this redundancy. The generally flow chart of BM3D is described in Figure 2.1.1 Specifically, in Step1 we first apply a basic estimation to find blocks that are similar with the current block and form a 3D group together. Then we apply a separable 3D transform on the 3D group. The separable 3D transform is composed by a basic 2D transform and a 1D Haar transform on the third dimension. Finally, a primary estimate of the original image is obtained by aggregation. Aggregation is performed by a weighted averaging at those pixels positions where there are overlapping block-wise estimates. In Step2, similarly grouping and 3D collaborative filtering are applied to both the primary estimate and the noisy image. The different in this step is the 3D filtering in Step2 makes use of the energy spectrum of the primary estimate obtained in Step1. So we can apply DCT and Haar-wavelet Wiener filtering to get the final estimate of u. The final estimate output is obtained by aggregation: ( ) Where is the estimated estimate of each image block in different block groups via collaborative filter. C. Isotropic filtering By Riesz theorem, isotropic filtering of image is equivalent to a convolution of the image by a linear symmetric kernel. The most famous isotropic filter is undoubtedly the Gaussian kernel: ( ) where h is the standard deviation and the estimated image is then represented as the convolution of the noisy observation and the Gaussian kernel: In practice the filtering range is the rectangle neighborhood around the current pixel. Though Gaussian isotropic filtering is effective to remove slightly contaminated images and simple to implement, usually the texture and edge will also be ( ) blurred which limits the denoising performance of this class of algorithms. D. Anisotropic filtering In order to alleviate the blurring effect introduced by the isotropic filtering, the anisotropic filtering attempts to exploit the edge information and only apply convolution on noisy image v at pixel locations that along the direction orthogonal to the gradient direction ( ).i.e. ( ) ( ) ( ) ( ) ( ) This algorithm is first proposed by Perona and Malik [] and can reduce image noise without removing significant parts of the image content, such as edges, lines or other details that are important for the interpretation of the image. The implementation of anisotropic filter is usually done by iteration scheme. Formally speaking, let I denote the input image, the partial differential equation can be described as follows: ( )

Where denotes the Laplacian operator and denotes the gradient operator. c(x, t) is the evolution coefficient that controls the rate of iteratively diffusion. c is usually chosen as a function of image gradient so as to preserve the textures and edges. In Perona and Maliks paper, two functions are proposed for c: ( ( ) )
( )

( ) Where factor K controls the sensitivity to edges. The first function privileges high-contrast edges over low-contrast edges. And the second privileges wide regions over smaller ones. E. Non-local filtering The isotropic and anisotropic filtering share the same basic idea: denoising is achieved by local linear convolution. However, for some images with evident textures, the local algorithms may not work well. So the non-local algorithms try to make use of the image information of a wider range to achieve better denoising performance. The representational algorithm is proposed by Buades [7], in this paper the basic idea is to find out and make use of all the pixels whose neighborhood is similar with that of the current pixel, i.e.

ELEC533 Final Project {

( )

( )

()
( ) [ ( ) ( )]

( ) }

( )

( )

There are several differences between non-local filtering and local filtering algorithms. The first one is that the range of neighboring pixel used for denoisng is larger, theoretically all the pixels within the image can be used to denoisng the current pixel in Non-local filtering. The second difference is that the similarity measurement, which decides the averaging weight, is not just depend on the intensity value of one pixel, but the similarity between rectangle neighborhoods. However, shortcomings of non-local filtering algorithm are also obvious: a larger filtering range usually leads to oversmoothing artifacts that textures and edges are unnecessarily blurred. Also, the non-local algorithm suffers from relatively higher computational complexity. F. SUSAN filtering SUSAN filtering [8] is a kind of neighborhood filtering which shares the same idea as Bilateral filtering [9]. Instead of consider only the spatial neighborhood which is close to the current pixel spatially, SUSAN filter weights pixels both by the Euclidean distance and the Intensity-space distance, i.e. ( ) () ( ) ( ) {
[ ( ) ( )]

The above function is strictly convex, so that the minimum exits and is computable. In this algorithm, the noise v(x) - u(x) is treated as an error. In practice, some textures are usually presented in the error. One possible solution to this problem is an iterative algorithm proposed by Burger [11] as follows: 1. First solve the original total variation model: 2. 3. ( ) ( ) ( ) ( ) ( ) ( )

Perform a correction step:

Iteration to obtain the final output:

The solution converges after plenty of iterations and serves as the output of Total Variation Minimization algorithm.

III. EXPERIMENTAL EVALUATION In my experiments, six image denoising algorithms abovementioned are evaluated under a wide range of AWGN ( ). And 8 test images are used to make the results more reasonable and convincing. The denoising performance is evaluated under both the subjective criterion of visual quality and the objective criterion, which is the absolute PSNR of the estimated image as well as PSNR increment compared with the noisy image v. A. Parameters Selection In my implementation, I make use of training images to select rational and adaptive model parameters to achieve best-so-far performance for each algorithm.

By exploiting the inherent intensity structure of the image, The SUSAN filter can preserves sharp edges and textures when removing the noise, which is also an important property of anisotropic filter. Another advantage of SUSAN filter (Bilateral filter) is that unlike the anisotropic filtering, it is a non-iterative edgepreserving denoising algorithm, which makes it widely used, even adopted by the well-known Adobe Photoshop. G. Total variation minimization This algorithm regards the image denoising problem as a minimization problem. It was first introduced by Rudin, Osher and Fatemi [10]. Given a noisy image observation v, the Total Variation Minimization algorithm try to estimate the original image u via the solution of the following expression: ( ) ( ) under the constrains ( ( ) ( ) ( )) ( ) ( ) ( ) ( )

Figure 3.1.1 Adaptive

for Gaussian isotropic filter

By introducing a Lagrange multiplier , this minimization problem becomes:

Figure 3.1.2 Adaptive

for iterative anisotropic filter

ELEC533 Final Project

Denoisng Peformance Comparison on me.jpg 16 14 12 10 8 6 4 2 0
BM3D Isotropic Filter Anisotropic Filter Non-local Filter SUSAN Filter TV Minimization

Figure 3.1.3 Adaptive

of intensity for SUSAN filter

PSNR increment compared with noisy image

10

15

20 25 Variance of AWGN

30

35

40

Figure 3.2.2 Performance comparison on a natural image We can see from Figure 3.2.1-2: For natural images (such as one of my photos) BM3D can achieve incomparable denosing performance. As for other algorithms, Total Variation Minimization, SUSAN filter, Non-local filter and Isotropic filter can achieve satisfactory results that the PSNR increments converge to approximately 8-9 dB. For artificial images with strong textures and edges (such as montage), besides BM3D, the edge-preserving algorithms, SUSAN filtering and Anisotropic filtering, can achieve appealing results which is comparable with theTVM algorithm. However, understandably, the Isotropic filter shows a poor performance. In both occasions, a sharp performance drop is observed for the Anisotropic filter, which I will explain in the next section. C. Overall performance evaluation
Average Denoisng Peformance Comparison 14 Denoisng Peformance Comparison on montage.png 14

Figure 3.1.4 Adaptive for iterative solution of TVM B. Performance comparison w.r.t same test images I test the denoising performance of each algorithm on eight images measured by PSNR increment of the restored image compared with the noisy image . Below are several numerical comparisons of the six denoising algorithms on different test images. where, The x coordinator is the standard variance of AWGN, ranging from 5 to 40. The y coordinator is the PSNR increment of the denoised image comparing with the noisy image. Different algorithms are marked in different colors.

12

PSNR increment compared with noisy image

12

Average PSNR increment

10

10

BM3D Isotropic Filter Anisotropic Filter Non-local Filter SUSAN Filter TV Minimization

BM3D Isotropic Filter Anisotropic Filter Non-local Filter SUSAN Filter TV Minimization

0 5

10

15

20 25 Variance of AWGN

30

35

40

0 5

10

15

20 25 Variance of AWGN

30

35

40

Figure 3.3.1 Average PSNR increment comparison The figure above shows the average denoising performance of each algorithm on the whole test image database.

Figure 3.2.1 Performance comparison on image with textures

ELEC533 Final Project We can conclude that statistically: 1. BM3D achieves apparently the best objective denoising performance. 2. TVM, SUSAN filter and Anisotropic filter (AF) achieve the betterthanaverage denoising performance. 3. The performance of Isotropic filter (IF) and Non-local filter (NL) is relatively poorer. The overall image denoising performance can be stated as follows: which is substantially consistent with our prior expectation on these denoising algorithms. The reason for the poor performance of Non-local algorithm is that the good result of Non-local filtering requires highly regularity of image textures, which is generally not true for natural images. D. Performance comparison w.r.t different noise variance In this section, lets consider how denoising performance evolves with different degree of noise for each algorithm. The following conclusions can be drawn based on Figure 3.3.1.: When the noise variance is relatively small, the denoising performance follows: Notice that the SUSAN filter performs much poorer when the variance is small, that is because the inner scheme of SUSAN filter is similar with Gaussian isotropic filter. Usually the blurring artifact will become more prominent when the noise variance is small. When the noise variance is relatively large, the denoising performance follows: Notice that a sharp drop is observed for Anisotropic filter when the noise variance becomes large. This is probably because of the fact that the anisotropic filtering is essentially a diffusion process, when the signal-noise-ratio is large the severe error propagation will usually lead to an unstable solution. E. Local vs Non-local Denoising performances of local filtering (Isotropic filter) and the Non-local filter are compared in this section.
Local vs Non-local claire.jpg 8 7 6 5 4 3 2 1 0
PSNR increment compared with noisy image

Local vs Non-local montage.png 8 7 6 5 4 3 2 1 0

PSNR increment compared with noisy image

Isotropic Filter Non-local Filter

10

15

20 25 Variance of AWGN

30

35

40

Figure 3.5.2 Performance comparison on artificial image We can see that for natural images, the Non-local filter have slightly better results for small noise variance and slightly poorer results for large noise variance compared with Isotropic filter, so basically they are similar in denoising performance. However, for artificial images (such as montage), because of the fact that much more textures exist, the Non-local filter can make use of plenty of similar image patches to denoise the current pixel. So the Non-local filtering can achieve a huge performance gap over the local Gaussian filter, which suffers from the unnecessary removal of important high frequency contents. In addition, the computational complexity of Non-local filter is much larger than that of Gaussian isotropic filtering, which diminishes the advantage of non-local denoising algorithms to some extent. . F. Isotropic vs Anisotropic Also, we test the denoising performance of Isotropic filtering and the Anisotropic filtering. The motivation is to find out under what circumstances the Anisotropic filter will have a better performance than classical Isotropic filter as expected.
Isotropic vs Anisotropic me.jpg 12

PSNR increment compared with noisy image

10

2
Isotropic Filter Anisotropic Filter

Isotropic Filter Non-local Filter

0 40

10

15

20 25 Variance of AWGN

30

35

10

15

20 25 Variance of AWGN

30

35

40

Figure 3.5.1 Performance comparison on natural image

Figure 3.6.1 Performance comparison on natural image

ELEC533 Final Project

Isotropic vs Anisotropic montage.png 9 8 7 6 5 4 3 2 1 0

Isotropic Filter Anisotropic Filter

Anisotropic vs SUSAN fingerprint.png 4

PSNR increment compared with noisy image

PSNR increment compared with noisy image

3.5

2.5

1.5

1
Anisotropic Filter SUSAN Filter

10

15

20 25 Variance of AWGN

30

35

40

0.5 5

10

15

20 25 Variance of AWGN

30

35

40

Figure 3.6.2 Performance comparison on artificial image From the two figures above we can observe that: For natural images, usually there arent a lot of textures and edges, Anisotropic can only achieve a moderate performance improvement over Isotropic filter. Whats worse, Anisotropic filter may even perform poor when the noise variance is very large, the reason is that the implementation of Anisotropic filter is a iterative algorithm, the error propagation may be catastrophic in such conditions. But for artificial images with stronger textures, such as montage, the Anisotropic filter can achieve a larger gap over Isotropic filter on PSNR increment than denoising natural images. That is because the Anisotropic filter apply filtering only on directions orthogonal to the edges, whereas the Isotropic filter apply filtering the image without taking any use of the inherent structure information of the image. G. Anisotropic vs SUSAN SUSAN filter and Anisotropic filter are both well-known edge-preserving denoising algorithms. In this experiment, I test and compare the numerical performances of these two algorithms
Anisotropic vs SUSAN montage.png 9 8 7 6 5 4 3 2 1

Figure 3.6.2 Performance on image with strong texture We can see from Figure 3.6.1-2 that both algorithms can achieve appealing results on textured images. Particularly, for images with moderate textures and edges, the two algorithms can effectively remove the noise without sacrificing important image features. As for images with strong structures, the SUSAN filter can achieve a slightly better PSNR increment than the Anisotropic filter. Which is consistent with the experimental conclusions in [8]. IV. CONCLUSIONS & DISCUSSIONS In this work we have implemented and evaluated six classes of most popular image denoisng algorithms. The theoretical analysis and experimental evaluations show valuable results on those denoising algorithms: 1. BM3D algorithm can achieve the best denoising performance under most occasions in real time. 2. Gaussian kernel isotropic filtering can achieve satisfactory results when the noise variance is large, but the visual quality is poor in most cases because of the removal of the textures and contours. 3. Iterative anisotropic filtering can effectively preserve the edges and textures with higher curvature, but the denoising performance is greatly constrained by the error propagation when noise variance becomes large. . 4. Non-local filtering is effective for denoising images with regular textures. However for general occasions over-smoothing are usually observed. And the inner block-matching scheme is relatively time-consuming. 5. SUSAN filtering (or Bilateral filtering) is another edgepreserving denoising algorithm, which can effectively preserve textures and edges with higher contrast, in most cases it can achieve stable better-than-average denoising performances. 6. Total Variation Minimization algorithm achieves denoising by minimizing the image variance under the constraints of noise statistics. With an iterative solution the computational complexity is greatly reduced and the subjective and objective denoising performances are better than all the others except for BM3D.

PSNR increment compared with noisy image

Anisotropic Filter SUSAN Filter

10

15

20 25 Variance of AWGN

30

35

40

Figure 3.7.1 Performance on image with moderate texture

ELEC533 Final Project In a nutshell, although image denoising is well-known an ill-pose problem, current algorithms can achieve satisfactory performances in most cases. Put another way, there is still much work to be done on this topic, future research directions as far as I can see could be the hybrid optimization-based denoising algorithms.
K. Dabov, A. Foi, Image denoising by sparse 3D transform-domain collaborative filtering. IEEE Trans. Image Processing 2007 [5] M. Lindenbaum, M. Fischer, On Gabor contribution to image enhancement. Pattern Recognition, 1994. [6] P. Perona, J. Malik, Scale space and edge detection using anisotropic diffusion. IEEE Trans. PAMI 1990. [7] A. Buades, B. Coll, A non-local algorithm for image denoising. Proc. CVPR 2005. [8] S. Smith, J. Brady, SUSAN-a new approach to low level image processing, IJCV 1997 [9] C. Tomasi, R. Manduchi, Bilateral filtering for gray and color images, Proc. ICCV 1998 [10] L. Rudin, S. Osher, Total variation based image restoration with free local constrains, Proc. ICIP. 1994. [11] S. Osher, M. Burger, Using Geometry and iterated refinement for inverse problems: Total variation based image restoration, JSIAM.2004 [4]

REFERENCES
[1] [2] [3] http://en.wikipedia.org/wiki/Main_Page/ http://www.cs.tut.fi/~foi/GCF-BM3D/ V. Katkovnik, A. Foi, K. Egiazarian, and J. Astola, From local kernel to nonlocal multiple-model image denoising, IJCV 2010.

APPENDIXES

Below are some subjective denoising performance comparisons From left to right: original image, noisy image, denoised image From top to bottom: BM3D, IF, AF, NL, SUSAN, TVM A. Noise standard variance

ELEC533 Final Project

ELEC533 Final Project

B. Noise standard variance

ELEC533 Final Project

ELEC533 Final Project C. Noise standard variance

ELEC533 Final Project