Nonlocal Total Variation Based Speckle

Denoising Model

Arundhati Bagchi Misra and Hyeona Lim

Abstract A large range of methods covering various fields of mathematics are

available for denoising an image. The initial denoising models are derived from
energy minimization using nonlinear partial differential equations (PDEs). The
filtering models based on smoothing operators have also been used for denoising.
Among them the recently developed nonlocal means method proposed by Buades,
Coll and Morel in 2005 is quite successful. Though the method is very accurate, it
is very slow and hence quite impractical. In 2008, Gilboa and Osher extended
some known PDE and variational techniques in image processing to the nonlocal
framework and proposed the nonlocal total variation method for Gaussian noise.
We used this idea to develop a nonlocal model for speckle noise. Here we have
extended the speckle model introduced by Krissian et al. in 2005 to the nonlocal
framework. The Split Bregman scheme is used solve this new model.

Keywords Image denoising Speckle denoising models
Nonlocal means



Nonlocal PDE Nonlocal TV

A. B. Misra (&)
Department of Mathematical Sciences, Saginaw Valley State University, University Center,
MI 48710, USA
e-mail: abmisra@svsu.edu
H. Lim
Department of Mathematics and Statistics, Mississippi State University, Mississippi,
MS 39762, USA
e-mail: hlim@math.msstate.edu

Mohan S. and S. S. Kumar (eds.), Proceedings of the Fourth International Conference 517
on Signal and Image Processing 2012 (ICSIP 2012), Lecture Notes in Electrical
Engineering 221, DOI: 10.1007/978-81-322-0997-3_46, Ó Springer India 2013
518 A. B. Misra and H. Lim

1 Introduction

Image restoration is a very important process and is often necessary as a pre-

processing for other imaging techniques such as segmentation and compression. In
general, an observed image f is represented by the equation

f ¼ u þ n; ð1Þ

where u is the original noise free image, f is the observed noisy image and n is the
Gaussian noise. Here u; f : X  I R2 ! I R. For any denoising model, the main
objective is to reconstruct u from an observed image f .
In 1992 Rudin et al. [8] proposed the total variation (TV) denoising model as
the minimization problem:
Z
min jrujdx ð2Þ
u
X

subject to the constraints,

Z Z
fd x ¼ u dx ð3Þ
X X

Z
1
and ð f ÿ uÞ2 dx ¼ r2 ; ð4Þ
2
X

where r is the standard deviation of the noise n. These constraints ensure that the
resulting image and the observed images are quite close to each other.
Combining the above constraints the TV functional is obtained by
Z Z
k
FðuÞ ¼ jrujdx þ ð f ÿ uÞ2 dx: ð5Þ
2
X X

Here k is a constraint parameter. The equivalent Euler–Lagrange equation gives

the TV denoising model as
 
ou ru
ÿr
¼ kðf ÿ uÞ: ð6Þ
ot jruj

To avoid singularities, it was regularized by using jruj  jre uj ¼ ðu2x þ u2y þ e2 Þ1=2 .
Nonlocal Total Variation Based Speckle Denoising Model 519

2 Preliminaries

Here we present the initial speckle denoising model and the filtering based non-
local means method.

2.1 Speckle Denoising Model

In 2005 Krissian et al. [6] considered the speckle noise model by

pffiffiffi
f ¼ u þ un; ð7Þ

where u is the desired image to find, n is Gaussian noise and f denotes the observed
f ÿu
image. Thus, we have n ¼ pffiffiffi . Then replacing f ÿ u in (5) with the new expression
u
for n, the minimization functional for speckle denoising is given by
Z "  #
k f ÿu 2

FðuÞ ¼ jruj þ pffiffiffi dx: ð8Þ
2 u
X

From energy minimization of this functional, the TV-based speckle denoising

model can be derived as:

u2
 
ou e ru
ÿ jr uj r
e
¼ k jre uj ðf ÿ uÞ: ð9Þ
ot f þ u jr uj

2.2 Nonlocal Means Method

In 2005 Buades et al. proposed the new state of art image denoising algorithm
known as nonlocal means algorithm [2]. The algorithm is given by the formula
Z
1 ðGa Ijuðxþ
Þÿuðyþ
Þj2 Þð0Þ
NL½uŠðxÞ ¼ eÿ h2 uðyÞdy; ð10Þ
CðxÞ
X
Z
ðGa Ijuðxþ
Þÿuðzþ
Þj2 Þð0Þ
where CðxÞ ¼ eÿ h2 dz. Here Ga is the Gaussian kernel with stan-
X
dard deviation a, h is a filtering parameter and uðx þ
Þ denotes the neighborhood
of the pixel x. Here each pixel value is denoised using the weighted average of all
the pixels in the image. Thus for a given discrete noisy image u ¼ fuðiÞ : i 2 Ig,
the estimated value NL½uŠðiÞ, for a pixel i, is computed as
520 A. B. Misra and H. Lim

X
NL½uŠðiÞ ¼ wði; jÞuðjÞ; ð11Þ
j2I

where the weight wði; jÞ depends onXthe similarity of the i and j pixels and satisfies
the conditions 0  wði; jÞ  1 and j
wði; jÞ ¼ 1. In this paper the authors mea-
sured the similarity of a square neighborhood Ni of a fixed size at each pixel i as a
decreasing function of the weighted Euclidean distance, jjuðNi Þ ÿ uðNj Þjj22;a ,
where j 2 I and a [ 0 is the standard deviation of the Gaussian kernel. The weight
function is defined as
2
1 ÿjjuðNi ÞÿuðN j Þjj2;a
wði; jÞ ¼ e h2 ; ð12Þ
CðiÞ
where CðiÞ is the normalizing constant given by
jjuðNi ÞÿuðNj Þjj2
2;a
eÿ
X
CðiÞ ¼ h2 : ð13Þ
j

The similar neighborhoods have a very small Euclidean distance which in turn
results in larger weight. Since the nonlocal means uses the similarity neighborhood
for denoising, it works really well for periodic or patterned textured cases. But
though the method is very good in removing noise, it is very slow and hence quite
impractical.

2.3 Nonlocal TV Model

In 2008, Gilboa and Osher extended some known PDE and variational techniques
in image processing to the nonlocal framework [3]. The motivation behind this
was to make any point interact with any other point in the image. Since the
classical derivatives were local operators, they had to redefine the required
operators in [3] following the ideas of Zhou and Scholkopf [10, 11]. Gilboa and
Osher
R proposed the nonlocal TV (NLTV) model based on the NLTV operator
( X jrNL uj), as follows:

ou
¼ jNL ðuÞ ÿ kðu ÿ f Þ
ot Z
ð14Þ
¼ wðx; yÞðuðyÞ ÿ uðxÞÞðjrNL ujÿ1 ðxÞ þ jrNL ujÿ1 ðyÞÞdy ÿ kðu ÿ f Þ;
X

where rNL ð
Þ ¼ nonlocal gradient, divNL ð
Þ ¼ nonlocal divergence and jNL ð
Þ
rNL
¼ divNL jr NL j
denotes the nonlocal curvature. Since the steepest descent scheme
(14) is very slow, a faster numerical scheme was introduced by Bresson in [1]. The
Nonlocal Total Variation Based Speckle Denoising Model 521

scheme is based on the Split Bregman method introduced in [4]. It was proved to
be very fast for regular TV.

3 Nonlocal TV Based Speckle Denoising Model

We now present our new model for speckle denoising. We extend the idea pre-
sented in Sect. 2.3 and develop the following nonlocal TV based speckle denoising
model:
Z "  #
f ÿu 2

min FðuÞ; FðuÞ ¼ jrNL uj þ k pffiffiffi dx ð15Þ
u u
X

where rNL is a nonlocal gradient defined in [3]. For faster computation, we adopt
Split Bregman scheme [4] for finding a solution to our new model (15). We first
develop in Sect. 3.1 the Split Bregman scheme for Krissian et al. [6].

3.1 Split Bregman Scheme for Krissian et al. Model

As discussed in Sect. 2.1, the minimization functional was given by (8). For Split
Bregman scheme, we introduce d ¼ ru and construct the unconstrained functional
as

ðf ÿ uÞ2
Z
b
min FðuÞ; FðuÞ ¼ jdj þ k dx þ kd ÿ ru ÿ bk22 : ð16Þ
u u 2
X

Here b is a penalty parameter. We can now split this in two subproblems of u and d
given as:

ðf ÿ uÞ2
Z
b
min k dx þ kd ÿ ru ÿ bk22 ; ð17Þ
u u 2
X

Z
b
min jdjdx þ kd ÿ ru ÿ bk22 : ð18Þ
d 2
X

Now,
522 A. B. Misra and H. Lim

! " #
o ðf ÿ uÞ2 ÿ2uðf ÿ uÞ ÿ ðf ÿ uÞ2
k ¼k
ou u u2
 
ðu ÿ f Þð2u ÿ u þ f Þ ð19Þ
¼k
u2
k
¼ 2 ðu ÿ f Þðu þ f Þ:
u
Therefore the optimality condition for u in (17) gives us
kðu þ f Þ
ðu ÿ f Þ þ b  ðd ÿ ru ÿ bÞ ¼ 0
u2
ð20Þ
u2 u2
 
) kÿb D u ¼ kf ÿ b  ðd ÿ bÞ:
uþf uþf

The optimality condition can be discretized using the definition of discrete gra-
dient, divergence and Laplacian as discussed in [3]. This gives
!
u2ij u2ij 
k þ 4b uij ¼ kfij þ b uiþ1;j þ uiÿ1;j þ ui;jþ1 þ ui;jÿ1
uij þ fij uij þ fij
ð21Þ
þ dx;iÿ1;j ÿ dx;ij þ dy;i;jÿ1 ÿ dy;ij

ÿ bx;iÿ1;j þ bx;ij ÿ by;i;jÿ1 þ by;ij :

Thus, we get
2
1 h ukij 
ukþ1
ij ¼  kfij þ b ukiþ1;j þ ukiÿ1;j þ uki;jþ1 þ uki;jÿ1
ukij
2
ukij þ fij
k þ 4b k
uij þfij
i
k k k k
þ dx;iÿ1;j ÿ dx;ij þ dy;i;jÿ1 ÿ dy;ij ÿ bkx;iÿ1;j þ bkx;ij ÿ bky;i;jÿ1 þ bky;ij :
ð22Þ

Since there is no coupling between elements of d, the optimal value of d is

computed using the shrinkage operator described in [7, 9]:

rukþ1
ij þ bkij
dijkþ1 ¼ maxfjrukþ1
ij þ bkij j ÿ 1=b; 0g: ð23Þ
jrukþ1
ij þ bkij j

The variable b is initialized to zero and is updated after each Bregman iteration as:
Nonlocal Total Variation Based Speckle Denoising Model 523

bkþ1
ij ¼ bkij þ rukþ1
ij ÿ dijkþ1 : ð24Þ

The scheme (22)–(24) provides a much faster solution for the TV based speckle
denoising model introduced by Krissian et al. [6].

3.2 Split Bregman scheme for nonlocal TV speckle model

For our new model (15) the Split Bregman functional will be of the form

ðf ÿ uÞ2 b
Z
min FðuÞ; FðuÞ ¼ jdj þ k þ kd ÿ rNL u ÿ bk22 : ð25Þ
u u 2
X

Then the optimality condition for u gives us the equation

u2 u2
 
kÿb DNL u ¼ kf ÿ b divNL ðd ÿ bÞ: ð26Þ
uþf uþf

Denoting the discretized points x; y 2 X  X by i and j, and using the discrete

definition for divNL and Dbadhbox [1, 3], the discrete minimization scheme is given
as:
!
u2i X
kui ÿ b wij ðuj ÿ ui Þ
ui þ f i j
! ð27Þ
u2i X pffiffiffiffiffiffi
¼ kfi ÿ b wij ðdij ÿ dji ÿ bij þ bji Þ :
ui þ fi j

Hence, the iteration steps are:

"
1 u2i X
ukþ1
i ¼ u2i
b wij ukj þ kfi
kþb
P
wij ui þ fi j
ui þfi j
!# ð28Þ
u2i X pffiffiffiffiffiffi
ÿb wij ðdij ÿ dji ÿ bij þ bji Þ ;
ui þ f i j

Table 1 Summary of numerical results

Krissian et al. Nonlocal means NLTV
Images Time (s) PSNR Time (s) PSNR Time (s) PSNR
Lenna (PSNR ¼ 24:51) 0:72 26:39 11:46 28:21 4:67 29:10
Gallstones 0:14 ÿ 11:92 ÿ 4:62 ÿ
524 A. B. Misra and H. Lim

pffiffiffiffiffiffi kþ1
0 1
kþ1 k
wij ðuj ÿ ui Þ þ bij
sffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
ffi
X 2 1
dijkþ1 ¼ qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi max@ wij ðukþ1 j ÿ ukþ1 i Þ2 þ bkij ÿ ; 0A;
P kþ1
ÿ ukþ1 Þ2 þ bkij
2 b
j wij ðuj
j
i

ð29Þ

pffiffiffiffiffiffi kþ1
bkþ1
ij ¼ bkij þ wij ðuj ÿ ukþ1
i Þ ÿ dijkþ1 : ð30Þ

(28–30) provides a fast and accurate nonlocal scheme for TV based speckle
denoising model.

4 Numerical Results

The numerical results are displayed in the next pages. We have considered the
images of size 128  128 and the summary of results are in Table 1. Other than the
visual results, we have used peak-signal-to-noise ratio (PSNR) for synthetic
images, which was mentioned in [5], to measure the efficiency of the models.A
cleaner image provides a higher PSNR value. We  have  provided the absolute
fpÿu
residual ðjf ÿ ujÞ and the speckle noise residual n ¼ ffiffi for all images. It is u
evident that the new nonlocal speckle model works best among the models
compared here. For this model only 1–2 iterations were enough for all images. The
Krissian et al. is not denoising very well compared to the new one. The residual
shows presence of edges and fine textures due to the blurring effect of TV. But the
Krissian et al. is certainly the fastest one. The nonlocal means is not working well
for fine textures and slower than the new model. It is also the slowest of all.
Nonlocal speckle model maintains texture and removes noise very well. The noise
residuals show that nonlocal speckle model has picked up more noise compared to
the other ones.
Nonlocal Total Variation Based Speckle Denoising Model 525

Synthetic image result (Lenna)

526 A. B. Misra and H. Lim

Ultrasound image result (Gallstones)

5 Conclusion

In this paper we have incorporated the positive aspects of both PDE and filtering
models. The PDE based TV model is very fast but has a blurring effect. Nonlocal
Nonlocal Total Variation Based Speckle Denoising Model 527

means has very effective denoising properties but it is very slow. Therefore, have
developed an accurate model using both of them. We have extended the existing
Krissian et al. speckle denoising model [6] to a nonlocal framework, and provided
an efficient nonlocal TV based speckle denoising model. We have also developed a
numerical scheme for solving this new model. The numerical scheme is based on
the Split Bregman method introduced by Gilboa and Osher. The results show the
new model is more accurate than both the Krissian et al and nonlocal means
model. It is also much faster than nonlocal means.

References

1. Bresson X (2009) A short note for nonlocal tv minimization. Technical report

2. Buades A, Coll B, Morel JM (2005) A non-local algorithm for image denoising. In: IEEE
computer society conference on computer vision and pattern recognition, CVPR 2005, vol 2,
pp 60–65. doi:10.1109/CVPR.2005.38
3. Gilboa G, Osher S (2008) Nonlocal operators with applications to image processing.
Multiscale Model Simul 7(3):1005–1028. http://dblp.uni-trier.de/db/journals/mmas/
mmas7.html#GilboaO08
4. Goldstein T, Osher S (2009) The split bregman method for l1-regularized problems. SIAM J
Imaging Sci 2:323–343. doi:10.1137/080725891, http://portal.acm.org/
citation.cfm?id=1658384.1658386
5. Kim S, Lim H (2007) A non-convex diffusion model for simultaneous image denoising and
edge enhancement. Electron J Differ Equ Conference 15:175–192
6. Krissian K, Kikinis R, Westin CF, Vosburgh K (2005) Speckle-constrained filtering of
ultrasound images. In: CVPR ’05: Proceedings of the 2005 IEEE computer society
conference on computer vision and pattern recognition (CVPR’05), vol 2. IEEE Computer
Society, Washington, DC, pp 547–552. doi:10.1109/CVPR.2005.331
7. Li C (2003) An efficient algorithm for total variation regularization with applications to the
single pixel camera and compressive sensing. Master’s thesis, Rice University, Houston,
Texas
8. Rudin LI, Osher S, Fatemi E (1992) Nonlinear total variation based noise removal
algorithms. Physica D: Nonlinear Phenom 60(1–4):259–268. doi:10.1016/0167-
2789(92)90242-F
9. Wang Y, Yin W, Zhang Y (2007) A fast algorithm for image deblurring with total variation
regularization. Technical report, Department of Computational and Applied Mathematics,
Rice University, Houston, Texas
10. Zhou D, Schlkopf B (2004) A regularization framework for learning from graph data. In:
ICML workshop on statistical relational learning, pp 132–137
11. Zhou D, Schlkopf B (2005) Regularization on discrete spaces. In: Pattern recognition.
Springer, Heidelberg, pp 361–368