Is Denoising Dead?

IIVP Mini-Project
Group Members:
IIT2013142, IIT2013178, IIT2013180
IIT2013181, IIT2013182, IIT2013202

oising problem. They have estimated a lower bound on the m

ean squared error of the denoised result and compared the p
erformance of state-of-the-art denoising methods with this b
ound.
Abstract
It has been shown that despite the phenomenal recent progr
ess in the quality of denoising algorithms, some room for im
provement still remains.
To conclude, it has been said that image denoising is not dea
d.

hly susceptible to noise. Manufacturers therefore rely on de noising alg

orithms. Recently proposed state-of-the art techniques produce impres
sive, but comparable results, forcing us to think if weve reached the lim
it of de noising.
Earlier studies have formulated expressions for optimal filtering param
eters that define resolution limits to recovering lost information, witho
ut defining any statistical performance limits of de noising.
Besides proposing an upper limit of extent of de noising, the study has
also led to understanding the fundamental limit of increasing number o
f sensors in imaging system with acceptable image quality being made
possible by noise suppression.

Introduction

The image formation model is defined as

Where yi = observed pixel intensity
zi = actual pixel intensity
i = noise

Introduction(Contd.)

Here noise is assumed to be independent identically distri

buted(IID), with zero mean and variance of 2.
Aim of de-noising algorithms is to recover the noise-free pi
xel intensity .

actual patch intensity, yi is vectorized noisy image patch and

i is noise patch.
For calculation of performance limits, noise-free image has b
Patch-Based
een assumed to be Image
available Model
and then aim is find out how we
ll, in terms of MSE, the image can be denoised. AWGN has be
en taken into consideration
Image is assumed to be made up of clusters, containing patc
hes of similar geometrical structure.

Patch-Based Image Model(Contd.)

The patch based image model is defined as

Where yi = vectorized noisy image patch
zi = actual patch intensity
i = vectorized noise patch

Here2 vectorized noise patch has co-variance

of I.

Patch-Based Image Model(Contd.)

Each image is considered to be composed of a finite number

of clusters.
Each such cluster contains patches of the image which have
similar geometric structure.
For instance, considering Fig. 1 (next slide), there are four
relevant clusters. Namely, these describe the horizontal
edges, the vertical edges, the corners,4 and the flat
regions.
It is worth noting that clusters are composed of regions which
may have quite different actual grey values, but that are
nonetheless visually similar (flat, edge, etc.).

Patch-Based Image Model(Contd.)

e between this estimator'sexpected valueand the true value of the par

ameter being estimated.
An estimator or decision rule with zero bias is calledunbiased. Otherw
ise the estimator is said to bebiased.
Bias in estimators for high-quality image de-noising should be expected
as
It is well known that unbiased estimators dont always exist
Biased estimators result in lower MSE, so its advantageous to work wit
h them
Unbiased estimators often have unacceptably high variance

Biasing in De-noising

Local behaviour of the bias function has been assumed to be

affine, which is reflective of all leading state-of-the art algorit
hms.

Biasing
Where, zin
is aDe-noising
random vector(Contd.)
having pdf p(z)

b(z) is bias of estimator

Matrix M and vector u are the parameter of affine based model.

Authors have experimentally showed that the affine

model for bias is good local fit.

Why Affine Model ?

Experiment to substantiate claim that bias model is

approximately affine
Noisy images are simulated by corrupting the 512x512 textured grass

image with 10 different realizations of additive white Gaussian noise of

standard deviation 25.
The noisy images are then denoised with each of the 4 state-of-the-art
methods, namely BM3D, K-SVD, SKR and K-LLD
The mean denoised image is obtained for each method
From this, the bias vectors are obtained for each method using nonoverlapping 11x11 patches. The bias vectors of all such patches are
tiled to form the method bias images.
Predicted bias patches are then computed for each patch in8/11/16
every
case.

Why Affine Model ? (Contd.)

For the quantitative evaluation, we use the coefficient of
determination which can be defined as
Where i indexes all the patches in the image,
is the actual bias of the estimated intensity of the ith patch
is the mean bias obtained by the denoising method across all
patches in the
image and
is the predicted bias obtained from the estimated parameters
and of the affine model
8/11/16

Why Affine Model ? (Contd.)

It has been empirically determine that R

values greater than

0.6 give a reasonably good estimate.

Looking at the observations, it becomes clear that affine
model is a good local fit.
8/11/16

In statistics and signal processing literature, several bounds were devel

oped to evaluate performance limits of estimation.
Some bounds were developed for the estimation of a deterministic par
ameter, while other address the Bayesian setting where the parameter
of interest is a random variable.
Bayesian version of the CRLB has been used where the CRLB is a lower
bound on the variance of an unbiased estimator

Optimal Bias Bayesian Cramer-Rao Lower

Bound (OB CRLB)

Optimal Bias Bayesian Cramer-Rao Lower

Bound (OB CRLB) (Contd.)
Bound on the conditional covariance of estimate
has been formulated using conditional pdf p(y|
) for unbiased case is given by
Here J is Fisher Information Matrix given by
Most denoising methods are biased in nature an
d this bias needs to be taken into account to obt
ain a useful lower bound. For such cases, the CR
LB on the covariance of the biased estimate is gi
ven by

Where I is Identity Matrix

and M is the gradient of
bias

Calculation of Lower Bound on MSE

We can calculate lower bound on
conditional MSE of z as

By law of total expectation,

overall Bayesian Mean
Square Error can be
expressed as

The pdf p(z) is referred to as Prior Knowledge and most de

noising methods make use of it to achieve improved

performance
Although it seems that complete knowledge of p(z) is
necessary for evaluation of lower bound, it has been shown
that only the first and second order moments p(z) are
sufficient to obtain a good approximation of above

Deriving the Bayesian MSE Bound

To obtain lower bound on MSE, we need to establish optimal

values of M and u in order to minimize the the RHS of

previous equation
Denoting these optimal values by M* and u*, we have

Optimum M* and u* can be obtained by differentiating above

w.r.t Q and M and solving simultaneously

8/11/16

Deriving the Bayesian MSE Bound (Contd.)

Upon solving the equations simultaneously, we get
Substituting above values of M* and u* and simplifying, we
get

Hence the lower bound on MSE becomes

8/11/16

Fisher Information Matrix

The only unknown value of the expression for lower

bound on MSE is J, which is Fisher Information Matrix

(FIM) given by

where N = total no of patches taken into account for

denoising
I = Identity Matrix
2 = Variance of z

Results for Simulated Images

Adjacent figure shows a simulated

image (of size 220x220) that was

generated to provide a proof of
concept for our MSE bounds
calculation.
The image consists of simple
repeating patterns (stripes), each 2
pixels wide. The image is made up
of two gray levels (75 and 200).
8/11/16

Results for Simulated Images (Contd.)

Its easy to see that for a sufficiently large patch size the

image patches will all be similar in geometric structure, and,

hence, no clustering is necessary for this particular image.
Fixing the patch size to be 11x11, the performance bounds of
denoising this particular image was calculated and compared
with various state-of-the-art denoising methods
MSE bound is quite small as a result of larger number of
identical patches being available. Also, the image consists of
a very simple repeating pattern leading to rather small
variability in geometric structure of the image patches. This
makes its easier to denoise as opposed to more complex
8/11/16
natural images.

Results for Simulated Images (Contd.)

Shown aside is another simulated

image (of size 200x200)

As opposed to the stripes image,
the edges vary in directionality.
Clearly, such an image requires the
use of multiple clusters to capture
the different geometric structures.
8/11/16

Results for Simulated Images (Contd.)

As shown earlier, we make use of four clusters to capture the

smooth, horizontal and vertical edges, and the corner regions.

It can be seen that this image is more difficult to denoise than

the stripes image and the predicted MSE bound is

considerably lower than the MSE obtained by any of the stateof-the-art denoising methods.
8/11/16

Results for General Images With Single Cluste

ring

8/11/16

General Test Images

8/11/16

Results for General Images With Clustering (Co

ntd.)

8/11/16

Results for General Images With Clustering

8/11/16

Conclusion
The results showed that there is room for improvement in de

noising a class of sufficiently simple images.

On the other hand, images rich in texture have already been
de noised close to the limit of performance.
Future Scope : The bounds formulation can also be extended
to study performance limits of de noising videos.

8/11/16

Thank You