Review on CNN Based Image Denoising
Dinsha Babu and Sajeev K. Jose
Department of Electronics and Communication Engineering
Government College of Engineering Kannur
Abstract—In the last few years, convolutional neural networks
(CNNs) have attracted good attention in the area of denoising.
There have a lot of approaches to remove noise from an affected
image and to reconstruct a clean image with high visual quality.
Some of the non-CNN related approaches are much efficient to
remove noise from affected images. But these methods also show
some issues like complex optimization, Computational expenses
etc. while performing denoising. To beat the disadvantages of
these denoising approaches, CNN related methodologies are
created to enhance the denoising performance. These denoising
strategies are drawing in acceptable considerations for their
efficient performance in image rebuilding. They utilize some
techniques like residual learning to enhance the denoising per-
formance. So this paper provides a review on some CNN related
image denoising approaches.
Index Terms—Denoising, CNN, Residual learning
I. I NTRODUCTION
Image denoising is a notable old style strategy use for
image reclamation and it is dynamic subject in low level
vision. Image denoising shows wide applications in numerous
fields like human entertainment and pathological analysis.
In the course of recent years, numerous endeavours have
been devoted to making various calculations for denoising
of images. The fundamental objective of denoising is to
recuperate an unmistakable image from a noise affected image.
The corruption model utilized in image denoising is shown as
a = b + m, where a is noise affected image, b is a noiseless
image and m is the added Gaussian noise, m has a standard
deviation σ. Noise defilement consistently influences the visual
nature of the actual image. Noise expulsion from an affected
image is a crucial step for computer vision assignment and
image handling. Additive white Gaussian noise or AWGN is
a typical noise, which generally influence the image during
handling. In early techniques for removing noise from affected
image, analysts eliminated the AWGN by presenting various
filters like bilateral filter and Gaussian filter or varying the co-
efficients using transforms like wavelet and Fourier transforms.
However, some of the customary techniques have restricted
and imperfect image priors, so it can’t efficiently remove
noise from images. As expressed in the Bayesian hypothesis,
assessing prior is significant for removing noise. For instance,
wavelet transformation with a prior of Markov random field
Dinsha Babu is an M. Tech student associated with Department of Elec-
tronics and Communication Engineering, Government College of Engineering
Kannur, Kerala, India (e-mail: dinshababudinu@gmail.com)
Sajeev K. Jose is a faculty of Department of Electronics and Communica-
tion Engineering, Government College of Engineering Kannur, Kerala, India
(e-mail: sajeev@gcek.ac.in)
178
Electronic copy available at: https://ssrn.com/abstract=3791105
Government College of Engineering Kannur (GCEK)
is utilized to eliminate noise from pictures. Joining the self-
similarities and sparse representation can lessen the storage
for picture denoising and enhance the performance.
Over the last few years, various techniques that determine
specific image priors have been utilized for removing noise
from affected image. It comprise of Markov random field mod-
els (MRF), gradient models, sparse models and nonlocal self-
similarity models (NSS), among these NSS based strategies
are more famous technique for denoising. The NSS infers
that a few patterns happen over and again in a picture and
the picture patches with comparable patterns can be found
away from one another. The execution of NSS prior has
enhanced the exhibition of noise removal fundamentally, and
numerous on-going denoising calculations can be named NSS
based strategies. NSS models like nonlocally centralised sparse
representation (NCSR) [8], Weight nuclear norm minimization
(WNNM) [9] and Block-matching and 3D filtering (BM3D)
[7] are extremely successful for picture denoising measure. In
any case, these conventional denoising techniques have a few
impediments, they have to set the boundaries physically to get
the optimal outcomes and they utilize complex optimization
calculation to improve the denoising efficiency, so it expands
the computational expense.
To beat the disadvantages of prior related denoising tech-
niques, some discriminative learning related methodologies are
as of late created to learn picture prior models. Plain MLP
technique proposed by Burger et a [20] is the main discrim-
inative denoising strategy which furnishes similar execution
with BM3D. Cascade of shrinkage fields model (CSF) that
consolidates the random field related model and quadratic op-
timization. Teachable nonlinear response dissemination model
(TNRD) [11] is also a well-known learning related technique
used for noise removal from images. However, these strategies
are restricted in a particular prior model.
As of late, neural network (NN) based denoising strategies
are drawing in acceptable considerations for their efficient
performance in image rebuilding. They first train the network,
then the network accept input as noisy patches and the noise-
less clear patches are estimated from the noise patches. Each
network contains set of non-linear activations and convolution
operations. It distinguishes the hidden prior of image from
the training set for image recovery. Deep learning approaches
have best learning capacity and adaptable network design,
which improve the efficiency of denoising. CNN is a deep
learning approach, which have pulled in more consideration in
denoising of noisy images [12]. Rectifier Linear Unit (ReLU)
[13], residual learning [15] and batch normalization (BN) [14]
are utilized in CNN to quicken the training of network and
improve the efficiency of denoising. When sample data is
given to the network, its distribution changes after it goes
through the convolution layer. This problem is termed as inter-
nal covariate shift problem. This issue can be diminished by
utilizing batch normalization method. Batch normalization first
normalizes the sample data then it utilizes scale and shift tasks
to recover the training data distribution. Activation function is
placed after BN of each layer. ReLU is a commonly using
activation function in CNN. In learning related denoising, the
efficiency of noise removal increase as the network depth
increase. However, expanding the network depth may prompt
the disappearing gradient issues. Applying residual learning
approach to the CNN is a good choice to remove this issue. It
primarily adds the output of different feature layers with input
image and given as an input of the next layer to ensure the
performance.
CNN is appropriate for equal calculation on current pow-
erful GPU and it can be utilized to enhance the run time
execution. The combination of CNN and qualities of denoisng
phase is helpful to eliminate obscure noise. Consolidating
CNN and nature of pictures is exceptionally viable to acquire
a noiseless image. CNN shows better modelling capacity; this
is one of the best advantages of CNN to use as a denoiser.
The primary CNN-based denoiser consists of 5 layers and it is
invented by Jain and Seung [18]. However, this strategy can’t
beat with the notable BM3D denoising approach [7] due to
the shallow network layer.
II. LITERATURE REVIEW
A. DnCNN based Denoising
The current discriminative learning approaches like TNRD
[11] and CSF [10] are based on analysis model, so they
cannot perfectly capture all features of image. Also they use
handcrafted parameters. Their network training occurs at a
particular noise level. Denoising convolutional neural network
(DnCNN) [1] utilizes BN and residual learning [14, 15] to
quicken the network training and enhance the efficiency of
noise removal. Noise reduction in DnCNN occurs in all noise
level.
As opposed to straightforwardly recovering the clean image
a, DnCNN works in a way to estimate the residual picture w,
i.e., w is the contrast between the noisy and clear images.
The usage of BN procedure in DnCNN helps to enhance
and maintain the network training efficiency. Reconciliation
of both BN and residual learning approach is helpful to
enhance the overall performance of denoising. The trained
DnCNN network with a particular noise level can results a
good recovery of image from Gaussian noise when compare to
other traditional methods of noise reduction such as WNNM,
TNRD and BM3D.
To complete a particular task using CNN model includes 2
steps. That are, (I) design the corresponding network architec-
ture, (ii) learning of model using trained data. Designing of
network architecture involves fixing of network depth, which
dependent on patch sizes utilized in traditional methods of
179
Electronic copy available at: https://ssrn.com/abstract=3791105
Government College of Engineering Kannur (GCEK)
Fig. 1. Block diagram of DnCNN [1].
denoising. For network learning, reconciliation of both BN
and residual learning approach is helpful to enhance the
overall performance of denoising. There have three sorts of
layers in this approach, ReLU and convolution are include
in first layer, where rectified linear unit (ReLU) [13] is the
activation function used for nonlinearity. Second layer includes
Convolution, ReLU and BN, where BN helps to accelerate
network training and enhance the performance. Last layer is
a convolution layer. DnCNN can easily sort out clean image
from noisy image via hidden layers by consolidating ReLU
with convolution. Applying zero padding to the input image
helps to maintain the size of each and every feature maps in
inner layers unchanged. Zero padding also avoids boundary
artifacts.
For a certain noise level, current discriminative denoising
strategies like MLP [20], TNRD and CSF can only train a
single model. In any case, those strategies can’t be applied
to non-Gaussian type noise such as JPEG de-blocking and
SISR. But in case of DnCNN, it is conceivable to use one
CNN model for a few general picture denoising functions, for
example, Gaussian noise removal with obscure noise level,
JPEG de-blocking using diverse quality elements and SISR
using various up-scaling components. DnCNN shows the good
PSNR value than other contending strategies like BM3D and
TNRD. Proposed DnCNN shows a moderately high speed on
CPU, also it is quicker than two discriminative approaches,
MLP and CSF. In any case, DnCNN can’t exceed the BM3D
and TNRD in speed on CPU.
B. BMCNN based Denoising
Non-local self-similarity (NSS) prior-based techniques and
CNN based methods are two main algorithms of image
denoising. Methods built on NSS are used on Standard and
repeated image patterns and CNN based techniques are used
on irregular structures. Denoising based on NSS integrates
similar input image patches into a 3D block then the block
is denoised. But it indicates certain demerits. First, as the
block denoising phase is made based on a particular prior,
this is tough to satisfy combined features of an image. Image
priors are human observation-based, Therefore, it might not be
optimal. Some parameters of this method should be adjusted
manually. The optimization process in NSS based techniques
such as NCSR [8], WNNM [9] and LSSC are complex and
time consuming.
A novel method called block-matching convolutional neural
network (BMCNN) [2] combine NSS prior method with CNN
based techniques. Patch based techniques are mainly used in
many method of image denoising. So in this method equivalent
local patches of input noisy image are combined to form a 3D
block like in BM3D. An existing algorithm for denoising is
used to denoise the noisy input image patches to avoid the
noise during block matching. It provides more detailed result.
This denoised input image is then termed as a pilot image for
the process of block matching. After block matching, resulting
pilot patch blocks are used as a reference for again aggregating
identical patches from input image. Then the denoised patches
and noisy input patches are grouped together, because some
information may loss during denoising process, so nnoisy
patch of inputs will help to reconstruct the image features.
These blocks are given as an input to the CNN based denoising
network. DnCNN [1] can be used as training based method.
It has better denoising performance. BM3D [7] can be used
as NSS based method for preprocessing. BM3D is faster than
other NSS based methods.
Fig. 2. Block diagram of BMCNN [2].
BMCNN outperform DnCNN algorithm. BMCNN shows
the highest PSNR average for all noise level. BMCNN is
also much quicker than other approaches based on NSS.
Performance of DnCNN with the formulation is saturated,
so for deeper network it is difficult perform better. BMCNN
can include extra information on the network. But BMCNN is
slower when compare to other CNN based denoising methods.
Because this method is a two-step process that includes the
denoising based on both NSS and CNN. So it results a large
computational cost. Block matching phase in BMCNN is
difficult to design with GPU.
C. ResNet based Denoising
Residual Denoising Network is a deep neural network
for solving denoising problem [3]. ResDNet architecture is
inspired by powerful optimization strategy and image regu-
larization approaches. This approach leads to a high quality
reconstruction of image and use less trainable parameters.
Residual network is similar to DnCNN that is the network
act as a noise estimator by subtracting the network output
from its input image. Unlike DnCNN, ResDNet accept two
inputs that are distorted input and the variance of the noise.
ResDNet works on wide range of noise level. DnCNN has
an internal mechanism to estimate the noise variance so it
can apply for different noise levels. But if the noise statistics
diverge from training condition then the internal mechanism
will fail. It affects the denoising performance of DnCNN
180
Electronic copy available at: https://ssrn.com/abstract=3791105
Government College of Engineering Kannur (GCEK)
[28]. So ResDNet gives better result than DnCNN. But in
ResDNet denoising, each noise level requires the training of
deep network. So it affects the storage capacity of devices,
which implemented this network.
ResDNet consist of 5 blocks. First block is a convolutional
layer with 64 filters. Second block is a non-linear block.
Non-linear block have a parametrized rectified linear unit
activation function (PReLU), it is followed by a convolutional
layer with 64 filters. The output of non-linear block is given
to a transposed convolution layer it reduces the number of
channels from 64 to 3. Transposed convolution layer follows a
projection layer. Noise variance is given as an additional input
to the projection layer, which is used to normalize the noise
realization estimate to get the correct variance, before this is
subtracted from the network input. Final output is clipped to
get the output intensity range between 0 and 255. Inputs of
each convolution layers are padded to ensure that each feature
map shows the same spatial size as the input image. Normally
most of the deep learning networks use zero padding but
reflective padding is used in this network. Also this network
does not use batch normalization after convolutions, rather it
use parametric convolution representation.
Fig. 3. Block diagram of ResDNet [3].
D. FFDNet based Denoising
Most discriminative learning strategies for denoising utilize
an explicit model for each and every noise level. So it is
important to utilize different models for denoising pictures
with different noise levels. Likewise they are not adaptable
to handle with spatially variant noise. NSS based denoising
strategies like WNNM and BM3D are adaptable in different
noise levels. Yet, their improvement calculations are time con-
suming. It can’t straightforwardly eliminate spatially variant
noise. Likewise NSS based techniques generally use hand-
made image priors like nonlocal self-similarity and sparsity
and they are feeble to identify complex image forms.
Fast and flexible denoising convolutional neural network
(FFDNet) [4] can be utilized to beat these issues. It use tunable
noise level map as input to manage a wide range of noise level,
additionally the noise level map control the trade-off between
detail preservation and noise reduction. FFDNet is versatile
to manage spatially varying noise by characterizing a non-
uniform noise level map. The mapping function of DnCNN
model is a = F (b; θσ) where b is the input noisy observation,
a is the desired output and θσ is the model parameter with
fixed noise level σ. Model parameter vary with change of
noise level. For FFDNet, a = F (b, N ; θ), where N is a noise
level map. Here the noise level map is displayed as an input
and the model parameter don’t change with noise level. So
FFDNet is adaptable to deal with spatially variant noise than
other existing CNN based strategies. FFDNet work on down-
sampled subimages, which helps to speed-up the testing and
training speed and broaden the receptive field, also it increase
the efficiency of network.
A reversible down-sampling operator is utilized to reshape
the input image into four down-sampled sub-pictures. Orthog-
onal initialization technique is utilized to the convolution filter
to empower noise level map to control the trade-off between
detail preservation and noise reduction without any visual
artifacts. The tunable noise level map M and down-sampled
sub-images are joined together and given as an input to the
network. Each layer in the network consists of three operations
that are Convolution (Conv), Rectified Linear Units (ReLU)
[13] and Batch Normalization (BN) [14]. Zero padding is
applied after every convolution to make the feature map size
unchanged. Finally sub-pixel convolution layer is placed in the
network to invert the down-sampling process. Output of the
organization is given to an up-scaling operator; it is used to
deliver the assessed clean image. FFDNet operates on down-
sampled subimages so there is no need of utilizing widened
convolution dilated convolution to additionally increase the
receptive field. Down-sampled sub-images denoising can ef-
fectively increase the receptive field and it improves efficiency.
Fig. 4. Block diagram of FFDNet [4].
Sub-sampling and sub-pixel convolution in FFDNet are
effective to reduce memory storage. So FFDNet is memory
friendly. Compare to DnCNN, FFDNet doesn’t predict the
noise, because FFDNet do not use residual learning for net-
work design. FFDNet outperform NSS based methods like
BM3D and WNNM for a wide range of noise level. But when
compare to the DnCNN, FFDNet is less efficient when the
noise level is low and it outperforms DnCNN as the noise
level increases. FFDNet can remove strong noises, because it
has large receptive field than DnCNN. But DnCNN has good
modelling capacity, so it is suitable for denoising lower noises.
Run time of FFDNet is better than DnCNN on GPU and CPU.
If the input noise level is lower than the actual noise level,
the noise cannot be perfectly removed. So high noise level is
required for better denoising. But using high noise level will
also results the removal of some image details with noise.
E. EDCNN based Denoising
The current CNN based approaches cannot remove the
high level noise completely and perfectly. Enhanced deep
convolutional neural network (EDCNN) [5] can be used to
overcome this issue. It utilizes both global and local residual
learning approaches and residual excitation approach for noise
removal. The local residual learning (LRL) approach reduces
gradient vanishing. Global residual learning (GRL) approach is
181
Electronic copy available at: https://ssrn.com/abstract=3791105
Government College of Engineering Kannur (GCEK)
utilized to obtain mapping between input and residual images,
which eliminate the clean picture with activities in the inner
layers and it additionally help the network to train deep
layers without any problem. Residual excitation (RE) help to
make short ways from input picture to output block. Residual
excitation reduces gradient vanishing and helps to obtain final
output clear image with more details. EDCNN contain 52
feature layers, so it has large network depth compare to other
CNN networks for image denoising.
EDCNN includes three sections, first one is low-level data
extraction block (LIB), second block is noise feature extraction
blocks (NFB) and the last one is output block (OB). LIB
have the activation function ReLU [13] to remove linearity
of convolution output, two convolution layers and BN [14]
layer. BN works between convolution and ReLU. NFB block
creates a short way between outputs of current block and
next block with the help of residual learning approach. GRL
approach helps to remove low-level noise from corrupted
images. But for high-level noise, the variation between noise
image a and noiseless clean image b is large. This issue can
be sort out by using RE strategy, it helps to decrease the
variation between noise image a and noiseless clean image b.
So the resulting output clean image acquires more information
from input image. Each NFB block comprises of eight layers
of convolution. Output block contain one convolution layer.
EDCNN has good denoising performance than DnCNN and
it provide more details and reconstruct sharp edges when
compare with other CNN and NSS related noise removal
approaches.
Fig. 5. Block diagram of EDCNN [5].
F. ECNDNet based Denoising
The current deep CNN related networks can be utilized
as good denoising approach. But they show some drawbacks
like, difficulty to train images and performance saturation
challenges faced by deeper network. Most of the CNN based
methods like BMCNN [2] and DCCRDN [3] have some
drawbacks, they results the vanishing of gradient when the
network is very deep and their computational cost is very high.
Enhanced convolutional neural denoising network (ECND-
Net) [6] is a novel denoising method, which uses BN and
residual learning approach to overcome the training issues and
to get a better denoising. Residual learning method [15] helps
to avoid the vanishing of gradient [19] and batch normalisation
enhance the training efficiency of model by normalizing the
data. Dilated convolution technique [17] is used to increment
the receptive field and enhance the network performance;
it utilizes dilated filter and dilation factor to obtain more
acquired data. Extending the CNN’s receptive field is helpful
for getting more highlights for picture denoising. Two well-
known approaches to grow the receptive field are expanding
the network depth and extending the network width. But
extending the network width makes more parameters; it leads
to the network over-fitting and large computational expense.
Expansion of network depth leads to gradient vanishing prob-
lem. As a result, using dilated convolution in network is a
better decision to expand the receptive field. ECNDNet has 17
layers so it reduces the computational cost. Second, fifth, ninth
and twelfth layers of ECNDNet contains dilated convolution,
it additionally diminishes the computational expense. The
first furthermore, sixteenth layers contain both ReLU and
convolution. Dilated convolution, BN and ReLU are included
in the second, fifth, ninth and twelfth layers of ECNDNet. In
particular, the dilated factor in dilated convolution is crucial
to expand the receptive field.
Fig. 6. Block diagram of ECNDNet [6].
Both ECNDNet and DnCNN [1] have same network depth,
but ECNDNet is better than DnCNN in run time. It has high
PSNR value compare to other existing CNN based denoising
methods.
III. C ONCLUSION
There have a lot of approaches to remove noise from an
affected image and to reconstruct a clean image with high
visual quality. Different CNN and non-CNN related denoising
approaches have the ability to provide a good image with
efficient performance. But some of the CNN related noise
removing approaches shows excellent result when compare
to current popular non-CNN approaches. Because of the good
modelling capacity shown by the CNN network, it can denoise
the images with good efficiency. BN, ReLU and residual
learning are some techniques utilized in CNN network to
enhance the denoising performance. These techniques help to
quicken the training phase of the network and to fasten the
over-all process. Some of the CNN networks for denoising are
reviewed in the paper. DnCNN, BMCNN, FFDNet, ResNet,
EDCNN and ECNDNet are the reviewed networks for noise
recovery, and they shows comparatively good PSNR value than
that of other non-CNN approaches.
182
Electronic copy available at: https://ssrn.com/abstract=3791105
Government College of Engineering Kannur (GCEK)
R EFERENCES
[1] K. Zhang, W. Zuo, Y. Chen, D. Meng, and L. Zhang, “Beyond a gaussian
denoiser: Residual learning of deep cnn for image denoising,” IEEE
Trans. Image process., 2017.
[2] Ahn, B., Cho, N.I.: ‘Block-matching convolutional neural network for
image denoising’, arXiv preprint arXiv:1704.00524, 2017.
[3] Kokkinos, F., Lefkimmiatis, S.: ‘Deep image demosaicking using a
cascade of convolutional residual denoising networks’, arXiv preprint
arXiv:1803.05215, 2018. 1983.
[4] Zhang, K., Zuo, W., Zhang, L.: ‘FFDNet: toward a fast and flexible
solution for CNN based image denoising’, IEEE Trans. Image Process.,
2018, pp. 4608–4622..
[5] Chunwei Tian, Yong Xu, Lunke Fei, Junqian Wang, Jie Wen, Nan
Luo.: ‘Enhanced CNN for image denoising ‘,CAAI Transactions on
Intelligence Technology, IET, 2019.
[6] Haizang Zou, Rushi Lan, Yanru Zhong, Zhenbing Liu, Xiaonan Luo,
“EDCNN: A Novel Network for Image Denoising,” in IEEE Interna-
tional Conference on Image Processing, 2019.
[7] K. Dabov, A. Foi, V. Katkovnik, and K. Egiazarian, “Image denoising by
sparse 3-D transform-domain collaborative filtering,” IEEE Transactions
on Image Processing, vol. 16, no. 8, pp. 2080–2095, 2007.
[8] W. Dong, L. Zhang, G. Shi, and X. Li, “Nonlocally centralized sparse
representation for image restoration,” IEEE Transactions on Image
Processing, vol. 22, no. 4, pp. 1620–1630, 2013.
[9] S. Gu, L. Zhang, W. Zuo, and X. Feng, “Weighted nuclear norm
minimization with application to image denoising,” in IEEE Conference
on Computer Vision and Pattern Recognition, 2014, pp. 2862–2869.
[10] U. Schmidt and S. Roth, “Shrinkage fields for effective image restora-
tion,” in IEEE Conference on Computer Vision and Pattern Recognition,
2014, pp. 2774–2781.
[11] Y. Chen and T. Pock, “Trainable nonlinear reaction diffusion: A flexible
framework for fast and effective image restoration,” to appear in IEEE
transactions on Pattern Analysis and Machine Intelligence, 2016.
[12] K. Simonyan and A. Zisserman, “Very deep convolutional networks for
large-scale image recognition,” in International Conference for Learning
Representations, 2015.
[13] A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification
with deep convolutional neural networks,” in Advances in Neural
Information Processing Systems, 2012, pp. 1097–1105.
[14] S. Ioffe and C. Szegedy, “Batch normalization: Accelerating deep
network training by reducing internal covariate shift,” in International
Conference on Machine Learning, 2015, pp. 448–456.
[15] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for
image recognition,” in IEEE Conference on Computer Vision and Pattern
Recognition, 2016, pp. 770–778.
[16] Lefkimmiatis, S.: Universal denoising networks: A novel cnn architec-
ture for image denoising. In: Proceedings of the IEEE Conference on
Computer Vision and Pattern Recognition. (2018) 3204–3213.
[17] K. Zhang, W. Zuo, S. Gu, and L. Zhang, “Learning deep CNN denoiser
prior for image restoration,” in IEEE Conference on Computer Vision
and Pattern Recognition, 2017, pp. 3929–3938.
[18] V. Jain and S. Seung, “Natural image denoising with convolutional
networks,” in Advances in Neural Information Processing Systems,
2009, pp. 769–776.
[19] T. Tong, G. Li, X. Liu, and Q. Gao, “Image superresolution using
dense skip connections,” in IEEE International Conference on Computer
Vision, 2017, pp. 4809–4817.
[20] H. C. Burger, C. J. Schuler, and S. Harmeling, “Image denoising: Can
plain neural networks compete with BM3D?” in IEEE Conference on
Computer Vision and Pattern Recognition, 2012, pp. 2392–2399.