862
3, JULY 2020
Underwater Image Enhancement Using a Multiscale
Dense Generative Adversarial Network
Yecai Guo , Hanyu Li, and Peixian Zhuang
Abstract—Underwater image enhancement has received much
attention in underwater vision research. However, raw underwater
images easily suffer from color distortion, underexposure, and fuzz
caused by the underwater scene. To address the above-mentioned
problems, we propose a new multiscale dense generative adversar-
ial network (GAN) for enhancing underwater images. The residual
multiscale dense block is presented in the generator, where the mul-
tiscale, dense concatenation, and residual learning can boost the
performance, render more details, and utilize previous features,
respectively. And the discriminator employs computationally light
spectral normalization to stabilize the training of the discrimina-
tor. Meanwhile, nonsaturating GAN loss function combining L1
loss and gradient loss is presented to focus on image features of
ground truth. Final enhanced results on synthetic and real under-
water images demonstrate the superiority of the proposed method,
which outperforms nondeep and deep learning methods in both
qualitative and quantitative evaluations. Furthermore, we perform
an ablation study to show the contributions of each component and
carry out application tests to further demonstrate the effectiveness
of the proposed method.
Index Terms—Dense concatenation, generative adversarial net-
work (GAN), multiscale, residual learning underwater image
enhancement.
I. INTRODUCTION
ECENTLY, the underwater imaging has played an impor-
R tant role in deep ocean exploration, underwater robotics,
and sea life monitoring. However, raw underwater images sel-
dom fulfill the requirements concerning image processing. Due
to light attenuation and scattering in the water, color distortion,
underexposure, and fuzz are three major problems of image
degeneration [1]. First, considering the depth, light conditions,
water type, and different light wavelengths, the color of under-
water images is often distorted [2]. Second, the absorption of
light energy results in underexposure. The objects at a distance
of more than 10 m from the camera lenses are almost indis-
tinguishable [3]. Third, the fuzz of underwater images can be
Manuscript received December 7, 2018; revised March 5, 2019; accepted
April 9, 2019. Date of publication June 4, 2019; date of current version July
14, 2020. This work was supported in part by the National Natural Science
Foundation of China under Grant 61701245, in part by the Startup Founda-
tion for Introducing Talent of Nanjing University of Information Science and
Technology 2243141701030, and in part by a project funded by the Prior-
ity Academic Program Development of Jiangsu Higher Education Institutions.
(Corresponding author: Yecai Guo.)
Associate Editor: H. Zheng.
The authors are with the School of Electronic and Information Engi-
neering, Nanjing University of Information Science and Technology, Nan-
jing 210044, China (e-mail: guo-yecai@163.com; lihanyu1204@163.com;
zhuangpeixian0624@163.com).
Digital Object Identifier 10.1109/JOE.2019.2911447
0364-9059 © 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.
See https://www.ieee.org/publications/rights/index.html for more information.
Authorized licensed use limited to: Tsinghua University. Downloaded on December 20,2020 at 12:09:37 UTC from IEEE Xplore. Restrictions apply.
IEEE JOURNAL OF OCEANIC ENGINEERING, VOL. 45, NO.
attributed to the following two factors. The abundant suspended
particles cause shifts in light scattering and propagation direc-
tion. Furthermore, both suspended particles and water affect the
scene contrast by reflecting light to the camera lenses [4].
To improve underwater images, traditional methods include
enhancement methods and restoration methods. Image enhance-
ment methods [3]–[6] without underwater physical parameters
focus on adjusting image pixel values to produce appealing
results. Image restoration techniques [7]–[11] consider the
degradation model to enhance underwater images. However,
various complex underwater physical and optical factors are re-
quired that make these traditional methods inflexible to be imple-
mented. Due to the lack of abundant training data, these methods
display poor generalization performance in different underwater
images, and enhanced images of some scenes tend to be overen-
hanced or underenhanced. Deep convolutional neural networks,
powerful supervised learning models, obtain convincing success
on low-level vision tasks, e.g., image superresolution [12], image
deraining [13], and image denoising [14], and some researchers
apply deep learning to underwater image processing [15]–[22].
In this paper, we develop a trainable multiscale dense gener-
ative adversarial network (GAN), and the main contributions of
this paper are summarized as follows.
1) We propose a novel multiscale dense block (MSDB) with-
out constructing the underwater degeneration model and
image prior. The effective combination of residual learn-
ing, dense concatenation, and multiscale can correct color
casts and recover image details to improve subjective and
objective evaluations. In addition, the ablation study is
conducted to demonstrate the effect of each component in
the proposed block.
2) The meaningful adversarial loss including the L1 and gra-
dient loss is adopted to preserve image features of ground
truth. Meanwhile, the spectral normalization stabilizes the
training of the discriminator, which is computationally
light, fast, and easy to incorporate into GAN training.
3) With two no-reference metrics adopted for the underwa-
ter environment, numerous experiments are provided to
demonstrate the superiority of the proposed method on
both synthesized and real-world underwater images. Fi-
nally, we carry out application tests to further show the
effectiveness of the proposed method.
II. RELATED WORK
Given the importance of underwater vision, many methods
toward underwater images enhancement have been proposed
GUO et al.: UNDERWATER IMAGE ENHANCEMENT USING A MULTISCALE DENSE GENERATIVE ADVERSARIAL NETWORK
in recent years. Existing approaches of improving underwater
image quality can be summarized into the following categories.
A. Enhancement-Based Methods
Image-enhancement-based methods focus on adjusting image
pixel values to produce a subjectively and visually appealing
image. The literature [3] derives the inputs and weights from
a raw underwater image. There are two inputs that one white
balanced version discards unwanted color casts of the subsea
images and another filtered version renders the details. Ad-
ditionally, four weight maps aim to determine which pixel is
advantaged to appear in the restored output. However, the en-
hanced image easily becomes overenhanced or underenhanced.
The integrated color model with the Rayleigh distribution [4]
minimizes overenhanced and underenhanced regions, but it in-
troduces noise in the output results. A retinex-based (RB) ap-
proach [5] is proposed to enhance a single underwater image, and
there are three major steps: a simple and effective color correc-
tion strategy, a variational RB framework, and a postprocessing
on fuzz and underexposure. This approach reduces underwa-
ter blue–green effect effectively and removes amplified noise.
These enhancement-based methods improve underwater scene
contrast and image quality to some extent, but output images in
some scenes become overenhanced or underenhanced, simulta-
neously their methods reckon without the complex underwater
physical parameters.
B. Restoration-Based Methods
In image restoration techniques, the goal is to recover under-
water images by constructing the degradation model and then
estimating model parameters. The dark channel prior method
[23] assumes that, in most local patches of haze-free outdoor
images, there will be some pixels with very low intensities in
at least one color channel. It then uses this assumption to esti-
mate the transmission and restores the image. Intricate under-
water images are similar to hazy images to some extent (e.g.,
backscatter). Therefore, some researchers apply this method to
process underwater images. The underwater dark channel prior
(UDCP) [7] proposes a novel prior based on observing the ab-
sorption rate of red channel in abundant underwater images to
restore high-quality images. However, the UDCP is sensitive to
the variations in an underwater scene. Similarly, the red channel
method [8] recovers degraded images by restoring the colors
associated with short wavelengths. However, many physical pa-
rameters and underwater optical property are required, making
these methods inflexible to be implemented. Owing to a lack
of abundant training data, these methods based on dark channel
prior exhibit poor performance for marine scenarios.
C. Deep-Learning-Based Methods
Relying on abundant training data, deep-learning-based meth-
ods are capable of improving image quality in different under-
water scenes. Combined with the physical model, WaterGAN
[16] uses in-air images with corresponding depth information to
generate the synthetic image for specific underwater scenarios.
Authorized licensed use limited to: Tsinghua University. Downloaded on December 20,2020 at 12:09:37 UTC from IEEE Xplore. Restrictions apply.
863
The literature [21] develops a weak underwater image color cor-
rection model based on the cycle-consistent adversarial network
(CycleGAN) [24] and a multiterm loss function. Considering
that CycleGAN can translate an image from one domain to an-
other domain without paired training data or depth pairings, an
underwater GAN (UGAN) [17] employs it as a degradation pro-
cess to generate paired training data, and then uses the model
based on pix2pix [25] to improve underwater image quality.
Computing the gradient penalty [26] adopted by a UGAN is
more time consuming than the spectral normalization [27].
Differing from a previous application, we propose an effective
block designed for underwater image enhancement with residual
learning, dense concatenation, and multiscale operation, which
would be demonstrated to be effective in ablation study. The
spectral normalization is utilized to stabilize the training of the
discriminator, which has been demonstrated to be computation-
ally light, fast, and easy to incorporate into GAN training [27].
In addition, the proposed network performs well in terms of both
subjective and objective evaluations on up to 215 real underwater
images.
III. METHODOLOGY
GANs [28] have attracted favorable attention in machine
learning community not only for its ability to learn the target
probability distribution but also for its theoretically attractive
aspects. Inspired by GANs, we propose an underwater GAN
(UWGAN) to learn a nonlinear mapping between the nondis-
torted image and the distorted image. The proposed network
generates enhanced results by leveraging on an end-to-end and
data-driven training mechanism. As shown in Fig. 1, the pro-
posed model contains two components, a generator network
G and a discriminator network D. Residual MSDB (RMSDB)
is employed in a fully convolutional network of the generator.
The generator is designed to synthesize the underwater images,
whereas the discriminator is designed to distinguish the synthe-
sized images produced by the generator from the corresponding
real underwater images. We use nonsaturating loss, L1 loss, and
gradient loss to produce visually pleasing images.
A. Generator Network
In recent years, a large number of feature extraction modules
have been designed. The widely used inception architecture [29]
aims to find out an optimal local sparse structure in a network
structure. However, these different scale features concatenate
in a simple fashion at the end of the block, partly leading to
the underutilization of feature maps [30]. Moreover, the litera-
ture [31] proposes a deep residual learning framework to ease the
optimization process of networks. Simultaneously, they could
easily enjoy more competitive results. After that, the dense block
[32] is designed to strengthen feature propagation and encourage
feature reuse.
Inspired by the above-mentioned feature extraction modules,
we propose a novel MSDB. Fig. 2 depicts the detailed struc-
ture of our MSDB. Each concatenation operation has three or
four feature maps to take full advantage of the local features of
the image, one of which directly comes from the output of the
864 IEEE JOURNAL OF OCEANIC ENGINEERING, VOL. 45, NO. 3, JULY 2020

Fig. 1. Architectures of generator and discriminator networks. “Conv” denotes convolution layer, whereas “Deconv” denotes deconvolution layer. MSDB
represents the multiscale dense block and “BN” represents batch normalization. Spectral normalization is used in the convolution layer of the discriminator.

1
F1 = L(ω5×5 ∗ Xn−1 ) (3)
2
T2 = L(ω3×3 ∗ [T1 , F1 , Xn−1 ]) (4)
2
F2 = L(ω5×5 ∗ [T1 , F1 , Xn−1 ]) (5)
3
Xn = L(ω1×1 ∗ [T2 , F2 , O1 , Xn−1 ]) (6)

where ω represents the weights, and the biases are omit-

Fig. 2. Architecture of the MSDB. “Contact” denotes dense concatenation
operation.
previous layer. The two middle paths have different kernel sizes
to detect the feature maps at different scales. The last 1 × 1
convolution can be introduced as the bottleneck layer, which
promotes feature fusion and improves computational efficiency.
The operation can be expressed as follows:
1
O1 = L(ω1×1 ∗ Xn−1 )
1
T1 = L(ω1×1 ∗ Xn−1 )
ted for simplifying notations. The convolution operation is
marked by “∗.” The superscripts denote the location of
the convolutional layer, whereas the subscripts denote the
size of the corresponding convolutional kernel. L(x) denotes
Leaky ReLU (LReLU) activation function [33] and [T1 , F1 ,
Xn−1 ], [T1 , F1 , Xn−1 ], [T2 , F2 , O1 , Xn−1 ] refer to the concate-
nation of the feature maps.
To facilitate the concatenation operation, each layer in an
MSDB uses the convolutional kernel with stride 1. The 1 × 1
convolutional layer in end of the block reduces the feature map
to the number of input channels to the MSDB, thus the input
and output of our block have exactly the same number in terms
of feature maps. The distinctive operation allows multiple MS-
DBs to be connected together. We add skip connection to the
MSDB, which serves as one block and further encourages the
flow of information and gradient. The RMSDB combines two
blocks to gain comparable performance. In Fig. 3, we notice that
more than two blocks improve the performance but introduce too
many parameters and increase the training time. Therefore, the
proposed network takes two blocks as the final version.
In Tables I and II, RMSDB stands for residual MSDB
and BN for batch normalization [34]. The format [filter_h,
filter_w, stride] is kernel size and h × w× channels is output
shape. The slopes of all LReLU activation functions is set to
0.2.
In the first two layers of network, we use two convolutional
layers with 7 × 7 kernels and 64 feature maps (3 × 3 kernels
(1) and 128 feature maps for the second layer) followed by BN
and LReLU activation. The first two layers of convolution can
(2) reduce the feature map size and extract preliminary features. As

Authorized licensed use limited to: Tsinghua University. Downloaded on December 20,2020 at 12:09:37 UTC from IEEE Xplore. Restrictions apply.
GUO et al.: UNDERWATER IMAGE ENHANCEMENT USING A MULTISCALE DENSE GENERATIVE ADVERSARIAL NETWORK
Fig. 3. Enhanced result with the proposed network under the different number of blocks. We obtain the results on the 215 real underwater images and average
them.
TABLE I
GENERATOR NETWORK
TABLE II
DISCRIMINATOR NETWORK
a result, the RMSDB can be connected at the output of the first
two layers and extract more features. Two deconvolution layers
are employed to reconstruct an image. The last deconvolution
layer maps to the number of input channels, which uses a Tanh
function to match the input distribution of [−1, 1].
B. Discriminator Network
The proposed discriminator network consists of five layers
with spectral normalization [27], as shown in Fig. 1, similar to
the work of 70 × 70 PatchGAN. As depicted in Table II, BN is
not applied to the first layer and the last layer. All remaining con-
volutional layers follow the same basic design, a convolution-
BN–LReLU layer. PatchGAN is first used in pix2pix [25] and
then extends to apply in later CycleGAN [24]. Such a Patch-
GAN that has fewer parameters than a full-image discrimina-
tor can handle arbitrarily sized images in a fully convolutional
Authorized licensed use limited to: Tsinghua University. Downloaded on December 20,2020 at 12:09:37 UTC from IEEE Xplore. Restrictions apply.
865
Fig. 4. The loss curve of discriminator. “−SN” represents the discriminator
without spectral normalization.
manner [25]. Spectral normalization restricts the Lipschitz con-
stant of the discriminator to stabilize the training of the discrim-
inator [27]. In addition, this procedure is computationally light
and easy to implement. As shown in Fig. 4, compared to the
discriminator without spectral normalization, the discriminator
with spectral normalization owns a stable and falling curve.
C. GAN Objective Function
The proposed generator produces an image to fool the discrim-
inator, which is designed to distinguish between synthesized and
real-world underwater images. We let x be an in-air image and y
be the same image with degradation. The proposed loss function
includes nonsaturating GAN loss, L1 loss, and gradient loss as
follows:
LUWGAN = min max V (D, G) + λ1 LL1 (G) + λg Lg (G). (7)
G D
The nonsaturating GAN loss can be expressed as follows:
min max V (D, G) = Ex∼ptrain (x) [log D(x)]
G D
+ Ey∼pgen (y) [log(1 − D(G(y)))] (8)
866
Fig. 5. Qualitative comparisons for sample results on real underwater images.
where D(x) represents the probability that x comes from the real
underwater images rather than the output of generator G(y). It is
well known that the nonsaturating loss outperforms the minimax
variant [35]. λ1 and λg are the weight of L1 distance and gradient
loss, respectively.
To give results some sense of ground truth and stabilize train-
ing process, we explore this option by using L1 distance and
gradient loss as follows:
LL1 (G) = E[||x − G(y)||1 ]
Lg (G) = E[||∇(x) − ∇(G(y))||1 ].
IV. EXPERIMENTS
In this section, we first discuss the detail setup of the pro-
posed network. We then show the performance of the proposed
method by comparing it with other nondeep and deep learning
methods on both synthetic and real underwater images. Finally,
an ablation study and application tests further demonstrate the
superiority of the proposed method.
A. Setup
1) Data Set: The proposed method is conducted in a paired
system by using the training data from the literature [17]. UGAN
divides subsets of Imagenet [36] containing underwater images
into two categories based on subjective vision. Let x be the set of
underwater images with no distortion, and y be the set of under-
water images with distortion. CycleGAN can learn the mapping
Authorized licensed use limited to: Tsinghua University. Downloaded on December 20,2020 at 12:09:37 UTC from IEEE Xplore. Restrictions apply.
IEEE JOURNAL OF OCEANIC ENGINEERING, VOL. 45, NO. 3, JULY 2020
function f : x → y as well as g : y → x. Finally, 6128 image
pairs for training data are generated by degenerating in-air im-
ages in x with f . Simultaneously, CycleGAN learns a mapping
g : y → x, which is similar to image enhancement and is used
as a comparison method. We select 119 real underwater images
from the related papers and seafood breeding bases in Zhangzi-
dao, China, and select 96 images from Imagenet [36] and SUN
[37]. The test set contains a total of 215 real underwater images.
2) Training Details: In our training process, training and
(9)
test images have dimensions 256 × 256 × 3 and are normalized
(10) between [−1, 1]. We use λ1 = 60, λg = 10, and LReLU with
a slope of 0.2 and the Adam algorithm [38] with the learning
rate of 0.0001. The batch size is set to 32. The discriminator
updates five times per generator update. The entire network was
trained on a GTX 1070 Ti using the TensorFlow framework for
60 epochs.
3) Compared Methods: We compare the proposed model
with other enhancement methods on both synthesized and real-
world underwater images. These competitive methods include
FusionEnhance (FE) [3], RB [5], UDCP [7], CycleGAN [24],
weakly supervised color transfer (WSCT) [21], and UGAN [17].
B. Real-World Underwater Enhancement
We first evaluate the proposed method on test set with nondeep
and deep learning methods. As shown in Fig. 5, FE has obvious
reddish color shift due to the inaccurate color correction algo-
rithm. RB can enhance underwater image quality meanwhile
GUO et al.: UNDERWATER IMAGE ENHANCEMENT USING A MULTISCALE DENSE GENERATIVE ADVERSARIAL NETWORK 867

Fig. 6. Qualitative comparisons of different enhancement methods on starfish image.

TABLE III
UNDERWATER IMAGE QUALITY EVALUATION OF DIFFERENT ENHANCEMENT METHODS ON REAL-WORLD UNDERWATER IMAGES

generates some darkish images. We note that UDCP aggravates
the blue–green effect. CycleGAN has the limited positive effect
on images because the image-to-image translation is not well
suited for underwater images enhancement. WSCT introduces
greenish tone in some patches of underwater images partly be-
cause of the lack of techniques to stabilize GAN training. For
example, in the first image of WSCT, the background intro-
duces green color deviation. From Fig. 6, the color of starfish is
not clear enough. Compared with other methods, the proposed
method not only restores visually appealing results in underwa-
ter scenarios but also enhances the underwater images even in
situations where other methods fail.
To make our results more convincing, we employ two no-
reference metrics to evaluate the underwater images, e.g., un-
derwater color image quality evaluation (UCIQE) [39] and un-
derwater image quality measure (UIQM) [40]. The UCIQE for a
quantitative evaluation utilizes a linear combination of chroma,
saturation, and contrast, which aims to quantify the nonuniform
color cast, blurring, and low contrast, respectively. Similarly,
the UIQM comprises three properties of underwater images,
such as the underwater image colorfulness measure (UICM), the
underwater image sharpness measure (UISM), and the underwa-
ter image contrast measure (UIConM). Higher values of UCIQE
and UIQM denote better images quality.
In our assessment experiments, we employ UCIQE and UIQM
to evaluate the test set. Table III lists the average values ob-
tained by different methods on 215 images. The best results of
UCIQE and UIQM metrics are marked in bold. The number
following the mean denotes the variance. FE has obvious red-
dish color shift due to the inaccurate color correction algorithm,
causing the decreased value of UICM. Therefore, FE ranks the
fifth under UIQM. The CycleGAN is trained for 50 000 itera-
tions and 100 000 iterations with no added benefit, because the
image-to-image translation with a cycle consistency loss is not
well suited for the underwater scenario. UGAN utilizes many
convolution layers with up to 512 kernels to enhance under-
water images, resulting in too many network parameters. The
proposed method combined with the effective block achieves
higher metrics while using fewer parameters than UGAN. It can
be seen that the UIQM of the proposed method is larger than
the other methods, the UCIQE is also larger than most methods,
and the variance is smaller than most methods.

Authorized licensed use limited to: Tsinghua University. Downloaded on December 20,2020 at 12:09:37 UTC from IEEE Xplore. Restrictions apply.
868
Fig. 7. Qualitative comparisons of different enhancement methods on synthetic underwater images.
TABLE IV
UNDERWATER IMAGE QUALITY EVALUATION OF DIFFERENT ENHANCEMENT
METHODS ON THE SYNTHETIC UNDERWATER IMAGES
TABLE V
UNDERWATER IMAGE QUALITY EVALUATION OF DIFFERENT VARIANTS
OF THE PROPOSED METHOD
C. Synthetic Underwater Enhancement
We also evaluate the proposed method on 60 synthesized im-
ages, which come from WaterGAN obtained using the MHL
data sets [16]. As shown in Fig. 7, FE and RB introduce
reddish color shift meanwhile UDCP fails to correct the under-
water color. CycleGAN and WSCT have a greenish tone in some
Authorized licensed use limited to: Tsinghua University. Downloaded on December 20,2020 at 12:09:37 UTC from IEEE Xplore. Restrictions apply.
IEEE JOURNAL OF OCEANIC ENGINEERING, VOL. 45, NO. 3, JULY 2020
Fig. 8. The examples of artifacts. “−DC” represents the UWGAN without
dense concatenation operation.
patches of underwater images. Compared with UGAN, such as
cabinet and the color of wall, the proposed method renders more
details and reduces color shifts.
Considered the underwater environment, we also employ
UCIQE and UIQM metrics to evaluate synthesized underwa-
ter images. In Table IV, the average of the results is listed, and
the values in bold denote the results that surpass all competing
methods. The number following the mean denotes the variance.
It can be seen that the UCIQE score of the proposed method
is higher than the other methods, the UIQM score is relatively
high, and the variance is smaller than the other methods.
D. Ablation Study and Application Tests
1) Ablation Study: The ablation study aims to reveal the
effect of each component. We carry out the test in 119 real
GUO et al.: UNDERWATER IMAGE ENHANCEMENT USING A MULTISCALE DENSE GENERATIVE ADVERSARIAL NETWORK 869

Fig. 9. Keypoint matching test on original underwater images and enhanced images.

Fig. 10. Canny edge detection on original underwater images and enhanced images.

underwater images considering that these methods are variants
of the proposed method and the following statements hold:
1) UWGAN removes residual learning operation (−RL);
2) UWGAN removes dense concatenation operation (−DC);
3) UWGAN removes multiscale operation (−MS).
The best results of UCIQE and UIQM metrics are marked
in bold. In Table V, we notice that both residual learning and
multiscale operation could improve the UCIQE and UIQM of
the underwater images. Compared with the proposed network
without dense concatenation, we take several dense concatena-
tion operations in the specified convolutional layer to improve
the UICM and UIConM. As seen in Fig. 8, the UWGAN with
dense concatenation operation removes unpleasant artifacts at
the cost of the decreased performance of UCIQE, which aims at
better subjective perception.
2) Application Tests: Some feature-related algorithms, in-
cluding SIFT [41] and Canny [42], are employed to further
demonstrate the effectiveness of the proposed methods. As
shown in Figs. 9 and 10, the enhanced images render more key
matching points and more edge detection feature than the orig-
inal underwater images.
V. CONCLUSION
In this paper, we present a method for underwater image
enhancement through GAN. Meanwhile, the proposed MSDB
combined with residual learning can improve network perfor-
mance, the multiterm loss function can produce visually pleas-
ing enhanced results. Numerous experiments are performed to
demonstrate the superiority of the proposed method both on
synthetic and real underwater images. In addition, the ablation
study is carried out to show the contributions of each component,
and the application tests further demonstrate the effectiveness of
the proposed method.
We find that our model cannot generate aesthetically pleasing
synthesized underwater images. We leave this part in our future
work.
ACKNOWLEDGMENT
The authors would like to thank the anonymous reviewers for
their constructive and valuable comments. They also would like
to thank C. Li for providing help. Finally, the authors would like
to thank J. Li, K. A. Skinner, and J. Zhang.
REFERENCES
[1] D. M. Kocak, F. R. Dalgleish, F. M. Caimi, and Y. Y. Schechner, “A focus
on recent developments and trends in underwater imaging,” Mar. Technol.
Soc. J., vol. 42, no. 1, pp. 52–67, 2008.
[2] H. Lu, Y. Li, and S. Serikawa, “Underwater image enhancement using
guided trigonometric bilateral filter and fast automatic color correction,”
in Proc. IEEE Int. Conf. Image Process., 2013, pp. 3412–3416.

Authorized licensed use limited to: Tsinghua University. Downloaded on December 20,2020 at 12:09:37 UTC from IEEE Xplore. Restrictions apply.
870
[3] C. Ancuti, C. O. Ancuti, T. Haber, and P. Bekaert, “Enhancing underwater
images and videos by fusion,” in Proc. IEEE Conf. Comput. Vision Pattern
Recognit., 2012, pp. 81–88.
[4] A. S. A. Ghani and N. A. M. Isa, “Underwater image quality enhancement
through integrated color model with Rayleigh distribution,” Appl. Soft
Comput., vol. 27, pp. 219–230, 2015.
[5] X. Fu, P. Zhuang, Y. Huang, Y. Liao, X.-P. Zhang, and X. Ding, “A retinex-
based enhancing approach for single underwater image,” in Proc. IEEE
Int. Conf. Image Process., 2014, pp. 4572–4576.
[6] C.-Y. Li, J.-C. Guo, R.-M. Cong, Y.-W. Pang, and B. Wang, “Underwater
image enhancement by dehazing with minimum information loss and his-
togram distribution prior,” IEEE Trans. Image Process., vol. 25, no. 12,
pp. 5664–5677, Dec. 2016.
[7] P. L. Drews, E. R. Nascimento, S. S. Botelho, and M. F. M. Campos, “Un-
derwater depth estimation and image restoration based on single images,”
IEEE Comput. Graph. Appl., vol. 36, no. 2, pp. 24–35, Mar./Apr. 2016.
[8] A. Galdran, D. Pardo, A. Picon, and A. Alvarez-Gila, “Automatic red-
channel underwater image restoration,” J. Vis. Commun. Image Represent.,
vol. 26, pp. 132–145, 2015.
[9] N. Wang, H. Zheng, and B. Zheng, “Underwater image restoration via
maximum attenuation identification,” IEEE Access, vol. 5, pp. 18941–
18952, 2017.
[10] Y. Wang, H. Liu, and L.-P. Chau, “Single underwater image restoration
using adaptive attenuation-curve prior,” IEEE Trans. Circuits Syst. I, Reg.
Papers vol. 65, no. 3, pp. 992–1002, Mar. 2018.
[11] Y.-T. Peng and P. C. Cosman, “Underwater image restoration based on im-
age blurriness and light absorption,” IEEE Trans. Image Process., vol. 26,
no. 4, pp. 1579–1594, Apr. 2017.
[12] C. Dong, C. C. Loy, K. He, and X. Tang, “Learning a deep convolutional
network for image super-resolution,” in Proc. Eur. Conf. Comput. Vision,
2014, pp. 184–199.
[13] X. Fu, J. Huang, D. Zeng, Y. Huang, X. Ding, and J. Paisley, “Removing
rain from single images via a deep detail network,” in Proc. IEEE Conf.
Comput. Vision Pattern Recognit., 2017, pp. 3855–3863.
[14] K. Zhang, W. Zuo, Y. Chen, D. Meng, and L. Zhang, “Beyond a Gaussian
denoiser: Residual learning of deep CNN for image denoising,” IEEE
Trans. Image Process., vol. 26, no. 7, pp. 3142–3155, Jul. 2017.
[15] S. Anwar, C. Li, and F. Porikli, “Deep underwater image enhancement,”
2018. [Online]. Available: https://arxiv.org/abs/1807.03528
[16] J. Li, K. A. Skinner, R. M. Eustice, and M. Johnson-Roberson, “Water-
GAN: Unsupervised generative network to enable real-time color correc-
tion of monocular underwater images,” IEEE Robot. Autom. Lett., vol. 3,
no. 1, pp. 387–394, Jan. 2018.
[17] C. Fabbri, M. J. Islam, and J. Sattar, “Enhancing underwater im-
agery using generative adversarial networks,” 2018. [Online]. Available:
https://arxiv.org/abs/1801.04011
[18] Y.-S. Shin, Y. Cho, G. Pandey, and A. Kim, “Estimation of ambient light
and transmission map with common convolutional architecture,” in Proc.
MTS/IEEE OCEANS Conf., Monterey, CA, USA, 2016, pp. 1–7.
[19] X. Yu, Y. Qu, and M. Hong, “Underwater-GAN: Underwater image
restoration via conditional generative adversarial network,” in Proc. Int.
Conf. Pattern Recognit., 2018, pp. 66–75.
[20] Y. Hu, K. Wang, X. Zhao, H. Wang, and Y. Li, “Underwater image restora-
tion based on convolutional neural network,” in Proc. Asian Conf. Mach.
Learn., 2018, pp. 296–311.
[21] C. Li, J. Guo, and C. Guo, “Emerging from water: Underwater image
color correction based on weakly supervised color transfer,” IEEE Signal
Process. Lett., vol. 25, no. 3, pp. 323–327, Mar. 2018.
[22] X. Chen, J. Yu, S. Kong, Z. Wu, X. Fang, and L. Wen, “Towards quality
advancement of underwater machine vision with generative adversarial
networks,” 2017. [Online]. Available: https://arxiv.org/abs/1712.00736
[23] K. He, J. Sun, and X. Tang, “Single image haze removal using dark channel
prior,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 33, no. 12, pp. 2341–
2353, Dec. 2011.
[24] J.-Y. Zhu, T. Park, P. Isola, and A. A. Efros, “Unpaired image-to-image
translation using cycle-consistent adversarial networks,” in Proc. IEEE
Conf. Comput. Vision Pattern Recognit., 2017, pp. 2223–2232.
[25] P. Isola, J.-Y. Zhu, T. Zhou, and A. A. Efros, “Image-to-image transla-
tion with conditional adversarial networks,” 2017. [Online]. Available:
https://arxiv.org/abs/1611.07004
[26] I. Gulrajani, F. Ahmed, M. Arjovsky, V. Dumoulin, and A. C. Courville,
“Improved training of Wasserstein GANs,” in Proc. Adv. Neural Inf. Pro-
cess. Syst., 2017, pp. 5767–5777.
[27] T. Miyato, T. Kataoka, M. Koyama, and Y. Yoshida, “Spectral normaliza-
tion for generative adversarial networks,” 2018, arXiv:1802.05957.
[28] I. Goodfellow et al., “Generative adversarial nets,” in Proc. Adv. Neural
Inf. Process. Syst., 2014, pp. 2672–2680.
Authorized licensed use limited to: Tsinghua University. Downloaded on December 20,2020 at 12:09:37 UTC from IEEE Xplore. Restrictions apply.
View publication stats
IEEE JOURNAL OF OCEANIC ENGINEERING, VOL. 45, NO. 3, JULY 2020
[29] C. Szegedy et al., “Going deeper with convolutions,” in Proc. IEEE Conf.
Comput. Vision Pattern Recognit., 2015, pp. 1–9.
[30] J. Li, F. Fang, K. Mei, and G. Zhang, “Multi-scale residual network for im-
age super-resolution,” in Proc. Eur. Conf. Comput. Vision, 2018, pp. 517–
532.
[31] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image
recognition,” in Proc. IEEE Conf. Comput. Vision Pattern Recognit., 2016,
pp. 770–778.
[32] G. Huang, Z. Liu, L. Van Der Maaten, and K. Q. Weinberger, “Densely
connected convolutional networks,” in Proc. IEEE Conf. Comput. Vision
Pattern Recognit., 2017, pp. 4700–4708.
[33] A. L. Maas, A. Y. Hannun, and A. Y. Ng, “Rectifier nonlinearities improve
neural network acoustic models,” in Proc. Int. Conf. Mach. Learn., vol. 30,
no. 1, pp. 1–6, 2013.
[34] S. Ioffe and C. Szegedy, “Batch normalization: Accelerating deep network
training by reducing internal covariate shift,” in Proc. 32nd Int. Conf.
Mach. Learn., 2015, vol. 37, pp. 448–456.
[35] K. Kurach, M. Lucic, X. Zhai, M. Michalski, and S. Gelly, “The GAN
landscape: Losses, architectures, regularization, and normalization,” 2018.
[Online]. Available: https://arxiv.org/abs/1807.04720
[36] O. Russakovsky et al., “ImageNet large scale visual recognition chal-
lenge,” Int. J. Comput. Vision, vol. 115, no. 3, pp. 211–252, 2015.
[37] J. Xiao, J. Hays, K. A. Ehinger, A. Oliva, and A. Torralba, “Sun database:
Large-scale scene recognition from abbey to zoo,” in Proc. IEEE Comput.
Soc. Conf. Comput. Vision Pattern Recognit., 2010, pp. 3485–3492.
[38] D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,”
2014. [Online]. Available: https://arxiv.org/abs/1412.6980
[39] M. Yang and A. Sowmya, “An underwater color image quality evaluation
metric,” IEEE Trans. Image Process., vol. 24, no. 12, pp. 6062–6071,
Dec. 2015.
[40] K. Panetta, C. Gao, and S. Agaian, “Human-visual-system-inspired un-
derwater image quality measures,” IEEE J. Ocean. Eng., vol. 41, no. 3,
pp. 541–551, Jul. 2016.
[41] D. G. Lowe, “Distinctive image features from scale-invariant keypoints,”
Int. J. Comput. Vision, vol. 60, no. 2, pp. 91–110, 2004.
[42] J. Canny, “A computational approach to edge detection,” IEEE Trans.
Pattern Anal. Mach. Intell., vol. PAMI-8, no. 6, pp. 679–698, Nov. 1986.
Yecai Guo received the Ph.D. degree in underwater
acoustic engineering from Northwestern Polytechni-
cal University, Xi’an, China, in 2003.
He is currently a Professor with Nanjing Univer-
sity of Information Science and Technology, Nanjing,
China. His research interests include meteorological
telecommunication technology, underwater commu-
nication theory, and their applications.
Dr. Guo was a national winner of National Out-
standing Doctoral Dissertations in 2006, the train-
ing object of “six talents peak” in Jiangsu province
in 2008, and one of the leaders of the advantage discipline “sensor network
and modern meteorological equipment” of colleges and universities of Jiangsu
province in 2009, respectively.
Hanyu Li received the B.S. degree in the department
of automation from Nanhang Jincheng College, Nan-
jing, China, in 2017. He is currently working toward
the M.S. degree in image processing and deep learn-
ing with the School of Electronic and Information
Engineering, Nanjing University of Information Sci-
ence and Technology, Nanjing, China.
His research interests include generative adversar-
ial network and underwater image processing.
Peixian Zhuang received the Ph.D. degree in sparse
representation, compressed sensing, and Bayesian
machine learning from Xiamen University, Xiamen,
China, in 2016.
He is currently a Lecturer with the School of Elec-
tronic and Information Engineering, Nanjing Univer-
sity of Information Science and Technology, Nanjing,
China. His research interests include image process-
ing and Bayesian machine learning.
Dr. Zhuang was the recipient of the Best Ph.D.
Thesis Award in Fujian Province in 2017.