Received September 2, 2019, accepted September 21, 2019, date of publication September 24, 2019,

date of current version October 8, 2019.

Digital Object Identifier 10.1109/ACCESS.2019.2943516

View-Consistent Intrinsic Decomposition

for Stereoscopic Images
DEHUA XIE , (Student Member, IEEE), SHUAICHENG LIU , (Member, IEEE),
YINGLONG WANG , (Student Member, IEEE), SHUYUAN ZHU , (Member, IEEE),
AND BING ZENG , (Fellow, IEEE)
Institute of Image Processing, University of Electronic Science and Technology of China, Chengdu 611731, China
Corresponding author: Bing Zeng (eezeng@uestc.edu.cn)
This work was supported in part by the National Natural Science Foundation of China under Grant 61872067, Grant 61502079, Grant
61672134, and Grant 61720106004, and in part by the ‘‘111’’ Project under Grant B17008.

ABSTRACT In this paper, we focus on the intrinsic image decomposition problem for stereoscopic image
pairs. The existing methods cannot be applied directly to decompose stereoscopic images, as it often produces
inconsistent reflectance (albedo) and 3D artifacts after the decomposition. We propose a straightforward yet
effective framework that enables a high-quality decomposition for stereoscopic pairs. First, retinex-based
constraints are employed to coarsely classify the observed image gradients into two categories that are caused
by reflectance changes and illumination variations, respectively. Second, reflectance-consistent constraints
are added to control the reflectance consistency between the left and right views. Since this problem is highly
ill-posed, we further analyze local and non-local image textures regularized by super-pixels within and across
two views to reduce reflectance ambiguity. Lastly, absolute-scale constraints are employed to normalize
the decomposition results. Extensive experiments on the real-world stereoscopic images and synthetic
stereoscopic images reveal that our method can readily achieve high-quality decomposition performance.

INDEX TERMS Intrinsic image decomposition, reflectance, shading, stereoscopic image.

I. INTRODUCTION the presence of poor lighting conditions. On the other hand,

From the perspective of image formation, several key factors,
such as surface material and textures, lighting conditions,
shape of objects, and viewing angle, jointly determine the
final appearance of the scene. These factors are referred to
as intrinsic properties [1] and are vital to image formation
process. Among these properties, surface material and tex-
tures are usually characterized by reflectance image, while
the composite product of lighting conditions and shape of
objects are described by shading image. Reflectance image
and shading image are also called as intrinsic images.
Intrinsic images play a critical role in image analysis
and processing. For instance, image recoloring [2] and
image re-texturing [3] tasks prefer a pure reflectance image
without intensity variations caused by non-uniform lighting
conditions and non-flat surface shape. Similarly, shape-from-
shading algorithms infer more accurate surface geometry
given a shading image with less texture residuals [4]. Seg-
mentation and face recognition [5] often gets degraded in
The associate editor coordinating the review of this manuscript and
approving it for publication was Sudhakar Radhakrishnan
high-quality intrinsic decomposition can increase the perfor-
mances of such vision problems. Therefore, intrinsic decom-
position has attracted increasing interests in both academic
and industrial communities [6]–[18].
Given an observed image I, it can be decomposed into a
reflectance image R multiplying a shading image S pixel-
by-pixel. Without further constraints, the decomposition is
a highly ill-posed problem in that the number of unknown
variables (R, S) are twice the known variables I. Much
progress has been made in intrinsic image decomposition
during the past decades. Land and McCann first introduced
the retinex theory in 1971 [19], which was extended to two
dimensional images by Horn [20], [21]. Since the retinex
theory provides a computational approach of coarsely classi-
fying the observed image gradients into reflectance gradients
and shading gradients, it thereafter becomes a fundamental
constraint to decompose intrinsic images.
Grosse et al. generated a ground-truth dataset and pro-
vided a comprehensive review [22]. From then on, a lot of
methods, such as optimization method [23], [24], statistical
. inference method [25], and dense CRF-based method [11]

This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see http://creativecommons.org/licenses/by/4.0/
VOLUME 7, 2019 140355

FIGURE 1. A stereoscopic image recoloring example based on our proposed stereoscopic intrinsic image decomposition. (a) Left view. (b) Right view.
(c) Left recoloring result. (d) Right recoloring result.
were put forward to improve the performance. Meanwhile,
many image cues, including textures [26], [27], color con-
stancy [28], [29], and reflectance sparsity [30], are utilized as
constraints to regularize the decomposition. More recently,
Luo et al. presented a novel optimization that combines pho-
tometric stereo and retinex constraints into the optimization,
which achieves some state-of-the-art results according to
MIT intrinsic image dataset [31]. However, all the above men-
tioned methods cannot be applied directly to stereoscopics
because they are designed for monocular images that usually
take no account of the consistency across different views.
In recent years, lots of topics related to stereoscopic images
and videos have been explored, such as stereoscopic movie
making [32], stereoscopic warping [33]–[35], cloning [36],
inpainting [37], stitching [38], [39], retargeting [40], edit-
ing [41]–[43], stabilization [44], [45], relighting [46], and so
on. However, few work has targeted the problem of stereo-
scopic intrinsic image decomposition [47].
Stereoscopic images generally possess a large overlapped
region between two views. On one hand, the intrinsic image
decomposition for a stereoscopic image pair needs to take
the reflectance consistency of the overlapped region into
consideration. On the other hand, inspired by the work of
Luo et al. [31], stereoscopic images provide the possibility
that two views could offer more constraints to reduce the
ambiguity during the decomposition. Fig. 1 shows a stereo-
scopic image recoloring example, which is produced on the
basis of our stereoscopic intrinsic decomposition.
In this work, we propose a stereoscopic intrinsic image
decomposition framework. Firstly, the retinex theory is
employed as soft constraints to coarsely classify stereoscopic
image gradients into two classes: reflectance gradients and
shading gradients. Secondly, reflectance-consistency con-
straints are considered for the matched pixels across two
views to facilitate a view-consistent decomposition. Thirdly,
we add local and non-local constraints within and across
views for an improved regularization. Finally, absolute-scale
constraint is added to normalize the decomposition results
into the visualization range. Optimizing all the above con-
straints in a unified energy can produce high-quality and
view-consistent decomposition results. Extensive experi-
ments on real-world stereoscopic images and synthetic
140356
D. Xie et al.: View-Consistent Intrinsic Decomposition for Stereoscopic Images
stereoscopic images will be conducted to validate the effec-
tiveness of our method both qualitatively and quantitatively.
To sum up, the main contributions of our paper are as
follows.
• A unified framework is proposed to enable the intrinsic
image decomposition for stereoscopic images.
• A stereoscopic dataset is presented, which includes both
real-world stereoscopic images and synthetic stereo-
scopic images.
The rest of the paper is organized as follows. Section II
reviews some typical intrinsic image decomposition algo-
rithms for monocular image and related works on
stereoscopic image/video. In addition, intrinsic image
decomposition datasets are also summarized in this section.
Section III describes our proposed stereoscopic intrinsic
image decomposition framework that includes the retinex
term, the reflectance-consistent term, the local and non-local
term, and the absolute-scale term. Extensive experiments on
both real-world stereoscopic images and synthetic stereo-
scopic images are carried out in Section IV. Finally, Section V
concludes the paper, together with discussion on the limita-
tions and future works.
II. RELATED WORKS
A. INTRINSIC IMAGE DECOMPOSITION
As early as in 1964, Land conducted a series of color
constancy experiments on the ‘‘Mondrian World’’ patterns
and proposed the retinex theory [48]. In 1974, Land’s
retinex theory was extended into two dimensional images by
Horn [20], [21], who was able to decompose an
‘‘Mondrian World’’ pattern into the reflectance component
and the shading component. In 1978, Barrow and Tenebaum
firstly put forward the concept of intrinsic images [1]. Here-
after, decomposing an observed image into the reflectance
image and the shading image has become a longstand-
ing research topic. Based on the retinex theory, Horn pro-
posed the classic retinex intrinsic image decomposition
algorithm. Through selecting a threshold to separate the
intensity gradient into the reflectance gradient and the shad-
ing gradient, the reflectance image and the shading image
can be reconstructed by solving a linear Poisson equation.
VOLUME 7, 2019
D. Xie et al.: View-Consistent Intrinsic Decomposition for Stereoscopic Images
However, the retinex assumption is only met for simple cases.
In real world, the scene captured by cameras and illumination
conditions are far more complex. Under this circumstance,
it is hard to obtain sound intrinsic decomposition results by
the classic retinex algorithm. Besides, choosing a suitable
threshold to separate the intensity gradient is a non-trial task.
Therefore, various methods have been proposed to improve
the classic retinex algorithm. On one hand, due to unpleasant
lighting conditions, many photographs have serious shadow
regions, which greatly degrade the performance of the classic
retinex algorithm. To overcome this problem, Some works
presented a new intrinsic image decomposition algorithm
by adopting image sequences [6] or videos [2], [49]. While
taking image sequences or videos as input is in favor of elim-
inating shadows, it is unfavorable to obtain image sequences
or videos.
On the other hand, since it is difficult for the classic
retinex method to choose a threshold to properly tell the
reflectance gradient from the shading gradient among inten-
sity gradients, a lot of learning based methods have been
presented to address this problem. In 2005, Tappen et al.
trained a classifier to distinguish the reflectance gradi-
ent from the shading gradient [7]. More recently, many
works have been reported to improve the quality of
intrinsic image decomposition based on the deep learning
technique [13], [14], [17], [18], [50]–[52]. Nevertheless, one
drawback of deep learning based intrinsic image decompo-
sition methods is short of enough ground-truth data for the
model training and verification.
Meanwhile, various image cues and priors have been used
to better constrain this problem. In 2009, Bousseau et al. pro-
posed an interactive way to target the intrinsic image decom-
position [8]. Barron and Malik proposed a new method based
on color constancy and constraints on reflectance and shading
to decompose a single object [10], [29]. Later on, they pre-
sented an SIRFS engine to inference the shape, illumination
and reflectance from shading by a single image [25], [53].
Concurrently, Shen et al. utilized reflectance sparsity char-
acteristics [9], [26], [30] to add constraints on reflectance
and achieved more pleasant results. Shen et al. also pre-
sented several new methods based on constraints and energy
minimization [23], [24].
In addition, there are some works that combine some
related tasks in computer vision with the intrinsic image
decomposition problem. For instance, in 2015, Bi et al. pro-
posed an pipeline that combines the edge-preserving flatten
task and the intrinsic image decomposition problem [12].
At the same time, Luo et al. come up with a photometric
stereo based intrinsic image decomposition method to obtain
the accurate surface normal and the reflectance. In 2016,
Kim et al. presented a unified framework that simultane-
ously predicts the depth information and recovers intrinsic
images from a single image [54]. These aforementioned
works mainly focused on the intrinsic image decomposi-
tion on the single object images or the laboratorial images.
In 2014, Bell et al. presented a dense CRF-based intrinsic
VOLUME 7, 2019
image decomposition algorithm that targets the real world
images [11].
The most related work to ours is Zhao’s and Kang’s
works [27], [55]. In 2012, Zhao et al. presented a closed-form
solution to the intrinsic image decomposition for a single
image. More specifically, this work presents a unified frame-
work that can be transformed into a quadratic form objec-
tive function and obtains a closed-form solution. However,
this method can not be extended directly to intrinsic image
decomposition for stereoscopic images. In 2014, Kang et al.
investigated image recoloring for stereoscopic images [55].
They presented a solution to stereoscopic image recoloring
on the basis of intrinsic image decomposition. Nevertheless,
Kang’s method is quite time-consuming for users to fine tune
the parameters. In addition, they only focused on the task of
image recoloring rather than the intrinsic image decomposi-
tion for stereoscopic images.
B. STEREOSCOPIC IMAGE/VIDEO
Stereoscopic image/video is becoming an emerging and inter-
esting topic with the success of stereoscopic devices and 3D
movies. A lot of works related to stereoscopic image or video
have been explored. For instance, image warping is one of the
most basic manipulations in image processing, which has also
been studied extensively on stereoscopic images [33], [35].
In 2008, Wang et al. explored stereoscopic inpainting with
color and depth joint completion [37]. Similarly, image
stitching for stereoscopic image has also been utilized to
extend the field of view [39]. In 2012, Lee et al. presented
an image retargeting method for stereoscopic image [40].
Furthermore, stereoscopic image editing has been widely
studied, such as stereoscopic image composition and stereo-
scopic image content replacement [41], [43]. Meanwhile,
researchers also studied the stabilization problem for stereo-
scopic video [39], [44], [45]. However, few work has focused
on the intrinsic image decomposition for stereoscopic images
or videos. Without the separation of reflectance and shading
from image, it is hard to consistently manipulate two views
of stereoscopic images or videos. To this end, we need to
spend more efforts on the intrinsic image decomposition for
stereoscopic images or videos in that intrinsic images usually
facilitate all kinds of image processing tasks.
C. INTRINSIC IMAGE DECOMPOSITION DATASETS
Although the intrinsic image decomposition has a history
of several decades, efforts on building the dataset for this
problem does not make too much progress. The earliest and
most influential dataset was established by Grosse et al. [22].
In this work, Grosse and his co-authors employed experimen-
tal method to generate 20 classes of reflectance and shading
ground-truth, which is called as MIT intrinsic image dataset.
There is no doubt that MIT intrinsic image dataset has greatly
promoted the development of intrinsic image decomposition.
Another important intrinsic image decomposition dataset
is developed by Bell et al. [11]. This dataset is commonly
referred to IIW dataset. Through crowdsourcing, they built
140357
 D. Xie et al.: View-Consistent Intrinsic Decomposition for Stereoscopic Images

5230 sparse reflectance images ground-truth for the real
world scenes. This dataset is conductive to the advancement
of intrinsic image decomposition, and especially critical to
deep learning based intrinsic image decomposition methods.
Due to the difficulty to obtain the reflectance and the
shading ground-truth from an observed image, especially for
the real world captures, to the best of our knowledge, there
is no available intrinsic image decomposition ground-truth
dataset for stereoscopic images or videos. In this paper,
we make efforts to build a stereoscopic dataset that contains
both real-world and synthetic stereoscopic images to address
the intrinsic image decomposition problem for stereoscopic
images.
III. OUR PROPOSED METHOD
λr , λc , λl , λnl , and λa are constraints weights. The above men-
tioned terms contain constraints imposed on the reflectance
component or the shading component. Each of these terms
will be elaborated in detail one by one in the following
subsections. The parameter settings for λr , λc , λl , λnl , and λa
will be given in Section IV.
B. RETINEX CONSTRAINTS
According to the retinex theory, larger gradients in an image
are usually attributed to the changes in reflectance, while
smaller gradients are caused by shading variations. Similar
to the work [27], the retinex term Er (sl,r ) is defined as
XX X
n 2
Er (sl,r ) = [(sm
l,r − sl,r )
l,r m n∈N (m)

The formation model of a stereoscopic image pair can be + ω(m, n)(rl,r

m n 2
− rl,r ) ], (4)
described by the following equation:
where m stands for an arbitrary pixel in one view, N (m)
Il,r (x, y) = Rl,r (x, y) ∗ Sl,r (x, y). (1) denotes the four adjacent neighbor of the pixel m, and ω(m, n)
is the balancing weight that is used to determine the main
where (x, y) indexes the coordinate of a given pixel; Il,r rep-
factor to give rise to the image gradient. There are many
resents the left and right views of a stereoscopic image pair,
different ways to judge whether reflectance change or shading
Rl,r denotes the left and right reflectances, and Sl,r means
variation is the main factor that leads to the image gradi-
the left and right shadings (assumed to be monochrome),
ent [11], [22], [27]. Here, we adopt the chromaticity distance
respectively. It is worth to notice that Il,r (x, y) and Rl,r (x, y)
of the neighboring pixels as the numerical metric to deter-
are a triple that contains the red, green, and blue channels,
mine the main factor of the image gradient. The balancing
and Sl,r (x, y) is also a triple for three channels. The opera-
weight ω(m, n) is defined as follows
tor ‘‘*’’ denotes the pixel-wised multiplication. From Equ. 1, (
we know that a pixel’s intensity Il,r (x, y) equals its reflectance 0 D(m, n) > τ,
Rl,r (x, y) multiplying its shading Sl,r (x, y). ω(m, n) = (5)
100 D(m, n) < τ,
A. PROBLEM FORMULATION where τ is a threshold and will be discussed in Section IV.
In order to distinguish image gradients into the reflectance D(m, n) is the chromaticity distance between the pixel m and
gradients and the shading gradients more conveniently, the pixel n, and defined as follows:
we first take the logarithm on both sides of Equ. 1. Then, 
hm, ni

Equ. 1 is transformed into the following equation: D(m, n) = 2 × 1 − . (6)
kmk2 · knk2
il,r (x, y) = rl,r (x, y) + sl,r (x, y), (2) In Equ. 6, m and n denote the chromaticity values of pixel
where the logarithm of Il,r (x, y), Rl,r (x, y), and Sl,r (x, y) are m and pixel n, respectively. The calculation of chromaticity
denoted by il,r (x, y), rl,r (x, y), and sl,r (x, y), respectively. value of a given pixel can be found in [11]. hm, ni is the inner
From Equ. 2, we know that the reflectance rl,r is equivalent product of m and n, and kmk2 and knk2 represent the 2-norms
to il,r − sl,r . In the rest of this Section, we use il,r − sl,r to of m and n, respectively.
represent the reflectance component rl,r . In Equ. 5, when the chromaticity distance of neighboring
Since the intrinsic decomposition is a highly ill-posed pixel m and n is larger than τ , ω(m, n) is assigned with 0; oth-
problem, it is hard to separate a stereoscopic image pair il,r erwise, ω(m, n) is assigned with 100. Combined with Equ. 4,
into the reflectance rl,r and the shading sl,r only on the basis some facts can be figured out that larger chromaticity dis-
of Equ. 1. To solve this problem, we propose an algorithm that tance between the neighboring pixels allows the reflectance
decomposes a stereoscopic image pair into the reflectance changes, while shading variations mainly bear the image
component and the shading component by optimizing the gradient under smaller chromaticity distance between the
following energy with respect to the shading component sl,r : neighboring pixels.
Because rl,r = il,r − sl,r , Equ. 4 can be reformulated into:
arg min E(sl,r ) = λr Er (sl,r ) + λc Ec (sl,r ) XX X h
sl,r
Er (sl,r ) = (1 + ω(m, n))(1sm,nl,r )
2
+ λl El (sl,r )+λnl Enl (sl,r )+λa Ea (sl,r ). (3) l,r m n∈N (m)
i
Here, Er (sl,r ), Ec (sl,r ), El (sl,r ), Enl (sl,r ), and Ea (sl,r ) are + (1im,n m,n m,n
l,r ) − 2ω(m, n)1il,r 1sl,r ,
2
(7)
the retinex term, reflectance-consistent term, local term,
non-local term, and absolute scale term, respectively. where 1sm,n n m,n
l,r = sl,r − sl,r and 1il,r = il,r − il,r .
m m n

140358 VOLUME 7, 2019

D. Xie et al.: View-Consistent Intrinsic Decomposition for Stereoscopic Images

C. REFLECTANCE-CONSISTENCY CONSTRAINTS
In general, stereoscopic images have large overlapped
regions. It is a hard constraint that the overlapped regions
between the left and right views share the same reflectance
value. Therefore, the intrinsic image decomposition for
stereoscopic images needs to take the reflectance-consistency
constraints into consideration. In the process of constructing
the reflectance-consistency term, a high-accuracy disparity
calculation between the left and right views is needed. The
dense disparity estimation methods are well documented
in [56]. In our implementation, we adopt a dense flow algo-
rithm proposed by Liu et al. [57] to calculate the opti-
cal flow of a stereoscopic image pair to find the matched
pixel pairs between two views. We encourage the matched
pixel pairs between the left and right views to share the
FIGURE 2. An illustration of image regions with similar image color and
same reflectance value. Thus, the reflectance-consistent term texture distributing locally.
Ec (sl,r ) is defined as follows:
X 0
Ec (sl,r ) = (rlm − rrm )2 , (8)
m,m0 Similarly, because rl,r = il,r − sl,r , we can re-write
Equ. 11 as
where rl and rr are the left and right reflectances, respectively.
m and m0 are a matched point pair between the left and
X X X
l,r − il,r − sl,r + sl,r ) . (12)
n 2
El (sl,r ) = (im n m
right views. Suppose that the coordinate of pixel m is (x, y). l,r Pi ∈0l m,n∈Pi
Then, the coordinate of the matched pixel m0 (x 0 , y0 ) can be
l l

calculated from the flow method as follows: An illustration of image regions with similar image color
and texture locally is shown in Fig. 2. Here, we sample several
(x 0 , y0 ) = (x + 4x, y + 4y). (9) pairs of points (yellow dots highlighted in Fig 2(b) and (c))
In Equ. 9, (4x, 4y) is the optical flow calculated by Liu’s in each super-pixel(Fig. 2(d)). The number of sampled points
method [57]. is determined by the size of a super-pixel. From Fig. 2,
By substituting rl,r with il,r − sl,r , we can re-write Equ. 8 pixels in the same super-pixel have similar reflectance, and
as therefore we add local constraints on super-pixels to con-
strain these pixels within the same super-pixel sharing sim-
m0 m0 2
X
l − ir − sl + sr ) .
(im m
Ec (sl,r ) = (10) ilar reflectance value. It is worthwhile to point out that
m,m0 local constraints are considered in the left and right views
In Equ. 10, similar to the meaning of rl and rr , il and ir are separately.
the left and right views of a stereoscopic image pair, sl and sr
are the left and right shadings. E. NON-LOCAL CONSTRAINTS
Similar to local constraints, there are also many image regions
D. LOCAL CONSTRAINTS with similar image color and texture distributing non-locally
Local texture cues have shown their effectiveness to reduce as depicted in Fig. 3.
the ambiguity [9], [27], [30]. In a real world scene, there are Accordingly, we add non-local constraints on the
many local regions that share similar textures and reflectance. super-pixels with similar image color and texture non-locally,
Therefore, it is reasonable to add constraints on pixels that which can further reduce the ambiguity. To be concrete, for
belong to the same image region. In this paper, we employ each super-pixel, we find K nearest neighborring super-pixels
a SLIC super-pixel method [58], [59] to segment an image in terms of image color and texture across the left and right
into different super-pixels. We add local constraints on the views. Then, we sample several pairs of pixels from the
pixels within the same super-pixel to encourage them to share super-pixels as illustrated in Fig. 3. We can notice that there
similar reflectance value, which gives rise to the local term are many similar non-adjacent super-pixels within and across
El (sl,r ) as follows two views. In our work, the non-local term Enl (sl,r ) is defined
X X X as
El (sl,r ) = m
(rl,r n 2
− rl,r ) , (11) X X
l,r Pi ∈0l m,n∈Pi Enl (sl,r ) = m
(rl,r n 2
− rl,r ) . (13)
l l
j j
Pinl ,Pnl ∈0nl m∈Pinl ,n∈Pnl
where Pil represents a super-pixel, 0l denotes the whole set
of all super-pixels, and m and n are two different pixels from where 0nl is the union of all super-pixels from the left and
the same super-pixel. right views. Compared to the local constraints, the difference

VOLUME 7, 2019 140359


FIGURE 3. An illustration of image regions with similar image color and
texture distributing non-locally.
between Equ. 11 and Equ. 13 is that pixels m and n are not in
the same super-pixel but in different super-pixels.
By substituting rl,r with il,r − sl,r , we can re-write Equ. 13
as
X X
l,r −il,r −sl,r +sl,r ) . (14)
n 2
Enl (sl,r ) = (im n m
j j
Pinl ,Pnl ∈0nl m∈Pinl ,n∈Pnl
F. ABSOLUTE SCALE CONSTRAINTS
Finally, we add the absolute scale constraints to push the
brightest pixels in the left and right views toward the unit
intensity value, which further disambiguate the shading
image. The absolute scale term Ea (sl,r ) is defined as follows
XX
l,r − 1) ,
2
Ea (sl,r ) = (sm (15)
l,r m∈Pa
where Pa denotes the brightest pixel or pixels.
Combining Equ. 3 with Equ. 7, Equ. 10, Equ. 12, Equ. 14
and Equ. 15, we find that the total energy function turns out
to be a quadratic function with respect to variable sl,r .
From the optimization theory, it is easy to figure out that the
total energy function Equ. 3 has a closed-form solution. In our
implementation, we formulate this closed-form solution into
a sparse linear system. 1 Then, the left shading sl and the
right shading sr can be obtained by solving the above sparse
linear system. Meanwhile, the left reflectance rl and the right
reflectance rr can be readily calculated by Equ. 2.
IV. EXPERIMENTAL RESULTS
In this section, we conduct extensive experiments both on
real-world stereoscopic images and the synthetic stereoscopic
images dataset. To begin with, all the parameters are set
as follows: λs = λr = λl = λnl = 1, λa = 1000,
1 In our implementation, the Matlab ‘\’ solver is adopted.
140360
D. Xie et al.: View-Consistent Intrinsic Decomposition for Stereoscopic Images
and τ = 0.0009, which are used by all test examples in our
implementation. All above parameters are set by extensive
experiments.
A. EXPERIMENTS ON REAL-WORLD
STEREOSCOPIC IMAGES
All real-world stereoscopic images are taken by a stereo-
scopic camera FinePix REAL 3D W3. Since the real-world
stereoscopic images are hard to obtain the ground-truth of
their reflectance and shading, Fig. 4 only shows the visual
results. As shown in Fig. 4, our proposed approach has suc-
cessfully decomposed the input image into its correspond-
ing reflectance component and shading component. Specif-
ically, the reflectance components, as shown in the third
and fourth columns in Fig. 4 (c) and Fig. 4 (d), only keep
color and texture information, while the shading compo-
nents, as shown in the fifth and sixth columns in Fig. 4 (e)
and Fig. 4 (f), contain lighting variations and shadow
effects.
To verify the individual contribution of the reflectance-
consistency term, the local and non-local term, we have also
conducted ablation experiments. As described in Section III,
the reflectance-consistency term encourages the reflectance
of the matched pixels between the left and right views to be
the same, we have conducted such a comparison experiments
with and without the reflectance-consistency constraints.
Fig. 5 reveals the individual contribution of the
reflectance-consistency term. The left and right reflectance
image, and the left and right shading image obtained with
and without reflectance-consistency constraints are shown
in Fig. 5 (b), Fig. 5 (c), Fig. 5 (d) and Fig. 5 (e), respec-
tively. From Fig. 5, it is clear that the decomposition results
with the reflectance-consistency term have better consistent
reflectance as compared to the decomposition results that are
generated without considering the reflectance-consistency
constraints.
Fig. 6 shows the individual contribution of the local and
non-local terms in the proposed decomposition framework.
In Fig. 6, the similar color and texture regions of the left
and right views obtain more consistent reflectance under
the situation where the local and non-local constraints are
considered in our decomposition framework.
B. EXPERIMENTS ON SYNTHETIC STEREOSCOPIC IMAGES
To quantitatively evaluate the performance of our proposed
method, we build a synthetic stereoscopic images dataset with
ground-truth of their corresponding reflectance and shading.
All synthetic stereoscopic images and their corresponding
reflectance and shading ground-truth are rendered by Maya
software. The procedure of rendering this dataset is described
as follows:
1) Create a scene model [60] with a certain number of
objects, the material type of every object is set as
Lambertian model, add color and texture information
if necessary.
VOLUME 7, 2019
D. Xie et al.: View-Consistent Intrinsic Decomposition for Stereoscopic Images

FIGURE 4. Stereoscopic intrinsic image decomposition on real-world stereoscopic images. (a) Left view. (b) Right view. (c) Left reflectance. (d) Right
reflectance. (e) Left shading. (f) Right shading.

FIGURE 5. Decomposition results with and without the reflectance-consistency constraints on real-world stereoscopic images. (a) Left (up) and
right (down) views. (b) Left (up) and right (down) reflectance without the reflectance constraints. (c) Left (up) and right (down) reflectance with
reflectance constraints. (d) Left (up) and right (down) shading without the reflectance constraints. (e) Left (up) and right (down) shading with the
reflectance constraints.

2) Create a stereo camera with proper view angle, dispar- 3) Render the scene without lighting, reflectance
ity and other camera settings. ground-truth (Rleft , Rright ) can be generated.

VOLUME 7, 2019 140361

 D. Xie et al.: View-Consistent Intrinsic Decomposition for Stereoscopic Images

FIGURE 6. Decomposition results with and without the local and non-local constraints on real-world stereoscopic images. (a) Left (up) and right (down)
views. (b) Left (up) and right (down) reflectance without the local and non-local constraints. (c) Left (up) and right (down) reflectance with the local and
non-local constraints. (d) Left (up) and right (down) shading without the local and non-local constraints. (e) Left (up) and right (down) shading with the
local and non-local constraints.

FIGURE 7. An example of synthetic stereoscopic images and its reflectance and shading ground-truth. (a) Left view. (b) Right view. (c) Left reflectance
ground-truth. (d) Right reflectance ground-truth. (e) Left shading ground-truth. (f) Right shading ground-truth.

4) Render the scene with the default or arbitrary light-
ing, synthetic stereoscopic images (Ileft , Iright ) can
be generated.
5) Transform Ileft , Iright , Rleft , Rright into logarithm domain
to get ileft , iright , rleft , rright respectively, obtain sleft , sright
by ileft −rleft and iright −rright . Take the inverse logarithm
of sleft , sright , shading ground-truth (Sleft , Sright ) can be
obtained.
Fig. 7 is an example of synthetic stereoscopic images and
its reflectance and shading ground-truth. Fig. 7(a) and (b) are
the left and right views of the synthetic stereoscopic images,
Fig. 7(c) and (d) are the left and right reflectance ground-
truths, and Fig. 7 (e) and (f) are the left and right shading
ground-truths, respectively.
The local mean squared error (LMSE) has been proposed
in the previous work [22] to evaluate decomposition algo-
rithms, as defined below:
arg min kXgtω − αX ω k2 .
X
LMSE(Xgt , X ) =
α
ω∈W
where Xgt and X denote the ground-truth vector and the
estimated vector, respectively, ω represents a sliding window,
W stands for the sliding window set of the vector, and α is
a scalar that allows to minimize the difference between the
ground-truth sliding window (Xgtω ) and the estimated sliding
window (X ω ). Generally, the sliding step is set to the half of
sliding window size.
The LMSE metric evaluation on synthetic stereoscopic
images are summarized in Table 1. The corresponding visual
decomposition results are shown in Fig. 8.
The individual contribution of reflectance consistency
term and local and non-local terms on the synthetic stereo-
scopic images are also verified by the similar way as the
experiments done on real-world stereoscopic images. The
average LMSE performance on the graphic stereoscopic
renderings under different conditions are listed on Table 2.
As is shown in Table 2, the reflectance consistency term
and local and non-local terms generally have positive impact
on our proposed decomposition framework. More specifi-
cally, reflectance consistent constraints show more important
impact on the final decomposition results.
To sum up, our proposed decomposition framework can
effectively produce better reflectance and shading image
qualitatively and quantitatively, which can be seen from
Fig. 4, Fig. 8 and Table 1.
(16)
C. COMPARISONS WITH MONOCULAR
INTRINSIC DECOMPOSITION
To testify the superiority of our stereoscopic intrinsic image
decomposition with compared to the intrinsic image decom-
position algorithm for monocular image, we conduct more
comparative experiments on our datasets.
Fig. 9 shows a visual comparison between Zhao’s
monocular intrinsic decomposition [27] and our proposed

140362 VOLUME 7, 2019

D. Xie et al.: View-Consistent Intrinsic Decomposition for Stereoscopic Images

TABLE 1. The LMSE of our proposed method experiments on synthetic stereoscopic images. R denotes reflectance red channel, G means reflectance
green channel, B stands for reflectance blue channel.

FIGURE 8. Stereoscopic intrinsic image decomposition on the synthetic stereoscopic images. (a) Left view. (b) Right view. (c) Left reflectance
ground-truth. (d) Right reflectance ground-truth. (e) Left shading ground-truth. (f) Right shading ground-truth. (g) Left reflectance. (h) Right reflectance.
(i) Left shading. (j) Right shading.

stereoscopic decomposition on real-world stereoscopic
image. Obviously, the decomposition results in Fig. 9 (b) pro-
duced by our method have more consistent reflectance com-
pared to Zhao’s method. It’s noted that the left and right
reflectance and shading image in Fig. 9 (a) are generated
by applying Zhao’s method to the left and right image,
respectively.
Similarly, we compare Zhao’s method and our method on
the synthetic stereoscopic images on the examples shown
in Fig. 8. The average LMSE of Zhao’s method and our

VOLUME 7, 2019 140363


TABLE 2. The average LMSE performance on the synthetic stereoscopic
images under different conditions.
FIGURE 9. Comparison between monocular intrinsic image
decomposition and our proposed method on real-world stereoscopic
image. (a) Result of [27] (b) Our method.
method on the synthetic stereoscopic images are shown
in Table 3.
From Table 3, it shows that the intrinsic decomposition
method for monocular images can not be directly applied
to decompose stereoscopic images in that their performance
basically worse than the decomposition algorithm designed
for stereoscopic images.
140364
D. Xie et al.: View-Consistent Intrinsic Decomposition for Stereoscopic Images
TABLE 3. The average LMSE performance of Zhao’s method [27] and our
method on the synthetic stereoscopic images.
V. CONCLUSIONS AND FUTURE WORKS
In this paper, we studied the intrinsic decomposition prob-
lem for stereoscopic image pairs. Our method combined
several kinds of constraints, namely the retinex constraints,
the reflectance-consistent constraints, the local and non-local
constraints, and the absolute scale constraints, into a unified
framework to decompose a stereoscopic image pairs into the
reflectance images and the shading images. Experimental
results on both real-world stereoscopic image and synthetic
stereoscopic images demonstrate that our proposed method
can produce more consistent results as compared to tradi-
tional methods that are developed for monocular images.
In addition, a real-world stereoscopic image and synthetic
stereoscopic images dataset has been made available for the
public.
One limitation of our method is that it only considers the
scene with Lambertian surface and simple lighting condi-
tions. In the real world, however, scenes are not necessarily
Lambertian and some objects in the scenes have highlight
effects. In addition, the illumination conditions are usually
complex. In our future works, we will focus on general scenes
and take into considerations the natural and complex lighting
conditions.
ACKNOWLEDGMENT
The authors would like to thank Dr. Shurong Song for her
generously providing image rendering models and material
to generate synthetic stereoscopic images.
REFERENCES
[1] H. Barrow and J. Tenenbaum, ‘‘Recovering intrinsic scene characteristics
from images,’’ Comput. Vis. Syst., vol. 2, pp. 3–26, 1978.
[2] G. Ye, E. Garces, Y. Liu, Q. Dai, and D. Gutierrez, ‘‘Intrinsic video and
applications,’’ ACM Trans. Graph., vol. 33, no. 4, p. 80, 2014.
[3] J. Shen, X. Yan, L. Chen, H. Sun, and X. Li, ‘‘Re-texturing by intrinsic
video,’’ Inf. Sci., vol. 281, pp. 726–735, Oct. 2014.
[4] R. Zhang, P.-S. Tsai, J. E. Cryer, and M. Shah, ‘‘Shape-from-shading:
A survey,’’ IEEE Trans. Pattern Anal. Mach. Intell., vol. 21, no. 8,
pp. 690–706, Aug. 1999.
[5] T. Chen, W. Yin, X. S. Zhou, D. Comaniciu, and T. S. Huang, ‘‘Total vari-
ation models for variable lighting face recognition,’’ IEEE Trans. Pattern
Anal. Mach. Intell., vol. 28, no. 9, pp. 1519–1524, Sep. 2006.
[6] Y. Weiss, ‘‘Deriving intrinsic images from image sequences,’’ in Proc.
IEEE Int. Conf. Comput. Vis., vol. 2, Jul. 2001, pp. 68–75.
VOLUME 7, 2019
D. Xie et al.: View-Consistent Intrinsic Decomposition for Stereoscopic Images
[7] M. F. Tappen, W. T. Freeman, and E. H. Adelson, ‘‘Recovering intrinsic
images from a single image,’’ IEEE Trans. Pattern Anal. Mach. Intell.,
vol. 27, no. 9, pp. 1459–1472, Sep. 2005.
[8] A. Bousseau, S. Paris, and F. Durand, ‘‘User-assisted intrinsic
images,’’ ACM Trans. Graph., vol. 28, no. 5, pp. 130-1–130-10,
Dec. 2009.
[9] L. Shen and C. Yeo, ‘‘Intrinsic images decomposition using a local and
global sparse representation of reflectance,’’ in Proc. IEEE Conf. Comput.
Vis. Pattern Recognit., Jun. 2011, pp. 697–704.
[10] J. T. Barron and J. Malik, ‘‘Shape, albedo, and illumination from a single
image of an unknown object,’’ in Proc. IEEE Conf. Comput. Vis. Pattern
Recognit., Jun. 2012, pp. 334–341.
[11] S. Bell, K. Bala, and N. Snavely, ‘‘Intrinsic images in the wild,’’ ACM
Trans. Graph., vol. 33, no. 4, p. 159, 2014.
[12] S. Bi, X. Han, and Y. Yu, ‘‘An L1 image transform for edge-preserving
smoothing and scene-level intrinsic decomposition,’’ ACM Trans. Graph.,
vol. 34, no. 4, p. 78, Jul. 2015.
[13] T. Narihira, M. Maire, and S. X. Yu, ‘‘Learning lightness from human
judgement on relative reflectance,’’ in Proc. IEEE Conf. Comput. Vis.
Pattern Recognit., Jun. 2015, pp. 2965–2973.
[14] M. Janner, J. Wu, T. D. Kulkarni, I. Yildirim, and J. Tenenbaum, ‘‘Self-
supervised intrinsic image decomposition,’’ in Proc. Adv. Neural Inf. Pro-
cess. Syst., 2017, pp. 5938–5948.
[15] Y. Wang, K. Li, J. Yang, and X. Ye, ‘‘Intrinsic decomposition from a single
RGB-D image with sparse and non-local priors,’’ in Proc. IEEE Int. Conf.
Multimedia Expo., Jul. 2017, pp. 1201–1206.
[16] H. Yue, J. Yang, X. Sun, F. Wu, and C. Hou, ‘‘Contrast enhancement based
on intrinsic image decomposition,’’ IEEE Trans. Image Process., vol. 26,
no. 8, pp. 3981–3994, Aug. 2017.
[17] G. Han, X. Xie, J. Lai, and W. S. Zheng, ‘‘Learning an intrinsic image
decomposer using synthesized RGB-D dataset,’’ IEEE Signal Process.
Lett., vol. 25, no. 6, pp. 753–757, Jun. 2018.
[18] Z. Li and N. Snavely, ‘‘Learning intrinsic image decomposition from
watching the world,’’ 2018, arXiv:1804.00582. [Online]. Available:
https://arxiv.org/abs/1804.00582
[19] E. H. Land and J. J. McCann, ‘‘Lightness and retinex theory,’’ J. Opt. Soc.
Amer., vol. 61, no. 1, pp. 1–11, 1971.
[20] B. K. P. Horn, ‘‘Determining lightness from an image,’’ Comput. Graph.
Image Process., vol. 3, no. 4, pp. 277–299, 1974.
[21] B. K. P. Horn, Robot Vision. Cambridge, MA, USA: MIT Press, 1986.
[22] R. Grosse, M. K. Johnson, E. H. Adelson, and W. T. Freeman, ‘‘Ground
truth dataset and baseline evaluations for intrinsic image algorithms,’’ in
Proc. IEEE Int. Conf. Comput. Vis., Sep. 2009, pp. 2335–2342.
[23] J. Shen, X. Yang, Y. Jia, and X. Li, ‘‘Intrinsic images using optimiza-
tion,’’ in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2011,
pp. 3481–3487.
[24] J. Shen, X. Yang, X. Li, and Y. Jia, ‘‘Intrinsic image decomposition using
optimization and user scribbles,’’ IEEE Trans. Cybern., vol. 43, no. 2,
pp. 425–436, Apr. 2013.
[25] J. T. Barron and J. Malik, ‘‘Shape, illumination, and reflectance from
shading,’’ IEEE Trans. Pattern Anal. Mach. Intell., vol. 37, no. 8,
pp. 1670–1687, Aug. 2015.
[26] L. Shen, P. Tan, and S. Lin, ‘‘Intrinsic image decomposition with non-
local texture cues,’’ in Proc. IEEE Conf. Comput. Vis. Pattern Recognit.,
Jun. 2008, pp. 1–7.
[27] Q. Zhao, P. Tan, Q. Dai, and L. Shen, ‘‘A closed-form solution to retinex
with nonlocal texture constraints,’’ IEEE Trans. Pattern Anal. Mach. Intell.,
vol. 34, no. 7, pp. 1437–1444, Jul. 2012.
[28] G. D. Finlayson, S. D. Hordley, and P. M. Hubel, ‘‘Color by correlation:
A simple, unifying framework for color constancy,’’ IEEE Trans. Pattern
Anal. Mach. Intell., vol. 23, no. 11, pp. 1209–1221, Nov. 2001.
[29] J. T. Barron and J. Malik, ‘‘Color constancy, intrinsic images, and shape
estimation,’’ in Proc. Eur. Conf. Comput. Vis. Berlin, Germany: Springer,
2012, pp. 57–70.
[30] L. Shen, C. Yeo, and B.-S. Hua, ‘‘Intrinsic image decomposition using
a sparse representation of reflectance,’’ IEEE Trans. Pattern Anal. Mach.
Intell., vol. 35, no. 12, pp. 2904–2915, Dec. 2013.
[31] T. Luo, J. Shen, and X. Li, ‘‘Accurate normal and reflectance recovery
using energy optimization,’’ IEEE Trans. Circuits Syst. Video Technol.,
vol. 25, no. 2, pp. 212–224, Feb. 2015.
[32] B. Mendiburu, 3D Movie Making: Stereoscopic Digital Cinema from Script
to Screen. New York, NY, USA: Taylor & Francis, 2009.
[33] Y. Niu, W.-C. Feng, and F. Liu, ‘‘Enabling warping on stereoscopic
images,’’ ACM Trans. Graph., vol. 31, no. 6, p. 183, 2012.
VOLUME 7, 2019
[34] S.-J. Luo, I.-C. Shen, B.-Y. Chen, W.-H. Cheng, and Y.-Y. Chuang,
‘‘Perspective-aware warping for seamless stereoscopic image cloning,’’
ACM Trans. Graph., vol. 31, no. 6, p. 182, 2012.
[35] S.-P. Du, S.-M. Hu, and R. R. Martin, ‘‘Changing perspective in stereo-
scopic images,’’ IEEE Trans. Vis. Comput. Graphics, vol. 19, no. 8,
pp. 1288–1297, Aug. 2013.
[36] W.-Y. Lo, J. van Baar, C. Knaus, M. Zwicker, and M. Gross, ‘‘Stereoscopic
3D copy & paste,’’ ACM Trans. Graph., vol. 29, no. 6, p. 147, 2010.
[37] L. Wang, H. Jin, R. Yang, and M. Gong, ‘‘Stereoscopic inpainting: Joint
color and depth completion from stereo images,’’ in Proc. IEEE Conf.
Comput. Vis. Pattern Recognit., Jun. 2008, pp. 1–8.
[38] F. Zhang and F. Liu, ‘‘Casual stereoscopic panorama stitching,’’ in Proc.
IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2015, pp. 2002–2010.
[39] H. Guo, S. Liu, T. He, S. Zhu, B. Zeng, and M. Gabbouj, ‘‘Joint video
stitching and stabilization from moving cameras,’’ IEEE Trans. Image
Process., vol. 25, no. 11, pp. 5491–5503, Nov. 2016.
[40] K.-Y. Lee, C.-D. Chung, and Y.-Y. Chuang, ‘‘Scene warping: Layer-based
stereoscopic image resizing,’’ in Proc. IEEE Comput. Vis. Pattern Recog-
nit., Jun. 2012, pp. 49–56.
[41] C.-H. Chang, C.-K. Liang, and Y.-Y. Chuang, ‘‘Content-aware display
adaptation and interactive editing for stereoscopic images,’’ IEEE Trans.
Multimedia, vol. 13, no. 4, pp. 589–601, Aug. 2011.
[42] R. F. Tong, Y. Zhang, and K. L. Cheng, ‘‘StereoPasting: Interactive compo-
sition in stereoscopic images,’’ IEEE Trans. Vis. Comput. Graphics, vol. 19,
no. 8, pp. 1375–1385, Aug. 2013.
[43] N. Bonneel, B. Kovacs, S. Paris, and K. Bala, ‘‘Intrinsic decompositions
for image editing,’’ Comput. Graph. Forum, vol. 36, no. 2, pp. 593–609,
2017.
[44] F. Liu, M. Gleicher, H.-L. Jin, and A. Agarwala, ‘‘Content-preserving
warps for 3D video stabilization,’’ ACM Trans. Graph., vol. 28, no. 3,
pp. 44:1–44:9, Aug. 2009.
[45] F. Liu, Y. Niu, and H. Jin, ‘‘Joint subspace stabilization for stereo-
scopic video,’’ in Proc. IEEE Int. Conf. Comput. Vis., Dec. 2013,
pp. 73–80.
[46] S. Duchêne, C. Riant, G. Chaurasia, J. L. Moreno, P.-Y. Laffont, S. Popov,
A. Bousseau, and G. Drettakis, ‘‘Multiview intrinsic images of outdoors
scenes with an application to relighting,’’ ACM Trans. Graph., vol. 34,
no. 5, p. 164, 2015.
[47] D. Xie, S. Liu, K. Lin, S. Zhu, and B. Zeng, ‘‘Intrinsic decomposition for
stereoscopic images,’’ in Proc. IEEE Int. Conf. Image Process., Sep. 2016,
pp. 1744–1748.
[48] E. H. Land, ‘‘The retinex,’’ Amer. Sci., vol. 52, no. 2, pp. 247–264, 1964.
[49] A. Meka, M. Zollhöfer, C. Richardt, and C. Theobalt, ‘‘Live intrinsic
video,’’ ACM Trans. Graph., vol. 35, no. 4, p. 109, 2016.
[50] A. S. Baslamisli, H.-A. Le, and T. Gevers, ‘‘CNN based learning using
reflection and retinex models for intrinsic image decomposition,’’ 2017,
arXiv:1712.01056. [Online]. Available: https://arxiv.org/abs/1712.01056
[51] J. Shi, Y. Dong, H. Su, and S. X. Yu, ‘‘Learning non-Lambertian object
intrinsics across shapenet categories,’’ in Proc. IEEE Conf. Comput. Vis.
Pattern Recognit., Jul. 2017, pp. 5844–5853.
[52] L. Lettry, K. Vanhoey, and L. van Gool, ‘‘Unsupervised deep
single-image intrinsic decomposition using illumination-varying
image sequences,’’ 2018, arXiv:1803.00805. [Online]. Available:
https://arxiv.org/abs/1803.00805
[53] J. T. Barron and J. Malik, ‘‘Intrinsic scene properties from a single
RGB-D image,’’ IEEE Trans. Pattern Anal. Mach. Intell., vol. 38, no. 4,
pp. 690–703, Apr. 2016.
[54] S. Kim, K. Park, K. Sohn, and S. Lin, ‘‘Unified depth prediction and intrin-
sic image decomposition from a single image via joint convolutional neural
fields,’’ in Proc. Eur. Conf. Comput. Vis. Cham, Switzerland: Springer,
2016, pp. 143–159.
[55] J. Kang, B. Jiang, J. Chen, and X. Liu, ‘‘Stereoscopic image recoloring via
consistent intrinsic decomposition,’’ in Proc. Int. Conf. Virtual Reality Vis.,
Aug. 2014, pp. 272–277.
[56] D. Scharstein and R. Szeliski, ‘‘A taxonomy and evaluation of dense two-
frame stereo correspondence algorithms,’’ Int. J. Comput. Vis., vol. 47,
nos. 1–3, pp. 7–42, Apr. 2002.
[57] S. Liu, P. Tan, L. Yuan, J. Sun, and B. Zeng, ‘‘MeshFlow: Minimum
latency online video stabilization,’’ in Proc. Eur. Conf. Comput. Vis. Cham,
Switzerland: Springer, 2016, pp. 800–815.
[58] P. F. Felzenszwalb and D. P. Huttenlocher, ‘‘Efficient graph-based
image segmentation,’’ Int. J. Comput. Vis., vol. 59, no. 2, pp. 167–181,
Sep. 2004.
140365

[59] R. Achanta, A. Shaji, K. Smith, A. Lucchi, P. Fua, and S. Süsstrunk,
‘‘SLIC superpixels compared to state-of-the-art superpixel methods,’’
IEEE Trans. Pattern Anal. Mach. Intell., vol. 34, no. 11, pp. 2274–2282,
Nov. 2012.
[60] S. Song, F. Yu, A. Zeng, A. X. Chang, M. Savva, and T. Funkhouser,
‘‘Semantic scene completion from a single depth image,’’ 2017,
arXiv:1611.08974. [Online]. Available: https://arxiv.org/abs/1611.08974
DEHUA XIE (S’16) received the B.S. degree
from the Hunan University of Technology, China,
in 2011, and the M.S. degree from the University
of Electronic Science and Technology of China,
Chengdu, China, in 2014, where he is currently
pursuing the Ph.D. degree. His research topics
include image processing and computer vision.
SHUAICHENG LIU (M’15) received the B.E.
degree from Sichuan University, Chengdu, China,
in 2008, and the M.S. and Ph.D. degrees from
the National University of Singapore, Singapore,
in 2014 and 2010, respectively. In 2014, he joined
the University of Electronic Science and Tech-
nology of China and is currently an Associate
Professor with the School of Information and
Communication Engineering, Institute of Image
Processing, Chengdu, China. His research inter-
ests include computer vision and computer graphics.
YINGLONG WANG (S’16) received the B.S.
degree from the School of Information Science
and Engineering, Lanzhou University, Lanzhou,
China, in 2011. He is currently pursuing the
Ph.D. degree with the University of Electronic Sci-
ence and Technology of China, Chengdu, China.
His research interests include image processing
and machine learning.
140366
D. Xie et al.: View-Consistent Intrinsic Decomposition for Stereoscopic Images
SHUYUAN ZHU (S’08–A’09–M’13) received
the Ph.D. degree from the Hong Kong Univer-
sity of Science and Technology, Hong Kong,
in 2010. From 2010 to 2012, he was with the
Hong Kong University of Science and Technology
and Hong Kong Applied Science and Technology
Research Institute Company Limited, respectively.
In 2013, he joined the University of Electronic
Science and Technology of China and is currently
a Professor with the School of Information and
Communication Engineering, Chengdu, China. His research interests include
image/video compression, image processing, and compressive sensing.
He has more than 60 research publications and recipient of the Top 10% paper
award at the IEEE ICIP-2014 and the Top 10% paper award at VCIP-2016.
He is a member of the IEEE CAS Society. He served as the Committee
Member for the IEEE ICME-2014, VCIP-2016, and PCM-2017. He was the
special Session Chair of Image Super-Resolution at the IEEE DSP-2015.
BING ZENG (M’91–SM’13–F’16) received the
B.Eng. and M.Eng. degrees in electronic engineer-
ing from the University of Electronic Science and
Technology of China (UESTC), Chengdu, China,
in 1983 and 1986, respectively, and the Ph.D.
degree in electrical engineering from the Tam-
pere University of Technology, Tampere, Finland,
in 1991. He was a Postdoctoral Fellow with the
University of Toronto, from September 1991 to
July 1992, and as a Researcher with Concor-
dia University, from August 1992 to January 1993. Then, he joined the
Hong Kong University of Science and Technology (HKUST). After 20 years
of service, HKUST, he returned to UESTC, in 2013, through Chinas 1000-
TalentScheme. At UESTC, he leads the Institute of Image Processing to work
on image and video processing, 3-D and multiview video technology, and
visual big data. During his tenure with the HKUST and UESTC, he graduated
more than 30 Master’s and Ph.D. students, recipient of the about 20 research
grants, filed eight international patents, and published more than 250 articles.
Three representing works are as follows: one article on fast block motion
estimation, published in the IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS
FOR VIDEO TECHNOLOGY (TCSVT), in 1994, has so far been SCI-cited more
than 1000 times (Google-cited more than 2100 times), and currently stands
at the 7th position among all articles published in this Transactions; one
article on smart padding for arbitrarily-shaped image blocks, published in
the IEEE TCSVT, in 2001, leads to a patent that has been successfully
licensed to companies; and one article on directional discrete cosine trans-
form, published in the IEEE TCSVT, in 2008, recipient of the 2011 IEEE
CSVT Transactions Best Paper Award. He was also a recipient of the Best
Paper Award at China-Com three times (2009 Xi’an, 2010 Beijing, and
2012 Kunming). He was also a recipient of a 2nd Class Natural Science
Award (the first recipient) from the Chinese Ministry of Education, in 2014.
He was an IEEE Fellow, in 2016 for contributions to image and video coding.
He was a General Co-Chair of the IEEE VCIP-2016, held in Chengdu, China,
in November 2016. He is currently on the Editorial Board of the Journal of
Visual Communication and Image Representation and serves as a General
Co-Chair of PCM-2017. He served as an Associate Editor for the IEEE
TCSVT for eight years and recipient of the Best Associate Editor Award,
in 2011.
VOLUME 7, 2019