128
1, MARCH 2016
Fast Coding Quad-Tree Decisions Using Prediction Residuals
Statistics for High Efficiency Video Coding (HEVC)
Hui Li Tan, Member, IEEE, Chi Chung Ko, Senior Member, IEEE, and Susanto Rahardja, Fellow, IEEE
Abstract—High Efficiency Video Coding (HEVC) is the latest video
coding standard to meet market demands for real-time high quality
video codecs. As compared to its predecessor H.264/AVC, HEVC can
achieve significant compression gains but with higher encoding com-
plexity. Therefore, for real-time applications, significant encoding time
reduction is still necessary. In the HEVC test model, the large number of
coding quad-tree decisions to be tested during rate-distortion optimiza-
tion would result in high encoding time. Hence, we propose a method to
reduce the high encoding time by pruning the coding quad-trees using
prediction residuals statistics. Experimental results from HM16.3-based
implementations show that the proposed residual-based pruning method
can reduce encoding time by an average of about 44% with an average
of about 1.0% coding loss.
Index Terms—High Efficiency Video Coding (HEVC), Coding
Quad-Tree.
I. I NTRODUCTION
ECENT advancements in integrated circuits and multimedia
R technology have made consumer mobile devices ubiquitous,
allowing video contents to be broadcasted/transmitted anytime and
anywhere. This directly increases market demands for real-time high
quality video codecs. For instance, mobile video calling and con-
ferencing applications require consumer mobile devices to encode
and decode video contents in real-time. Furthermore, online social
networking and broadcasting platforms have been extending their
capabilities to allow consumers to live stream events using mobile
devices. Even commercial broadcasting companies have been relying
on small consumer grade devices for live news updates from places
with accessibility constraints. As these devices are being equipped
with increasingly high resolution content capture systems, consumers
will have the capability to produce higher quality video contents. In
order for the contents to be readily available for consumption, there is
a need to have high performance video codecs that is able to stream
high quality video contents within similar bandwidth and time con-
straints. Nonetheless, the video codecs must remain low in complexity
to run effectively on these low power devices with smaller integrated
circuits.
High Efficiency Video Coding (HEVC) [1], [2] is the lat-
est video coding standard developed by the Joint Collaborative
Team - Video Coding (JCT-VC), which comprises the ITU-T Video
Coding Experts Group (VCEG) and the ISO/IEC Moving Pictures
Experts Group (MPEG). As compared to the existing H.264/AVC
Manuscript received June 19, 2015; revised October 11, 2015; accepted
October 21, 2015. Date of publication January 12, 2016; date of current
version March 2, 2016. (Corresponding author: Susanto Rahardja.)
H. L. Tan is with the Institute for Infocomm Research, A*STAR,
Singapore 138632, and also with the National University of Singapore,
Singapore 117583 (e-mail: hltan@i2r.a-star.edu.sg).
C. C. Ko is with the National University of Singapore, Singapore 117583
(e-mail: elekocc@nus.edu.sg).
S. Rahardja is with Northwestern Polytechnical University, Xi’an,
Shaanxi, 710072, P.R. China, and also with the National University of
Singapore, Singapore 117583 (e-mail: susanto@nwpu.edu.cn).
Color versions of one or more of the figures in this paper are available
online at http://ieeexplore.ieee.org.
Digital Object Identifier 10.1109/TBC.2015.2505406
0018-9316 c 2016 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.
See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
IEEE TRANSACTIONS ON BROADCASTING, VOL. 62, NO.
standard [3], [4], the HEVC standard approximately doubles data
compression rate for equal perceptual video quality [5]. Designed to
address increased video resolutions, the attractive compression ben-
efits offered by HEVC will ease the bandwidth bottleneck for the
deployment of HD and beyond HD contents.
Nonetheless, HEVC is reported to be several times more complex
than H.264/AVC [2] and particularly suffers from high encoding time.
The high encoding time of HEVC is attributed to various new cod-
ing tools and features that are introduced to provide increased coding
flexibility. For instance, HEVC employs a flexible hierarchical pic-
ture partitioning structure that enables the use of large and multiple
sizes of processing units. HEVC also employs more comprehensive
intra prediction, inter prediction, transform and quantization, a new
loop filter, and an enhanced version of entropy coding. Such sophis-
ticated tools come with performance improvement but at the expense
of higher coding complexity, and this complexity would consider-
ably hamper its usage particularly in applications where real-time is
required.
In this paper, we propose a method to reduce the high encoding
time of HEVC, by pruning its coding quad-trees using prediction
residuals statistics. The proposed residual-based pruning method
introduces little modifications and insignificant computational com-
plexity overheads to the existing encoder, but can effectively reduce
redundant mode and partition checks.
This paper is organized as follows. First, an overview of the HEVC
picture partitioning structure is provided in Section II. Then, prior
works on fast encoding methods for coding quad-tree decisions are
discussed in Section III. Subsequently, our proposed method is pre-
sented in Section IV, and the experimental results are shown in
Section V. Finally, the conclusion is discussed in Section VI.
II. OVERVIEW OF HEVC P ICTURE PARTITIONING S TRUCTURE
As shown in Fig. 1, like its predecessor standards such as MPEG-4
and H.264/AVC, HEVC adopts the conventional approach of using
temporal and spatial prediction, followed by transform coding of the
prediction residuals and entropy coding of the quantized transform
coefficients with other coding parameters. Nonetheless, HEVC uses
a flexible hierarchical picture partitioning structure to enable large
and multiple sizes of coding units (CUs), prediction units (PUs), and
transform units (TUs) [2].
A CU defines a region sharing the same prediction mode (i.e.,
SKIP, INTER, or INTRA), a PU defines a region sharing the same
prediction information (i.e., intra prediction modes, motion parame-
ters, etc.), and a TU defines a region sharing the same transformation
and quantization. As shown in Fig. 2, in a coding quad-tree repre-
sentation, a picture is first divided into non-overlapping coding tree
units (CTUs), which can then be recursively divided into smaller
CUs. A leaf CU, which is a CU that is not divided into sub-CUs,
can either be SKIP, INTER or INTRA coded. The CTU size is 64×64
(depth 0), and the minimum CU size is 8×8 (depth 3) [6]. An INTER
or INTRA coded CU can then be further divided into PUs of various
partitions. The possible PU partitions for a CU of size 2N × 2N are
IEEE TRANSACTIONS ON BROADCASTING, VOL. 62, NO. 1, MARCH 2016
Fig. 1. A block-based hybrid video encoder. An approach to encoding time
reduction is by judiciously pruning the rate-distortion search space in the
circled region.
Fig. 2. An illustration of the nested coding and transform quad-trees in the
HM16.3 reference software encoder. Blue bubbles: Coding quad-tree repre-
sentation, where a largest coding tree unit (CTU) can be recursively divided
into smaller coding units (CUs). Green bubbles: A leaf CU can be further
divided into prediction units (PUs). Yellow bubbles: Transform quad-tree
representation, where a transform unit (TU) can be recursively divided into
smaller TUs.
2N × 2N, N × 2N, 2N × N, 2N × N2 , 2N × 3N N 3N
2 , 2 × 2N, and 2 × 2N
for INTER coded CUs, and 2N × 2N and N × N for INTRA coded
CUs [6]. Finally, in a transform quad-tree representation nested in a
CU, a TU can be recursively divided into smaller TUs. The TU size
can range from 32 × 32 down to 4 × 4, with a maximum of three
levels allowed for the transform quad-tree depth within a CU [6].
The coding and transform quad-trees recursively decompose an
image into variable unit sizes so that regions of different characteris-
tics can be better encoded. For instance, regions of high stationarity
and homogeneity can be encoded with larger unit sizes with smaller
side-information overheads. However, according to the rate-distortion
optimization (RDO) framework [7], [8], at a unit level, an operating
point on the convex hull of all rate-distortion (RD) pairs is found by
minimizing the RD cost function over all sets of decisions made for
the unit, i.e.,
min J = Di (i ) + λRi (i ), (1)
i
where i is the set of decisions made for unit i, and Di (i ) and
Ri (i ) are the distortion and rate, respectively achieved by using i .
Each choice in the nested quad-trees amounts to a point on the
operational RD plane. Hence, the flexibility provided by the nested
quad-trees greatly increases the operational RD search space and
computational complexity of RDO at the encoder. Fast decision algo-
rithms, which judiciously prune the RD search space, are necessary
for real-time broadcasting applications [9]–[11].
One possible way to reduce the RD search space is by avoiding
full branching of the coding quad-trees. Referring to Fig. 4, in the
129
reference software description [12], the best prediction mode and
partition are first determined by assuming that the CU has not been
divided before recursively repeating the same process for each of its
sub-CUs. A final decision as to whether the CU is to be divided or
not is then made only after all the sub-CUs have been analyzed. This
corresponds to traversing the coding quad-tree in a depth-first man-
ner; the coding quad-tree decisions are made bottom up. Hence, if the
initial branching of the coding quad-trees can be avoided, the com-
putational complexity of RDO at the encoder can be greatly reduced.
In this paper, we explore the use of prediction residuals statistics for
pruning the coding quad-trees.
III. P RIOR W ORKS ON FAST E NCODING M ETHODS
The “Early SKIP Decision (ESD)” method [13], [14], “Coding-
Block-Flag Fast Condition (CFM)” method [15], and “Early CU
Condition (ECU)” method [16], [17] are some methods already
present in the reference software and can be enabled with the encoder
parameters ESD, CFM, and ECU respectively [2]. The “Early SKIP
Decision (ESD)” method checks for the motion vector difference and
coding-block-flag (cbf) when a unit is encoded with CU mode INTER
with partition 2N × 2N. Zero cbf indicates that there is no non-zero
quantized transform residuals to be coded. If the motion vector differ-
ence and cbf are both zero, the best interim CU mode is set as SKIP,
and all remaining CU mode checks at the current depth are skipped.
The “Coding-Block-Flag Fast Condition (CFM)” method checks for
the cbf when a unit is encoded with CU mode INTER with any par-
tition. If the cbf is zero, the best interim CU mode is set as INTER
with the current partition, and all remaining CU mode checks at the
current depth are skipped. The “Early CU Condition (ECU)” method
checks for the best interim CU mode at the current depth before the
CU mode checks at the next depth. If the best interim CU mode at
the current depth is SKIP, the best CU mode is set as SKIP, and all
remaining CU mode checks are skipped, i.e., the coding quad-tree is
terminated.
There are also a few prior works on coding quad-tree mode and
partition fast decision methods which are not present in the reference
software. Some of these methods attempt to prune the search space by
terminating the coding quad-trees early. The “Fast Encoder (FEN)”
method [18] checks for the RD cost when a unit is encoded with
CU mode SKIP. If the SKIP RD cost is less than a threshold, the
coding quad-tree is terminated. The threshold is set as the aver-
age RD cost of a number of previously SKIP coded CUs of the
same CU size. An attempt to improve the “Fast Encoder (FEN)”
method adjusts the weights appended to the average RD cost of a
number of previously SKIP coded CUs of the same CU size [19].
The default weight is 1.0, and is increased by 0.5 if either one
of the neighboring CU is SKIP coded or its CU of higher depth
is SKIP coded. The proposed method in [19] obtained an average
encoding time reduction of 46% with 1.4% average coding loss,
when benchmarked against HM3.0 for the Random Access setting
with High Efficiency configuration. Furthermore, the “Early Partition
Decision (EPD)” method [20] checks for the interim RD cost when
a unit is encoded with CU mode INTER with any partition. If the
interim RD cost is less than a threshold, the coding quad-tree is
terminated. Similar to the “FEN” method, the threshold is set to
the average RD cost of a number of previously SKIP coded CUs
of the same CU size. The “CU Depth Pruning (CDP)” method
[20] terminates the coding quad-trees at the higher branches by
traversing the coding quad-trees in a pseudo breadth-first manner.
When benchmarked against HM4.0, their proposed “EPD” method
obtained an average encoding time reduction of 36% with 0.8%
average coding loss, while their proposed “CDP” method obtained
130 IEEE TRANSACTIONS ON BROADCASTING, VOL. 62, NO. 1, MARCH 2016

an average encoding time reduction of 9% with 0.1% average

coding loss.
Some other coding quad-tree mode and partition fast decision
methods attempt to prune the search space by skipping certain depths
of the coding quad-trees. Frame-level and CU-level depth statistics
are used in skipping rarely used CU depths [21]. The CU depth statis-
tics of the CUs in a frame are used at the frame-level, while the CU
depth statistics of the left, top-left, top, top-right and co-located CUs
are used at the CU-level. When benchmarked against TMuC0.9 on
some 720p (1280 × 720) and 1080p (1920 × 1080) test sequences for
only quantization parameter QP = 24, the proposed method in [21]
obtained an average encoding time reduction of 42%. Similarly, the
“Adaptive Depth Range Determination (ADRD)” method [22] uses
the CU depth statistics of the left, top, top-left, and co-located CUs
for skipping rarely used CU depths. The “ADRD” method is cou-
pled with three coding quad-tree early termination methods based
on (i) motion homogeneity checking of the current, left, top, top-left
and co-located CUs; (ii) RD cost checking of the current CU, with
the threshold set as a weighted RD cost of the left, top, top-left and Fig. 3. Plot of log thresholds vs QP.
co-located CUs; and (iii) SKIP mode checking, by skipping further
branching if the current and parent CUs are SKIP coded. Their pro-
posed “ADRD” method obtained an average encoding time reduction and the absolute difference between the prediction residuals variances
of 42% with 1.3% average coding loss, when benchmarked against of the two 2N × N PUs,
    
HM2.0. In another work, the coding quad-tree mode decision prob-  
u2 = V Xim¯ ,U − V Xim¯ ,D , (3)
lem is casted as a Bayesian decision problem [23]. Their proposed
method obtained an average encoding time reduction of 41% with where V(X) is the variance of the prediction residuals in partition X.
1.9% average coding loss, when benchmarked against HM4.0 using The proposed method terminates a coding quad-tree early when
four HEVC test sequences. In this paper, we use prediction residuals the prediction residuals statistics are both less than a threshold, i.e.,
statistics for pruning the coding quad-trees. The prediction residu-
u1 < ncf (QP), (4)
als statistics are obtained from the current CU and can be computed
independently of its spatially and temporally neighbouring CUs. We and
then apply a QP-dependent threshold for coding quad-tree decisions
using the prediction residuals statistics. u2 < ncf (QP), (5)
where n is the number of pixels in a CU, f (QP) is a quanti-
IV. P ROPOSED M ETHOD zation parameter (QP)-dependent threshold, and c is a constant
experimentally set to trade off complexity and coding performance.
Referring again to Fig. 2, each frame is divided into non-
The prediction residuals statistics are computed over different CU
overlapping CTUs, where Xi0 denotes the ith 0 CTU within the frame. sizes and hence is normalized by the number of pixels in a CU.
Let M denote the maximum depth of the coding quad-tree, and m0
Furthermore, the QP-dependent threshold would allow a higher
denotes a parameter for deciding the minimum CU size. Then, a CU
threshold and hence fewer splits into smaller CUs for larger QPs.
at depth m, where 0 ≤ m < M, is of size 2m0 +(M−m) × 2m0 +(M−m) .
As in [24], we model our QP-dependent threshold as
Therefore the CTU size is 2m0 +M × 2m0 +M while the minimum CU
size is 2m0 +1 × 2m0 +1 . According to the HEVC common test con- 2
f (QP) = , (6)
ditions [6], M = 4 and m0 = 2, i.e., the CTU size is 64 × 64, and 12
the minimum CU size is 8 × 8. where  is the quantization step size [4]. Eq. (6) is essentially the
Each CU at depth m is denoted by Xi0 ,i1 ,··· ,im , where as mentioned mean-squared quantization error [25].
above, i0 indexes the location of the root CTU within the frame, while An analysis of the empirical QP-dependent threshold for the
each subsequent index, i1 , · · · , im (0 ≤ i1 , · · · , im ≤ 3), specifies the prediction residuals statistics is performed using the HEVC test
index of the CU within its parent. Therefore, Xi0 ,i1 ,··· ,im−1 ,im is the sequences [6] on HM16.3 [12]. Two test sequences are used from
ith
m CU of Xi0 ,i1 ,··· ,im−1 . For example, if the parent CU is X0 , its four each resolution class. For each video sequence, the prediction resid-
sub-CUs would be X0,0 , X0,1 , X0,2 , and X0,3 . Similarly, if the parent uals statistics and ground truth split decisions (split decisions by
CU is X0,1 , its four sub-CUs would be X0,1,0 , X0,1,1 , X0,1,2 , and reference software with exhaustive CU search) are collected for eight
X0,1,3 . This notation would allow us to uniquely identify each CU video frames over QP = {17, 18, . . . , 41, 42}. For each QP, we find
within a frame. For convenience, we would also use Xim¯ to denote gQP to minimize the mean-squared error between y and ỹ, where
¯ denotes the list of indices i0 , i1 , · · · , im .
Xi0 ,i1 ,··· ,im , i.e., im y and ỹ denote the ground truth split decisions and predicted split
Let the two N × 2N PUs of CU Xim¯ be denoted by Xim¯ ,L and Xim¯ ,R decisions, respectively.
respectively. Similarly, let the two 2N ×N PUs of CU Xim¯ be denoted  max{u1 , u2 }
by Xim¯ ,U and Xim¯ ,D respectively. The proposed method terminates a 0 (no split) if < gQP
ỹ = n
coding quad-tree early by using prediction residuals statistics. The 1 (split) if otherwise.
prediction residuals statistics are computed as the absolute difference
between the prediction residuals variances of the two N × 2N PUs, Fig. 3 shows a plot of log(gQP ) against QP. log(gQP ) is approx-
     imately a linear function of QP, i.e., gQP is approximately a
  QP−4
u1 = V Xim¯ ,L − V Xim¯ ,R , (2) logarithmic linear function of QP. Since  ≈ 2 6 [4],  is also
IEEE TRANSACTIONS ON BROADCASTING, VOL. 62, NO. 1, MARCH 2016
Fig. 4. An illustration of the proposed method. The mode and partition
decision for a CU in the HM16.3 reference software encoder is as shown.
The proposed method terminates a coding quad-tree early when the prediction
residuals statistics are both less than a QP-dependent threshold.
approximately a logarithmic linear function of QP. Fig. 3 shows a
plot of log(cf (QP)) against QP. High correlation of gQP and f (QP)
can be observed (breaking down only at very high QP above 41),
supporting the use of f (QP) for coding quad-tree decisions using the
prediction residuals statistics.
The proposed method is illustrated in Fig. 4. As can be seen, the
proposed method involves little modifications to the existing encoder,
since the prediction residuals statistics are obtained from the PUs
directly within a CU, independently of the spatially and temporally
neighbouring CUs. The computational complexity overhead is in
computing the prediction residuals statistics for the pruning condition.
Nonetheless, these computations of simple second order prediction
residuals statistics are operations with very low complexity compared
to the encoding processes that can be pruned, e.g., mode decision and
motion estimation.
V. E XPERIMENTAL R ESULTS
The proposed method is implemented in HM16.3 [12], and tested
using the common test conditions [6]. The HEVC test sequences
cover a range of resolutions, including Class A (2560 × 1600 pix-
els), Class B (1920 × 1080 pixels), Class C (832 × 480 pixels),
Class D (416 × 240 pixels) and Class E (1280 × 720 pixels). Using
HM16.3 as the anchor, the performances of our algorithm in Random
Access (RA) and Low Delay (LD) settings, for both Main and Main10
configurations are assessed. The RA setting corresponds to storage
or entertainment applications, while the LD setting corresponds to
video conferencing applications. The Main and Main10 configura-
tions correspond to 8 and 10 bit-per-sample storage and output,
respectively. For each method and configuration, a video sequence
is coded at four quantization parameters, QP = {22, 27, 32, 37}. The
coding performances of the proposed method are measured by the
Bjontegaard Delta PSNR (BD-PSNR) [26], and their encoding times
are presented as a percentage of the anchor encoding run-times. We
experimented with c = 10i , where i = {−5, . . . , 0}, and found 0.001
give the best trade-off between complexity and coding performance.
131
TABLE I
C ODING LOSSES (BD-R ATE ) AND E NCODING T IMES
(E NC T IME ) FOR R ANDOM ACCESS (RA) S ETTING
W ITH M AIN AND M AIN 10 C ONFIGURATIONS
TABLE II
C ODING LOSSES (BD-R ATE ) AND E NCODING T IMES (E NC T IME ) FOR
L OW D ELAY (LD) S ETTING W ITH M AIN AND M AIN 10
C ONFIGURATIONS
Therefore, c is set as 0.001 for all combinations of RA and LD
settings with Main and Main 10 configurations.
The coding losses and encoding times for RA setting with Main
and Main10 configurations are shown in Table I while the coding
losses and encoding times for LD setting with Main and Main10
configurations are shown in Table II. The proposed residual-based
pruning method reduces encoding time by an average of 44% with
an average of about 1.0% coding loss.
The simulation results of “Early SKIP Decision (ESD)”
method [13], [14], “Coding-Block-Flag Fast Condition (CFM)”
method [15], “Early CU Condition (ECU)” method [16], [17] and the
proposed residual-based pruning method are summarized in Table III.
For a clearer illustration of the performance-complexity trade-offs, a
plot of coding losses against encoding times for the RA and LD
with Main configuration is also shown in Fig. 5. For both RA and
LD settings, it can be seen that the operating points of the proposed
and “ECU” methods lie on the convex hull of the operating points.
This indicates that, as compared to the “CFM” and “ESD” methods,
the proposed and “ECU” methods are able to achieve more compet-
itive complexity-performance trade-offs. This could be attributed to
a multi-depths pruning approach for the proposed and “ECU” meth-
ods as compared to a single-depth pruning approach for the “CFM”
and “ESD” methods. As compared to the “ECU”, while the proposed
method suffered additional coding losses, it further reduced encoding
time by almost 10%. The proposed method is useful when encoding
time reduction is more critical than bandwidth reduction, such as in
low-delay video conferencing applications.
It should be noted that the encoding time reductions shown in
Fig. 5 result from both the encoding time overheads to compute the
analysis algorithms and the encoding time savings due to the prun-
ing algorithms. The complexity of the analysis algorithms of “ESD”,
“ECU”, and “CFM” are low, since these analysis algorithms only
involve checking of coding-block-flags, SKIP mode, and motion vec-
tor differences. Similarly, the complexity of our analysis algorithm
is low, as only simple second order prediction residuals statistics are
used. Therefore, the encoding time reductions of all these methods
largely indicate the efficiencies of the pruning algorithms.
132
TABLE III
C ODING L OSSES (L UMA BD-R ATE ) AND E NCODING T IMES (E NC T IME ) FOR A LL C OMBINATIONS OF R ANDOM
ACCESS (RA) AND L OW D ELAY (LD) S ETTINGS W ITH M AIN AND M AIN 10 C ONFIGURATIONS
Fig. 5. Plot of coding loss vs encoding time for various fast encoding methods
under Random Access (RA) Main and Low Delay (LD) Main configurations.
To better understand the encoding time overheads of our method,
we turn on the prediction residuals statistics analysis algorithm
but turn off the pruning algorithm at every CU. The encoding
time increase is close to 0%, suggesting that the prediction resid-
uals statistics analysis algorithm incurs insignificant encoding time
overhead.
We now discuss other prior fast encoding methods which are
not present in the reference software. As these prior methods are
proposed based on earlier HM versions and are evaluated using dif-
ferent test conditions, the performance-complexity operating points
of these prior methods may not be directly comparable to the
performance-complexity operating points of our proposed method.
Nonetheless, as the source codes of these prior methods are not
publicly available, evaluation of these prior methods on HM16.3
is not viable. In general, these prior methods reported about 40%
encoding time reduction, with the Bayesian method [23] and the
IEEE TRANSACTIONS ON BROADCASTING, VOL. 62, NO. 1, MARCH 2016
“ADRD” method [21] reporting encoding time reduction of over 40%.
The reported encoding time reduction of the Bayesian method
and the “ADRD” method are respectively 41% and 42%. Nonetheless,
the reported coding loss of the Bayesian method is close to a high 2%.
As for the “ADRD” method, the reported coding loss is 1.3%, which
is similar/slightly higher than ours. When making pruning decision
for a CU, different from the “ADRD” method which involves statistics
from the current CU and its spatially and temporally neighbouring
CUs, our proposed method only involves prediction residuals statis-
tics from the PUs directly within the current CU. Therefore, our
proposed method could be more parallel processing friendly due to
its independency on neighboring CUs.
VI. C ONCLUSION
HEVC is important to the economical adoption of the emerg-
ing high quality digital visual content formats. Nonetheless, further
encoding time reduction is necessary for its use in real-time appli-
cations. In this paper, we proposed a method for reducing the high
encoding time of HEVC, by pruning its coding quad-trees using pre-
diction residuals statistics. Our experimental results show that the
proposed residual-based pruning method effectively reduces redun-
dant mode and partition checks; encoding time is reduced by an
average of about 44% with an average of about 1.0% coding loss. By
using only simple second order prediction residuals statistics from
a current CU independently of its spatially and temporally neigh-
bouring CUs, the proposed method introduces little modifications
and insignificant computational complexity overheads to the existing
encoder.
In the future, we will investigate ways for reducing unneces-
sary symmetric motion partition (SMP) and asymmetric motion
partition (AMP) checks.
ACKNOWLEDGMENT
The main author would like to thank Dr. Chuohao Yeo and
Dr. Yih Han Tan for their suggestions.
R EFERENCES
[1] High Efficiency Video Coding, document ITU-T Rec. H.265, Oct. 2014.
[2] F. Bossen, B. Bross, K. Suhring, and D. Flynn, “HEVC complexity
and implementation analysis,” IEEE Trans. Circuits Syst. Video Technol.,
vol. 22, no. 12, pp. 1685–1696, Dec. 2012.
[3] Advanced Video Coding for Generic Audio-Visual Services, document
ITU-T Rec. H.264 and ISO/IEC 14496-10 (AVC), ITU-T and ISO/IEC
JTC 1, May 2003.
[4] I. E. Richardson, H.264 and MPEG-4 Video Compression: Video Coding
for Next-Generation Multimedia. Chichester, U.K.: Wiley, 2004.
[5] J. Vanne, M. Viitanen, T. D. Hamalainen, and A. Hallapuro,
“Comparative rate-distortion-complexity analysis of HEVC and AVC
video codecs,” IEEE Trans. Circuits Syst. Video Technol., vol. 22, no. 12,
pp. 1885–1898, Dec. 2012.
IEEE TRANSACTIONS ON BROADCASTING, VOL. 62, NO. 1, MARCH 2016
[6] F. Bossen, Common HM Test Conditions and Software Reference
Configurations, document JCTVC-L1100, JCT-VC, Geneva,
Switzerland, Jan. 2013.
[7] A. Ortega and K. Ramchandran, “Rate-distortion methods for image
and video compression,” IEEE Signal Process. Mag., vol. 15, no. 6,
pp. 23–50, Nov. 1998.
[8] G. J. Sullivan and T. Wiegand, “Rate-distortion optimization for video
compression,” IEEE Signal Process. Mag., vol. 15, no. 6, pp. 74–90,
Nov. 1998.
[9] C. Grecos and M. Y. Yang, “Fast inter mode prediction for P slices in
the H264 video coding standard,” IEEE Trans. Broadcast., vol. 51, no. 2,
pp. 256–263, Jun. 2005.
[10] J. Hou, S. Wan, Z. Ma, and L.-P. Chau, “Consistent video quality control
in scalable video coding using dependent distortion quantization model,”
IEEE Trans. Broadcast., vol. 59, no. 4, pp. 717–724, Dec. 2013.
[11] Y. Wang, L.-P. Chau, and K.-H. Yap, “Bit-rate allocation for broadcast-
ing of scalable video over wireless networks,” IEEE Trans. Broadcast.,
vol. 56, no. 3, pp. 288–295, Sep. 2010.
[12] K. McCann et al., High Efficiency Video Coding (HEVC) Test Model
16 (HM16) Encoder Description, document JCTVC-R1002, JCTVC,
Sapporo, Japan, Jul. 2014.
[13] J. Kim, J. Yang, K. Won, and B. Jeon, “Early determination of mode
decision for HEVC,” in Proc. Pict. Coding Symp. (PCS), Kraków,
Poland, May 2012, pp. 449–452.
[14] J. Yang, J. Kim, K. Won, H. Lee, and B. Jeon, Early SKIP Detection
for HEVC, document JCTVC-G543, JCT-VC, Geneva, Switzerland,
Nov. 2011.
[15] R. H. Gweon, Y.-L. Lee, and J. Lim, Early Termination of CU Encoding
to Reduce HEVC Complexity, document JCTVC-F045, JCT-VC, Turin,
Italy, Jul. 2011.
[16] K. Choi, S. H. Park, and E. S. Jang, Coding Tree Pruning Based
CU Early Termination, document JCTV-F092, JCT-VC, Turin, Italy,
Jul. 2011.
133
[17] K. Choi and E. S. Jang, “Fast coding unit decision method based
on coding tree pruning for high efficiency video coding,” Opt.
Eng., vol. 51, no. 3, 2012, Art. ID 030502. [Online]. Available:
http://dx.doi.org/10.1117/1.OE.51.3.030502
[18] K. McCann, B. Bross, S.-I. Sekiguchi, and W.-J. Han, High Efficiency
Video Coding (HEVC) Test Model 2 (HM 2) Encoder Description,
document JCTVC-D502, JCT-VC, Daegu, Korea, Oct. 2010.
[19] J. Kim, S. Jeong, S. Cho, and J. S. Choi, “Adaptive coding unit early
termination algorithm for HEVC,” in Proc. IEEE Int. Conf. Consum.
Electron. (ICCE), Las Vegas, NV, USA, Jan. 2012, pp. 261–262.
[20] H. L. Tan, F. Liu, Y. H. Tan, and C. Yeo, “On fast coding tree block
and mode decision for high-efficiency video coding (HEVC),” in Proc.
IEEE Int. Conf. Acoust. Speech Signal Process. (ICASSP), Kyoto, Japan,
Mar. 2012, pp. 825–828.
[21] J. Leng, L. Sun, T. Ikenaga, and S. Sakaida, “Content based hierarchi-
cal fast coding unit decision algorithm for HEVC,” in Proc. Int. Conf.
Multimedia Signal Process. (CMSP), vol. 1. Guilin, China, May 2011,
pp. 56–59.
[22] L. Shen, Z. Liu, X. Zhang, W. Zhao, and Z. Zhang, “An effective CU
size decision method for HEVC encoders,” IEEE Trans. Multimedia,
vol. 15, no. 2, pp. 465–470, Feb. 2013.
[23] X. Shen, L. Yu, and J. Chen, “Fast coding unit size selection for HEVC
based on Bayesian decision rule,” in Proc. Pict. Coding Symp. (PCS),
Kraków, Poland, May 2012, pp. 453–456.
[24] Y. H. Tan, C. Yeo, H. L. Tan, and Z. Li, “On residual quad-tree cod-
ing in HEVC,” in Proc. IEEE 13th Int. Workshop Multimedia Signal
Process. (MMSP), Hangzhou, China, Oct. 2011, pp. 1–4.
[25] H. R. Wu and K. R. Rao, Digital Video Image Quality and Perceptual
Coding (Signal Processing and Communications). Boca Raton, FL,
USA: CRC Press, 2005.
[26] G. Bjontegaard, Calculation of Average PSNR Differences Between RD
Curves, document VCEG-M33, ITU-T Q6/SG16, Austin, TX, USA,
Apr. 2001.