Professional Documents
Culture Documents
21 Candidate Position Subsampling Technique
21 Candidate Position Subsampling Technique
21 Candidate Position Subsampling Technique
7, JULY 2010
Abstract—The candidate position subsampling technique lower bounds than the SAD, this method is quicker. The
(CPST) basically chooses candidates in a search window at a third method is to use a pixel-subsampling technique [5]–[7].
sampling rate. The 2:1 CPST chooses half the candidates, and
When calculating the SAD, not all pixels are considered, but
then selects one or more candidates that are considered as to be
close to the optimal motion vector before conducting a fine search. pixels on the subsampling pattern are considered. This method
The fine search is conducted by checking four neighbors of the guarantees an improvement in speed by as much as the rate
chosen candidate(s) referred to as winner(s). The CPST can be of pixel-subsampling.
combined with a fast optimal block-matching algorithm, such as The third method is usually combined with the candidate
the multilevel successive elimination algorithm (MSEA), in order
position subsampling technique (CPST) [5], [7]. The CPST
to reduce the number of computations used in rejecting the non-
best candidate. We propose a new 2:1 CPST fitted to the MSEA. consists of three steps. First, candidates for the optimal
The proposed algorithm adopts a new condition for the winner motion vector (MV) are selected by subsampling. Second,
which helps to find the best candidate efficiently. Moreover, a one or more candidates that are considered to be close to the
fast motion estimation step is used to reduce the number of optimal MV are selected from candidates that are chosen in
computations of the MSEA, and the peak signal-to-noise ratio
the first step. The candidates that are selected in the second
(PSNR) compensation step is adopted to guarantee that the PSNR
performance of the proposed algorithm is very close to that of step are called winners. Finally, neighbors of each winner are
the full search. Experimental results show that the proposed checked in the fine search step.
algorithm reduces the computational loads of the MSEA by Recently, there has been an attempt to combine the ad-
47.26% on average with only −0.027 dB PSNR degradation in justable partial distortion search (APDS) algorithm (an algo-
the worst case.
rithm that uses the second method of using lower bounds
Index Terms—Block matching, candidate position, fast full of the SAD [3]) and the CPST [8]. The first and second
search, motion estimation, subsampling, successive elimination. candidates that have the minimum SAD are selected as win-
I. Introduction ners from candidates to which the 2:1 subsampling pattern
is applied. Finally, the four neighbors of each winner are
M OTION estimation via a block-matching algorithm
(BMA) has been used in almost all the video coding
standards, including ITU-T H.263/H.264 and ISO/IEC MPEG-
checked. However, this algorithm does not perform well if the
multilevel successive elimination algorithm (MSEA) [4], one
of the most well-known algorithms using the second method,
1/2/4. Although the full-search (FS) algorithm is the optimal
is used instead of the APDS. This results mainly from the
BMA with regard to motion estimation accuracy, its computa-
inadequate specification of criteria for selecting the winners.
tional complexity limits practical usage. Hence, many BMAs
In this letter, we propose a 2:1 candidate position subsam-
have been developed to reduce the complexity. The methods
pling technique that is used with the MSEA. The proposed
used in BMAs can be divided into three types.
algorithm shows almost the same peak signal-to-noise ratio
The first method is to use only a subset of all the search
(PSNR) performance as a FS algorithm (in the worst case,
locations in the search window [1], [2]. This method reduces
−0.027 dB degradation) with only almost half the computa-
computational complexity because the sum of absolute differ-
tions of the MSEA. This performance is achieved by adopting
ence (SAD) is not computed for all possible candidates. The
a new condition for the winner, the fast motion estimation,
second method is to adopt lower bounds for the SAD [3], [4].
and the PSNR compensation steps.
Before calculating the SAD of the current candidate, lower
bound values are obtained, and the necessity of calculating II. Related Work
the SAD is determined by comparing lower bound values with
the minimum SAD obtained from the previous candidates. The APDS algorithm [3] and the MSEA [4] are both known
Given that fewer computations are required to compute the to be fast optimal block-matching algorithms, which means
that the algorithm can achieve the same PSNR performance
Manuscript received September 26, 2008; revised August 7, 2009 and as the FS algorithm, but with fewer computations. The basic
December 30, 2009. Date of publication May 27, 2010; date of current version
July 16, 2010. This paper was recommended by Associate Editor H. Yu. procedure of the APDS and the MSEA is to calculate lower
The authors are with the School of Electrical Engineering and Com- bounds (also called boundary values) from the lowest to
puter Science, Korea Advanced Institute of Science and Technology, Dae- the highest and to check whether each boundary value is
jeon 305-701, Korea (e-mail: seagall@kaist.ac.kr; honestok@gmail.com; dj-
park@ee.kaist.ac.kr). larger than the minimum SAD value that was obtained from
Digital Object Identifier 10.1109/TCSVT.2010.2051365 previously searched positions. This value will be referred to as
1051-8215/$26.00
c 2010 IEEE
LEE et al.: 2:1 CANDIDATE POSITION SUBSAMPLING TECHNIQUE FOR FAST OPTIMAL MOTION ESTIMATION 1053
TABLE I
Algorithm 1: Pseudo code for the APDS and the MSEA
Number of Cases in Which the Optimal MV Is Missed With
(x, y) = initial position Different Conditions
Find SAD of (x, y)
SAD min = SAD of (x, y) Name Format Total Condition 1 Condition 2 Condition 3
(x, y) = next position Mobile CIF 12 586 1222 466 285
while !(last candidate position) do Foreman CIF 9650 2754 1792 536
for k=1 to number LB do Fun CIF 7106 622 233 219
Compute LB[k] Paris CIF 3093 325 116 59
if LB[k]>SAD min then Mobile CCIR601 15 362 2521 1304 800
go to (#) Football CCIR601 18 277 2309 946 1010
Flower CCIR601 16 908 2621 1756 1447
end
Tennis CCIR601 8178 5556 4680 2985
end Average 11395.0 2241.3 1411.6 917.6
SAD min = LB[number LB] Probability of Missing MV 0.197 0.124 0.081
MV = (x, y)
(#) (x, y) = next position Condition 1: Choose the one candidate with the minimum SAD. Condition 2: Choose the
end two candidates with the first and second minimum SADs. Condition 3: Choose candidates
updating the temporary minimum SAD.
· LB: Lower bound, MV: Motion vector, (x, y) : Current candidate
Fig. 1. Search patterns. (a) 2:1 candidate subsampling pattern. (b) Fine
search pattern.
Fig. 2. Sample plot for the proposed condition in 1-D space.
the temporary minimum SAD. If a certain lower bound of the III. Proposed 2:1 Candidate Position Subsampling
current position is larger than the temporary minimum SAD, Technique for the MSEA
it is deemed unnecessary to compute the SAD of the current
position. This procedure is presented in Algorithm 1. A. New Condition for Winner
The main difference between the APDS and the MSEA The purpose of a fine search is to find the optimal MV in
is the way of obtaining lower bounds. The APDS calculates the case that the optimal MV is not in the pattern yielded by
boundary values by using the partial distortion technique subsampling. Therefore, the best winner out of the candidates
which is derived from the fact that the SAD value for the selected by subsampling is the one that has the optimal MV
N × N block size is always larger than the partial sum of as a neighbor if the fine search checks four neighbors of
the absolute value for difference between pixel-values in the the winner, as depicted in Fig. 1(b). If the correlation between
current and reference blocks. By contrast, the MSEA obtains the SAD values of the optimal MV and its neighbors is high,
boundary values by using the triangle inequality as a basis: the candidate selected by subsampling that has the minimum
|a + b| ≤ |a| + |b|. SAD is likely to be the best winner. In most cases, it is
The CPST addresses the issue of which candidates are acceptable that the correlation is high, because the maximum
to be checked and the search order of selected candidates, distance between the winner and a neighbor is just one
while the APDS and the MSEA are the candidate-rejection pixel.
algorithms (CRAs) that correspond to the inside of a while- Our experiments show that when the condition of selecting a
loop in Algorithm 1. In [8], the CPST consists of three steps candidate that has the minimum SAD is used, the probability
with the APDS as the CRA. In the first step, candidates are of missing the optimal MV is 0.197, as shown in Table I,
searched using the candidate subsampling pattern, as depicted where the total number (Total) stands for the number of cases
in Fig. 1(a). In the second step, two candidates with the first that the optimal MV is not in the subsampling pattern. The
and second minimum SAD values are selected as winners. simulation was executed using the same conditions as those
Finally, the four candidates nearest neighbors to each winner in Table III.
are searched, as in Fig. 1(b). Missing the optimal MV means degradation of the PSNR
Given that, in the second step, two candidates with the first performance. To improve the PSNR performance, the condi-
and second minimum SAD values are selected, Algorithm 1 tion for selecting the winner needs to be changed. One way is
should be changed slightly. A noteworthy change is that a to choose multiple winners using the increasing order of the
certain lower bound is compared not with the first minimum SAD values as a basis. For instance, two candidates with the
SAD but with the second minimum SAD. This modification first and second minimum SAD are selected as winners in [8].
was also mentioned in [8]. The condition used in [8] reduces the probability of missing
1054 IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 20, NO. 7, JULY 2010
TABLE III
Computations and PSNR Performance Comparison of Different Algorithms
Mobile (CIF) Foreman (CIF) Fun (CIF) Paris (CIF)
Total PSNR to FS Total PSNR to FS Total PSNR to FS Total PSNR to FS
FS 668516.3 0.000 668516.3 0.000 668516.3 0.000 668516.3 0.000
MSEA 33416.5 0.000 41878.2 0.000 38361.9 0.000 15265.40 0.000
Conv CPST 32919.3 −0.090 33540.9 −0.336 29052.0 −0.013 12237.9 −0.008
Prop CPST1 28986.4 −0.020 29248.2 −0.030 25521.9 −0.018 11488.9 −0.002
Prop CPST2 30999.7 −0.019 29345.5 −0.019 27157.9 +0.001 11707.2 +0.005
Proposed 20270.8 −0.027 24708.8 −0.018 21028.3 +0.003 10555.5 +0.001
Mobile (CCIR601) Football (CCIR601) Flower (CCIR601) Tennis (CCIR601)
Total PSNR to FS Total PSNR to FS Total PSNR to FS Total PSNR to FS
FS 2636526.3 0.000 2636526.3 0.000 2636526.3 0.000 2636526.3 0.000
MSEA 113322.0 0.000 216924.0 0.000 131796.0 0.000 823889.0 0.000
Conv CPST 137153.0 −0.074 145081.0 −0.010 106872.0 −0.023 496364.0 −0.194
Prop CPST1 73229.1 −0.014 126763.0 −0.016 78660.1 −0.014 459910.0 −0.054
Prop CPST2 77345.7 −0.010 140055.0 +0.073 82295.7 +0.002 482898.0 +0.004
Proposed 60985.1 −0.015 113452.0 +0.075 61163.8 +0.008 434055.0 +0.032
if any MVs in the three MVs of the left, top, and top-right minimum SAD value (Condition 3). The proposed condition
blocks of a current block for predicting the optimal MV is does not change the basic procedure in Algorithm 1, but is able
not available, it is replaced with (0, 0). All versions of the to find winners effectively. Adopting a new winner condition
proposed algorithm adopt MSEA as the CRA. yields a 16.05% reduction in computation and an improvement
The test platform for experiments was based on a simple in PSNR of +0.073 dB over Conv CPST. It is noticeable
code only to perform motion estimation. Search order for that the worst degradation of PSNR performance is reduced
candidate subsampling is in Fig. 1(a). Simulations were per- from −0.336 dB (Foreman in CIF) to −0.054 dB (Tennis in
formed in eight sequences with the two different formats: CIF CCIR601) by using a new winner condition.
(352 × 288, 50 frames, 25 frames/s) and CCIR601 (720 × 480, 2) PSNR Compensation (Prop CPST2): The PSNR Com-
30 frames, 30 frames/s) [11]. Only luminance components of pensation step is performed if SAD(MVFS ) is larger than
each sequence were used. The block size was set to 16 × 16. SADmax , in Fig. 3. Our simulation shows that the probability
The search range to ±15 for CIF and ±30 for CCIR601. of conducting the PSNR compensation step is 0.1358. It is
We set one operation of addition, subtraction, absolute value, natural to think that the PSNR compensation increases the
comparison, and shift each to equal one operation. computational load of Prop CPST1 by 13.58%. However, it
requires only 5.76% additional computations, giving a PSNR
B. Experimental Results improvement of +0.026 dB, because some area of the new
1) New Condition for Winner (Prop CPST1): Even search window overlaps with the original search window and
though Conv CPST subsamples candidate positions at a 2:1 the temporal minimum SAD value is already very small.
rate, an ineffective condition for choosing the winner results in It is remarkable that the PSNR compensation step reduced
slower performance. Conv CPST requires more computations the worst PSNR degradation from −0.054 dB (Tennis in
even than the MSEA for the Mobile sequence in CCIR601. CCIR601) to −0.019 dB (Mobile and Foreman in CIF).
This is because the fact that lower bounds are compared, not 3) Fast Motion Estimation (Proposed): The proposed
with the first minimum SAD, but with the second minimum algorithm applies the fast ME step to Prop CPST2 to reduce
SAD, while the MSEA always compares lower bounds with the number of computations by having the lower temporal
the first minimum SAD. minimum SAD value for the same candidate. On average the
Prop CPST1 only changes the condition for winner in proposed algorithm requires 15.38% fewer computations than
the Conv CPST (Condition 2 in Table I) into the proposed Prop CPST2. However, there is a small difference between
condition of whether or not a MV updates the temporal the PSNR performances of Prop CPST2 and the proposed al-
1056 IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 20, NO. 7, JULY 2010