H264/AVC

Video Coding Standard

Nhóm 8: Nguyễn Hà Thu
Nguyễn Tiến Thành
School of Electronics and Telecommunications
Hanoi University of Science and Technology

3/29/19 1
REFERENCES
• H.264/MPEG-4 AVC Video Compression Tutorial [1]
• Overview of the H.264/AVC Video Coding Standard [2]
• ITU-T Recommendation H.264: Advanced video coding for
generic audiovisual services [3]
• P. List, A. Joch, J. Lainema, G. Bjøntegaard, and M. Karczewicz,
“Adaptive deblocking filter,” IEEE Trans. Circuits Syst. Video
Technol., vol.13, pp. 614–619, July 2003. [4]
• H. Malvar, A. Hallapuro, M. Karczewicz, and L. Kerofsky, “Low-
Complexity transform and quantization in H.264/AVC,” IEEE
Trans. Circuits Syst. Video Technol., vol. 13, pp. 598–603, July
2003. [5]

3/29/19 2
INTRODUCTION

History of Video Standard

3/29/19 3
Input Video Signal
• A video frame can be considered
contains two interlaced fields:
- Top field: row 0, 2, 4, …
- Bottom field: row 1, 3, 5, …
• If the two fields of a frame were
captured at different time
instants, the frame is referred to
as an interlaced frame,
otherwise it is referreas a
progressive frame.
• In H264/AVC, the coding
representation based on
geometric concepts rather than
on timing concepts.
3/29/19 4
Partioning of the Pictures
• Slice
- A picture can be split into one
or several slices.
- Slices are self contained. It can
be correctly decoded without
using data from the other slice.
- It is a sequence of
macroblocks.

• Macroblock
- It is the basic processing unit.
- Contains 16x16 luma samples
and two 8x8 chroma samples.

3/29/19 5
Overview Block Diagram

3/29/19 6
WORK ASSIGNMENT

Name Works
Nguyễn Hà Thu Transform, Quantization,
Intra Prediction, Deblocking Filter
Nguyễn Tiến Thành Inter Prediction (Motion Estimation
+ Motion Compensation)

3/29/19 7
Macroblock coding structure

3/29/19 8
TRANSFORM

3/29/19 9
TRANSFORM
• Transform

• Inverse Transform

3/29/19 10
QUANTIZATION
• Quantization:
x = QPmod6
MF = [ a c a c
cbcb
acac
c b c b]
a = M(x, 0)
b = M(x, 1)
c = M(x, 2)
3/29/19 11
INTRA PREDICTION
• Intra prediction is used to
eliminate spatial
redundancies.
• It attempts to predict the
current block by
extrapolating the neighboring
pixels from adjacent blocks
in a defined set of different
directions.
• Each slice can be predicted Intra 4x4 prediction conducted
by using Intra 4x4 or Intra using samples A-Q
16x16 prediction.

3/29/19 12
INTRA PREDICTION
Intra 4x4
• There are 9 Intra 4x4 modes:
- Eight modes coresponding to
eight directions.
- DC mode.

Eight prediction direction of

Intra 4x4 prediction

3/29/19 13
INTRA PREDICTION
• Intra 16x16
- Performs on the whole 16x16 block
- Is more suited for coding very smooth areas of a picture.
• The chroma samples of a macroblock are predicted using
a similar prediction.

3/29/19 14
DEBLOCKING FILTER
• Block based coding accidentally cause visible artifacts by
two main reasons:
- Coarse quantization of the transform coefficient cause
disturbing discontiniutis at the block boundaries.
- Motion compensated block generated by inter prediction.
-> Need the filter to remove the artifacts.
• Deblocking filter can be used as post filter and in loop filter.
- Post filters operate on buffer to display video (optional).
- In loop filters operate in the loop coding (compulsory).

3/29/19 15
DEBLOCKING FILTER
• The purpose: distinguish the true edges and those
created by quantization of the DCT coefficients.
• The basic idea:
- If a relatively large absolute difference between
samples near a block edges is measured, it is quite
likely a blocking artifact.
- If the magnitude of that difference is so large,
the edge is more likely be the true edges

3/29/19 16
DEBLOCKING FILTER

Filter strength depends on coding mode

3/29/19 17
DEBLOCKING FILTER
•Principle:

p0 and q0, p1 and p1 should be filtered

or not depend on these conditions:

with .
q2 and p2 be filtered if

3/29/19 18
DEBLOCKING FILTER
• and are threholds, denpend on the quantization
parameter QP:

ÞWith QP <= 16, and are set to 0, the filter is

turn off.

3/29/19 19
DEBLOCKING FILTER
• Filtering process:
- In a frame, marcroblock are filtered in the raster
scan order.
- In a macroblock, vertical edges are filtered first, then
horizontal edges.

3/29/19 20
DEBLOCKING FILTER
• Filtering for BS = 1 to 3
If mentioned conditions are true, the sample values will be
replaced by the filtered value:

• The value of is calculated in a two-step process, with the

calculation of an initial value , followed by clipping of this
value before it is applied in the above equations.

• p1 will be filtered by:

3/29/19 21
DEBLOCKING FILTER
• Filtering for BS = 1 to 3
• is also calculated in a two-step process.
• The initial value is computed as:

• and then obtained by limit value:

With c1 is depending on BS and QP.

c0 is set by c1 then increment by 1 when p1/q1 is
filterd
3/29/19 22
DEBLOCKING FILTER
• Filtering for BS = 4
Depending the image content, a stronger or weaker filter
will be applied.
If this contrain hold true:
The stronger filter will be used.

Else only p0 be modified

3/29/19 23
DEBLOCKING FILTER

3/29/19 24
INTER- PREDICTION

Block diagram of H264/AVC encoder with motion compensation

3/29/19 25
MOTION ESTIMATION
• Use to reduce temporal redundancy between
consecutive frame
• Partition with variable block shape and block size,
small block 4x4
• Quarter-pel accuracy of motion vector

3/29/19 26
MOTION ESTIMATION
• Block Matching Algorithm (BMA)

• Identify a block from the reference

frame (in search region) that best
match the current frame block Current Frame

Motion Vector (MV) consist of a

pair (MVx, MVy) for horz. and vert.

• BMA composed of:

 Matching criterion
Reference Frame
 Searching strategy
3/29/19 27
MATCHING CRITERION
• Block Distortion Measure (BMD)
• Mean Square Difference (MSD or MSE):

• Sum of Square Difference (SSD or SSE):

3/29/19 28
MATCHING CRITERION
• Mean Absolute Difference (MAD or MAE):

• Sum of Absolute Difference (SAD or SAE):

3/29/19 29
SEARCHING STRATEGY
• Optimal search algorithm:
• Full Search (FS)
• Successive Elimination Algorithm (SEA)

• Sub-optimal search algorithm:

• Three Step Search (TSS)
• Four Step Search (4SS)
• Diamond Search (DS)

• Adaptive search algorithm:

• UMHexagonS
• Enhanced Predictive Zonal Search (EPZS)

3/29/19 30
Full Search (FS)
• Or Exhaustive Search (ES) is a brute force
method to find motion vector
• Search all the (2W+1)2 candidate block position
in the search window of size W
• Give the optimum quality of compression
• But quite slow and require more computation
time

3/29/19 31
 Successive Elimination Algorithm
•• A type of fast full search algorithm
• Use the Minkowski inequality in order to calculate the best
match faster:

• If the difference between the block sum of candidate Y and

block sum of reference block X is greater than the
minimum SAD block Y must not the best match
• Calculating the block sum difference is much faster than
calculating the SAD
3/29/19 32
Three Step Search (TSS)
• Use a 9x9 grid with 8 check point and
initial step size
• Search for the point having minimum
BDM among all the eight points
• After that, the center is moved to point
of minimum BDM and the step size is
halved
• The search terminate when minimum
Search Path for TSS
BDM at center point or cannot
decrease step size
3/29/19 33
Four Step Search (4SS)
• Use the center-biased search pattern with a nine checking
point in a 5x5 window
• The center of the search window is then shifted to
minimum BDM point
• If the minimum BDM point is center of the search window
 go to final step
• Finally, use a 3x3 search
window, motion vector
correspond to the
minimum BDM point
5x5 pattern 3x3 pattern
3/29/19 34
Diamond Search (DS)
• Use two search patterns:
• Large Diamond Search Pattern (LDSP): 9 check point
• Small Diamond Search Pattern (SDSP): 5 check point

• Searching procedure:
• Use LDSP pattern repeatedly until the minimum BDM point
at the center
• Switch to SDSP, the point
with minimum BDM in
SDSP provides the best
match motion vector LDSP SDSP
3/29/19 35
Unsymmetrical-cross Multi-Hexagon-
grid Search (UMHexagonS)
• MV & SAD prediction modes:
• Median prediction:

3/29/19 36
Unsymmetrical-cross Multi-Hexagon-
grid Search (UMHexagonS)
• MV & SAD prediction modes:
• UpLayer prediction:

• Corresponding-block prediction:

3/29/19 37
Unsymmetrical-cross Multi-Hexagon-
grid Search (UMHexagonS)
• MV & SAD prediction modes:
• Neighboring reference-picture prediction:

3/29/19 38
Unsymmetrical-cross Multi-Hexagon-
grid Search (UMHexagonS)
• Cost Function:

with:

: Lagrange multiplier
: predicted vector for motion vector

3/29/19 39
Unsymmetrical-cross Multi-Hexagon-
grid Search (UMHexagonS)
• Start search point decision:

with:

3/29/19 40
Unsymmetrical-cross Multi-Hexagon-
grid Search (UMHexagonS)
• Step1: Unsymmetrical
Cross Search
• Step2: 5x5 Exhaustive
Search
• Step3: Multi-Hexagon
Grid Search
• Step4: Extended
Hexagon-based Search UMHexagonS Search Pattern, W = 16

3/29/19 41
Unsymmetrical-cross Multi-Hexagon-
grid Search (UMHexagonS)
• Step1: Unsymmetrical
Cross Search:

with:
(Start)

3/29/19 42
Unsymmetrical-cross Multi-Hexagon-
grid Search (UMHexagonS)
• Step2: 5x5 Exhaustive
Search:

with:
(Step1)

3/29/19 43
Unsymmetrical-cross Multi-Hexagon-
grid Search (UMHexagonS)
• Step3: Multi-Hexagon
Grid Search:

with:
k correspond with each hexagon

• Sixteen points hexagon pattern (16-HP) is used

• 16-HP hexagon is extended with different scale factor (k
from to 1 to W/4)
3/29/19 44
Unsymmetrical-cross Multi-Hexagon-
grid Search (UMHexagonS)
• Step4-1: Iterative
Hexagon Search:

with:
(Step3)

• If , go to Step4-2, else go to Step4-1

3/29/19 45
Unsymmetrical-cross Multi-Hexagon-
grid Search (UMHexagonS)
• Step4-2: Small Diamond
Search:

with:
(Step4-1)

• If , Stop, else go to Step4-2

3/29/19 46
FAST FRACTION-PEL MOTION
ESTIMATION
• Full Fractional Pel Search (FFPS)

• Paraboloid Prediction Fractional Pel Search

(PPFPS)

• Hierachical Fractional Pel Search (HFPS)

• Center Biased Fractional Pel Search (CBFPS)

3/29/19 47
Hierarchical Fractional Pel Search
(HFPS)
• Step 1: Check eight ½
pel position  best ½ pel
motion vector
• Step 2: Check eight ¼
pel position  best ¼ pel
motion vector
• Step 3: Check eight 1/8
pel position  best 1/8
HFPS algorithm
pel motion vector
3/29/19 48
Center Biased Fractional Pel
Search (CBFPS)
• Step 1: Predict
fractional-pel MV:

(= 4 for ¼ pel)
• Using diamond search
pattern
CBFPS algorithm

3/29/19 49
Center Biased Fractional Pel
Search (CBFPS)
• Step 2: Using Cost function
for (Pred_x, Pred_y) and 4
vertex of diamond
• Step 3: If the minimum
Cost point is at the center,
go to Step 4, else iterate
Step 2 & Step 3
• Step 4: Choose minimum
Cost point as MV CBFPS algorithm

3/29/19 50
MOTION COMPENSATION
• The accuracy of motion compensation is in unit
of one quarter of the distance between samples
• If motion vector points to an integer-sample
position  prediction signal corresponding
samples of the reference frame
• Otherwise, motion vector points to non integer
samples  using interpolation to generate
noninteger position
3/29/19 51
MOTION COMPENSATION
• mvLX: luma motion vector given in quarter-luma-
sample units
• (xAL , yAL): location given in full-sample units of
the upper-left of the current partition
• (xIntL , yIntL): luma location given in full-sample
units
• (xFracL , yFracL): offset given in quarter-sample
units
3/29/19 52
MOTION COMPENSATION
• For each luma sample location (xL,yL) inside
partition:

• Corresponding predicted luma sample value is

predLXL[xL, yL]
3/29/19 53
MOTION COMPENSATION
• The sample at half
sample position (b, h)
is derived by applying
a 6-tap filter to the
nearest integer
position sample in the
horizontal and vertical
direction

3/29/19 54
MOTION COMPENSATION
•

3/29/19 55
MOTION COMPENSATION
•The sample at half
sample position j is
derived by:

or

3/29/19 56
MOTION COMPENSATION
•The sample at quarter
sample (a, c, d, n, f, i, k, q)
position is derived by:

3/29/19 57
MOTION COMPENSATION
•The sample at quarter
sample (e, g, p, r)
position is derived by:

3/29/19 58
MOTION COMPENSATION
• Luma location offset in fractional-sample units
(xFracL , yFracL) specifies predicted luma
sample value predPartLXL according below
table

3/29/19 59