Professional Documents
Culture Documents
Digital Video Compression Fundamentals and Standards
Digital Video Compression Fundamentals and Standards
Outline
Introduction
Video Compression Standards
Simulation Reference Software
Future Work and Conclusions
2008/12/26
Outline
Introduction
Video Compression Standards
Simulation Reference Software
Future Work and Conclusions
2008/12/26
Introduction (1/2)
Why video compression technique is
important ?
One movie video without compression
720 x 480 pixels per frame
30 frames per second
Total 90 minutes
Full color
The total quantity of data = 167.96 G Bytes !!
2008/12/26
Introduction (2/2)
What is the difference between video
compression and image compression?
Temporal Redundancy
Interframe Coding
Remove temporal redundancy
2008/12/26
N 1 N 1 N 1
8
(2 x 1)u
(2 y 1)
v
F ( x, y, t ) 3 C (u )C (v)C ( w) ( x, y, t ) cos
cos
N
2 N
2 N
t 0 x 0 y 0
cos
(2t 1) w
2 N
where N 8 and C ( k )
2008/12/26
2008/12/26
Matching Function
The dissimilarity D( s, t ) between two blocks n and n1
p
D( s, t ) M [ n( x, y ), n 1( x Vx, y Vy )]
Vy 1 Vx , 1
M (u , v) (u - v) 2
M (u, v) | u - v |
Current Frame
Motion
Vector
|A|=12
12 22 33 44 55 66 77 88
12 22 33 44 55 66 77 88
12 22 33 44 55 66 77 88
12 22 33 44 55 66 77 88
11 23 34 44 55 66 77 88
11 23 34 44 55 66 77 88
Search
11 23 34 44 55 66 77 88
Range
11 23 34 44 55 66 77 88
2008/12/26
11 22 33 44 55 66 77 88
11 22 33 44 55 66 77 88
11 22 33 44 55 66 77 88
11 22 33 44 55 66 77 88
11 22 33 44 55 66 77 88
11 22 33 44 55 66 77 88
11 22 33 44 55 66 77 88
11 22 33 44 55 66 77 88
1
1
1
1
0
0
0
0
0
0
0
0
1
1
1
1
0
0
0
0
1
1
1
1
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
720 480
16
16 30 170 G instructions/sec
128 128
16 16
2008/12/26
Current Frame
10
2008/12/26
11
2
4
2008/12/26
12
2
2
2008/12/26
2
3 3 3
1 3 2 3
3 3 3
2
2
13
(
x
d
2,l
l
1,l
xl
%
The new motion vector is dl (x) d l (x) ql (x)
2008/12/26
14
2008/12/26
15
2008/12/26
16
Outline
Introduction
Video Compression Standards
Simulation Reference Software
Future Work and Conclusions
2008/12/26
17
2008/12/26
18
19
20
GOP
I
2008/12/26
N=9
M=3
Bidirectional Motion
Compensation
Digital Video Compression Fundamentals an
21
Residue
Entropy
Coding
Inter-frame
Q-1
Motion
Compensation
Motion Vector
IDCT
Motion
Estimation
Frame
Memory
2008/12/26
22
D(t ) (t ) (t )
(t ) is the input image
^
DCT
N 1 N 1
2
(2 x 1)u
(2 y 1)
v
F (u , v) C (u )C (v) f ( x, y ) cos
cos
N
2 N
2 N
x 0 y 0
for u 0,..., N 1 and v 0,..., N 1
1/ 2 for k 0
where N 8 and C (k )
1 otherwise
2008/12/26
23
Qintra
16
19
22
22
26
26
27
16 19 22 26 27 29 34
16 17 18 19
16 22 24 27 29 34 37
17 18 19 20
18 19 20 21
22 26 27 29 34 34 38
22 26 27 29 34 37 40
19 20 21 22
Qintra
20 21 22 23
26 27 29 32 35 40 48
27 29 32 35 40 48 58
21 22 23 24
22 23 24 26
27 29 34 38 46 56 69
29 35 38 46 56 69 83
23 24 35 27
2008/12/26
20 21 22 23
21 22 23 24
22 23 24 25
23 24 26 27
25 26 27 28
26 27 28 30
27 28 30 31
28 30 31 33
24
( x, y )
n
is the compensated image
2008/12/26
Target frame
25
2008/12/26
26
2008/12/26
Interlaced Scan
27
2008/12/26
Luminance MB
28
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
2008/12/26
Zig-zag
1 1 1
0 0 0
1 1 1
0 0 0
1 1 1
0 0 0
1 1 1
0 0 0
scan
1 1
0 0
1 1
0 0
1 1
0 0
1 1
0 0
2D DCT
4
0.72
0
0.85
0
1.27
0
3.62
Alternate
0 0 0 0
0 0 0 0
0 0 0 0
0 0 0 0
0 0 0 0
0 0 0 0
0 0 0 0
0 0 0 0
scan
0 0
0 0
0 0
0 0
0 0
0 0
0 0
0 0
0
0
0
0
0
0
0
0
29
Qintra
16 16 16 16
8 16 19 22 26 27 29 34
16
16
22
24
27
29
34
37
16 16 16 16
16 16 16 16
19 22 26 27 29 34 34 38
22
22
26
27
29
34
37
40
Q 16 16 16 16
22 26 27 29 32 35 40 48 inter 16 16 16 16
26
27
29
32
35
40
48
58
16 16 16 16
26 27 29 34 38 46 56 69
16 16 16 16
27 29 35 38 46 56 69 83
16 16 16 16
16
16
16
16
16
16
16
16
16
16
16
16
16
16
16
16
16
16
16
16
16
16
16
16
16
16
16
16
16
16
16
16
30
Base Layer
Bits Enhance
Enhanced Layer
High quality service
For HDTV
Entropy
Coding
Q-1
DCT
Entropy
Coding
Q
Q-1
Q-1
Motion
Compensation
Bits Base
Base
Layer
IDCT
Motion
Estimation
Frame
Memory
2008/12/26
31
H.264/AVC
Variable Block Size
Multiple Reference Frames
Integer Transform
Intra Prediction
In-loop Deblocking Filtering
1/4-pel Resolution Motion Estimation
CAVLC
2008/12/26
32
16 x 8
0
8x8
1
8x4
0
2008/12/26
0
1
8 x 16
4x8
8x8
4x4
33
2008/12/26
34
2
1
C
1 1 1 1
1 2 2 1
44 Block Size
Separable Integer Transform
The transform coefficients are CXCT
2008/12/26
35
0.5
0.6325 0.3162 0.3162 0.6325
T
Y AXA
0.5
0.5
0.5
0.5
0.3162
0.6325
0.6325
0.3162
0.5
0.5
1 1 1 1
2 1 1 2
=
1 1 1 1
1 2 2 1
x00
x
10
x20
x01
x11
x21
x02
x12
x22
x30
x31
x32
x00
x01
x02
x10
x20
x30
x11
x21
x31
x12
x22
x32
1 1 1 1
2 1 1 2
=
1 1 1 1
1 2 2 1
C
2008/12/26
x00
x
10
x01
x11
x02
x12
x20
x21
x22
x30
x31
x32
0.5
x03
x13
x23
x33
x03
x13
x23
x33
0.5
0.5
0.3162
0.3162 0.5 0.6325
0.5 0.3162 0.5 0.6325
0.5
0.6325
0.5
0.3162
0.6325
0.5
0.3162
0.5
0.5
0.5
0.5
1 1 1 2 0.1581
0.1
0.1581
0.1
1 1 1 2
0.25 0.1581 0.25 0.1581
1 2 1 1
0.1
0.1581
0.1
0.1581
0.5
x03
x13
x23
x33
0.6325
CT
0.5
36
Intra Prediction
Predict the similarity between the neighboring pixels in one
frame in advance, and exploit differential coding transform
coding to remove the redundancy.
Intra
Prediction
Vertical
Transform/
Quantization
DC
+ + + +
Horizontal
+
+
+
+
Vertical right
2008/12/26
Entropy
Coding
Diagonal down-left
Diagonal right
Mean
Horizontal down
Vertical left
Horizontal up
37
QP
p3 p2 p1 p0
2008/12/26
38
2008/12/26
39
Coder
Controller
Transform/
Quantization
Motion
Compensation
Intra-frame
Prediction
Residue
Inverse
Transform/
De-Quantization
Entropy
Coding
Motion Vector
De-blocking
Filter
Motion
Estimation
2008/12/26
40
Outline
Introduction
Video Compression Standards
Simulation Reference Software
Future Work and Conclusions
2008/12/26
41
2008/12/26
42
2008/12/26
43
Outline
Introduction
Video Compression Standards
Simulation Reference Software
Future Work and Conclusions
2008/12/26
44
Future Work
Fast Mode Decision Algorithm
Interpolation Filter Design
Deblocking Filter Design
DCT-Based Motion Estimation
Implementation Based on TI DSP
2008/12/26
45
2008/12/26
46
2008/12/26
47
2008/12/26
48
Block Based ME
O(N2M2)
Concept
Pseudo Phase
j
F
{
x
(
t
)}
e
F {x(t )}
Similar to
2008/12/26
49
TI DM642
2008/12/26
50
Conclusions
Thank You
Q&A
2008/12/26
51
References (1/2)
[1] Yun Q.Shi and Huifang Sun, Image and Video Compression for Multimedia
Engineering: Fundamentals, Algorithms, and Standards, CRC press, 2000.
[2] Yao Wand, Jorn Ostermann and Ya-Qin Zhang, Video Processing and
Communications, Prentice Hall, 2007.
[3] Richardson, Lain E. G., Video Codec Design: Developing Image and Video
Compression Systems, John Wiley & Sons Inc, 2002.
[4] Barry G, Haskell, Atul Puri and Arun N. Netravali, Digital Video : An
Introduction to MPEG-2, Boston : Kluwer Academic, 1999.
[5] T. Wiegand, G. J. Sullivan, G. Bjontegaard, and A. Luthra, Overview of the
H.264/AVC video coding standard, IEEE Trans. on Circuits and systems for
video Technology, vol. 13, no. 7, pp. 560-576, July 2003.
[6] G. Sullivan and T. Wiegand, Video Compression - From Concepts to the
H.264/AVC Standard, Proceedings of the IEEE, Special Issue on Advances in
Video Coding and Delivery, December 2004.
[7]
2004.
2008/12/26
52
References (2/2)
[8] Thomas Wedi, Adaptive Interpolation Filters and High-Resolution
Displacements For Video Coding, IEEE Trans. on Circuits and Systems For
Video Technology, vol.
[9] Dong-Hwan Kim, Hwa-Yong Oh, Oguzhan Urhan, Sarp Ertrk and Tae-Gyu
Chang, Optimal Post-Process/In-Loop Filtering for Improved Video
Compression Performance, IEEE Trans. on Consumer Electronics, vol. 53, no.
4, Nov. 2007.
[10] Shu-Fa Lin, Meng-Ting Lu, and Homer Chen, Fast Multi-Frame Motion
Estimation for H.264 and Its Applications to Complexity-Aware Streaming,
IEEE International Symposium on Circuits and Systems, 2005.
[11] Kai-Ting Cheng and Hsueh-Ming Hang, Acceleration and Implementation
of H.264 Encoder and Scalable Extension of H.264 Decoder on TI DSP
Platform, master thesis, June 2007
2008/12/26
53