Prosopo

.

:
.

.
.

: ,
,
.

.
.

.
.
.
,
, .
,
Acknowledgments.
.
: -,
.
, ,
. , .
(constructive) .
.
, ,

.
.

,
.
1.1:
A2.1: mug shot
A2.2:
A3.1:
16
A4.1:
19
A4.2:
22
A4.3:
23
A4.1:
26
5.1: Radon
44
5.1:
48
5.2:
48
5.3:
49
5.4: -
49
B1.1:
57
B1.2: pixels
59
B1.3:
59
B1.4:
60
1.5:
62
1.6:
64
1.7:
65
2.1:
70
2.1:
72
2.2:
76
2.3:
77
2.4:
78
2.5:
78
3.1:
82
3.2:
85
3.1: studio
86
3.2:
87
ii
3.3:
88
4.1:
91
4.1:
96
4.2:
96
4.3:
97
4.4:
98
1.1:
104
1.1:
107
2.1:
115
2.1: KLT VKLT
115
2.2: KL VKL
116
2.3: KLT VKLT
116
3.1: DCT
118
3.2: DCT
118
4.1:
123
4.2:
124
4.3:
125
4.1: s s*
130
5.1: SOM
133
5.2:
134
5.2: SOM
135
2.1: , ,
146
2.2:
148
2.1:
150
2.3: 20
151
2.4: 5
152
3.1: delighted eager
155
3.1:
161
3.2:
161
3.3: 19 FDP FAPs 3.3
162
3.4:
162
3.5:
163
4.1:
170
iii
4.1:
172
4.2:
173
4.3:
174
4.2: HMM
175
iv
Mug shot :
. , .
False alarm:
Dismissal:
FACS - Facial Action Coding System:

Facial Definition Parameter set FDP: 3D
MPEG-4
Facial Animation Parameter set FAP:

MPEG-4
Apex:
3- :
shot:
- head format: , ,
- .

.................................................................................................................................Error! Bookmark not defined.
.................................................................................................................................Error! Bookmark not defined.
..................................................................................................................................Error! Bookmark not defined.
..................................................................................................................................Error! Bookmark not defined.
1. ............................................................................................................................................................................. 6
2. ........................................................................................................................ 8
3. ....................................................................................................................................... 9
4. .................................................................................................................................. 11
5. ........................................................................................... 12
........................................................................................................................................................................................ 12
..................................................................................................................................................................... 12
........................................................................................................................................................................................... 13
...................................................................................................................................................... 13
1. ..................................................................... 14
. ................................................................................................................................ 14
; .......................................................................................... 14
. ................................................................................................................................................. 14
................................................................................................................................................................................................................ 14
............................................................................................................................................................................ 15
............................................................................... 15
-.......................................................................................................................................................................................... 15
............................................................................................................................................................................................................ 15
2. ........................................................................................... 16
........................................................................................................................................................................................................... 16
...................................................................................................................................................................... 16
............................................................................................................................................................................................................... 17
...................................................................................................................................................... 18
1. ........................................................................................................................................................................... 19
2. ................................................................................................................................................................. 21
2.1 - .........................................................................................................21
() ................................................................................................................................................................................. 23
() ............................................................................................................................................... 23
() ................................................................................................................................................................................. 23
2.2 ................................................................................24
gray scale ........................................................................................... 24
2.3 ................................................................................27
2.4 ...............................................................................................................................................................27
3. - ............................................................................................................................. 28
4. ................................................................................................................................................. 32
4.1 ....................................................................................................................................................35
4.2 ..........................................................................................................................................................................35
1. ........................................................................................................................................................................... 37
2. ............................................................................................................... 38
2.1 ......................................................................38
3. ................................................................................. 39
4. .............................................................................................................................. 40
vi
5. ................................................................................................................................. 41
............................................................................................................................................................ 41
5.1 ...................................................................................................................................................................42
5.2 .................................................................................................................................................................42
() . ................................................................................................................. 42
() ...................................................................... 44
() .......................................................................................................................................................................... 46
() .................................................................................. 48
1. .......................................................................................................................... 56
2. ....................................................................................................... 59
2.1 Gabor..................................................................................................62
3. ......................................................................................................................... 63
4. 3- ........................................................................................................................................................... 65
1. ........................................................................................................................................................................... 67
2. ..................................................................................................... 68
3. ........................................................................................................................... 71
4. ...................................................................................................................................... 73
5. ............................................................................................................... 74
6. ............................................................................................................................................ 75
6.1 ...........................................................................................................................75
6.2 ........................................................................................................76
6.3 .....................................................................................................................77
6.4 ............................................................................................................78
7. .................................................................................................................................................................. 79
1. ........................................................................................................................................................................... 80
2. ..................................................................................................................... 82
2.1 ...............................................................................................................................................84
2.2 .........................................................................................................................84
3. .......................................................................................................... 84
4. ............................................................................................................................................ 86
1. ........................................................................................................................................................................... 90
2. ................................................................................................................................................... 91
3. ..................................................................................................... 92
4. ................................................................................................................................. 93
4.1 ..............................................................................................................93
4.2 .....................................................................................................94
4.3 ...................................................................................................................................................95
4.4 ..........................................................................................................................95
5. ............................................................................................................................................ 95
6. .................................................................................................................................................................. 97
1. ......................................................................................................................................................................... 101
vii
2. .............................................................................................................................................. 102
3. ............................................................................................................ 104
................................................................................................................................................ 104
........................................................................................................................................................................................ 105
........................................................................................................................................................................ 105
4. ................................................................................................................. 105
1. ......................................................................................................................................................................... 108
2. .......................................................................................................................... 109
3. M ............................................................................................... 109
3.1 Karhunen-Loeve.......................................................................................................................................111
3.2 Fisher Linear Discriminant FLD..............................................................................................................................................114
1. ......................................................................................................................................................................... 116
2. KLT ........................................................................................................................ 117
3. ............................................................................................................................................... 118
4. ......................................................................................................................................... 120
1. ......................................................................................................................................................................... 122
2. ...................................................................................................... 122
3. ............................................................................................................ 123
4. ............................................................................................................................................... 124
5. .......................................................................................................................................... 124
6. ................................................................................................................................................................ 125
1. ......................................................................................................................................................................... 126
2. .......................................................................................................................................... 126
2.1 ...........................................127
3. ........................................................................ 130
4. ............................................................................................................................................. 130
5. .......................................................................................................................................... 134
6. ................................................................................................................................................................ 135
1. ................................................................................................................. 136
2. ........................................................................................................................................................ 136
2.1 .......................................................................................................................................136
2.2 ...................................................................................................................138
2.3 - Luminance Invariant Representation ...........................................139
3. ............................................................................................................................................ 139
3.1 ....................................................................................................................................139
3.2 ....................................................................................................................................140
3.3 .....................................................................................................................141
4. ............................................................................................................................................. 141
5. ......................................................................................................................................... 143
6. ................................................................................................................................................................ 144
viii
1. ......................................................................................................................................................................... 148
2. ............................................................................................................................................... 148
3. FLD .............................................................................................................................................................. 149
3.1 PCA ..............................................................................................150
3.2 ........................................................................151
4. ................................................................................................................................. 152
4.1 MLP ..............................................................153
4.2 ......................................................................................................................................................154
4.3 PCA ..........................................................................................................155
5. ................................................................................................................................................................ 155
1. ......................................................................................................................................................................... 157
2. ............................................................... 159
3. FDPs ................................................................................................. 160
3.1 ............................................................................................................163
3.2 ......................................................................................................................163
4. ....................................................................................................................................... 164
4.1 ................................................................165
4.2
........................................................................................................................................................168
4.3 ..................................................................................................169
5. ........................................................................................ 171
6. .......................................................................................................................................... 171
7. ................................................................................................................................................................ 172
1. ........................................................................................................................................ 175

............................................................175
1.1 ........................................................................................................................................................................175
1.2 ...................................................................................................................................176
2 ............................................................................................................. 176
3. HMM...................................................................................................................................................... 179
4. .......................................................................................................................................... 181
5. ................................................................................................................................................................ 181
............................................................................................................................................................................. 190
ix
mpeg-4 ............................................................................................................. 191

FAPs ............................................................................................................................................................................................ 191
FDPs ............................................................................................................................................................................................ 192
A1

.
.

.

XY
1.1:

:
, ,
.

, , , .
,
2

.

.
.
.

, .
1.1
.

.
,

.
,
.

.

.
:

.
.

.
.

.

.
.
3

.

.
.
.
.

.
, .

.
.
.

.
.
,
, .

-
.

.

.
.

.
.

4
.

, .
.
.
, ,
.
.
!

. --

.
A2
.

.
. , ,
, .
1.
.
. ,
.
:
.

.

.
.
, , , .

. ,
, ,
, ,
.
.

, ()
.
, PINs .

.
, .
6
,
, ,
.

. ,
, , ,
, - ,
.
: ,
.
, , , ,
, .
:
(1) - .
(2) .
, , , - ,
.
(3)
.
-.
,
.
,
.

20 .
, ,
,
.
.

.

70. 80 .
90 ,
IEEE Transactions on
7
PAMI, sessions- (IEEE ICIP,

IEEE ICASSP, IEEE Multimedia).
[1].
.
2.
2.1.

.

, ,
.
:
- - -
.

. - ,

, ,
- 4 5 2.1.
,

.

2.1. , .
1, 2, 3, 10 11 .
4-7
. 8 9
.
2.1
.
.

.
.

- .
8
, ,
, ,
.
,
,

Mug shots
,
.
, ,

,
,

mug shots

(aging)
10
11
studio

1()
1()
,
,

A2.1:
3.
mug shot .
,
- -.
,
.
, ,
. ,
, .
mug shots 2.1() 2.1()
9
()
()
()
()
A2.1: () Mug shot (front view) () Mug shot

(profile) () () , .
mug shot ,
, .
2.1() 2.1().
mug shots .
mug shot
. ,
.

, .
()
()
A2.2: () ()
2 1
.
.
. 2.2() ().
2

. 1,
,
10
. 1 2

.
4-7
. 4
.
. ,
,

. 5

, , .
, , ,
. 6
. 7
. 4-7

.
,
.
,
. 4-7

.
8 9

. 4-6
.
4.
3 2 ,
.
.
,
. ,
. , 3-
, ,
.
11

.

.
5.
.
.
.
, ..
.
.

.
.

.
, .. , .

.
, . ,
512x512, 6 bpp,
8 bpp. 512x512
.
.

,
.. .

. , ,
, ,
. , ,
, , ,
.
.

12
. , 90 ().

.
.
.
: .

. ,
.
().

, , .
.

.
13
A3
.
- -
.
.
,
.
1.
.
: () ,
, , () ,
,
, , , ,
, ()
.
;

.
( , .)

.
, , - .
.
.
, ,
.
,
, .
14

. , ..
,
.

.

, .
.
-

.
.
.
,
,
.
.
, ,
.
.

.
(multiresolution)
. ,
, ,
,
mug shot .
mug shot .

.
15
,
.
2.
,
, , ,
.
.
, ,
.

.
, . ,
.

.

,
. ,
, ,
.
. ,
-
.
.
, , .
,
,
.
,
. -
- ,
, .

.
. 3.1
16
() 64x88 () 32x44 () 16x22. 32x44

16x22 , .
16x16 .
.
()
()
()
()
A3.1: () 64x88 () 32x44 ()

16x22 () 64x88 1 bpp
.
3.1(), 1bpp,
.
32x32 4 bpp .
.

. 700
.
.
.

-.. - .
.
, .

.
. . Bartlett [2]
()
.
,
.
,
.
17
: , , .
, ,
. , , ,
. .
[3]
.
.
.

. [3]
.
1000 10
, . ,
,
.

, ,
. :
, : , , , : , , .

.
.
.
18
A4
. ,
, .
.
.

. ,
,
.
1.
Francis
Galton.
2-

.
.
:
(1).
().
.
(2).
( z
- . A4.1).
(3).
(4).
,
.
19
x
z
A4.1:
Baron
150
512x480
6 bpp
128x120
Buhr
100
512x512
8 bpp
Campell
50
256x256
6 bpp
Craw et al
1000
128x128
20
Goldstein et al
255
Harmon et al
124
97 +27
Harmon et al
130
81 +49
Kaufman
120
240x356
1 bpp
Nixon
128x128
8 bpp
Petajan
244x248
Riccia
506
Wong
256x256
8 bpp
Wu and Huang
18
Olivetti R.L.
400
112x92
8 bpp
40 10

A4.1:

4.1
, .
20
.
(manually).
2.
:
(

).
2.1 -

.
, ,
.
.
,
, (
), (moment invariants),
(Fourier descriptors).

, , ,
. [7]:
() .
, ,
..
, , active contours. 1. A4.2
,
.
.
21
, , ,
, , , , , ,
D( , ), D( , ),
D( , ),
D( , )
, D(, )
, , ,
, , , ,
, , , ,
, ,
, , ,
, D(, ), D(, ),
D(, ), D(, ), D( , ),
D( , ), D( , ),
D( , ), D( , ),
D( , ), D( , ),
D( , ), D( , )
( , ), ( , ),
( , )
D( , )/ D(, )
D(, )/ D( , )
, ,
A4.2:
. D(x,y)=
x y
22
()

.
(x,y) [ x1 , x 2 ] [ y1 , y 2 ] .
:
V ( x) =
y2
y = y1
I ( x, y ) , H ( y ) =
x2
I ( x, y )
(4.1)
x = x1
A4.2:

(central moments) moment invariants. moment invariants

, [8].
()
Fourier, Hadamard
. Fourier (signatures)
.
.
()

. .
Karhunen Loeve -KL SVD -Singular Value
Decomposition. L

4.3.
. L 1.
SVD 4 5.
23
1330
324
11
11
+ 216
A4.3:
2.2

.
.
, gray scale ,
. ,
gray scale . , 32x32
4 bpp ,
gray scale ,
.
gray scale
4.3
. R1 R2
kl gray scale . :
M 1 = max{SVD{
R1
R2
}}
mean( R1) mean( R 2)
=>
R1 R2
M2=
R1
R2
diag{
R1
R2
} => Frobenius
R1 R2 .
M3 =
1
k l
R1
R2
mean(R1) - mean( R2)
=> -pixel pixel-
R1 R2
24
M4 =
diag{ R1 - R2
R1 - R 2 } => Frobenius
R1 R2
M 5 = corr( R1, R 2) => R1 R2
Olivetti Research Laboratory (ORL)

. gray scale
8 bpp.
.
:
(1) 28x23 .
- .
(2) .
- - .
(3) () 1.51, 3.81 19.82 sec

200 .
(4) .
(sec)
1.51
28x23
(sec)
3.81
(sec)
56x46
19.82
112x92
A4.3:
5
25
28x23
56x46
112x92
A4.4:
5 .

.
28x23
56x46
112x92
A4.5:
5 .

.
4.4 ,
,
. , 4.5
,
, . 4.4 4.5
.
26
2.3

.
.
MPEG-4.
,
, ,
.
. 4.6
.
A4.1: frontal views.

profile

frontal view
1-2-3
d(14,15) /d(4,9)
7-8-9
d(12,13) /d(14,15)
d(8,10) /d(2,8)
d(5a, 5b) /d(14,15)
A4.6: (
4.1). d(i,j) i j.
2.4
gray scale
4.2 4.6. Campell [4]

32x32 gray scale .
27
,
, .
Galton [5][6] ,
.
. ,
, .
().
3. -

. , ,
,
.
mug shots.
.
.
Govindaraju et al. [7][8], ,
, , ,
.
: , ,
, .

. ,
,
. (semantic)
.
Hough.
( )
.
. : () ()
4.1, z- -
y. ,
.
10
false alarms ( ).

[9].

28
.

, .
[10]
. ,
,
.
.

.
Craw [11]
-template matching.
,
.
, ,
.

.
.
. ,
.
.
.
.
l1 .
, Craw [12]
. (indexing)
mug shot . 40
.
Shepherd [13]
.
(polygon template) coarse to fine.
, , .
simulated annealing .

.
.

29
feature experts.
50 , 43 , 7
.
,
.
simulated annealing
.
[14]
. ,

. (x0, y0) a, b.

- z -
y .

Karhunen-Loeve.

. DFFS (Distance-From-Feature-Space)
pixel . pixel
(
, , ).
N , {I 1 , I 2 ,.... I N } I i R nxm . I i
{v1 , v 2 ,.... v N } vi R L L = n m . KL
:
(4.2)
L = F T SF
S , F S , L
.
yi x i -
- KL:
)
yi = F k T xi
(4.3)
N
x)i = xi - m , m = v j F k k
j =1
S .
30
:
) (
) (
e KLT ( k , i ) = ( xi - xi ) T ( xi - xi )
x(i = F k yi ,
(4.4)
)
yi = F k T xi
DFFS :
x opt = arg m i n[e KLT ( k , i )]
i
(4.5)

[16].
.
.
.

.
.
Yulie [17] deformable templates
( ). ,
gray scale .

, .
.
, . ,
.
, , ,
.
:
,
.
.
O Nixon [18]
Hough.
. Craw [11]
,
.
.
31

-
- .
,
, .
.
,
,
.
4.
.
: () ()
,
.

.
Galton [5][6] 19 . - -
.

.
. ,
.
:
(1) .
(2) 1

(3)
(4)
.

.
- 100 ( 4.1).
10-40.
32
Galton [5][6] .
9 4.1, , ,
,
.
. Galton :
.

Z
.
yiY ziZ .

.
Harmon [20] Galton
. Galton
, - 4.1.

, .
Y Z
:
d 2 (Y , Z ) =
i =1
y i2 - z i2
s i2
(4.6)
n 2 . 256
, .

:
(1)
(2)
(3)
, .
.
[21] ,
. 11- ,
,
D12, D14, D15, D16, D18 D79 Dij
33
i j.
,
, . ,

().
10-
. [22]
.
(contrast) .
Kaufman Breeding [23]
.
. moment invariants.
.
.
- range finders.
,
.
Goldstein [24]. ,
. 34
22, (..
). 1-5
( - ). 255 .

, , , , , , .

.
. 6-7
. Kaya Kobayashi [25]
,
.
O Buhr [19] 33 12
. 21 -..
- 4 , 4 , 2
.
. Wong [26]

34
.
(clustering).
Baron [27] gray scale .
, 20x23 ,
, . ,
, 15x16 pixels. ,

.

.
.
.
Preston [28] Taylor [29]
. ,
-6 10 .
:
4.1
gray scale
.
. 4.2 4.6

.
4.2
.
. :
(1) . .

.

.
(2) .
.
. k-means
SOM.
35
(3) .
, .
.

.
(4) . gray scale
.
.
(5) .
.
.
.
4.3 - 4.5
. 1
3
. ,

.
,
gray scale .
36
A5

: () , ()
()
.

. ,
,
.
1.
. Shlosberg [30]
:
- A-R (attention-rejection), - P-U (pleasantnessunpleasantness), (level of activation).

. Ekman Friesen [31]

. , , , ,
.
. animation

Terzopoulos Waters [43]
.
(analysis
by synthesis).

.
.
37
2.
[31],
Darwin [32]
[33]-[35].
, ,

[36].
2.1

:
apex.

.
.

, ,
.

[36][37].

.
Ekman
-
.
Ekman Friesen [38]

,
.
.

.

- [39].

.
38

,
,
.
.
3.
[38]
-
- , .
-
.
. Bassili
[40]

.
,
,
ness .
Bassili

.
.
.
,
.

.
,
, , -
.

.
Ekman
Friesen[31],
FACS - Facial Action Coding System. FACS
AU
action units. AU
39
.
AU AU. 46 AU
12 .
FACS
[41]-[47]. Ekman
EMFACS AU .
FACSAID
FACS [48].
FACS
ISO MPEG-4 [49].
(Facial Definition Parameter set, FDP)
(Facial Animation Parameter set, FAP)
,
. FDP ,
, ,
,
setup.
,
. (FAPs)
. ,
,
.
(Facial
Animation Parameters Units).

.
.
.
3.
4.
,
.
(amygdala)
[51].

[52].
, [53].
40

.
5.

. : () ,

, () ,
.
0.5 4 [35].
,
.

.

.
.
.
[54]-[59].
[54]-[59] [60]-[62],
, .

.

. , ,
.
, active contours, [63],
wavelets [64] RBF [65], affine

.

-pose estimation-
[66][67].
41
5.1
mug shot
[68].
,
, .
.

. [69].
[70],
, , PCA
.
.
86% -
92%-
.
.
, ,
: ()
, , ,
() .
5.2
() .

.
,
.
Horn Schunck [71],
.
,
, [45].
[72] .
, ,
.
.
[73][74]
MPEG. pixel
42

pixel pixel .
pixels ,
.
.

. , ,
.

.

.

, ,
.
: ()
, ()
.

. Mase [45]
AUs FACS.

.
,

.

. ,
.

.
k-means.
, ,
. 4
Mase
.
43
Yacoob Davis [75] ,

,
.
Ekman Friesen [38]
Bassili [40],
.
[38][40]. Yacoob Davis,

, :

.
3.
Li, Roivainen Forchheimer [44]
. FACS
.
-rigid-
. 6 AUs
.
()

, ,
:

.
zoom .
pixels
[76].
Fk Fk +1
. pixel p k ( x, y) k-
2nx2n bk ( x, y ) (MAD-Mean Absolute
Difference):
e k ( x, y) = bk ( x, y ) - bk +1 ( x, y ) =
( x + l , y + m) - p k +1 ( x + l , y + m)
(5.1)
l = -n m= -n
44

ek ( x, y ) - 4.
v k ( x, y ) block bk ( x, y )
bk +1 ( x, y ) :
v k ( x, y ) = (v x , v y ) = arg min
( v x , v y )Q l = - n m = - n
( x + l , y + m) - p k +1 ( x + l - v x , y + m - v y )
(5.2)
Q = {-q,..., q} {-q,..., q} .
,
(v x , v y ) Q . -
- ,
.
(x,y) v k ( x, y ) = a k ( x, y)e jfk ( x , y ) .
Radon , :
R(q ) =
( x, y )
(5.3)
x =t cos q -u sin q , y =t sin q + u cos q
u = -
t u x y
5.1.
y
a(x,y)
5.1: Radon
Radon, , 00
900
. ,
Yale -82% 87.5%
[76].
45

.
()
-
.
MPEG-4 3.

.
idden
Markov odels (HMM), recurrent neural networks .
[76][77]
.
, coarse-to-fine
wavelets [78].
.
Ohya [79]-[81] Hidden Markov Models
.
HMMs . [79] wavelets

wavelet- [81]
Fourier. Fourier
,
.
[80] HMM

[79].

.

. :
,
, , , [82],[84]-[87],
[88].
Lucas-Kanade [83],
.
46

.
.
,
- .
Yacoob [89][90],
30 .

[73]. , ,
,

.
Rosenblum Yacoob [91] [69],
.
Thalmann [92]
.

.

. Essa Pentland [93]-[95]
FACS.
3-
. 3-
,
.
Terzopoulos Waters [43] Platt Badler [96],
mesh- ,
. [43]
,
.
.
: , .
.
Terzopoulos Waters
.
47

.
98% - 52 .
3- .
()
5.1()

.
5.1() .
5.1(), 5.2() 5.3()
, .
5.1(), 5.2() 5.3()
Radon 900 . 5.1
(), 5.2() 5.3()
.
.
, .

, . :
.
:

.
- 4, 4.1-
HMM -

[97].
HMMs
3.
HMM ,
, . 5.4 ,
, :

.
,
.
48
5.1: () 01, 01
03, 03 apex, apex release, () Radon,
, 900 01 03, 01 apex
(dotted line), apex release.
5.2: () 01, 01
05, 05 09, 09 apex, apex release, () Radon,
, 900 01 05
(dashed line), 01 09, 01 apex (dotted line).
49
5.3: () 01, 01
04, 04 apex, apex release, () Radon,
, 900 01 04, 01
apex (dotted line), apex release.
5.4: () ()
HMM, activation - evaluation.
50
[1].
[2].
[3].
[4].
[5].
[6].
[7].
[8].
[9].
[10].
[11].
[12].
[13].
[14].
[15].
[16].
[17].
[18].
[19].
[20].
[21].
[22].
[23].
[24].
P. Chellapa, C. Wilson and S. Sirohey, Human and Machine Recognition of Faces: A Survey, Proc.
IEEE, vol. 83, no. 5, pp. 705-740, 1995.
J.C. Bartlett, S. Hurry and W. Thorley, Typicality and familiarity of faces, Memory Cognition 12, pp. 219228, 1984.
L.D.Harmon, The recognition of faces, Scientific America. 229, pp.71-82, Oct. 1973
R.A.Campell, S. Cannon, G. Jones and N. Morgan, Individual face identification by computer vision,
Proc. Conf. Modeling Simulation Microcomputer, pp. 62-63, 1987.
Sir F. Galton, Personal identification and description - I, Nature, pp.173-177, June 1888.
Sir F. Galton, Numeralized profile for classification and recognition, Nature 83, pp.127-130, March 1910.
V. Govindaraju, D.B.Sher, R.K.Srihari and S.N.Srihari, Locating of human faces in newspapers
photographs, Proc. of CVBR, pp. 549-554, 1989.
V. Govindaraju, S.N.Srihari and D.B.Sher, A computational model for face location, Proc. of 3rd Int. Conf.
on Computer Vision, pp. 718-721, 1990.
T. Sakai, M. Nagao and S. Fujibayshi, Line extraction and pattern recognition in a photograph, Pattern
Recognition 1, pp. 233-248, 1969.
M. Kelly, Visual identification of people by computer, Technical Report AI-130, Stanford, CA, 1970
I. Craw, H . Ellis and J.R.Lishman, Automatic extraction of face-features, Pattern Recognition Letters 5,
pp.183-187, 1987.
I. Craw, D. Tock and A. Benett, Finding face features, Proc. of 2nd European Conference on Computer Vision,
pp.92-96, 1992.
J. Sepherd, An interactive computer system for retrieving faces, Ellis (Eds), Dordrecht: Nijhoff, pp. 398409, 1985.
S. Sirohey, Human face segmentation and identification, Technical Report CAR-TR-695, University of
Maryland, MD, 1993.
Sir F. Galton, Personal identification and description - I, Nature, pp.201-203, June 1888.
J.L.Perry and J.M.Carney, Human face recognition using a multilayer perceptron, Proc. of Int. Conf. on
Neural Networks II, pp. 413, January 1990.
.L.Yulie, D.S.Cohen and P.W.Hallinan, Feature extraction of faces using deformable templates, Proc. of
CVBR, pp. 104-109, 1989.
M. Nixon, Eye spacing measurements for facial recognition, SPIE Proc. 575, Applications of Digital Image
Processing VIII, pp. 279-285, 1985.
R. Buhr, Analyse und klassifikation von gesichtsbildern, ntzArchiv 8, pp. 245-256, 1986.
L.D.Harmon, Automatic recognition of human face profiles, Proc. of 3rd Int. Joint Conf. on Pattern
Recognition, pp. 183-188, 1976.
L.D.Harmon, S.C.Kuo, P.F.Ramig and U. Raudkivi, Identification of human face profile by computers,
Pattern Recognition 10, pp. 301-312, 1978.
L.D.Harmon, M.K.Khan, R. Lasch and P.F.Ramig, Machine identification of human faces, Pattern
Recognition 13, pp. 97-110, 1981.
G.J.Kaufman K.J.Breeding, The automatic recognition of human faces from profile silhouettes, IEEE
Trans. on System Man and Cybernetics 6, pp. 113-121, 1976.
A.J.Goldstein, L.D.Harmon and A.B.Lesk, A basic study on human face recognition, Proc. of IEEE 59,
pp. 748-760, May 1971.
51
[25]. Y. Kaya and K. Kobayashi, Identification of human faces, Frontiers of Pattern Recognition, pp. 265-289,
Academic Press, New York 1971.
[26]. K.H.Wong, H.M.Law and P.W.M.Tsang, A system for recognizing human faces, Proc. of ICASSP, pp.
1638-1642, 1989.
[27]. R.J.Baron, Mechanisms of human facial recognition, Int. Journal of Man Machine Studies 15, pp. 137-178,
January 1990.
[28]. K. Preston Jr, Computing at the speed of light, Electronics 38, pp. 72-83, 1965.
[29]. W.K.Taylor, Machine learning and recognition of faces, Electronic Letters 3, pp. 436-437, 1967.
[30]. H. Shlosberg, Three dimensions of emotion, Psychol. Rev 61, pp. 81-88, 1954.
[31]. P. Ekman and W.V.Friesen, Manual for the Facial Action Coding System, Consulting Psychologists Press,
Palo Alto, 1978.
[32]. C. Darwin, The Expression of Emotions in Man and Animals, John Murray, 1872, reprinted by University of
Chicago Press, 1965.
[33]. P. Ekman, Darwin and Facial Expressions, Academic Press, 1973.
[34]. M. Davis and H. College, Recognition of Facial Expressions, Arno Press, New York, 1975.
[35]. K. Scherer and P. Ekman, Approaches to Emotion, Lawrence Erlbaum Associates, 1984.
[36]. P. Ekman, T. Huang, T. Sejnowski and J. Hager, NSF Planning Workshop on Facial Expression
Understanding, Technical report, National Science Foundation, Human Interaction Lab, 1992.
[37]. W. E. Rinn, The Neuropsychology of Facial Expression: A Review of Neurological and Psychological
Mechanisms for Producing Facial Expressions, Psychological Bulletin, vol. 95, pp. 52-77, 1984.
[38]. P. Ekman and W. Friesen, Unmasking the Face, Prentice-Hall, 1975.
[39]. S. Shibui, H. Yamada, T. Sato and K. Shigemasu, Categorical Perception and Semantic Information
Processing of Facial Expressions, Perception, vol. 28 S, p. 114, 1999.
[40]. J. N. Bassili, Emotion recognition: The role of facial movement and the relative importance of upper and
lower areas of the face, Journal of Personality and Social Psychology, 37, 2049-2059, 1979.
[41]. M. Bartlett, J. Hager, P. Ekman and T. Sejnowski, Measuring Facial Expressions by Computer Image
Analysis, Psychophysiology, vol. 36, pp. 253-263, 1999.
[42]. I. Essa and A. Pentland, A Vision System for Observing and Extracting Facial Action Parameters, Proc.
Intern. Conf. on CVPR, pp. 76-83, 1994.
[43]. D. Terzopoulos and K. Waters, Analysis and Synthesis of Facial Image Sequences using Physical and
Anatomical Models, IEEE Trans. on PAMI, Vol. 15, No. 6, 569-579, 1993.
[44]. Li, Roivainen and Forcheimer, 3-D motion estimation in model-based facial image coding, IEEE Trans.
on PAMI, Vol. 15, 545-555, 1993.
[45]. K. Mase, Recognition of facial expression from optical flow, IEICE Trans., vol. E74, pp. 3474-3483,
1991.
[46]. C. Pelachaud, N. Badler and M. Viaud, Final Report to NSF of the Standards for Facial Animation
Workshop, Technical report, NSF, University of Pennsylvania, Philadelphia, PA 19104-6389, 1994.
[47]. P. Ekman, T. Huang, T. Sejnowski and J. Hager (Eds), Final Report to NSF of the Planning Workshop on
Facial Expression Understanding, Technical report, NSF, Human Interaction Lab., UCSF, CA 94143,
1993.
[48]. P. Ekman, J. Hager and E. Rosenberg, ACSAID: A Computer DataBase for Predicting Affective
Phenomena from Facial Movement, http:/www.nirc.com/facsaid.html.
[49]. ISO/IEC JTC1/SC29/WG11 MPEG96/N1365, MPEG4 SNHC: Face and Body Definition and
Animation Parameters, 1996.
[50]. R. Cabeza and J. Nyburg, Cognitive Neuroscience 9, 1-26, 1997.
[51]. V. Bruce and A. Young, In the Eye of the Beholder: The science of face perception, Oxford Univ. Press, 1998.
52
[52]. Phillips et al., Nature 389, 495-8, 1997.

[53]. S. Scalaidhe et al., Science 278, 1135-08, 1997.
[54]. S. McKenna and S. Gong, Tracking Faces, Proc. of the 2nd Intl. Conf. on Automatic Face and Gesture
Recognition, pp. 271-276, 1996.
[55]. J. Crowley and F. Berard, Multi-Modal Tracking of Faces for Video Communications, Proc. of IEEE
CVPR, pp. 640-645, Puerto Rico, 17-19 June 1997.
[56]. H. Graf, E. Cosatto, D. Gibbon, M. Kocheisen, and E. Petajan, Multi-modal System for Locating Heads
& Faces, Proc. Int. Conf. on Automatic Face & Gesture Recognition, pp. 88-93, Vermont, USA, Oct. 1996.
[57]. M. Collobert et al, Listen: A System for Locating and Tracking Individual Speakers, Proc. of the Intl. Conf.
on Automatic Face and Gesture Recognition, pp. 283-288, Vermont, October 1996.
[58]. M. Hunke and A. Waibel, Face Locating and Tracking for Human Computer Interaction, IEEE
Computers, pp. 1277-1281, November 1994.
[59]. S. Basu, I. Essa and A. Pentland, Motion Regulization for Model-based Head Tracking, Proc. 13th Intern.
Confer. On Pattern Recognition, August 1996.
[60]. P. Fieguth and D. Terzopoulos, Color-based Tracking of Image Regions with Changes in Geometry and
Illumination, Proc. of IEEE CVPR, pp. 403-410, 1996.
[61]. J. Terillon, M. David and S. Akamatsu, Automatic Face Detection in Natural Scene Images Using a Skin
Color Model & Moments, Proc. Int. Conf. Automatic Face & Gesture Recognition, Nara, Japan, April 1998.
[62]. Y. Raja, S. Mckenna and S. Gong, Tracking and Segmenting People in Varying Lighting Conditions
Using Color, Proc. Int. Conf. Automatic Face and Gesture Recognition, pp. 228-233, Nara, Japan, April 1998.
[63]. T. Maurer and C.V.D. Malsburg, Tracking and Learning Graphs on Image Sequences of Faces, Proc. of
the Intl. Conf. on Automatic Face and Gesture Recognition, pp. 176-181, Vermont, USA, Oct. 1996.
[64]. V. Kruger and G. Sommer, Affine Face Tracking Using a Wavelet Network, Proc. Intl. Workshop on
Recognition, Analysis and Tracking of Faces and Gestures in Real-time Systems, Corfu, Greece, Sept. 1999.
[65]. N. Arad and D. Reisfeld, Image Warping using few Anchor Points and Radial Functions, Computer
Graphics Forum, vol 14 (1), 35-46, 1994.
[66]. S. Gong, S. McKenna and S. Collins, An Investigation into Face Pose Distributions, Proc. of the Intl. Conf.
on Automatic Face and Gesture Recognition, pp. 265-270, Vermont, 1996.
[67]. A. Zelinsky and J. Heinzmaan, Real-time Visual Recognition of Facial Gestures for HCI, Proc. of the Intl.
Conf. on Automatic Face and Gesture Recognition, pp. 351-356, October 1996.
[68]. A. Young and H. Ellis, Handbook of Research on Face Processing, Elsevier Science Publishers, 1989.
[69]. C. Padgett and G. W. Cottrell, Representing Face Images for Emotion Classification, Advances in Neural
Information Processing Systems, Vol. 9, p. 894, The MIT Press, 1997.
[70]. C. Padgett, G. Cottrell and B. Adolps, Categorical Perception in Facial Emotion Classification, Proc. of the
Cognitive Science Conference, vol. 18, pp. 249-253, 1996.
[71]. B. Horn and B. Schunk, Determining optical flow, Artificial Intelligence, Vol.17, 185-203, 1981.
[72]. D. Heeger, Optical flow using spatiotemporal filters, Intern. Journal of Computer Vision 1, 279-302, 1988.
[73]. M. Abdel-Mottaleb, R. Chellappa and A. Rosenfeld, Binocular motion stereo using MAP estimation,
IEEE Conference on Computer Vision and Pattern Recognition, 321-327, 1993.
[74]. P. Anandan, A computational work and an algorithm for the measurement of visual motion,
International Journal of Computer Vision 2, 283-310, 1989.
[75]. Y. Yacoob and L. Davis, Computing spatio-temporal representations of human faces, Proc. of the
Computer Vision and Pattern Recognition Conference, 70-75, 1994.
[76]. N. Tsapatsoulis, I. Avrithis and S. Kollias, On the use of Radon Transform for Facial Expression
Recognition, Proc. of the 5th Intl. Conf. on Information Systems Analysis and Synthesis, Orlando, U.S.A, July 1999.
[77]. M. Black and P. Anandan, The Robust Estimation of optical flow, Proc. Int. Conf. on Computer Vision,
Berlin, Germany, 231-236, 1993.
53
[78]. Y. T. Wu, T. Kanade, J. Cohn and C-C. Li, Optical Flow Estimation Using Wavlet Motion Model, 1997.
[79]. T. Sakaguchi, J. Ohya and F. Kishino, Facial expression recognition from image sequence using Hidden
Markov Model, VLBV 95, A-5, 1995.
[80]. T. Otsuka and J. Ohya, Recognition of facial expressions using HMM with continuous output
probabilities, Proc. of 5th IEEE International Workshop on Robot and Human Communication RO-MAN, 323328, 1996.
[81]. T. Otsuka and J. Ohya, Recognizing Multiple Persons Facial Expressions using HMM based on
Automatic Extraction of s from Image Sequences, Proc. IEEE Int. Conf. on Image Proc., vol. 2, 546549, 1997.
[82]. J. Lien, T. Kanade, J. Cohn and C-C. Li, Subtly Different Facial Expression Recognition and Emotion
Expression Intensity Estimation, Proc. of IEEE CVPRpp. 853-859, Santa Barbara, CA, 1998.
[83]. B. Lucas and T. Kanade, An Iterative Image Registration technique with an Application to Stereo
Vision, Proc. of the 7th Intl. Joint Conf. on AI, 1981.
[84]. J. Cohn, A. Zlochower, J. Lien and T. Kanade, Automated Face Analysis by Feature Point Tracking Has
High Concurrent Validity with Manual FACS Coding, Psychophysiology, vol. 26, pp. 35-43, 1999.
[85]. H. Wu et al, Face and Facial Feature Extraction from Color Images, Proc. of the Intl. Conf. on Automatic Face
and Gesture Recognition, pp. 345-350, October 1996.
[86]. R. Herpes et al, Edge and Keypoint Detection in Facial Regions, Proc. of the Intl. Conf. on Automatic Face and
Gesture Recognition, pp. 212-217, Vermont, USA, October 1996.
[87]. R. Herpes et al, An Attentional Processing Strategy to Detect and Analyse the Prominent Facial
Regions, Proc. of the Intl. Conf. on Automatic Face and Gesture Recognition, Zurich, Switzeland, pp. 214-220,
1995.
[88]. M. Black, Y. Yacoob, A. Jepson and D. Fleet, Learning Parameterized Models of Image Motion, Proc. of
IEEE CVPR, pp. 561-567, 1997.
[89]. Y. Yacoob and L. Davis, Recognizing human facial expressions, The Second Workshop on Visual Form,
Capri, 584-593, 1994.
[90]. Y. Yacoob and L. Davis, Recognizing human facial expressions from long image sequences using optical
flow, IEEE Transactions on Pattern Analysis and Machine Intelligence 18(6), 636-642, 1996.
[91]. M. Rosenblum, Y. Yacoob and L. Davis, Human Emotion Recognition from Motion Using a Radial
Basis Function Network Architecture, IEEE Trans. On NNs vol. 7, no 5, 1996.
[92]. N. Thalmann, P. Kalra and M. Escher, Face to Virtual Face, Proc. of the IEEE, vol.86, pp.870-883, 1998.
[93]. I. Essa and A. Pentland, Coding, Analysis, Interpretation and Recognition of Facial Expressions,
Technical Report No. 325, M.I.T. Media Laboratory, 1995.
[94]. I. Essa, T. Darrell and A. Pentland, Tracking facial motion, Proceedings of the Workshop on Motion of Nonrigid
and Articulated Objects, 36-42, IEEE Computer Society, 1994.
[95]. I. Essa, S. Sclaroff and A. Pentland, Physically-based modeling for graphics and vision, in R. Martin (Ed)
Directions in Geometric Computing. Information Geometers, U.K., 1993.
[96]. S. Platt and N. Badler, Animating facial expression, Proc. of ACM SIGGRAPH Conference 15(3):245-252,
1981.
[97]. N. Tsapatsoulis, M. Leonidou and S. Kollias, Facial Expression Recognition Using HMM with
Observation Dependent Transition Matrix, Proc. of MMSP98, Portofino CA, December 1998.
54
55
B1

.

. .
:

y 4.1.
, .
.
.
,

.
1.

.
,
. ,
,
. ,
(active contours) , .
, ,

.
rc.
L
(2n+1)(2n+1), n ( n=1).
:
N (t ij ) = {t i - k , j -l | k , l = -n,..., n}
(1.1)
56
(2n+1)(2n+1) pixel tij=(i,j). :

H (i, j ) = max [I ( N )]
(1.2)
p ij N
L(i,j) :
L (i , j ) = min [I ( N ) ]
(1.3)
p ij N
()
()
()
()
B1.1: () () ()
() ( )
D=H-L
. , ,
D . 1.1
.

, . , pixels
,
(
). pixels
. ,
D .
c=m+ :
m=
s=
1
M N
1
M N
D(i, j)
(1.4)
i =1 j =1
M
D(i, j) - m
(1.5)
i =1 j =1
[X 0 , Y0 , A, B, Q] = arg max E[x 0, y 0, a, b,q ]

x0 , y0 ,a ,b,q
(1.6)
57
X0,Y0 , , z
4.1,
E [x 0 , y 0, a, b, q ] =
B(t )
ij
(1.7)
pij S
(i - x 0 )2 ( j - y 0 )2
+
1
.
05
S [x0 , y 0 , a, b, q ] = t ij : 0.95
a2
b2
(1.8)
(x0, y0) a, b.
i cos q
j = - sin q

sin q x
cos q y
(1.9)

,
D :
] D (t )
E x 0 , y 0, a, b, q =
ij
(1.10)
pij S
pixels
,
.
, .
,
5- . ,
.
a/b
[1.1 1.6]. KL
pixels 2-
.

x y. x
,
58
a/b. y a/b
. 1.3
y.
z .

.
1.2.
. .
()
()
()
1.2: pixels
.
()
()
()
1.3:
2.

.
, .
mug shot
z.
59

. -
, , ,
1.4(). ,
.
()
()
1800
1600
1400
1200
1000
800
600
400
20
40
60
()
80
100
120
()
1.4: () , ()
, ()
pixels
, ()

D. .
pixels
. 1.11 Drc
:
ve ( y) =
D(i, y )
(1.11)
i =1

.
60

, :
y 0 = arg max (v e (i ))
i
(1.12)
z
. z
.
[y1 y2][1 r] y1<y0< y2
y=y0.
:
he ( x) =
y2
I ( x, i )
(1.13)
i = y1

:
he ( x ) =
y2
D ( x, i )
(1.14)
i = y1
1.13 1.14
:
x0 = arg min (he (i ))
(1.15)
x 0 = arg max ( he (i ))
(1.16)
1.14 1.16
.
. 1.13 1.15
.
1.11 1.16 .
,

..
. 1.13 1.15
1.14
1.16. 1.5 .
61
1.5:
2.1 Gabor

. pixel
.
Gabor.
.
Gabor pixel .

pixel. ,
.

.
Gabor .
:
Yk ( x ) =
k 2 x2
k2
exp
2s 2
s2
( )]
exp(- ikx) - exp s -2
(1.17)
x pixel , k ,
.
.
,
.

. :
x=
2
j
> x TH
(1.18)
, j
.
62
.

, .
.
.

.
,

.

.
Gabor :
x 2
y2
g ( x, y; u 0 , v0 ) = exp - 2 +
+
2
j
u
x
+
v
y
p
[
]
0
0
2
2s c 2s y
(1.19)
x y
. (uo,vo)
. Gabor , .
wavelet :
{[
)]
F l ( x, y , q ) = exp - l 2 x ' 2 + y ' 2 + jpx '
(1.20)
x=xcos+ysin y=xsin+ycos.
.
.
.
,
.
Gabor .
,
Fourier .
,
.
3.

. .
63

.
.
, . ,
1.
,
.
.
, 1.6().

. ui-1 , ui , ui+1
ri -1 , ri , ri +1 . l i
ui-1 , ui+1 mi ri l i .
ui 1.6():
(1.21)
Di = u i - l i
:
(1.22)
s i = sign( ri - rmi )
rmi mi .
()
()
1.6:

.
64

1.21 1.22.
.
4. 3-

. 6.1
. .
1.7:

profile
1-2-3
d (14,15) d (4,9)
7-8-9
d (12,13) d (14,15)
d (8,10) d (2,8)
d (5a ,5b) d (14,15)
5-6-7
d (4,6) d (14,15)
3-4-5
d (12a ,12b) d (14,15)
4-5-6
d (12c,12d ) d (14,15)
d (8,9) d (14,15)
d (8a ,8b) d (14,15)
65
1.1:

FDP
MPEG-4. (
frontal),
. 6.1

. 6.1
.
[31]-[33].
66
.
,
. [1],
.
.

. ,
, /
. . ,
, .
1.
face detection-
.
[2].

,
, [3][4].

.
[5][6],
[7], . ,
,
. ,
,
.

.
[1]
. [1]
YCrCb.

67
[7].

.
Cr-Cb
.

false alarms- .
dismissals- .
Y .

,
, .
: ()

, ()

.
2.
[8][9]
Cr-Cb [Y, Cr, Cb]. Wang 0,

.
.

. Wang

/ .
0 C
, x
Cr, Cb pixel :
1
exp{- (x - 0 ) T C -1 ( x - 0 )}
2
P(x | 0 , C) =
k
(2p ) 2 C
1
2
(2.1)
k = = 2
68
0 C pixels
, , .
, pixels pixels
,
/ .
pixels , MxN p(x) pixel
x pixel X C :
X I, Xc I , X Xc =I
:
X : {xI | p(x)CI }
C I = m x + s x , m x =
sx =
1
M N -1
1
M N
{ p( x) -m }
x
p( x )
pixels
xI
xI
L X m =
1
x
L xX
pixels X.
, (2.1),
pixels pixels
pixels 0
:
0 = (1 - m) 0 + m
(2.2)

pixels m .
X ,
C I .
.
, X pixels
.
m
.
pixels
:
69
p (x | w 1 )
> th x 1
p (x | w 0 )
(2.3)
p ( x | w1 )
< th x 0
p (x | w 0 )
(2.4)
th =
c01 p (w 0 )
c10 p (w1 )
(2.5)
1 pixel 0 . p(0)
p(1) a priori 0 1 . c01 c10
pixel 1 false alarm- 0
dismissal- .

.
(2.2)
.
.
False Alarms (%)
Dismissals (%)
35
30
25
20
15
10
5
0
Anchorpersons
Various
Webcameras
Photos
Scenes
2.1:

2.1
.
studio (
) .

(various scenes -, )
.

.
70
3.

. opening closing [11]
.
(morphological distance transform)
[8]
.
:
pixels pixel .
Si, i = 1n :
S i S j = i j,
US
=X.
i =1
D( S i ) Si 2.1 ()-
{ S1 S2 ... S n } Si, i = 1n,
d ( S i ) = max{D ( S i )} . S i > S j d ( S i ) > d ( S j ) .
{ S1 > S2 ... S n } .
S1 :
q = max{D ( S i )} q = d ( S1 ) . (markers)
i
:
: {x | D(x) q }, D(x) x.
M S1 . opening by reconstruction [11]
1, 2, 1 = S1 2 =
- S1 2.1() () .
2
S2 , S3 ,..., S n .
S i , S j , i j, d ( S i ) = d ( S j ) , nm
nm 2 M S i ,
M S j . M 1
nm1 = 1 M 1 M . S Sj
.
71
nm 2 M S i 1
S .
()
()
()
()
()
()
2.1: () ()
() () ()
marker
() (), ()

.
, .

72
. 2.1 .
2.1()
2.1().
4.
,
.
,
.

.

1.2 1.6.
. , ,
active contour models snakes- deformable templates [30]. affine
, ,
affine invariants [29].
.
splines
.
.
. -compactness-
:
gX =
4p a X
rX2
(2.7)
rX pixels - aX
pixels- X.
,
[0,1].
elongation-
,
Karhunen-Loeve . N1 x y
X. 22
(x, y) :
73
C=
1
[x - m x
N
y - m y ]T [ x - m x
y - my]
(2.8)
C .
:
l X = l2 / l1
(2.9)
1, 2 C .
(2.9)
.
,
,
[0,1]. ,
z ,
, 0.44 0.79 0.59 0.91
.
. ,
.

.
5.
,
,
:
M (u , q ) u(h,v),
h v, ( z) . F
, ,
A F
A M u :
A - M (u , q )
r (u , q ) = min
A F
r a b
r = 1 - c
(2.10)
h 2
- , , c
v 3
(0<c<0.5) a = mean(A) , b = mean(M )

. r
F .
74
[U , Q] = arg min{r (u ,q )}
M (U , Q) (3). A*
F.
, ,
.

3 U h

v
2*h . U
.

.
6.

.
.
.
.
6.1

.
5, .
: -
CMU, 20 ( ). 32
120x128 ,
.

.
.
: 2.1.
100
Pentium II / 233 MHz.

75
. -dismissal rate-
-false alarm rate.
Dismissal Rate (DR) (%)
False Alarm Rate (FAR) (%)
Mean Execution Time (MET) (sec)
131.7
2.1:
: .
.
/
.
2.2:
6.2
.
5 .

.
, ,
5.
: -
.
CIF format .
10 .
.
DR (%)
FAR (%)
MET (sec)
10
10
225.4
10
10
2.2:
76
: 2.2.
300 -10 30 -
.
.
: 2.2,
,
40 . ,

.
,
.
2.3:
6.3
2 5,
.
.
: -
CIF format.
.
: 2.3.
100 145
.
, . 2.5
,
.
.
, 24x24 pixel ,

.
77
:
,
.

. 2.4()
-
. -
2.4()
.
2.3
.
DR (%)
FAR (%)
MET (sec)
1.7
2.3:
()
()
2.4:
2.5:
6.4
.
,
,
78
.
, .
: B 6.2-
.
: 2.4.
600 -20 30 .
.
.
DR (%)
FAR (%)
MET (sec)
2.1
1.4
2.4:
: B
.

25%.
7.

, ,
, , ,
. .

. ,
, 100
,
.
79

,
(news-on-demand NoD),
.
teletext.
studio .

studio .
.

.

.
1.
,
, ,
,
[13].
, ,
MPEG-4 MPEG-7 [17].

: ()
, , ,
[22]. ,
, reporter ,
.

. ()

,
[19].
80

.

.
,
(NoD), [15][18][20][22].

teletext .
[21].

.

.
[19]
, ,
[20], [19]
idden Markov Models [16].
,
,
.
: () studio
, , ,
, () ,
, ,
, ()
.

,
.
, ,
,
.
.
81
2.
3.1.

.

,
,

3.1:
.

. :
: .
,
studio, ,
, , ,
studio .
: studio
.
.

.

.
.
.

:
82
studio (anchorperson shots): :
, ,

(report/interview shots): :
-
.
(static shots): :
.
(outdoor shot): :
( )

[14].

.
.
.

studio .
83
2.1
studio
.
, studio
.
:
I RI,GI,BI nm l R R (i, j ) hR ,
l G G (i, j ) hG , l B B(i, j ) hB , " i, j , {S1 , S 2 ,..., S N }
min{l R , l G , l B } min{S1 } , max{S N } max{h R , hG , h B } .
v I I :
v I = [ n1 ( RI ) n 2 ( RI )...n N ( RI ) n1 (G I ) n 2 (G I )...n N (G I ) n1 (B I ) n 2 ( B I )...n N ( B I )]T
ni (k ) pixels k { RI,GI,BI } S i .
J :
d = vI - v J .
2.2

false alarms studio
.
.

.
. z

.
3.

.

.

84
. ,
.
Si

m(Si)
(Si) ( pixels )
c(Si) .
x(Si )
Cr, Cb Si :
1
c(S i ) = exp{- (x( S i ) - 0 ) T C -1 (x(S i ) - 0 )}
2
(3.1)
f(Si) c(Si)
. l(Si) g(Si)
Si ,
3.2 l(Si) g(Si). f(Si)
: f ( S i ) = c( S i ) g ' ( S i ) l ' ( S i )
0.3
0.4
0.8
()
0.9
0.5
0.6
0.9
()
3.2: () ()
.

.
.

- ,

, .
85

, ,
.
.
4.
4 (A5, ET1, MEGA
ANT1) studio .
10 10
38428824bpp. studio-
3.1
.
()
()
3.1: () , ()
.
3.2.
100 15
. 3.3()

. 3.3()
- .
,
, false alarms.
3.3()-
.
studio .

.
: ()
- false
alarms, () , ()
86

.
. false alarms

2.2- 3.3().
, ,
.
,
,
,
. ,

2.1 3.3().
3.2: , MEGA,
100 , 15
.
precision recall
3.1.
.
precision [19]
false alarm- recall

dismissal.
2
.
87
1
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
100
200
300
400
500
600
700
800
900
1000
600
700
800
900
1000
600
700
800
900
1000
600
700
800
900
1000
()
1
0. 9
0. 8
0. 7
0. 6
0. 5
0. 4
0. 3
0. 2
0. 1
0
100
200
300
400
500
()
1
0. 9
0. 8
0. 7
0. 6
0. 5
0. 4
0. 3
0. 2
0. 1
0
100
200
300
400
500
()
100
90
80
70
60
50
40
30
20
10
0
100
200
300
400
500
()
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
600
650
700
750
800
850
900
()
3.3: () (01000), () (
), ()
, () ()
600-900.
88
A5 (a)
0.92
0.94
0.65
0.82
N/A
N/A
0.75
0.85
A5 (b)
0.95
1.00
0.83
0.94
050
1.00
0.73
0.87
ET-1
1.00
1.00
0.71
0.88
0.66
1.00
0.81
0.93
MEGA (a)
0.93
0.93
0.76
0.86
0.75
0.75
0.67
0.86
MEGA (b)
0.96
1.00
0.84
0.91
N/A
N/A
0.74
0.81
ANT1
0.93
0.94
0.77
0.88
0.75
0.66
0.85
0.86
0.95
0.97
0.76
0.88
0.67
0.85
0.76
0.86
3.1: Precision Recall

.
-anchorperson shots-
.
.

.
.
.
89
.
.
,
, .
.
,
,
, .
, , ,
.
.
1.

[23][28]. VIRAGE, QBIC,
Photobook VisualSEEk,
.
: () , ()
() , , ,
[24][13].

.

.
.

,
.
.

.
90

2.

- : ()
2
, ()

.

()
4.1:
2.

. -
2.

91
.
.
4.1.
/ ,
.

. :
() , () , ()
, ()
() .
3.

:
.
2. 0 4 ( 4

)
. I Y , I C r , I Cb
YCrCb , Si pixels-
mC r (S i ) =
1
Si
p S i
mC b ( Si ) =
1
Si
p S i
Cr
( p)
Cb
( p) Cb
Cr
Si.

I Y
g(Si)
(Si)

(indexing).

.
92
4.

.

- : ()
()
, .

.
4.1

F .
2
:
1
exp{- (x - 0 ) T C -1 (x - 0 )}
2
P( x | 0 , C) =
k
(2p ) 2 C
1
2
(4.1)
I Y , I C r , I Cb YCrCb
F :
mCr ( F ) =
1
F
Cr
( p)
(4.2)
mCb ( F ) =
1
F
Cb
( p)
(4.3)
pF
pF
m = [ mCr ( F ) mCb ( F )]T .

(4.1) :
1
exp{- (x - 0 )T C -1 (x - 0 )}
2
P(x | 0 , C) =
k
(2p ) C
2
0 = m 0 + (1 - m)
1
2
(4.4)
(4.5)
m (m =0.4 )

.
93
S i(k ) i- k- , x( S i( k ) ) = [ mCr (S i( k ) ) mCb ( S i( k ) )]T

c(S i(k ) )
S i(k ) (4.4)
:
1
c(S i( k ) ) = exp{- ( x( S i( k ) ) - 0 )T C -1 ( x( S i( k ) ) - 0 )}
2
(4.6)
:
k 0 = arg max{max{c ( S i( k ) )}}
k
(4.7)
(4.7)
.

.
.

.
4.2
F
n( F ) = [n00 ( F ) n20 ( F ) n02 ( F ) n30 ( F ) n03 ( F ) n12 ( F ) n21 ( F )]T . S i(k ) i-
k- n( S i(k ) )
:
k0 = arg min{ n( S i(0k ) ) - n( F ) }
(4.8)
i0 = arg min { n( S i( k ) ) - n( F ) }
(4.9)
.
(4.8)

.
.

.

94
x, y
z 4.1.
.
4.3

.
F (F)
. S i(k ) i- k-
A( S i(k ) )
:
k0 = arg min{ A( S i(0k ) ) - A( F ) }
(4.10)
i0 = arg min { A( S i( k ) ) - A( F ) }
(4.11)

(4.10)
.
4.4

.

4.1 m .
5.

4.
,
. 200 , 156
.
.
95
m=0.4
0.9992
0.9872
0.9735
0.9591
4.1:
0.0873
0.0883
0.0969
0.0985
4.2:
4.1
.
m=0.4.
96
4.1()
. 4.2
.
.
. 4.3
.

.
4.4
.
.
6.

, .

.

.

.
m=0.7
0.6369
0.5525
0.1581
0.1224
4.3:
97
0.9998
0.9444
0.9165
0.8416
4.4:
98
[1].
[2].
[3].
[4].
[5].
[6].
[7].
[8].
[9].
[10].
[11].
[12].
[13].
[14].
[15].
[16].
[17].
[18].
[19].
[20].
[21].
H. Wang and S.-F Chang, A Highly Efficient System for Automatic Face Region Detection in MPEG
Video, IEEE Trans. on Circuits and Systems for Video Technology, special issue on Multimedia Systems and
Technologies, August 1997.
A. Samal and P.A. Iyengar, Automatic Recognition and Analysis of Human Faces and Facial Expressions:
A Survey, Pattern Recognition, Vol. 25, No. 1, pp. 65-77, 1992.
G. Yang and T.S. Huang, Human Face Detection in Complex Background, Pattern Recognition, Vol. 27,
No. 1, pp. 55-63, 1994.
K.C. Yow and C. Cipolla, Feature-based Human Face Detection in Complex Background, Image and
Vision Computing Recognition, Vol. 15, pp. 713-735, 1997.
C. Garcia and G. Tziritas, Face Detection Using Quantized Skin Color Regions Merging and Wavelet
Packet Analysis, IEEE Trans. on Multimedia, vol. 1, no. 3, pp.264-277, September 1999.
Y. Avrithis, N. Tsapatsoulis and S. Kollias, Color-Based Retrieval of Facial Images, Proc. of EUSIPCO
2000, Tampere, Finland September 2000.
Y. Avrithis, N. Tsapatsoulis and S. Kollias, Broadcast News Parsing Using Visual Cues: A Robust Face
Detection Approach, Proc. of IEEE International Conference on Multimedia and Expo, July 2000, New York
City, USA
N. Tsapatsoulis, N. Doulamis, A. Doulamis, and S. Kollias, Face Extraction from Non-uniform
Background and Recognition in Compressed Domain, Proc. of ICASSP98, Seattle WA, May 1998.
T. Rzeszewski, A Novel Automatic Hue Control System, IEEE Trans. Consumer Electron., vol. CE-21, pp.
155-162, May 1975.
L. A. Harwood, A Chrominance Demodulator IC with Dynamic Flesh Correction, IEEE Trans.
Consumer Electron., vol. CE-22, pp. 111-117, Feb 1976
P. Maragos, Morphological Signal and Image Processing, Digital Signal Processing Handbook, V. Madisetti
and D. Williams, eds., IEEE Press.
N. Tsapatsoulis, Y. Avrithis and S. Kollias Face Detection for Multimedia Applications, Proc. of ICIP
2000, September 2000, Vancouver, BC, Canada
Y. Avrithis, A. Doulamis, N. Doulamis and S. Kollias, A Stochastic work for Optimal Key
Extraction from MPEG Video Databases, Computer Vision and Image Understanding 75 (1/2), pp. 3-24, July
1999.
P. Bouthemy, M. Gelgon and F. Ganancia, A Unified Approach to Shot Change Detection and Camera
Motion Characterization, IEEE Trans. CSVT 9 (7), pp.1030-1044, Oct. 1999.
M. Brown, J. Foote, G. Jones, K. Sparck-Jones and S. Young, Automatic Content-Based Retrieval of
Broadcast News, Proc. of ACM Multimedia Conference, San Francisco, CA, Nov. 1995.
S. Eickeler, A. Kosmala and G. Rigoll, A New Approach to Content-Based Video Indexing Using
Hidden Markov Models, Proc. of WIAMIS, Belgium, June 1997.
ISO/IEC JTC1/SC29/WG11, MPEG-7: Context and Objectives (v.5), Doc. N1920, 1997.
B. Merialdo, Automatic Indexing of TV News, Proc. of WIAMIS, Belgium, June 1997.
A. Merlino, D. Morey and M. Maybury, Broadcast News Navigation Using Story Segments, Proc. of
ACM Multimedia Conference, Seattle, WA, Nov. 1997.
Y. Nakamura and T. Kanade, Semantic Analysis for Video Contents Extraction Spotting by
Association in News Video, Proc. of ACM Multimedia Conference, Seattle, WA, Nov. 1997.
S. Tsekeridou and I. Pitas, Audio-Visual Content Analysis for Cont-Based Video Indexing, Proc. of Int.
Conf. on Multimedia Computing and Systems, Florence, Italy, June 1999.
99
[22]. H.J. Zhang, S.Y. Tan, S. Smoliar and G. Yihong, Automatic Parsing and Indexing of News Video,
Multimedia Systems 2, pp. 256-266, 1995.
[23]. D. Androutsos, K. N. Plataniotis and A. N. Venetsanopoulos, Extraction of Detailed Image Regions for
Content-Based Image Retrieval, Proceedings of IEEE ICASSP, Seattle WA, USA, May 1998.
[24]. A. Alatan, L. Onural, M. Wollborn, R. Mech, E. Tuncel and T. Sikora, Image Sequence Analysis for
Emerging Interactive Multimedia Services - The European Cost 211 work, IEEE Trans. Circuits and
Systems for Video Technology, Vol. 8, No. 7, pp. 802- 813, Nov. 1998.
[25]. O. J. Morris, M. J. Lee and A. G. Constantinides, Graph Theory for Image Analysis: an Approach based
on the Shortest Spanning Tree, IEE Proceedings, Vol. 133, pp.146-152, April 1986.
[26]. P. J. Mulroy, Video Content Extraction: Review of Current Automatic Segmentation Algorithms,
Proceedings of Workshop on Image Analysis and Multimedia Interactive Systems (WIAMIS), Louvain-la-Neuve,
Belgium, June 1997.
[27]. Y. Rui, T. S. Huang and S.-F. Chang, Digital Image/Video Library and MPEG-7: Standardization and
Research Issues, Proc. of IEEE ICASSP, pp. 3785-3788, Seattle, WA, May 1998.
[28]. Special Issue on Segmentation, Description and Retrieval of Video Content, IEEE Trans. Circuits and
Systems for Video Technology, Vol. 8, No. 5, 1998.
[29]. Y. Avrithis, Y. Xirouhakis and S. Kollias Affine-Invariant Curve Normalization for Shape-Based
Retrieval, Proc. of International Conference on Pattern Recognition (ICPR), Barcelona, Spain, September 2000.
[30]. A. D. Bimbo and P. Pala, Visual Image Retrieval by Elastic Matching of User Sketches, IEEE Trans.
PAMI 19 (2), pp. 121-132, 1997.
[31]. K. Karpouzis, G. Votsis, N. Tsapatsoulis and S. Kollias, Compact 3D Model Generation based on 2D
Views of Human Faces: Application to Face Recognition, in Machine Graphics and Vision, vol. 7, no.1-2,
1998.
[32]. N. Tsapatsoulis, K. Karpouzis, G. Votsis and S. Kollias, Analysis by synthesis of facial images based on
frontal and profile views, in Proceedings of the IWSNHC3DI97, Rhodes, Greece, September 1997.
[33]. G. Votsis, N. Tsapatsoulis, K. Karpouzis and S. Kollias, A Simplified Representation of 3D Human
Faces adapted from 2D Images, in Proceedings of the NMBIA98, Glasgow, UK, July 1998.
100

.
.
. ,
,
.
1.
, ,

.

slides []. slides

.

.

.
.

.
, bias ,
.

.

.

.
101
.
slides
.
.
2.
I nm L,
l I (i, j ) h , " i, j . T (k ) , k = 1,..., N l T (k ) h
T (1) = l , T ( N ) = h . S (k ) , k = 1,..., N , :
1 I (i, j ) T ( k )
S ( k ) (i, j ) =
0 I (i, j ) < T (k )
S ( N ) S ( N -1) ... S (1) = L .

opening by reconstruction markers m : Op ( r ) ( X , m) .
R (k ) :
R ( k ) = Op ( r ) ( S ( k ) , R ( k +1) ) R (N ) = Op ( r ) ( S ( N ) , M ) M S ( N ) .
R (k ) .
::
a ( k ) = R ( k ) - R ( k +1) a (N ) = 0
c (k ) =
4 p R (k )
P
(k ) 2
(5.1)
(5.2)
P (k ) R (k ) .
(1) R (k )
Op (r ) (2) R (k ) .
D dilation
erosion G= D -
I R (k ) - R ( k +1 ) :
b ( k ) = max G{R ( k ) - R ( k +1) }
(5.3)
:
(1) M S (N ) R (N ) = Op ( r ) (S ( N ) , M )
102
(2) R ( k ) = Op ( r ) (S ( k ) , R ( k +1) ) k
a (k ) c (k )
b (k ) .
R (k ) .
(3) R (q ) (2).
S (k ) ( R (k ) S (k ) ) = 0 , " k q
(4) (1)-(4). S (N ) =0 (1) M S ( N -1)

S ( N -1)
=0 M S ( N - 2) .
R (q )
. ,
R (q ) .
:
()
()
()
()
()
()
()
()
103
()
()
()
()
5.1:
5.1() .

opening by reconstruction . 5.1()

. ,
5.1()
. 5.1() ()
5.1() 5.1(). 5.1()
.
.
3.

.
.
[33].

Y, Cr, Cb
Txt :
Txt = 1 -
1
1+ s 2
(Y probe)
(5.4)
s2
(Yprobe).
104

RBF (Radial Basis Function)
: , , , , .

5.1.
45% : 20% => , 15% => ,
10% => , (11)+(12)+(13)=6
2+.
(%)
0 25 %
26 50 %
14
1+
58
2+
51 75 %
9 12
3+
76 %
13
4+
5.1:
4.

. 28 . 24bit
480640.
: ()
.
3. ()
slides
.

- .
. :

. ,
.
105

.
(S Sensitivity) (PPV - Positive Predictive Value).

:
S=
TP
TP + FN
(5.5)
TP (true positive)
FN (false negative)
.

:
PPV =
TP
TP + FP
(5.6)
FP (false positive)
.
5.1 .

OR. .
5.2
.

.
.
. ,
100% .

contrast 5.2- .
, ,
FP.
.

.
.
106
-
.
S +/- SD
PPV +/- SD
Exp1 - Exp2
72.6 +/- 15.1
74.5 +/- 14.3
System - Exp1
84.6 +/- 8.7
61.7 +/- 15.2
System - Exp2
86.8 +/- 7.4
65.8 +/- 14.7
System OREx
83.3 +/- 9.6
75.6 +/- 13.1
5.2:
5.2: contrast
107
:
,
.
1.
, ,
, ,

.
,
,
[1]. ,
[3][4][6]. ,
,
[6]. Karhunen-Loeve Principal Component Analysis
,
[4][6], .

[15] [16].

.
.
.

.
,
, .

.
108

.
2.

- head
format.
- .
1.1.

(
" "

)
"
"
1.1: .

.
.

, , ,
.

.
3. M

: .
109
,

pixels. :
x = {x i , i S}
(1.1)
S . x
pixels ,
:
x = [ x1 , x 2 ,... x n ]T
(1.2)
n = S pixels . x n n n .
: .

.
,
, (1.1) (1.2), ,

:
f ( x) = [ f 1 ( x), f 2 ( x),... f m ( x)]T
(1.3)
f 1 (), f 2 (),... f m () . m<<n

..

n . e1 , e2 , e3 ,...e n
x :
x=
ei
(1.4)
i =1
xi = x ei x ei x
x = [ x1 , x 2 ,... x n ]T .
(1.2) ei = [0,0,...,0,1,0,...,1]T
i- Fourier ei =
1
n
[1, e
j 2p
i
n
,e
j 2p 2
i
n
,..., e
j 2p ( n -1)
i
n T
] .
x i 0 i m x
m- :
110
x = [ x1 , x 2 ,... x m ]T
(1.5)

[21].

. x
c k , k = 1,2,...K , x
c k 0 :
k 0 = arg min d ( x, c k )
k
(1.6)
d ( x, c k ) x c k .
n ( ) (1.6) :
k 0 = arg min x - c k
k
(1.7)
ck c k = c k
(1.7) x c k :
k 0 = arg max x, c k
k
(1.8)

(1.7) (1.8)
x ck .
, ,
,
.
(ML-maximum likelihood) :
k 0 = arg min log p( x | c k )
k
(1.9)
p ( x | c k ) x k- . ML

.
3.1 Karhunen-Loeve

.
Karhunen-Loeve.
Fourier DCT, Karhunen-Loeve
111
.
x.
:
m x = E{x}
(1.10)
E{} .
:
C = E{ ( x - mx ) ( x - m x ) T }
(1.11)
KL C
eigenfaces . C u1 , u 2 ,...., u n
l1 l 2 .... l n x :
x=
ui
(1.12)
i =1
i i >m- :
x
u i , x = [ x1 , x 2 ,... x m ]T
(1.13)
i =1
(1.13) m<n
x m
. i u i
- , m
.
KL
,
- 1.1.
u1

4

3

2

2
1.1:

112

:
k 0 = arg min x - ck
(1.14)
KL mx
C. N {I 1 , I 2 , .... I N }
I i lxm . pixels I i
x1 , x 2 ,..., x N xi R n n = l m . mx
C :
mx =
(1.15)
i =1
C=
1
N
(x
- m x ) ( xi - m x ) T =
i =1
1
XX T
N
(1.16)
X = [ x1 - m x , x2 - m x ,..., x N - m x ] . KLT
:
L = FT C F
(1.17)
F C , L
.

, C R n n .
90x100, C R 9000 x 9000 .
(singularity) C
N > n 2 .
C
SVD X. XX T
X T X N N .
N << n - .
X r, r N SVD X :
X=
l k u k v Tk
(1.18)
k =1
113
u k v k lk
XX T , X T X . (1.18) vk
u k :
1
lk
X vk = u k
(1.19)
3.2 Fisher Linear Discriminant FLD

FLD.
w1 , w 2 ,..., w L N1 , N 2 ,..., N L . M 1 , M 2 ,..., M L
.
nn b nn :
Sw =
p (w i ) C i =
i =1
Sb =
p(w ) E{[ x - M ] [ x - M
i
]T | w i }
(1.20)
i =1
p(w ) (M
i
- M ) (M i - M )T
(1.21)
i =1
p (w i ) (a priori) Ci w i .
FLD ,
- ,
n
Y T Sb Y
Y T Sw Y
Sw-1 S b
:
S w-1 S b Y = Y D
(1.22)
Y, D n n Sw-1 S b .
b Sw-1 S b
. Y D
b [21].
FLD PCA
.

.
.
114
FLD : ()
- ( )
Ci b , ()
Sw-1 S b
PCA .
PCA

FLD .
115
KL
KL
(VKLT).
(singularity) . VKLT
KLT.
KLT- ,
.
VKLT KLT ,
.
1.

, ,
- y- - z- [3][4].
.

,
[2].
-
.
Karhunen
Loeve (KLT- Karhunen Loeve Transform) Eigenfaces.
, (PCA- Principal Component
Analysis)
1.
: ()

.
, ()
,
, singular.
116
KL
(VKLT).
(singularity) .
VKLT KLT.
KLT-

. VKLT KLT
.
2. KLT
eigenfaces KL
[4][6].
KLT .
{I 1 , I 2 , .... I N } I i nxm
. ,
:
N
Ro = ( I j - M ) ( I j - M ) T
(2.1)
j =1
:
N
Co = ( I j - M ) T ( I j - M )
(2.2)
j =1
M = I j . Ro R nxn , Co R mxm
j =1
C (1.16)
1. :
L 1 = F 1 T Ro F 1
(2.3)
L 2 = F 2 T Co F 2
(2.4)
F 1 F 2
, , .

.
q 1 2
L 1 L 2 Yi :
117
)
Yi = F 1q T I i F 2 q
(2.5)
I i = I i - M
VKLT, KLT
: .
3.
VKLT.
S = {I 1 , I 2 ,.... I N }
V = {v1 , v 2 , .... v N } , F k , F 1q F 2l
F - (1.17), F 1 F 2 k ,q l
( ).

KLT VKLT, k ,q l.
,
k = l q .
:
)
(
)
(
e KLT ( k , j ) = ( v j - v j ) T (v j - v j )
v( j = F k y j ,
)
y j = Fk T v j
(2.6)
)
(
)
(
eVKLT (q , l , j ) = vec( I j - I j ) T vec ( I j - I j )
(2.7)
vec( I ) I,
(
T )
Yi = F 1q I i F 2l , I i = F 1q Yi F 2l T .
(2.6) (2.7)
KLT VKLT :
meKLT ( k ) =
1
N
meVKLT (q , l ) =
eKLT (k , j)
(2.8)
j =1
1
N
eVKLT (q, l, j)
(2.9)
j =1
118
, ,
.

.
KLT VKLT
,
.
I i vi . Ti
I i Li I i
. t i li . KLT,
:
)
teKLT ( k ) = F k T vi - F k T t i
(2.10)
T )
T
le KLT ( k ) = F k v i - F k l i
(2.11)
vi t i li .
I i Ti Li VKLT:
)
teVKLT (q , l ) = vec(F 1q T I i F 2l ) - vec(F 1q T Ti F 2l )
(2.12)
)
T
T
leVKLT ( q , l ) = vec(F 1q I i F 2l ) - vec(F 1q Li F 2 l )
(2.13)

(2.10) (2.13) (2.8)
(2.9).

. :
.

: , , , ,
.
.
.

( ).
119

KLT VKLT k = l q .
4.

ORL.
.
KLT 64x72.
2.1: () ()
() ()
45
40
35
30
25
20
KLT
15
VKLT
10
10
15
20
25
2.1: KLT VKLT

.
2.1
KL VKL. VKLT ,
. 2.3, 2.4
2.5, VKLT KLT,
KLT . VKLT
120
0.3
0.2
KLT
0.1
VKLT
10
15
20
25
2.2: KL VKL
.
40
30
20
KLT
10
VKLT
10
15
20
25
2.3: KLT VKLT

.
16
25
36
49
(%)
KLT
54
63
75
82
VKLT
62
69
81
85
2.1: KLT VKLT
121
DCT
DCT
JPEG 88- .
DCT
.

.
1.

DCT, JPEG, .

-

. , ,
mug shot ,
.
2.
3.1.
JPEG
- .
PCA .
.

-
.
.
122

(JPEG )
PCA
3.1: .
3.2:
DCT
3.
3.2.
- DCT ,
, .

. ,

.
123
5 88 512512
20480 - PCA .
FLD
. PCA FLD
1. - DCT
JPEG
.
4.

.
.
JPEG.
. x
y , z =x y c=max(z)
. u
x :
u = yi , i = arg max{
k
max( x y k )
}
x yk
,
DCT .

.

PCA
.
5.
ORL
mug shot .
(KL)
64x72. ,
PCA DCT .
.
ORL ,
x, y z. eigenfaces,
124

DCT .
3.1.
BDCT (Block DCT) 72 (
8x8 64x72)
. ,
BDCT
.
(%) ( )
KLT
44 (16)
54 (25)
65 (36)
77 (49)
DCT
52 (16)
59 (25)
73 (36)
84 (49)
BDCT
81 (72)
81 (144)
85 (216)
88 (288)
3.1: .
6.

DCT.
mug shot
,
.
. PCA
FLD 1.
125

.
(singular vectors)
. :
;,
.
(singular values).
, DCT
JPEG 3.

.
1.

.

.
.
,
. ,

- face versions.

.
.
DCT
[7].
2.
.

(Singular Value Decomposition -SVD),
A mxn.
126

.
A mxn ( m>n,
) rank(A) = k . U mxm , Vnxn
mxn :
T
A=U V
(4.1)
= diag (l1 , l 2 ,.., l k ,0,..,0) and l1 l 2 .. l k .
l2i ,
A T A AA T , i
A . U = (u1 ,.., u k , u k +1 ,.., u m ) V = (v 1 ,.., v k , v k +1 ,.., v n )

u i vi
l2i A A , AA
T
x nx1 A :
x nx1 = [l1l2 ...lk 0...0]
(4.2)
A l1 l 2 .. l k , x nx1
.
, x nx1
[5].
. , ,
,
.
2.1
A .
A
. A .
A A l1 , l2 ,.., lk l1* , l*2 ,.., l*k .
(a)(aA) T :
(a )(aA )T
- l*2 I = 0
AAT - (
(4.3)
1 *2
l I) = 0
a2
A A :
(l , l ,.., l ,0,..,0)
*
1
*
2
*
k
= a (l1 , l 2 ,.., lk ,0,..,0 )
(4.4)
127

.

. 4.1
.

. 4.1()
.
.

.
()
()
25
20
15
10
1 .5
2 .5
3 .5
4.5
()
4.1:
.

.

.
.
128
x nx1 = [l1l2 ..lk 0..0]T .

:
a nx1 = {li = l1 , i = 1..k | li = 0, i = k + 1.. n}
b nx1 = {li =
c nx1 = {li =
(4.5)
l1
, i = 1..k | li = 0, i = k + 1 .. n}
i
l1
i2
(4.6)
, i = 1..k | l i = 0, i = k + 1: .. n}
(4.7)
4.2

.
, (4.5)-( 4.7),
.
2 50
2 00
1 50
1 00
50
10
()
2 24.5
224
2 23.5
223
2 22.5
222
10
()
2 50
2 00
1 50
1 00
50
10
0
1
10
()
250
200
150
100
50
()
4.2: () .
(),(),()

(4.5),( 4.6) (4.7) .
129
3.
I S ,
, .
:
I = U I I VI T =
T
Ij v Ij
(4.8)
T
Sj u Sj v Sj
(4.9)
Ij
j =1
S = U S S VS T =
j =1
I S
)
S , face version S , (4.10):

)
S = U S I VS T =
u
Ij
(4.10)
T
Sj v Sj
j =1
(4.8), (4.9), (4.10) S

US , VS I
I . S I
4.3.

)
S face version S .
()
()
()
4.3: () () ()

-Face version.
4.
face
versions .

DCT
. 10%
130
.
3 [7].

- 5.
2 4.
face
versions .
4.1
88.
, 10%
DCT.
. 6
99%.
ORL.
DCT ( 88)
1
(%)
81
95
99
4.1:
- .

face versions .
S m
Frobenious (l2)
S k I :
S m = arg min S k - I
k
= arg min U S k I VS k
k
- U I I VI
T
F
(4.11)
.
(4.11).
Frobenious :
I - S
2
F
= tr ( I - S )T ( I - S )
= tr ( I T I ) + (S T S ) + (S T I ) + ( I T S )
}
131
= tr ( I T I ) + tr (S T S ) - tr (S T I ) - tr ( I T S )
= tr ( I T I ) + tr (S T S ) - 2 tr ( I T S )
(4.12)
tr ( A) A F Frobenious
.
(4.12) :
T
tr ( I T I ) = tr{(U I I VI ) T (U I I VI )}
= tr (VI I T U I T U I I VI T )
= tr (VI I T I VI T )
k
= tr ( li2 v Ii v Ii T )
i =1
= li2 v Ii
(4.13)
i =1
v Ii i- IT I k .

v Ii
= 1 U I , VI (5.5.2)
:
tr ( I T I ) =
2
i
(4.14)
i =1
tr ( S T S ) = l2i ( S )
i =1
:
I - S
2
F
=2
2
i
- 2 tr ( I T S )
(4.15)
i =1
tr ( I T S )

2
i
i =1
I T S :
T
T
T
T
I T S = VI I U I U S I VS = VI RVS
(4.16)
132
l12 

l1 l2 ...
T
T
R = I U I U S I = l1 l 2 l22 ...
...
...
...
(4.17)
 u Ii u Sj . R

l12 
0
...
2
=
R
0
l 2 ...
...
...
...
(4.18)
tr ( I T S ) :
V T ) = tr (
tr ( I T S ) tr ( VI R
S
l v Ii v S i T )
i =1
= l2i < v Ii , v Si >
(4.19)
i =1
l < v Ii , v Si >
i =1
1 i = j
1 i = j
< v Ii , v Sj >=
=
0 i j
0 i j
tr ( I T S )
I - S
I - S
= 2(
i =1
i =1
l2i - l2i < v Ii , v Si >)

k
2
i
(4.20)
i =1
S m = arg min S k - I
k
= arg min U S k I VS k
k
- U I I VI
T
F
(4.21)

S m = arg max
L
l < v Ii , v S L i >
(4.22)
q q<<k
q=5. (4.22) :
133
S m = arg max
L
l < v Ii , v S L i >
(4.23)
(4.11)
5.
ORL.
s, s*
:
s* S m = arg min S k - I
s S m = arg min S k - I
k
= arg min U S k I VS k - U I I VI
= arg min U S k S k VS k
k
- U I I VI
(4.24)
(4.25)
(4.25) (4.24).
.
4.2.

1
(%)
s
76
79
83
s*
77
85
91
4.2: , s s*

face version.

.

.

. 4.1
, ,

.

4.2-
134

- , .
Mean
Distance
2.8
2.6
_____ : Mean distance (s)
- - + - - : Mean Distance (s*)
2.4
2.2
1.8
0
10
15
Database Faces
4.1:
s s*.
0.26
Mean
Distance
0.24
0.22
_______ : Mean Distance (s)

- - + - - - : Mean Distance (s*)
0.2
0.18
M.D. for target

face (s)
0.16
M.D. for target

face (s*)
0.14
0.12
1
1.5
2.5
3.5
4.5
Group of candidate faces
4.2:
, s s*.
6.

.

.
.
135

, , , .

, ,
.
, o
, , ,
. ,
.

, ,
.
1.
.
.
1.

.

.
.
2.

.
2.1
(Moments)
[18].
pixels
136

.
. (
)
.
g(x,y)
:
m uv =
g ( x, y ) x
x
yv
u, v=0,1,2,3
(5.1)
.

.
.
, (central
moments) m uv
.
m uv = g ( x, y )( x - x ) u ( y - y ) v
x
where x =
m10
m00
and y =
m 01
m 00
(5.2)
( x, y ) .
(Centroid).
:
m 00 = m 00 ,
m10 = m 01 = 0 ,
m11 = m11 - y m10 ,
m 30 = m30 - 3 x m 20 + 2 x m10 ,
m12 = m12 - 2 y m11 - x m02 + 2 y 2 m10 ,
m 20 = m20 - x m10 ,
m 02 = m02 - y m01 ,
m 03 = m 03 - 3 y m02 + 2 y m01 ,
m 21 = m21 - 2 x m11 - y m 20 + 2 x 2 m01 .
:
n 00 = m 00
n ij =
m ij
( m 00 )
k=
i+ j
+1, j + i 0
2
(5.3)
n10 = n 01 = 0
n11 0 . .
.

137
,
. (Scale Invariant
Representation-SIR) : SIR = [n 00 , n 20 , n 02 , n30 , n 03 , n12 , n 21 ]T

[15].

(Self Organized Maps - SOM)
Kohonen [19].

.
, .
SOM
. SOM

.
SOM SIR .

.
5.1. SOM Kohonen

.

- SIR
5.1: SOM
2.2
4,

. , A
l1 > l 2 > .. > l k , x nx1 .
[8], x nx1
138
CIR -Content Invariant Representation-

CIR = [l1 l 2 ,.., lk ] .
2.3 - Luminance Invariant Representation

.
. 4 [20] -
.
.
LIR Luminance Invariant Representation-
U s , Vs - 4.
3.
5.2:

CIR, LIR SIR.

.
SIR
LIR
CIR
5.2:
3.1
SIR, SOM,
.
, , SI
SOM. .

SIR ( 5.3).
139
SIR
x
x
5.3: SOM

.
LVQ (Learning Vector Quantization) ,
SOM . SOM
LVQ
5.1 l1.
(%)
l1- (%)
LVQ (%)
67
79
85
SOM
70
83
96
5.1:
3.2
,

.

.
4.
, .

.
140
3.3
4

.

.
, .
I = U I I VI T .
Yi
I Yi
: Y i = U I y i VI T .
Ym Frobenius
Y i I :
- I = U V T - U V T = U ( - )V T
Ym = arg min Y
i
I yi I
I I I
I
yi
I
I
(5.4)
Y i = Yi = y i , (5.4) :
Ym = arg min yi - I = arg min

i
i
(l
j
yj
- lIj ) 2
(5.5)
l yj j- Yi .
(5.5),
Frobenius Y i I
.
4.

.
.
, ,
.
, .
, ,
, (
141
). IF-THENELSE .

I S ,
I L I C
SIR, LIR CIR . :

:
SOM SI .
S
L:
m LI (m =5%
).
C:
m CI .
CS :
k (k<<m) SIR.
LS :
k SIR.
SL :
k LIR.
CL :
k LIR.
SC :
k CIR.
LC :
k CIR.
1:
IF ( I S = I L OR
2:
IF ( I S
OR
IS CS
) THEN = I S
3:
IF ( I L S L OR
IL CL
) THEN = I L
4:
IF ( I C
IC LC
5:
IF = THEN
LS
SC
OR
IS
= I C ) THEN = I S ELSEIF
IL
= I C THEN = I L
) THEN = I C

.

. 1
.

142
. ,

.
2, 3 SIR.
, .

.
.
4 LIR. 1-3
SIR.
. LIR
.
LIR.
, .
CIR .
. ,

.
5.
Bern.
,
. 100
. 5.2,

.
94%.
(%)
SIR
96
CIR
63
LIR
66
( )
(%)
(%)
(%)
60
95
52
50
55
99
69
71
72
94
5.2:
143
6.

.
(
, ,
)
.
.
.
144
[1].
[2].
[3].
[4].
[5].
[6].
[7].
[8].
[9].
[10].
[11].
[12].
[13].
[14].
[15].
[16].
[17].
[18].
[19].
[20].
[21].
IEEE, vol. 83, no. 5, pp. 705-740, 1995.
J. Daugman, Face and Gesture Recognition: Overview, IEEE Trans. on PAMI, vol. 19, no 7, pp 675676, 1997.
A. Doulamis, N. Tsapatsoulis, N. Doulamis and S. Kollias Innovative Techniques for the Recognition of
Faces Based on Multiresolution Analysis and Morphological Filtering, Proc. of IWISP, Manchester,
November 1996.
B. Moghaddam and A. Pentland, Probabilistic Visual Learning for Object Representation, IEEE
Trans.on PAMI, vol. 19, pp. 696-710, 1997.
N. Tsapatsoulis, K. Karpouzis, G. Votsis and S. Kollias, Analysis by Synthesis of Facial Images Based on
Frontal and Profile Views, Proc. of IWSNHC3DI Conference, Rhodes, September 1997.
M. Turk and A. Pentland, Eigenfaces for Recognition, Journ. Cognitive Neuroscience, vol. 3, no. 1, 1991.
Z. Hong, Algebraic feature extraction of image for recognition, Pattern Recognition, vol.24, pp. 211-219,
1991.
L. Chiariglione, MPEG and Multimedia Communications, IEEE Trans. on Circuits and Systems for Video
Techn. vol. 7, no. 1, pp. 5-18, Feb. 1997.
A. Doulamis, N. Doulamis, and S. Kollias, Retranable Neural Networks for Image Analysis and
Classification Proc. of IEEE Int Conf. on Syst. Man & Cybern., Orlando, October 1997.
S. Kollias, N. Doulamis, and A. Doulamis, Improving the Performance of MPEG Compatible Encoders
Using on Line Retrainable Neural Networks, Proc. of ICIP97, Santa Barbara, October 1997.
P. Maragos, Differential Morphology and Image Processing, IEEE Trans. Image Processing, vol. 5, pp. 922937, June 1996.
E. Reusens, T. Ebrahimi, C. Le Buhan, R. Castagno, V. Vaerman, C. de Sola Fabregas, S. Bhattacharjee, F.
Bossen, and M. Kunt, Dynamic Approach to Visual Data Compression. IEEE Trans. on Cir. & Syst. Fo
Video Techn., vol. 7, pp. 197-211, February. 1997.
G. Cottrell and M. Flemming, Face recognition using unsupervised feature extraction, Proc. Int. Conf.
Neural Network, pp 322-325, Paris 1990.
S. Lawrence, et. al,. Face Recognition: A Convolutional Neural Network Approach, IEEE Trans. on
Neural Networks, Vol. 8, No. 1, pp. 98-113, 1997.
P. Maragos, Morphological Signal and Image Processing, Digital Signal Processing Handbook, V. Madisetti
and D. Williams, eds., IEEE Press.
David Vernon, Machine Vision, Prentice Hall, 1991.
T. Kohonen, Self-Organisation and Associative Memory, Berlin: Springer, 1988.
M. Leonidou, N. Tsapatsoulis and S. Kollias, An Illumination Invariant Face Recognition Algorithm, in
Proc of the JCIS 98, North Carolina, USA, October 1998.
. Fukunaga, An Illumination Invariant Face Recognition Algorithm, 2nd ed, New York: Academic
Press, 1990.
,
.

.
.

. , ,
,
.

.

.
.

.
.

.
:
, .
-apex.
,
,

..
.
[25]
. ,
,
.

.
.
.

:
.
.
,
, .

.
.
.
.

:
.

: ,
.
.

.

,
. :
. PCA
FLD .
multilayer perceptron backpropagation.
1.

. .
, () () ,
Brunelli Poggio [11],

- [12],
.

(MLP)
( , PCA-FLD).
2.

CMU .
20 ( ), 120x128
, , .
1 2.
.
35x37 77 .
.

. 64
(16x416 4 ).
pixel pixel . 2.1
, , .
2.1: , ,
16
.

, 75%
50%
.
, ,
,
.
M 3 =
1
k l
R1
R2
mean( R1) - mean( R2) ,

.
3. FLD

[12]-[18] 1.

.
. 1 PCA

. [15] k
, ,

.
1 FLD

.
.
Ci .
Sw-1 S b .
PCA
KL
FLD -
.
3.1 PCA
PCA
[13],
.
PCA :
N {I 1 , I 2 , .... I N } I i R nxm ,
, , , {v1 , v 2 , .... v N } vi R L L = n m ,
. KLT :
L = FT S F
(2.1)
S , F S , L
.
x = [ x1 , x 2 ,... x m ]T v i
m m
- KLT:
T )
x ( i ) = F k vi
v)i = v i - m ,
(2.2)
N
m = v j F k
j =1
k .
2.2 36
. , ,
12 18 ,
.
2..2: 36
64
, , .
3.2
PCA
x ( i ) = [ x i1 , x i 2 ,...x im ]T . w1 , w 2 ,..., w L , N1 , N 2 ,..., N L ,
. m1 , m 2 ,..., m L
, :
mi =
1
Ni
Ni
x ((ki )) , M =
k =1
1
L
Li
(2.3)
i =1
x ((ki )) k- w i
L=4 , ,
- N1 = N 2 = ... = N L = 16 .
b :
Sw =
p(w i ) C i =
i =1
Sb =
p(w ) E{[ x
(i )
- mi ] [ x (i ) - mi ]T | w i }
(2.4)
i =1
p(w ) (m
i
- M ) (m i - M ) T
(2.5)
i =1
a priori p (w i )
, . Ci :
Ci =
1
Ni
Ni
( x
(i )
(k )
- mi ) ( x ((ki )) - mi ) T
(2.6)
k=
, -
Sw-1 S b :
S w-1 S b Y = Y D
(2.7)
Y, D m m .
:
y = Y T x y = YkT x
(2.8)
Yk = [y 1 ,y 2 ,...y k ] k (k<m)
Sw-1 S b . :
k 0 = arg min y - YkT mi
i
(2.9)
.
FLD PCA 100%

- 60%.

.
4.
multilayer perceptron (MLP)
[19][20].
feedforward-
.
.

.
.

,
.
, 2.1,
. n
n p<<n .
.

.
.
[30]
-
p PCA.
2.1:

-35x37- pixels
x ( i ) = [ x i1 , x i 2 ,...x im ]T
PCA.
.
backpropagation [23].
4.1 MLP
20 .
35x37 , 20 4 .
100%
75%.
25%,

. 4060%. 2.3 20 .
, ,
. 2.2 2.3
MLP
.

( ).

( ).
2.3: 20 hidden-
MLP- 1295x20x4
, , .
4.2
5.
100% , 64/5 = 12.8
. 20

2.4.
,
.

, ,
, .

.
.
z
.

.
2.4: 5 hidden-
MLP- 1295x5x4
, , .
4.3 PCA

36 12 4.
12 .
PCA
FLD.
36
10 -
. 65%,
FLD.
.
5.

.
()
()
.
. PCA-

. MLP backpropagation
,

. ,
multilayer perceptron,

,

.

:
1) :
(a). - 35x37 pixel
(b).
(, )
(c).

2)
3)
: ()
, ()

[24][25] [26]
.
[27, 28]
[29]
.
A FDP FAP MPEG4
Facial
Definition Parameter Set (FDP) Facial Animation Parameter Set (FAP) [1] MPEG-4.
MPEG-4, FAP
.
[24].

FDP.
,
,

.
.
[3]
.
1.
, ,
[4].
[5]
.
[3][6] .

. Whissel [3]

- activation - evaluation.
,
.

. 2.1 2.1
,
,
Whissel ( delighted 4.2 eager 5).
MPEG MPEG-4
.
-Facial Animation Parameters (FAPs)-
- Facial Definition Parameter Set (FDP)- MPEG-4
.
FAP FDP
: .
FDP,
. 1.
MPEG-4
.
()
()
3.1: -delighted
-eager
3.2: () FDP ()
FAPs (The Facial Animation Parameter Units -ES = ESo/1000, NS
= ENSo/1000, MNS = MNSo/1000, MW = MWo/1000) [1]
,
FDP

.
,
,
[3].
Afraid
Bashful
Disgusted
Guilty
Patient
Surprised
Activation
Evaluation
4.9
2
5
4
3.3
6.5
3.4
2.7
3.2
1.1
3.8
5.2
Angry
Delighted
Eager
Joyful
Sad
Activation
Evaluation
4.2
4.2
5
5.4
3.8
2.7
6.4
5.1
6.1
2.4
3.1: [3] [6]
2.
(Facial Definition Parameter set FPD)
(Facial Animation Parameter set FAP)
, ,
. FAP
,
, .
(FDPs)
.
.
(FDPs)
FAPUs (FAP).
(FAPs)
.

.

.
(Facial Animation
Parameters Units).

.
.
FDP
.
,
, .

, FDP ,
FAPs. FAP,
FDP

.
3. FDPS
[4] ,
,
.

.

. , ,
.
FDP
.
3.2
FAPs [1].
[2][3], ,
Ekman
- MediaLab .
3.3 FDPs FAPs
.
FAP FDP
FDP.
: FAPs
-apex- . fi-NEUTRAL
.
squeeze_l_eyebrow (+)
squeeze_r_eyebrow (+)
lower_t_midlip(-)
raise_b_midlip (+)
raise_l_i_eyebrow (+)
raise_r_i_eyebrow (+)
close_t_r_eyelid (-)
close_t_l_eyelid (-)
close_b_r_eyelid (-)
close_b_l_eyelid (-)
close_t_l_eyelid (+)
close_t_r_eyelid (+)
raise_l_m_eyebrow (-)
raise_r_m_eyebrow (-)
raise_l_o_eyebrow (-)
raise_r_o_eyebrow (-)
close_b_l_eyelid (+)
close_b_r_eyelid (+)
raise_l_o_eyebrow (+)
raise_r_o_eyebrow (+)
raise_l_m_eyebrow (+)
raise_r_m_eyebrow (+)
squeeze_l_eyebrow (-)
squeeze_r_eyebrow (-)
open_jaw (+)
stretch_l_cornerlip (+)
stretch_r_cornerlip (+)
lift_r_cheek (+)
lift_l_cheek (+)
lower_t_midlip (-)
OR open_jaw (+)
raise_b_midlip (-)
lower_t_midlip (-)
open_jaw (+)
squeeze_l_cornerlip (+) AND / OR squeeze_r_cornerlip (+)
raise_l_o_eyebrow (+)
raise_r_o_eyebrow (+)
squeeze_l_eyebrow (+)
squeeze_r_eyebrow (+)
open_jaw (+)
OR
close_t_l_eyelid (-)
close_t_r_eyelid (-)
lower_t_midlip (-)
OR lower_t_midlip (+)
3.2: FAPs
FAP
squeeze_l_eyebrow
f1 = s (1,3) / F1 = f1 - f1-NEUTRAL
F1 < 0
squeeze_r_eyebrow
f2 = s (4,6) / F2 = f2 - f2-NEUTRAL
F2 < 0
lower_t_midlip
f3 = s (16,30) / F3 = f3 f3-NEUTRAL
F3 < 0
raise_b_midlip
f4 = s (16,33) / F4 = f4 f4-NEUTRAL
F4 < 0
raise_l_I_eyebrow
f5 = s (3,8) / F5 = f5 f5-NEUTRAL
F5 > 0
raise_r_I_eyebrow
f6 = s (6,12) / F6 = f6 f6-NEUTRAL
F6 > 0
raise_l_o_eyebrow
f7 = s (1,7) / F7 = f7 f7-NEUTRAL
F7 > 0
raise_r_o_eyebrow
f8 = s (4,11) / F8 = f8 f8-NEUTRAL
F8 > 0
raise_l_m_eyebrow
f9 = s (2,7) / F9 = f9 f9-NEUTRAL
F9 > 0
raise_r_m_eyebrow
f10 = s (5,11) / F10 = f10 f10-NEUTRAL
F10 > 0
open_jaw
f12 = s (30,33) / F11 = f11 f11-NEUTRAL
F11 > 0
close_t_l_eyelid close_b_l_eyelid
f12 = s (9,10) / F12 = f12 f12-NEUTRAL
F12 < 0
close_t_r_eyelid close_b_r_eyelid
f13 = s (13,14) / F13 = f13 f13-NEUTRAL
F13 < 0
stretch_l_cornerlip stretch_r_cornerlip
f14 = s (28,29) / F14 = f14 f14-NEUTRAL
F14 > 0
Squeeze_l_eyebrow & squeeze_r_eyebrow
f15 = s (3,6) / F15 = f15 f15-NEUTRAL
F15 < 0
3.3(): FAPs FDPs.

: s(i,j) FDPs i j
Mouth_corners_down
f16 = s ([16,17],15) / F16 = f16 f16-NEUTRAL
F16 > 0
Mouth_vertical_asymmetry
f17 = slope (16,17) f21
f17 0
Mouth_occlusion (detection failure)
f18 = boolean value
Head_up/down
Head_ rotation
Head _tilt
f19 =
ENSo
/ F19 = f19 f19-NEUTRAL
ESo
F19 < 0
f20 =
ESo
/ F20 = f20 f20-NEUTRAL
ENSo
F20 < 0
f21 = slope ([7,8],[11,12])

3.3(): MPEG-4-

f21 0
3.1
FAPs
1 [8].
.
3.2
3.3
FAPs,
.
: MediaLab,
, , ,
3.3() . FDP
1
.

.
(NeuralWare) 15
. 3.1.
8
15 .
F15-

.
FDP, open_eyelid close_eyelid,
.
animation- .
3.1
raise_l_i_eyebrow raise_r_i_eyebrow.
,
.

.

. , 3.2,
.

.
,
TMR PHYSTA MediaLab.

. 3.2.

.

.

(
)
F1
F2
F3
F4
F5
F6
F7
F8
F9
F10
F11
F12
F13
F14
F15
F10
F11
F12
F13
F14
3.1:
.
(
)
F1
F2
F3
F4
F5
F6
F7
F8
F9
3.2:
, .
4.
3.4.

, FAPs 3.3,
.
singleton.
i, i=1,,15 Fi, = 1 2 15
( X ) . singleton x0 X
A' ( x) ( X ) A' ( x 0 ) = 1 A' ( x) = 0
x x0 .
9
7
10
13
ESo
12
14
4
11
ENSo
ENSo
15
16
18
17
ENSo
ENSo
19
ENSo
3.3: 19 FDP
FAPs 3.3
n n
.
, ,
, , .
, ,
. (. medium open_jaw)
IF-THEN 3.2
3.4. 3.2 3.4 .
FAPs
3.4:
4.1
i, i=1,,15
Fi. [1] FAPs
.
.
i 3.4
:
Ekman MediaLab.

. :
mij ij , Fj i ( i
={1 , 2 , 3 , 4 , 5 , 6 })
4. j FAP
[1]- :
j = min{mij - 3 s ij } max{mij + 3 s ij }
i
(3.1)
FAP ( F11) :
j = min{m ij - 3 s ij } 0 j = 0 max{mij + 3 s ij }
i
(3.2)
.
3.2 3.4 . 3.2
, , FAP
. 3.2 MPEG-4

. FAP 3.2
. 3.4
.
3.4 11
.
3.5:
A (1) , A ( 2) ,..., A (15) i, i=1,,15

- A (i ) = k i . : A (i ) = { A1(i ) , A2(i ) ,..., Ak(ii ) }
m (ij ) , j=1, , ki.
c (ij ) s (ij )
3.5.
c (ij ) s (ij ) mij s ij
3.5.
F1 (ES)
F2 (ES)
F3 (MNS)
F4 (MNS)
F5 (ENS)
F6 (ENS)
F7 (ENS)
F8 (ENS)
F9 (ENS)
F10 (ENS)
F11 (MNS)
F12 (IrisD)
F13 (IrisD)
F14 (MW)
F15 (ES)
Mean
StD
Mean
StD
Mean
StD
Mean
StD
Mean
StD
Mean
StD
Mean
StD
Mean
StD
Mean
StD
Mean
StD
Mean
StD
Mean
StD
Mean
StD
Mean
StD
Mean
StD
-57
28
-58
31
-73
51
*
*
-83
48
-85
51
-66
35
-70
38
-149
40
-144
39
*
*
*
*
*
*
*
*
-69
51
*
*
*
*
*
*
*
*
85
55
80
54
*
*
*
*
*
*
*
*
*
*
-153
112
-161
109
*
*
-56
35
*
*
*
*
-271
110
*
*
*
*
*
*
*
*
*
*
24
22
25
22
*
*
-254
133
-242
122
234
98
*
*
*
*
*
*
-234
109
-177
108
*
*
*
*
*
*
*
*
-80
53
-82
54
*
*
-203
148
-211
145
*
*
-52
34
*
*
*
*
*
*
218
135
104
69
111
72
*
*
*
*
72
58
75
60
291
189
244
126
249
128
*
*
*
*
*
*
*
*
*
*
543
203
224
103
211
97
54
31
55
31
144
64
142
62
885
316
254
83
252
81
-82
39
86
60
3.4: FAPs
(, , , , , )
4.2
3.2 3.4
3.6.
i, i=1,,15 Fi,
3.5, = 1 2 15 ( X )
. B = {B1 , B2 ,..., B6 }
. x X
A' ( x) ( X ) :
A' ( x) = { a1, a 2 ,..., a15 } .
3.6
Xi
F1
F2
F3
F4
F5
[m11 - 3 s 11 0]
A1(1)
c1(1) = m11
s1(1) = s 11
[m12 - 3 s 12 0]
A1( 2 )
s1( 2) = s 12
A2(3)
c1( 2 ) = m12
( m + m 43 )
= 33
2
(3 )
c 2 = m13
A1( 4 )
c1( 4 ) = m 43
s1( 4) = s 43
A2( 4 )
c 2( 4 ) = m54
s 2( 4) = s 54
A3( 4 )
c3( 4 ) = m64
s 3( 4) = s 64
A1(5)
c1(5 ) = m15
( m + m55 )
= 25
2
(5 )
c3 = m65
s1( 5) = s 15
[m33 - 3 s 33
[m44 - 3 s 44
[m15 - 3 s 15
m13 + 3 s `13 ]
m64 + 3 s 64 ]
m65 + 3 s 65 ]
A1(3)
A2(5)
c1(3)
c 2(5)
A3(5)
A1( 6)
F6
[m16
- 3 s 16 m66 + 3 s 66 ]
F7
[m17
- 3 s 17 m67 + 3 s 67 ]
F8
[m18 - 3 s 18
m68 + 3 s 68 ]
F9
[m19 - 3 s 19
m69 + 3 s 69 ]
s1( 3) = max{s 33 , s 43 }
s 2( 3) = s 13
s 2( 3) = max{s 25 , s 55 }
s 3( 5) = s 65
s1( 6 ) = s 16
A3( 6)
c1( 6) = m16
( m + m56 )
= 26
2
( 6)
c3 = m 66
A1( 7 )
c1( 7 ) = m17
s1( 7 ) = s 17
A2( 7 )
c 2( 7 ) = m67
s 2( 7 ) = s 67
A1(8)
c1(8) = m18
s1(8) = s 18
A2(8)
c 2(8) = m68
s 2(8) = s 68
A1(9 )
c1(9 ) = m19
s1( 9) = s 19
A2(9 )
c 2(9 ) = m 49
( m + m59 )
= 39
2
s 2( 9) = s 49
A2( 6)
A3(9 )
c 2( 6)
c3(9 )
s 2( 3) = max{s 26 , s 56 }
s 3( 6 ) = s 66
s 3( 9) = max{s 39 , s 59 }
F10
[m
1,10
- 3 s 1,10 m 6,10 + 3 s 6,10 ]
A4(9 )
c 4(9 ) = m69
s 4( 9) = s 69
A1(10)
c1(10 ) = m1,10
s1(10 ) = s 1,10
A2(10)
c 2(10 ) = m 4,10
s 2(10 ) = s 4,10
A3(10)
[0
m 6,11 + 3 s 6,11
s 4(10 ) = s 6,10
A1(11)
c1(11) = m5,11
s1(11) = s 5,11
A2(11)
c 2(11) = m6,11
s 2(11) = s 6,11
A1(12)
F12
F13
[m
[m
3,12
4,13
- 3 s 3,12 m5,12 + 3 s 5,12 ]
- 3 s 4,13 m 5,13 + 3 s 5,13 ]
A2(12)
F15
[m
6,14
[m
1,15
(10 )
4
c1(12 ) =
c3(12 ) =
A1(13)
c1(13) =
A2(13)
(14 )
1
( m3,12 + m4 ,12 )
2
( m5,12 + m6 ,12 )
2
( m3,13 + m 4,13 )
2
s 2(12 ) = s 2,12
s 3(12 ) = max{s 5,12 , s 6,12 }
s1(13) = max{s 3,13 , s 4 ,13 }

s 2(13) = s 2,13
c 2(13) = m 2,13
c3(13) =
( m5,13 + m 6,13 )
(14 )
1
2
= m 6,14
- 3 s 6,14 m3,14 + 3 s 3,14 ]
A2(14)
c 2(14 ) = m3,14
- 3 s 1,15 m 6,15 + 3 s 6,15 ]
A1(15)
A2(15)
s1(12 ) = max{s 3,12 , s 4,12 }
c 2(12 ) = m 2,12
A3(12)
A3(13)
F14
( m3,10 + m5,10 )
2
= m 6,10
(10 )
4
F11
c3(10 ) =
c1(15) =
( m1,15 + m 4,15 )
2
s 3(13) = max{s 5,13 , s 6 ,13 }

s1(14 ) = s 6,14
s 2(14 ) = s 3,14
s1(15 ) = max{s 1,15 , s 4,15 }
c 2(15) = m6,15
s 2(15 ) = s 6,15
3.5:
4.3
rij i-
A(ij ) :
rij = max{a i A(ij ) }
(3.3)
h(k) k.
4 h(4) = 7 5 h(5) = 5.
2.6
-
-. - t(a,b)=ab -
u(a,b)=max(a,b).
b(l) , x, l
( l={1 , 2 , 3 , 4 , 5 , 6 })
T (l ) ( X ) , U (l ) ( X ) , , ,
, l T ( 4) = { A1( 3) , A1( 4 ) , A2( 9) , A2(10 ) , A1(12) , A1(13) , A1(15) }
U (4) = { A1(11) , A1(14 ) } - b(l) :
b (l ) =
A (j i ) T ( l )
rij (max
{rij }
i)
A j U (l )
(3.4)
h (l )

:
q = arg max b(l )
(3.5)
1: 1 A(1) 2 A( 2 ) 3 A(3) 5 A(5) 6 A( 6 ) 7

1
1
2
1
1
A1( 7 ) 8 A1(8 ) 9 A1( 9) 10 A1(10 ) 15 A1(15 ) b 1
2: 5 A( 5) 6 A( 6 ) 12 A(12 ) 13 A(13) 15 A(15 ) b
2
2
2
2
1
2
3: 9 A(9 ) 10 A(10 ) 12 A(12) 13 A(13) 14 A(14) { 3
3
3
1
1
2
A1( 3) 11 A1(11) 4 A2( 4) } b 3
4: 3 A(3) 4 A( 4) 9 A( 9) 10 A(10 ) 12 A(12) 13
1
1
2
2
1
A1(13) 15 A1(15 ) {11 A1(11) 14 A1(14) } b 4
5: 4 A( 4 ) 11 A(11) 12 A(12 ) 13 A(13) { 1 A(1) 2
2
1
3
3
1
A1( 2) 5 A2( 5) 6 A2( 6 ) 9 A3( 9) 10 A3(10) } b 5
6: 4 A( 4) 5 A( 5) 6 A( 6 ) 7 A( 7 ) 8 A(8 ) 9
3
3
3
2
2
A4( 9) 10 A4(10) 11 A2(11) 12 A3(12) 13 A3(13) 14 A1(14 )
15 A2(15 ) b 6
3.6:
5.
4
. ()
3.3
()
.

.

.

. ,
,
.
-
.
ax ay X Y
m (ij ) A(ij ) Y
X :
1:
Fi

2:
O s rij i- A(ij )
: rij = max{a i A' (ij ) } A' (ji )
A(ij ) m ' (ji ) =
3:
ay
ax
m (ji )
ax ay Whissel [3]
6.

.
-apex-
. 3.7
3.8.
: () 80 CMU
, , () 60
Yale ,
, () 100 MediaLab
, , , , () 30
.
FAPs .

PHYSTA
.
.
85%
76%
92%
73%
67%
94%
PHYSTA
68%
61%
87%
64%
58%
85%
3.7:
: 3.7 PHYSTA
.
PHYSTA
.

. 2.8
.
5
.
48%
60%
52%
61%
65%
75%
3.8:
7.
FAP FDP
MPEG-4
.

.
, ,
.

.
HIDDEN
MARKOV MODELS
,
HMM , ,
HMM
.
.
.
1.
4.1.

.
,
1 2 .
.

4.1:
1.1

:
Fk Fk +1
. pixel p k ( x, y) k- 2nx2n
bk ( x, y ) (MAD-Mean Absolute
Difference):
ek ( x, y) = bk ( x, y ) - b k +1 ( x, y ) =
l =-n m =- n
( x + l , y + m) - p k +1 ( x + l , y + m)
(4.1)

ek ( x, y) . mb s b
ek k- .
:
I = {bk | ek > (mb + s b )}
(4.2)
v k ( x, y) bk ( x, y )
bk +1 ( x, y ) :
v k ( x, y ) = (v x , v y ) = arg min
( v x , v y )Q l = - n m = - n
( x + l , y + m) - p k +1 ( x + l - v x , y + m - v y )
(4.3)
Q = {-q,..., q} {-q,..., q} .
,
(v x , v y ) Q

.
MPEG-1 MPEG-2.
1.2

.
-median filtering-
.
.
2
S
.
4.1.
Bassili [33] S ,
, :
S = {s1 , s 2 , s 3 , s 4 , s 5 , s 6 }
(4.4)
()
()
()
()
()
()
4.1: () s1 () s2 () s3 () s4 ()
s5 () s6
Ai (s j ) i- s j , v k = k e jj k
v k - 4.3.
,
- :
{ Ai : v k s j , ( 2i - 1 )
p
p
j k < ( 2i + 1 ) | i = 1...8, j = 1...6}
8
8
(4.5)
i (s j ) s j :
i (s j ) =
(4.6)
vk Ai ( s j )
i
S .
48
.
24

.
, ,
- .
4.2:
. 1: 2: 3: 4:
5: 6: 7: 8: .
4.3:
3. HMM
HMMs ,

- ,
.
HMMs [34].
HMM.
HMM 4.2.
HMMs , , , ,
- . ,
G1 , G2 , G3 G4 .
, ,
. HMMs.

.
. codebook-
.

.
G j
Gauss :
1
exp{- (O i - j ) T C j -1 (O i - j )}
2
b j (O i ) =
k
(2p ) 2 C j
1
2
(4.7)
O i i- ( ), j
G j , C j k
O i ( k=24).
j C j .
Baum-Welch
( j ) ( C j ).
a mn Gm Gn
: () a mn =0 n<m
() a mm =1- a mn -
.
,
O = O1O 2 ...OT , lm (A m , B m , m ) 4.2
:
1) Q * ,
O , Viterbi:
Q * = arg max{P (Q / O, l m )}
(4.8)
2) O
Q * :
P * = P (O / Q * , l m )
(4.9)
P(O | 1 )
1=(A1, B1, )
P(O | 2 )
2=(A2, B2, )
.
.
.
.
.
.
P(O | 6 )
6
=(A , B , )
4.2 HMM
*=argmax[P(O|)]
4.
MediaLab
project PHYSTA 30
.
HMM
. HMM
15
. 5

. 4.1.
10
11
14
12
14
13
67%
73%
93%
80%
93%
87%
4.1:
5.

.
.

.

.
[1].
[2].
[3].
[4].
[5].
[6].
[7].
[8].
[9].
[10].
[11].
[12].
[13].
[14].
[15].
[16].
[17].
[18].
[19].
[20].
M. Tekalp, Face and 2-D Mesh Animation in MPEG-4, Tutorial Issue On The MPEG-4 Standard, Image
Communication Journal, Elsevier, 1999.
G. Faigin, The Artist's Complete Guide to Facial Expressions, Watson-Guptill, New York, 1990.
C. M. Whissel, The dictionary of affect in language, R. Plutchnik and H. Kellerman (Eds) Emotion:
Theory, research and experience: vol 4, The measurement of emotions. Academic Press, New York, 1989.
P. Ekman and W. Friesen, The Facial Action Coding System, Consulting Psychologist Press, San Francisco,
CA, 1978.
EC TMR Project PHYSTA Report, Development of Feature Representation from Facial Signals and
Speech, January 1999.
R. Plutchik, Emotion: A Psychoevolutionary Synthesis, Harper and Row, New York, 1980.
F. Parke and K. Waters, Computer Facial Animation, A K Peters, 1996.
Kin-Man Lam and Hong Yan, An Analytic-to-Holistic Approach for Face Recognition Based on a Single
Frontal View, IEEE Trans. on PAMI, vol. 20, no. 7, July 1998.
K. Karpouzis, N. Tsapatsoulis and S. Kollias, Moving to Continuous Facial Expression Space using the
MPEG-4 Facial Definition Parameter (FDP) Set, in Proc. of the Electronic Imaging 2000, San Jose, CA, USA,
Jan. 2000.
IEEE, vol. 83, no. 5, pp. 705-740, 1995.
R. Brunelli and T. Poggio, Face recognition: Features versus templates, IEEE Transactions on Pattern
Analysis and Machine Intelligence, Vol. 15, No. 10, 1993.
M. Kirby and L. Sirovich, Application of the Karhunen-Loeve procedure for the characteri-zation of
human faces, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 12, No. 1, 1990, pp. 103108.
M. Turk and A. Pentland, Eigenfaces for recognition, Journal of Cognitive Neuroscience, Vol. 3, pp.71-86,
1991.
G.W. Cottrell and J. Metcalfe, EMPATH: Face, emotion, and gender recognition using holons, In R.P.
Lippman, J. Moody and D.S. Touretzky (Eds.) Advances in Neural Information Processing Systems 3, San Mateo:
Morgan & Kaufman, pp. 564-571, 1991.
A.J. O'Toole, H. Abdi, K.A. Deffenbacher and D. Valentin, A low-dimensional representation of faces in
the higher dimensions of the space, Journal of the Optical Society of America A, Vol. 10, 405-411, 1993.
P.N. Belhumeur, J.P. Hespanha and D.J. Kriegman, Eigenfaces vs. Fisherfaces: Recognition using class
specific linear projection, In Proceedings of the European Conference on Computer Vision, 1996.
P.J.B. Hancock, V. Bruce and A.M. Burton, Testing principle component representations for faces, Proc.
4th Neural Computation and Psychology Workshop, London, 9-11 April 1997, Eds. J.A. Bullinaria, D.W.
Glasspool, and G. Houghton, pp. 84-97, London, Springer, 1997.
B. Moghaddam, W. Wahid and A. Pentlant, Beyond Eigenfaces: Probabilistic Matching for Face
Recognition, Proc. 3nd IEEE Intl Conf. on Automatic Face and Gesture Recognition, Nara, Japan, 1998.
M.S. M.S. Gray, D.T. Lawrence, B.A. Golomb and T.J. Sejnowski, A perceptron reveals the face of sex,
Neural Computation, Vol. 7, No. 6, pp. 1160-1164, 1995.
N. Intrator, D. Reisfeld, and Y. Yeshrun, Face Recognition using a Hybrid Supervised/ Unsupervised
Neural Network, Pattern Recognition Letters, Vol. 17, pp. 67-76, 1996.
[21]. J. Daugman, Complete discrete 2-D Gabor transforms by neural networks for image analysis and
compression, IEEE Transactions on Acoustic, Speech and Signal Processing, Vol. 36, No. 7, pp. 1169-1179,
1988.
[22]. J. Buhmann, J. Lange and C. von der Malsburg, Distortion invariant object recognition by mat-ching
hierarchically labeled graphs, In IJCNN International Conference on Neural Networks, Washington, DC, Vol. 1,
pp. 155-159, 1989.
[23]. Z. Zhang, M. Lyons, M. Schuster and S. Akamatsu, Comparison between geometry-based and Gaborwavelets-based facial expression recognition using multi-layer perceptrons, In Proceedings of the 3rd IEEE
International Conference on Automatic Face & Gesture Recognition, Nara, Japan, 1998.
[24]. I. Essa, T. Darrell and A. Pentland, Tracking facial motion, Proceedings of the Workshop on Motion of Nonrigid
and Articulated Objects, IEEE Computer Society, pp. 36-42, 1994.
[25]. Y. Yacoob and L..S. Davis, Recognizing human facial expressions from long image sequences using
optical flow, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 18, No. 6, pp. 636-642, 1996.
[26]. C. Padgett and G.W. Cottrell, and R. Adolphs, Categorial perception in facial emotion classification,
manuscript, 1998.
[27]. M. Stewart Bartlett, P.A. Viola, T.J. Sejnowski, B.A. Golomb, J. Larsen, J.C. Hager, P. Ekman, Classifying
facial action, Advances in Neural Information Processing Systems 8, D. Touretzky, M. Mozer, and M. Hasselmo
(Eds.), MIT Press, Cambridge, MA, 1996.
[28]. J. Lien, T. Kanade, J.F. Cohn and C.C. Li, A multi-method approach for discriminating betweem similar
facial expressions, including expression intensity estimation, In Proceedings of the IEEE Conference on
Computer Vision and Pattern Recognition, pp. 853-859, 1998.
[29]. R. Cowie, E. Douglas-Cowie, N. Tsapatsoulis, G. Votsis, S. Kollias, W. Fellenz and J. Taylor, Emotion
recognition in human-computer inter-action, to appear in the IEEE Signal Processing Magazine, January 2001.
[30]. H. Boulard and Y. Camp, Autoassociation by Multilayer Perceptrons and Singular Value
Decomposition, Biological Cybern, Vol. 59, pp. 291-294, 1988.
[31]. I. Essa and A. Pentland, Coding, Analysis, Interpretation, and Recognition of Facial Expressions, M.I.T.
Media Laboratory, Perceptual Computing Group Report No. 325, 1995.
[32]. N. Tsapatsoulis, N. Doulamis, A. Doulamis, and S. Kollias Face Extraction from Non-uniform
[33]. J.N. Bassili,. Emotion Recognition: The role of facial movement and the relative importance of upper and
lower areas of the face, Journal of Personality and Social Psychology, vol. 37, 2049-2059, 1979.
[34]. L.R. Rabiner, A tutorial on HMM and Selected Applications in Speech Recognition, IEEE Proc., vol.77,
no. 2, 1989.
[35]. T. Otsuka and J. Ohya Recognizing Multiple Persons Facial Expressions Using HMM Based on
Automatic Extraction of Significant frames from Image Sequences, Journ. Cognitive Neuroscience, vol. 3, no.
1, 1991.
[36]. C. Kobayashi and J. Hara, Dynamic Recognition of Basic Facial Expressions by Discrete time Recurrent
Neural Networks, Proc. of International Joint Conference on Neural Networks, pp.155-158, 1993.
[37]. Review of existing techniques for human emotion understanding and their applications in humancomputer interaction, Technical Report, Research contract FMRX-CT97-0098 (DG12-BDNC), October
1998.

.

.

-
. 1
.

:
.

.

.
, ,
. ,

. 2

.
, ,
. ,

-
.
.
per person . ,
,
.
3.

.
.
.
, - -
.

.

.
.

- .
MPEG4 .
.
,
.

: ()

, , ()

.
KARHUNEN-LOEVE-
Karhunen-Loeve
eigenfaces. (103)
.
Karhunen-Loeve, eigenfeatures.
mn . i
i=1,2,3,,M Xi 1xMN.
:
R = (X i - X m ) (X i - X m )
[6]
i =1
X m =
1
M
[7]
(). xMN
.
R = U L U T
[8]
. U
:
U = [u1 | u 2 | K | umn ]
[9]
.
.
, ,
, .
.
Xi :
X = a1u1 + a2 u2 + ...
[10]
,
.
X => [a1 a 2 ... a k ]
[11]

eigenfaces
eigenfeatures.
,
.

,
.
.
.
.
.
.

(eigen-templates).
.
.
eigenfeatures
.
. .

(
).
.

.
GABOR
Gabor pixel .

pixel.
.

.
,
.
Gabor .
:
Yk ( x ) =
( )]
k 2 x2
k2
-2
exp
2s 2 exp(- ikx) - exp s

s2
[12]
x , k ,
k/=ct.
.
,
.

. :
x=
2
j
> xTH
[13]
j
.
.

.
.
.

.
video ,

.

. Gabor :
x 2
y2
g ( x, y; u 0 , v0 ) = exp- 2 +
+
2
p
j
[
u
x
+
v
y
]
0
0
2
2s c 2s y
[14]
x y
. (uo,vo)
. Gabor , .
wavelet :
{[ (
)]
F l (x , y ,q ) = exp - l2 x ' 2 + y '2 + jpx '
[15]
x=xcos+ysin y=xsin+ycos.
.
.
.
,
. ,
. :
F l ( x, y ,q ) = exp - l2 x 2 + y 2 + jp ( x cos q + y sin q )
[16]
:
cos=2uo
sin=2vo
[17]
.
Gabor .

Fourier .
,
.
MPEG-4
video

.
MPEG-4 .

video
.
,
,
.

.

.
MPEG-4 :
1)

,
-, .
2)

.
MPEG-4
,
.
MPEG-4 :
MPEG-4
(Facial Definition Parameter set FPD)
(Facial Animation Parameter set FAP)

, . FAP set

, .
FDP set
setup.
.
FAPs
(FAPs)
.

.

.

(Facial Animation Parameters Units).

.
.
.. .
,
, ,
.
FAPUs 2.1.
1 10.
3 :
FDPs
(FDPs)
.
.
(FDPs)
FAPUs
(FAP). .
:
( )
()
(, , , ) ()
.
. 3D
.
.

MPEG-4,
, (. [11]).
FAP name
FAP description
Bidirectional
/Unidirectional
Direction of movements
for positive intensities
Measurement
units
move_h_l_eyeball
Horizontal movement of the left

eyeball
Right
Degrees
move_h_r_eyeball
Horizontal movement of the right

eyeball
Right
Degrees
move_v_l_eyeball
Vertical movement of the left

eyeball
Downward
Degrees
move_v_r_eyeball
Vertical movement of the right

eyeball
Downward
Degrees
?enlarge_l_pupil?
Enlarge th left pupil
Outward
ES
?enlarge_r_pupil?
Enlarge th right pupil
Outward
ES
close_upper_l_eyelid
Vertical movement of the upper left

eyelid
Downward
ENS
close_upper_r_eyelid
Vertical movement of the upper

right eyelid
Downward
ENS
close_lower_l_eyelid
Vertical movement of the lower left

eyelid
Downward
ENS
close_lower_r_eyelid
Vertical movement of the lower

right eyelid
Downward
ENS
move_o_l_eyeball
Outside/inside movement o the left

eyeball
Forward
ES
move_o_r_eyeball
Outside/inside movement o the

right eyeball
Forward
ES
raise_l_o_eyebrow
Vertical movement of the outer part

of the left eyebrow
Upward
ENS
raise_r_o_eyebrow
Vertical movement of the outer part

of the right eyebrow
Upward
ENS
raise_l_m_eyebrow
Vertical movement of the middle

part of the left eyebrow
Upward
ENS
raise_r_m_eyebrow

part of the right eyebrow
Upward
ENS
raise_l_i_eyebrow
Vertical movement of the inner part

of the left eyebrow
Upward
ENS
raise_r_i_eyebrow
Vertical movement of the inner part

of the right eyebrow
Upward
ENS
squeeze_l_eyebrow
Horizontal movement of the left

eyebrow
Inwards
ES
squeeze_r_eyebrow
Horizontal movement of the right

eyebrow
Inwards
ES
stretch_l_nose
Stretch/squeeze movement of the

left nostril
Toward the edge of the face
ENS
stretch_r_nose
Stretch/squeeze movement of the

right nostril
Toward the edge of the face
ENS
raise_nose
Vertical movement of the nose
Upward
ENS
open_jaw
Rotational movement of the jaw
Downward
MNS
move_hori_jaw
Left/Right movement of the jaw
Right
MNS
move_fwd_jaw
Forward movement of the jaw
Forward
MNS
depress_chin
Upward
and
compressing
movement of the chin (like in
sadness)
Upward
Intensity 1-10
puff_l_chek
Puffing movement of the left cheek
Left
ES
puff_r_cheek
Puffing movement of the right

cheek
Right
ES
lift_l_cheek
Lifting movement of the left cheek
Upward
ENS
lift_r_cheek
Lifting movement of the right cheek
Upward
ENS
lower__lowerlip
Vertical movement of the lowerlip
Downward
MNS
lower_l_cornerlip
Vertical movement of th left corner

of the lips
Downward
MNS
lower_r_cornerlip
Vertical movement of the right

corner of the lips
Downward
MNS
raise_l_cornerlip
Vertical (upward) movement of the

left corner of the lips
Upward
MNS
raise_r_cornerlip
Vertical (upward) movement of the

right corner of the lips
Upward
MNS
raise_u_midlip

part of the upper lip
Upward
MNS
raise_l_midlip

part of the lower lip
Upward
MNS
push_pull_upperlip
Protrunding or sucking movement

of the upper lip
Forward
MNS
push_pull_lowerlip
Protrunding or sucking movement

of the lower lip
Backward
MNS
stretch_l_cornerlip
Stretch the left corner of the lips
Left
MW
stretch_r_cornerlip
Stretch the right corner of the lips
Right
MW
squeeze_l_cornerlip
Squeeze the left corner of the lips
Left
MW
squeeze_r_cornerlip
Squeeze the right corner of the lips
Right
MW
?move_h_tongue_tip?
Left/Right movemnt of the tip of

the tongue
Right
MW
?raie_tongue_tip?
Vertical movement of the tip of the

tongue
Up
MW
?proturd_tongue?
Horizontal proturding movement of

the tongue body
Forward
MW
?raise_tongue?
Vertical movement of the tongue

body
Up
MW
?tongue_roll?
Rolling of the tongue in the U shape
Towards the center of the

face
Degrees
?raise_left_ear?
Raise the left ear
Up
ENS
?raise_right_ear?
Raise the right ear
Up
ENS
?pull_left_ear?
Pull left ear forward with possible

rotation
Forward
ENS
?pull_right_ear?
Pull right ear forward with possible

rotation
Forward
ENS
F3.1: FAPs MPEG-4
F3.1: FDP points MPEG4
Aristotle,3

Prosopo

Uploaded by

Copyright:

Available Formats

You might also like

Prosopo

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Prosopo

Uploaded by

Copyright:

Available Formats

.

A2.1: mug shot

2.1: KLT VKLT

2.3: KLT VKLT

3.1: delighted eager

3.3: 19 FDP FAPs 3.3

Facial Animation Parameter set FAP:

mpeg-4 ............................................................................................................. 191

PAMI, sessions- (IEEE ICIP,

A2.1: () Mug shot (front view) () Mug shot

() 64x88 () 32x44 () 16x22. 32x44

A3.1: () 64x88 () 32x44 ()

mean(R1) - mean( R2)

=> -pixel pixel-

Olivetti Research Laboratory (ORL)

A4.1: frontal views.

d(5a, 5b) /d(14,15)

Yacoob Davis [75] ,

x =t cos q -u sin q , y =t sin q + u cos q

[52]. Phillips et al., Nature 389, 495-8, 1997.

(2n+1)(2n+1) pixel tij=(i,j). :

[X 0 , Y0 , A, B, Q] = arg max E[x 0, y 0, a, b,q ]

exp(- ikx) - exp s -2

F l ( x, y , q ) = exp - l 2 x ' 2 + y ' 2 + jpx '

d (5a ,5b) d (14,15)

d (12a ,12b) d (14,15)

d (8a ,8b) d (14,15)

(0<c<0.5) a = mean(A) , b = mean(M )

False Alarm Rate (FAR) (%)

Mean Execution Time (MET) (sec)

studio (anchorperson shots): :

3.1: Precision Recall

m = [ mCr ( F ) mCb ( F )]T .

S i(k ) i- k- , x( S i( k ) ) = [ mCr (S i( k ) ) mCb ( S i( k ) )]T

S ( N ) S ( N -1) ... S (1) = L .

(4) (1)-(4). S (N ) =0 (1) M S ( N -1)

72.6 +/- 15.1

74.5 +/- 14.3

84.6 +/- 8.7

61.7 +/- 15.2

86.8 +/- 7.4

65.8 +/- 14.7

83.3 +/- 9.6

75.6 +/- 13.1

f 1 (), f 2 (),... f m () . m<<n

3.2 Fisher Linear Discriminant FLD

2.1: KLT VKLT

2.3: KLT VKLT

2.1: KLT VKLT

= diag (l1 , l 2 ,.., l k ,0,..,0) and l1 l 2 .. l k .

A . U = (u1 ,.., u k , u k +1 ,.., u m ) V = (v 1 ,.., v k , v k +1 ,.., v n )

= a (l1 , l 2 ,.., lk ,0,..,0 )

x nx1 = [l1l2 ..lk 0..0]T .

S , face version S , (4.10):

(4.8), (4.9), (4.10) S

l12 < u I1 , u S1 >

= l2i < u Ii , u Si > < v Ii , v Si >

, u Si > < v Ii , v Si >