28 Orbicularis oris Lips sucked

41 Relaxation of levator palpebrae superioris Upper eyelid droop

42 Orbicularis oculi Eyelid slit
43 Relaxation of levator palpebrae superioris; Eyes closed
orbicularis oculi, pars palpebralis
44 Orbicularis oculi, pars palpebralis Eyes squinted
45 Relaxation of levator palpebrae superioris; Blink
orbicularis oculi, pars palpebralis
46 Relaxation of levator palpebrae superioris; Wink
orbicularis oculi, pars palpebralis
Non-additive combinations represent further
Table 2. Miscellaneous Actions. complexity. Following usage in speech science, we
AU Description of Movement refer to these interactions as co-articulation effects.
8 Lips toward An example is the combination AU 12+15, which
19 Tongue show often occurs in embarrassment. While AU 12 raises
21 Neck tighten the cheeks, its action on the lip corners is modified by
29 Jaw thrust the downward action of AU 15. The resulting
30 Jaw sideways appearance change is highly dependent on timing.
31 Jaw clench The downward action of the lip corners may occur
32 Bite lip either simultaneously or sequentially. To be
33 Blow comprehensive, a database should include individual
34 Puff action units and both additive and non-additive
35 Cheek suck combinations, especially those that involve co-
36 Tongue bulge articulation effects. A classifier trained only on
37 Lip wipe single action units may perform poorly for
38 Nostril dilate combinations in which co-articulation effects occur.
39 Nostril compress
2.3 Deliberate versus spontaneous expression
2.2 Transitions among expressions
Most face expression data have been collected by
A simplifying assumption in previous research is asking subjects to perform a series of expressions.
that expressions are singular and begin and end from These directed facial action tasks may differ in
a neutral position. In reality, facial expression is appearance and timing from spontaneously occurring
more complex, especially at the level of action units. behavior [5]. Deliberate and spontaneous facial
Action units may occur in combinations or show behavior are mediated by separate motor pathways,
serial dependence. Transitions from action units or the pyramidal and extra-pyramidal motor tracks,
combination of actions to another may involve no respectively [16]. As a consequence, fine-motor
intervening neutral state. Parsing the stream of control of deliberate facial actions is often inferior
behavior is an essential requirement of a robust facial and less symmetric to that which occurs
analysis system, and training data are needed that spontaneously. Many people, for instance, are able to
include dynamic combinations of action units, which raise their outer brows spontaneously while leaving
may be either additive or non-additive. their inner brows at rest; few can perform this action
An example of an additive combination is voluntarily. Spontaneous depression of the lip
smiling (AU 12) with mouth opening, which would corners (AU 15) and raising and narrowing the inner
be coded as AU 12+25, AU 12+26, or AU 12+27 corners of the brow (AU 1+4) are common signs of
depending on the degree of lip parting and whether sadness. Without training, few people can perform
and how far the mandible was lowered. In the case these actions deliberately, which incidentally is an
of AU 12+27, for instance, the facial analysis system aid to lie detection [5]. Differences in the temporal
would need to detect transitions among all three organization of spontaneous and deliberate facial
levels of mouth opening while continuing to actions are particularly important in that many pattern
recognize AU 12, which may be simultaneously recognition approaches, such as Hidden Markov
changing in intensity. Modeling, are highly dependent on the timing of
appearance change. Unless a database includes both

486