Face as Mouse Through Visual Face Tracking

Jilin Tu, Thomas Huang Hai Tao

Elec. and Comp. Engr. Dept. Elec. Engr. Dept.

Univ. of Ill. at Urbana and Champ. Univ. of Calif. at Santa Cruz
Urbana, IL 61801 Santa Cruz, CA 95064
jilintu,huang@ifp.uiuc.edu tao@soe.ucsc.edu

Abstract By retrieving human motion parameters in video cap-

tured at realtime by a camera hooked to computer, mouse
This paper introduces a novel camera mouse driven by cursor can be controlled by the motion parameters. This new
3D model based visual face tracking technique. While cam- mouse tool is called camera mouse[13]. A camera mouse is
era becomes standard configuration for personal com- usually composed of a visual tracking module and a mouse
puter(PC) and computer speed becomes faster and control module. Visual tracking module retrieves motion
faster, achieving human machine interaction through vi- parameters from the video, and mouse control module spec-
sual face tracking becomes a feasible solution to hand-free ifies the rule of control. The framework can be illustrated by
control. The human facial movement can be decom- Fig. 1.
posed into rigid movement, e.g. rotation and translation,
and non-rigid movement, such as the open/close of mouth,
eyes, and facial expressions, etc. We introduce our visual
face tracking system that can robustly and accurately re-
trieve these motion parameters from video at real-time.
After calibration, the retrieved head orientation and trans-
lation can be employed to navigate the mouse cursor, and
the detection of mouth movement can be utilized to trig-
ger mouse events. 3 mouse control modes are investigated
and compared. Experiments in Windows XP environ-
ment verifies the convenience of navigation and operations
using our face mouse. This technique can be an alter-
native input device for people with hand and speech
disability and for futuristic vision-based game and inter-
face.

Figure 1. The framework of camera mouse

1. Introduction
People have long been speculating the possibility of in-
teracting with computer in an natural way, instead of us-
ing hand-controlled input devices, e.g. mouse and keyboard.
However in order to do that, the computer has to accom-
modate to human’s natural sensing and perceptual behav-
iors which is usually not trivial. One potential solution is
to track human body movement in video input, from which
the intention of human can be inferred for computer to re-
spond to. This kind of technology is called perceptual user
interface[21][20].
Among human body parts, face has been the most stud-
ied for visual human tracking and perceptual user interface,
because face appearance is more statistically consistent in
color, shape and texture, and thus allow computer to de-
tect and track with robustness and accuracy. With differ-
ent assumptions, people have proposed to navigate mouse
with the movement of eyes[14][12][4][16], nose[8][9], and
nostrils[6][1], etc. For eye tracking, people usually employ
infrared lighting cameras, and take advantage of the fact
that the iris of human eye has large infra-red light reflec-

Proceedings of the Second Canadian Conference on Computer and Robot Vision (CRV’05)
0-7695-2319-6/05 $ 20.00 IEEE
 tion. For nose tracking, [8] claimed that defining nose as
an extremum of the 3D curvature of the nose surface makes
nose the most robust feature for tracking with high accuracy.
For nostril tracking, skin-color region is usually extracted
first, and nostril can be distinguished by its dark color and
unique contour shape. By tracking the X-Y facial feature
coordinates, mouse cursor can be navigated. However, we
also notice the movement and location of facial feature in
video usually does not coincide with people’s focus of at-
tention on screen. This indeed makes the navigation oper-
ations un-intuitive and inconvenient. In order to avoid that
problem, people have proposed to navigate mouse cursor by
3D head pose. The estimation of 3D head pose usually re-
quires tracking of more than one feature. Head pose can
be inferred by stereo triangulation if more than one cam-
era are employed[16][3] or by inference from anthropolog-
ical characteristics of face geometry[18][10]. Based on the
technical developments in this area, some commercial prod-
ucts have been developed in recent years[15] [11][7].
For mouse control module, the conversion from
human motion parameters, i.e. position and/or rota-
tion(orientation), etc., to mouse cursor navigation can
be categorized into direct mode, joystick mode, and dif-
ferential mode. For direct mode, a one-to-one mapping
from the motion parameter domain to screen coordi-
nates is established by calibration off-line or by design
based on the a prior knowledge about the human-monitor
setting[18]. Joystick mode navigate mouse cursor by the di-
rection(or the sign) of the motion parameters. And the
speed of the cursor motion is determined by the magni-
tude of the motion parameters[9]. In differential mode, the
cumulation of displacement of the motion parameter dis-
placements drives the navigation of the mouse cursor,
and some extra motion parameter switches on/off the cu-
mulation mechanism so that the motion parameter can
be shifted backwards without influence the cursor posi-
tion. Therefore this mode is very similar to standard mouse
mode as user can lift mouse and move back to the ori-
gin on mouse pad after performing a mouse dragging
operation[9].
After mouse cursor is navigated to desired location, the
execution of mouse operations, such as mouse button clicks,
is carried out according to further interpretation of user’s
motion parameter changes. The most straightforward in-
terpretation is to threshold some specified motion parame-
ters. In [13], the mouse clicks is generated based on ”dwell
time”, e.g. a mouse click is generated if the user keeps the
mouse cursor still for 0.5s. In [4], the confirmation and can-
celation of mouse operations is conveyed by head nodding
and head shaking. Timed finite state machine is designed to
detect the nodding and shaking from raw motion parame-
ters.
In this paper, we proposed to use 3D model-based visual
Proceedings of the Second Canadian Conference on Computer and Robot Vision (CRV’05)
0-7695-2319-6/05 $ 20.00 IEEE
face tracking approach[17] to retrieve facial motion para-
meters for mouse control. This approach utilizes only one
camera as video input, but is able to retrieve 3D head mo-
tion parameters. It is also not merely a facial feature tracker,
the non-rigid facial deformation is also formulated as a lin-
ear model and can be retrieved. Based on the motion pa-
rameters retrieved by our system, we designed 3 different
mouse control modes. In the experiments, the controllabil-
ity of the 3 mouse control modes is compared. At last, we
demonstrate how our system can be utilized to play com-
puter card game in Windows XP environment.
The rest of the paper is organized as follows: Section
2 summarizes our face model. Section 3 explains how our
tracking system works. Section 4 describes the design of
our mouse control strategies. Section 5 shows the experi-
mental evaluation of the controllability of our system. And
Section 6 summarizes the paper and gives some analysis of
future directions.
2. 3D face modelling
The 3D geometry of human facial surface can be rep-
resented by a set of vertices {(xi , yi , zi )|i = 1, ..., N } in
space, where N is the total number of vertices. In order to
model facial articulations, a so-called Piecewise Bezier Vol-
ume Deformation Model is developed[17]. With this tool,
some pre-specified 3D facial deformations can be manually
crafted. These crafted facial deformations are called Action
Units(AU)[5]. For our tracking system, 12 action units are
crafted as shown in Fig. 1.
AU Description
1 Vertical movement of the center of upper lip
2 Vertical movement of the center of lower lip
3 Horizontal movement of left mouth corner
4 Vertical movement of left mouth corner
5 Horizontal movement of right mouth corner
6 Vertical movement of right mouth corner
7 Vertical movement of right eyebrow
8 Vertical movement of left eyebrow
9 Lifting of right cheek
10 Lifting of left cheek
11 Blinking of right eye
12 Blinking of left eye
Table 1. Action Units
If human face surface is represented by concate-
nating the vertices coordinates into a long vector
V = (x1 , y1 , z1 , x2 , y2 , z2 , ..., xN , yN , zN )T , the Action
Units can be modeled as the vertices displacement of the de-
formed facial surface from a neutral facial surface, i.e.,
 (k) (k) (k) (k) (k) (k)
∆V (k) = (∆x1 , ∆y1 , ∆z1 , ∆x2 , ∆y2 , ∆z2 , ..., 3. Visual face tracking
(k) (k) (k)
∆xN , ∆yN , ∆zN )T , where k = 1, ..., K with K be-
ing the total number of key facial deformations. There- 3.1. Initialization of face tracking
fore an arbitrary face articulation can be formulated
as The face modeling equation 3 defines a highly nonlin-
ear system. Fortunately visual face tracking is not a process
V = V̄ + Lp (1)
of finding solution with random initial guess. A initial so-
where V̄ is the neutral facial surface, and L3N ×K = lution is usually provided with high accuracy by manual la-
{∆V (1) , ∆V (2) , ..., ∆V (K) } is the Action Unit ma- beling or by automatic detection of the face to be tracked in
trix, and pK×1 is the AU coefficients that define the articu- the first frame of the video.
lation. An illustration of such synthesis is shown in Fig. 2. Our system provided an automatic tracking initialization
procedure. In the first frame of the video, we do face de-
tection using Adaboosting algorithm[22]. After face detec-
tion, the location of facial features are identified by ASM
techniques[2][19]. And finally, the generic 3D face model
is adapted and deformed to fit to the detected 2D facial fea-
tures. In the worst case, if the automatic procedure goes
awry, user is also provided with GUI tool to fine tune the
initialization result. A initialization result is show in Fig-
ure 3.

(a) (b)

Figure 2. The modeling of facial articula-

tions.(a)Neutral face; (b)A smiling face syn-
thesized by linear combination of AUs.

Taking head rotation and translation into account, the

motion at each vertex Vi , i = 1, ..., N on face surface in
video can be formulated as

Vi
= R(θ, φ, ψ)Vi + T (2)

where R(θ, φ, ψ) is rotation matrix, and T = [Tx Ty Tz ]

defines head translation.
Assuming pseudo-perspective camera model with pro-
jection matrix


f s/z 0 0
M=
0 f s/z 0
where f denotes the focal length, s denotes scaling factor,
the projection of the face surface to image plane can be de-
scribed as
ViImage = M R(θ, φ, ψ)(V̄i + Li p) + T (3)
where ViImage is the projection of the i − th vertex node on
face surface. Therefore a face motion model is character-
ized by rigid motion parameters including rotation θ, φ, ψ,
translation T = [Tx Ty Tz ], and non-rigid facial motion pa-
rameter, the AU coefficient vector p.
Figure 3. The initialization of our tracking
system
3.2. Tracking by iteratively solving differen-
tial equations.
Though the system is nonlinear, at each video frame it
can be approximated locally by first order Taylor expan-
sion with the solution at previous frame as origin. There-
fore tracking can be formulated as a iterative process of
solving linearized differential equations. The displacement
of ViImage , i = 1, ..., N can be estimated from video se-
quence as optical flow ∆ViImage = [∆Xi , ∆Yi ]T ,i =
1, ..., N . The displacement of model parameters, i.e. dW =
[dθ, dφ, dψ]T , dp, dT = [dTx , dTy , dTz ]T can be computed

Proceedings of the Second Canadian Conference on Computer and Robot Vision (CRV’05)
0-7695-2319-6/05 $ 20.00 IEEE
 with Least-Mean-Square error once the Jacobian matrix
∆ViImage
dW,dp,dT for each vertex node Vi , i = 1, ...N is computed.
More details of the computation can be found in [17].
The whole tracking procedure is illustrated by Fig. 4. At
the first frame of the video, the model parameter is initial-
ized. From the second frame on, the optical flow at each
vertex on face surface is computed, the displacement of the
model parameters is estimated by solving the system of lin-
ear equations and the model parameters are updated accord-
ingly. This procedure iterates for each frame, and for each
frame, it also iterates in a coarse-to-fine fashion.
Figure 4. The flowchart of our tracking proce-
dure
3.3. Fast computation of optical flow
One of the most time-consuming operation for our track-
ing procedure is the computation of optical flow. We com-
pute optical flow by template matching through normalized
correlation. Given the texture template centered at (X, Y ) at
frame t−1, the location of the maximal normalized correla-
tion of the templates in searching window at frame t defines
the optical flow at (X, Y ). Denote the template centered at
(X, Y ) as {t(i, j)}. and the template centered at (x, y) as
{f (x + i, y + j)}. The standard normalized correlation is
defined by Eq. 4. After some manipulation of the equation
with the purpose of reducing double decision computations
and intermediate variable storage, the formula for fast com-
putation is Eq. 5. We notice the intermediate variables in
Eq. 5 are summations in the template windows which can
be sped up by using integral image techniques[22]. Sup-
pose the template searching area is of size M by N, and
the template size is m by n, about 2mnMN multiplication
and 5mnMN additions/subtraction are required for using
Eq. 4. While the number of operations of Eq. 5 reduces to
mnMN+6((M+m)(N+n)+MN) summation/subtractions and
mnMN+(M+m)(N+n) multiplications. Therefore comput-
ing normalized correlation using Eq. 5 is about 5 times
faster for addition/subtraction operations and about twice
Proceedings of the Second Canadian Conference on Computer and Robot Vision (CRV’05)
0-7695-2319-6/05 $ 20.00 IEEE
faster for multiplication operations comparing to using Eq.
4.
3.4. Performance of our tracking system
Our tracking system currently runs on a Dell worksta-
tion with dual CPU of 2GHz and SCSI RAID hard drive.
The setup of our camera system is shown in Figure 5. The
camera is mounted beneath the screen, and looks upward
to the user’s face. Sequences(120 frames for about 10 sec-
onds) of the head pose parameters Tx , Ty , Tz , Rx , Ry , Rz
and AU coefficients related to mouth and eyes estimated
by the tracker are shown in Figure 6. The figure indicates
the user moved his head horizontally (in x coordinate) and
back-forth(in z coordinate), rotated his head about y axis
(yaw movement) in the first 40 frames, and opened his
mouth wide at about the 80-th frame, and blinked his eyes
at about the 20-th, 40-th, and 85-th frame.
Figure 5. The setup of our camera mouse sys-
tem
The frame rate of the tracker achieves 22 fps when only
rigid motion parameters are computed and 13 fps when all
motion parameters are computed. Because the optical flow
is computed by normalized correlation, the tracker is ro-
bust to illumination changes during tracking. Because the
LMSE solution of the tracker is globally optimal in na-
ture, it can handle partial occlusions of the face. Besides,
the LMSE model fitting error and the confidence of the op-
tical flow computation(derived from the sum of the normal-
ized correlation at all vertices) indicates whether the tracker
has failed or not, and can trigger automatic re-initialization
  m2  n2
(f (x + i, y + j) − µf )(t(i, j) − µt )
i=− m
j=− n
N Corr(x, y) =  m2  n2
2 2
 m2  n2 (4)
{( i=− m j=− n (f (x + i, y + j) − µf )2 )( i=− m
2 1/2
j=− n (t(i, j) − µt ) )}
2 2 2 2

where
 m2  n2  m2  n2
i=− m j=− n f (x + i, y + j) i=− m j=− n t(i, j)
µf (x, y) = 2 2
, µt (x, y) = 2 2

mn mn

 m2  n2
mn i=− m j=− n f (x + i, y + j)t(i, j) − µf µt
N
Corr(x, y) =  m2  n2
2 2
 m2  n2 (5)
{(mn i=− m j=− n f 2 (x + i, y + j) − µ2f )(mn i=− m
2 2 1/2
j=− n t (i, j) − µt )}
2 2 2 2

where
m n m n

2 
2 
2 
2

µf (x, y) = f (x + i, y + j), µt (x, y) = t(i, j)

i=− m n
2 j=− 2 i=− m n
2 j=− 2

of the tracker. All these advantages make our tracking sys- else y=0;
tem a good candidate for the visual tracking module in cam- return y; }
era mouse. The ∆(x) function is a step function in which the constants
are specified empirically. We found it is easier for the user
4. Mouse cursor control to learn to move the mouse cursor with desired speed and
to keep cursor still at desired location by changing his/her
The direct mode, joystick mode and differential mode head pose with such a step control function.
are implemented for the mouse control module. For the di- For differential mouse control mode, we have the follow-
rect mode, the face orientation angle Rx , Ry (the rotation ing control rules
angle with respect to x and y coordinate)are mapped to the
mouse cursor coordinates (X, Y ) on the screen. As the re- X t+1 = X t + α∆Ryt bt
liable tracking range of Rx and Ry is about 40o , and the Y t+1 = Y t + β∆Rxt bt
resolution of the screen is 1600 × 1200, we therefore em-
pirically let the mapping function to be where
 
0 Tzt < Tz0
X = 40(Ry − Ry0 ) bt =
1 Tzt >= Tz0
Y = 30(Rx − Rx0 )
Therefore the mouse is navigated by the cumulation of
where Rx0 and Ry0 are the initial face orientation angles.
head orientation displacements ∆Rxt and ∆Ryt with head
For joystick mouse control mode, the following control
moving toward to the camera turns on the mouse drag-
rule is employed. ging state, head moving away from the camera turns on the
0 mouse lifting state.
X t+1
= X + ∆(Ry − Ry )
t

0
The variations in nonrigid motion parameters trigger
Y t+1
= Y + ∆(Rx − Rx )
t
mouse button events. While there are 12 AUs for selec-
tion, not all of them are good for triggering mouse event.
with the rule function ∆(x) defined as
Ideally the detection of AU should be robust against head
double Delta(double x){ pose change and noisy outliers. AU7 and AU8(eyebrows
double y; raising) are not good because eyebrow movements are rela-
if(fabs(x)>15) y=sgn(x)*64; tive subtle for detection. AU9, AU10(cheek lifting) are not
if(fabs(x)>10) y=sgn(x)*16; good because the lack of texture on cheek makes the esti-
else if(fabs(x)>5) y=sgn(x)*4; mation unreliable. And AU11 and AU12(eye blinking) are
else if(fabs(x)>3) y=sgn(x)*1; not good because user may blink his eye unintentionally

Proceedings of the Second Canadian Conference on Computer and Robot Vision (CRV’05)
0-7695-2319-6/05 $ 20.00 IEEE

Figure 6. The motion parameters of head
moving with eye blinking and mouth opening
facial deformation
and user may have difficulty to do click-and-drag opera-
tion with their eyes closed. We chose using the detection
of mouth opening to trigger left-button-click event, and the
detection of mouth corner stretching to trigger right-button-
click event.
Our camera mouse system is seamlessly integrated with
Windows XP operating system. Once the camera mouse
mode is turned on, user can comfortably navigate in Win-
dows and interact with commercial softwares using his face.
5. Experiments
We implemented the three control modes and compared
their controllability by clicking on a target center of a pic-
ture 10 times in a painting software on a screen of resolution
Proceedings of the Second Canadian Conference on Computer and Robot Vision (CRV’05)
0-7695-2319-6/05 $ 20.00 IEEE
of 1600 by 1200. The clicking results are illustrated in Fig.
7. From the figure, we can notice the average localization er-
ror for direct mode is about 10 pixels, that for joystick mode
is about 3 pixels, and that for differential mode is about 5
pixels. The localization error is mostly caused by measure-
ment noise introduced during the tracking process, and par-
tially due to human can not really hold his head tightly still.
We then evaluated the reliability of detection of mouth
opening and stretching. When the user is assuming near-
frontal view(within ±30o ) and the tracker is in benign track-
ing status, the detection of mouth opening and stretching
achieves 100% with zero false alarm.
(a) (b) (c)
Figure 7. The illustration of clicking ac-
curacy using 3 control modes. (a)Direct
mode;(b)Joystick mode; (c) Differential mode
Since our system is seamlessly integrated to Windows
XP environment, we can use our camera mouse to navigate
in Windows environment. Fig. 8 shows the user is activat-
ing the windows Start menu by a mouth opening. Fig. 9 and
Fig. 10 show how the user plays Solitaire and Minesweeper
using the camera mouse.
After experimenting with the 3 mouse control modes, we
list the comparisons as follows:
Direct mode Mouse cursor is correlated with focus of at-
tention. However, the noise in motion parameter esti-
mation is amplified on screen and the localization vari-
ance is large.
Joystick mode The cursor localization variance is small.
The mouse cursor is still correlated with the direction
of focus of attention, but not as strongly as direct mode.
However such inconsistency can be compensated by
eye gaze if the inconsistency is not too large.
Differential mode This mode is supposed to match
most with human’s mouse-control habit. How-
ever, this mode require user to move head frequently,
and mouse cursor is not correlated with human’s fo-
cus of attention which could be disturbing if the
user wish to read the screen at the same time. Be-
sides, its localization error is not comparable to Joy-
stick mode.

Figure 8. Using camera mouse to navigate in
Windows XP. The user is opening his mouth
to activate Start menu.
6. Conclusion
In this paper, we investigated the state of the art of cam-
era mouse techniques. In particular, we proposed to use a
3D model based visual face tracker to control the mouse
and carry out mouse operations. The implementation of our
face trackers is detailed. Based on the estimated rigid and
non-rigid facial motion parameters, 3 mouse control modes,
direct mode, joystick mode and differential mode, are im-
plemented for the mouse control. The experiments verified
the effectiveness of our camera mouse system. In particular,
the accuracy of mouse navigation using the 3 mouse con-
trol modes is evaluated numerically, and the pros and cons
of each control mode is summarized.
We believe an optimal solution for mouse control is a
combination of direct mode and joystick mode. The direct
mode can navigate the mouse cursor to the area of inter-
est on screen, then joystick mode can be triggered to fine
tune the cursor location according to user’s desire. The two
modes can be switched on/off by the detection of some non-
rigid facial deformation, i.e., the blinking of eye.
An interesting future direction is how to estimate human
emotion based on the visual tracking of user facial motions.
Some mouse events could be triggered by user emotions,
and the application(such as computer tutoring software) can
respond to user in a proactive way according to user’s emo-
tion(or spiritual state).
We also realize the visual face tracking system also play
a very important role in camera mouse. Our current system
is still not fully optimized in algorithm and in implementa-
Proceedings of the Second Canadian Conference on Computer and Robot Vision (CRV’05)
0-7695-2319-6/05 $ 20.00 IEEE
Figure 9. Using camera mouse to play Win-
dows card game Solitaire. The user is drag-
ging Spade-7 from right toward Heart-8 in the
left by turning his head with mouth opened.
tion. We still need to further improve our system so that it
can run faster and can achieve more robust tracking perfor-
mance in the context of camera mouse application.
References
[1] V. Chauhan and T. Morris. Face and feature tracking for
cursor control. In 12th Scandinavian Conference on Image
Analysis, Bergen, 2001.
[2] T. F. Cootes and C. J. Taylor. Active shape model search
using local grey-level models: A quantitative evaluation. In
4th British Machine Vision Conference, page 639648. BMVA
Press, 1993.
[3] T. Darrel, N. Checka, A. Oh, and L.-P. Morency. Exploring
vision-based interfaces: How to use your head in dual point-
ing tasks. Technical Report AIM-2002-001, Artificial Intel-
ligence Laboratorty@MIT, 2002.
[4] J. W. Davis and S. Vaks. A perceptual user interface for
recognizing head gesture acknowledgements. In Workshop
for Perceptive User Interface, pages 1–7, Orlando, FL, 2001.
[5] P. Ekman and W. Friesen. Facial action coding system. Psy-
chologist Press., Palo Alto, 1978.
[6] L. El-Afifi, M. Karaki, and J. Korban. ’hand-free interfacee’-
a fast and accurate tracking procecdure for real time human
computer interaction. In FEA Student Conference, American
University of Beirut, 2004.
[7] S. Eye. http://www.smarteye.se/.
[8] D. Gorodnichy. On importance of nose for face tracking. In
5th Intern. Conf. on Automatic Face and Gesture Recogni-
tion, Washington, DC, USA., 2002.
[9] D. Gorodnichy, S. Malik, and G. Roth. Nouse ’use your nose
as a mouse’ - a new technology for hands-free games and in-
 [19] J. Tu, Z. Zhang, Z. Zeng, and T. Huang. Face localization via
hierarchical condensation with fisher boosting feature selec-
tion. In IEEE Computer Society Conference on Computer
Vision and Pattern Recognition (CVPR’04), volume 2, pages
719–724, 2004.
[20] M. Turk, C. Hu, R. Feris, F. Lashkari, and A. Beall. Tla
based face tracking. In Intern. Conf. on Vision Interfacec,
pages 229–235, Calgary, 2002.
[21] M. Turk and G. Robertson. Perceptual user interfaces.
In Communications of the ACM, volume 43, pages 32–34,
2000.
[22] P. Viola and M. Jones. Fast and robust classification using
asymmetric adaboost and a detector cascade. In Advances
in Neural Information Processing System, volume 14. MIT
Press, Cambridge, MA, 2002.

Figure 10. Using camera mouse to play Win-

dows game Minesweeper. The user is sweep-
ing the remaining squares at the cursor lo-
cation(where the 2 mines have been marked)
by simultaneous mouth opening and stretch-
ing which corresponds to clicking the left and
right mouse button simultaneously.

terfaces. In Intern. Conf. on Vision Interface, pages 354–361,

Calgary, 2002.
[10] Y. Hu, L. Chen, Y. Zhou, and H. Zhang. Estimating face pose
by facial asymmetry and geometry. In the 6th Intl. Conf. on
Automatic Face and Gesture Recognition, 2004.
[11] M. V. Inc. www.mousevision.com.
[12] R. J. Jacob. What you look at is what you get: Eye
movement-based interaction techniques. In Human Factors
in Computing Systems, pages 11 – 18, 1990.
[13] B. M., G. J., and F. P. The camera mouse: Visual tracking
of body features to provide computer access for people with
severe disabilities. IEEE Trans Neural Syst Rehabil Eng.,
10(1):1–10, 2002.
[14] C. Morimoto, D. Koons, A. Amir, and M. Flickner. Real-
time detection of eyes and faces. In Workshop on Percep-
tural User Interfacecs, San Fransisco, 1998.
[15] C. Mouse. http://www.cameramouse.com/.
[16] R. Ruddarraju and et al. Perceptual user interfaces using
vision-based eye tracking. In the 5th international confer-
ence on Multimodal interfaces, pages 227 – 233, Vancouver,
British Columbia, Canada, 2003.
[17] H. Tao and T. Huang. Explanation-based facial motion track-
ing using a piecewise bezier volume deformation model. In
Proc. IEEE Comput. Vision and Patt. Recogn., volume 1,
pages 611–617, 1999.
[18] K. Toyama. Look, ma-no hands! hands-free cursor control
with real-time 3d face tracking. In Workshop on Perceptual
User Interfaces, San Fransisco, 1998.

Proceedings of the Second Canadian Conference on Computer and Robot Vision (CRV’05)
0-7695-2319-6/05 $ 20.00 IEEE