Proceedings of the 25" Annual International Conference of the IEEE EMBS
Caneun, Mexico * September 17-21,2001
Performance of a Head-Movement Interface for Wheelchair Control
P. B. Taylor', H. T. Nguyen'
'Faculy of Engineering, University of Technology, Sydney, NSW, Australia
Abslracl- Head movement has been used as a control
interface for people with motor impairments in a range of
applications. Chin operated joysticks and switch arrays have
been incorporated in control systems for electric wheelchairs
but have several disadvantages, including being dilfcult to
operate and aesthetically unattractive. A prototype wheelchair
control interface has been developed that makes use of an
artificial neural network (ANN) to recognize commands given
by head movement. This paper presents the results of an
experimental investigation of the ANN'S performance in terms
of classification accuracy and delay. It goes on to compare the
results of disabled with able-bodied users, and assesses the
effect of providing real-time feedback to the user.
The results obtained indicate that ANN techniques can be
used to classify head movements sufficiently quickly and
accurately to be used in a practical interface. The provision of
graphical real-time feedback does not appear to be crucial, but
may be of benefit for particular cases.
Keywords- head movement, neural network,
classification performance, power wheelchair control.
1. INTRODUCTION
Head movement has been used as a control interface for
people with motor impairments in a range of applications.
Chin operated joysticks and switch arrays have been
incorporated in control systems for electric wheelchairs hut
have several disadvantages, including being difficult or
unintuitive to operate and aesthetically unattractive.
More advanced interfaces have been developed in
several projects, but none have yet become commonplace.
Roy [I] showed that Artificial Neural Networks (ANNs) can
be used to recognize gestures. This work was able to reach a
classification accuracy of 80%.
Joseph [2] described a prototype wheelchair interface
using a triaxial accelerometer to measure head position and
an Artificial Neural Network (ANN) to interpret commands
given by head movements. The ANN in this system was
reported to have an accuracy of approximately 92%.
Another head movement based interface was described
by Bergasa [3]. This system generated commands by using
fuzzy logic to recognize the user winking , hiding their lips,
or tilting or turning their head, based on images captured by
a CCD camera. A practical issue reported for this system
was that it performed well in indoor environments with
suitable illumination, but this was reduced as light
conditions deteriorated.
0-7803-7789-3/03/$17.0002003 IEEE
Barea [4] presented initial work on an interface based on
electroculography (EOG), where the user gives commands
to the control system by eye movement. The measured EOG
potentials are passed into an ANN, and the output of this
network further analyzed to identify commands. The
performance of the ANN was not reported, but results
indicating that an able-bodied user used the interface to
control a wheelchair along a given path. A similar system,
combining EOG and ekctromyography (EMG) was also
presented by Law [5], along with results indicating that a
user was able to use the system, to perform 3 tasks.
Min [6] described two wheelchair control interfaces,
using head and shoulder movements. One interface allowed
users to give commands by either tilting or pitching their
head. The other used vertical movements of the shoulders.
Sensor data were translat,ed into commands using thresholds
and heuristics such as hysteresis. Results presented showed
that a group of six disabled users were able to use the
prototype interfaces to complete a short navigation task. The
head movement interface was reported to have a recognition
rate of approximately 75%, and the shoulder movement
approximately 70%.
A prototype wheelchair control interface reported in
Taylor [7] makes use of an ANN to recognize commands
given by head movement, similar to the system proposed in
[Z]. Users give commands to the system by nodding or
tilting their head. The movement of the user's head is
detected by analyzing data from a two-axis accelerometer,
collected with a sampling. period of 100 ms. The input to the
ANN is comprised of a window of 20 samples from each
axis. When the neural network has recognized a movement,
a simple set of heuristics is used to determine the resulting
speed and change of direction of the wheelchair. Results
presented in [7] indicated that ANN techniques could
achieve classification accuracies greater than 95% for able-
bodied users in a laborata'ry setting.
This paper presents the results of an experimental
investigation of the performance of a pretrained ANN in
terms of classification a.ccuracy and delay. It goes on to
compare the results of disabled with able-bodied users, and
assesses the effect of providing real-time feedback to the
user.
Feedback to the user has several forms. Real-time
graphical displays of the accelerometer data allow the user
to track the deviation of their head from the neutral position
over the preceding 100 samples. Boolean outputs from the
classifier and the numerical output values from the ANN
inform the user of how the classifier is interpreting their
movements.
1590
 The performance of the neural network is of
considerable importance in the development of the system.
The sensitivity and specificity of the classifier and the delay
between user's movement and its classification are of
particular significance, as these will affect the safety and
utility of the system.
11. METHODOLOGY
Data was collected from six adults, aged between 19
and 56, with approval from the UTS Human Research
Ethics Committee and informed consent from the
volunteers. Of these, two had high-level spinal cord injuries
(C4 and C5) and were not able to use a standard joystick to
control a wheelchair. The remaining four did not have
conditions affecting their movement.
A pre-trained ANN was used to classify the windowed
TABLE
I
CONFUSION MATRICES FOR ABLE-BODIED SUBJECTS, WITH AND WITHOUT THE DISPLAY OF REAL-TIME, GRAPHICAL BEDBACK.
data as corresponding to four types of head movement
commands: left, right, forwards or backwards. If the window
was not classified as belonging to one of the four classes, it
was interpreted as being neutral, or equivalently that no
command had been given in that window.
Data for each person was collected in two periods of ten
minutes, with the user being prompted to give a specified
movement every 6 seconds. Each specified movement was
chosen randomly from the following: neutral, forward nod,
forward hold, backward nod, backward hold, left nod, left
hold, right nod and right hold. The difference between a nod
and a hold movement is that a nod moves from the neutral
position to a maximum value and immediately begins
returning to the neutral, whereas a hold movement pauses
briefly at the maximum value before returning. A graphical
real-time display of sensor data and classifier output was
displayed to the subject for the first period of ten minutes.
This was then hidden for the second period and no feedback

-
Actual classification With feedback Actual classification - Without feedback
Able-bodied subjects
Forward Back Left Right Neutral Forward Back Left Right Neutral
Forward 80 0 0 4 6 86 0 0 I 2
Back 0 89 I 0 0 ' 0 94 I 0 0
Expected
classification Left 0
0 85 0 0 0 0 91 0 0
Right 0 0 0 86 I 0 0 0 91 0
Neutral 0 0 0 0 45 0 0 0 0 48

TABLE2
CONFUSION MATNCES FOR DISABLED SUBJECTS, WITH AND WITHOUT THE DISPLAY OF REAL-TIME, GRAPHICAL FEEDBACK.

Actual classification -With feedback Actual classification - Without feedback

Disabled subiccts
Forward Back Left Rieht Neutral Forward Back Left Rieht Neutral
~ ~ ~ ~~ ~ ~

Forward 37 0 0 2 3 41 0 0 0 3
Back 0 42 0 0 0 0 44 0 0 0
Expected
classification 0 0 45 0 0 0 2 38 I 2
Right 2 0 0 35 5 0 1 I 29 II
Neutral 0 0
--
0 0 23 0 0 0
-
0 22

TABLE3
CONFUSION MATRICES FOR ALL ABLE-BODIED SUBJECTS. AND ALL DISABLED SUBJECTS.

Actual classification -Able-bodied Actual classification - Disabled

All results
Forward Back Left Right Neutral Forward Back Left Rieht Neutral
Forward 166 0 0 5 8 78 0 0 2 6
Back 0 183 2 0 0 0 86 0 0 0
Expected
classification Left 0 0 176 0 0 0 2 83 I 2
Right 0 0 0 177 1 2 I 1 64 16
Neutral 0 0 0 0 93 0 0 0 0 45

1591
 TABLE
4
S l . U S l l l\'llY, SI'ECIFICITY. POSITIVL PP.EOIClI\'E V A L L S (PPV) A S 0 H I G A ' I IVE PREDICTIVE VAL.CI' (XPV) OFTHI( ARTlblCIAI. NEURAL
N L T H O F X AVF.RA(IED FOR T l l l i 4 POSSlRLt CLASSIFICATIONS.

Sensitivity Specificity PPV NPV

Able-bodied, with GUI 0.967 0.996 0.986 0.990

Able-bodied, without GUI 0.989 0.998 0.995 0.997
Disabled, with GUI 0.929 0.993 0.974 0.981
Disabled, without GUI 0.877 0.992 0.969 0.967
All able-bodied 0.978 0.997 0.990 0.994
All disabled 0.903 0.993 0.971 0.974

TABLE5
MEAN AND STANDARD DEVIATION OF THE DELAY (mr)BETWEEN MOVEMENT ANI1 CLASSIFICATION

Fonvard Back Lett Right

Mean SD Mean SD Mean SD Mean SD

Ablc-bodied, with GUI 1308 149 1278 98 996 100 983 165
Able-bodied, without GUI 1296 86 1212 75 969 IO9 948 94
Disabled, with GUI 1387 753 1624 802 1119 199 . 1178 461
Disabled, without GUI 1378 383 1285 206 1192 822 1260 663
All able-bodied 1302 120 I244 93 982 Ios 965 135
All disabled 1382 589 1445 592 1153 576 1215 557

was provided. IV. DISCUSSION

The raw sensor and classifier data was analyzed then to
produce the results described below. Each IO-minute period
was subdivided into the six-second windows corresponding
to each specified movement. Windows that contained
obvious errors by the subject were removed fiom the
analysis. The start of the movement within each window
was determined to be the point where the deviation fiom the
neutral position reached 25% of the maximum value on the
relevant axis. The classification of the movement was
determined as the first classification made by the ANN after
the start of the movement. The delays between the start of
the movement and the time of classification were then
calculated, and the confusion matrix relating the expected
and actual classification updated.
111. RESULTS
The confusion matrices show that the classifier tends to
fail to recognize movements at all rather than classify them
as another type of movement. This is also reflected in the
high positive predictive value and specificity.
As can be seen in the tables, the sensitivity, specificity,
positive predictive value (PPV) and negative predictive
value (NPV) of the ANN ranged from 0.877 to 0.989, 0.992
to 0.998,0.969 to 0.995 and 0.967 to 0.997 respectively, and
mean classification delays ranged from 948 to 1624 ms.
Although the number of subjects involved in this trial is
small, the results obtained lead to several useful
observations. The performance of the classifier and
similarities between the ;six subjects appear to validate the
approach used in developing of the prototype. The results
also suggest points needing further examination and
development.
No significant difference was found between specificity
of the ANN for able-bodied and disabled subjects. The
specificity was high for all of the subjects, indicating that
the neural network is unlikely to recognize one type of
movement as another.
Comparing the delays between the able bodied and
disabled subjects, it appe:ars that although the delay for the
disabled group is generally slightly longer, the difference
between the two groups is not statistically significant. It is,
however, notable that the able-bodied group had a smaller
variance in the delay. The implications of this larger
variance are not clear, but an increase in sample size may
assist in determining this.
The provision of graphical feedback made no significant
difference to the neural network statistics and classification
delays for the able-bodied subjects, and only sensitivity and
NPV were significantly reduced for the disabled subjects.
The majority of the misalassifications leading to this result
were associated with the "Right" class of one subject. This

1592
suggests that real-time feedback is useful where the electrooculography," IEEE Trms. Neural S w e m and
movement being recognized is close to the user's Rehobilitotion Engineering, vol. IO, pp. 209-218,2002.
[SI C. K. H. Law, M. Y. Y. Leung, Y. Xu, and S. K. TSO,"A cap as
comfortable range in that direction. interface for wheelchair control," Proc. IEEENSJ lnternmionol
There is a noticeable difference in the sensitivity of the Con/ on Intelligent Robols o n d s y s l e m , vol. 2, pp. 1439-1444,
neural network between the able-bodied and disabled 2002.
subjects. Again, this is largely a result of the relatively large [6] JLW. Min. K. Lee, S.-C. Lim, and D.-S. Kwon, "Human-
friendly interfaces of wheelchair robotic system for handicapped
number of misclassifications of the Right movement for one persons," Proc. IEEE/RSJ Intonational Con/: on Intelligent
subject, both with and without feedback. This highlights the Robots ondSysrem, vol. 2, pp. 2505-1510,200Z.
need to test the control system with individual potential 171 P. Taylor, H. Nguyen, and A. Craig, "Head Movement
users of the control system. It also suggests that Recognition for Power Wheelchair control," Proc. Annu. Con/
Engineering andphysical Sciences in Medicine, vol. I . pp. 135,
performance of the classifier could be improved by adapting 2002.
it to each particular user.
A practical system should include a fast acting input to
allow the user to react to sudden, unexpected events, for
example, to quickly stop the wheelchair after a
misclassification or in an emergency. The high PPV
obtained indicate that the ANN is able to operate without
producing large numbers of false positives. However, it is
also clear that a small number of false positives can be
expected, and the control interface must be designed to
allow for this. The delays observed indicate that use of the
existing ANN alone is not sufficient to achieve this.
However, use of adaptive algorithms or additional interface
techniques may solve this.

V. CONCLUSION

The results obtained indicate that ANN techniques can

be used to classify head movements sufficiently quickly and
accurately to be used in a practical interface. The provision
of real-time feedback does not appear to be crucial, but may
be of benefit for particular cases.

ACKNOWLEDGMENT

The authors thank Adrian Byak, Allan McCabe and

Matthew Peters for their assistance in the collection of data
for this experiment.

REFERENCES
D.M. P. Roy, M., "Computer recognition of athetoid cerebral
palsy movement using ncural networks," Proc. 16th Con/ IEEE
Engineering in Medicine and Biology Society, vol. I, pp. 458-
459,1994.
T. Joscph and H. Nguycn, "Neural network control of
wheelchain using telemcvic head movement," Pmc. 20th Con/:
IEEE Engineering in Medicine and Biology Society, vol. 5 , pp.
2731-2733, 1998.
L. M. Bcrgasa, M. Mazo, A. Gardel, R. Barca, and L. Boquete,
"Commands generation by facc movements applied LO the
guidance of a wheelchair for handicapped people," Proc. 1Slh
CO$ Panern Recognition, vol. 4, pp. 660-663,2000,
R. Barea, L. Boquctc, M. M a o , and E. Lopez, "System for
assisted mobility using eye movementS based on

1593