Proceedings of the 20th Annual Znternational Conferenceof the ZEEE Engineering in Medicine and Biology Society, Vol.
20, No 5,1998
NEURAL NETWORK CONTROL OF WHEELCHAIRS USING
TELEMETRIC HEAD MOVEMENT
Trevor Joseph and Hung Nguyen, Member IEEE
Electrical Engineering, University of Technology, Sydney
PO Box 123, Broadway NSW 2007, Australia
htn@ee.uts.edu.au
Abstract
Powered wheelchairs are traditionally used by people with
insuflcient upper body strength and dexterity to operate a
manual wheelchair. However, the operation of these
wheelchairs can still be a diBcult and stressful task. Head
movement is a natural form of pointing and can be used to
directly replace the joystick whilst still allowing for similar
control. Through the use of artiJicia1intelligence, a trainable
wheelchair controller can be designed which provides an
alternative control method with improved posture, ease of use
and attractiveness. A computer simulation of this head-
controlled wheelchair has been successfilly designed and
tested. It consists of a motion detector for head motion
measurement, a telemetry systemfor the elimination of wiring,
and neural networks to provide the system with the ability to
be trained for each individual operator irrespectively of their
disability.
Introduction
Powered wheelchairs are normally used by people who do not
have sufficient upper body strength to operate a manual
wheelchair. The operation of these wheelchairs is still a
difficult and demanding task. To overcome the problems
associated with joystick control, a new variety of interfaces
have been designed to aid in the use of the joystick. This
includes repositioning or adding mechanical attachments to
the end of the joystick so that it can be operated by a person’s
elbow or chin. Other replacements for the joystick include
eyewink control (Crisman, 1991) and voice control (Mazo,
1995a).
Head movement is a natural form of pointing and can be used
to directly replace the joystick whilst still allowing for similar
control. This method of wheelchair control has been used
previously by sufferers of high level spinal lesions and motor
neurone disease. One head movement-sensing device (the
Oxford Optical Pointer) was developed by Tew (1988). This
device used a photo quadrant sensor which is made up of four
photodiodes fabricated on one substrate. As a light spot from
’ For the motion detector and transmission module as shown in
an Mared emitting diode passed over the detector, the
relative proportion of photocurrents fiom each quadrant was
used to determine the light source position.
Recently, Coyle (1995) developed a system which uses a
combination of technologies. This system used the standard
hand controlled joystick, an ultrasonic non-contact head
0-7803-5164-9/98/$10.000 1998 IEEE 2731
controller and a small vocabulary voice recognition system.
The ultrasonic head control system consisted of three sets of
transmitterheceiver units which were mounted on the fiame
of the user’s head. They emitted inaudible sound waves which
were picked up and sent back when they hit an object. The
time’ of travel of the sound was measured and used to
calculate the distance from the transducer to the object. The
combination of all three transducers was used to determine
the current position of the user’s head.
In this paper, we describe the development of an alternative
telemetric head movement device for the control of power
wheelchairs. In contrast to the other head pointing devices
where infrared or ultrasonic transmitters are used, we use
radio fiequency technology and digital telemetry. This system
has been designed to enable disabled people to control a
powered wheelchair easily, accurately and comfortably. It has
also been designed to adapt to each individual user using
neural networks.
Methods
As shown in Figure 1, the entire system consists of four
modules: a motion detector and transmission module, a
receiver and data acquisition module, a neural network
module and a computer simulation module.
.
L
II
I
U
Figure 1: System block diagram
Fig 2, we use the Microstrain triaxial accelerometer telemetry
system. This module is responsible for accepting the head
motion input from the operator and transmitting it via the
telemetry system. The position of the operator’s head is
sampled at a rate of 28 sampleshecond. The x, y and z
coordinates are measured. These coordinate values are
incorporated into a data packet which consists of eight bytes.
Pulse code modulation (PCM) is then used to modulate the
signal on the carrier frequency of 916.5 MHz (radio
frequency). The PCM signal is then transmitted.
Figure 2: Motion detector and transmission module
In the receiver and data acquisition module as shown in
Figure 3, the PCM signal is received, filtered and
demodulated into the original packets. The x, y and z
coordinates of the motion are then extracted from each data
packet. The checksum byte is compared with a sum of the
data bytes to check if an error in transmission has occurred. If
an error has occurred, the data is tagged for later analysis.
The motion coordinates are then normalised to a value range
of zero-to-one which is required by the neural networks
module. Here, twenty data coordinates are saved to disk to
form part of the neural network training set. The receiver,
filter and demodulation subsystem is part of the Microstrain
telemetry system. The remainder of this module needs to be
designed in software.
Figure 3: Receiver and data acquisition module
The neural network module as shown in Figure 4 is
responsible for deciding what head motion the user has just
performed based on pattern recognition of the data received
from the motion detector.
Figure 4: Neural network module
This module consists of three functional submodules: training
algorithm, weight vector storage and testing algorithm. In the
first submodule, the neural network training algorithm is
implemented using the error back propagation and delta
learning training rule. In the second submodule, the neural
network weight vectors, which contain vital information
about the recognition of patterns and trends in the system, are
stored or retrieved. In the third submodule, the weight
2732
vectors are used to determine which direction the operator has
moved hisher head.
The structure of the chosen neural network is shown in Figure
5. For the input layer, we choose 61 neurons which contain
60 head motion input values and one bias input (-1). The first
20 values are x-coordinates, the next 20 values are y-
coordinates and the last 20 values are z-coordinates. For the
output vector, eight neurons are required. Each output neuron
corresponds to one of the eight head motion directions that
the network must classify. They are north, south, east, west,
northeast, northwest, southeast and , southwest. With eight
output neurons, the hidden iayer needs to consist of at least
six neurons plus one bias neuron.
Due to the necessary augmentation of inputs and of the
hidden layer by one fixed input to account for bias (or offset),
this neural network should have 61 input neurons, 7 hidden
neurons, and 8 output neuron. There are two weight matrices
for
- the neural network the hidden layer weight matrix
W(6x 61) and the output layer weight matrix W(8x7) .
In this paper, the neural network is trained using the back-
propagation learning algorithm in which synaptic strengths
are systematically modified so that the response of the
network increasingly approximates the command from the
user. This is an optimisation problem and the optimisation
algorithm we use here is simply negative gradient descent.
The learning algorithm modifies both the output layer weight
matrix W and the hidden layer weight matrix w
so that the
error value decreases.
Hidden Layer Output Layer
Weight Matrix
Weight Matrix
F W
Hidden Nodes Output Nodes
Figure 5: Feedforward multilayer neural network
The major software design consideration for this system is the
operating system used for the computer simulation. As the
DOS operating system is being used, there is a constraint on
the programming language used to develop the software.
Borland Turbo C++ for DOS was chosen as the best choice
for software development.
Four programs are needed to be developed. These are head
motion recording, neural network training, neural network
testing, and real-time computer simulation programs.
The first program is required for the data acquisition module
of the system. It is responsible for the recording of single
head movements which are then used to build up the training
set for the neural network module. The second program is
responsible for implementing the neural network training
algorithm. The third program is used for testing the neural
network training procedure. If the neural network is unable to
classify a very high percentage of the training set then more
training or a new set of random weight matrices is required.
The fourth program is responsible for graphical simulation of
the wheelchair control. It is by far the most complex of the
four programs.
Results
The success of this system is highly dependent on the neural
network training. The sample patterns (in this case head
movement recordings), which form a training sequence, are
presented to the neural network classifier along with the
correct response. The classifier then modifies its parameters
(weights) by means of iterative supervised learning. The
network just learns from experience by comparing the
targeted correct response with the actual response. Note also
that in addition to eight motion directions, this system can
also recognise the ninth motion, sharp random head shaking,
which is used as a safety switch to slow down and stop the
wheelchair in an emergency.
After comprehensive testing with various sets of weight
matrices, the final set of weight matrices was chosen. The
overall complete system results are as follows. The final
system was able to detect head motions and identify them
with a success of 91.7%. The system was able to correctly
detected 95% of head shaking - movements. The decision
making algorithm was shown to function correctly for every
possible combination of current speed and direction and new
head motion (all 320 cases = 8 directions x 5 speeds x 8 head
movements).
Conclusion
We have developed successfully a telemetric head-movement
system suitable for the control of powered wheelchairs. This
system was also tested using a neck brace to limit head
movement with a consistently high success rate of correct
recognition (around the 90% mark). The system has many
2733
benefits for the disabled wheel chair operator: visual benefit
(no wiring, small motion detector), operator mobility and
posture (full mobility, upright posture), individual operator
training, and ease of installation.
References
VI Adam JA (1994) Technology combats disabilities. IEEE
Spectrum October: 24-26.
[2] Brown KE, Inigo RM, Johnson BW (1990) Design,
Implementation, and Testing of an Adaptable Optimal
Controller for an Electric Wheelchair. IEEE
Transactions on Industrial Educations, 26: 1144-1157.
[3] Coyle ED (1995) Electronic Wheelchair Controller
Designed for Operating by Hand-Operated Joystick,
Ultrasonic Non-Contact Head Control and Utterance
from a Small Word-Command Vocabulary. IEE
Colloquium (Digest), 55: 3/1-3/4.
[4] Cooper RA (1995) Intelligent Control of Power
Wheelchairs. IEEE Engineering in Medicine and
Biology, 14:423-431.
[5] Crisman EE, Loomis A, Shaw R, and Laszewski Z
(1991) Using the Eye Wink Control Interface to Control
a Powered Wheelchair. Proceedings of the Annual
International Conference of the IEEE Engineering in
Medicine and Biology Society, 13:1821-1822.
[6] Kauffmann C, Zasadzinski M, and Darouach (1993)
Robust Optimal Control Design applied to an Electric
Wheelchair. Proceedings of the IEEE International
Conference on Systems, Man and Cybernetics, 3559-
564.
71 Mazo M, Rodriguez FJ, Lazoro JL,, Urena J, Garcia JC,
Santiso E, and Revenga PA (1995a) Electronic Control
of a Wheelchair guided by voice commands. Control
Eng. Practice, 3: 665-674.
81 Mazo M, Rodriguez FJ, Lazoro JL,, Urena J, Garcia JC,
Santiso E, Revenga PA, and Garcia JJ (1995b)
Wheelchair for Physically Disabled People with Voice,
Ultrasonic and Infrared Sensor Control. Autonomous
Robots, 2: 203-226.
[9] Ouiguini R, and Belloulou B (1996) Navigation of an
Autonomous Wheelchair in a Structured Environment.
Proceedings of the IEEE International Conference on
Industrial Electronics, Control, and Instrumentation, 2:
749-754.
[ 101 Schalk TB (1987) A Voice-Activated Wheelchair.
Speech Technology, 4: 44-48.
[ l l ] Tew AI (1988) The Oxford Optical Pointer: A
Direction-sensing Device with Proportional Electric
Output. Med. & Biol. Eng. & Comp., 26: 68-74.
[ 121 Witters DM, Ruggera PS (1994) Electromagnetic
Compatibility (EMC) of Powered Wheelchairs and
Scooters. Proceedings of the Annual International
Conference of the IEEE Engineering in Medicine and
Biology Society, 16: 894-895.