IEEE T R A N S A C T I O N S ON B I O M E D I C A L E N G I N E t R l N G . VOL 36.
N O I , J A N U A R Y 1989
Dynamical Neural Network Organization of the
Visual Pursuit Svstem
L. KELLER,
Abstract-The central nervous system is a parallel dynamical system
which connects sensory input with motor output for the performance
of visual tracking. This paper applies elementary control system tools
to extend dynamical neural network models to the visual smooth pur-
suit system. Observed eye position responses to target motions and
characteristics of the plant (eye muscles and orbital mechanics) place
dynamical constraints on the interposed neural network controller. In
the process of constructing a model for the controller, we show two
previous pursuit system models, using efference copy and feedforward
compensation, are equivalent from an input-output standpoint. We in-
troduce a controller model possessing a potentially highly parallel im-
plementation and offer an example with supporting neural firing rate
data. Changes in time delays or other system dynamics are expected to
lead to compensatory adaptive changes in the controller. A scheme to
noninvasively simulate such changes in system dynamics was devel-
oped. Actual physiologic data of adaptive responses to increased time
delay is presented as an example of the utility of this parallel control-
ler. Compensatory changes in our parallel controller model are easily
predicted. These results suggest a productive interaction between
neural network modeling, neurophysiology, and control systems engi-
neering.
I. INTRODUCTION
T HE FIELD of neural networks is characterized by ex-
plorations of properties of many simple dynamic sys-
tems interconnected to collectively form a system which
does computation. One common form can be considered
to be a function from a set of input vectors to a set of
output vectors. These networks serve as associative mem-
ories, pattern recognizers, and optimization schemes.
Good examples in this area include Kohonen and Oja [ 11,
Anderson [ 2 ] , and Hopfield and Tank [3]. Dynamical is-
sues in such systems concern convergence behavior in
“training” sessions or convergence in pattern completion
or optimization solutions.
Less common are neural net applications where dynam-
ics are central to the goal of the network. Here the net-
work is considered to be a functional from a set of inputs
(over time) to a set of outputs. Such applications often
Manuscript received February 8, 1988: revised July I , 1988. This work
was supported by Grants National Eye Institute EY0S913, EY06860.
EY06883, and EY06860, the Rachael Atkinson Fellowship, and the Smith-
Kettlewell Eye Research Foundation.
D. C . Deno is with the Department of Electrical Engineering and Com-
puter Science, University of Berkeley, CA 94720.
E. L. Keller is with the Department of Electrical Engineering and Com-
puter Science, University of California, Berkeley, CA, and the Smith-Ket-
tlewell Eye Research Foundation. San Francisco, CA 941 IS.
W . F. Crandall is with the Smith-Kettlewell Eye Research Foundation,
San Francisco. CA 941 IS.
IEEE Log Number 8824419.
0018-9294/89/0100-0085$01.OO
J
AND WILLIAM F. CRANDALL
use ideas from control systems engineering. For example,
Cannon and Robinson [4] use state feedback and pole-
placement to form an integrator from biologically plau-
sible elements. Closer to the present paper is an interest-
ing simulation by Barto et al. [5] of an inverted pendulum
controller with parameters “learned” by a neural net-
work. Sompolinsky and Kanter [6] describe a neural net-
work which is capable of recalling temporal patterns and
suggest this may be usefui in understanding rhythmical
patterns in biological motor systems.
The visual tracking system is attractive because it ap-
pears to consist of a simple sensory input to motor output
loop. To achieve good tracking behavior, the brain must
serve as a controller which compensates for time delays
and dynamics and is capable of adaptively responding to
changes in time delay or dynamics. From this perspec-
tive, the brain is a dynamical, highly parallel, adaptive
control system. This paper develops a dynamical neural
network which helps to unify two apparently disparate
models of smooth pursuit and clarify dynamical element
organization. Realization issues are addressed, physio-
logic signal correlates suggested, and a framework offered
to understand how adaptive changes might be imple-
mented to compensate for eye muscle or neural dynamics
changes.
A . Experimental Methods
Experimental data and models developed in this report
are taken from eye movement recordings in alert behaving
monkeys. Human pursuit eye movement responses are
similar, but slower. This paper concentrates on the pursuit
response to ramp target motions where the visual target
begins to move at a constant but unpredictable speed, at
a random epoch, and in an unpredictable direction. Pur-
suit responses to predictable target motion probably in-
volve a predictive neural network that is not considered
here.
The pursuit eye movements studied in this report were
generated by visual target motion produced by an oscil-
loscope projector system [7]. Eye position was measured
by the implanted induction eye coil technique [8].
Changes in plant dynamics were electronically simulated
by modifying target position based on eye position sig-
nals. Data were collected by digitizing the eye and target
position and velocity signals at 0.5 or 1 kHz sampling
rates. Real-time experiment control and data acquisition
0 1989 IEEE
86 I E E E T R A N S A C T I O N S O N B I O M E D I C A L E N G I N E E R I N G , VOL. 36, N O . I. J A N U A R Y 1989
Time (msec) from Target Motion
Fig. I . A representative smooth pursuit system response. Velocity is in
degree/second. A 2 O o / s step in target velocity is shown as the dashed
lines. The eye velocity recording is the jagged curve and the blank space
in this recording shows where a saccade was removed. The solid smooth
curve shows the best nonlinear least-squares fit to the underlying data.
Even within a set of trials in one animal there is Some variability in exact
delay, damping, and oscillation frequency. Since point-by-point aver-
aging in the face of this variability can obscure significant features we
use analyses of individual trials to yield parameterizations from which a
. I . .
Drototvpe response is derived from the median or mean parameter set.
Our model simulations are based on similarly generated prototype re-
sponses.
were accomplished by a set of laboratory computers (Ap-
ple 11, IBM-AT, and a PDP-11). Linear system manipu-
lation was aided by Program CC [9] and nonlinear nu-
merical simulation performed with Simnon [ IO].
B. Smooth Pursuit Major Attributes
The visual tracking system provides eye movements
which keep images of moving objects near the fovea where
visual acuity is highest. The eyes move in response to
target motion in a feedback-control system which seeks
to reduce fixation error. Eye movements take several
forms: smooth pursuit (sustained, low velocity motions
which help match eye and target velocity), saccades (ep-
isodic, high velocity movements to reduce position error),
and vergence (disconjugate movements which align im-
ages from near-field objects on the two retinas). The
smooth pursuit system has proven to be a fruitful subject
of study from both neurophysiology and engineering per-
spectives.
When tracking is elicited by a sudden target motion, the
usual response is to attempt to reduce this error by eye
movements consisting of both smooth pursuit and sac-
cadic components. Smooth pursuit eye movements with-
out corrective saccades can be induced by superimposing
a step in target position away from the fovea in the direc-
tion opposite to target motion (the Rashbass step-ramp
paradigm [ I I]). If the step size is chosen so that the eye
is just beginning to move as the target crosses the fovea,
then the saccadic portion of the pursuit movement will be
suppressed.
Fig. 1 shows a typical response in one monkey to the
step-ramp paradigm when the visual target velocity was
2Oo/s to the left. A major feature of the response is the
delay from the onset of target motion to the apparent ini-
tiation of the response. This delay is typically 100 ms but
can be as short as 80 ms or as long as 150 ms in some
animals. Other features of the pursuit responses are the
rapid initial eye acceleration immediately following the
period of the delay, the overshoot of eye velocity with
respect to target velocity, the subsequent damped oscil-
lation of eye velocity about true target speed, and the final
close approximation of steady-state eye speed to target
speed. All these features are similar to those previously
reported in man [12], [13] and monkey [14], [7].
11. THEORETICAL DEVELOPMENT
A . Oculomotor Physiology
to this system is the strut-
turd subtraction of target and eye position which occurs
on the retina as shown in ~ i qa). ~ The. current dogma is
that this visual error information is the primary input into
the pursuit system [ 141 and that neither target nor eye po-
sition information are directly available to the
system. From photoreceptors information then
travels to visual cortex and then to temporal cortical re-
gions believed specialized for motion analysis [ 151 where
cells respond preferentially to certain target speeds and
directions and are less influenced by retinotopic location.
These temporal cortex cells respond to target motion with
delays on the order of 80 ms. This velocity error infor-
mation is believed to be a major drive behind the smooth
pursuit system. Together with target position information
and higher cortical inputs such as volition, the velocity
information then passes to the pontine region of the brain
stem and then to the cerebellum where it interacts with
the signals helping to coordinate information that now
specifies motor commands [ 141. The information is then
reintegrated in the brain stem and sent to oculomotor nu-
clei where the information travels through the cranial
nerves to the eye muscles which rotate the eye.
B. Development of a Mathematical Model
From the conceptual information flow diagram of Fig.
2(a) we incorporate some linear systems dynamical no-
tation in Fig. 2(b). Cortical motion processing, convert-
ing position information into velocity information, is rep-
resented by a differentiator ( s in Laplace notation). The
loop delay has been lumped into a single element e-sT.
The transition from sensory information into motor infor-
mation is begun in the neural controller element labeled
Gc. This scheme also includes the generally accepted no-
tion that neural circuits which immediately precede ocu-
lomotor neurons provide both direct (velocity, 7 )and in-
tegrated (position, 1 / s ) signals to these neurons. The
“neural integrator” is assumed to have a time constant
much larger than 1-2 s and can thus be approximated by
an ideal integrator 1 /s. From data collected in vergence
studies [16] it appears that a step change in motor unit
firing rate results in eye position changes well modeled as
a two-pole plant with time constants of 0.2 and 0.015 s.
Higher order lumped parameter models of the eye, mus-
cles, and connective tissue [17] differ little in predicted
responses. Nonlinearities in the plant due to saturation are
unlikely in smooth pursuit where velocities are an order
of magnitude less than during saccades.
Fig. 2(c) shows the (parallel) premotor path effectively
DEN0 el u l . : NEURAL NETWORK ORGANIZATION OF VISUAL PURSUIT SYSTEM
Fig. 2. Basic structure of the smooth pursuit system. (a) Information flow
diagram. Retinal position information is the difference between target
and eye position. Representation of this information is unspecified arid
nonlinear dynamics are permitted. Abbreviations: cerebel. (cerebellum);
integr (integrator); propor. (direct proportiona' pathway). (b) Mathe-
matical model using Laplace notation. Delay has been lumped into a
single element located where the majority of delay exists. The s denotes
differentiation to obtain velocity information. The controller Gc is un-
specified and possibly nonlinear. The plant GP is replaced with a linear
systems model. Abbreviations: T(target position); E (eye position); RPE
(retinal position error). (c) Equivalent model. Velocity can be considered
to be the input and output of the smooth pursuit system. The premotor
integral and direct paths create a zero which may cancel the plant's slow
pole 7 ,since neural signals preceding this point seem to code for desired
eye velocity. (d) Final reduced model. Models of the pursuit system share
this basic arrangement but differ in realizations for Gc and G p . The con-
troller has input node A (delayed retinal velocity error, RVE) and output
node B (eye velocity command).
canceling the slow dynamics of the plant, as is commonly
assumed [ 131. It also shows that the input and output sig-
nals can be considered to be target velocity T and eye
velocity E , consistent with conventional descriptions of
the smooth pursuit system. Finally, this simplified con-
ceptual scheme can be formatted as shown in Fig. 2(d) in
which velocity inputs and outputs are explicitly shown.
Whether or not there is cancellation of the neural integra-
tor by differentiation and cancellation of a plant pole by
the premotor path, the resulting scheme may still be rep-
resented as in Fig. 2(d). All models and simulations of
the system can start with this basic topology-they will
differ only in the dynamics (possibly nonlinear) assigned
to the controller Gc and the premotor circuit/plant Gp.
C. Previous Pursuit System Models
Models of the pursuit system can be divided into three
broad classes [Fig. 3(a)-(c)] based on the controller model
Gc. Initial efforts to model smooth pursuit tracking de-
scribed the system as a simple proportional error feedback
servo that attempted to reduce the retinal slip velocity of
the selected target image [18]. Fig. 3(a) illustrates a sim-
ple velocity-servo in which Gc becomes a scalar gain fac-
tor K . This model fails on two counts. First, there is a
stability incompatibility in the requirement for both high
gain and a significant delay [ 191. Second, the model fails
87
c- "G ,)
Fig. 3. Various controller models. (a) Simple proportional gain. (b) Rob-
inson et al. [I31 (linearized). (c) Lisberger er al. [21] (linearized). (d)
Alternative cancellation scheme. A rearrangement of cancellation
schemes to emphasize three main components: G&! which embodies a
(inverse) model of the plant, e-'TMwhich is a model of total loop delay,
and GDa desired response element (e) Partial fraction re-representation
of D above. The straight forward compensation in D above can be dis-
guised as a parallel set of dynamical elements.
to account for observed periods of steady-state tracking
where E 1 T[13].
The second class of models, illustrated in Fig. 3(b),
may be called an efferent copy feedback model. A copy
of the neural command for desired eye velocity (node B )
is passed through a central model of the plant dynamics
and total loop delay e-"'GpM. The resulting output signal
summed with the delayed retinal velocity error signal
(node A ) recreates a central replica of target velocity TR.
This is a valuable transformation and some oculomotor
neurophysiology supporting a reconstructed TR has re-
cently been described in the cerebellar vermis by Suzuki
and Keller [20]. Taken together, Figs. 2(d) and 3(b) in-
dicate that this arrangement creates an effectively open-
loop system in which reconstructed target velocity di-
rectly generates eye velocity through a central processor
G;7. It is then clear that all the dynamic features observed
in E (Fig. 1) for step inputs in target velocity must be
generated by Gh. The recent model by Robinson et al.
[13] uses this sort of efferent feedback. By judicious in-
clusion of nonlinear elements their model displays appli-
cability over an extended range of target velocities.
The third class of model is exemplified by Lisberger et
88 IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING. VOL. 36. NO. I , JANUARY 1989

al. [21] as shown in Fig. 3(c). It uses the retinal motion
signal as the input to a parallel network which produces
weighted combinations of retinal position, velocity, and
acceleration errors (RPE, RVE, and RAE respectively).
These signals are then combined and fed through a final
‘‘velocity memory” integrator to produce the eye velocity
command. The necessity for the final integrator stems
from open-loop pursuit experiments of Lisberger et al.
[21]. If the visual feedback loop is electronically opened
by fixing the target’s position on the retina, the animal’s
eyes continue to move for a few seconds. Since retinal
position and velocity error can be fixed at zero, some type
of velocity holding circuit is needed and can be modeled
as an integrator. This model is attractive because the coef-
ficients of RPE, RVE, and RAE were directly estimated
by experimentally imposing controlled position, velocity,
or acceleration errors on the retinal input during open-
loop tracking. This model does not seem to extend as
readily to a wide range of velocities as the efferent feed-
back type [ 131, but it too can produce many features of
the monkey smooth pursuit response.
D. A Parallel Pursuit System Model
We introduce a variation on the controller schemes de-
scribed above which is shown in Fig. 3(d). For both con-
ceptual simplicity and because signals which precede the
neural integrator seem similar to eye velocity, we precede
the plant GPwith an inverse plant model GiL. For a linear
plant this implicitly requires GP be stable and minimum
phase. At the input of this inverse plant is thus a central
replica of eye velocity which, when passed through a
model of the loop delay e-FTM, can cancel retinal feedback
ical concepts. One question that arises from cancellation
models such as Fig. 3(b) and (d) is how to create internal
model delays on the order of 100 ms? It seems awkward
to string together 100 1 ms synaptic delay elements and
switch some in or out to adapt to different delays. Alter-
natively, one may approximate delays by systems which
use a sort of mutual cancellation, e.g., Pad6 approxima-
tions. Time delays realized in this manner make the tran-
sition to our parallel neural network controller of Fig. 3(e)
natural.
A . Controller Example
An example may help clarify our approach. A prototype
smooth pursuit response (similar to Fig. 1) for tracking a
10-20” /s targets was well fit by a linear, time-invariant,
deterministic, continuous system with transfer function
~-”G,(s) =
e-’.’”(5s +200)
+ +
0 . 0 0 3 ~ ~0 . 2 1 5 ~ ~6s + 200’
A single-pole linear plant model
1
Gp(S) = 0.015s +1
is assumed as motivated earlier and thus it is straight for-
ward to determine Gc from Fig. 3(d). Replacinge-”M with
a fifth-order Pade approximation leads to an 8th-order ra-
tional polynomial expression for Gc whose partial frac-
tion expansion consists of five terms. Under the condition
where delays T = T, = 0.1 s, a parallel controller rep-
resentation [as in Fig. 3(e)] is
G,(s) =
9.524
+ 8.659s + 713.5
+ 42.632
~

S (s - 2.715f
and thereby produce a central replica of the target’s ve-
locity. The element labeled Go denotes the desired dy-
+ + 695.6
.0436s
namics (possibly nonlinear) for the pursuit response.
As a conceptual tool, and for correspondence with (s + 22.59f + 93.762
neural recording data discussed later, we develop a par-
allel feedforward dynamical neural network representa-
+ 6.777s + 1402 - .00356
(s + 146.2f + 100.82 S + 39.57’
tion for this controller. Assuming Go is linear and choos-
ing rational polynomial approximations for GD and e-’TM, Because the first two terms dominate, the controller could
a partial fraction expansion of Gc leads to a network of consist of two parallel dynamical pathways.
the form shown in Fig. 3(e). This basic manipulation If T and T, were increased to 0.18 s, and the controller
shows, in the linear case at least, that the two classes of adaptively responded to preserve the same step response
models (parallel feedforward and delayed efferent feed- (except for an additional 80 ms delay) the controller rep-
back) can be shown to be equivalent. This creates a bridge resentation would be
between the Lisberger and Robinson perspectives. Fur-
thermore, issues of plasticity and adaptation are more eas-
5.405
G ~ ( s=) - + 12.90s + 220.1
ily addressed from the twin perspectives of Fig. 3(d) and
S (s - 3.326f + 29.752
(e). One can immediately predict what a corrective re- 0.2078s + 734.2
sponse to a change in system time delay, gain, or dynam- +
ics would entail (as done below). Finally, we are free to + 4.563)2 + 58.772
(s

+ 6.483s + 1107 - 0S.000000

select Go to match an arbitrary input-output pair, con- 12
trary to the claim by Robinson et al. [ 131.
( s + 97.93)2 + 79.102 + 40.00 ‘

111. NEURALNETSA N D NEUROPHYSIOLOGY Here, in addition to the first two terms, the third term
Models constructed from input-output responses are not plays a significant role. An adaptive change in Gc might
unique. Realizations are often selected based on simplic- consist of changing the handful of parameters involved in
ity and consistency with both current data and physiolog- these three terms.
D E N 0 er ul : NEURAL NETWORK ORGANIZATION OF VISUAL PURSUIT SYSTEM
B. Neurophysiologic Correlates
One possibility is that there are sets of brain cells whose
discharge envelopes approximate each of the five hypoth-
esized parallel signals in our expansion. There are, how-
ever, many equivalent schemes where cells carry linear
combinations of this model's paths.
For the case where T = 0.1 s and two parallel controller
paths predominate we next illustrate that neural firing rate
data can be consistent with the partial fraction expansion
above. The top panel of Fig. 4 shows the location of the
two significant outputs which together comprise the con-
troller. In the middle and lower panels of Fig. 4, the out-
puts of the first and second terms are paired with neural
discharge records actually recorded in the brainstem of a
monkey shown such a target motion. These cellular re-
sponses are examples of a variety of dynamic neural sig-
nals recorded with microelectrodes in the pontine nuclei
of monkeys (Keller and May, unpublished data). This re-
gion of the primate brain receives parallel inputs from a
wide variety of extrastriate cortical regions thought to be
involved in processing visual motion information. It has
outputs to regions of the cerebellum involved with smooth
eye movements [22]. Thus, hypothetical responses car-
ried in these two paths may have some basis in neuro-
physiology.
Without reference to the model, these neural responses
could be difficult to fit into a functional relationship with
the generation of pursuit. A direct interpretation of the
neural response shown on the middle panel of Fig. 4 is as
the integral of the retinal velocity slip signal. The func-
tion of the cell depicted in the lower panel may be to pro-
vide an early boost in eye velocity (at the cost of later
oscillations in eye velocity about the true target velocity).
Note that this neural signal is not in phase with either eye
acceleration or retinal acceleration error, as reflected in
the expression for G,( s). In fact, in a highly parallel sys-
tem each neuron engaged in control could possess a dif-
ferent response pattern. This highlights one of the major
differences of our parallel model from that of Lisberger et
al. [21] where only retinal velocity error and its deriva-
tives and integrals make up the parallel paths.
IV. CALIBRATION
A N D PLASTICITY
The requirement for calibration to maintain accurate in-
ternal correlates of dynamic elements is often pointed out
in discussions of the vestibulo-ocular reflex (VOR) (e.g.,
[24]). The plasticity of neural gains in the VOR has been
clearly demonstrated by the telescopic spectacle and sim-
ilar paradigms. Adjustments in synaptic efficacy are con-
sidered likely mechanisms for gain alterations required to
keep the essentially open-loop VOR accurate.
The utility of compensators that employ (explicitly or
implicitly) models of the plant's delay and dynamics was
discussed above. The models proposed did not include
descriptions of how such internal models are kept in ac-
curate form. Upon suddenly perturbing the system's delay
or dynamics, the immediate resulting response should re-
89
m
I I
0 l5 1
Time (msec) from Target Motion
Fig. 4. Parallel controller diagram and outputs. The plant was assumed to
be linear single pole, the desired response assumed to be a linear second-
order system, and the model time delay replaced by a fifth-order Pad6
approximation. The resulting transfer function from A to E , after partial
fraction expansion, consisted of only two dominant terms or elements.
Top: the two parallel controller elements G,, and Gcz are shown within
the pursuit system model. Middle: the G,, term was effectively an in-
tegrator and its output is shown by the smooth curve. It is matched with
neural firing rate data from single-unit recording during such a pursuit
trial. Both records are synchronized on onset of target motion. Lower:
the other term GC2was a second-order system with an output shown by
the smooth curve. Again, neural firing rate data from single-unit record-
ing to show the existence of units with activity that can plausibly resem-
ble the outputs of such simple parallel dynamical elements. All controller
outputs are to the same scale.
flect sensitivity to an imperfect internal model, and the
long term adaptation to such a change would reflect a re-
calibration of this internal model.
It is not possible to actually modify the plant's dynam-
ics or delay in normal human subjects or without invasive
procedures on primates. The diagrams of Fig. 5 show an
approach to provide the retina with information consistent
with modified plant dynamics. To the extent that the ret-
ina constitutes the sole or major feedback pathway in pur-
suit, this external manipulation must completely simulate
the desired change to the plant.
Consider a modified plant whose dynamics are that of
the original plant in series with a new element wch ad-
ditional dynamics, gain, and time delay Gp =
GpGAePSTA [Fig. 5(a)]. The old plant is a special case
where GA(s) = 1 and TA = 0. Next, we move the
GAe-sTAblock maintaining the same retinal signal and
overall input-output relation. As shown in Fig. 5(b), this
results in an element that uses current and past eye posi-
tion to modify the actual target position T (perc%ived by
the subject) from the "planned" target position T :
T = f + ( 1 - GAe-"A)E.
90 IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING, VOL. 36, NO. I , JANUARY 1989

E 40 1 n
:
V
e
1

r - - - - - - - - -I

E
GAe-"b
B
z
V
e
1
0
C
(b)
Y
i
Fig. 5. Target modification scheme to permit simulation of altered plant
dynamics. (a) Motivational diagram. Consider the old plant G , to be in
series with a new dynamical element GAe-'TAthereby forming the new E 60
plant Gp. (b) Equivalent diagram. Simulation of altered plant by elec-
tronic modification of target position utilizing an element with transfer :
function 1 - G,e-"7'. The term G , may incorporate dynamics (possibly V 40
nonlinear). e
1
0
F:

We have implemented digitally an element G,( s) = 1 - t.

GAePSTA that operates at rates of 1-2 kHz for both vertical
and horizontal directions and allows selection of nearly
any time delay, gain, and single-pole location.
Some preliminary results for delay modification are
presented in Fig. 6. The top panel shows a fairly oscil-
latory pursuit trial without saccades and under the control
condition GA = 1 and TA = 0. The smooth curve is a
least-squares fit of a second order linear system step re-
sponse to this pursuit response. The oscillation period is
about 300 ms. The middle panel of Fig. 6 is a represen-
tative eye velocity response a few trials after TAhas been
increased to 80 ms. The peak eye velocity has increased
to nearly 6Oo/s and the oscillation period is approxi-
mately 600 ms. After three days of three-hour training
sessions with TA fixed at 80 ms, with randomized target
speed and direction, a typical eye velocity record was se-
lected and appears on the lower panel of Fig. 6. Evidence
supporting an adaptive system is shown by a decreased
oscillation period of about 400 ms.
Our observations for the pursuit system's initial re-
sponse to an acute increase in time delay are similar to
those described recently by Goldreich and Lisberger [23].
They increased delay by reducing target intensity and by
Y
-200 0 200 400 600 800 1000 1200
Time (msec) from Target Motion
Fig. 6 . Illustration of time delay adaptation. Velocity is in degree/second.
Top: fairly oscillatory smooth pursuit response under control conditions
to a 20"/s target. The smooth curve is the result of a least-squares fit to
the data from -200 ms to 1200 ms. Middle: same monkey as above with
TAacutely set to 80 ms to simulate an extra 80 ms delay. The oscillation
period has gone from nearly 300 ms above to about 600 ms. Lower: after
three days of three-hour training sessions with TA at 80 ms a typical
pursuit response now has an oscillation period of about 400 ms indicating
partial adaptation to the additional time delay.
alter the visual feedback need not differentiate between
these two classes of models.
Adaptation to such a change in delay means that the
neural delay correlate can be reconfigured. One possibil-
ity is the connection of additional neurons in series to pro-
long compensatory delay time T,. Alternatively, in the
spirit of our Fig. 3(e) and the partial fraction expansion
example earlier, several parallel-path synaptic strengths
might be altered to effect a compensatory delay change in
the controller.
V . DISCUSSION A N D CONCLUSIONS

an electronic means similar to that described above. In
their experiment, the oscillation period of the monkey
pursuit response increased monotonically with added de-
lay in the feedback loop.
They claim that the parallel feedforward class of models
can reproduce this effect but that the efferent feedback
class cannot. We have run similar simulations on both
classes of models and have confirmed their observation
for simulations where the parameters are adjusted so that
the eye shows a pronounced oscillation at the nominal de-
lay. However, when the simulation is adjusted to produce
a more damped response (which fits many animals that
we have observed) both models make similar predictions.
From the equivalence of the linear controllers shown in
Fig. 3(d) and (e) it should be clear that experiments which
Neural networks have appeared in conjunction with
adaptive control as early as 1960 with the paper by Wid-
row and Hoff [25]. The ADELINE (ADAptive LInear
NEuron) pursued a least mean square algorithm based on
performance feedback to form the "best" linear switch-
ing circuit. Although this paper does not mention mech-
anisms governing adaptive control, the topic has been re-
viewed recently by Astrom [26]. Instead we focus on
application of neural nets and dynamical systems theory
to visual tracking system neurobiology.
This paper has emphasized a role for internal models of
plant dynamics and loop delay in compensation. We have
illustrated that these models may be either explicit
[lumped feedback, Fig. 3(d)] or implicit [parallel feed-
forward, Fig. 3(e)]. The lumped-feedback controller rep-
D E N 0 cl 01.: N E U R A L NETWORK ORGANIZATION OF VISUAL PURSUIT SYSTEM
resentation used cancellation to explicitly recreate a cen-
tral signal corresponding to target position or velocity.
This strategy for the oculomotor system is attributed to
Young et al. [27]. We use these two extremes to motivate
descriptions of compensation schemes which help provide
fast accurate tracking in the face of significant time de-
lays.
Linear system theory is not wholly adequate to analyze
the smooth pursuit system. It is used, in part, because it
is a familiar and tractible analytic tool for this extension
into dynamical neural networks. Lisberger et al. [21] and
Robinson er al. [ 131 have found nonlinear effects in mon-
keys at velocities as low as 5 ” / s . Our simulations and
experimental data show that linear model parameters (cal-
culated from our animal’s pursuit) provide adequate pre-
dictions of pursuit response if target velocities remain in
the 20” /s and less range. Over a broader range of inputs,
the smooth pursuit system behaves nonlinearly and there
are many ways one can seek to model these effects. The
Lisberger et al. model directly determines parameters
which govern nonlinearity. However, their controller
seems less rich than those of Robinson er al. or ours since
the linearized controller is restricted to multiple poles at
the origin.
Our pursuit system model of Fig. 3(d) permits a direct
correspondence between internal models and system be-
havior. Adaptive responses by the controller to changes
in system dynamics [which follow directly from Fig. 3(d)]
are readily reexpressed for the parallel realization of Fig.
3(e). Neural firing rate data was offered which was com-
patible with a parallel feedforward realization. Evidence
supporting partial adaptation to increased time delay was
given. In fact, since 100-300 ms delays from sensory in-
put to motor response are so common one may expect de-
lay compensation in biologic motor control to be omni-
present. These may be exciting grounds for interaction
between mathematical possibility, engineering plausibil-
ity, and experimental neurophysiology.
REFERENCES
[ I ] T. Kohonen and E. Oja, “Fast adaptive formation of orthogonalizing
filters and associative memory in recurrent networks of neuron-like
elements,” Biol. Cybern., vol. 21, pp. 85-95, 1976.
[2] J . A. Anderson, “Cognitive and psychological computation with
neural models,” IEEE Trans. Syst., Man, Cybern., vol. SMC-13, pp.
799-815, 1983.
[3] J. J . Hopfield and D. W . Tank, “Computing with neural circuits: A
model,” Science, vol. 233, pp. 625-633, 1986.
[4] S . C. Cannon and D. A. Robinson, “An improved neural-network
model for the oculomotor system: More realistic neuron behavior,”
Biol. Cybern., vol. 53, pp. 1-16, 1985.
[ 5 ] A. G . Barto, R. S . Sutton, and C. W . Anderson, “Neuronlike adap-
tive elements that can solve difficult learning control problems,” IEEE
Trans. Syst., Man, Cybern., vol. SMC-13, pp. 834-846, 1983.
[6] H. Sompolinski and I . Kanter, “Temporal association in assymetric
neural networks,” Phys. Rev. Lerr., vol. 57, pp. 2861-2864, 1986.
[7] E. L. Keller and N. S . Khan, “Smooth-pursuit initiation in the pres-
ence of a textured background in monkey,” Vision Res., vol. 26, pp.
943-955, 1986.
[8] A. Fuchs and D. Robinson, “A method for measuring horizontal and
vertical eye movements chronically in the monkey,” J . Appl. Phys-
id,, vol. 21, pp. 1068-1070, 1966.
91
[9] Program CC-Computer-aided control system design package, Syst.
Technol;, Inc. Hawthorne, CA 90250.
[IO] K. J . Astrom, ”A Simnon tutorial. CODEN: LUTFDZ/(TFRT-
3168);’ Lund Inst. Technol., Lund, Sweden, 1982.
[ 1 I] C. Rashbass, “The relationship between saccadic and smooth track-
ing eye movements,” J . Physiol., vol. 159, pp. 326-338, 1961.
1121 J . Pola and H. J . Wyatt, “Target position and velocity: The stimulus
for smooth pursuit eye movements,” Vision Res., vol. 20, pp. 523-
534, 1979.
131 D. A. Robinson, J. L. Gordon, and S . E. Gordon, “A model of the
smooth pursuit eye movement system,” B i d . Cybern., vol. 5 5 , pp.
43-57, 1986.
141 S . G. Lisberger and L. E. Westbrook, “Properties of visual inputs
that initiate horizontal smooth pursuit eye movements in monkeys,”
J . Neurosci., vol. 5 , pp. 1662-1673, 1985.
151 D. C. van Essen, J . H . R. Maunsell, and J . L. Bixby, “The middle
temporal visual area in the macaque: Myeloarchitecture, connections,
functional properties, and topographic organization,” J . Comp. Neu-
rol., vol. 199, pp. 293-326, 1985.
161 E. L. Keller, “Accomodative vergence in the alert monkey,” Vision
R e s . , vol. 13. pp. 1565-1575. 1973.
171 D . A. Robinson, “The mechanics of human saccadic eye move-
ment,” J . Neurophys., vol. 174, pp. 245-264, 1964.
181 L. Young and L. Stark, “Variable feedback experiments testing a
sampled data model for eye tracking movements,” IEEE Trans. Hu-
man Fucrors Electron. vol. HFE-4, pp. 28-51, 1963.
191 D. A. Robinson, “Models of oculomotor neural organization,” in
The Control of Eye Movements. P. Bach-Y-Rita and C . C. Collins,
Eds., New York: Academic, 1971, pp. 519-538.
[20] D. A. Suzuki and E. L. Keller, “The role of the posterior vermis of
monkey cerebellum in smooth pursuit eye movement control. 11. Tar-
get velocity-related Purkinjie cell activity,” J . Neurophysiol., vol.
59, pp. 19-40, 1988.
[21] S . G. Lisberger, E. J . Morris, and L. Tychsen, “Visual motion pro-
cessing and sensory-motor integration for smooth pursuit eye move-
ments,” Ann. Rev. Neurosci., vol. 10, pp. 97-129, 1987.
1221 J . G . May, E. L. Keller, and D. A. Suzuki, “Smooth-pursuit eye
movement deficits with chemical lesions in the dorsolateral pontine
nucleus of the monkey,” J . Neurophysiol., vol. 59. pp. 952-977,
1988.
[23] D. Goldreich and S . G. Lisberger, “Evidence that visual inputs drive
oscillations in eye velocity during smooth pursuit eye movements in
the monkey,” Soc. Neurosci. Absrr., vol. 13, p. 170, 1987.
[24] F . A. Miles and S . G. Lisberger, “Plasticity in the vestibulo-ocular
reflex: A new hypothesis,” Ann. Rev. Neurosci., vol. 4, pp. 273-
299. 1981.
[25] B . Widrow and M. E. Hoff, “Adaptive switching circuits,” presented
at IRE WESCON Convention Record, pp. 96- 104, 1960.
[26] K. J . Astrom, “Adaptive feedback control,” Proc. IEEE, vol. 75,
pp. 185-217, 1987.
[27] L. Young, J . Forster, and N. Van Houtte, “A revised stochastic sam-
ple data model for eye tracking movements,” presented at Fourth Ann.
NASA-Univ. Conf. Manual Contr., Univ. Mich., Ann Arbor, MI,
pp. 489-508, 1968.
D. Curtis Deno (M’78) was born in Schenectady,
NY, in 1954. In 1976 he received the B S degree
in electrical engineering and biomedical engineer-
ing and the M.S. degree i n biomedical engineer-
ing from Rensselaer Polytechnic Institute, Troy,
NY. He received the M.D. degree from Hahne-
mann University, Philadelphia, PA in 1980.
He completed a one-year internship with the
Department of Medicine, Albany Medical Col-
lege, Albany, NY in 1981 and subsequently did
three years as an NIH postdoctoral fellow in phys-
iology, including one year of clinical trauma research, at Albany Medical
College Since 1984 he has been working on the Ph D degree in electrical
engineering from the University of California, Berkeley, CA, where he has
advanced to candidacy. His major interests include. control systems, ma-
chine and biological vision, neural networks, dynamical systems theory,
real-time control, neurobiology, physiology, and medicine.
Dr. Den0 is a member of Sigma Xi and Alpha Omega Alpha.
1' 2