Analysis of Unconventional Evolved Electronics

Adrian Thompson & Paul Layzell

Centre for Computational Neuroscience and Robotics,
School of Cognitive & Computing Sciences,
University of Sussex, Brighton BN1 9QH, UK.
adrianth, paulla @sussex.ac.uk

1 Introduction form the task. Supercially, this suggests an increased

chance of success, and a guaranteed utility of the nal
In recent years, several research groups have demon- result, in that we know it is likely to be `well-behaved'
strated the potential for articial evolution to design
electronic circuits automatically. An `evolutionary al-
gorithm' is implemented, usually as a computer pro-
gram, which mimics some aspects of Darwinian evolu-
tion: small random variations (`mutations') are repeat-
edly made to one or more candidate circuit designs. Nat-
and amenable to analysis using established techniques.
The unconstrained approach, however, is alluring
(Thompson, 1998). Evolution is free to explore very un-
usual designs: circuits with strange structures and intri-
cate dynamical behaviours beyond the scope of conven-
ural selection is replaced by a `tness' measure of the de- tional design and analysis. In this larger search-space,
gree to which a circuit meets the target engineering spec- there is the possibility of better solutions, if we are pre-
ication of behaviour, size, power-consumption, and so pared to set aside our prejudices (based on existing de-
on. Mutations resulting in a poorer tness measurement sign methods) of how an electronic circuit should be. The
are rejected, whereas those producing an improvement unrestricted design-space may also contain more evolv-
(or at least no deterioration) are allowed to persist and be able circuits: those more susceptible to gradual adap-
built upon by further rounds of variation and selection. tation, through a sequence of small changes, towards
Commencing with either randomly generated circuit de- meeting the specication. Attempts to aid evolution by
signs, results of previous evolutionary experiments, or restricting it to conventional architectures, usually the
an initial hand-designed attempt, repeated cycles of au- products of top-down design, may actually hinder it if
tomated evolution can eventually produce circuits which these structures are not so easily improved through evo-
satisfy the specication. Many elaborations to this basic lution's more bottom-up activities. Furthermore, if the
scheme exist, such as allowing `recombination' of features most promising application domains are the ones prob-
from two or more individuals in an evolving population lematic for conventional design, then the exploration of
of variants. new strategies is appropriate. Finally, if we are to learn
Electronics design through articial evolution is be-
coming practical for some real-world applications (Sip-
per, Mange & Perez-Uribe, 1998), especially in do-
mains where conventional design methodologies strug-
gle. These include automated analogue circuit design
(Rutenbar, 1993), and design under dicult implemen-
tation constraints such as size, power-consumption, or
even fault-tolerance (Thompson, 1997a). Here there are
applications even for relatively simple evolved circuits.
When setting-out to evolve a circuit for some task,
or be inspired by the products of articial evolution, then
we must give it the freedom to produce circuits working
in an unexpected way.
Clearly, to learn from an evolved circuit, we need to be
able to discern some of its principles of operation. How-
ever, even if the goal is merely to evolve a circuit that
works (and we don't need to know how), some degree of
analysis may still increase its utility as an engineering
product. In particular, if bounds on possible long-term
changes in the circuit's behaviour can be derived, then
there is a fundamental decision to be made: whether the circuit can be applied with condence more widely.
the evolutionary process is free to explore any possible The need for extended consistent performance is di-
design, or whether it is constrained to encourage `sensi- cult to accommodate within an evolutionary framework,
ble' circuits more like those arising from conventional de-
sign methods. In many ways, the constrained approach
seems like a good idea. The search-space of possible de-
signs through which evolution must navigate is reduced
in size, to a set of circuits that we can readily under-
stand, and some of which we know should be able to per-
because usually the tests for tness measurement of can-
didate solutions (the bottleneck in the evolutionary pro-
cess) are as brief as possible. The evolutionary approach
can be made more viable if, through analysis, it can be
predicted that a circuit will perform adequately in the
long term, even though it was never tested for long dur-
ing its evolution.
There are two components to long-term stability of be-
haviour. First, the circuit must be insensitive to certain
variations in its implementation or environment, or able
to adapt to them. Examples include thermal drift, noise,
tions (Simon, 1996). Systems designed by humans can
usually be understood in this way, because of the `di-
vide and conquer' approach universally adopted to tackle
complex designs.
Although an evolved system may have particular func-
and ageing eects in semiconductor devices and inte-
grated circuits. Second, it is possible for even simple dy-
namical systems to display intermittent behaviour over
long time-scales (Pomeau & Manneville, 1980). This is
not due to any external
uctuations, but is a property of Hence, what to many people is the essence of under-
the system's own dynamics. Circuits can be constructed
which will inevitably | though after a long period of
normal operation | suddenly and unpredictably change
in their qualitative mode of behaviour; possibly forever,
or perhaps to return to normal operation for another long
interval (Jeeries & Deane, 1997). An evolutionary algo-
tions localised in identiable subsystems, this is not al-
ways so. Dynamic systems theory (Burton, 1994) pro-
vides a mathematical framework in which systems can
be characterised without a functional decomposition.
standing | being able to point at parts of the whole and
say what function they perform | is not always possible
for evolved systems. In this case, more precisely formu-
lated questions regarding the organisation of behaviour
must replace fuzzy notions of `understanding' or `expla-
nation' rooted in functional decomposition. In our case,
rithm, unless using debilitatingly long tness evaluation these questions are centred around the suitability of an
tests, would be blind to this pathological behaviour, and evolved circuit for engineering applications. Addressing
could present such a circuit as a solution to the engineer- these questions, such as those regarding long-term dy-
ing specication. Inherently erratic dynamics of this kind
can also interact with the time-dependent exogenous in-
uences mentioned above. If analysis can provide re-
assurance against long-term sporadic misbehaviour, the
circuit is rendered more useful.
namics, is what we mean by `analysis.'
The successful action of a circuit can be considered
as a property of the interface between its inner mecha-
nisms and the external environment (Simon, 1996): the
inner has been adapted so that the behaviour at the in-
In critical applications, complex circuits can be em- terface satises the specication. Observations at the
bedded within an error-recovering framework (Anderson, interface (eg. at input and output connections) during
1985). The error recovery mechanisms themselves can be normal circuit operation may reveal little about the inner
simpler, and perhaps formally veried. For example, if a mechanisms, but instead will largely re
ect the demands
failure condition is detected, the circuit responsible could
be automatically reset to a safe initial state. The more
a circuit's potential failure modes are understood, the
more feasible it becomes to construct a resilient system
containing it.
When encouraging evolution to explore beyond the
scope of conventional design, there can be engineering
of the specication. Analysis therefore requires internal
probing, and/or observation under abnormal conditions,
either internal or external.
There are surprisingly many tactics that can be used
to piece-together an analysis:
benets to some sort of analysis even without a com-
plete understanding of how the circuit works. In the
next section, we discuss what `analysis' is, and how it
can be done. We then apply these techniques to illu-
minate an unconventional evolved circuit, showing that
one can indeed address some analysis questions even if
the circuit is so strange that parts of its inner workings
remain unknown.
2 The Analyst's Toolbox
Analysis of exotic evolved circuits is dierent to that
undertaken as part of orthodox design. At an abstract
level, the appropriate tools are sometimes more akin to
neuroscience than to electronic engineering. It is espe-
cially important to recognise that an evolved system may
not have a clear functional decomposition. A functional
analysis decomposes the system into semi-independent
subsystems with separate roles; the subsystems interact
largely through their functions and independently of the
details of the mechanisms that accomplish those func-
 Probing and abnormal conditions. Abnormal
conditions include: manipulation of the input sig-
nals, varying temperature or power-supply voltage,
replacing parts of the circuit with alternative or non-
functional pieces, and injecting externally generated
signals at internal points. Monitoring an internal
voltage always has some side-eect, often placing a
mostly-reactive load at the probing point. This may
have negligible consequences, but potentially per-
turbs the measurement, or even stops the circuit from
working altogether. Probing internal signals of a cir-
cuit implemented on a VLSI (Very Large-Scale Inte-
gration) chip can require routing extra connections to
reach the external pins of the device, with a danger
of further disrupting the circuit under study.
 Mathematical techniques, including standard
electronics theory, are preferable for their rigour and
generality. If a whole unconventional evolved circuit
is mathematically intractable, there may still be parts
of it which yield.
  Simulation of a circuit allows rapid and extensive was present at the input, and a steady low for the
interactive exploration. As we shall see, circuits other. It was evolved directly on a Xilinx XC6216 Field-
evolved not in simulation, but using real recong-
urable hardware, may rely on detailed hardware prop-
erties not easily modelled in a simulation. Attempts
at simulation can at least help to clarify the extent
of this dependence.
 Synthesis: a circuit can be implemented using al-
ternative electronic devices. For example, a circuit
Programmable Gate Array (FPGA). In simple terms,
this VLSI device consists of an uncommitted array of
components (or `cells'). Switches made from transistors
(multiplexers) control how each component behaves, and
how they are connected to each-other. The settings of
the switches are controlled by the contents of RAM mem-
ory bits distributed throughout the chip; this memory
can be rapidly and repeatedly written to by a host mi-
evolved on a single VLSI recongurable chip, might
then be constructed out of a number of hard-wired
small-scale chips. This provides easy access for prob-
ing and manipulation of `internal' signals, and again
can clarify what aspects of the hardware are impor-
croprocessor, causing any of a vast number of possible
electronic circuits to be physically instantiated in silicon.
Evolution was allowed to exploit the capabilities of the
FPGA as freely as possible. Each candidate design was a
tant to the circuit's operation. setting of the conguration bits for a 10  10 corner of the
array of cells. To test its tness, each design was con-
 Power Consumption for the most common VLSI gured onto a real FPGA (no simulation), half-second
technology (`CMOS'), is related to the rate of change bursts of 1kHz and 10kHz test stimuli were applied to its
of the internal voltages. After removing any power- input, and a tness score was automatically assigned ac-
supply smoothing capacitors, power consumption can cording to the degree to which the output approximated
be monitored with high temporal precision, while the the desired discrimination behaviour. The behaviour of
circuit is exposed to test conditions. the nal circuit (shown in the diagram) is near-optimal
according to this tness measure.
 Electromagnetic emissions, resulting from rapidly The FPGA chip is intended for digital designs: the
changing electrical signals, can sometimes be de- cells can be congured to perform various Boolean func-
tected using a tuned radio receiver. Circuit activity tions. However, being made of real transistors, a cell
within a chip, which might be dicult to monitor is really an analogue component behaving in continuous
directly, can thus be roughly characterised. time. These analogue cells have very high gain (ampli-
 Evolutionary history: The mechanism underlying cation); for most inputs their output rapidly saturates
a task-achieving behaviour may be more apparent either fully high or fully low. Consequently, if certain
soon after its evolutionary origin, rather than after design-rules are followed, then the system's behaviour
evolution has rened it closely to match the speci- can be abstracted to a binary description: a digital logic
cation. It may be possible to identify the innovation system. The basic digital design rule is that recurrent
(perhaps caused by one or more mutations) giving connections | paths through the circuit by which a com-
rise to the behaviour's origin in an ancestor, and to ponent's output can (indirectly) later aect its own input
relate this to the operation of the nal circuit. | must only be allowed to operate at particular discrete
instants. This gives the components time to saturate to
 Population diversity: Sometimes there can be sev- their extreme (digital) states before their inputs change
eral slightly dierent (but related) forms of high- again. In synchronous design, this is done on the tick-
tness circuit in an evolutionary population, which
can help to reveal the basic mechanisms used.
Although unconventional evolved circuits can seem
dauntingly unfamiliar, the analyst is far from powerless.
3 Case Study
We now present an analysis of probably the most bizarre,
mysterious, and unconventional unconstrained evolved
circuit yet reported. The aim is to explore how analysis
may be able to abate some of the worries associated with
employing very unusual evolved circuits in an engineering
application.
The task of the evolved circuit (Fig. 1) was simply
to distinguish between 1kHz and 10kHz audio tones,
producing a steady high output whenever one tone
ing of a global `clock', whereas in asynchronous digital
design more local co-ordination is used.
In our experiment, no constraints were placed on the
circuit's structure or dynamical behaviour. No clock
was provided, so evolution was searching the space of
continuous-time dynamical systems made from the high
gain analogue components normally used as logic gates.
Evolution was free to explore the full repertoire of the
FPGA's possible behaviours, of which digital systems
constitute but a small subset. The components behave
on the timescale of nanoseconds, and one of the original
motivations of the experiment was to see if the dynamics
of the FPGA could be organised to give an orderly be-
haviour on a very dierent timescale: the periods of the
two tones to be distinguished are 1ms and 0.1ms. The
experiment (fully described in (Thompson, 1998)) was
 Out

In

In:

1kHz 10kHz 1kHz

Out:

Figure 1: The circuit for analysis, after pruning. For simplicity, only the connections between the cells are shown,
not their functions, which are also under evolutionary control. Of the signals arriving from its four neighbours, those
used as inputs to a cell's function are marked with small boxes. Each of a cell's four outputs (to the four neighbours)
can be driven either directly from one of the cell's inputs, or by the output of the cell's function.
successful, and the resulting circuit of Fig. 1 is consid-
erably smaller than would be achieved by conventional
methods given the same resources.
Until the present study, one could only speculate as to
the circuit's means of operation, so unusual are its struc-
ture and dynamics. It was clear that continuous time
staying constant until the input next goes to 0.
Part C of the circuit is based around a 2:1 multiplexer.
When Part B is in the static state, the multiplexer selects
the input marked `1' to be its output. This input comes
from the multiplexer's own output via an even number
of inversions, resulting in no net logic inversion but a
played an important role. If the circuit was congured
onto a dierent, but nominally identical, FPGA chip
(not used during evolution), then its performance was
degraded. If the temperature is raised or lowered, then
the circuit still works, but the frequencies between which
it discriminates are raised or lowered. (Digital circuits
usually display unchanged behaviour followed by brittle
failure on approaching their thermal limits.)(Thompson,
1997b) These initial observations warranted a concerted
application of our tactics for unconventional analysis.
time delay of around 9ns. Under certain conditions, it is
possible for such a loop to oscillate (at least transiently),
but the most stable condition is for it to settle down to
a constant logic state. The output of the whole circuit is
taken from this loop. As this Part C loop provides the
only possibility for circuit activity during a high input,
the next step in the analysis was to inspect the output
very carefully while applying test inputs.
We had already observed that the output only ever
changes state (high)low or low)high) on the falling
In Figure 1, the rst analysis step has already been edge of the input waveform (Fig. 1). Although never
taken. Of the 10  10 corner of FPGA cells, only those required to do so during evolution, we discovered that
actively contributing to the behaviour are shown. All the output also responds correctly to the width of sin-
of the cells not shown can be simultaneously `clamped' gle high-going pulses. Figure 3 shows a low)high out-
without aecting the output. To clamp a cell, the cong-
uration sent to the FPGA was changed so that the cell's
function produced a constant output, and this sourced
all four of the cell's output connections. Where a con-
nection is shown passing through a clamped cell, all of
the cell's attributes except for this connection could be
clamped. The cells shaded grey in the diagram were enig-
matic. Although not connected to the rest of the circuit,
if they were clamped then the output became less re-
liable, brie
y changing to spurious states occasionally. time' which either dies away or results in a state change.
One of the aims of further analysis was to learn the in-
uence of these `grey cells'.
put transition occurring after a short pulse; further short
pulses leave the output high, but a single long pulse will
switch it back to the low state. The output assumes the
appropriate level within 200ns after the falling edge of
the input. The circuit does not respond to the width of
low-going pulses, and recognises a high-going pulse de-
limited by as little as 200ns of low input at each end of
the pulse. The output is perfectly steady at logic 1 or 0,
except for a brief oscillation during the 200ns `decision
This is astonishing. During the single high-going
pulse, we know that parts A and B of the circuit are
If the input frequency was gradually changed from
1kHz to 10kHz, the output (low at 1kHz) would be-
gin rapidly to alternate between low and high, spending
more time high as the frequency was increased, even-
tually staying steady high for frequencies near 10kHz.
This binary behaviour of the output voltage suggested
that perhaps part of the system could be understood in
`reset' to a static state within the rst 20ns (the pulse
widths are vastly longer: 500s and 50s correspond to
1kHz and 10kHz). Our observations at the output show
that Part C is also in a static state during the pulse. Yet
somehow, within 200ns of the end of the pulse, the cir-
cuit `knows' how long it was, despite being completely
inactive during it.
digital terms. By temporarily making the assumption This is hard to believe, so we have reinforced this nd-
that all of the FPGA cells were acting as Boolean logic ing through many separate types of observation, and all
gates in the normal way, the FPGA conguration could
be drawn as the logic circuit diagram of Figure 2.
The logic circuit diagram shows several continuous-
time recurrent loops (breaking the digital design rules),
so the system's behaviour is unlikely to be fully captured
by this Boolean level of abstraction. However, it contains
many `canalysing' functions (Kauman, 1993), such as
AND and OR: functions where one input can assume
a value which makes the other inputs irrelevant. It so
happens that whenever our circuit's input is 1, all of the
recurrent loops in Parts A & B are broken by a cascade of
canalysing functions. Within 20ns of the input becoming
1, A and B satisfy the digital design rules, and all of their
agree that the circuit is inactive during the pulse. Power
consumption returns to quiescent levels during the pulse.
Many of the internal signals were (one at a time) routed
to an external pin and monitored. Sometimes this prob-
ing altered (or destroyed) the circuit's behaviour, but
we have observed at least one signal from each recur-
rent loop while the circuit was successfully discriminat-
ing pulse-widths, and there was never activity during the
pulse. We were concerned that perhaps, because of the
way the gates are implemented on the FPGA, it was pos-
sible that glitches (very short-duration pulses) were able
to circulate in the circuit while our logic-analysis pre-
dicts it should be static; possibly these glitches could be
gates deterministically switch to xed, static logic values, so short as to be unobservable when routed to an exter-
 Part B

11
2

O
3 0 U
2 IOB T
P
2 1 U
I
N 1 1 T
2 1
P IOB 4 8
U 2
T 2 3
2 1 2 1 2 1

7
2
6 Part C
Part A
Figure 2: The logic circuit representation. The numbers in hexagons are rough estimates of time delays (in ns),
based on the maximum values specied by the manufacturer. `IOB' refers to the inverting Input/Output Blocks that
interface the edges of the recongurable array to the physical pins of the chip.

In:
(single pulse) < 200us

< 200ns
Out:

Circuit Static
Oscillation Oscillation
Activity: (Canalysed)

Figure 3: The response to a single high-going pulse.

nal pin. Hence, we hand-designed a high-speed `glitch- have successfully modelled this in simulation. The time
catching' circuit (basically a
ip-
op) as a conguration delays on the connections from A to B & C are crucial.
of two FPGA cells. Glitches suciently strong to circu- This explains the in
uence of the `grey cells', which are
late for tens of microseconds could be expected to trigger all immediately adjacent to (or part of) the path of these
the glitch-catcher, but it detected no activity in any of signals. Varying the time delays in the simulation pro-
the recurrent loops during an input pulse. duces a similar result to interfering with the grey cells.
We performed a digital simulation of the circuit (us- Mostly, the loop of part C serves to maintain a constant
ing PSPICE), extensively exploring variations in time- and steady output even while the rest of the circuit oscil-
delays and parasitic capacitances. The simulated circuit lates, but immediately after an input pulse it has subtle
never reproduced the FPGA conguration's successful
behaviour, but did corroborate that the transient as the
circuit enters its static state at the beginning of an in-
put pulse is just a few tens of nanoseconds, in agreement
with our experimental measurements of internal FPGA
signals, and according with the logic analysis. We then
built the circuit out of separate CMOS multiplexer chips,
mimicking the way that the gates are actually imple-
mented by multiplexers on the FPGA, and also mod-
elling the relative time-delays. Again, this circuit did not
work successfully, and | despite our best eorts | never
produced any internal activity during an input pulse.
transient dynamics interacting with those of A & B.
4 Conclusion
We argued that allowing evolution to explore highly un-
conventional circuits can be advantageous. Analysis of
the evolved circuits enhances their utility, but requires
novel approaches. There are numerous tactics that can
be used to piece-together answers to analysis questions
even for seemingly impenetrable circuits. We applied
many of these techniques to the most advanced uncon-
ventional circuit yet produced. We still do not under-
We then went back to nd the rst circuit during the
evolutionary run which responded at all to input fre-
quency. Its behaviour is shown in Figure 4. During a
pulse, the output is steady low. After the pulse, the
output oscillates at one of two dierent frequencies, de-
pending on how long the preceding pulse was. These
stand fully how it works: the core of the timing mech-
anism is a subtle property of the VLSI medium. We
have ruled out most possibilities: circuit activity (in-
cluding glitch-transients and beat-frequencies), metasta-
bility (Marino, 1981), and thermal time-constants from
self-heating. Whatever this small eect, we understand
oscillations are stable and long-lasting. The dierences that it is amplied by alterations in bistable and tran-
are minor between this circuit and its immediate evo- sient dynamics of oscillatory loops, and in detail how this
lutionary predecessor (which displays no discrimination, is used to derive an orderly near-optimal output. Cer-
always oscillating at the lower of the two frequencies).
In fact, there are no dierences at all in the logic circuit
diagrams; the changes do things like alter where a cell's
function takes an unused input from. This lends further
support that circulating glitches are not the key: there
was no change to the implementation of the recurrent
loops.
We see bistable oscillations reminiscent of Fig. 4 at
tain peripheral cells ne-tune particularly sensitive time
delays. On the key question of long-term consistency
of behaviour, we know that the entire FPGA circuitry
is strongly reset to a deterministic stable logic state for
every high half-cycle of the input waveform. Long-term
pathologies are therefore highly unlikely, demonstrating
that analysis eort can sometimes remove worries related
to the use of highly unconventional circuits in practical
applications.
internal nodes of part A of the nal circuit. On being
released from the canalysed stable state, the dierence in Reference
the rst 100ns of oscillatory behaviour in part A is used
by parts B & C to derive a steady output according to
the pulse width. There is some initial state of the part A
dynamics which is determined by the input pulse length.
This initial state does not arise from any circuit activity
in the normal sense: the circuit over the entire array
of cells was stable and static during the pulse. It is a
particular property of the FPGA implementation, as it is
not reproduced in simulation or when the circuit is built
from separate small chips. One guess is that the change
in initial state results from some slow charge/discharge
of an unknown parasitic capacitance during the pulse,
but we cannot yet be sure.
We understand well how parts B & C use A's initial
oscillatory dynamics to derive an orderly output, and
Anderson, T. (Ed.). (1985). Resilient Computing Sys-
tems. Collins, London.
Burton, T. D. (1994). Introduction to Dynamic Systems
Analysis. McGraw-Hill.
Jeeries, D. J., & Deane, J. H. B. (1997). Noise, traps,
and snags in iterating systems. In Proc. 1997 Eur.
Conf. on Circuit theory and Design (ECCTD'97)
Budapest.
Kauman, S. A. (1993). The Origins of Order. Oxford
University Press.
Figure 4: Behaviour of the rst frequency-discriminating ancestor. The upper waveform is the output, and the lower
the input; we see the behaviour immediately after the falling edge of a single input pulse. Left: long pulse, Right:
short pulse. The input to the FPGA is actually delayed  40ns by an intervening buer, relative to the wave seen
here.

Marino, L. R. (1981). General theory of metastable oper-

ation. IEEE Transactions on Computers, C-30 (2),
107{115.
Pomeau, Y., & Manneville, P. (1980). Intermittent tran-
sition to turbulence in dissipative dynamical sys-
tems. Communications in Mathematical Physics,
74, 189{197.
Rutenbar, R. A. (1993). Analog design automation:
Where are we? Where are we going?. In Proc.
IEEE Custom Integrated Circuits Conference, pp.
13.1.1{13.1.8.
Simon, H. A. (1996). The Sciences of the Articial (3rd
edition). MIT Press.
Sipper, M., Mange, D., & Perez-Uribe, A. (Eds.). (1998).
Evolvable Systems: From biology to hardware.
Proc. 2nd Int. Conf., Vol. 1478 of LNCS. Springer-
Verlag.
Thompson, A. (1997a). Evolving inherently fault-
tolerant systems. Proc Instn Mechanical Engrs,
Part I, 211, 365{371.
Thompson, A. (1997b). Temperature in natural and arti-
cial systems. In Husbands, P., & Harvey, I. (Eds.),
Proc. 4th Eur. Conf. on Articial Life (ECAL'97),
pp. 388{397. MIT Press.
Thompson, A. (1998). Hardware Evolution: Automatic
design of electronic circuits in recongurable hard-
ware by articial evolution. Distinguished disser-
tation series. Springer-Verlag.