Evolution of Robustness in an Electronics Design

Adrian Thompson and Paul Layzell

Centre for Computational Neuroscience & Robotics, and
Centre for the Study of Evolution; School of Cognitive & Computing Sciences,
University of Sussex, Brighton BN1 9QH, UK
adrianth, paulla @sussex.ac.uk,
WWW home page: http://www.cogs.susx.ac.uk/users/adrianth/

Abstract. Evolutionary algorithms can design electronic circuits that

conventional design methods cannot, because they can craft an emergent
behaviour without the need for a detailed model of how the behaviours
of the components aect the overall behaviour. However, the absence
of such a model makes the achievement of robustness to variations in
temperature, fabrication, etc., challenging. An experiment is presented
showing that a robust design can be evolved without having to resort to
conventional restrictive design constraints, by testing in dierent condi-
tions during evolution. Surprisingly, the result tentatively suggests that
even within the domain of robust digital design, evolution can explore
beyond the scope of conventional methods.

1 Rationale
Evolutionary algorithms can design electronic circuits that conventional elec-
tronics design methods cannot. Take a basic notion of an evolutionary process
as the repeated action of selection upon individuals replicating with heritable
variations [6]. Then an evolutionary algorithm can proceed by making undirected
stochastic variations; a selection mechanism makes those variations that give rise
to better observed behaviour more likely to persist, to be further embellished by
subsequent rounds of variation and selection.
In contrast, every step in the derivation of a circuit design using conventional
methods is taken with respect to some model of how it will aect nal circuit
behaviour. Even in a stage of iterative modication and testing, the alterations
are chosen with reference to some model of their expected eect.
A circuit's behaviour may be termed emergent [3, 1, 2] if it cannot feasibly
be predicted in detail given only a knowledge of the individual components and
how they are interconnected. An evolutionary algorithm, needing no model to
predict the eects of the component-level variations applied, can craft an emer-
gent circuit behaviour, whereas conventional methods can not. The latter must
constrain the components' interactions, designing within the subset of possible
circuit structures and dynamics for which the necessary models are tractable.
Previous experiments have conrmed that an evolutionary algorithm can de-
rive a circuit beyond the scope of conventional methods in this way [9, 13], but
raised the issue of robustness. Here, robustness is the ability of a circuit to oper-
ate adequately over a specied range of variations in temperature, fabrication,
power-supply voltage, and so on. Notice that these perturbations are often at
the level of physical components. The constraints of conventional design, in al-
lowing models of how the components aect the overall behaviour, also simplify
the achievement of robustness. How much robustness is required is part of the
specication of an application, but some is nearly always necessary.
The main benet of allowing evolution to explore designs without constraint
is that the circuits produced can be dierent to those otherwise obtainable, e.g.
[5]. This novelty means that the circuits might be better in some circumstances,
for example being smaller, more power ecient, or displaying graceful degrada-
tion. This paper reports an experiment showing that evolutionary design can be
induced to produce a robust circuit, without having to impose the customary
constraints on what designs can be explored. The paper nishes with a discussion
of the wider implications of the result.

2 Experiment
The circuit to be evolved has one input and one output. As the input is stimu-
lated with a square wave of either 1kHz or 10kHz, the output is to give a steady
high voltage for one input frequency, and a steady low for the other. This fre-
quency discrimination task was used in the earlier experiments, and was chosen
to be as simple as possible while raising the issues of interest. In particular,
some sort of stable dynamics is needed within the circuit. The tone discrimina-
tion task can also lead the way to more sophisticated cases of pattern recognition
over time, such as word recognition [11]. We seek to explore new means to gener-
ate robust behaviour, which does not necessarily imply that the behaviour could
not be achieved in some other way using conventional methods.
The circuit was evolved to be a conguration of the now-obsolete Xilinx
XC6216 eld-programmable gate array (FPGA). Each candidate design (evolu-
tionary variant) was tested by conguring four XC6216 chips having the con-
ditions shown in Table 1. Evaluations at the target task were then performed
on all four chips simultaneously and independently. The evolutionary tness of
the candidate was taken to be the worst of the four measurements. This multi-
FPGA, multi-condition apparatus, called `The Evolvatron', was fully described
in [12].
Only the conguration of a 10  10 region of the 64  64 array of cells on
the FPGA was subject to evolution. For each chip of Table 1, the 10  10 region
was translated in position by a dierent but constant amount, as shown. Only
nearest-neighbour interconnections between cells were enabled. With reference
to the datasheet [15], this leaves 10 multiplexers (`muxes') to be congured in
each of the 100 cells: four 4:1 neighbour muxes, three 4:1 `X' muxes to select the
inputs to the cell's function unit; and within the function unit, two 4:1 muxes
(Y1 and Y2), and the single 2:1 CS mux selecting either a combinatorial or a
Chip Fabrication Package Interface Temperature PSU Output load Position
1 Seiko PQFP parallel in PC PC's 5V s.m. - (37,30)
2 Yamaha PLCC serial ambient 5V lin. 1k
(32,0)
3 Yamaha PGA serial 60 C 4.75V s.m. - (63,0)
4 Seiko PGA serial 27 C 5.25V s.m. - (37,54)
Table 1. Conditions of the chips used for tness evaluation. `Temperature' is that of
the surface of the chip package. See [14] for the thermal properties of each package.
Power supply units (PSUs) were either switched-mode (s.m.) or linear regulated (lin.).
`Position' gives the (row, column) location of the north-west corner of the 10  10
circuit within the 64  64 array, with (63,0) being the extreme north-west corner of the
array.

sequential output. The RP (register protect) mux, which can be set to prevent
the cell function unit's ip- op from ever changing, was always set to OFF.
In contrast to the earlier experiments, a clock signal was supplied to all of
the FPGAs, and was the clock input to the rising-edge triggered D-type ip- op
inside every cell's function unit. The clock source was a single external crystal
oscillator, nominally 6MHz. Under evolutionary control, via the Y1, Y2 and CS
muxes, each cell's function unit could be synchronous to the clock (function
output taken from the ip- op), asynchronous to the clock ( ip- op output not
selected), or mixed (function output is a combinatorial function of the possibly-
asynchronous inputs as well as of the ip- op's output). Hence, there is no
constraint that evolution must produce a synchronous digital design: the clock
is a stable dynamical resource to be used or ignored as selection deems t, rather
than being an enforced synchronisation signal [10]. No design rules were imposed.
The evolutionary algorithm was a (1+1) Evolution Strategy (ES) [7, 8]. A
mutation operator was dened to select one of the 100 cells at random, then one
of that cell's 10 muxes at random, and then to randomly recongure that mux
to select a dierent input. In the ES, each new variant was produced by applying
the mutation operator three times to the parent.
With a new variant congured to the FPGAs, the inputs were stimulated with
a sequence of 100ms bursts of 1kHz and 10kHz tones. The tones were shued
into a random order for each trial, such that there were an equal number of each
frequency. The single source of the tones was a 1.000000MHz crystal oscillator
module, followed by hardware dividing the frequency by either 100 or 1000.
Separate analogue circuits integrated the output of each chip over each 100ms
tone, giving a value proportional to the average output voltage during that time.
Let the integrator reading of FPGA chip at the end of test tone number be
ct (t=1,2,... T). Let 1 be the set of 1kHz test tones, and 10 the set of 10kHz
c t

i S S

test tones. Then the evaluation of the performance of the conguration on chip
c was calculated as:
 
c = 1  X ct X ct  c (1)
E
2 t2S1
T
i i

t2S10 
P
and the tness was MIN4c=1 ( c ). The term c was used towards the end of
F E P

the run to penalise solutions having spikes or glitches in their output. For each
FPGA, a separate resetable counter made from 74LS devices counted the number
of low)high logic transitions on that FPGA's output during each test tone. If
the number of low)high transitions during tone t was c( ), then a counter
would return the value c ( ) = MIN( c( ) 255 ). Then c was dened as
N t

R t N t ; P

T
c =  1 X MAX( c ( ) 1 0 ) (2)
P w
2 t=1 T
R t ;

where is a weighting factor, initially zero.

w

The ES algorithm proceeded as follows, where is an integer initially 1 but

L

later increased to lengthen the trials:

 Download the initial parent conguration to the FPGAs.
 Measure parent over 50  tones.
F L

REPEAT
 Generate three mutations (the variation operator) and
partially recongure each FPGA with all three.
 y Measure offspring over a total of 24  test tones,
F L

aborting after 8  or 16  tones if offspring parent.

L L F < F

Before the nal group of 8  test tones (if reached), all FPGAs are
L

hardware-reset and completely recongured from scratch with the

new variant.
 IF ( offspring  parent)
F F

The ospring becomes the new parent.

All three mutations are left in place.
ELSE
The parent is left unchanged:
partially recongure the FPGAs to undo the three mutations.
END IF
 IF (15 ospring have been generated without any replacing the parent)
Hardware reset the FPGAs & recong. from scratch with the parent.
Extend the parent's evaluation by a further 10  test tones. L

Wait a further 15 ospring before doing this again.

END IF
END REPEAT

The initial parent was found through random search for a starting point giving
better than trivial performance. Partial reconguration is used above because
some of the FPGAs are recongured via a relatively slow serial interface. The
simple variable-length evaluation scheme copes with measurement noise and the
stochastic behaviours of some candidate congurations. Termination was by a
human observer. The algorithm relies on the existence of pathways of change
that make no immediate dierence to the tness, but that alter the opportunities
for later improvement. See [4] for a discussion of such `neutral networks' in the
context of evolutionary electronics design. One reason for choosing such a simple
algorithm was to simplify the study of the role of neutrality, to be reported
elsewhere.
The initial parent conguration ( =0.43 with =0) was found after test-
F w

ing 75679 randomly generated individuals, using only the tness measuring step
marked y above (with =1), and with a hardware reset and complete recon-
L

guration of the FPGAs between individuals. In the main phase of evolution

( =1, =0), 25000 triple-mutations were accepted out of 861348 attempted (al-
L w

though most evaluations would be aborted early) resulting in a circuit with t-

ness =6.17. Then was increased to 1.4/255 and a further 10 triple-mutations
F w

were accepted out of 265 trials. Then with =2, =1.0, a further 870 out of
L w

32338 triple-mutations were accepted. Finally, with =5, =5.0, a further 3000
L w

out of 131005 mutants were accepted. Each of these latter phases resulted in a
reduction in the number of unwanted spikes in the output, eventually eliminating
them entirely.

3 Results and Preliminary Analysis

The nal circuit works near-perfectly on all of the FPGAs and conditions of
Table 1. The only room for improvement is that sometimes when the input
frequency is changed, the circuit's output responds after several cycles of the
input, rather than after the minimum of a single half-cycle. For steady 1kHz or
10kHz input, in 24-hour tests no unwanted transitions in the output occurred.
As a further test of robustness, the nal circuit was tested on six XC6216 FP-
GAs that were not used during evolution, including samples from both foundries.
Each chip was rst frozen to ' 50 C using freezing spray, then the circuit's
operation was monitored as the chip warmed to room temperature. The same
near-perfect performance as before was seen in all of these conditions, indicating
that the evolved design is usefully robust.
Figure 1 shows the functional part of the circuit. The conguration can be
changed to clamp to constant values those cells and connections not shown,
without aecting the behaviour.
We now wish to discern whether there has been any advantage in allowing
evolution the freedom to explore beyond conventional digital design rules. To see
if any or all of the circuit's behaviour ts into a digital-logic model of operation,
it was simulated in a well-known digital simulator (PSpiceTM ). Working with
noise-free binary signals, for each kind of component there is a model describing
the internal propagation times from the component's inputs to its output. At the
start of the simulation, the propagation times of each component are increased
by a load model, to take account of what loads that output must drive.
The simulation model of the FPGA used components from the 74AC family,
with the propagation times changed to match the approximate gures given in
the datasheet [15]. Where necessary, the simulator defaults for the relationships
between minimum, typical, and maximum timings were used. The other details,
such as the loading model, were unchanged. The recongurable multiplexers
 Output
9,4

8,3 8,4

7,0 7,1 7,2 7,3 7,5

Input 6,0 6,1

5,0 5,3

Fig.1. The functional part of the evolved conguration. The large boxes represent
the cells. A wire shown originating from the perimeter of a cell is the output of that
cell's function unit. Inputs to a cell's function unit are indicated with small squares.
If a cell's function unit output is taken from it's ip- op (synchronous), then a small
triangle is drawn at the bottom of the cell. The congurations of the function units
are not shown. The cells are labelled with their (row, column) co-ordinates within the
10  10 evolved array.

were replaced by inverters or buers hard-wired according to the nal evolved

conguration. The resulting model is shown in Figures 2 and 3, which also show
all the details not visible in Figure 1.
The circuit worked rst time in simulation, without any need to ne-tune the
component timings. This means it does not rely on any aspects of the chips not
captured in the simulation. In particular, it does not rely on any analogue eects,
in contrast to an earlier experiment in which robustness was not an objective
[13]. On a Pentium 233MHz processor with 32MB of memory, the simulation
ran ' 62000 times slower than real time.
Using the schematic and simulation traces, the circuit appears to operate as
shown in Figure 4. First, the input is `retimed' (sampled) to the 6MHz clock
I

in cell (5,0). When the retimed input is high, cell (7,0) toggles at the clock
R

frequency. When is low, this oscillation stops. Since the sampling in cell (5,0)
R

delays by up to (but less than) a clock period with respect to , the number
R I

of times the oscillator toggles is completely determined by how long the raw
input is high, and hence on the input frequency. For some input frequencies,
the oscillator toggles an odd number of times, so nishes in a dierent state,
whereas for others it does not. This is the heart of the timing mechanism; cell
(7,2) simply holds the nal value of the previous oscillation while the next one
is going on, and this is the output of the circuit.
The core mechanism described so far uses only three cells, but would, at
best, produce a constant output for one input frequency and an output toggling
every cycle at the other input frequency. The function of all of the other cells
in the circuit is to correct this, by delaying the retimed input. The delay on
 8,0 8,1 8,2
RSr8c2

OPr7c0 REr8c0 REr8c0 REr8c1 REr8c1 REr8c2

Fig. 2. The left half of the simulation model.

7,0 7,1 7,2

OPr6c1 OPr7c3 REr7c1
Hi Hi Hi

HI

HI

HI
1
OPr7c0 1
OPr7c1 1
OPr7c2
D S Q D S Q D S Q
0
R Q'
0
R Q'
RSr8c2 0
R Q'
Clk Clk Clk
Hi Hi OPr6c1 REr7c1 Hi

HI

HI

HI
6,0 6,1
OPr5c0 OPr7c1
Hi

HI
Edge OPr6c0 OPr6c1
OPr6c0 1
D S Q
1
0
0
R Q'
Clk
RWr6c1
Hi

HI
Clk
5,0 OPr7c1 RWr6c1 CLK

Lo Clock:
LO
ONTIME = 65.6ns
Hi
HI

Input OFFTIME = 103.1ns

1
OPr5c0
D S Q
0
R Q'
Clk
Hi Lo Edge
HI

LO
 9,4
OPr8c3
Hi

HI
OPr9c4
REr8c2 1
D S Q
0
R Q'
Clk
Hi

HI
8,3 8,4
Fig. 3. The right half of the simulation model.

OPr7c5 OPr9c4
Hi Hi

HI

HI
OPr8c3 OPr8c4
OPr5c3 1
D S Q
1
D S Q
0 0
R Q' R Q'
Clk Clk
Hi Hi

HI

HI
7,3 7,5
OPr7c5 OPr8c4
HI Hi Hi

HI
OPr7c2 OPr7c3 OPr7c5
1 1
D S Q D S Q
0 0
R Q'
Hi R Q'
HI
Clk Clk
Hi Hi
HI

HI
5,3
Lo
LO

REr7c1 OPr5c3
1

0
Hi
HI
 t R control

delay edges
I sample R tR toggle at sample
input output
to clock clock freq. and hold
tF
go/stop hold/sample

Fig. 4. An outline of the mechanism.

falling edges is constant, but for rising edges is variable, and is a function of the
present output of the circuit. This is arranged so that if the input is 1kHz while
the output is high, or if the input is 10kHz while the output is low, then the
oscillator will have time for an odd number of toggles in a high half-cycle of the
input, and the output will change state. If the input is 1kHz while the output
is low, or 10kHz while it is high, then the oscillator toggles an even number of
times and the output is constant. The implementation of the variable delay is
not yet understood.
The simulator is able to report whenever there is a ip- op data-to-clock
setup time violation: that is, when the input to a ip- op changes so soon before
the clock edge arrives that the resulting output is uncertain. Setup time viola-
tions are only ever reported in the cell (5,0) which retimes the input: the other
nine ip- ops in the evolved design operate in a reliable clocked fashion.

Fig. 5. Typical cell outputs during simulation.

4 Discussion
The observed robustness, the success of the simulation, and the absence of setup
time violations, together provide good evidence that the circuit operates as a
well-behaved digital design. However, the simulation waveforms (Figure 5) show
many spikes and glitches of various durations that are determined by analogue
time delays through the circuit. Such dynamics are avoided in digital design as
leading to unreliable or non-robust operation. Presumably the present design
evolved to be insensitive to these, perhaps completely ignoring them.
Together, these ndings give a fascinating result. The circuit has evolved to
achieve robustness by nding a well behaved, clocked, digital mode of operation.
However, it is not at all clear what digital design rules could have been formu-
lated in advance, that would have guaranteed robust digital operation and yet
permitted this evolved circuit.
A surprising hypothesis is suggested: Even within robust digital design, un-
constrained evolution can produce circuits beyond the scope of conventional
design rules. Previously it had been assumed that the domain of robust digital
design was fully covered by conventional design rules, and that novel territory
for evolutionary exploration must lie in other domains [13]. Given the undoubted
utility of robust digital designs, it is an exciting possibility that evolution could
explore novel regions of design space, containing circuits that may be better for
some applications.
More work is needed to investigate this bold idea, and the results presented
here do not yet conclusively prove it. Future work should also look at the char-
acteristics of the evolutionary intermediates that aided the discovery of the nal
solution. Finally, it should be noted that in nature there are numerous strategies
for robustness other than clocked digital design [10], and these may be of interest
to evolutionary electronics when using analogue components.

Acknowledgement
to Phil Husbands, Inman Harvey and Ricardo Salem Zebulum. This work was primar-
ily funded by EPSRC, and we also thank the following for support: Xilinx, British
Telecommunications, Hewlett-Packard, Zetex, and Motorola.

References
1. P. Cariani. Emergence and articial life. In C. G. Langton, C. Taylor, J. D. Farmer,
and S. Rasmussen, Eds, Articial Life II, pp. 775{797. Addison Wesley Longman,
1992.
2. J. P. Crutcheld and M. Mitchell. The evolution of emergent computation. Proc.
Nat. Acad. Sci. USA, 92(23):10742{10746, 1995.
3. S. Forrest. Emergent computation | self-organizing, collective, and cooperative
phenomena in natural and articial computing networks. Physica D, 42(1-3):1{11,
1990.
 4. I. Harvey and A. Thompson. Through the labyrinth evolution nds a way: A silicon
ridge. In T. Higuchi and M. Iwata, Eds, Proc. 1st Int. Conf. on Evolvable Systems
(ICES`96), vol. 1259 of LNCS, pp. 406{422. Springer-Verlag, 1997.
5. J. R. Koza, F. H. Bennett III, M. A. Keane, et al. Searching for the impossible
using genetic programming. In W. Banzhaf, J. Daida, A. E. Eiben, et al., Eds, Proc.
Genetic and Evolutionary Computation conference (GECCO-99), pp. 1083{1091.
Morgan Kaufmann, 1999.
6. J. Maynard Smith. Evolutionary Genetics. Oxford University Press, 1989.
7. I. Rechenberg. Cybernetic solution path of an experimental problem. Royal Air-
craft Establishment, Library Translation 1122, 1965. Reprinted in `Evolutionary
Computation | The fossil record', D. B. Fogel, Ed., chap. 8, pp. 297-309, IEEE
Press 1998.
8. H.-P. Schwefel and G. Rudolph. Contemporary evolution strategies. In F. Moran,
A. Moreno, J. J. Merelo, and P. Chacon, Eds, Advances in Articial Life: Proc.
3rd Eur. Conf. on Articial Life, vol. 929 of LNAI, pp. 893{907. Springer-Verlag,
1995.
9. A. Thompson. Silicon evolution. In J. R. Koza, D. E. Goldberg, D. B. Fogel, and
R. L. Riolo, Eds, Genetic Programming 1996: Proc. 1st Annual Conf. (GP96), pp.
444{452. Cambridge, MA: MIT Press, 1996.
10. A. Thompson. Temperature in natural and articial systems. In P. Husbands and
I. Harvey, Eds, Proc. 4th Eur. Conf. on Articial Life (ECAL'97), pp. 388{397.
MIT Press, 1997.
11. A. Thompson. Exploring beyond the scope of human design: Automatic generation
of FPGA congurations through articial evolution (Keynote). In Proc. 8th Annual
Advanced PLD & FPGA Conference, pp. 5{8. Miller Freeman, 12th May 1998.
Ascot, UK.
12. A. Thompson. On the automatic design of robust electronics through articial
evolution. In M. Sipper, D. Mange, and A. Perez-Uribe, Eds, Proc. 2nd Int. Conf.
on Evolvable Systems (ICES'98), vol. 1478 of LNCS, pp. 13{24. Springer-Verlag,
1998.
13. A. Thompson, P. Layzell, and R. S. Zebulum. Explorations in design space: Uncon-
ventional electronics design through articial evolution. IEEE Trans. Evol. Comp.,
3(3):167{196, 1999.
14. Xilinx. Packages and thermal characteristics V1.2, August 1996. In The Pro-
grammable Logic Data Book, chapter 10. Xilinx, Inc., 1996.
15. Xilinx. XC6200 eld programmable gate arrays. Data Sheet, Xilinx, Inc., April
1997. Version 1.10.