Agile Artificial Intelligence in Pharo: Implementing Neural Networks, Genetic Algorithms, and Neuroevolution Alexandre Bergel

Agile Artificial Intelligence in Pharo:
Implementing Neural Networks, Genetic

Algorithms, and Neuroevolution
Alexandre Bergel
Visit to download the full and correct content document:
https://ebookmeta.com/product/agile-artificial-intelligence-in-pharo-implementing-neur
al-networks-genetic-algorithms-and-neuroevolution-alexandre-bergel/
More products digital (pdf, epub, mobi) instant
download maybe you interests ...
Agile Visualization with Pharo: Crafting Interactive

Visual Support Using Roassal Alexandre Bergel
https://ebookmeta.com/product/agile-visualization-with-pharo-
crafting-interactive-visual-support-using-roassal-alexandre-
bergel/
Artificial Neural Networks and Machine Learning ICANN

2021 30th International Conference on Artificial Neural
Networks Bratislava Slovakia I Igor Farkaš (Editor)
https://ebookmeta.com/product/artificial-neural-networks-and-
machine-learning-icann-2021-30th-international-conference-on-
artificial-neural-networks-bratislava-slovakia-i-igor-farkas-
editor/
Using Artificial Neural Networks for Timeseries

Smoothing and Forecasting Case Studies in Economics
Studies in Computational Intelligence 979 Jaromír
Vrbka
https://ebookmeta.com/product/using-artificial-neural-networks-
for-timeseries-smoothing-and-forecasting-case-studies-in-
economics-studies-in-computational-intelligence-979-jaromir-
vrbka/
Artificial Neural Networks Methods in Molecular Biology

2190 Hugh Cartwright (Editor)
https://ebookmeta.com/product/artificial-neural-networks-methods-
in-molecular-biology-2190-hugh-cartwright-editor/
2021 30th International Conference on Artificial Neural
Networks Bratislava Slovakia II Lecture Notes in
Computer Science 12892 Igor Farkaš (Editor)
machine-learning-icann-2021-30th-international-conference-on-
artificial-neural-networks-bratislava-slovakia-ii-lecture-notes-
in-computer-science-12892-igor-farkas-edit/

2022 31st International Conference on Artificial Neural
Networks Bristol UK September 6 9 2022 Proceedings Part
II Elias Pimenidis
machine-learning-icann-2022-31st-international-conference-on-
artificial-neural-networks-bristol-uk-
september-6-9-2022-proceedings-part-ii-elias-pimenidis/

I Elias Pimenidis
september-6-9-2022-proceedings-part-i-elias-pimenidis/

III Elias Pimenidis
september-6-9-2022-proceedings-part-iii-elias-pimenidis/
Artificial Neural Networks with Java: Tools for

Building Neural Network Applications, 2nd Edition Igor
Livshin
https://ebookmeta.com/product/artificial-neural-networks-with-
java-tools-for-building-neural-network-applications-2nd-edition-
igor-livshin/
Agile Artificial
Intelligence in
Pharo
Implementing Neural Networks,
Genetic Algorithms, and Neuroevolution
—
Alexandre Bergel
www.allitebooks.com
Agile Artificial Intelligence
in Pharo
Implementing Neural Networks,
Genetic Algorithms,
and Neuroevolution
Alexandre Bergel
www.allitebooks.com
Agile Artificial Intelligence in Pharo: Implementing Neural Networks, Genetic
Algorithms, and Neuroevolution
Alexandre Bergel
Santiago, Chile
ISBN-13 (pbk): 978-1-4842-5383-0 ISBN-13 (electronic): 978-1-4842-5384-7

https://doi.org/10.1007/978-1-4842-5384-7
Copyright © 2020 by Alexandre Bergel

This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of the
material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation,
broadcasting, reproduction on microfilms or in any other physical way, and transmission or information
storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now
known or hereafter developed.
Trademarked names, logos, and images may appear in this book. Rather than use a trademark symbol with
every occurrence of a trademarked name, logo, or image we use the names, logos, and images only in an
editorial fashion and to the benefit of the trademark owner, with no intention of infringement of the
trademark.
The use in this publication of trade names, trademarks, service marks, and similar terms, even if they are not
identified as such, is not to be taken as an expression of opinion as to whether or not they are subject to
proprietary rights.
While the advice and information in this book are believed to be true and accurate at the date of publication,
neither the authors nor the editors nor the publisher can accept any legal responsibility for any errors or
omissions that may be made. The publisher makes no warranty, express or implied, with respect to the
material contained herein.
Managing Director, Apress Media LLC: Welmoed Spahr
Acquisitions Editor: Steve Anglin
Development Editor: Matthew Moodie
Coordinating Editor: Mark Powers
Cover designed by eStudioCalamar
Cover image designed by Freepik (www.freepik.com)
Distributed to the book trade worldwide by Apress Media, LLC, 1 New York Plaza, New York, NY 10004,
U.S.A. Phone 1-800-SPRINGER, fax (201) 348-4505, e-mail orders-ny@springer-sbm.com, or visit www.
springeronline.com. Apress Media, LLC is a California LLC and the sole member (owner) is Springer Science
+ Business Media Finance Inc (SSBM Finance Inc). SSBM Finance Inc is a Delaware corporation.
For information on translations, please e-mail editorial@apress.com; for reprint, paperback, or audio rights,
please email bookpermissions@springernature.com.
Apress titles may be purchased in bulk for academic, corporate, or promotional use. eBook versions and
licenses are also available for most titles. For more information, reference our Print and eBook Bulk Sales
web page at http://www.apress.com/bulk-sales.
Any source code or other supplementary material referenced by the author in this book is available to
readers on GitHub via the book’s product page, located at www.apress.com/9781484253830. For more
detailed information, please visit http://www.apress.com/source-code.
Printed on acid-free paper
www.allitebooks.com
Table of Contents
About the Author�� xi
About the Technical Reviewer�� xiii

Acknowledgments��xv
Introduction��xvii
Part I: Neural Networks�� 1

Chapter 1: The Perceptron Model�� 3
1.1 Perceptron as a Kind of Neuron�� 3
1.2 Implementing the Perceptron�� 5
1.3 Testing the Code�� 10
1.4 Formulating Logical Expressions�� 13
1.5 Handling Errors�� 15
1.6 C
ombining Perceptrons�� 16
1.7 T raining a Perceptron�� 19
1.8 D
rawing Graphs�� 24
1.9 Predicting and 2D Points�� 25
1.10 Measuring the Precision�� 31
1.11 H
istorical Perspective�� 33
1.12 E xercises�� 34
1.13 What Have We Seen in This Chapter?�� 34
1.14 Further Reading About Pharo�� 35
iii
www.allitebooks.com
Table of Contents
Chapter 2: The Artificial Neuron�� 37

2.1 Limit of the Perceptron�� 37
2.2 Activation Function�� 38
2.3 The Sigmoid Neuron�� 40
2.4 Implementing the Activation Functions�� 41
2.5 Extending the Neuron with the Activation Functions�� 43
2.6 Adapting the Existing Tests�� 46
2.7 Testing the Sigmoid Neuron�� 46
2.8 Slower to Learn�� 48
Chapter 3: Neural Networks�� 53

3.1 General Architecture�� 53
3.2 Neural Layer�� 54
3.3 Modeling a Neural Network�� 59
3.4 Backpropagation�� 62
3.4.1 Step 1: Forward Feeding�� 63
3.4.2 Step 2: Error Backward Propagation�� 64
3.4.3 Step 3: Updating Neuron Parameters�� 66
Chapter 4: Theory on Learning�� 69

4.1 Loss Function�� 69
4.2 Gradient Descent�� 73
4.3 Parameter Update�� 76
4.4 Gradient Descent in Our Implementation�� 78
4.5 Stochastic Gradient Descent�� 79
4.6 The Derivative of the Sigmoid Function�� 87
4.8 F urther Reading�� 88
iv
www.allitebooks.com
Table of Contents
Chapter 5: Data Classification�� 89

5.1 Training a Network�� 89
5.2 Neural Network as a Hashmap�� 92
5.3 Visualizing the Error and the Topology�� 93
5.4 Contradictory Data�� 98
5.5 Classifying Data and One-Hot Encoding�� 99
5.6 The Iris Dataset�� 100
5.7 Training a Network with the Iris Dataset�� 102
5.8 The Effect of the Learning Curve�� 103
5.9 Testing and Validation�� 106
5.10 Normalization�� 109
5.11 Integrating Normalization into the NNetwork Class�� 114
Chapter 6: A Matrix Library�� 117

6.1 Matrix Operations in C�� 117
6.2 The Matrix Class�� 119
6.3 Creating the Unit Test�� 122
6.4 Accessing and Modifying the Content of a Matrix�� 123
6.5 Summing Matrices�� 125
6.6 Printing a Matrix�� 127
6.7 E xpressing Vectors�� 128
6.8 F actors�� 129
6.9 Dividing a Matrix by a Factor�� 131
6.10 M
atrix Product�� 132
6.11 M
atrix Subtraction�� 133
6.12 Filling the Matrix with Random Numbers�� 135
6.13 Summing the Matrix Values�� 135
6.14 Transposing a Matrix�� 136
6.15 Example�� 137
v
Table of Contents
Chapter 7: Matrix-Based Neural Networks�� 141

7.1 Defining a Matrix-Based Layer�� 141
7.2 Defining a Matrix-Based Neural Network�� 145
7.3 Visualizing the Results�� 149
7.4 Iris Flower Dataset�� 150
Part II: Genetic Algorithms�� 153

Chapter 8: Genetic Algorithms�� 155
8.1 Algorithms Inspired from Natural Evolution�� 155
8.2 Example of a Genetic Algorithm�� 156
8.3 Relevant Vocabulary�� 157
8.4 Modeling Individuals�� 158
8.5 Crossover Genetic Operations�� 165
8.6 Mutation Genetic Operations�� 169
8.7 P arent Selection�� 173
8.8 E volution Monitoring�� 179
8.9 T he Genetic Algorithm Engine�� 181
8.10 T erminating the Algorithm�� 188
8.11 T esting the Algorithm�� 190
8.12 V isualizing Population Evolution�� 191
Chapter 9: Genetic Algorithms in Action�� 195

9.1 Fundamental Theorem of Arithmetic�� 195
9.2 The Knapsack Problem�� 197
9.2.1 The Unbounded Knapsack Problem Variant�� 198
9.2.2 The 0-1 Knapsack Problem Variant�� 200
9.2.3 Coding and Encoding�� 202
9.3 Meeting Room Scheduling Problem�� 202
vi
Table of Contents
9.4 Mini Sodoku�� 204

Chapter 10: The Traveling Salesman Problem�� 209

10.1 Illustration of the Problem�� 209
10.2 Relevance of the Traveling Salesman Problem�� 210
10.3 Naive Approach�� 211
10.4 Adequate Genetic Operations�� 218
10.5 The Swap Mutation Operation�� 218
10.6 The Ordered Crossover Operation�� 219
10.7 Revisiting Our Large Example�� 222
Chapter 11: Exiting a Maze�� 225

11.1 Encoding the Robot’s Behavior�� 225
11.2 Robot Definition�� 225
11.3 Map Definition�� 227
11.4 Example�� 231
Chapter 12: Building Zoomorphic Creatures�� 237

12.1 Modeling Join Points�� 238
12.2 Modeling Platforms�� 242
12.3 Defining Muscles�� 244
12.4 Generating Muscles�� 249
12.5 Defining
the Creature�� 253
12.6 Creating Creatures�� 255
12.6.1 Serialization
and Materialization of a Creature�� 257
12.6.2 Accessors and Utility Methods�� 258
12.7 Defining the World�� 259
12.8 Cold Run�� 262
vii
Table of Contents
Chapter 13: Evolving Zoomorphic Creatures�� 265

13.1 Interrupting a Process�� 265
13.2 Monitoring the Execution Time�� 266
13.3 The Competing Conventions Problem�� 267
13.4 The Constrained Crossover Operation�� 268
13.5 Moving Forward�� 269
13.6 Serializing
the Muscle Attributes�� 273
13.7 Passing Obstacles�� 274
13.8 Climbing Stairs�� 277
Part III: Neuroevolution�� 281

Chapter 14: Neuroevolution�� 283
14.1 Supervised, Unsupervised Learning, and Reinforcement Learning�� 283
14.2 Neuroevolution�� 284
14.3 Two Neuroevolution Techniques�� 285
14.4 The NeuroGenetic Approach�� 285
14.5 Extending the Neural Network�� 286
14.6 NeuroGenetic
by Example�� 287
14.7 The Iris Dataset�� 292
14.8 Further
Reading About NeuroGenetic�� 294
Chapter 15: Neuroevolution with NEAT�� 295

15.1 Vocabulary�� 295
15.2 The Node Class�� 297
15.3 Different Kinds of Nodes�� 298
15.4 Connections�� 304
15.5 The Individual Class�� 306
15.6 Species�� 313
15.7 Speciation�� 315
viii
Table of Contents
15.8 The Crossover Operation�� 318

15.9 Abstract Definition of Mutation�� 320
15.10 Structural Mutation Operations�� 321
15.10.1 Adding a Connection�� 322
15.10.2 Adding a Node�� 325
15.11 Non-Structural Mutation Operation�� 326
15.12 Logging�� 326
15.13 NEAT�� 330
15.14 Visualization�� 340
15.15 The XOR Example�� 346
15.16 The Iris Example�� 351
Chapter 16: The MiniMario Video Game�� 355

16.1 Character Definition�� 356
16.2 Modeling Mario�� 360
16.3 Modeling an Artificial Mario Player�� 360
16.4 Modeling Monsters�� 361
16.5 Modeling
the MiniMario World�� 363
16.6 Building
the Game’s Visuals�� 368
16.7 Running MiniMario�� 372
16.8 NEAT
and MiniMario�� 373
Afterword: Last Words�� 377
Index�� 379
ix
About the Author
Alexandre Bergel, Ph.D., is a professor (associate) at the
Department of Computer Science (DCC), at the University of
Chile and is a member of the Intelligent Software Construction
laboratory (ISCLab). His research interests include software
engineering, software performance, software visualization,
programming environment, and machine learning. He is
interested in improving the way we build and maintain
software. His current hypotheses are validated using rigorous
empirical methodologies. To make his research artifacts useful
not only to stack papers, he co-founded Object Profile.
xi
About the Technical Reviewer
Jason Whitehorn is an experienced entrepreneur and
software developer and has helped many companies
automate and enhance their business solutions through data
synchronization, SaaS architecture, and machine learning.
Jason obtained his Bachelor of Science in Computer Science
from Arkansas State University, but he traces his passion
for development back many years before then, having first
taught himself to program BASIC on his family’s computer
while in middle school.
When he’s not mentoring and helping his team at work,
writing, or pursuing one of his many side-projects, Jason enjoys spending time with his
wife and four children and living in the Tulsa, Oklahoma region. More information about
Jason can be found on his website: https://jason.whitehorn.us.
xiii
Acknowledgments
Agile Artificial Intelligence in Pharo is the result of a long and collective effort made by
the ESUG community and beyond. The writing and the necessary research of the book
was sponsored by Lam Research and ESUG. Thank you! You made the book happen!
Many people helped get the book in shape. In no particular order, we are deeply
grateful to CH Huang, Chris Thorgrimsson, Milton Mamani, Jhonny Cerezo, Oleks
Zaytsev, Stéphane Ducasse, Torsten Bergmann, Serge Stinckwich, Alexandre Rousseau,
Sean P. DeNigris, Julián Grigera, Cesar Rabak, Yvan Guemkam, John Borden, Sudhakar
Krishnamachari, Leandro Caniglia, mldavis99, darth-cheney, Andy S., Jon Paynter,
Esteban Lorenzano, Juan-Pablo Silva, Francisco Ary Martins, Norbert Fortelny, forty,
Sebastián Zapata, and Renato Cerro.
Publishing a book involves some legal aspects that need to be carefully considered.
We thank Fernanda Carvajal Gezan and Rosa Leal, from the University of Chile.
We also thank the Apress team for trusting in the project and thank Jason Whitehorn,
who tech-reviewed the book.
xv
Introduction
Artificial Intelligence (AI) is radically changing the way we use computers to solve
problems. For example, by exploiting previous experience, which may be expressed in
terms of examples, a machine can identify patterns in a given situation and try to identify
the same patterns in a slightly different situation. This is essentially the way AI is used
nowadays. The field of AI is moving quickly, and unfortunately, it is often difficult to
understand.
The objective of the Agile Artificial Intelligence in Pharo book is to provide a practical
foundation for a set of expressive artificial intelligence algorithms using the Pharo
programming language. The book makes two large contributions over existing related
books. The first contribution is to bring agility in the way some techniques related to
artificial intelligence are designed, implemented, and evaluated. The book provides
material in an incremental fashion, beginning with a little perception and ending with a
full implementation of two algorithms for neuroevolution.
The second contribution is about making these techniques accessible to
programmers by detailing their implementation without overwhelming the reader
with mathematical material. There is often a significant gap between reading
mathematical formulas and producing executable source code from those formulas,
unfortunately. The book is meant to be accessible to a large audience by focusing on
executable source code.
Overall, this book details and illustrates some easy-to-use recipes to solve actual
problems. Furthermore, it highlights some technical details of these recipes using the
Pharo programming language. Agile Artificial Intelligence in Pharo is not a book about
how to use an existing API provided by external libraries. Instead, this book guides you to
build your own API for artificial intelligence.
Book Overview
Agile Artificial Intelligence in Pharo is divided into three parts, each targeting a specific
topic within the field of artificial intelligence—neural networks, genetic algorithms, and
neuroevolution.
xvii
Introduction
The first part of the book is about neural networks. A neural network is a
computational metaphor simulating the interaction occurring between biological
neurons. The chapter begins with the implementation of a single neuron and shows its
limitations in terms of what it can achieve. Neural networks are then presented to solve
more complex problems. Various examples involving relatively simple data classification
tasks are presented.
The second part of the book covers genetic algorithms (GAs). The GA is a
computational metaphor simulating the evolution occurring in biological species. GAs
provide a way to solve problems without knowing the structure and shape of the solution
in advance. GAs simulate the way biological species evolve over time. For two candidate
solutions, as soon as the machine is able to say which one is closer to the solution, then
GAs may be considered to solve the problem. Numerous examples are provided in this
second part of the book, including an implementation of zoomorphic creatures, which
is a simulation of artificial life. We define a zoomorphic creature as an artificial organism
able to evolve in order to move itself through obstacles.
The third part of the book covers the field of neuroevolution, which is a combination
of genetic algorithms and neural networks. The evolution of neural networks is called
neuroevolution. Instead of training a neural network, as in classical deep learning (Part
1 of the book), neuroevolution begins with extremely simple networks and incrementally
adds complexity to them. Evolution makes those networks able to solve particular tasks.
This third part uses a Mario Bros-like game, which is used to build an artificial player
using neuroevolution.
I nstalling Pharo
Pharo works on the three common platforms, Mac OSX, Windows, and Linux. The web
page at https://pharo.org/download gives a very detailed instruction set and some
links to download Pharo. Pharo is easy to install. Just a matter of a few clicks.
The content of the book is known to work up until Pharo 9. The code provided in
the book does not heavily rely on the Pharo runtime. So the code provided in this book
should be easy to adapt to future versions of Pharo or to another dialect of Smalltalk.
xviii
Introduction
Accompanying Source Code

Agile Artificial Intelligence in Pharo is a book about programming. It provides and
details a sizable amount of source code. Most of the code in the book is self-contained.
This means that no external libraries are used besides the Pharo core and the Roassal
visualization engine. Roassal is used to visually explore data and build a user interface.
Readers may prefer to transcribe the code into Pharo or use our dedicated Git repository
at https://github.com/Apress/agile-ai-in-pharo.
A script that begins with ellipses (i.e., ...) means that you need to append the script
to the last one seen before.
The code provided in this book is known to run on Pharo 8 and 9. To load the code,
you simply need to open a playground and execute the following code:
Metacello new
baseline: 'AgileArtificialIntelligence';
repository: 'github://Apress/agile-ai-in-pharo/src';
load.
The GitHub repository contains the scripts folder, which contains all the scripts
and code snippets provided in the book.
The book focuses on Pharo; however, at the cost of a few small adaptations, all the
provided code will run on an alternative Smalltalk implementation (e.g., VisualWorks).
Who Should Read This Book?

This book is designed to be read by a wide audience of programmers. As such, there is no
need to have prior knowledge of neural networks, genetic algorithms, or neuroevolution.
There is even no need to have a strong mathematical background. We made sure that
there is no such prerequisite for most of the chapters. Some chapters require mild
mathematical knowledge. However, these chapters are self-contained and skipping them
will not negatively affect your overall understanding.
The book exposes some sophisticated AI techniques through the lenses of Pharo.
Readers will acquire the theoretical and practical tools to be used in Pharo. Note
that people willing to learn Pharo through AI are encouraged to complement it with
additional sources of information.
xix
Introduction
The book is not made for people who want to learn about AI techniques without
heavily investing in a programming activity. Instead, Agile Artificial Intelligence in Pharo
is made for programmers who are either familiar with Pharo or are willing to be.
The Pharo Experience

The code provided in this book uses the Pharo programming environment.
Programming in Pharo is a fantastic and emotional experience. Literally. Pharo gives
meaning to Agile programming that cannot be experienced in another programming
language, or at least, not to the same degree. We will try to convey this wonderful
experience to the readers.
Pharo has a very simple syntax, which means that code should be understandable as
soon as you have some programming knowledge. Chapter 2 briefly introduces the Pharo
programming language and its environment in case you want to be familiar with it.
Why did we pick Pharo for this book? Pharo is a beautiful programming language
with a sophisticated environment. It also provides a new way of communication
between a human and a machine. By offering a live programming environment
and a language with minimal syntax, programmers may express their thoughts
in an incremental and open fashion. Although a number of similar programming
environments exist (e.g., Scratch and Squeak), Pharo is designed to be used in an
industrial software development setting.
Pharo syntax is concise, simple, unambiguous, and requires very little explanation
to be fully understood. If you do not know Pharo, we encourage you to become familiar
with the basics of its syntax and programming environment. Chapter 2 should help
in that respect. The debugger, inspector, and playground are unrivaled compared to
other programming languages and environments. Using these tools really brings an
unmatchable feeling when programming.
xx
Introduction
A
dditional Reading
Agile Artificial Intelligence in Pharo provides a gentle introduction to Pharo in Chapter 2.
However, the presentation of Pharo is shallow and not complete. Readers who do
not know Pharo and but have experience in programming may find the chapter to be
enough. Readers who want to deepen their knowledge may want to look for additional
sources of learning. Here are some good readings on Pharo:
• http://pharo.org is the official website about Pharo.
• https://mooc.pharo.org is probably the most popular way to learn

Pharo. It provides many short videos covering various aspects of
Pharo.
• http://books.pharo.org offers many valuable books and booklets

on Pharo.
• http://agilevisualization.com describes the Roassal visualization

engine, which also contains a gentle introduction to Pharo.
Visualization is omnipresent in Agile Artificial Intelligence in Pharo. Roassal is used

in many chapters and the reader is welcome to read Agile Visualization to become
familiar with this wonderful visualization toolkit.
xxi
PART I
Neural Networks
CHAPTER 1
The Perceptron Model

All major animal groups have brains made of neurons. A neuron is a specialized cell that
transmits electrochemical stimulation using an axon to other neurons. A neuron receives
this nerve impulse via a dendrite. Since the early age of computers, scientists have tried
to produce a computational model of a neuron. The perceptron was one of the first
models to mimic the behavior of a neuron.
This chapter plays two essential roles in the book. First, it presents the perceptron,
a fundamental model on which neural networks are based. Second, it also provides a
gentle introduction to the Pharo programming language. The chapter builds a simple
perceptron model in Pharo.
1.1 Perceptron as a Kind of Neuron

A perceptron is a kind of artificial neuron that models the behavior of a biological
neuron. A perceptron is a machine that produces an output for a provided set of input
values. Figure 1-1 gives a visual representation of a perceptron.
A perceptron accepts one, two, or more numerical values as inputs. It produces
a numerical value as output (the result of a simple equation that we will see shortly).
A perceptron operates on numbers, which means that the inputs and the output are
numerical values (e.g., integers or floating point values).
Figure 1-1. Representing the perceptron
3
© Alexandre Bergel2020
A. Bergel, Agile Artificial Intelligence in Pharo, https://doi.org/10.1007/978-1-4842-5384-7_1
Chapter 1 The Perceptron Model
Figure 1-1 depicts a perceptron. A perceptron is usually represented as a circle with

some inputs and one output. Inputs are represented as incoming arrows located on
the left of the central circle and the output as an outgoing arrow on the right of it. The
perceptron in Figure 1-1 has three inputs, noted x1, x2, and x3.
Not all inputs have the same importance for the perceptron. For example, an input
may be more important than other inputs. Relevance of an input is expressed using
a weight (also a numerical value) associated with that input. In Figure 1-1, the input
x1 is associated with the weight w1, x2 with the weight w2, and x3 with w3. Different
relevancies of some inputs allow the network to model a specialized behavior. For
example, for an image-recognition task, pixels located at the border of the picture
usually have less relevance than the pixels located in the middle. Weights associated
with the inputs corresponding to the border pixels will therefore be rather close to zero.
In addition to the weighted input value, a perceptron requires a bias, a numerical value
acting as a threshold. We denote the bias as b.
A perceptron receives a stimulus as input and responds to that stimulus by
producing an output value. The output obeys a very simple rule: if the sum of the
weighted inputs is above a particular given value, then the perceptron fires 1; otherwise,
it fires 0. Programmatically, we first compute the sum of the weighted inputs and
the bias. If this sum is strictly above 0, then the perceptron produces 1; otherwise, it
produces 0.
Formally, based on the perceptron given in Figure 1-1, we write z = x1 * w1 + x2 *
w2 + x3 * w3 + b. In the general case, we write z = ∑i xi * wi + b. The variable i
ranges over all the inputs of the perceptron. If z > 0, then the perceptron produces 1 or
if z ≤ 0, it produces 0.
In the next section, we will implement a perceptron model that is both extensible
and maintainable. You may wonder what the big deal is. After all, the perceptron model
may be implemented in a few lines of code. However, implementing the perceptron
functionality is just a fraction of the job. Creating a perceptron model that is testable,
well tested, and extensible is the real value of this chapter. Soon will see how to train a
network of artificial neurons, and it is important to build this network framework on a
solid base.
4
1.2 Implementing the Perceptron

In this section, we will put our hands to work and implement the perceptron model in
the Pharo programming language. We will produce an object-oriented implementation
of the model. We will implement a class called Neuron in a package called
NeuralNetwork. The class will have a method called feed, which will be used to compute
two values—z and the perceptron output.
This code will be contained in a package. To create a new package, you first need
to open a system browser by selecting the corresponding entry in the Pharo menu. The
system browser is an essential tool in Pharo. It allows us to read and write code. Most of
the programming activity in Pharo typically happens in a system browser.
Figure 1-2 shows a system browser, which is composed of five different parts. The top
part is composed of four lists. The left-most list gives the available and ready-to-be-used
packages. In Figure 1-2, the names Announcement, AST-Core and Alien are examples of
packages. The Announcement package is selected in the figure.
The second list gives the classes that belong to the selected package. Many classes
are part of the Announcement package, including the classes called Announcement,
AnnouncementSet, and Announcer.
The third list shows the method categories of the selected class. Method categories
sort methods into logical groups to clarify their purpose and make them easier to find.
Think of them as a kind of package for methods. Since no class is selected in the figure,
no method category is listed.
The right-most list shows the methods of the selected class, filtered by the selected
method category if any. Since no class is selected, no methods are listed. The bottom
part of a system browser displays source code, which is one of the following:
Selection Code Displayed
Method Selected method source code

Class Selected class definition
None New class template
5
Figure 1-2. The Pharo system browser
Right-click the left-most top list to create a new package, named NeuralNetwork. This
package will contain most of the code we will write in this first part of the book.
Select the package NeuralNetwork you just created and modify the template in the
bottom pane as follows:
Object subclass: #Neuron

instanceVariableNames: 'weights bias'
classVariableNames: ''
package: 'NeuralNetwork'
You then need to compile the code by “accepting” the source code. Right-click
the text pane and select the Accept option. The Neuron class defines two instance
variables—weights and bias. Note that we do not need to have variables for the
inputs and output values. These values will be provided as message arguments and
6
returned values. We need to add some methods to define the logic of this perceptron. In
particular, we need to compute the intermediate z and the output values. Let’s first focus
on the weights variable. We will define two methods to write a value in that variable and
another one to read from it.
You may wonder why we define a class called Neuron and not Perceptron. In the
next chapter, we will extend the Neuron class by turning it into an open abstraction for
an artificial neuron. This Neuron class is therefore a placeholder for improvements we
will make in the subsequent chapters. In this chapter we consider a perceptron, but in
the coming chapter we will move toward an abstract neuron implementation. The name
Neuron is therefore better suited.
Here is the code of the weights: method defined in the Neuron class:
Neuron>>weights: someWeightsAsNumbers
"Set the weights of the neuron.
Takes a collection of numbers as argument."
weights := someWeightsAsNumbers
To define this method, you need to select the Neuron class in the class panel (second
top list panel). Then, write the code given without Neuron>>, which is often prepended in
documentation to indicate the class that should host the method. It is not needed in the
browser because the class is selected in the top pane. Figure 1-3 illustrates this. Next, you
should accept the code (again by right-clicking the Accept menu item). In Pharo jargon,
accepting a method has the effect of actually compiling it (i.e., using the Pharo compiler
to translate the Pharo source code into some bytecodes understandable by the Pharo
virtual machine). Once it’s compiled, a method may be executed. The code defines the
method named called weights: which accepts one argument, provided as a variable
named someWeightsAsNumbers.
The weights := someWeightsAsNumbers expression assigns the value
someWeightsAsNumbers to the variable weights.
7
Figure 1-3. The weights: method of the Neuron class
Your system browser should now look like Figure 1-3. The weights: method writes a
value to the variable weights. Its sibling method that returns the weight is
Neuron>>weights
"Return the weights of the neuron."
^ weights
The ^ character returns the value of an expression, in this case the value of the
variable weights.
8
Similarly, you need to define methods to assign a value to the bias variable and to
read its content. The method bias: is defined as follows:
Neuron>>bias: aNumber
"Set the bias of the neuron"
bias := aNumber
Reading the variable bias is provided by the following:
Neuron>>bias
"Return the bias of the neuron"
^ bias
So far, we have defined the Neuron class, which contains two variables (weights and
bias), and four methods (weights:, weights, bias:, and bias). We now need to define
the logic of this perceptron by applying a set of input values and obtaining the output
value. Let’s add a feed: method that does exactly this small computation:
Neuron>>feed: inputs
| z |
z := (inputs with: weights collect: [ :x :w | x * w ]) sum + bias.
^ z > 0 ifTrue: [ 1 ] ifFalse: [ 0 ].
The feed: method simply translates the mathematical perceptron activation

formula previously discussed into the Pharo programming language. The expression
inputs with: weights collect: [ :x :w | x * w ] transforms the inputs and
weights collections using the supplied function. Consider the following example:
#(1 2 3) with: #(10 20 30) collect: [ :a :b | a + b ]
The expression #(1 2 3) is an array made of three numbers—1, 2, and 3. The

expression evaluates to #(11 22 33). Syntactically, the expression means that the
literal value #(1 2 3) receives a message called with:collect: with two arguments,
the literal array #(10 20 30) and the block [ :a :b | a + b ]. You can verify the
value of that expression by opening a playground (accessible from the Tools top
menu). A playground is a kind of command terminal for Pharo (e.g., xterm in the UNIX
world). Figure 1-4 illustrates the evaluation of the expression (evaluated either by
choosing Print It from the right-click menu or using the adequate shortcut—Cmd+p on
OSX or Alt+p on other operating systems).
9
Figure 1-4. The playground
We can now play a little bit with the perceptron and evaluate the following code in
the playground we just opened:
p := Neuron new.
p weights: #(1 2).
p bias: -2.
p feed: #(5 2)
This piece of code evaluates to 1 (since (5*1 + 2*2)-2 equals to 7, which is greater
than 0), as shown in Figure 1-5.
Figure 1-5. Evaluating the perceptron
1.3 Testing the Code

Now it is time to talk about testing. Testing is an essential activity whenever we write
code using Agile methodologies. Testing is about raising the confidence that the code we
write does what it is supposed to do.
Although this book is not about writing large software artifacts, we do write source
code. And making sure that this code can be tested in an automatic fashion significantly
improves the quality of our work. More importantly, most code is read far more
10
often than it is written. Testing helps us produce maintainable and adaptable code.
Throughout this book, we will improve our code base. It is very important to make sure
that our improvements do not break existing functionalities.
For example, we previously defined a perceptron and informally tested it in a
playground. This informal test costs us a few keystrokes and a little bit of time. What if we
could repeat this test each time we modified our definition of perceptron? This is exactly
what unit testing is all about.
We will now leave the playground for a while and return to the system browser to
define a class called PerceptronTest:
TestCase subclass: #PerceptronTest

instanceVariableNames: ''
classVariableNames: ''
package: 'NeuralNetwork'
The TestCase class belongs to the built-in Pharo code base. Subclassing it is the first
step to creating a unit test. Many perceptrons will be created by the tests we define. We
can define the method as follows:
PerceptronTest>>newNeuron
"Return a new neuron"
^ Neuron new
Tests can now be added to PerceptronTest. Define the following method:
PerceptronTest>>testSmallExample
| p result |
p := self newNeuron.
p weights: #(1 2).
p bias: -2.
result := p feed: #(5 2).
self assert: result equals: 1.
The testSmallExample method tests that the code snippet we previously gave
returns the value 1. You can run the test by clicking the gray circle located next to the
method name. The gray circle turns green to indicate that the test passes (see Figure 1-6).
11
Figure 1-6. Testing the perceptron
A green test means that no assertion failed and no error was raised during the test
execution. The testSmallExample method sends the assert:equals: message, which
tests whether the first argument equals the second argument.
EXERCISE: So far, you have only shallowly tested this perceptron. You can improve
these tests in two ways:
• Expand testSmallExample by feeding the perceptron p with different

values (e.g., -2 and 2 gives 0 as a result).
• Test the perceptron with different weights and biases.
In general, it is a very good practice to write a thorough suite of tests, even for a small
component such as this Neuron class.
12
Another random document with
no related content on Scribd:
probable that the French Government itself had this paragraph
inserted, and it was, moreover, concocted in complete ignorance of
the Spanish Constitution and of the laws governing the election of a
King. This, which was the only new feature, was a barefaced
invention. It had already been mentioned in all the papers that Prim’s
speech of the 11th of June referred to the Prince of Hohenzollern,
and that had caused no excitement in France. Is the present
agitation then a coup monté? Does the French Government insist
upon a ‘row’? Has Louis Napoleon chosen Grammont in order to
pick a quarrel with us? At any rate he has been unskilful in his
treatment of this question. The general moral to be drawn as often
as possible is: the French Government is, after all, not quite so
shrewd as people believe. The French have succeeded in many
things with the assistance of 300,000 soldiers, and owing to that
success they are regarded as immensely clever. Is that really so?
Circumstances show that it is not.”
July 10th, evening.—Received further series of sketches and
drafts for articles from Bucher, who acts as the mouthpiece of the
Chancellor’s views and intentions.
1. For the Spenersche or Kölnische:—“Those foreign Powers that
are not concerned in the differences respecting the Spanish throne
are as desirous to maintain peace as Germany herself. Their
influence will, however, be neutralised by Grammont’s ill-considered
threats. Should the German Governments consider the security of
our frontier to be seriously threatened, they would scarcely come to
a decision without convoking Parliament.”
2. “The French are running amuck like a Malay who has got into
a rage and rushes through the streets dagger in hand, foaming at the
mouth, stabbing every one who happens to cross his path. If France
is mad enough to regard Germany as a fit object for a vicarious
whipping, nothing will restrain her, and the result will be that she will
herself receive a personal castigation.”
3. “The semi-official journals in Paris pretend that attention has
been attracted there by the numerous cipher despatches exchanged
between Berlin and Madrid, and that they have been clever enough
to decipher them. We do not know whether many despatches have
passed between the two capitals mentioned, but we remember a
communication which was made to Parliament some time ago by
Count Bismarck, according to which the cipher system of our Foreign
Office is based on a vocabulary of about 20,000 words, each one of
which is represented by a group of figures arbitrarily chosen. It is
impossible to ‘decipher’ such a system in the same way as those
based on an altered alphabet and other old methods. In order to
read such a despatch, it is essential to have the vocabulary. Does
the cleverness on which the Parisians pride themselves consist in
having stolen the key to our ciphers? This would be in contradiction
with the original statement that the Prince of Hohenzollern’s
candidature first became known through a communication from Prim.
It would, therefore, appear that the official press wants to clear the
Government of the reproach of incapacity by a subsequent invention,
acting, on the maxim that it is better to be taken for a rogue than a
fool.”
4. “According to a private telegram from Paris to the Berliner
Boersen Zeitung, our Ambassador there, together with the second
Secretary of Embassy, left for Ems on receipt of a Note delivered to
him immediately after the Cabinet Council at Saint Cloud. We have
made inquiries in the proper quarter as to the accuracy of this report,
and have received the following answer: Note delivered. ‘Not a
shadow of truth. Werther’s journey was decided upon and
announced in Paris long before the agitation began.’”
5. “As was already known, Prim intended this year, as on
previous occasions, to visit Vichy. This would have led to a meeting
between himself and the Emperor Napoleon and a discussion of the
succession to the Spanish throne. It is also reported that the Prince
of Hohenzollern was not indisposed to try confidentially to bring
about an understanding with the Emperor. All this has been rendered
impossible by the abrupt tone of the Duc de Grammont. As Prim’s
visit to Vichy has long since been announced in the newspapers,
and the near relationship as well as the personal friendship which
hitherto existed between the Prince of Hohenzollern and the
Emperor rendered both meetings probable, it is hard to avoid the
suspicion that the French Government, dreading insurmountable
domestic difficulties, desires to inflame French vanity in favour of a
war, which would at the same time promote the dynastic views of the
Empress Eugénie.”
July 12th.—Received from Secretary Wollmann a note from
Bucher in Varzin which is intended for me. It has been sent to the
Secretary of State, in order that he should say whether there is any
objection to its being used in the press. He has no objection, and so
it goes to the newspapers. It runs as follows: “The Imparcial
publishes a letter from Paris to the effect that the furious article in the
Constitutionnel reproaching Prince Hohenzollern with his relationship
to Murat, has been revised by the Emperor himself.”
In the evening the Minister returned. He is dressed in plain
clothes and looks very well.
July 13th.—Called early to the Chief. I am to wait until a
statement appears in the press to the effect that the renunciation of
Prince Hohenzollern was in consequence of pressure from Ems, and
then to contradict it. “In the meantime (said the Minister) the
Norddeutsche should only say that the Prince’s present decision has
not been altogether unexpected. When he accepted the throne
which had been offered to him he had obviously not foreseen that his
decision would occasion so much excitement in Paris. For more than
thirty years past the best relations existed between Napoleon and
the Hohenzollern family. Prince Leopold could not, therefore, have
apprehended any antipathy to his candidature on the part of the
Emperor. As his candidature suddenly became known after the
Cortes had been adjourned till November, it may well have been
assumed that there would be time enough in the interval to sound
the Emperor as to his views. Now that this assumption (here the
Chancellor began to speak more slowly as if he were dictating),
which, up to the acceptance of the Crown by the Prince, was still
quite legitimate, had proved to be partly erroneous, it was scarcely
probable that the Prince would, on his own responsibility, be
disposed to cope single handed with the storm which his decision
had raised, and might yet raise, in view of the apprehensions of war
of the whole European world, and the influence brought to bear upon
him from London and Brussels. Even a portion of the responsibility of
involving the great European nations, not only in one war, but
possibly in a series of wars, would weigh very heavily upon a man
who could not claim to have assumed it as part of the duty of the
Royal office which he had already accepted. That was more than
could well be expected of a Prince who only occupied a private
position. It was the offensive tone of Grammont that alone prevented
Prussia from exercising her influence with the Prince.”
The following is to be published in other papers: “It cannot be
denied that a Spanish Government disposed to promote the cause of
peace and to abstain from conspiring with France would be of
considerable value to us. But if, some fourteen days ago, the
Emperor Napoleon had addressed himself confidentially to Berlin, or
indicated that the affair was attracting attention, Prussia, instead of
adopting an indifferent attitude, would have co-operated in pacifying
public opinion in Paris. The situation has been entirely altered
through the aggressive tone of Grammont’s speech, and the direct
demands addressed to the King, who is staying in privacy at Ems for
the benefit of his health, unaccompanied by a single Minister. His
Majesty rightly declined to accede to these demands. That incident
has created so much indignation in Germany, that many people feel
disappointed at Prince Leopold’s renunciation. At any rate, the
confidence in the peaceful intentions of France has been so
thoroughly shaken that it will take a considerable time to restore it. If
commerce and trade have been injured by the evidence which has
shown us what a den of brigands we have to deal with in France, the
people of that country must fasten the responsibility on the personal
régime under which they at present live.”
The Minister also desires it to be incidentally remarked in the
non-official press that of the South German Courts those of Munich
and Carlsruhe had given the most satisfactory declarations in this
affair, while on the other hand that of Stuttgart had expressed itself
evasively.
Finally, I am to communicate to one of the local papers that
Count Bismarck has been sent for to Ems to consult with the King as
to summoning Parliament. Breaking off a cure which he was
undergoing, the Chancellor has remained in Berlin in order to await
there the further instructions of his Majesty, or ultimately to return to
Varzin. The Count then added: “Later on I will call for you several
times, as there is something more to be prepared for the
Norddeutsche. We shall now be shortly interrupted.” The Crown
Prince arrived five minutes afterwards and had a long interview with
the Minister.
July 14th.—Our newspapers to call attention to the loyal attitude
of Würtemberg, “which in consequence of a misunderstanding has
been represented in some journals as evasive.”
July 15th.—I am to send the following démenti to Wolf’s
Telegraphic Agency for circulation: “The news published by the
Spenersche Zeitung respecting the opening of Parliament is not
quite accurate. It was proposed a week ago by the Chancellor while
in Varzin that it should be convoked as soon as the Government Bills
were ready for submission to it. His Majesty shares this view, and the
Federal Council has accordingly been summoned for to-morrow,
Saturday, morning to consider those measures.”
In the evening the Chancellor dictated an article for the
Kreuzzeitung on the confusion by the public between personal and
private proceedings of the King and his official acts. It ran as follows:
“It appears from the Mazaredo pamphlet that the Hereditary Prince
of Hohenzollern informed the King at Ems of his acceptance of the
offer of the Spanish throne, probably towards the end of June. His
Majesty was then at Ems for the purpose of taking the waters, and
certainly not with the intention of carrying on business of State, as
none of his Ministers had been summoned thither. As a matter of
fact, only so much has become public respecting the King’s reply to
the communication of the Hereditary Prince (it was in the form of a
letter written in his Majesty’s own hand) that the Sovereign was not
pleased at the news, although he did not feel called upon to offer any
opposition. In the whole affair no State action of any kind has been
taken. This constitutional aspect of the situation does not appear to
have been properly appreciated up to the present in public
discussions of the question. The position of the King in his private
correspondence was confounded with his position as head of the
State, and it was forgotten that in the latter capacity, according to the
Constitution, the co-operation of the Ministry is necessary to
constitute a State action. It is only the French Cabinet that appears
to have thoroughly realised this distinction, inasmuch as it brought
the whole force of its diplomacy to bear upon the person of the
Sovereign, who was staying at a watering-place for the sake of his
health, and whose private life was not protected by the usual
etiquette, in order to force him under official pressure into private
negotiations which might afterwards be represented as
arrangements with the Government.”
July 19th.—About an hour after the opening of Parliament in the
Royal Palace (1.45 p.m.) Le Sourd, the French Chargé d’Affaires,
delivered Napoleon’s declaration of war at the Foreign Office.
Towards 5 o’clock in the evening I was called to the Minister, who
was in his garden. After searching for him for some time I saw him
coming through one of the long shady alleys to the left which led to
the entrance in the Königgrätzer Strasse. He was brandishing a big
stick. His figure stood out against the yellow evening sunshine like a
picture painted on a gold ground. He stopped in his walk as I came
up to him, and said: “I wish you to write something in the
Kreuzzeitung against the Hanoverian nobles. It must come from the
provinces, from a nobleman living in the country, an Old Prussian—
very blunt, somewhat in this style: It is reported that certain
Hanoverian nobles have endeavoured to find pilots and spies in the
North Sea for French men-of-war. The arrests made within the last
few days with the assistance of the military authorities are
understood to be connected with this affair. The conduct of those
Hanoverians is infamous, and I certainly express the sentiments of
all my neighbours when I put the following questions to the
Hanoverian nobles who sympathise with those traitors. Have they
any doubt, I would ask them, that a man of honour could not now
regard such men as entitled to demand honourable satisfaction by
arms whether their unpatriotic action was or was not undertaken at
the bidding of King George? Do they not, as a matter of course,
consider that an affair of honour with them is altogether out of the
question, and should one of them be impudent enough to propose
such a thing, would they not have him turned out of the house by the
servants or eject him propriæ manu after having, of course, put on a
pair of gloves to handle him with? Are they not convinced that such
miscreants can only be properly described by the good old Prussian
word Hundsvott (scurvy, infamous rogues), and that their treason
has branded their posterity to the third and fourth generations with
indelible disgrace? I beg them to answer these questions.”
Evening.—In an article in the Liberté of the 18th instant, that
paper reminds Italy that she owes her liberation to France, and that
in 1866 it was France who brought about the Italian alliance with the
Berlin Cabinet. It then maintains that, in view of the seriousness of
approaching events, Victor Emmanuel, with truly chivalrous
sentiment, has not for a moment hesitated to assure the French of
his unconditional support. With reference to this article our papers
should observe: “Up to the present the French have played the part
of masters to the whole world, and Belgium, Spain, and the King of
Prussia have in turn experienced their arrogance. Their behaviour
was somewhat like that of the Sultan towards his Khedive, it was a
kind of megalomania based upon the bayonet. Their presumption is
now beginning to waver, so they court the assistance of those good
friends whom they pretend to have placed under obligations to
them.”
The Minister subsequently dictated the following, to be worked up
for the German newspapers outside Berlin, such as the Kölnische
Zeitung, and for the English and Belgium journals: “According to
confidential communications from loyal Hanoverian circles, amongst
other decisive factors which led the French to the declaration of war,
were the reports sent to Paris by Colonel Stoffel, the Military
Plenipotentiary in Berlin. Stoffel’s information was, it appears, less
accurate than abundant, as none of those who supplied him with it
being prepared to forego the payments they received from him
merely because they had nothing to say, they occasionally invented
the news of which they warranted the correctness. The
Plenipotentiary had, it is said, been informed that the arming of the
Prussian infantry, both as regards rifles and ammunition, was at
present undergoing a thorough transformation, and that
consequently a moment so favourable as the present for attacking
Prussia would hardly occur again, inasmuch as on the completion of
this change the Prussian armaments would have been
unassailable.”[4]
2. “It now appears to be beyond all doubt that the French
Government was aware of the candidature of the Prince of
Hohenzollern for months past, that they carefully promoted it and
foolishly imagined it would serve as a means of isolating Prussia and
creating a division in Germany. No trustworthy information has been
received as yet as to whether and how far Marshal Prim had
prepared the way for this intrigue, in agreement with the Emperor
Napoleon. But doubtless that point will ultimately be cleared up by
history. The sudden disappearance of Spain from the political field as
soon as the differences between France and Prussia broke out gives
matter for reflection and suspicion. It cannot but be regarded as
strange that after the zeal shown by the Spanish Government in the
matter of the Hohenzollern candidature had been raised to boiling
point it should have suddenly fallen below zero, and that the
relations of Marshal Prim to the French Cabinet should now appear
to be of the most friendly character, while the Spaniards seem no
longer to feel any irritation at the interference of France in their
internal affairs.”
3. “Rumours were circulated this afternoon to the effect that the
former French Military Plenipotentiary, Baron Stoffel, had been
insulted in the street. On closer inquiry it was ascertained that some
individuals who knew Stoffel followed him in the street, and on his
reaching his house struck the door with their sticks. The police
intervened energetically on the first report of this matter and have
taken measures to prevent a repetition of such conduct and to
provide that Baron Stoffel shall not be interfered with on his
departure this evening. Excesses of this description are, however,
highly reprehensible, even when they are confined to words. The
former representatives of France are under the protection of
international law and of the honour of Germany until they have
crossed the frontier.”
July 21st.—Keudell asked me this morning if I knew Rasch, the
journalist, and if I could say where he was now to be found, in Berlin
or elsewhere. I replied that I had seen him in Schleswig in 1864,
afterwards at a table d’hôte at the Hotel Weissberg, in the Dessauer
Strasse, where he lodged at the end of February. I knew nothing
more about him, but had heard that he was extremely conceited,
almost to the point of madness—a political visionary who desired to
convert the whole world to republicanism. I was not aware of his
whereabouts in Berlin, but would make inquiries at Weissberg’s.
Keudell told me to hunt him up and ask him whether he would go to
Garibaldi and urge him to undertake an expedition against Rome, at
the same time carrying him money from us. I pointed out that Rasch
was perhaps too vain to keep his own counsel. Keudell consoled
himself with the idea that he would doubtless prove a good patriot. I
declined to treat with Rasch in the matter, as I could not speak to him
in my own name but in that of the Foreign Office, and that could be
better done by some official of higher rank, who would make a
greater impression upon Rasch. Keudell seemed to recognise the
justice of this view. I made inquiries and was able to report on the
same evening that Rasch was staying at Weissberg’s.
Called to the Minister in the evening. He showed me an extract
from the National Zeitung, and observed: “They say here that the
English would not allow the French to attack Belgium. Well and
good, but how does that help the Belgians if the protection comes
too late? If Germany were once defeated (which God forbid!) the
English would not be able to assist the Belgians in the least, but
might, on the contrary, be thankful if they themselves remained safe
in London.”
I am further to call attention to the “manner in which France is
begging for help on all sides—that great warlike nation which makes
so much parade of its victories, representing them as having always
been won solely by the force of its own arms. They go begging (use
that expression) to Italy, to Denmark, to Sweden, and above all to
the German States, to whom they promise the same brilliant destiny
which they have already prepared for Italy—political independence
and financial ruin.”
Called up to the Minister again later. I am to secure the insertion
of the following in the non-official German papers and in the Belgian
and English press: “The English Government observe their neutrality
in connection with the war that has now broken out in a liberal and
conscientious spirit. They impartially permit both sides to purchase
horses and munitions of war in England. It is unfortunate, however,
that France alone can avail herself of this liberality, as will appear
from a glance at the geographical position of the two countries and
from the superiority of the French at sea. Then quote what Heffter
(the book must be in the library) has to say on this kind of neutrality,
and observe that the English jurists describe it more tersely as
‘fraudulent neutrality.’”
July 23rd.—Called to the Minister five times to-day. The press
should urge the prosecution and seizure of Rothan, an Alsacian who
speaks German, hitherto French Chargé d’Affaires at Hamburg, who
has been a zealous spy and instrument of French intrigue in North
Germany, and who is now understood to be wandering along the
coast between the Elbe and Ems, as also that of the ex-Hanoverian
officer, Adolf von Kielmansegg, respecting whom further particulars
are to be obtained from the Ministry of the Interior. The Count further
wants the press to give a list of the names of the Bavarian members
of Parliament who voted for the neutrality of that State in the national
war, mentioning their professions but without any further remarks.
“Give it first in Brass,” (i.e., Norddeutsche Allgemeine Zeitung,) he
added. “You will find such a list amongst the documents. The
complaints as to the manner in which England understands neutrality
must be continually renewed. The English Government does not
forbid the export of horses, though only France can avail herself of
that facility. Colliers are allowed to load at Newcastle and to supply
fuel for the French men-of-war cruising in the North Sea. English
cartridge factories are working for the French army under the eyes of
the Government. In Germany the painful feeling has become more
and more widespread that, under Lord Granville, England, while
nominally maintaining neutrality, favours France in the manner in
which it is really observed.”
About 11 p.m. I was again called to the Minister. The reports
respecting the English coal ships to be at once sent by a Chancery
attendant to Wolf’s Telegraphic Agency for circulation to the
newspapers.
In this connection may be mentioned an Embassy report from
London, dated the 30th of July, to the following effect: Lord Granville
had asked the Ambassador if he had not stirred up the authorities in
Berlin against the English Government. The reply was in the
negative. The Ambassador had only carried out his instructions.
Public opinion in Germany influenced the Government, just as the
German press influenced public opinion. The manner in which
neutrality was observed on the part of England had excited the
greatest indignation in Germany. The action of the English
Government, which indeed recognised that France was in the wrong,
but failed to give expression to that conviction, was also bitterly
resented there. Granville replied that once it had been decided to
remain neutral that neutrality must be maintained in every respect. If
the export of contraband of war were forbidden, the French would
regard it as an act of one-sided hostility, while at the same time it
would ruin English trade in the branches affected by such prohibition,
and favour American manufacturers. For the present, every one in
England approved of the maintenance of neutrality, and therefore in
a general way no change was possible in these matters. At the same
time, the English Government was ready, in case of complaints
reaching them in an official way respecting any acts of illegality, to
institute an inquiry into the facts and secure the punishment of the
guilty parties. It did not seem impossible to prevent the supply of
English coal to French men-of-war. Next Monday a Bill was to be
submitted to Parliament for the amendment of the laws regulating
neutrality. The report concluded as follows: “England is in many
respects well disposed towards us, but will for the present remain
neutral. If we make further attacks upon English public opinion
through our official press in connection with these grievances, it will
serve no purpose but to conjure up future difficulties. Granville is not
what we might desire, but he is not prejudiced against us. He may
become so, however, if he is further provoked by us. We can hardly
succeed in overthrowing him, and if we did his probable successor
would in all likelihood be much worse than himself.”
July 24th.—I am instructed by the Count to send an article to the
Kölnische Zeitung respecting the Dutch coal question. He gave me
the following information on this subject: “Holland asked us to again
permit the passage of Prussian coal down the Rhine, and requested
that a large transport of Rhenish coal intended for Holland should be
allowed to pass the frontier. It was only to be used in factories, and
the Government of the Netherlands would prohibit its re-exportation.
Prussia willingly agreed to this, but shortly afterwards it was
ascertained that foreign vessels were being loaded with coal in
Dutch ports, and the Government of the Netherlands subsequently
informed us that in promising to prevent the re-exportation they had
overlooked the circumstance that their treaty with France did not
permit this. Thereupon as a matter of course the export of Prussian
coal to Holland was prohibited. In the interval, however, they seem to
have secured a sufficient supply in Holland to provide the French
fleet for a considerable time. That is a very suspicious method of
observing the neutrality promised by the gentlemen at the Hague.”
Bucher brings me the following paragraph from the Chief, which
is to be inserted in the Spenersche Zeitung, or some other non-
official organ, and afterwards in the North German Correspondence:
“In 1851 a literary gamin in Paris was commissioned to conjure up
the Red Terror in a pamphlet, which proved very useful to the
President Louis Napoleon, enabling him to escape from a debtors’
prison and ascend the Imperial throne. The Duc de Grammont now
tries to raise the Spanish Terror in order to save the Emperor from
the necessity of accounting for the hundred millions which he
diverted from the State Treasury into his private purse. The literary
gentleman in question was made a Prefect. What reward can
Grammont have had in view?”
Evening.—The Minister wishes an article to be prepared for
circulation in the German press describing the French and French
policy under the Emperor Napoleon. This is to be first sent to the
Spenersche Zeitung, while the Literary Bureau is to secure the
insertion of the principal points in a condensed form in the
Magdeburg papers and a number of the smaller journals to-morrow.
The Count said (literally): “The French are not so astute as people
generally think. As a nation they resemble certain individuals
amongst our lower classes. They are narrow-minded and brutal,—
great physical force, boastful and insolent, winning the admiration of
men of their own stamp through their audacity and violence. Here in
Germany the French are also considered clever by persons who do
not think deeply, and their Ministers are regarded as great statesmen
because of their insolent interference in the affairs of the whole
world, and their desire to rule everywhere. Audacity is always
impressive. People think their success is due to shrewd political
calculation, but it is actually due to nothing else than the fact that
they always keep 300,000 soldiers ready to back up their policy. That
alone, and not their political intelligence, has enabled them to carry
things with such a high hand. We must get rid of this fiction.... In
political affairs the French are in the fullest sense of the word a
narrow-minded nation. They have no idea how things look outside of
France, and learn nothing about it in their schools. The French
educational establishments, for the greater glory of France, leave
their pupils in the crassest ignorance as to everything beyond her
frontiers, and so they have not the slightest knowledge of their
neighbours; that is the case with the Emperor, or at least he is not
much better, to say nothing of Grammont, who is an ass (Rindvieh).
Napoleon is ignorant at bottom, although he has been educated in
German schools. His ‘Cæsar’ was intended to conceal that fact. He
has forgotten everything. His policy was always stupid. The Crimean
War was against the interests of France, which demanded an
alliance or at least a good understanding with Russia. It was the
same with the war in Italy. There he created a rival in the
Mediterranean, North Africa, Tunis, &c., who may one day prove
dangerous. The Italian people are much more gifted than the French,
only less numerous. The war in Mexico and the attitude adopted in
1866 were blunders, and doubtless in storming about as they do at
present the French feel conscious that they have committed another
blunder.”
July 25th.—At 11 o’clock this morning Count Bismarck and his
family took the Holy Communion at their residence. He asked
whether any one in our bureau desired to join them, but no one
offered to do so. I was for a moment tempted, but reconsidered the
matter. It might look as if I wished to recommend myself.
Copies of the Benedetti draft treaty are sent to Auber (the French
Press Agency) and Heide.
July 27th.—It is to be stated either in the Norddeutsche or the
Spenersche Zeitung that secrecy respecting confidential
communications between great States is, as a rule, more carefully
observed and maintained than the public imagines. Nevertheless,
the French misrepresentation of Prussia’s attitude in the affair of the
candidature for the Spanish throne (in Grammont’s despatch of the
21st of July) obliged the authorities here to disregard these
considerations of discretion. Benedetti’s proposal has therefore been
published and it may be followed by other documents of the same
description. The Count concluded his directions as follows: “We are
at least entitled to tell the truth with discretion in presence of such
indiscreet lies.”
Bucher brings me from the Minister the following sketch of a
paragraph for the press: “The despatch of the Duc de Grammont, the
full text of which now lies before us, is a desperate attempt to prove
that the origin of the situation which they have themselves created
was the Hohenzollern candidature, and to conceal the motive which
they confessed on many other occasions—namely, the conquest by
France of the left bank of the Rhine and of Belgium. The
inconsistency of the whole assertion is made clear by the
circumstance that the offer of the Spanish throne to the Hereditary
Prince of Hohenzollern was first made in a letter dated the 14th of
February of the present year. Therefore, there can be no connection
between this offer and the conversations in March, 1869, between
Benedetti and Von Thile, which were the outcome of aspirations or
proposals frequently ventilated in the press (also with reference to
Prince Frederick Charles). In 1851 the President Louis Napoleon
succeeded in obtaining credence both at home and abroad for
certain fictions, so long as that was necessary for the attainment of
his object. The fiction which is now circulated, at a somewhat late
hour, to the effect that the Prince of Hohenzollern was the candidate
of Prussia is refuted in advance by the fact, which has been well
known for a long time, that the Prussian Government as well as the
officials of the Confederation, had absolutely no knowledge of, or
connection with, the Spanish proposal. It was resolutely opposed by
his Majesty the King, as the head of the Hohenzollern family, until
last June, when at Ems he reluctantly withdrew his opposition when
it was represented to him that otherwise Spain would fall into the
hands of the Republicans. We find it difficult to understand what
interest the French Government can have in circulating such lies
now that war has actually broken out. The attempt of the Duc de
Grammont to conjure up the spectre of a restoration of Charles V.’s
monarchy can only be explained by the complete isolation of the
French mind. That apparition had no sooner manifested itself than it
vanished before the angry contempt of public opinion, which
resented being supposed capable of such credulity.”
The Chancellor desires to see the following considerations
reproduced in the evening papers: “The Benedetti document is by no
means the only one dealing with the matter in question. Negotiations
were also carried on by others, as, for instance, by Prince Napoleon
during his stay in Berlin. Since French diplomacy was ignorant
enough to believe that a German Minister who followed a national
policy could for a moment think of entertaining such proposals, it had
only itself to thank if it was befooled with its own schemes so long as
such fooling appeared calculated to promote the maintenance of
peace. Even those who pursue the most ignorant and narrow-
minded policy must ultimately come to recognise that they have
hoped for and demanded impossibilities. The bellicose temper which
now prevails in Paris dates from such recognition. The hopes of
German statesmen that they would be able to befool the French until
a peaceful régime was established in France by some transformation
of her despotic constitution have unfortunately not been realised.
Providence willed it otherwise. Since we can no longer maintain
peace it is not necessary now to preserve silence. For we preserved
silence solely in order to promote the continuance, and, if possible,
the permanency, of peaceful relations.”... The Minister concluded:
“You can add, too, that the question of French Switzerland was also
mentioned in the negotiations, and that it was hinted that in
Piedmont they knew quite well where the French districts begin and
the Italian districts leave off.”
July 28th.—I see the original of Benedetti’s draft treaty, and I am
to receive a photographic copy of it similar to that which has been
prepared for distribution amongst foreign Governments.
Bucher handed me the following sketch of an article, received by
him from the Minister, which is to be inserted in some organ not
apparently connected with the Government: “Those who now hold
power in Spain declare that they do not wish to interfere in the
conflict between Germany and France, because the latter might
create internal difficulties for them. They allow Bonaparte to prohibit
their election of the King of their own choice. They look on calmly
with folded arms while other nations go to war over a difference that
has arisen out of a question of Spanish domestic interest. We had
formed quite another opinion of the Castilian gentilhomme. The
Spanish temper seems to resemble that of Gil Blas, who wanted to
fight a duel with the army surgeon but observed that the latter had an
unusually long rapier.”
July 30th, 10 p.m.—The Minister desires that attention should be
again called to the manner in which the French are looking about for
foreign assistance, and he once more gives a few points: “France is
begging in all directions, and wants in particular to take Italy into her
pay. Here, as everywhere, she speculates upon the worst elements,
while the better elements will have nothing to do with her. How does
that harmonise with the greatness of the nation which ‘stands at the
head of civilisation,’ and whose historians always point out that it
was only defeated at Leipzig because its opponents were four to
one? At that time they had half Germany, Italy, Holland, and the
present Belgium on their side. To-day, when they stand alone, they
go round hat in hand to every door, and seek mercenaries to
reinforce their own army, in which they can therefore have but very
little confidence.”
July 31st.—This morning received from Roland one of the
photographic copies of the Benedetti draft.

Agile Artificial Intelligence in Pharo: Implementing Neural Networks, Genetic Algorithms, and Neuroevolution Alexandre Bergel

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Agile Artificial Intelligence in Pharo: Implementing Neural Networks, Genetic Algorithms, and Neuroevolution Alexandre Bergel

Uploaded by

Copyright:

Available Formats

Agile Artificial Intelligence in Pharo:

Implementing Neural Networks, Genetic

Agile Visualization with Pharo: Crafting Interactive

Artificial Neural Networks and Machine Learning ICANN

Using Artificial Neural Networks for Timeseries

Artificial Neural Networks Methods in Molecular Biology

Artificial Neural Networks and Machine Learning ICANN

Artificial Neural Networks and Machine Learning ICANN

Artificial Neural Networks and Machine Learning ICANN

Artificial Neural Networks with Java: Tools for

ISBN-13 (pbk): 978-1-4842-5383-0 ISBN-13 (electronic): 978-1-4842-5384-7

Copyright © 2020 by Alexandre Bergel​

About the Technical Reviewer������������������������������������������������������������������������������� xiii

Part I: Neural Networks������������������������������������������������������������������������� 1

Chapter 2: The Artificial Neuron����������������������������������������������������������������������������� 37

Chapter 3: Neural Networks����������������������������������������������������������������������������������� 53

Chapter 4: Theory on Learning������������������������������������������������������������������������������� 69

Chapter 5: Data Classification�������������������������������������������������������������������������������� 89

Chapter 6: A Matrix Library���������������������������������������������������������������������������������� 117

Chapter 7: Matrix-Based Neural Networks����������������������������������������������������������� 141

Part II: Genetic Algorithms���������������������������������������������������������������� 153

Chapter 9: Genetic Algorithms in Action�������������������������������������������������������������� 195

Chapter 10: The Traveling Salesman Problem������������������������������������������������������ 209

Chapter 11: Exiting a Maze����������������������������������������������������������������������������������� 225

Chapter 12: Building Zoomorphic Creatures�������������������������������������������������������� 237

Chapter 13: Evolving Zoomorphic Creatures�������������������������������������������������������� 265

Part III: Neuroevolution���������������������������������������������������������������������� 281

Chapter 15: Neuroevolution with NEAT����������������������������������������������������������������� 295

Chapter 16: The MiniMario Video Game��������������������������������������������������������������� 355

Afterword: Last Words������������������������������������������������������������������������������������������ 377

Accompanying Source Code

Who Should Read This Book?

The Pharo Experience

• http://pharo.org is the official website about Pharo.

• https://mooc.pharo.org is probably the most popular way to learn

• http://books.pharo.org offers many valuable books and booklets

• http://agilevisualization.com describes the Roassal visualization

Visualization is omnipresent in Agile Artificial Intelligence in Pharo. Roassal is used

The Perceptron Model

1.1 Perceptron as a Kind of Neuron

Figure 1-1. Representing the perceptron

Figure 1-1 depicts a perceptron. A perceptron is usually represented as a circle with

1.2 Implementing the Perceptron

Selection Code Displayed

Method Selected method source code

Figure 1-2. The Pharo system browser

Object subclass: #Neuron

Figure 1-3. The weights: method of the Neuron class

Reading the variable bias is provided by the following:

The feed: method simply translates the mathematical perceptron activation

#(1 2 3) with: #(10 20 30) collect: [ :a :b | a + b ]

The expression #(1 2 3) is an array made of three numbers—1, 2, and 3. The

Figure 1-4. The playground

Figure 1-5. Evaluating the perceptron

1.3 Testing the Code

TestCase subclass: #PerceptronTest

Tests can now be added to PerceptronTest. Define the following method:

Figure 1-6. Testing the perceptron

• Expand testSmallExample by feeding the perceptron p with different

• Test the perceptron with different weights and biases.

You might also like

Copyright © 2020 by Alexandre Bergel

About the Technical Reviewer�� xiii

Part I: Neural Networks�� 1

Chapter 2: The Artificial Neuron�� 37

Chapter 3: Neural Networks�� 53

Chapter 4: Theory on Learning�� 69

Chapter 5: Data Classification�� 89

Chapter 6: A Matrix Library�� 117

Chapter 7: Matrix-Based Neural Networks�� 141

Part II: Genetic Algorithms�� 153

Chapter 9: Genetic Algorithms in Action�� 195

Chapter 10: The Traveling Salesman Problem�� 209

Chapter 11: Exiting a Maze�� 225

Chapter 12: Building Zoomorphic Creatures�� 237

Chapter 13: Evolving Zoomorphic Creatures�� 265

Part III: Neuroevolution�� 281

Chapter 15: Neuroevolution with NEAT�� 295

Chapter 16: The MiniMario Video Game�� 355

Afterword: Last Words�� 377

Accompanying Source Code

Who Should Read This Book?

The Pharo Experience

1.1 Perceptron as a Kind of Neuron

1.2 Implementing the Perceptron

1.3 Testing the Code