M1.

THE HISTORY
OF AI
UNIT 1.2 – FROM COGNITIVISM TO CONNECTIONISM

Raffaella Esposito/UNINA
SLIDE 1
Throughout the history of artificial intelligence, there have been moments of significant change
and transformation that have shaped the field and opened up new perspectives. Among them, the
transition from cognitivism to connectionism represents a crucial and revolutionary chapter in the
evolution of AI. This transitional moment redefined the way we conceive of intelligence and
opened the door to new paradigms and approaches to learning and knowledge representation.

SLIDE 2
Cognitivism, a current of thought in psychology, has used computer operation as a metaphor for
the mind (Neisser, Lindsay, Norman). This metaphor suggests that the mind is an information
processing system, like a computer, because information from our senses, such as visual or
auditory stimulus, is processed through cognitive processes that follow rules and patterns of
reasoning. These rules can be compared to the instructions of a computer, which guide how data
are manipulated and interpreted. Just as a computer uses algorithms to solve problems, the
human mind relies on cognitive processes that follow specific rules and strategies to deal with
cognitive challenges. For example, when we read a sentence, our cognitive system applies
grammatical rules to understand the meaning of the sentence.

SLIDE 3
Cognitivism modeling has led to important developments in the field of artificial intelligence. For
example, the application of rules of logical inference and symbolic representation has enabled the
creation of expert systems capable of making decisions and solving problems in specific domains.
Despite its successes, cognitivism encountered limitations that stimulated a search for new
approaches. A major challenge was the complexity of rules and symbolic representations, which
made it difficult to manage expert systems in complex and ever-changing scenarios. In addition,
the ability to adapt to new contexts and learn from unstructured data was limited. It was in this
context that connectionism took hold.

SLIDE 4
In fact, the computer is very efficient in solving tasks in which humans normally find difficulty, but
it is inefficient in dealing with tasks that belong to our daily life (that is why it cannot be called
“intelligent”).The tasks that computers perform follow precise rules and possess an analytical
solution, the latter are difficult to describe through precise rules and it is not always possible to
derive an analytical solution. Our brain contains 10^11 processing units (neurons), each of which
communicates with 10^4 neurons. They emit a response based on the global signal received and
their own threshold of activation. There is no specific portion of the nervous system that only
stores knowledge and another that only applies rules; moreover, the decision to emit a signal is
best described by the probability of emitting the signal, and the signal transmission system is
unreliable.

SLIDE 5
Connectionism brings a new understanding of information processing in living and artificial
systems. Unlike cognitivism and its computer metaphor to describe the human mind,
connectionism proposes an approach more inspired by the workings of the brain and neural
systems. This approach is based on the notion that information processing occurs through the
network of connections between processing units, similar to biological neurons, which operate in
parallel.
In connectionism, information is not represented by abstract symbols, but is distributed in a
connectionist manner, that is, through the strength and configuration of connections between
processing units. Artificial neural networks, the foundation of connectionism, are composed of a
set of interconnected processing units (that represent artificial neurons) that work together to
solve complex tasks.

SLIDE 6
This connectionist perspective carries with it several implications. First, connectionism recognizes
that interactions between processing units are fundamental. Connections between artificial
neurons can be modified based on activity and interaction with each other. This synaptic plasticity
allows neural networks to adapt to changes in the environment and emerge with complex
behaviors. In this sense, it emphasizes the importance of experience-based learning and
adaptation. Artificial neural networks learn from input data and self-organize, adjusting the
strength of connections to improve performance in the assigned task. This phenomenon allows
neural networks to adapt to changes in the environment and emerge with complex behaviors.
In addition, connectionism proposes a new conception of memory compared to the symbolic
model of cognitivism. In connectionism, memory is not considered as a separate entity or a
centralized storage structure, as it is in traditional computers. Instead, memory is distributed in
the connections between processing units within neural networks.
In the cognitivist model, memory is often described as a sequential storage system in which
information is stored and retrieved in a linear fashion. In contrast, in connectionism, information is
represented and stored through the forces and configurations of connections between processing
units, similar to biological neurons.
SLIDE 7
This conception of distributed memory in connectionism has several advantageous features. First,
distributed memory allows simultaneous and parallel access to information. Whereas in the
cognitivist model, memory access occurs sequentially, in connectionism, different processing units
can simultaneously access different parts of the stored information. This allows for faster and
more complex processing, as different processing units work in parallel, analyzing and combining
information simultaneously.
In addition, distributed memory in connectionism comes closest to the conception of memory in
the human brain. In the brain, information is represented and stored through the interconnections
between neurons, which form complex neural networks. This differs from computer memory, in
which information is stored in specific memory locations.

SLIDE 8
We can realize how different the connectionist paradigm is from that proper to the cognitive
sciences: in a connectionist model there is not one symbol that stands for another symbol, but the
representation is dispersed into what are called principal components: the symbol is broken down
into its distinctive features, which are also common to other symbols. That is why partial damage
to a neural network does not lead to a general inefficiency of the system, unlike what would
happen in a serial processing system. So, distributed memory allows for greater robustness of the
system. Depriving the network of a certain number of units or connections results in the loss of
some microtracts that do not decisively affect the general performance of the network itself: it
may happen, depending on the extent of the damage, that one or more symbols that the system
knew how to recognize will be lost, or that the responses will have a major error, but only very
extensive damage will permanently impair the functioning of the system. We can consider this
property of reconstruction of representations as a property emerging from a more general one,
such as that of generalization, i.e., being able to place within a generic set of belonging particular
representations, which have never before been presented to the network, only on the basis of the
examples given during the learning cycle.

SLIDE 9
In 1943, McCulloch, a neurophysiologist, and Pitts, a mathematician, published a paper entitled "A
Logical Calculus of Ideas Immanent in Nervous Activity“, in which they presented the first model
of an artificial neuron and proposed the assembly of many neurons into artificial neural networks.
This model was based on the functioning of neurons in the human brain and described how an
artificial neuron could receive input signals, process them through an activation function and
produce an output. They were not capable of learning, and synaptic weights had to be
predetermined. Each neuron was characterized by a threshold value and binary activation and
performed a simple logic function.
However, the importance of this discovery lay in the creation of a fundamental theoretical building
block for the construction of artificial neural networks, opening up new perspectives for artificial
intelligence.

SLIDE 10
Some time later, a scientist named Hebb proposed a new idea about how neural networks can
learn. He developed a rule called "Hebb's rule" that explains how connections between neurons
can become stronger when neurons are activated at the same time. Hebb's rule says that if two
neurons are active together, the connection between them becomes stronger. Some time later, a
scientist named Hebb proposed a new idea about how neural networks can learn. He developed a
rule called "Hebb's rule" that explains how connections between neurons can become stronger
when neurons are active at the same time. Hebb's rule says that if two neurons are active
together, the connection between them becomes stronger.

SLIDE 11
In the 1950s and 1960s, a researcher named Rosenblatt proposed a new class of neural networks
called perceptrons. These perceptrons consisted of a single neuron that received input from a
series of light receptors. What made perceptrons special was their ability to vary the strength of
the connections between the receptors and the neuron.
Rosenblatt invented a learning algorithm that allowed perceptrons to learn autonomously to
distinguish different configurations. When the perceptron gave an incorrect response, connections
between neurons were automatically changed. The goal was to find a learning rule that would
allow the perceptron to recognize figures correctly, so that the response error would correlate
with appropriate changes in connections.
At the same time, Widrow and Hoff developed another learning rule called the "delta rule." This
learning algorithm was more powerful in its error correction mechanism and had a greater ability
to generalize to new patterns than Rosenblatt's. The basic idea was to change the values of the
connections in proportion to the difference between the neuron's response and the correct
response.