Self-Organizing Map

Laxmidhar Behera
Department of Electrical Engineering
Indian Institute of Technology Kanpur
 What is a Self Organizing Map?

Supervised Learning: There is a target output for each input

pattern, and the network learns to produce the required outputs.

Unsupervised Learning: The networks learn to form their own classifications of the
training data without external help. To do this we have to assume that class membership
is broadly defined by the input patterns sharing common features, and that the network
will be able to identify those features across the range of input patterns.

Self-Organizing Map SOM: This is unsupervised learning based on competition. Here the
output neurons compete amongst themselves to be activated, and the winner takes the most
while neighbours have say in the decision making based on the lattice distance from the
winner. Such competition can be induced/implemented by having lateral inhibition
connections (negative feedback paths) between the neurons. The result is that the neurons are
forced to organize themselves.
 Topographic Maps
Neurobiological Motivation: Different sensory inputs (motor, visual, auditory, etc.) are
mapped onto corresponding areas of the cerebral cortex in an orderly fashion. i.e. inputs
are represented by topologically ordered computational maps
This form of map, known as a topographic map, has two important properties:
• At each stage of representation, or processing, each piece of incoming information is kept
in its proper context/ neighborhood.
• Neurons dealing with closely related pieces of information are kept close together so that
they can interact via short synaptic connections.

The principle of topographic map formation: “The spatial location of an output

neuron in a topographic map corresponds to a particular domain or feature drawn
from the input space”.

We will confine to KSOM introduced by the Finnish professor Teuvo

Kohonen in the 1980s is sometimes called a Kohonen map or network.
 The self-organization process involves four major components:

Initialization: All the connection weights are initialized with small random values.
Competition: For each input pattern, the neurons compute their respective values of a
discriminant function which provides the basis for competition. The particular neuron
with the smallest value of the discriminant function is declared the winner.

Cooperation: The winning neuron determines the spatial location of

a topological neighbourhood of excited neurons, thereby providing
the basis for cooperation among neighbouring neurons.

Adaptation: The excited neurons decrease their individual values of the

discriminant function in relation to the input pattern through suitable
adjustment of the associated connection weights, such that the response
of the winning neuron to the subsequent application of a similar input
pattern is enhanced
Cluster Formation without the preservation of Topology
A 2-d Kohonen Self Organizing map
Competition
Cooperation
2-d Input Space
 Evolution of Weights
Final Weights
L Shaped Input Space

Evolution of Weights

Initial Weights
Input Space
 Final Weights
Effect of the updating process: Moves the synaptic weight vector wi of winning
neuron i toward the input vector x. Weight vectors tend to follow the distribution of the
input vectors i.e it forms clusters.

In addition, the algorithm leads to a topological ordering of feature map in input space
(ie., adjacent neurons have similar weight vectors)
Evolution of Weights
Hand-Eye Coordination
Self organization has two
identifiable stages: ordering
and convergence.

Kohonen, Teuvo (1982). "Self-

Organized Formation of
Topologically Correct Feature
Maps". Biological Cybernetics.
43 (1): 59–69
Assignment: Solve n-city travelling salesman
problem using 1-d Kohonen SOM
Thank you