Alex’s lectures

Email for an appointment

Alex.holcombe@sydney.edu.au
t: @ceptional

• Brains and computers (today and 10 Oct)

Readings
http://rpubs.com/alexHolcombe/PSYC3014

1
What has given you the most
understanding of how our brains
make us smart?

http://socrative.com/

Room Code: ATHK 2

 Reductionism The practice of analysing and describing a complex
phenomenon in terms of its constituents, especially
Testm
when this is said to provide a sufficient explanation.

Scale of detail Number

brain
Hundreds

Thousands
Log area
Size
(mm) column Millions

layer
neuron 86 billion

dendrite Trillions

synapse Quadrillions

ion channel Quintillions

molecule 3
ion
What does a brain do? Th
- Lots. But which aspects are essential to understand human
abilities and behavior?

“
g
b
p
a
e
h
As psychologists, we want to understand behaviour Cognitive Neuroscience: The
Biology of the Mind, 4 edition
• Do we need to know about ion channels? th

Testm

Scale of detail Number of parts

brain
Hundreds

Thousands
Log area
Size
(mm) column Millions

layer
neuron 86 billion

dendrite Trillions

synapse Quadrillions

ion channel Quintillions

molecule 5
ion
 Cognitive Neuroscience: The
• Do we need to know about ion channels? Biology of the Mind, 4 edition
th

Testm

Does a chemist need to know about quarks?

Does an electrician need to know how a light bulb works?

6
As psychologists, we want to understand behaviour
• Do we need to know about ion channels? Testm

Scale of detail Number of parts

brain
Hundreds

Thousands
Log area
Size
(mm) column Millions

layer
neuron 86 billion

dendrite Trillions

synapse Quadrillions

ion channel Quintillions

molecule 7
ion
Alex’s lectures
• Brains and computers (Today and 10 Oct)

Readings
http://rpubs.com/alexHolcombe/PSYC3014

Theories, Models
• Reductionism –C
• Knowledge of brain needed for good
theories of how it would do something
• Understand role of models in
understanding brain and behavior
• Know what computational modeling is

8
Monosynaptic stretch reflex

9
Tom Carlson slides

10
Justin Harris’ slide

11
As psychologists, we want to understand thoughts and behaviour
• Do we need to know about ion channels? Testm

Scale of detail Number of parts

brain Cognitive Neuroscience: The

Hundreds Biology of the Mind, 4 edition
th

Thousands
Log area
Size
(mm) column Millions

layer
neuron 86 billion

dendrite Trillions

synapse Quadrillions

ion channel Quintillions

molecule
ion
role of Modeling/Simulation
• Build something which mimics the thing we
are trying to understand.
• Doesn’t duplicate. Strips away irrelevant
detail
– In modeling airplane aerodynamics, can we
use a scale model in a wind tunnel?
– In electronic circuit, does spatial arrangement
of components matter?
• If we can build something that mimics the
thing, our theory wasn’t complete B.S.

16
role of Modeling/Simulation
• Build something which mimics the thing we
are trying to understand.
• Doesn’t duplicate. Strips away irrelevant
detail
– In modeling airplane aerodynamics, can we
use a scale model in a wind tunnel?
– In electronic circuit, does spatial arrangement
of components matter?
• If we can build something that mimics the
thing, our theory wasn’t complete B.S.

• Often, the model uses something more

familiar than the hard thing we’re trying to
understand. Then we call it a metaphor

17
 Metaphors for modeling

Wikimedia commons
Ian
• Plato: wax tablet carried by his student 1st century
– “our minds contain a wax block, which may vary in size, cleanliness and
consistency in different individual . . ." Socrates
– - impressionability
– wax could be too hard, soft, of full of impurities.

• Clock 17th century

– Spurred the scientific revolution

Wikimedia commons
• Clockwork automaton
– Descartes: nerves = hydraulic pipes

• 20th century:
– Computer

• 21st century: 18
– ?
As psychologists, we want to understand human abilities
Testm

There’s more than one way to Different algorithms

– Achieve human abilities.

Deep learning

19
 Alphabetise a list
Different algorithms Brooks
Kandel
McClelland
Churchland
Smith
Gibson
Cavanagh
There’s more than one way to: Cajal
• achieve human abilities Hodgkin
Kanwisher
Treisman
• How do we get on the right track to model how Dennett
humans do it? Berkeley
Locke
Hebb
• Use our knowledge of neurons to help us Moser
theorise about how brain does things O’Keefe
Hubel
Pylyshyn
Kosslyn
Gazzaniga
Varela
LeDoux
Koch
Ramachandran
Rizzolatti
Diamond
Shatz
Shepherd
Huxley
 Naïve theories
• Some things dogs can do
– Run
– Catch frisbees
– Jump
– Bark
– Pant

1. Different parts of dog brain dedicated to

these different functions?

Knowledge of the brain has led us to take a different approach.

We investigate how semi-realistic models of neurons can do what a dog or human can do.
21
22
Goal: Explain behaviour

wikimedia
We think this requires considering some
aspects of neurons.

• Do we need to worry about every ion

channel, dendrite, and synaptic vesicle
to understand how neurons mediate
intelligent behaviour?

Licensed under: work of the US federal government

• Most neuroscientists think not, and will
therefore ignore many details.

• We will build a simplified simulation of

neurons.

synapse
23
Goal: Explain behaviour

wikimedia
We think this requires considering some
aspects of neurons.

What can one neuron do?

• Do we need to worry about every ion
channel, dendrite, and synaptic vesicle
to understand how neurons mediate
intelligent behaviour?

• Most neuroscientists think not, and will

therefore ignore many details.

• We will build a simplified simulation of

neurons.

The connectionist model approach

24
Goal: Explain behaviour

wikimedia
We think this requires considering some
aspects of neurons.

What can one neuron do?

• Do we need to worry about every ion
channel, dendrite, and synaptic vesicle
to understand how neurons mediate
intelligent behaviour?

• Most neuroscientists think not, and will

therefore ignore many details.

• We will build a simplified simulation of

neurons.

The connectionist model approach

 Brains and computers (26 Sep; 10 Oct) Testm

– Connectionist neural networks

• What can one or two neurons do?
– Simple enough to understand fully – reflex,
Pavlovian learning
– How connectionist networks are a
simplification of real neural functioning
• What can several neurons do?
– Connectionist network for word recognition
and memory (birds). Understand how they
work.
• Compare to a computer
– Box-and-arrow psychological theory often is
more like a computer’s style. Understand
how network functioning differs.

25
 Brains and computers (26 Sep; 10 Oct) Testm

Metaphors, Theories,
Models
Scale of detail • Knowledge of brain needed for
good theories of how it would
brain do something
• Understand role of metaphor in
understanding brain
Log area
• Understand what computational
Size
modeling is
(mm) column

layer
neuron

dendrite

synapse

ion channel
Everything should be made
molecule as simple as possible, but
ion no simpler - Einstein
Simplified brain:
neural network model AKA connectionist model
•Simplification: we represent the ability of neurons to
make the next one fire, ignoring the details of the
neurotransmitters or the membrane potentials
Licensed under: work of the US federal government

synapse

Kosslyn, S. M. & Koenig, O. (1992). Wet Mind: The new cognitive neuroscience. (octopus example) 27
What can one neuron do?
28

• not much?
Tom Carlson’s slide
– Decision of whether to transmit
• Each synapsing cell has different
number of ‘votes’, yay
(excitatory) or nay (inhibitory)
– Temporal, spatial summation

• Whether total activation exceeds

criterion level (threshold)

– Learning: Hebbian
From Irina’s slide
– Are the above all that’s
important? So assumes
connectionist / neural network
models…
29

From PSYC2010 in 2018

Complex coordinated patterns, not computation

Tom Carlson’s slide

 Irina Harris’ slide
Parallel distributed processing (PDP) network

Category
“bird”
emerges
from overlap
among all
Canary instances
Simulating lil’ brains tutorial

How can a bunch of neurons do something smart?

a TUTORIAL
Learning Outcomes
– Know what is a connectionist neural network
• J. Harris: How neurons can mediate conditioning, motor plans, map
memory
• Holcombe: first two lectures
• I. Harris: semantic connectionist memory
– Understand how a single model neuron works
– Understand how connecting these dumb units can
yield somewhat-smart behavior
• How they can learn a new memory
– No new ‘files’ created. Instead a new pattern of activity
• How they can retrieve a learned pattern
– “Content addressable”: if partial content provided, network activates related
• How they can mediate approach behavior
• How they can accomplish XOR (if A or B, do it. But if A and B, don’t do it)
 Te

Simulating lil’ brains tutorial

32
Simulating lil’ brains tutorial:
neural network model = connectionist model

government
Licensed under: work of the US federal

Neurons are pictured as circles.

while synapses or synapse-like elements are modeled

by "weights" or "connections."
synapse The number inside is the neuron’s "activation,” perhaps
its firing rate or voltage potential.

The synaptic terminal is represented by the semicircle.

Its color indicates the connection strength.
Red = excitatory connection
Blue = inhibitory connection

33
Synaptic strength = connection strength = “weight”
Simulating lil’ brains tutorial:

Simplified neural network

Linear activation rule:

simply pass on the overall votes/ stimulation

Linear is not enough for decisions!

You won’t be “71% accepted” to honours
This morning, you didn’t 60% get up and go to class, 40% didn’t
34
Simulating lil’ brains tutorial:

Simplified neural network

Threshold activation rule for D:
If stimulation > threshold, activate

Threshold = 60

Threshold = 60
Simulating lil’ brains tutorial

36
Simulating lil’ brains tutorial

How can a bunch of neurons do something smart?

a TUTORIAL
Learning Outcomes
– Know what is a connectionist neural network
• J. Harris: How neurons can mediate conditioning, motor plans, map
memory
• Holcombe: esp. first two lectures
• I. Harris: semantic connectionist memory
– Understand how a single model neuron works
– Understand how connecting these dumb units can
yield somewhat-smart behavior
• How they can learn a new memory
– No new ‘files’ created. Instead a new pattern of activity
• How they can retrieve a learned pattern
– “Content addressable”: if partial content provided, network activates related
• How they can mediate approach behavior
• How they can accomplish XOR (if A or B, do it. But if A and B, don’t do it)
 Brains and computing (26 Sep and 10 Oct) Testm

– Connectionist neural networks Metaphors, Theories,

• What can one or two neurons do? Models
– Simple enough to understand fully – reflex,
Pavlovian learning
• Knowledge of brain needed for
– How connectionist networks are a
good theories of how it would
simplification of real neural functioning do something
• What can several neurons do? • Understand role of metaphor
– Connectionist network for word recognition in understanding brain
and memory (birds). Understand how they
work. • Understand what
• Compare to a computer computational modeling is
– Naïve computer-style box-and-arrow
psychological theory. Understand how
network functioning differs.

38
 Naïve theories
Alert
• Some things attention can do
– Enhance
– Inhibit Interrupt
– Engage
– Disengage
Localize
– Move

Disengage
(Parietal)

Move

Engage

Inhibit
1. Different parts of brains do different
Traditional box-and-arrow psychological
attentional functions
theory of attention
 Understanding neural implem Object2
Spatial2
Alert
Less-retinotopic
layers
Interrupt m
od
el Top-down
no
to activation
Localize n
ex
am Spatial1 Object1

Disengage
(Parietal) Lateral inhibition-
competition V1

Move

Engage
2. Functions are distributed? (more
likely for lots of processing
Inhibit units interconnected)
Traditional box-and-arrow
psychological theory
1. Different parts of brains do different attentional functions? 40
(like if wrote conventional software program)
 Understanding neural implem
2. Functions are distributed? model not on exam
• Neurons representing
same location mutually
excite each other
• Neurons representing Spatial2
different locations inhibit Object2
each other Less-retinotopic
layers
• Neurons that do
recognition also “do”
attention- via lateral Top-down
inhibition activation
• Lateral inhibition causes Spatial1 Object1
units on all layers
representing a single object
Lateral inhibition-
to eventually all become V1
competition
active
• Posner cuing: cue pre-
activates the location,
helping subsequent target
to win the competition
sooner
41
O’Reilly, R. C., Munakata, Y., Frank, M. J., Hazy, T. E., et al. (2012).
Computational Cognitive Neuroscience. Free: http://ccnbook.colorado.edu
42
Irina Harris’ slide