Deep learning in neuroscience,

Tutorial 3
Yalda Mohsenzadeh, PhD
About me

Assistant Professor of Computer Science

Brain and Mind Institute
Western University (Canada)

Cognitive Neuroscience & AI Lab:

○ Machine Vision and AI

○ Human Perception and Memory

○ Cognitive Computational Neuroscience

Yalda Mohsenzadeh⦁ Deep Learning Week 2 ⦁ Day 1 ⦁ Tutorial 3 2
DNNs as models of the visual System

(Khaligh-Razavi et al. 2014, Yamins et al. 2014, Guclu et al. 2015, Cichy et al. 2016; Mohsenzadeh et al. 2020)

Yalda Mohsenzadeh⦁ Deep Learning Week 2 ⦁ Day 1 ⦁ Tutorial 3 3

Using an orientation discrimination task
reference
orientation
stimulus stimulus

Train a deep network to perform this task,

are its representations of the stimuli the
same as the brain’s representations?

choose left choose right

Yalda Mohsenzadeh⦁ Deep Learning Week 2 ⦁ Day 1 ⦁ Tutorial 3 4

A neural network for orientation discrimination

Jorge A. Menendez
Yalda Mohsenzadeh⦁ Deep Learning Week 2 ⦁ Day 1 ⦁ Tutorial 3 5
Convolutional neural network (in 2D)

input
filter
output

Pooling
(K=2) Pooling ⇒ stability w.r.t.
small translations

Vincent Dumoulin, Francesco Visin - A guide to convolution arithmetic for deep learning; Jorge A. Menendez

Yalda Mohsenzadeh⦁ Deep Learning Week 2 ⦁ Day 1 ⦁ Tutorial 3 6

Pytorch code for network

Yalda Mohsenzadeh⦁ Deep Learning Week 2 ⦁ Day 1 ⦁ Tutorial 3 7

 Comparing
representations
between brains
and DNNs

Yalda Mohsenzadeh ⦁ Deep Learning Week 2 ⦁ Day 1 ⦁ Tutorial 3 8

 Example unit activations from notebook

Brain V1 Responses Model Pool Layer Responses Model FC Layer Responses

Neural Response

Stimulus Orientation Stimulus Orientation Stimulus Orientation

Yalda Mohsenzadeh⦁ Deep Learning Week 2 ⦁ Day 1 ⦁ Tutorial 3 9
Representational Dissimilarity Matrix (RDM)
neural
activity
stimulus pattern

Compute
dissimilarity
(1 - corr_coef)

Yalda Mohsenzadeh⦁ Deep Learning Week 2 ⦁ Day 1 ⦁ Tutorial 3 10

RDMs from orientation discrimination task

Yalda Mohsenzadeh⦁ Deep Learning Week 2 ⦁ Day 1 ⦁ Tutorial 3 11

Representational Similarity Analysis (RSA)
Activation Patterns

Correlation of Model Layer RDM

versus

with Brain V1 RDM

Correlation
Neural Network Layer

versus

Pool Layer FC Layer

Region of Interest Neural Network Model Layer
Yalda Mohsenzadeh⦁ Deep Learning Week 2 ⦁ Day 1 ⦁ Tutorial 3 12
Python code (RSA)

Yalda Mohsenzadeh⦁ Deep Learning Week 2 ⦁ Day 1 ⦁ Tutorial 3 13

Representational Similarity Analysis (RSA)

Brain RDM Image pixels RDM

High

Dissimilarity
Neural Net RDM Behavior RDM
Low

Yalda Mohsenzadeh⦁ Deep Learning Week 2 ⦁ Day 1 ⦁ Tutorial 3 14

What have we learned?

● How to use deep learning to build a normative encoding model of the visual
system

● How to use RSA to evaluate how the model's representations match to those
in the brain

Yalda Mohsenzadeh⦁ Deep Learning Week 2 ⦁ Day 1 ⦁ Tutorial 3 15

 T-SNE Visualization of Population Responses

Brain V1 Responses Model Pool Layer Responses Model FC Layer Responses

Stimulus Orientation

Stimulus Orientation

Stimulus Orientation
Dimension 2

Dimension 1 Dimension 1 Dimension 1

Yalda Mohsenzadeh⦁ Deep Learning Week 2 ⦁ Day 1 ⦁ Tutorial 3 16