Scribd
Upload
40 views16 pages

DeepLearning Tutorial3

This document discusses using deep neural networks (DNNs) as models of the visual system and comparing their representations to those in the brain. It describes training a DNN on an orientation discrimination task and using representational similarity analysis to compare the network's internal representations to neural responses recorded from brain area V1. The analysis shows that the pooling layer representations best match V1, while the fully connected layer representations diverge. This demonstrates that DNNs can be used as normative models of the visual system and RSA is an effective way to evaluate how similar the models' representations are to those in the brain.

Uploaded by

Bharath kumar
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
40 views16 pages

DeepLearning Tutorial3

This document discusses using deep neural networks (DNNs) as models of the visual system and comparing their representations to those in the brain. It describes training a DNN on an orientation discrimination task and using representational similarity analysis to compare the network's internal representations to neural responses recorded from brain area V1. The analysis shows that the pooling layer representations best match V1, while the fully connected layer representations diverge. This demonstrates that DNNs can be used as normative models of the visual system and RSA is an effective way to evaluate how similar the models' representations are to those in the brain.

Uploaded by

Bharath kumar
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 16

Deep learning in neuroscience,

Tutorial 3
Yalda Mohsenzadeh, PhD
About me

Assistant Professor of Computer Science


Brain and Mind Institute
Western University (Canada)

Cognitive Neuroscience & AI Lab:

○ Machine Vision and AI

○ Human Perception and Memory

○ Cognitive Computational Neuroscience


Yalda Mohsenzadeh⦁ Deep Learning Week 2 ⦁ Day 1 ⦁ Tutorial 3 2
DNNs as models of the visual System

(Khaligh-Razavi et al. 2014, Yamins et al. 2014, Guclu et al. 2015, Cichy et al. 2016; Mohsenzadeh et al. 2020)

Yalda Mohsenzadeh⦁ Deep Learning Week 2 ⦁ Day 1 ⦁ Tutorial 3 3


Using an orientation discrimination task
reference
orientation
stimulus stimulus

Train a deep network to perform this task,


are its representations of the stimuli the
same as the brain’s representations?

choose left choose right

Yalda Mohsenzadeh⦁ Deep Learning Week 2 ⦁ Day 1 ⦁ Tutorial 3 4


A neural network for orientation discrimination

Jorge A. Menendez
Yalda Mohsenzadeh⦁ Deep Learning Week 2 ⦁ Day 1 ⦁ Tutorial 3 5
Convolutional neural network (in 2D)

input
filter
output

Pooling
(K=2) Pooling ⇒ stability w.r.t.
small translations

Vincent Dumoulin, Francesco Visin - A guide to convolution arithmetic for deep learning; Jorge A. Menendez

Yalda Mohsenzadeh⦁ Deep Learning Week 2 ⦁ Day 1 ⦁ Tutorial 3 6


Pytorch code for network

Yalda Mohsenzadeh⦁ Deep Learning Week 2 ⦁ Day 1 ⦁ Tutorial 3 7


Comparing
representations
between brains
and DNNs

Yalda Mohsenzadeh ⦁ Deep Learning Week 2 ⦁ Day 1 ⦁ Tutorial 3 8


Example unit activations from notebook

Brain V1 Responses Model Pool Layer Responses Model FC Layer Responses


Neural Response

Stimulus Orientation Stimulus Orientation Stimulus Orientation


Yalda Mohsenzadeh⦁ Deep Learning Week 2 ⦁ Day 1 ⦁ Tutorial 3 9
Representational Dissimilarity Matrix (RDM)
neural
activity
stimulus pattern

Compute
dissimilarity
(1 - corr_coef)

Yalda Mohsenzadeh⦁ Deep Learning Week 2 ⦁ Day 1 ⦁ Tutorial 3 10


RDMs from orientation discrimination task

Yalda Mohsenzadeh⦁ Deep Learning Week 2 ⦁ Day 1 ⦁ Tutorial 3 11


Representational Similarity Analysis (RSA)
Activation Patterns

Correlation of Model Layer RDM


versus

with Brain V1 RDM


Correlation
Neural Network Layer

versus

Pool Layer FC Layer


Region of Interest Neural Network Model Layer
Yalda Mohsenzadeh⦁ Deep Learning Week 2 ⦁ Day 1 ⦁ Tutorial 3 12
Python code (RSA)

Yalda Mohsenzadeh⦁ Deep Learning Week 2 ⦁ Day 1 ⦁ Tutorial 3 13


Representational Similarity Analysis (RSA)

Brain RDM Image pixels RDM

High

Dissimilarity
Neural Net RDM Behavior RDM
Low

Yalda Mohsenzadeh⦁ Deep Learning Week 2 ⦁ Day 1 ⦁ Tutorial 3 14


What have we learned?

● How to use deep learning to build a normative encoding model of the visual
system

● How to use RSA to evaluate how the model's representations match to those
in the brain

Yalda Mohsenzadeh⦁ Deep Learning Week 2 ⦁ Day 1 ⦁ Tutorial 3 15


T-SNE Visualization of Population Responses

Brain V1 Responses Model Pool Layer Responses Model FC Layer Responses

Stimulus Orientation

Stimulus Orientation

Stimulus Orientation
Dimension 2

Dimension 1 Dimension 1 Dimension 1


Yalda Mohsenzadeh⦁ Deep Learning Week 2 ⦁ Day 1 ⦁ Tutorial 3 16

You might also like