Neural Engineering Computation Representation and
Neural Engineering Computation Representation and
net/publication/3207658
CITATIONS READS
54 5,735
1 author:
Jag Sarangapani
Missouri University of Science and Technology
516 PUBLICATIONS 14,005 CITATIONS
SEE PROFILE
All content following this page was uploaded by Jag Sarangapani on 28 June 2013.
http://compneuro.uwaterloo.ca
MIT Press
Jan. 2003
I also want to acknowledge my interactions
with David Van Essen over the past 19
years and the support he has given me over
the last 10.
Implementation
“Theorems about Turing machines and
universal computation are only true in the
limit of infinite resources and infinite time.”
(John Denker and Yann LeCun, Bell Labs)
Vestibular Signals:
1. Angular Head Acceleration
2. Linear Head Acceleration
3. 6 Degrees of Freedom
4. ~1000 Neurons
Neuronal Systems are Analog
A/D
Converter
Bit Variables
Bit Representation
Neuronal Representation
System Variables
Neuron Variables
Neuronal Encoding
Neuronal Representation
System Variables
Neuron Variables
Neuronal Linear Decoding
Population Vector:
Georgopolous et al.
600
500
Firing Rate
400
Encoding 300
200
100
0
-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1
0.8
Estimate
0.6
Actual
0.4
0.2
Decoding
0
-0.2
(Linearity) -0.4
-0.6
-0.8
-1
-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1
x
Decoding Errors
Responses
Encode Decode
+Noise
+Spikes
Neuron Ensemble
1) Noise ~ 1/sqrt(N)
2) Static distortion ~1/N
3) Residuals from filtered spikes ~1/sqrt(N)
0
10
Distortion
Noise
Total
1/2
1/N
1/N
-1
10
RMS ERROR
RMS Errors
-2
10
-3
10
0 1 2 3
10 10 10 10
Number of Neurons
Number
Bits vs. Neurons
• Bits: (Almost infinite S/N ratio)
1. Precision scales as 2N, (N=#bits).
2. Fragile and costly in power.
Bits vs. Neurons
• Bits: (Almost infinite S/N ratio)
1. Precision scales as 2N, (N=#bits).
2. Fragile and costly in power.
• Neurons: (S/N Ratio ~5/1)
1. Precision scales as sqrt(N), (N=#Neurons).
2. Fails gracefully,
runs slower and/or loses precision…
Neurons work at the level of bits!!!
3 Key Elements of Framework
REPRESENTATION
(1)The higher-level variables are encoded
into the neuronal variables using nonlinear
spike generation rules,
Neuronal Representations
• Neuronal Variables: neuron outputs Examples
(as instantaneous firing rates or spike trains)
REPRESENTATION
(1)The higher-level variables are encoded
into the neuronal variables using nonlinear
spike generation rules,
(2) And decoded from the neuron variables
using linear decoding, where a weighted
average is taken over neural populations and
over time, i.e. linear filters.
Moving on to Spiking Neurons
Leaky Integrate and Fire Spiking Model
200
Firing Rate
150
On
100
50
x
n
irg
F
te
a
R
-50
-100
-150
-200
-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1
Input voltage
200
Firing Rate
150
Off On
100
50
x
n
irg
F
te
a
R
-50
-100
-150
-200
-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1
Input voltage
200
Firing Rate
150
Off On
100
50
x
n
irg
F
te
a
R
-50
-100
-150 Diff
-200
-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1
Input voltage
Reminder to run Matlab
On-Off Pair
2
1
On-Off Pair
--1
0
2<
-1
-2
0 0.05 0.1 0.15 0.2 0.25
1
Neurons
0.5
ns2
0
E
2nd 100
-0
.5
-1
0 0.05 0.1 0.15 0.2 0.25
time(sec)
On-Off Pair
2
1
On-Off Pair
--1
0
2<
-1
-2
0 0.01 0.02 0.03 0.04 0.05 0.06 0.07 0.08 0.09 0.1
0.6
0.4
2nd 100EnNeurons
0.2
s2
-0
.2
-0
.4
-0
.6
-0
.8
0 0.01 0.02 0.03 0.04 0.05 0.06 0.07 0.08 0.09 0.1
time(sec)
100 Neurons
1
0.5
100 Neurons
--1
0
2<
-0
.5
-1
0 0.01 0.02 0.03 0.04 0.05 0.06 0.07 0.08 0.09 0.1
0.6
0.4
2nd 100EnNeurons
0.2
s2
-0
.2
-0
.4
-0
.6
-0
.8
0 0.01 0.02 0.03 0.04 0.05 0.06 0.07 0.08 0.09 0.1
time(sec)
Linear decoding can do more
than support representation.
Population Representation
• Diversity of neuronal responses is important!
Representation 700
Neuron Tuning Curves
600
500
Firing Rate
400
Encoding 300
200
100
0
-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1
0.8
0.6
Estimate
Actual
0.4
0.2
Decoding
0
-0.2
(Linearity) -0.4
-0.6
-0.8
-1
-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1
x
Population Representation
• Linear decoders also support functions of x.
Representation 700
Neuron Tuning Curves Extracting 1
0.8
500
Firing Rate
0.7
0.6
400
0.5
Encoding 300
Quadratic 0.4
0.3
200
0.2
100
0.1
0 0
-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1 -1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1
x x
1 1
0.8 0.8
0.6
Estimate 0.6
Actual
0.4 0.4
0.2 0.2
Decoding Cubic
0 0
-0.2 -0.2
(Linearity) -0.4
-0.6
-0.4
-0.6
-0.8 -0.8
-1 -1
-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1 -1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1
x x
Neuron activities, ai(x)
0.35
0.3
0.25
0.2
0.15
0.1
0.05
0
-1 -0.5 0 0.5 1
x
Principle Components, ψ n(x)
4
-1
-2 ψ 1(x)
ψ 2(x)
-3 ψ 3(x)
ψ (x)
4
-4
-1 -0.5 0 0.5 1
x
ai(x) Along Principle Component Axes
0
ψ4(x)
-1
-2
2
4
0 2
0
-2
-2 -4
ψ3(x) ψ2(x)
Computing Functions by Linear Decoding
Moving on
3 Key Elements of Framework
u(t) x(t)
dx/dt
B 1/s
A
Neural Dynamics and
Control Theory
• Modern linear control theory
u(t) x(t)
dx/dt
1 B 1/s
h( s ) =
s
A
• Neural Dynamics
1
h( s ) = u(t) x(t)
1 + sτ syn B' 1/(1+sτ)
A'
Neural Dynamics
u(t) x(t)
B' 1/(1+sτ)
A'
• Filter - Integrator
• Oscillator
Neural Dynamics
u(t) x(t)
B' 1/(1+sτ)
A'
• Filter - Integrator
• Oscillator
higher-level description
hsyn(t) neuron
encoding soma output
input spikes
spikes decoding dynamics
matricies
hsyn(t)
dendrites
neuronal-level description
recurrent connections
Synthesis (details)
higher-level description
neuron
encoding soma output
input spikes
spikes decoding dynamics
matricies
dendrites
neuronal-level description
recurrent connections
Applications
• Sensory Systems:
1. Visual System (Brandon Westover, David Van Essen)
2. Sound Localization in Barn Owl (Brian Fischer)
3. Vestibular System (Dora Angelaki)
• Motor Systems:
1. Neural Integrator (Chris Eliasmith)---{S. Seung}
2. Lamprey (Chris Eliasmith)
3. Arm movements (Zoran Nenadic)
• Cortical Systems:
1. V1 (Brandon, Greg DeAngelis and David Van Essen)
2. MT (Harris Nover and Greg DeAngelis)
3. PDF’s (John Clark, Mike Barber)
Major Unanswered Questions
http://compneuro.uwaterloo.ca
Published by
MIT Press, on
Jan. 2003
http://compneuro.uwaterloo.ca
Neural-Simulations Matlab
http://compneuro.uwaterloo.ca
1. Gen_ensemble.m
• Creates neuronal ensembles that represent vector spaces.
2. Setupsim.m
• Sets up a simulation from a text file describing the
circuit.
3. Runsim.m
• Runs the simulation.
Ensemble 1 Ensemble 3
x(t) x y y(t)
k*y
[y,k]
k k(t)
Ensemble 4
Ensemble 2
Dynamic Filter.txt Connections
0.5
Outputs
0
0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5
1
0.5
3 <-- 4
-0.5
0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5
1
4 <-- 2
0.5
0
0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5
0.5
4 <-- 3
-0.5
0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5
time (sec)
Spike rasters
View publication stats