Dialogues on the role of top-down factors in sensory processing

Does the human brain implement predictive coding?

Karl Friston, University College London

How much about our interaction with – and experience of – our world can be deduced from basic
principles? This talk reviews recent attempts to understand the self-organised behaviour of embodied
agents, like ourselves, as satisfying basic imperatives for sustained exchanges with the environment.
In brief, one simple driving force appears to explain many aspects of action and perception. This
driving force is the minimisation of surprise or prediction error. In the context of perception, this
corresponds to Bayes-optimal predictive coding that suppresses exteroceptive prediction errors.
In the context of action, motor reflexes can be seen as suppressing proprioceptive prediction errors.
We will look at some of the phenomena that emerge from this scheme, such as hierarchical message
passing in the brain and the ensuing perceptual inference.
.
 Overview

The anatomy of inference graphical models

canonical microcircuits

Functional asymmetries spectral connections

modulatory connections

Action and perception inference and uncertainty

simulations of saccadic searches
 “Objects are always imagined as being present in the field of
vision as would have to be there in order to produce the same
impression on the nervous mechanism” - von Helmholtz

Hermann von Helmholtz Richard Gregory

Geoffrey Hinton

The Helmholtz machine and the

Bayesian brain

Thomas Bayes Richard Feynman

 “Objects are always imagined as being present in the field of
vision as would have to be there in order to produce the same
impression on the nervous mechanism” - von Helmholtz

Hermann von Helmholtz Richard Gregory

sensory impressions…

Plato: The Republic (514a-520a)

Bayesian filtering and predictive coding

prediction update

prediction error
 Minimizing prediction error

sensations – predictions

Prediction error

Action Perception
Change sensations Change predictions
 Generative models

what where
A simple hierarchy

Sensory
fluctuations
 From models to perception

A simple hierarchy
Generative model

Descending
predictions

ModelPredictive
inversion (inference)
coding

Expectations:

Ascending
prediction errors
Predictions:

Prediction errors:
 Canonical microcircuits for predictive coding

Haeusler and Maass: Cereb. Cortex 2006;17:149-162 Bastos et al: Neuron 2012; 76:695-711
 David Mumford
Predictive coding with reflexes Action

oculomotor
signals
reflex arc
proprioceptive input
pons

retinal input
Perception

Errors (superficial pyramidal cells)

frontal eye fields geniculate

Top-down or backward
predictions

Expectations (deep pyramidal cells)

Bottom-up or forward
prediction error
visual cortex
 Overview

The anatomy of inference graphical models

canonical microcircuits

Functional asymmetries spectral connections

modulatory connections

Action and perception inference and uncertainty

simulations of saccadic searches
 Errors (superficial pyramidal cells)
Forward transfer function
14

12

10

spectral power
Expectations (deep pyramidal cells) 8

4
Andre Bastos
2

0
0 20 40 60 80 100

0.3 V4 V1
0.25

0.2

0.15 superficial
0.1

0.05

0
0 20 40 60 80 100 120

spectral power
4
2
3

2
deep
1
1

0
0 20 40 60 80 100
frequency (Hz)

0
0 20 40 60 80 100 120

frequency (Hz) Backward transfer function

 Errors (superficial pyramidal cells)

Expectations (deep pyramidal cells)

Linear or driving connections

superficial

Nonlinear or modulatory
connections

deep

NMDA receptor density

 Interim summary

Hierarchical predictive coding is a neurobiological plausible scheme that the brain

might use for (approximate) Bayesian inference about the causes of sensations

Predictive coding requires the dual encoding of expectations and errors, with
reciprocal (neuronal) message passing

Much of the known neuroanatomy and neurophysiology of cortical architectures is

consistent with the requisite message passing

In particular, the functional asymmetries and laminar specificity of intrinsic and

extrinsic connections provide a formal perspective on spectral asymmetries and
nonlinear coupling in the brain.
 Overview

The anatomy of inference graphical models

canonical microcircuits

Functional asymmetries spectral connections

modulatory connections

Action and perception inference and uncertainty

simulations of saccadic searches
 Sampling the world to minimise uncertainty

Free energy minimisation Expected uncertainty

Likelihood World model Prior beliefs

“I am [ergodic] therefore I think”  “I think therefore I am [ergodic]”

 Sampling the world to minimise uncertainty

Free energy minimisation Expected uncertainty

stimulus visual input salience sampling

Perception as hypothesis testing – saccades as experiments

Parietal (where)
Frontal eye fields

Visual cortex

Fusiform (what) Pulvinar salience map

oculomotor reflex arc Superior colliculus

Saccadic eye movements

Saccadic fixation and salience maps

Action (EOG)
2

Hidden (oculomotor) states 0

-2
200 400 600 800 1000 1200 1400
time (ms)

Visual samples

Posterior belief
5

Conditional expectations 0

about hidden (visual) states -5

200 400 600 800 1000 1200 1400

time (ms)

And corresponding percept

Hermann von Helmholtz

“Each movement we make by which we alter the appearance of

objects should be thought of as an experiment designed to test
whether we have understood correctly the invariant relations of
the phenomena before us, that is, their existence in definite
spatial relations.”

‘The Facts of Perception’ (1878) in The Selected Writings of Hermann von

Helmholtz, Ed. R. Karl, Middletown: Wesleyan University Press, 1971 p. 384
 Thank you
And thanks to collaborators:

Rick Adams
Andre Bastos
Sven Bestmann
Harriet Brown
Jean Daunizeau
Mark Edwards
Xiaosi Gu
Lee Harrison
Stefan Kiebel
James Kilner
Jérémie Mattout
Rosalyn Moran
Will Penny
Lisa Quattrocki Knight
Klaas Stephan

And colleagues:

Andy Clark
Peter Dayan
Jörn Diedrichsen
Paul Fletcher
Pascal Fries
Geoffrey Hinton
James Hopkins
Jakob Hohwy
Henry Kennedy
Paul Verschure
Florentin Wörgötter

And many others