connectionist
connectionist
connectionist
2
Inspiration: The human brain
3
Biology
4
Neuron
5
How neurons work
6
How neurons work (cont’d)
7
How neurons change
8
Neurons as devices
9
How do neurons do it?
10
AI / Cognitive Science Implication
11
From a neuron to a perceptron
other units
a threshold that the sum of the weighted inputs are
compared against
the threshold has to be crossed for the unit to do something
(“fire”)
a single output to another bunch of units
what the unit decided to do, given all the inputs and its
threshold
12
A unit (perceptron)
w1
x1
x2 w2
x3 w3 O=f(y)
. y=wixi
.
. wn
xn
xiare inputs
wiare weights
wnis usually set for the threshold with xn =1 (bias)
y is the weighted sum of inputs including the
threshold (activation level)
o is the output. The output is computed using a
function that determines how far the
perceptron’s activation level is below or
above 0
13
Notes
14
Interesting questions for NNs
15
The simplest architecture: a single
perceptron
x1 w1
x2 w2
x3 w3 o
. y=wixi
.
. wn
xn
+1
x
y +1 xy
x+y-2
-2
1
x y x+y-2 output
1 1 0 1
1 0 -1 -1
0 1 -1 -1
0 0 -2 -1
17
Logical function or
+1
x
y +1 xy
x+y-1
-1
1
x y x+y-1 output
1 1 1 1
1 0 0 1
0 1 1 1
0 0 -1 -1
18
Training perceptrons
19
Training perceptrons: the intuition
20
Example: teaching the logical or function
21
b x y output
1 -1 -1 -1
1 -1 1 1
The training cycle 1
1
1
1
-1
1
1
1
22
b x y output
1 -1 -1 -1
1 -1 1 1
The final set of weights 1
1
1
1
-1
1
1
1
23
The general procedure
24
More on wi = c (d - f) xi
25
A data set for perceptron classification
26
A two-dimensional plot of the data points
27
The good news
28
The bad news: the exclusive-or problem
30
The adjustment for wki depends on the total
contribution of node i to the error at the output
31
Comments on neural networks
33
Nevertheless
34