Backpropagation Learning algorithm
How does a multi layer neural network learn???
– Back Propagation algorithm
Backpropagation Learning
Conti… Backpropagation learning
the network output.
Neural network notation
Understanding backpropagation pseudocode
inputs:
• Network: a multilayer feed-forward neural network
• Training records: a set of training vectors with associated outputs
• Learning rate: the learning rate (alpha symbol)
• To start the algorithm, we initialize our neural network and begin looping through
the input examples (until we encounter a terminating condition or a maximum
number of epochs).
• First, we compute the output of the current network for the current input
example.
• We compare this output to the actual output associated with the input and
compute the error (example_err).
• Now we’re ready to compute the weight updates leading to the output layer.
Updating the output layer weights
for all connections in the neural n/w
we can see we are working with the neuron j weight from the previous
hidden layer connecting to the current neuron i.
We’re multiplying the learning rate α by the incoming activation from
the j neuron.
This input is calculated by getting the net input to the j neuron and
then computing the activation of neuron j.
Computing
. the total weight input to the activation function for neuron
i,
we compute the dot product of the incoming weight vector Wj and the
activation vector Aj and then add in the bias term value
Error term
This update rule is similar to how we’d update a perceptron except we’re using the activations of the previous layers as
opposed to their raw input values.
This rule also contains a term for the derivative of the activation function to get the gradient of the activation function
Updating the hidden layers With the backpropagation
algorithm, we walk back across the hidden layers, updating the
connection between each one until we reach the input layer.
To update these connections we take the input from the
fractional error value computed previously and multiply it by
the activation (input) from the connection from the previous
layer and the learning rate.
The length of these learning steps, or the amount the weights that are changed with each iteration, is
known as the learning rate.
BACKPROPAGATION AND FRACTIONAL ERROR RESPONSIBILITY (extra)
