Unit 2 - Soft Computing
Unit 2 - Soft Computing
Unit 2 - Soft Computing
ARTIFICIAL NEURAL
NETWORKS: AN
INTRODUCTION
DEFINITION OF NEURAL NETWORKS
According to the DARPA Neural Network Study (1988, AFCEA
International Press, p. 60):
Massive parallelism
Learning ability
Generalization ability
Adaptively
Fault tolerance
W1
X1 Y
W2
X2
The figure shows a simple artificial neural net with two input neurons
(X1, X2) and one output neuron (Y). The inter connected weights are
given by W1 and W2.
ASSOCIATION OF BIOLOGICAL NET
WITH ARTIFICIAL NET
PROCESSING OF AN ARTIFICIAL NET
The neuron is the basic information processing unit of a NN. It consists
of:
1. A set of links, describing the neuron inputs, with weights W1, W2,
…, Wm.
x1-x2= 1
x1
MULTI LAYER ARTIFICIAL NEURAL NET
INPUT: records without class attribute with normalized attributes
values.
HIDDEN LAYER: the number of nodes in the hidden layer and the
number of hidden layers depends on implementation.
OPERATION OF A NEURAL NET
- Bias
x0 w0j
x1 w1j
∑ f
Output y
xn wnj
All change in weights (∆wij) in the previous epoch are below some
threshold, or
Activation Function
LAYER PROPERTIES
Input Layer: Each input unit may be designated by an attribute
value possessed by the instance.
Activation functions:
(A) Identity
(F) Ramp
CONSTRUCTING ANN
Determine the network properties:
• Network topology
• Types of connectivity
• Order of connections
• Weight range
Adaptive learning
Self-organization
Real-time operation
Fault tolerance via redundant information coding
Massive parallelism
Learning and generalizing ability
Distributed representation
McCULLOCH–PITTS NEURON
Neurons are sparsely and randomly connected
All but one neuron are excitatory (tend to increase voltage of other
cells)
x1 w1
w0
w2
x2 Σ o
n
. Σ
. wn
w i xi
. i=0
n
xn 1 if Σ wi xi >0
f(xi)= { i=0
-1 otherwise
PERCEPTRON LEARNING
wi = wi + ∆wi
∆wi = η (t - o) xi
where
t = c(x) is the target value,
o is the perceptron output,
η Is a small constant (e.g., 0.1) called learning rate.
Error: The error value is the amount by which the value output by
the network differs from the target value. For example, if we
required the network to output 0 and it outputs 1, then Error = -1.
Target Value, T : When we are training a network we not only
present it with the input but also with a value that we require the
network to produce. For example, if we present the network with
[1,1] for the AND function, the training value will be 1.
Use a set of sample patterns where the desired output (given the
inputs presented) is known.
Output Values
Output Layer
Adjustable
Weights
Input Layer
Input Signals
LAYERS IN NEURAL NETWORK
The input layer:
• Introduces input values into the network.
• No activation function or other processing.
Inputs
Hiddens
I0
Outputs
h0
I1 o0
h1
I2 o1
h2 Outputs
I3 Hiddens
Inputs
MULTILAYER FEEDFORWARD NETWORK:
ACTIVATION AND TRAINING
For feed forward networks:
• A continuous function can be
• differentiated allowing
• gradient-descent.
• Back propagation is an example of a gradient-descent technique.
• Uses sigmoid (binary or bipolar) activation function.
In multilayer networks, the activation function is
usually more complex than just a threshold function,
like 1/[1+exp(-x)] or even 2/[1+exp(-x)] – 1 to allow for
inhibition, etc.
GRADIENT DESCENT
Gradient-Descent(training_examples, η)
Will find a local, not necessarily global error minimum -in practice
often works well (can be invoked multiple times with different initial
weights)
Image processing.
Signature verification.
Bioinformatics.