Applications of AI

Unit 6
Applications of AI
6.1.1 Neural network
• A neuron is a cell in brain whose principle function is the collection,
Processing, and dissemination of electrical signals.
• Brains Information processing capacity comes from networks of such
neurons.
• Due to this reason some earliest AI work aimed to create such artificial
networks. (Other Names are Connectionism; Parallel distributed processing
and neural computing)
Biological neuron vs ANN
Application of AI 3
Biological neuron
Application of AI 4
Units of Neural Network
Nodes(units):
Nodes represent a cell of neural network.
Links:
Links are directed arrows that show propagation of information from one
node to another node.
Activation:
Activations are inputs to outputs from unit.
Weight:
Each link has weight associated with it which determines strength and sign
of the connection
Application of AI 5
Cont..
Activation function:
A function which is used do derive output activation from the input
activations to a given node is called activation function.
Bias Weight:
Bias weight is used to set the threshold for a unit. Unit is activated when
the weighted sum of real inputs exceeds the bias weight.
Application of AI 6
Artificial Neural Network(ANN)
•It is a system that is based on the biological neural network.
•They are a computational tool, based on the properties of biological neural systems that
attempts to mirror them.
•It is a system that is ideally suited for a wide range of tasks.
•An ANN is comprised of a network of artificial neurons (also known as "nodes").
Application of AI 7
Fig: An artificial Neuron
Application of AI
8
Cont..
•Receives n inputs Multiplied by its weight(W1 , W2 ,…, Wn ) which determines the
strength of the input vector.
•Computes the weighted sum of its input( ∑ )
•Applies an activation function to this sum of results
•Outputs result
Application of AI 9
Example 1
Application of AI 10
Example 2
6.1.2 Activation Function
•The transfer(activation) function translates the input signals to output signals.
•It defines the output of that node given an input or set of inputs.
•It is usually an abstraction representing the rate of action potential firing in the cell.
•It performs the mathematical operation on the signal output.
Cont..
The activation function is designed to meet two criteria:
1. We want the node(unit) to be “active” (near+1) when the “right ” input is
given, and “inactive”(near 0) when the “wrong” input is given.
2. The activation needs to be non-linear, otherwise the entire neural
network collapses into a single linear function.
Cont..
Although the possible number of activation functions is infinite, few of the activation
functions are regularly employed by majority of neural networks:
1. Unit step function
2. Ramp function
3. Piece-wise Linear function
4. Sigmoid function
Linear and Non-Linear[short glimpse]
• In short if out expected output reflects the
linear regression[explore] as shown below
then linear activation functions can be used:
(Top Figure).
• But in the in the second figure below linear
function will not produce the desired
results:(Middle figure)
• However, a non-linear function as shown
below would produce the desired results:
(Bottom figure)
1. Unit Step Function
•Unary Function
•Unary function is a simple type of function where the output generated is only one .
•Such that the unary function in linear function will only produce positive numbers over
the entire real number range.
Cont..
•It is most useful for the output layer of a predictive neural network.
•The mathematical function given by linear function is ;
Here, the weighted sum input of the neuron plus a linearly dependent bias
becomes the system output.
Figure: Graph of Linear Function
•Binary Function
It is mostly used in single layer networks. The output of this function is limited to one of
the two values:
Typically the binary function produces a binary value in response to the sign of the
input, emitting +1 if x is positive and 0 if not.
Figure: Graph of Binary Function
2. Ramp Function
•It is a combination of the linear activation function and the step
activation functions.
•It places the upper and lower bounds on the value of the function
produces and allows a linear response between the bounds.
The ramp function is defined by the equation:
Where y is the saturation value for the function and the points x=y and
x=-y are where the discontinuities in function exists.
Figure: Graph of Ramp Function
3. Piece-wise Linear Function
•It is also called saturating linear function and can have either a binary or
bipolar range for the saturation limits of the outputs.
•In fact, it is an approximate representation of a nonlinear function.
•It substitutes the global nonlinear function by a series of linear sub-
functions which are defined in properly partitioned sub-regions of the
original nonlinear function domain.

Figure: Graph of Piece-wise Linear Function
4. Sigmoid Function
•The non-linear curved S-shaped function.
•Most common type of activation used to construct the neural networks.
•It is mathematically well behaved, differentiable and strictly increasing
function.
•The sigmoidal function is achieved using exponential equation.
The sigmoidal transfer function can be written in the form:
This can be explained as:

0 for the large –ve input values,
1 for the large +ve values, with a smooth transition between the two.
ᾳ is slope parameter also called shape parameter.
Figure: Graph of Sigmoid Function
6.1.4 McCulloch and Pit model of Neural network
A simple mathematical model of neuron is devised by McCulloch and Pit is given in

the figure given below:
Cont..
• It fires when a linear combination of its inputs exceeds some threshold.
• A neural network is composed of nodes (units) connected by directed links. A
link from unit j to i serve to propagate the activation aj from j to i.
• Each link has some numeric weight Wj,i associated with it, which determines
strength and sign of connection.
• Each unit first computes a weighted sum of it’s inputs:
• Then it applies activation function g to this sum to derive the output:
• Here, ai output activation from unit j and Wj,i is the weight on the link j to this
node.
Cont..
Activation function is used to:
• Activate or Deactivate the units
• Makes the activation non-linear.
If the activations are linear entire neural network collapse into linear function.
Activation function is either:

• a step function or threshold function or
• is a sigmoid function
1/(1 + ex)
Realizing logic gates by using Neurons
• We can use McCulloch-Pitts neurons to implement the basic logic gates.
• All we need to do is find the appropriate connection weights and neuron
thresholds to produce the right outputs for each set of inputs.
• We shall see explicitly how one can construct simple networks that perform
NOT,
AND, and OR.
• It is then a well known result from logic that we can construct any logical
function from these three operations.
Cont..
What about XOR gate realization?

Finding Weights Analytically for the AND Network
We have two weights w1 and w2 and the threshold θ, and for each training
pattern we need to satisfy:
Neural network Architecture
• Single-Layer Feed-forward NNs
• One input layer and one output layer of processing units. No feed-back
connections. (For example, a simple Perceptron.)
• Multi-Layer Feed-forward NNs
• One input layer, one output layer, and one or more hidden layers of
processing units. No feed-back connections. The hidden layers sit in between
the input and output layers, and are thus hidden from the outside world.(For
example, a Multi-Layer Perceptron.)
• Recurrent NNs
• Any network with at least one feed-back connection. It may, or may not, have
hidden units. (For example, a Simple Recurrent Network.)
Single-layer feed-forward neural networks
a. ADALINE
• Acronym for Adaptive Lineardi Neuron or later Adaptive Linear
Element.
• Single layer ANN
• Network uses memistors(memory devices able to perform logical
operation and data processing)
• Based on McCulloch-Pitts neuron
• Consists of weight ,a bias and a summation function
• Consists of multiple nodes where each node accepts multiple inputs
and generates one output
Cont..
• Given the following variables as:
Single-layer feed-forward neural networks
b. Perceptron
• A neural network in which all the inputs connected directly to the outputs is
called a single-layer neural network, or a perceptron network.
• Since each output unit is independent of the others each weight affects only
one of the outputs
• Supervised learning technique
i. Set the weight and thresholds of the neuron to random values.
ii. present an input.
iii. Calculate the output of the neuron.
iv. Alter the weights to reinforce correct decisions and discourage wrong decisions, hence reducing
the error. So for the network to learn we shall increase the weights on the active inputs when we
want the output to be active, and to decrease them when we want the output to be inactive.
v. Now present the next input and repeat steps iii. - v.
Perceptron NN
Question of the day????
• Did you point out the major differences between Adaline and
perceptron?
Multilayer neural networks (perceptrons)
• The neural network which contains input layers, output layers and
some hidden layers also is called multilayer neural network.
• The advantage of adding hidden layers is that it enlarges the space of
hypothesis (???).
• Layers of the network are normally fully connected.
Multilayer NN
• Advantage of adding Hidden layer in multilayer neural network??
• Explore:
• Underfitting and overfitting
• One of the type of Supervised Learning we left in
Chapter 5.
• Can any of you spell its name????
Back propagation Algorithm
• It is a supervised learning method, and is an implementation of the Delta
rule.
• It requires a teacher that knows, or can calculate, the desired output for any
given input.
• The term is an abbreviation for "backwards propagation of errors".
• Backpropagation requires that the activation function used by the
artificial neurons (or "nodes") is differentiable(calculate slope)
• As the algorithm's name implies, the errors (and therefore the learning)
propagate backwards from the output nodes to the inner nodes.
• So technically speaking, backpropagation is used to calculate the gradient of
the error of the network with respect to the network's modifiable weights.
Cont..
•Backpropagation usually allows quick convergence on satisfactory
local minima for error in the kind of networks to which it is suited.
•Backpropagation networks are necessarily multilayer perceptrons
(usually with one input, one hidden, and one output layer).
•In order for the hidden layer to serve any useful function, multilayer
networks must have non-linear activation functions for the multiple
layers.
Algorithm[Alternate]
•Phase 1: Propagation
•Each propagation involves the following steps:
•Forward propagation of a training pattern's input through the
neural network in order to generate the network's output value(s).
•Backward propagation of the propagation's output activations
through the neural network using the training pattern target in
order to generate the deltas (the difference between the targeted
and actual output values) of all output and hidden neurons.
•.
Cont..
• Phase 2: Weight update
• For each weight, the following steps must be followed:
• The weight's output delta and input activation are multiplied to find the gradient
of the weight.
• A ratio (percentage) of the weight's gradient is subtracted from the weight.
• This ratio (percentage) influences the speed and quality of learning; it is called the
learning rate.
• The greater the ratio, the faster the neuron trains, but the lower the ratio, the
more accurate the training is.
• The sign of the gradient of a weight indicates whether the error varies directly
with, or inversely to, the weight. Therefore, the weight must be updated in the
opposite direction, "descending" the gradient.
• Phases 1 and 2 are repeated until the performance of the network is satisfactory
Algorithm:
• Step 0: Initialize the weights to small random values

• Step 1: Feed the training sample through the network and determine the
final output
• Step 2: Compute the error for each output unit, for unit k it is:
• Step 3: Calculate the weight correction term for each output unit, for unit
k it is:
Algorithm cont..
• Step 4: Propagate the delta terms (errors) back through the weights
of the hidden units where the delta input for the jth hidden unit is:
The delta term for jth hidden unit is:

• Step 5: Calculate the weight correction term for the hidden units:
• Step 6: Update the weights of output layer and hidden layer.

• Step 7: Test for stopping (maximum cycles, small changes, etc)
Note:
• There are a number of options in the design of a backpropagation
system;
– Initial weights – best to set the initial weights (and all other free
parameters) to random numbers inside a small range of values (say –0.5 to
0.5)
– Number of cycles – tend to be quite large for backpropation systems
– Number of neurons in the hidden layer – as few as possible
Summary of the backpropagation technique:
1. Present a training sample to the neural network.

2. Compare the network's output to the desired output from that sample.
Calculate the error in each output neuron.
3. For each neuron, calculate what the output should have been, and a
scaling factor, how much lower or higher the output must be adjusted to
match the desired output. This is the local error.
4. Adjust the weights of each neuron to lower the local error.
5. Assign "blame" for the local error to neurons at the previous level, giving
greater responsibility to neurons connected by stronger weights.
6. Repeat from step 3 on the neurons at the previous level, using each one's
"blame" as its error.
Application of Artificial Neural network??
• Brain modeling
• Models of human development – help children with developmental problems
• Simulations of adult performance – aid our understanding of how the brain works
• Neuropsychological models – suggest remedial actions for brain damaged patients
• Real world applications

• Financial modeling – predicting stocks, shares, currency exchange rates
• Other time series prediction – climate, weather, airline marketing tactician
• Computer games – intelligent agents, backgammon, first person shooters
• Control systems – autonomous adaptable robots, microwave controllers
• Pattern recognition – speech recognition, hand-writing recognition, sonar signals
• Data analysis – data compression, data mining
• Noise reduction – function approximation, ECG noise reduction
• Bioinformatics – protein secondary structure, DNA sequencing

Applications of AI

Uploaded by

Copyright:

Available Formats

Applications of AI

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Applications of AI

Uploaded by

Copyright:

Available Formats

Unit 6

•It is a system that is based on the biological neural network.

attempts to mirror them.

•It is a system that is ideally suited for a wide range of tasks.

•An ANN is comprised of a network of artificial neurons (also known as "nodes").

strength of the input vector.

•Computes the weighted sum of its input( ∑ )

•Applies an activation function to this sum of results

•The transfer(activation) function translates the input signals to output signals.

•It performs the mathematical operation on the signal output.

functions are regularly employed by majority of neural networks:

1. Unit step function

3. Piece-wise Linear function

the entire real number range.

•The mathematical function given by linear function is ;

becomes the system output.

•It is a combination of the linear activation function and the step

produces and allows a linear response between the bounds.

x=-y are where the discontinuities in function exists.

bipolar range for the saturation limits of the outputs.

•In fact, it is an approximate representation of a nonlinear function.

•It substitutes the global nonlinear function by a series of linear sub-

functions which are defined in properly partitioned sub-regions of the

original nonlinear function domain.

•The non-linear curved S-shaped function.

•Most common type of activation used to construct the neural networks.

•It is mathematically well behaved, differentiable and strictly increasing

•The sigmoidal function is achieved using exponential equation.

This can be explained as:

A simple mathematical model of neuron is devised by McCulloch and Pit is given in

• Then it applies activation function g to this sum to derive the output:

Activation function is either:

What about XOR gate realization?

• Given the following variables as:

• Step 0: Initialize the weights to small random values

The delta term for jth hidden unit is:

• Step 6: Update the weights of output layer and hidden layer.

1. Present a training sample to the neural network.

• Real world applications

You might also like