ARTIFICIAL INTELLIGENCE

(CSC4101)
-INTRODUCTION
-MACHINE LEARNING AND OPTIMIZATION MODULE
 OUTLINE
• What are Artificial Neural Networks?
• How Neurons perform calculation.
• Mathematical equations.
• Single Layer Perceptron & Derivation.
• Numerical of SLP.
• MATLAB/Program of SLP.

ARTIFICIAL INTELLIGENCE (CSC4101) 05/01/2021 2

MODELS
AN IDEA TO MAKE COMPUTERS IMITATE HUMAN BRAIN

ARTIFICIAL INTELLIGENCE (CSC4101) 05/01/2021 3

 HUMAN BRAIN
The brain is one of the largest and most complex organs in the human body. It is
made up of approx. 100 billion nerves that communicate in trillions of connections
called
synapses.

ARTIFICIAL INTELLIGENCE (CSC4101) 05/01/2021 4

 ARTIFICIAL NEURAL NETWORKS (ANN)
• Neural Networks (NNs) are networks of neurons, for example, as found in real (i.e.
biological) brains.
• Artificial Neurons are basic approximations of the neurons found in brains. They may
be physical devices, or purely mathematical constructs.
• Artificial Neural Networks (ANNs) are networks of Artificial Neurons, and hence
constitute basic approximations to parts of real brains. They may be physical devices,
or simulated on conventional computers.
• From a practical point of view, an ANN is just a parallel computational system
consisting of many simple processing elements connected together in a specific way in
order to perform a particular task (classification).
ARTIFICIAL INTELLIGENCE (CSC4101) 05/01/2021 5
 IMPORTANCE OF ANN
• They are extremely powerful computational method.
• Massive parallelism makes them very efficient.
• They can learn and generalize from training data – so there is no need for
enormous skills of programming.
• They are particularly fault tolerant – this is equivalent to the “graceful
degradation” found In biological systems.
• They are very noise tolerant – so they can cope with situations where normal
symbolic systems would have difficulty.

ARTIFICIAL INTELLIGENCE (CSC4101) 05/01/2021 6

 APPLICATIONS OF ANN
• Financial modeling – predicting/forecasting growth, stocks, shares, currency
exchange rates
• Other time series prediction – climate, weather, catastrophes, etc.
• Computer games – intelligent agents, backgammon, first person shooters
• Control systems – autonomous adaptable robots, path planning, etc.
• Pattern recognition – speech recognition, hand-writing recognition, sonar signals
• Data analysis – data compression, data mining, PCA, GTM
• Noise Reduction – Function approximation, E.g. ECG noise réduction
• Bioinformatics – protein secondary structure, DNA sequencing
ARTIFICIAL INTELLIGENCE (CSC4101) 05/01/2021 7
 LEARNING IN ANN
• There are many forms of neural networks. Most operate by passing neural
‘activations’ through a network of connected neurons.
• One of the most powerful features of neural networks is their ability to
learn
and generalize from a set of training data. They adapt the strengths/weights
of the connections between neurons so that the final output activations are
correct. There are three broad types of learning:
1. Supervised Learning “God then taught Adam
2. Reinforcement learning the names of all things”
3. Unsupervised learning
ARTIFICIAL INTELLIGENCE (CSC4101) 05/01/2021 8
 NEURONS – BIOLOGICAL AND ARTIFICIAL
The Nervous System: The human nervous system can be broken down into three stages
that may be represented (in block diagram form) as:
1. The receptors collect information from the environment – e.g. photons on the retina.
2. The effectors generate interactions with the environment – e.g. activate muscles.
3. The flow of information/activation is represented by arrows – feedforward and
feedback.
Naturally, in this module we will be primarily concerned with the neural network
which is
in the middle.

ARTIFICIAL INTELLIGENCE (CSC4101) 05/01/2021 9

 NEURONS – BIOLOGICAL AND ARTIFICIAL
(CONT.)

ARTIFICIAL INTELLIGENCE (CSC4101) 05/01/2021 10

 NEURONS – BIOLOGICAL AND ARTIFICIAL
(CONT.)
1. There are approximately 100 billion neurons in the human cortex, compared
with few processors in the most powerful parallel computers.
2. Each biological neuron is connected to several thousands of other neurons,
similar to the connectivity in powerful parallel computers.
3. Lack of processing units can be compensated by speed. The typical operating
speeds of biological neurons is measured in milliseconds (10-3 s), while a silicon
-9
chip can operate in nanoseconds (10 s).

ARTIFICIAL INTELLIGENCE (CSC4101) 05/01/2021 11

 NEURONS – BIOLOGICAL AND ARTIFICIAL
(CONT.)
4. The human brain is extremely energy efficient, using approximately 10-16 joules
per operation per second, whereas the best computers today use around 10-6
joules per operation per second.
5. Brains have been evolving for tens of millions of years, computers have been
evolving for tens of decades.

ARTIFICIAL INTELLIGENCE (CSC4101) 05/01/2021 12

 NEURONS – BIOLOGICAL AND ARTIFICIAL
1. (CONT.)
The majority of neurons encode their activations or outputs as a series of brief
electrical pulses (i.e. spikes or action potentials).
2. The neuron’s cell body (soma) processes the incoming activations and converts
them into output activations.
3. The neuron’s nucleus contains the genetic material in the form of DNA. This
exists in most types of cells, not just neurons.
4. Dendrites are fibers which emanate from the cell body and provide the
receptive zones that receive activation from other neurons.

ARTIFICIAL INTELLIGENCE (CSC4101) 05/01/2021 13

 NEURONS – BIOLOGICAL AND ARTIFICIAL
4. (CONT.)
Axons are fibers acting as transmission lines that send activation to other
neurons.
5. The junctions that allow signal transmission between the axons and dendrites are
called synapses. The process of transmission is by diffusion of chemicals
called neurotransmitters across the synaptic cleft.

ARTIFICIAL INTELLIGENCE (CSC4101) 05/01/2021 14

 NEURONS – THE MCCULLOCH-PITTS
NEURON

Logical Unit:
1. A set of synapses (i.e. connections) brings in
activations from other neurons.
2. A processing unit sums the inputs, and then
applies a non-linear activation function (i.e.
squashing/transfer/threshold function).
3. An output line transmits the result to neuron.
ARTIFICIAL INTELLIGENCE (CSC4101) 05/01/2021 15
 UNDERSTANDING NEURON
• The model presented by McCulloch and Pitts is the basic approximation of the
actual Neuron present in Human Brain.
• It is a computational model (logical) to perform similar computations as the
Human Brain does.
• It works on mathematical equations to compute output, train itself and organize the
inputs.

ARTIFICIAL INTELLIGENCE (CSC4101) 05/01/2021 16

UNDERSTANDING NEURON (CONT.)

• Inputs: x1, x2,…, xn Θ

• Output: y X1
y
• Processing: Summation ∑ X2 ∑|ƒ
…
and Activation function ƒ. Xn

• Bias: Θ

ARTIFICIAL INTELLIGENCE (CSC4101) 05/01/2021 17

 EQUATIONS
• 
Mathematically:
For every problem ’X’ there is a solution ‘Y’; Y=ƒ(X)
Simply: Y = ∑(inputs * weights) – Bias| Θ
Y = act
where n = number of inputs, m = number of neurons, Where I and j are iterators of
Input Signal and associated Neuron.

ARTIFICIAL INTELLIGENCE (CSC4101) 05/01/2021 18

 ACTIVATION FUNCTION
1. It is a mathematical equation which dictates the output of a Neuron.
2. Activation function decides whether the neuron should ‘fire’ or ‘activate’ itself
or not.
3. There are few types of activation functions. These types are used based on the
required output. For example: a binary classifier requires step function to decide
whether true or false (0 and 1).

ARTIFICIAL INTELLIGENCE (CSC4101) 05/01/2021 19

 STEP FUNCTION
•1.  A step function is a simple activation function in Neuron. If decides whether the
computed value is greater than or lesser than Threshold in order to activate itself.
2. Mathematically:
Act (v) =
3. It is used when the desired output is either Zero (0|false) or One (1|true)

ARTIFICIAL INTELLIGENCE (CSC4101) 05/01/2021 20

 SIGMOID FUNCTION
•1.  A more sophisticated activation function called sigmoid function used to output
value between Zero and One.
2. Mathematically:
Act (v) =
3. It is used when the desired output ranges between Zero (0|false) and One (1|
true). Multi-category classifier is based on such function.

ARTIFICIAL INTELLIGENCE (CSC4101) 05/01/2021 21

 TAN SIGMOID FUNCTION
•1.  A more sophisticated activation function called sigmoid function used to output
value between Zero and One.
2. Mathematically:
Act () =
Act (v) =
3. It is used when the desired output ranges between Negative One (-1) and One
(1).

ARTIFICIAL INTELLIGENCE (CSC4101) 05/01/2021 22

 BIAS | Θ
•1.  A bias (Θ) is constant added to each Neuron’s output or in activation function.
2. It allows the activation function to shift either right or to left (in a step function).
3. It’s purpose is only to fit the data accordingly (E.g. , where C is the constant
added to avoid the line through origin).

ARTIFICIAL INTELLIGENCE (CSC4101) 05/01/2021 23

SINGLE LAYER PERCEPTRON
SIMPLE NEURAL NETWORK

ARTIFICIAL INTELLIGENCE (CSC4101) 05/01/2021 24

 SINGLE LAYER PERCEPTRON
•1.  It is the most simple variant of neural network.
2. It has only two layers; one is called input layer (denoted by and comprises of
inputs) and second is called output layer (denoted by and composed of
Neurons).
3. So, to build a neural network, we need to introduce weights associated with
each synaptic link which starts from input () and ends at Neuron ().

ARTIFICIAL INTELLIGENCE (CSC4101) 05/01/2021 25

 SINGLE LAYER PERCEPTRON (DESIGNING)
 Neurons()

 Input Layer () 𝛚𝟏𝟏

 
1
𝑦
 

 
1 𝛚𝟐𝟏
 

𝛚𝟏𝟐
 

 
𝛚𝟑𝟏
  𝛚𝟐𝟐
 
2
𝑦
 

2 𝛚𝟑𝟐
 
 
3

ARTIFICIAL INTELLIGENCE (CSC4101) 05/01/2021 26

 EQUATIONS
•  Output function can be stated as:
=
=
=

• Error function can be stated as:

j = is the desired value.

• Update function can be stated as:

ARTIFICIAL INTELLIGENCE (CSC4101) 05/01/2021 27

END

Presentation made for:

ARTIFICIAL INTELLIGENCE – CSC4101

COMPUTER SCIENCE DEPARTMENT, SZABIST KARACHI.
ARTIFICIAL INTELLIGENCE (CSC4101) 05/01/2021 28