Intro DL

Introduction to Deep
Learning
Industrial
Revolution
• 1st Industrial Revolution
• 18-19th centuries in Europe and
America
• Agriculture experienced
industrialisation
• Driving Technologies
• Iron Industry
• Steam Engine
• Textile Industry.
Industrial
•Revolution
2nd Industrial Revolution
• 1870-1914 (before the World War I)
• In America, Europe, Japan
• Rapid growth of new industries and technology
• Focus on Mass production due to availability of Electricity, Oil
and Steel
• Driving Technology
• Telephone
• Light Bulb
• Internal Combustion Engine
• Henry Ford took idea of mass production from a slaughter
house in Chicago.
• The pigs hung from conveyor belts and each butcher performed
only a part of the butchering of the animal.
• Henry applied these principles in automobile production and
now the vehicles were produced in partial steps on the
conveyor belt.
.
• • This made process faster and cost reduced.
Industrial
•Revolution
3rd Industrial Revolution
• Late 1970s - Today
• Also called Digital Revolution
• Advancements in digital
electronics
• Driving Technologies
• PC
• Internet
• ICT
• Current Driving
Technologies
• Smart phones
• Wireless
• Mobile
Communications
• Big Data
• Cloud
• SNS
• Artificial Intelligence
Industrial
Revolution
• 4th Industrial Revolution
• Klaus Schwab- World Economic Forum, 2016
• Building on 3rd, characterised by fusion of technologies that is
blurring lines between physical, digital and biological spheres.
• In future, M L and DL are expected to drive improvements in all
technology areas like, 3D printing, robotics, Autonomous
vehicles, IOT, 5G etc
Applications of
DL
• Computer Visions and Image
Processing
• NLP: IBM Watson
• Prediction and Estimation
• Lettuce Bots
• AIVA
• Speech Recognition
• Siri
• Google’s Voice Search
• Autonomous Vehicles
• UAV (Unmanned Aerial Vehicle)
• USV. (Unmanned Surface Vehicle)
• https://www
.thinkautonomous.ai/blog/?p=deep-learning-in-self-driving-cars
• Robotics
• Industrial Robots
• Service Robots
• Cobot( Collaborative Robot)
• Mobile Robot
Why Deep Learning is Taking
Off?
•
Scale is driving the Deep Learning Process: Andrew Ng
• Scale traditionally is for
• Data
• Computation
• Now: Algorithms
• Eg: Switching from Sigmoid Function to Relu
Function.
• Learning is very slow in Sigmoid
Other
Reasons
• Support for Cloud
computing
• Hardware Innovation
• In Deep Learning the basic building block is Neural
Network.
• Hence, we will understand what NN means.
• Before that, we have a walk through in the history.
History of Deep
Learning
1871-1873
Staining
Theory
• Camillo Golgi discovered a
chemical reaction to examine
nervous tissue and he was also a
proponent of Reticular Theory
1888-1891: Neuron Doctrine
• Santiago Cajal used Golgi’s

method to again study the
nervous system.
• He proposed that nervous

system is made up of discrete
individual cells forming a
network
• In 1891 the term Neuron was coined by Heinrich Wilhem
Gottfried.
• He consolidated the Neuron Doctrine.
• There was a debate about a single cell vs dicrete network.

1950s
• In 1950s electron microscope

confirmed the neuron doctrine and
showed that the nerve cells are
individual cells interconnected through
synapse
• Network of many individual neurons.
What is a
Neuron?
The Human Brain has 100 billion neurons with 1

Quadrillion (10^15) synaptic
connections
• McCulloch Pitts Neuron
• Proposed simplified model of
neuron (1943) or how brain
works
• Brain takes input from
various factors and then
takes decision after
applying some function.
• We have come to Artificial
Neuron.
• Activation
Function
• Hard Output:
Binary values
(0/1) result from
Threshold T
activation
function
• Output = 1, if
y>T
• Else 0
• Instead of Hard Output, the focus is on Soft Output.
• Soft output values can be made using one of the following activation
function
Sigmoid
Function
• The value exists between
(0,1).
• Specially used for
models when the
probability is to be
predicted as the output.
Tanh Activation
Function
• Like Sigmoid, but better.
• Range of tank (-1,1)
• Advantage: Can map negative
inputs to negative value and the
zero inputs near zero.
ReLU (Rectified Linear Unit) Activation
Function
• Most used activation function.
• Widely used in Convolutional NN.
• Half Rectified. R(z) is zero when z is
less than zero and f(z) is equal to z
when it is above or equal to zero.
Neural Network
• House Price Prediction Example
• Linear Regression can be used to
predict the price of houses.
• But Prices can never be negative.
• S o instead of a straight line, we will
bend the curve so that it just ends
up at zero.
• It is a simple possible NN.
• This NN, takes size as the input,
computes the linear function, takes a
max of zero and observed value
and then outputs the estimated
price.
• A larger NN can be formed by taking many single neurons and
stacking them together.
• A single neuron can be compared to a Single Lego brick and a
larger network to a stack of Lego bricks.
Pictures of Various
NN
Perceptron
• Frank Rosenblatt came up with the

perceptron model for doing
computation.
• The perceptron may eventually be

able to learn, make decisions and
translate languages.
Perceptron
• Single Layer NN.
• Also termed as
Linear Binary
Classifier
• 4 parts
• Input
• Weights and
Bias
• Net sum
• Activation
Function
Equations
Bia
Pre activation s
Function
Mathematical Model for a Single

Activation Function: Neuron
Step Function
What is
Bias??
• The model has the
constraint to train itself
to pass through the
origin
• Many times
impossible to fit a
model that passes
through the origin.
• Add a constant and
allow the model to
train itself to find the
best fit for data.
• c= bias=constant
Perceptron Learning
Rule
• Learning rule means a procedure to modify weights and
biases of the network.
• Algorithm will automatically learn the optimal weight
coeﬃcients.
• Also called training a NN.
• https://mlweb.loria.fr/book/en/perceptron.html
•
Comparing to Biological
Neuron
• Input analogous to Dendrites
• Weights — - > Synapse
• Activation Function. — — > Electric
signal
Why Perceptrons can only Model Linearly Separable
Problems
Decision Boundary : Line

Only one boundary can be
produced
Do you feel that Perceptrons are of no

use???
Limitations of
Perceptron
• Used for binary Classification
• Are limited to solving problems that are linearly
separable.
MLP: Multi Layer
Perceptron
• Is the XOR problem linearly
separable.
• Truth Table
Idea of
MLP
• What is XO R ? ? ?
• OR but not AND
• Therefore, Non Linear

Problems can be
solved by linking
multiple perceptrons.
• This is the idea behind

MLP.
3 Layer Neural
Network
• Incorporating Multiple Perceptron into
Hidden Layers in the Perceptron can
solve the XOR problem.
• This is the foundation of Modern
Neural Networks
Example: Training a NN.
In the training phase weights need to be updated
such that the loss function is minimized.
To update the weights, it is needed to do Back

Propogation.

Intro DL

Uploaded by

Copyright:

Available Formats

Intro DL

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Intro DL

Uploaded by

Copyright:

Available Formats

Introduction to Deep

• Santiago Cajal used Golgi’s

• He proposed that nervous

• He consolidated the Neuron Doctrine.

• There was a debate about a single cell vs dicrete network.

• In 1950s electron microscope

The Human Brain has 100 billion neurons with 1

• Frank Rosenblatt came up with the

• The perceptron may eventually be

Mathematical Model for a Single

Decision Boundary : Line

Do you feel that Perceptrons are of no

• Therefore, Non Linear

• This is the idea behind

To update the weights, it is needed to do Back

You might also like