Scribd
Upload
261 views13 pages

Session 1

Uploaded by

aditya wisnugraha
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
261 views13 pages

Session 1

Uploaded by

aditya wisnugraha
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 13

Session 1

1
What is Neural Network

© Aditya Wisnugraha Sugiyarto


2
Activation Function

© Aditya Wisnugraha Sugiyarto


3
Cost Function: Classification Loss Function
1. Multi-Class Cross-Entropy Cost
ℓ 𝑦, 𝑦ො = − ෍ 𝑦𝑖 𝑙𝑜𝑔 𝑦ො𝑖
𝑖
2. Hinge Loss (for Binary Clasification)
In this function, we use 𝑦 = −1,1 and we must use 𝑡𝑎𝑛ℎ activation
function to get 𝑦ො = −1,1
ℓ 𝑦, 𝑦ො = max 0,1 − 𝑦 ∙ 𝑦ො
3. Kullback-Leibler Divergence Loss
This function, used to classify multi-class problems
𝑦𝑖
𝐷𝐾𝐿 𝑦‖𝑦ො = ෍ 𝑦𝑖 ln Source:
𝑦ො https://keras.io/api/losses/
𝑖

© Aditya Wisnugraha Sugiyarto


4
Optimization Algorithms

Note:
𝛼=
𝑙𝑒𝑎𝑟𝑛𝑖𝑛𝑔 𝑟𝑎𝑡𝑒
𝜃 =explanation:
For more 𝑝𝑎𝑟𝑎𝑚𝑒𝑡𝑒𝑟𝑠 link

© Aditya Wisnugraha Sugiyarto


5
Convolutional Neural
Network (CNN)

6
Architecture

© Aditya Wisnugraha Sugiyarto


7
Algorithms: Conv Layer

© Aditya Wisnugraha Sugiyarto


8
Algorithms: Conv Layer

© Aditya Wisnugraha Sugiyarto


9
Algorithms: Max Pooling

© Aditya Wisnugraha Sugiyarto


10
Algorithms: Flatten

© Aditya Wisnugraha Sugiyarto


11
Algorithms: Fully Connected Layer
Forward Propagation Backward Propagation
Input layer Output layer
1
𝑥𝑖 = 𝑎𝑖 , 𝑖 𝜕𝐶 𝜕𝐶 𝜕 𝑦ො
∈ 1,2,3,4, … , 𝑁 =
𝜕𝑧 𝑛+1 𝜕 𝑦ො 𝜕𝑧 (𝑛+1)
Output layer 𝜕𝐶 𝜕𝐶 𝜕𝑧 𝑛+1
● Transfer Function =
𝑧 2 =𝑊 1 𝑥+𝑏 1 𝜕𝑊 (𝑛) 𝜕𝑧 𝑛+1 𝜕𝑊 (𝑛)
● Activation Function 𝜕𝐶 𝜕𝐶 𝜕𝑧 𝑛+1
= (𝑛+1)
ෝ=𝑓 𝑧2
𝒚 𝜕𝑏 𝑛 𝜕𝑧 𝜕𝑏 𝑛
Calculate Error using Cost 𝜕𝐶
= (𝑛+1)
Function 𝜕𝑧
𝐶 = 𝑐𝑜𝑠𝑡(𝑦,
ො 𝑦) Weight and bias update using
optimization:
(𝑛) (𝑛) 𝜕𝐶
𝑊𝑡+1 = 𝑊𝑡 − 𝛼
𝜕𝑊 (𝑛)
(𝑛) (𝑛) 𝜕𝐶
𝑏𝑡+1 = 𝑏𝑡 − 𝛼
𝜕𝑏 𝑛 12
© Aditya Wisnugraha Sugiyarto
Source
A part of PPT: Workshop Introduction to Neural Network (13 Maret 2021)

© Aditya Wisnugraha Sugiyarto

You might also like