Hota-ML35

Birla Institute of Technology and Science Pilani, Hyderabad Campus
22.11.2024
BITS F464: Machine Learning (1st Sem 2024-25)

GENERATIVE ADVERSARIAL NETWORK (GAN)
Chittaranjan Hota, Sr. Professor
Dept. of Computer Sc. and Information Systems
hota@hyderabad.bits-pilani.ac.in
Generative Adversarial Network (GAN)
• GANs are neural networks (Deep ANNs) that learn to create synthetic data similar
to some known input data. Or have the capability to generate new data.
• Ex: Admission office (Discriminator) checking the transcripts of newly admitted
students( Generator)…
update
Real Samples
d
i
s
c
r
i
m
i
fake real
n
random seed
F a
A t
generator
K o
E r
update
Loss Functions in a GAN
D(x):Discriminator’s prob. that x is real; Ex:
Expected value over all real x; G(z):
Ex[log(D(x))] + Ez[log(1 – D(G(z)))]
Generator’s output when given noise is z;
D(G(z)): Discriminator’s prob. that a fake
instance is real; Ez: Expected value over all
random inputs to the generator.
Real
Samples Discriminator
Loss
Discriminator
Fake
Generator Generator
Samples Loss
Backpropagation in GAN
Backpropagation
Real
Samples
Discriminator
Discriminator Loss
Fake
Generator
Samples
Generator
Loss
Backpropagation in GAN
Real Discriminator
Samples Backpropagation Loss
Fake Discriminator
Generator
Samples
Generator
Loss
GAN Applications
Applications: synthetic image/ video generation (deepfake),
Image-to-image translation, Anomaly detection,…
(Deep Convolutional GAN: Better Quality) (Generated Vanilla GAN images in MNIST)
Image Source: https://theaisummer.com

StyleGAN:
https://github.com/NVlabs/stylegan (Tero Karras)

k-NN: k-Nearest Neighbor Algorithm
from sklearn.neighbors import KNeighborsClassifier

from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split class: blue
from sklearn.metrics import accuracy_score
iris = load_iris()
X = iris.data load dataset
k=5 ?
y = iris.target
X_train, X_test, y_train, y_test = train_test_split(X, y,
test_size=0.2, random_state=42)
k=3
knn = KNeighborsClassifier(n_neighbors=5) class: red
knn.fit(X_train, y_train) k=5

k=9
y_pred = knn.predict(X_test)
class: blue
accuracy = accuracy_score(y_test, y_pred)
print("Accuracy:", accuracy)
Discrete valued target fun: voting, and
Real-valued target fun: taking mean
Applications: Optical Character Recognition (OCR), Credit Scoring, Loan Approval.
How to choose a right value of ‘k’?
k=7 k=13
If ‘k’ = small value: Rule of thumb:

- Can lead to sensitive decision boundaries K = sqrt(N)
when data-distribution is Non-linear N: number of training points
- Can be sensitive to noise and outliers,
Else, use Cross-validation to
hence leading to Overfitting
decide the value of ‘k’.
- Hence, Low bias and High variance.
If ‘k’ = large value:

- Can lead to smoother decision boundary
as influence of individual neighbor is
diluted.
- Reduce the impact of noise and outliers.
- Can lead to Under-fitting in complex cases.
- Hence, High bias and Low variance.
Curse of Dimensionality: Distance computation complexity, Sparse data, Curse of Proximity
SVM: Intuition behind choice of surface
O+ O+
X X
X X
O+ O+
X X O+
O+
X O
+
Which one is the best separator out of these 4?

Some Noise in the Input Samples
A B C D
O+ O+
X X
X X
O+ O+
X X O+
O+
X O+
Which ones are better now? Which one is good ow?

Finding the Decision Boundary: Another Ex.
• Let {x1, ..., xn} be the data set and let yi ∈ {1,-1} be the class label of xi
Class 2 Support Vectors are the data
y=1
points:
y=1 - That lie on or within the margin
boundary.
y=-1 y=1 - Are misclassified or are close to
y=1 being misclassified.
y=-1 y=1
y=-1 Hence, the decision

y=-1
boundary should be
y=-1 as far away as
y=-1
m possible from the
Class 1 data of both the
Euclidean (L2 classes 
Norm) Maximum Margin
Classifier (m)
w12 + w22 + …+wn2
Support vectors are the data points that lie closest to the decision boundary (hyperplane)
and have the largest influence on determining the position and orientation of the boundary.

Hota-ML35

Uploaded by

Copyright:

Available Formats

Hota-ML35

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Hota-ML35

Uploaded by

Copyright:

Available Formats

Birla Institute of Technology and Science Pilani, Hyderabad Campus

BITS F464: Machine Learning (1st Sem 2024-25)

Image Source: https://theaisummer.com

https://github.com/NVlabs/stylegan (Tero Karras)

from sklearn.neighbors import KNeighborsClassifier

knn.fit(X_train, y_train) k=5

If ‘k’ = small value: Rule of thumb:

If ‘k’ = large value:

Which one is the best separator out of these 4?

Which ones are better now? Which one is good ow?

y=-1 Hence, the decision

You might also like