Note
Go to the end to download the full example code.
Warm-up: numpy¶
Created On: Dec 03, 2020 | Last Updated: Dec 03, 2020 | Last Verified: Nov 05, 2024
A third order polynomial, trained to predict \(y=\sin(x)\) from \(-\pi\) to \(pi\) by minimizing squared Euclidean distance.
This implementation uses numpy to manually compute the forward pass, loss, and backward pass.
A numpy array is a generic n-dimensional array; it does not know anything about deep learning or gradients or computational graphs, and is just a way to perform generic numeric computations.
99 643.6769083389396
199 445.27442884341883
299 309.2114167083651
399 215.7939055016693
499 151.58310952404253
599 107.3981291252633
699 76.95968841722055
799 55.96813684166337
899 41.47596606060701
999 31.460282267376883
1099 24.53118388900641
1199 19.732614341044325
1299 16.40620919807229
1399 14.09810424663235
1499 12.495075494068722
1599 11.380728856888002
1699 10.605410451518894
1799 10.065517101685948
1899 9.689254473338359
1999 9.426822691084933
Result: y = -0.023740828721341013 + 0.8467043842790039 x + 0.004095688003526444 x^2 + -0.09190279017529342 x^3
import numpy as np
import math
# Create random input and output data
x = np.linspace(-math.pi, math.pi, 2000)
y = np.sin(x)
# Randomly initialize weights
a = np.random.randn()
b = np.random.randn()
c = np.random.randn()
d = np.random.randn()
learning_rate = 1e-6
for t in range(2000):
# Forward pass: compute predicted y
# y = a + b x + c x^2 + d x^3
y_pred = a + b * x + c * x ** 2 + d * x ** 3
# Compute and print loss
loss = np.square(y_pred - y).sum()
if t % 100 == 99:
print(t, loss)
# Backprop to compute gradients of a, b, c, d with respect to loss
grad_y_pred = 2.0 * (y_pred - y)
grad_a = grad_y_pred.sum()
grad_b = (grad_y_pred * x).sum()
grad_c = (grad_y_pred * x ** 2).sum()
grad_d = (grad_y_pred * x ** 3).sum()
# Update weights
a -= learning_rate * grad_a
b -= learning_rate * grad_b
c -= learning_rate * grad_c
d -= learning_rate * grad_d
print(f'Result: y = {a} + {b} x + {c} x^2 + {d} x^3')
Total running time of the script: (0 minutes 0.237 seconds)
