Rate this Page

Warm-up: numpy#

Created On: Dec 03, 2020 | Last Updated: Dec 03, 2020 | Last Verified: Nov 05, 2024

A third order polynomial, trained to predict \(y=\sin(x)\) from \(-\pi\) to \(pi\) by minimizing squared Euclidean distance.

This implementation uses numpy to manually compute the forward pass, loss, and backward pass.

A numpy array is a generic n-dimensional array; it does not know anything about deep learning or gradients or computational graphs, and is just a way to perform generic numeric computations.

99 318.3087724652189
199 216.24129021282135
299 147.93935417139193
399 102.20082505291774
499 71.55011950306142
599 50.99496056476203
699 37.199496593745764
799 27.933334159510558
899 21.70424459408163
999 17.513191982479906
1099 14.690859910294348
1199 12.788503551972532
1299 11.505031714479554
1399 10.638263416881099
1499 10.052323780290877
1599 9.655820943256035
1699 9.387228998033102
1799 9.205090445353463
1899 9.081444149126302
1999 8.997413463631988
Result: y = 0.009688279330304214 + 0.8662886846593172 x + -0.0016713893981414 x^2 + -0.09468848636280486 x^3

import numpy as np
import math

# Create random input and output data
x = np.linspace(-math.pi, math.pi, 2000)
y = np.sin(x)

# Randomly initialize weights
a = np.random.randn()
b = np.random.randn()
c = np.random.randn()
d = np.random.randn()

learning_rate = 1e-6
for t in range(2000):
    # Forward pass: compute predicted y
    # y = a + b x + c x^2 + d x^3
    y_pred = a + b * x + c * x ** 2 + d * x ** 3

    # Compute and print loss
    loss = np.square(y_pred - y).sum()
    if t % 100 == 99:
        print(t, loss)

    # Backprop to compute gradients of a, b, c, d with respect to loss
    grad_y_pred = 2.0 * (y_pred - y)
    grad_a = grad_y_pred.sum()
    grad_b = (grad_y_pred * x).sum()
    grad_c = (grad_y_pred * x ** 2).sum()
    grad_d = (grad_y_pred * x ** 3).sum()

    # Update weights
    a -= learning_rate * grad_a
    b -= learning_rate * grad_b
    c -= learning_rate * grad_c
    d -= learning_rate * grad_d

print(f'Result: y = {a} + {b} x + {c} x^2 + {d} x^3')

Total running time of the script: (0 minutes 0.236 seconds)