beginner/examples_tensor/polynomial_numpy

Run in Google Colab

Colab

Download Notebook

Notebook

View on GitHub

GitHub

Warm-up: numpy

Created On: Dec 03, 2020 | Last Updated: Dec 03, 2020 | Last Verified: Nov 05, 2024

A third order polynomial, trained to predict \(y=\sin(x)\) from \(-\pi\) to \(pi\) by minimizing squared Euclidean distance.

This implementation uses numpy to manually compute the forward pass, loss, and backward pass.

A numpy array is a generic n-dimensional array; it does not know anything about deep learning or gradients or computational graphs, and is just a way to perform generic numeric computations.

99 5216.47175523742
199 3490.3964013339723
299 2337.9160915411635
399 1567.9729639779762
499 1053.282021549529
599 709.0038112631381
699 478.5625277643276
799 324.2107782377892
899 220.74987447579318
999 151.34860389058605
1099 104.75803126600022
1199 73.4553782505025
1299 52.406467728995445
1399 38.24017499820671
1499 28.697427788070115
1599 22.263244387608555
1699 17.92086201335166
1799 14.987344839351039
1899 13.003599414651541
1999 11.660741285846834
Result: y = 0.035867315677983444 + 0.8167154794951832 x + -0.0061877087891612165 x^2 + -0.08763712961547539 x^3

import numpy as np
import math

# Create random input and output data
x = np.linspace(-math.pi, math.pi, 2000)
y = np.sin(x)

# Randomly initialize weights
a = np.random.randn()
b = np.random.randn()
c = np.random.randn()
d = np.random.randn()

learning_rate = 1e-6
for t in range(2000):
    # Forward pass: compute predicted y
    # y = a + b x + c x^2 + d x^3
    y_pred = a + b * x + c * x ** 2 + d * x ** 3

    # Compute and print loss
    loss = np.square(y_pred - y).sum()
    if t % 100 == 99:
        print(t, loss)

    # Backprop to compute gradients of a, b, c, d with respect to loss
    grad_y_pred = 2.0 * (y_pred - y)
    grad_a = grad_y_pred.sum()
    grad_b = (grad_y_pred * x).sum()
    grad_c = (grad_y_pred * x ** 2).sum()
    grad_d = (grad_y_pred * x ** 3).sum()

    # Update weights
    a -= learning_rate * grad_a
    b -= learning_rate * grad_b
    c -= learning_rate * grad_c
    d -= learning_rate * grad_d

print(f'Result: y = {a} + {b} x + {c} x^2 + {d} x^3')