Scribd Logo
Upload

Welcome to Scribd!

  • 3 views

    Import As Import As Return

    Uploaded by

    0105cd211061

    Copyright:

    © All Rights Reserved
    Download as DOCX, PDF, TXT or read online from Scribd

    Import As Import As Return

    Uploaded by

    0105cd211061
    3 views2 pages

    Original Title

    65bb64a6e8e3a

    Copyright

    © © All Rights Reserved

    Available Formats

    DOCX, PDF, TXT or read online from Scribd

    Did you find this document useful?

    Is this content inappropriate?

    Copyright:

    © All Rights Reserved
    Download as DOCX, PDF, TXT or read online from Scribd
    Download as docx, pdf, or txt
    3 views2 pages

    Import As Import As Return

    Uploaded by

    0105cd211061

    Copyright:

    © All Rights Reserved
    Download as DOCX, PDF, TXT or read online from Scribd
    Download as docx, pdf, or txt
    You are on page 1of 2

    Experiment 4

    Aim: Implementation of Gradient Descent


    Algorithm. Theory:
    Gradient Descent Algorithm iteratively calculates the next point using gradient at the current
    position, scales it (by a learning rate) and subtracts obtained value from the current position
    (makes a step). It subtracts the value because we want to minimise the function (to
    maximise it would be adding). This process can be written as:

    There’s an important parameter η which scales the gradient and thus controls the step size. In
    machine learning, it is called learning rate and have a strong influence on performance.
    Algorithm:
    Gradient Descent method’s steps are:
    1. choose a starting point (initialisation)
    2. calculate gradient at this point
    3. make a scaled step in the opposite direction to the gradient (objective: minimise)
    4. repeat points 2 and 3 until one of the criteria is met:
     maximum number of iterations reached
     Step size is smaller than the tolerance (due to scaling or a small
    gradient). Python Code:
    import tensorflow as tf
    import matplotlib.pyplot as plt
    def fu(x,z):
    return x**2.0-x*3+z**2

    def fu_min():
    return 2**x-x*3+z**z

    def reset():
    x=tf.Variable(10.0)
    z=tf.Variable(10.0)
    return x,z

    x,z=reset()
    opt=tf.keras.optimizers.SGD(learning_rate=0.1)
    for i in range(50):
    with tf.GradientTape() as tape:
    y = fu(x, z)
    grads = tape.gradient(y, [x,z])
    processed_grads = [g for g in grads]
    grads_and_vars = zip(processed_grads, [x, z])
    print ('y = {:.1f}, x = {:.1f}, z = {:.1f}, grads0 = {:.1f}, gra
    ds1 = {:.1f} '.format(y.numpy(), x.numpy(), z.numpy(), grads[0].n
    umpy(), grads[1].numpy()))
    opt.apply_gradients(grads_and_vars)

    Output:

    You might also like