Scribd Logo
Upload

Welcome to Scribd!

  • 20 views

    Mini-Batch and Gradient Descent

    Uploaded by

    srija
    The document describes how to implement mini-batch gradient descent for model training. It involves making batches of the training data, storing the batches in a list, then training the model by iterating through the batches and calling partial_fit on each batch to update the model parameters. The batch size, max iterations, and other hyperparameters need to be tuned for optimal results. Gradient descent can also be implemented by setting the batch size equal to the full training set size.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd

    Mini-Batch and Gradient Descent

    Uploaded by

    srija
    20 views1 page
    The document describes how to implement mini-batch gradient descent for model training. It involves making batches of the training data, storing the batches in a list, then training the model by iterating through the batches and calling partial_fit on each batch to update the model parameters. The batch size, max iterations, and other hyperparameters need to be tuned for optimal results. Gradient descent can also be implemented by setting the batch size equal to the full training set size.

    Original Title

    Mini-Batch and Gradient Descent(1)

    Copyright

    © © All Rights Reserved

    Available Formats

    PDF, TXT or read online from Scribd

    Did you find this document useful?

    Is this content inappropriate?

    The document describes how to implement mini-batch gradient descent for model training. It involves making batches of the training data, storing the batches in a list, then training the model by iterating through the batches and calling partial_fit on each batch to update the model parameters. The batch size, max iterations, and other hyperparameters need to be tuned for optimal results. Gradient descent can also be implemented by setting the batch size equal to the full training set size.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Download as pdf or txt
    20 views1 page

    Mini-Batch and Gradient Descent

    Uploaded by

    srija
    The document describes how to implement mini-batch gradient descent for model training. It involves making batches of the training data, storing the batches in a list, then training the model by iterating through the batches and calling partial_fit on each batch to update the model parameters. The batch size, max iterations, and other hyperparameters need to be tuned for optimal results. Gradient descent can also be implemented by setting the batch size equal to the full training set size.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Download as pdf or txt
    You are on page 1of 1

    Mini-Batch

    1. Make Batches and Store

    make_batches(​batch_size​):
    iterator=0
    while iterator<training_size:
    temp_batch = range(start,start+batch_size)
    X_batch = X_train[temp_batch]
    y_batch = y_train[temp_batch]
    yield X_batch,y_batch
    iterator += batch_size

    batches = list(make_batches(batch_size=.....))

    2. Training

    model = linear_model.SGDRegressor(max_iter=....,tol=....,eta0=....,average=batch_size)

    for loop_counter in range(max_iterations):


    for X_batch,y_batch in batches:
    model.partial_fit(X_batch,y_batch)

    NOTE
    1. Use batch_size as 50 or 100 according to convenience.
    2. Max_iter, tol and eta0 are parameters that need to be tuned.

    Gradient Descent
    As already discussed there is no direct API/Class in scikit-learn to implement this algorithm. So,
    we use the mini-batch approach with ​batch_size ​as the entire dataset i.e, ​X_train.shape[0]

    You might also like