Gradient Descent Calculation Split in 2 Steps

Gradient Descent Calculation Split in 2 Steps
$\begingroup$

In a machine learning course I am taking we have an assignment with a notebook (In linear models). So far I have calculated the cost and the sigmoid in a logistic regression and the next exercise wants the gradient. I am pretty aware of the formula and how it works theoretically,but heres what got me confused : The function is split functions,the first one wants to calculate the derivatives of weights,bias and the 2nd to update them accordingly. My problem is this,I dont understand how by the GD formula I can "extract" the derivatives I need since in my understanding,it could be done simply in a loop in one function. I'll leave the code below in case anyone wants to take a look at it but please dont answer with code,i just want the theoretical direction. Thank you

First method

def gradient(w,b,X,Y,sigma):
    """
    Arguments:
    w -- weights
    b -- bias
    X -- input data
    Y -- target or label vector

    Return:
    dw -- gradient of the loss with respect to w (numpy array) 
    db -- gradient of the loss with respect to b (scalar)
    """
    # BEGIN CODE HERE
    bw = ... #(Optional)
    X =  ... #(Optional)
    grad = ... #(Optional)
    dw = ...
    db = ...

    #END CODE HERE

    return dw, db

Second method

def update_parameters(w,b,X,Y,num_iterations,learning_rate):
    """
    This function optimizes w and b by running a gradient descent algorithm

      Arguments:
      w -- weights
      b -- bias
      X -- input data
      Y -- target or label vector
      num_iterations -- number of iterations of the optimization loop
      learning_rate -- learning rate of the gradient descent update rule

      Returns:
      params -- dictionary containing the weights w and bias b
      grads -- dictionary containing the gradients of the weights and bias with respect to the cost function.
    """
    for i in range(num_iterations):
        w_prev = w
        b_prev = b
        # BEGIN CODE HERE
        # Cost and gradient calculation
        sigma, cost = ...
        dw, db = ...
        # update rule
        w = w_prev - ...
        b = b_prev - ...
        #END CODE HERE

        # Print the cost every 100 training iterations
        if i % 100 == 0:
            print ("Cost after iteration %i: %f" %(i, cost)) 

    return w,b,dw,db

$\endgroup$
1
Elena Rostova
Author

Elena Rostova

Elena Rostova holds a Master's degree in Public Health Journalism. She covers groundbreaking medical research, holistic wellness trends, mental health awareness, and nutritional science.