In a machine learning course I am taking we have an assignment with a notebook (In linear models). So far I have calculated the cost and the sigmoid in a logistic regression and the next exercise wants the gradient. I am pretty aware of the formula and how it works theoretically,but heres what got me confused : The function is split functions,the first one wants to calculate the derivatives of weights,bias and the 2nd to update them accordingly. My problem is this,I dont understand how by the GD formula I can "extract" the derivatives I need since in my understanding,it could be done simply in a loop in one function. I'll leave the code below in case anyone wants to take a look at it but please dont answer with code,i just want the theoretical direction. Thank you
First method
def gradient(w,b,X,Y,sigma):
"""
Arguments:
w -- weights
b -- bias
X -- input data
Y -- target or label vector
Return:
dw -- gradient of the loss with respect to w (numpy array)
db -- gradient of the loss with respect to b (scalar)
"""
# BEGIN CODE HERE
bw = ... #(Optional)
X = ... #(Optional)
grad = ... #(Optional)
dw = ...
db = ...
#END CODE HERE
return dw, db
Second method
def update_parameters(w,b,X,Y,num_iterations,learning_rate):
"""
This function optimizes w and b by running a gradient descent algorithm
Arguments:
w -- weights
b -- bias
X -- input data
Y -- target or label vector
num_iterations -- number of iterations of the optimization loop
learning_rate -- learning rate of the gradient descent update rule
Returns:
params -- dictionary containing the weights w and bias b
grads -- dictionary containing the gradients of the weights and bias with respect to the cost function.
"""
for i in range(num_iterations):
w_prev = w
b_prev = b
# BEGIN CODE HERE
# Cost and gradient calculation
sigma, cost = ...
dw, db = ...
# update rule
w = w_prev - ...
b = b_prev - ...
#END CODE HERE
# Print the cost every 100 training iterations
if i % 100 == 0:
print ("Cost after iteration %i: %f" %(i, cost))
return w,b,dw,db