Batch Gradient Descent algorithm in python is returning huge values

41 Views Asked by At

I'm trying to implement a Batch Gradient Descent algorithm in python that takes in the training set, the learning rate, and the number of iterations as input arguments, and returns the weights. However, when I run it, within a few iterations the values for the parameters get exponentially large and eventually it returns 'nan'.

x = [[2104] [1600] [2400] [1416] [3000] [1985] [1534] [1427] [1380] [1494] [1940] [2000] [1890] [4478] [1268] [2300] [1320] [1236] [2609] [3031] [1767] [1888] [1604] [1962] [3890] [1100] [1458] [2526] [2200] [2637] [1839] [1000] [2040] [3137] [1811] [1437] [1239] [2132] [4215] [2162] [1664] [2238] [2567] [1200] [ 852] [1852] [1203]]

y = [399900 329900 369000 232000 539900 299900 314900 198999 212000 242500 239999 347000 329999 699900 259900 449900 299900 199900 499998 599000 252900 255000 242900 259900 573900 249900 464500 469000 475000 299900 349900 169900 314900 579900 285900 249900 229900 345000 549000 287000 368500 329900 314000 299000 179900 299900 239500]

a = 0.01

num_iter = 100

def BGD ( x, y, a, num_iter):
    m = len(x) #number of samples
    n = x.shape[1] #number of features
    p = np.zeros(n) 
    b = 0
    for _ in range(num_iter):
        sum_p = np.zeros(n)
        sum_b = 0
        for i in range(m):
            sum_p = sum_p + ((np.dot(p,x[i])+b) - y[i]) * x[i]
            sum_b = sum_b + (((np.dot(p,x[i])+b) - y[i]))
        p = p - (a * (1/m) * sum_p)   
        b = b - (a * (1/m) * sum_b)
    return p, b

p, b = BGD(x, y, 0.01, 100)
print(p)
print(b)

I get the following:

RuntimeWarning: overflow encountered in add
  sum_p = sum_p + ((np.dot(p,x[i])+b) - y[i]) * x[i]
RuntimeWarning: invalid value encountered in subtract
  p = p - (a * (1/m) * sum_p)
[nan]
nan
0

There are 0 best solutions below