2. Simple regression problem

The return question

introduce

Optimize

Through samples wx1+b、y1,wx2+b、y2 etc. , It can be inferred that （w,b） Make for any xn, Estimate one yn, That's what we want to do .
The reason for the gradient descent may be , The derivative method is used to find the minimum value , That is, find the gradient , In higher numbers, let the derivative first （ gradient ） by 0 when , Then judge whether this point is the extreme point , Finally, find the minimum point .

Linear regression

The value range of the whole prediction is continuous .

Logical regression

The activation function is added on the basis of linear regression （ Compression function ）, For example, the predicted value is 0-1 The problem of .
for example ReLU function ：
R(z)=max(0,z)

classification

The classic case is MNIST Data set identification

Return to the problem of actual combat

points Data sets :

def computer_error_for_line_given_points(b,w,points):
    totalError = 0
    for i in range(0,len(points)):
        x = points[i,0]
        y = points[i,1]
        totalError += (y - (w * x + b))**2
    return totalError/ float(len(points))

def step_gradient(b_current,w_current,points,learningRate):
    b_gradient = 0
    w_gradient = 0
    N = float(len(points))
    for i in range(0,len(points)):
        x = points[i, 0]
        y = points[i, 1]
        b_gradient += (2/N) * ( w_current * x + b_current - y)
        w_gradient += (2/N) * x * ( w_current * x + b_current - y)
    new_b = b_current - (learningRate * b_gradient)
    new_w = w_current - (learningRate * w_gradient)
    return [new_b,new_w]

Optimize ：

def gradient_descent_ruuner(points,starting_b,starting_w,learning_rate,num_iterations):
    b = starting_b
    w = starting_w
    for i in range(num_iterations):
        b,w = step_gradient(b,w,np.array(points),learning_rate)
    return [b,w]

Running results ：