Reflection on pytorch back propagation

Realize a simple all to condense network , The work done is ：

Then the loss function is classical L2 Loss ：

The code is as follows , Here we set paranoia to 0：

class network(nn.Module):
    def __init__(self):
        super().__init__()
        self.w = torch.Tensor([1.0])
        self.w.requires_grad = True
    
    def forward(self, x):
        output = self.w * x
        return output



def gradient(x, y, w):
  return 2*x*(x*w - y)

if __name__ == '__main__':

    model = network()  
    x_data = torch.Tensor([1.0,2.0,3.0])
    y_data =  torch.Tensor([2.0,4.0,6.0])
    output = model(x_data)


    loss = (output-y_data).pow(2).mean()
    loss.backward()
    
    grads = gradient(x_data, y_data, model.w).mean()# Manual implementation   Back propagation 
    print(model.w.grad.data)  #
    print(grads)

  The results are consistent , All for -8.3333

Specific derivation ：

Let's revise it loss In the form of , The weight of each sample is different , Increase the weight of the last sample , You can see that the weight of the two samples is 1/4, The weight of the last sample is 1/2. From this point of view , In fact, the above is in accordance with the standard loss Back propagation , In fact, each sample has equal weight .

    model = network()  
    x_data = torch.Tensor([1.0,2.0,3.0])
    y_data =  torch.Tensor([2.0,4.0,6.0])
    output = model(x_data)

    loss = (output-y_data).pow(2)
    loss = (loss[0:2].mean() + loss[2])/2  # modify loss
    loss.backward()
    
    # grads = gradient(x_data, y_data, model.w).mean()# Manual implementation   Back propagation 
    print(model.w.grad.data)  #

The result is -11.5, Now calculate manually ：