Hands on deep learning pytorch version exercise solution-3.3 simple implementation of linear regression

1. If the total loss of small batch is replaced by the average loss of small batch , How do you need to change the learning rate ？
   Explain ： The default is actually the average (mean squared), The question is wrong , Do the opposite . The learning rate is divided by batch_size that will do
2. View the deep learning framework ⽂ files , What loss functions and initializations do they provide ⽅ Law ？⽤Huber The loss replaces the original loss , namely
   
   Provided loss As shown in the figure below , If you need specific understanding, you can use help(torch.nn.xxxLoss) Or Baidu query
   

# huber Loss correspondence Pytorch Of SmoothL1 Loss 
loss = nn.SmoothL1Loss(beta=0.5)
num_epochs = 3
for epoch in range(num_epochs):
    for X, y in data_iter:
        l = loss(net(X), y)
        # Start calculating the gradient 
        trainer.zero_grad()
        l.backward()
        trainer.step()  # Update all parameters 
    print("epoch: {}, loss:{}".format(epoch + 1, l))
# Start with 1 when loss The larger , Change to 0.5 Run again to reduce , It may be related to the number of iterations 
''' epoch: 1, loss:0.00011211777746211737 epoch: 2, loss:0.00013505184324458241 epoch: 3, loss:4.4465217797551304e-05 '''

3. How do you access the gradient of linear regression ？

net[0].weight.grad,net[0].bias.grad
''' (tensor([[-0.0040, 0.0027]]), tensor([0.0015])) '''

For observability , This time, in addition to highlighting loss Of , The code output of other problems is put into the code block .