Hands on deep learning pytorch version exercise solution -- implementation of 3-2 linear regression from scratch

practice

1. If we initialize the weight to zero , Will send ⽣ what . Is the algorithm still valid ？

\qquad Explain ： whole 0 Initialization is also a common choice , Compared with normal distribution initialization, it may move towards different local optima , The algorithm is still effective .

2. Suppose you are George · Simon · ohm , Try to build a relationship between voltage and current ⽴⼀ A model . You can make ⽤⾃ Dynamic differentiation to learn the parameters of the model ?

\qquad Explain ： In the example, the current is regarded as features, Voltage as labels, After tracing points and drawing, it is found that there is a linear relationship between them , Just walk through the code in the book again .

3. You can make... Based on Planck's law ⽤ Spectral energy density to determine the temperature of the object ？

\qquad Explain ： Energy density spectrum u The function of wavelength is ：

\qquad Write as a function of temperature as a function of energy density , namely ：

4. If you want to calculate ⼆ What problems may be encountered with the first derivative ？ How would you solve these problems ？

\qquad Explain ： The problem encountered should be that the calculation formula of first-order derivative function cannot be obtained directly . resolvent ： Find the first derivative and save the calculation diagram . See the following code for the specific method .

5. Why is it squared_loss Function needs to make ⽤reshape function ？

\qquad Explain ：$\hat{y}$ Is a column vector ,$y$ It's a line vector

6. Try to make ⽤ Different learning rates , Observe how fast the value of the loss function decreases .

\qquad Explain ： There is no unified answer to this , When running the model, you can record the current learning rate and training time, and then compare them .

7. If the number of samples cannot be batched ⼤ Small divisible ,data_iter Functional ⾏ For what will change ？

\qquad Explain ： The number of samples left at the end of execution cannot be divided , Will report a mistake

Fourth, ask code learning , Reference resources pytorch autograd Calculate the second derivative of scalar function

# The first 4 Ask for the second derivative ：f(x)=x^2+sinx→f'(x)=2x+cosx→f''(x)=2-sinx
# Here take x=π/2,π
import math
import torch
import numpy as np

x = torch.tensor([math.pi/2,math.pi],requires_grad=True)
y = x**2 + torch.sin(x)

# Compare with the real value 
true_dy = 2*x+torch.cos(x)# This result has two quantities , One is tensor, the other is gradient type 
true_d2y = 2 - torch.sin(x)

# Find the first derivative , Save the calculation diagram and then find the second derivative 
dy = torch.autograd.grad(y,x,
                         grad_outputs=torch.ones(x.shape),# This parameter should be manually specified 
                         create_graph=True,
                         retain_graph=True# Save calculation diagram 
                        )
# After the tensor, add .detach().numpy() Only tensor values can be output 
print(" First derivative true value ：{} \n First derivative calculation value ：{}".format(true_dy.detach().numpy(),dy[0].detach().numpy()))

# Find the second derivative . above dy The first element of is the first derivative 
d2y = torch.autograd.grad(dy[0],x,
                         grad_outputs=torch.ones(x.shape),
                         create_graph=False# No more calculation charts , Destroy the previous calculation diagram 
                         )
print("\n Second order conduction true value ：{} \n Second derivative calculation value ：{}".format(true_d2y.detach().numpy(),d2y[0].detach().numpy()))