Learning notes of "hands on learning in depth"

《 Hands-on deep learning 》 Learning notes

Data manipulation

Automatic derivation

Variable setting derivation

x = torch.ones(2, 2, requires_grad=True)

Obtain derivative

out.backward()

Here, the dependent variable is required to be scalar , If tensor, You need to provide a weight matrix with the same size as the dependent variable , The output dependent variable is transformed into a scalar by weighted summation of all elements , Then you can backward().
The reason is well understood ： There is no relationship between dependent variables , All dependent variable elements are simply put together , Therefore, their permutation can be regarded as a one-dimensional vector and linearly weighted to a scalar l On . The advantage of this is that the gradient is independent of the latitude of the dependent variable , Got l And the gradient of the independent variable , It doesn't matter what shape the dependent variable is tensor.

Linear regression

The gradient descent method solves the parameters

The above formula can be solved analytically .

The parameters to be solved can also be optimized by gradient descent through linear regression . Small batch random gradient descent is usually used （mini-batch stochastic gradient descent） Method , That is, calculate the average gradient in small batches , Multi batch optimization parameters .

Here is also multiplied by a learning rate , It is equivalent to step size , It can be a little bigger at first , The back should be smaller .

Fully connected layer （ Dense layer ）

Linear regression

1. Build simulation data , Confirm the input （features）, Input （label）, Parameters
2. Write data loader（ Split data into batch）
3. structure function (net) , loss And optimization method
4. iteration epoch solve
   

Linear regression concise version

1. data fetch
   

2. Define your own function, You need to give the number of parameters and forward function . In fact, it is the functional relationship between input and output . therefore , Give the calculation method of input and output , namely forward function . Neural network replaces this functional relationship with grid structure .
   

3. use torch Of net structure
   

4. Loss
   

5. An optimization method