Hands on learning and deep learning -- simple implementation of linear regression

1、 Imported packets

import numpy as np
import torch
import torch.utils.data as Data #PyTorch Provides  data  Package to read data . because  data  often ⽤ Make variable name , We will lead ⼊ Of  data  modular ⽤ Data  Instead of .
from torch import nn, optim
from torch.nn import init

2、 Generate data set , among features Training data characteristics ,labels Is the label .

num_inputs=2  # The number of entries 
num_examples=1000 # Training data set 
true_w=[2,-3.4] # Real weight 
true_b=4.2 # True deviation 
features=torch.tensor(np.random.normal(0,1,(num_examples,num_inputs)),dtype=torch.float)
#  The mean for 0, The variance of 1 The random number  , Yes num_examples Samples , The number of columns is num_inputs
labels=true_w[0]*features[:,0]+true_w[1]*features[:,1]+true_b
# lables Is equal to w For each column multiplied by features Each column of is then added , Finally, add the deviation true_b;
labels+=torch.tensor(np.random.normal(0,0.01,size=labels.size()),dtype=torch.float)
#  Added a noise , The mean for 0, The variance of 0.01, Shape and lables It's the same length 

3、 Reading data

PyTorch Provides  data  Package to read data .

batch_size=10  #10 Of data samples ⼩ Batch 
dataset=Data.TensorDataset(features,labels) # Combine the characteristics and labels of training data 
data_iter=Data.DataLoader(dataset,batch_size,shuffle=True) # Random read small batch 
# Next, read and print the first small batch data sample 
for X,y in data_iter:
    print(X,y)
    break

 

4、 Defining models  , how ⽤ nn.Module  Realization ⼀ A linear regression model 

class LinearNet(nn.Module): #nn Is the abbreviation of neural network 
    def __init__(self,n_feature):
        super(LinearNet, self).__init__()
        self.linear=nn.Linear(n_feature,1)
    def forward(self,x): # Return the forward propagation method of the output 
        y=self.linear(x)
        return y
net =LinearNet(num_inputs)
print(net)

 In fact, we can ⽤ nn.Sequential  Come more ⽅ Convenient construction ⽹ Collateral , Sequential  yes ⼀ An orderly container ,⽹ Collateral 
 The layer will be transmitted in accordance with ⼊ Sequential  The order of is added to the calculation diagram in turn .

#  How to write it ⼀
net = nn.Sequential(
 nn.Linear(num_inputs, 1)
 #  Here you can also pass ⼊ Other layers 
 )
#  How to write it ⼆
net = nn.Sequential()
net.add_module('linear', nn.Linear(num_inputs, 1))
# net.add_module ......
#  Write three 
from collections import OrderedDict
net = nn.Sequential(OrderedDict([
 ('linear', nn.Linear(num_inputs, 1))
 # ......
 ]))
print(net)
print(net[0])

 

 Can pass  net.parameters()  To see all the learnable parameters of the model , This function will return ⼀ individual ⽣ Make it .

for param in net.parameters():
 print(param)

5、 Initialize model

#init  yes  initializer( Initialization settings )  Abbreviated form 
init.normal_(net[0].weight,mean=0,std=0.01) # adopt  init.normal_  Initialize each element of the weight parameter to random sample at a mean value of 0、 The standard deviation is 0.01 Is a normal distribution 
init.constant_(net[0].bias,val=0) # The deviation is initialized to zero 

6、 Define the loss function 

loss=nn.MSELoss()# The mean square error is calculated using MSELoss class , Also called square L2 norm 

7、 Define optimization algorithms

#torch.optim  The module provides many common functions ⽤ Optimization algorithm ⽐ Such as SGD、Adam and RMSProp etc. 
optimizer=optim.SGD(net.parameters(),lr=0.03) # Take out all the parameters w and b, The learning rate is 0.03
print(optimizer)

About learning rate

#  If the learning rate is not specified for a certain parameter , Just make ⽤ Default learning rate of the outermost layer 
#  Adjust the learning rate 
for param_group in optimizer.param_groups:
 param_group['lr'] *= 0.1 #  The learning rate is the same as before 0.1 times 
print(optimizer)

 

8、 Training models Through the tune ⽤ optim Example of step Function to iterate over model parameters .

num_epochs = 3 # Do three cycles 
for epoch in range(1, num_epochs + 1): # Circular function 
    for X, y in data_iter:  # Take out small batches one at a time 
        output = net(X) # Output to net Inside 
        l = loss(output, y.view(-1, 1))
        optimizer.zero_grad() #  Gradient clear , Equivalent to net.zero_grad()
        l.backward() # Calculate the gradient 
        optimizer.step() # call step Function to update the model 
    print('epoch %d, loss: %f' % (epoch, l.item()))

⽐ Compare the learned model parameters with the real model parameters . We from  net  Get the required layer , And access to it ᯿（ weight ） And deviation （ bias ）

dense = net[0]
print(true_w, dense.weight)
print(true_b, dense.bias)