2 neural network toolbox NN

1. In practice , The most common practice is to inherit nn.Module, Write your own network layer . Here's how to use nn.Module Realize your own full connection layer .

Fully connected layer , Also known as affine layer , Input y and x Satisfy y=Wx+b,W,b It's a learnable parameter .

import torch as t
import torch.nn as nn
import torch.nn.functional as F
from torch.autograd import Variable as V

class Linear(nn.Module):# Inherit nn.Module
    def __init__(self,in_features,outfeatures):
        super(Linear,self).__init__()# Equivalent to nn.Module.__init__(self)
        self.w=nn.Parameter(t.randn(in_features,outfeatures))
        self.b=nn.Parameter(t.randn(outfeatures))
    def forward(self, x):
        x=x.mm(self.w)
        return x+self.b.expand_as(x)

    # expand_as (a)
    #  This is a tensor A built-in method of variable , If you use b.expand_as (a)
    #  Will be b To expand , Expand to a Dimensions , It should be noted that a The low dimension of needs to be better than b Big , for example b Of shape yes 3 * 1, If a Of shape yes 3 * 2
    #  No mistakes , But it is 2 * 2  It's going to be a mistake 
layer=Linear(4,3)
input=V(t.randn(2,4))
output =layer(input)
print(output)

Output ：

tensor([[-3.0932,  1.7944, -0.4888],
        [-1.8203,  1.4885, -0.2306]], grad_fn=<AddBackward0>)

so , The implementation of full connection layer is simple , But we need to pay attention to the following points ：

1） Custom layer Linear Must inherit nn.Module, And call... In its constructor nn.Module Constructor for , namely super(Linear,self)__init__(self).

2） In the constructor __init__ You must define your own learnable parameters , And encapsulate it into Parameter, As in this example, we put w,b Encapsulated into Parameter. It's a special Variable, But by default, it requires derivation （required_grad=True).

3）forward Function to realize the forward propagation process , Its input can be one or more variable, Yes x Any operation of must also be variable Supported operations .

4） No need to write back propagation function , Because its forward propagation is right variable To operate ,nn.Module Be able to use autograd Automatic back propagation , This is better than Function Simple and many .

 

2.Module Can automatically detect their own parameter, And as a learning parameter . except parameter,Module Also contains children Module, Lord Module Can recursively find sub Module Medium parameter.

Multilayer perceptron , It consists of two fully connected layers , use sigmoid Function as activation function .

class Perceptron(nn.Module):
    def __init__(self,in_features,hidden_features,out_features):
        nn.Module.__init__(self)
        #super (Perceptron, self).__init__ ()
        self.layer1=Linear(in_features,hidden_features)
        self.layer2 = Linear (hidden_features, out_features)
    def forward(self, x):
        x=self.layer1(x)
        x=t.sigmoid(x)
        return self.layer2(x)
perceptron =Perceptron(3,4,1)
for name,parameter in perceptron.named_parameters():
    print(name,parameter.size())#w,b

Output ：

layer1.w torch.Size([3, 4])
layer1.b torch.Size([4])
layer2.w torch.Size([4, 1])
layer2.b torch.Size([1]）

1） Constructors __init__ in , You can use the customized Linear layer （Module） As the present Module A child of the object of Module, His learnable parameters , It will also become the present Module The learnable parameters of .

2） In the forward propagation function , We consciously Name the output variables x, It's to make Python Recycle some output from the middle layer , thus Save memory .

3）Module in parameter The global naming conventions for are as follows ：

①Parameter Direct naming . for example self.param_name=nn.Parameter(t.randn(3,4)), Name it param_name.

② Son Module Medium parameter, Will add the current before its name Module Name . for example ：self.sub_module=SubModel(),SubModel There is one of them. parameter Is also called param_name, Then the two are combined parameter name Namely sub_module.param_name.

4） Arguments to the constructor , Such as nn.Linear(in_feature,out_feature,bias), We need to pay attention to the role of these three parameters .

attribute 、 Learnable parameters and subsets Module. Such as nn.Linear There is weight and bias Two learnable parameters , It doesn't contain children Module.

The shape of the input and output , Such as nn.linear The input shape for is （N,input_features), Output is （N,output_features),N yes batch_size.

Be careful ： Self defined layer There are assumptions about the input shape ： The input is not a single data , It is a batch. If you want to input a data , Must call unsqueeze(0) Function disguised as batch_size=1 Of batch.