Pytoch learning notes -- Summary of common functions 2

Catalog

1-- torch.nn.Sequential( ) function

2--torch.flatten() function

3--Model.train() Functions and Model.eval() function

​​​​​​​

1-- torch.nn.Sequential( ) function

effect ： adopt Sequential sequence Create neural network structure

Example ： call Block Function time , The in parentheses will be executed sequentially Function.

import torch

x = torch.randn(1, 1, 64, 64)

Block = torch.nn.Sequential(
    torch.nn.Conv2d(in_channels = 1, out_channels = 32, kernel_size = 3, stride = 1),
    torch.nn.ReLU(),
    torch.nn.Conv2d(in_channels = 32, out_channels = 32, kernel_size = 3, stride = 1),
    torch.nn.ReLU()
)

result = Block(x)
print(result)

The above code is equivalent to ：

import torch

Conv1 = torch.nn.Conv2d(in_channels = 1, out_channels = 32, kernel_size = 3, stride = 1)
Conv2 = torch.nn.Conv2d(in_channels = 32, out_channels = 32, kernel_size = 3, stride = 1),
Relu = torch.nn.ReLU()

x = torch.randn(1, 1, 64, 64)
print(x)

x = Conv1(x)
x = Relu(x)
x = Conv2(x)
result = Relu(x)

print(result)

############################

2--torch.flatten() function

effect ： Splicing by dimension ;torch.flatten(input, start_dim = 0, end_dim = -1);input For input tensor data ,start_dim Is the starting dimension of splicing ,end_dim Is the termination dimension of splicing .

Example ：

import torch

x = torch.randn(2, 3, 3)
print(x)

result1 = torch.flatten(x, start_dim = 0, end_dim = 2)
print(result1)

result2 = torch.flatten(x, start_dim = 0, end_dim = 1)
print(result2)

result3 = torch.flatten(x, start_dim = 1, end_dim = 2)
print(result3)

result ：

tensor([[[ 0.3546, -0.8551,  2.3490],
         [-0.0920,  0.0773, -0.4556],
         [-1.6943,  1.4517, -0.0767]],

        [[-0.6950,  0.4382, -1.2691],
         [-0.0252, -0.4980, -0.5994],
         [-0.2581, -0.2544, -0.6787]]]) #X

tensor([ 0.3546, -0.8551,  2.3490, -0.0920,  0.0773, -0.4556, -1.6943,  1.4517,
        -0.0767, -0.6950,  0.4382, -1.2691, -0.0252, -0.4980, -0.5994, -0.2581,
        -0.2544, -0.6787]) #result1

tensor([[ 0.3546, -0.8551,  2.3490],
        [-0.0920,  0.0773, -0.4556],
        [-1.6943,  1.4517, -0.0767],
        [-0.6950,  0.4382, -1.2691],
        [-0.0252, -0.4980, -0.5994],
        [-0.2581, -0.2544, -0.6787]]) #result2

tensor([[ 0.3546, -0.8551,  2.3490, -0.0920,  0.0773, -0.4556, -1.6943,  1.4517,
         -0.0767],
        [-0.6950,  0.4382, -1.2691, -0.0252, -0.4980, -0.5994, -0.2581, -0.2544,
         -0.6787]]) #result3

############################

3--Model.train() Functions and Model.eval() function

Model.train() Function is used for the training state of the model , Place the model in Training mode ;

Model.eval() Function is used for the test state of the model , Place the model in Test mode ;

The model is in two different modes of training and testing , Some functions have different functions , Such as ：Batch Normalization and Dropout（ Training mode will be enabled , In test mode, it is disabled ）;

If there are BN layer (Batch Normalization） and Dropout, You need to add model.train().model.train() It's a guarantee BN Layer can use The mean and variance of each batch of data . about Dropout,model.train() It's random Take part of the network connection To train and update parameters .

If there are BN layer (Batch Normalization） and Dropout, Add... When testing model.eval().model.eval() It's a guarantee BN The layer can be used Mean and variance of all training data , That is to say, in the process of testing, we should ensure that BN The mean and variance of the layers are constant . about Dropout,model.eval() Is to use All network connections , That is, there is no random abandonment of neurons .

After training train After sample , The resulting model model To test samples . stay model(test) Before , Need to add model.eval(), Otherwise , There is input data , Even without training , It also changes the weights . This is a model contains BN Layer and the Dropout The nature of that .

Reference link ​​​​​​​