Pytoch notes

It is mainly used to record some used but infrequently used pytorch Method , In alphabetical order .

C

1. nn.ChannelShuffle()
Function function ： Disturb the channel order of the original feature map , That is, the input characteristic diagram is （B,C,H,W）, The size of the output feature map is also （B,C,H,W）, however C The order of dimensions on the dimension will be disturbed . Parameters group(int) To indicate a division channel Number of groups .

channel_shuffle = nn.ChannelShuffle(2)
input = torch.randn(1, 4, 2, 2)
print(input)
''' [[[[1, 2], [3, 4]], [[5, 6], [7, 8]], [[9, 10], [11, 12]], [[13, 14], [15, 16]], ] '''
output = channel_shuffle(input)
print(output)
''' [[[[1, 2], [3, 4]], [[9, 10], [11, 12]], [[5, 6], [7, 8]], [[13, 14], [15, 16]], ]] '''

D

1. nn.Dropout(p=float)
Dropout The role of ： Set to prevent over fitting ,p Indicates that each neuron has p Probability of not being activated , It is generally used after full connection network mapping .

nn.Droupout(p=0.3)

P

1. nn.Parameter()
Except convolution in neural networks 、 Other operations such as full connection require some additional parameters, which will also be learned and trained along with the whole network . For example, the weight parameter in the attention mechanism 、Vision Transformer in positional embedding etc. .

#  With ViT Medium position embedding For example 
self.pos_embedding = nn.Parameter(torch.randn(1, num_patches+1, dim))