Pytorch notes torch nn. BatchNorm1d

1 The main parameters are introduced

torch.nn.BatchNorm1d(num_features, eps=1e-05)

num_features	Data dimensions that need to be normalized , Generally equal to dim_hid
eps	Value added to the denominator , To ensure numerical stability

2 num_feature analysis

batch normalization Yes, one. batch Each dimension is normalized separately

 

For example ：

a=torch.Tensor([[1,1,1],
                [2,-3,5],
                [0.1,-0.3,0.5],
                [1.2,-1.3,1.5]])
f1=torch.nn.BatchNorm1d(3)
f2=torch.nn.BatchNorm1d(4)
print(a,'\n',f1(a),'\n',f2(a.T))

'''
tensor([[ 1.0000,  1.0000,  1.0000],
        [ 2.0000, -3.0000,  5.0000],
        [ 0.1000, -0.3000,  0.5000],
        [ 1.2000, -1.3000,  1.5000]]) 
 tensor([[-0.1110,  1.3003, -0.5657],
        [ 1.3685, -1.4372,  1.6971],
        [-1.4425,  0.4106, -0.8485],
        [ 0.1849, -0.2738, -0.2828]], grad_fn=<NativeBatchNormBackward0>) 
 tensor([[ 0.0000e+00,  2.0203e-01,  2.2812e-08,  5.8423e-01],
        [ 0.0000e+00, -1.3132e+00, -1.2247e+00, -1.4075e+00],
        [ 0.0000e+00,  1.1112e+00,  1.2247e+00,  8.2323e-01]],
       grad_fn=<NativeBatchNormBackward0>)
'''

num_features yes 3, Represent each sample There are three feature, Then there are three groups of data to be normalized respectively