pytorch 筆記 torch.nn.BatchNorm1d

1 主要參數介紹

torch.nn.BatchNorm1d(num_features, eps=1e-05)

num_features	需要進行歸一化的數據維度，一般等於dim_hid
eps	加到分母上的值，以保證數值穩定性

2 num_feature分析

batch normalization是對一個batch裏面的每一個維度分別進行歸一化

 

舉一個例子：

a=torch.Tensor([[1,1,1],
                [2,-3,5],
                [0.1,-0.3,0.5],
                [1.2,-1.3,1.5]])
f1=torch.nn.BatchNorm1d(3)
f2=torch.nn.BatchNorm1d(4)
print(a,'\n',f1(a),'\n',f2(a.T))

'''
tensor([[ 1.0000,  1.0000,  1.0000],
        [ 2.0000, -3.0000,  5.0000],
        [ 0.1000, -0.3000,  0.5000],
        [ 1.2000, -1.3000,  1.5000]]) 
 tensor([[-0.1110,  1.3003, -0.5657],
        [ 1.3685, -1.4372,  1.6971],
        [-1.4425,  0.4106, -0.8485],
        [ 0.1849, -0.2738, -0.2828]], grad_fn=<NativeBatchNormBackward0>) 
 tensor([[ 0.0000e+00,  2.0203e-01,  2.2812e-08,  5.8423e-01],
        [ 0.0000e+00, -1.3132e+00, -1.2247e+00, -1.4075e+00],
        [ 0.0000e+00,  1.1112e+00,  1.2247e+00,  8.2323e-01]],
       grad_fn=<NativeBatchNormBackward0>)
'''

num_features是3，錶示每個sample有三個feature，那麼就有三組數據分別要進行歸一化