Pytorch crossentropyloss learning

class torch.nn.CrossEntropyLoss(weight=None,size_average=None,ignore_index=-100,reduce=None,reduction='elementwise_mean')

function ：
        Pass the input through softmax After activating the function , Then calculate it and target The cross entropy loss of . That is, the method will nn.LogSoftmax() and nn.NLLLoss() Combined with . Strictly speaking, the cross loss function should be nn.NLLLoss()

        Cross entropy loss (cross-entropy Loss) Also known as log likelihood loss (Log-likelihood loss), Logarithmic loss ; In the second classification, it can also be called logistic regression loss . The expression of cross entropy loss function is L=-sigama(y_ilog(x_i)).PyTorch First the input after softmax Activation function , Vector " normalization " In the form of probability , And then with target Calculate the cross entropy loss in the strict sense .

      In multi category tasks , Always use softmax Activation function + Cross entropy loss function , Because cross entropy describes the difference between two probability distributions , But the neural network outputs vectors , It's not in the form of a probability distribution . So we need to softmax The activation function performs a vector " normalization " In the form of probability distribution . So we need to softmax The activation function performs a vector " normalization " In the form of probability distribution , And then the cross entropy loss function is used to calculate Loss.PyTorch Of CrossEntropyLoss(), Will be nn.LogSoftmax() and nn.NLLLoss() Combined with ,nn.LogSoftmax() Equivalent to activation function ,nn.NLLLoss() Is the loss function , Combine it , So I think this function should be called softmax+ Cross entropy loss function .

example ：

import torch
import torch.nn as nn
import numpy as np
entroy = nn.CrossEntropyLoss()
input = torch.Tensor([[-0.7715,-0.6205,-0.2562]])
target = torch.tensor([0])
output = entroy(input,target)
print(output) # use CrossEntropyLoss The result of the calculation .
myselfout = -(input[:,0])+np.log(np.exp(input[:,0])+np.exp(input[:,1])+np.exp(input[:,2])) # Calculate the result with formula 
print(myselfout)
lsf = nn.LogSoftmax()
loss = nn.NLLLoss()
lsfout = lsf(input)
lsfnout = loss(lsfout,target)
print(lsfnout)