Loss function~

Concept ：

The loss function is the function used to calculate the difference between the tag value and the predicted value , In the process of machine learning , There are a variety of loss functions to choose from , A typical distance vector , Absolute vector, etc .

  The diagram above is a schematic diagram for automatic learning of linear equations . Thick lines are real linear equations , The dotted line is a schematic diagram of the iterative process ,w1 Is the weight of the first iteration ,w2 Is the weight of the second iteration ,w3 Is the weight of the third iteration . As the number of iterations increases , Our goal is to make wn Infinitely close to the real value .

In the figure 1/2/3 The three labels are 3 Predict in the next iteration Y Value and reality Y The difference between values （ The difference here means the loss function , Yes, of course , There are many formulas for calculating the difference in practical application ）, The difference here is represented by absolute difference on the diagram . There is also a square difference in multidimensional space , Mean square deviation and other different distance calculation formulas , That is, the loss function .

Common calculation methods of loss function ：

1. nn.L1Loss Loss function

L1Loss The calculation method is very simple , Take the average of the absolute error between the predicted value and the real value .

criterion = nn.L1Loss()
loss = criterion(sample, target)
print(loss)

# 1

The final output is 1

Calculation logic is as follows ：

• First calculate the sum of absolute differences ：|0-1|+|1-1|+|2-1|+|3-1| = 4
• And then average ：4/4 = 1

2. nn.SmoothL1Loss

SmoothL1Loss Also called Huber Loss, Error in （-1,1） It's the square loss , Other things are L1 Loss .

criterion = nn.SmoothL1Loss()
loss = criterion(sample, target)
print(loss)

# 0.625

The final output is 0.625

Calculation logic is as follows ：

• First calculate the sum of absolute differences ：
• And then average ：2.5/4 = 0.625

3. nn.MSELoss

Square loss function . The calculation formula is the average of the sum of squares between the predicted value and the real value .

criterion = nn.MSELoss()
loss = criterion(sample, target)
print(loss)

# 1.5

The final output is 1.5

Calculation logic is as follows ：

• First calculate the sum of absolute differences ：
• And then average ：6/4 = 1.5

 4. nn.BCELoss

Cross entropy for binary classification , Its calculation formula is complex , Here is mainly a concept , In general, it won't be used .

criterion = nn.BCELoss()
loss = criterion(sample, target)
print(loss)

# -13.8155