当前位置:网站首页>Pytorch crossentropyloss learning
Pytorch crossentropyloss learning
2022-07-01 18:02:00 【Dream without trace 123】
class torch.nn.CrossEntropyLoss(weight=None,size_average=None,ignore_index=-100,reduce=None,reduction='elementwise_mean')
function :
Pass the input through softmax After activating the function , Then calculate it and target The cross entropy loss of . That is, the method will nn.LogSoftmax() and nn.NLLLoss() Combined with . Strictly speaking, the cross loss function should be nn.NLLLoss()
Cross entropy loss (cross-entropy Loss) Also known as log likelihood loss (Log-likelihood loss), Logarithmic loss ; In the second classification, it can also be called logistic regression loss . The expression of cross entropy loss function is L=-sigama(y_ilog(x_i)).PyTorch First the input after softmax Activation function , Vector " normalization " In the form of probability , And then with target Calculate the cross entropy loss in the strict sense .
In multi category tasks , Always use softmax Activation function + Cross entropy loss function , Because cross entropy describes the difference between two probability distributions , But the neural network outputs vectors , It's not in the form of a probability distribution . So we need to softmax The activation function performs a vector " normalization " In the form of probability distribution . So we need to softmax The activation function performs a vector " normalization " In the form of probability distribution , And then the cross entropy loss function is used to calculate Loss.PyTorch Of CrossEntropyLoss(), Will be nn.LogSoftmax() and nn.NLLLoss() Combined with ,nn.LogSoftmax() Equivalent to activation function ,nn.NLLLoss() Is the loss function , Combine it , So I think this function should be called softmax+ Cross entropy loss function .
example :
import torch import torch.nn as nn import numpy as np entroy = nn.CrossEntropyLoss() input = torch.Tensor([[-0.7715,-0.6205,-0.2562]]) target = torch.tensor([0]) output = entroy(input,target) print(output) # use CrossEntropyLoss The result of the calculation . myselfout = -(input[:,0])+np.log(np.exp(input[:,0])+np.exp(input[:,1])+np.exp(input[:,2])) # Calculate the result with formula print(myselfout) lsf = nn.LogSoftmax() loss = nn.NLLLoss() lsfout = lsf(input) lsfnout = loss(lsfout,target) print(lsfnout)
边栏推荐
- Is the software of futures pioneer formal and safe? Which futures company is safer to choose?
- China PBAT resin Market Forecast and Strategic Research Report (2022 Edition)
- Kernel stray cat stray dog pet adoption platform H5 source code
- Product service, operation characteristics
- February 16, 2022 Daily: graph neural network self training method under distribution and migration
- L'ouverture d'un compte d'actions en ligne est - elle sécurisée? Fiable?
- JS how to convert a string with a delimiter into an n-dimensional array
- D @ safety and dip1000
- Yolov5 practice: teach object detection by hand
- Penetration practice vulnhub range Nemesis
猜你喜欢
(十六)ADC转换实验
【Try to Hack】vulnhub DC4
(十七)DAC转换实验
The difference and relationship between iteratible objects, iterators and generators
Fix the black screen caused by iPhone system failure
Technical secrets of ByteDance data platform: implementation and optimization of complex query based on Clickhouse
New patent applications and transfers
Apache iceberg source code analysis: schema evolution
Common design parameters of solid rocket motor
[splishsplash] about how to receive / display user parameters, MVC mode and genparam on GUI and JSON
随机推荐
transform. Forward and vector3 Differences in the use of forward
Replace UUID, nanoid is faster and safer!
Quick foundation of group theory (5): generators, Kelley graphs, orbits, cyclic graphs, and "dimensions" of groups?
The latest intelligent factory MES management system software solution
Data warehouse (3) star model and dimension modeling of data warehouse modeling
[splishsplash] about how to receive / display user parameters, MVC mode and genparam on GUI and JSON
PHP implements sensitive word filtering system "suggestions collection"
Good looking UI mall source code has been scanned, no back door, no encryption
Report on research and investment prospects of China's silicon nitride ceramic substrate industry (2022 Edition)
聊聊项目经理最爱使用的工具
PHP实现敏感词过滤系统「建议收藏」
Maizeer: the two batches of products reported by the media have been taken off the shelves and sealed, and consumer appeals are accepted
. Net cloud native architect training camp (permission system code implements actionaccess) -- learning notes
Irradiance, Joule energy, exercise habits
ISO 27001 Information Security Management System Certification
In depth Research Report on China's disposable sanitary products production equipment industry (2022 Edition)
From comedians to NBA Zhan Huang, check the encrypted advertisements during this super bowl
Heavy disclosure! Hundreds of important information systems have been invaded, and the host has become a key attack target
SLO is increasingly used to achieve observability | Devops
Kia recalls some K3 new energy with potential safety hazards