当前位置:网站首页>Performance measure of classification model
Performance measure of classification model
2022-07-07 16:11:00 【_ Spring_】
Catalog
Confusion matrix (Confusion matrix)
For dichotomies , The actual data can be divided into positive examples and negative examples . According to the discrimination category of the model and the actual category of the data , Four results can be obtained : Real examples (True positive), False positive example (False positive), True counter example (True negative), False counter example (False negative).
If the discrimination category is consistent with the actual category , It is true , atypism , False , in other words , Real examples Means , The classification of the model is consistent with the actual , All positive examples ; and False positive example The meaning of is false positive example : The discrimination is positive , But it's wrong ( Actually, it is a counterexample ); False counter example It's a false counterexample : It is judged as a counterexample , But the judgment is wrong , In fact, it is a positive example .
use TP、FP、TN、FN To represent the number of corresponding results , The confusion matrix of classification results can be obtained :
- | The prediction is positive | Negative prediction |
---|---|---|
Actual positive example | TP | FN |
Actual counterexample | FP | TN |
Consider an example , Now there is 100 people , One of the boys 70 people , girl student 30 people . There is a model to classify boys and girls . The discrimination result of the model is : schoolboy 60 people ( What is really a boy is 55 people , rest 5 People are girls ), girl student 40 people ( What is really a girl is 25 people , in addition 15 For boys ). Then the confusion matrix can be expressed as :
- | Predicted to be male | Predicted to be female |
---|---|---|
Actual boys | 55 | 15 |
Actual girls | 5 | 25 |
Commonly used evaluation index
Accuracy
The accuracy of Chinese Translation / precision .
It refers to , The proportion of samples with correct classification in the total sample books .
In the confusion matrix ,TP and TN All belong to the samples with correct classification , therefore ,
A c c u r a c y = T P + T N T P + F P + F N + T N Accuracy= \frac{TP+TN} {TP+FP+FN+TN} Accuracy=TP+FP+FN+TNTP+TN
Use the example of male and female students above to calculate Accuracy Words ,acc=(55+25)/100=0.80
Precision
Chinese translation is accuracy / Precision rate .
It refers to the sample in which the model is judged as a positive example , How many are real positive examples , therefore ,
P r e c i s i o n = T P T P + F P Precision = \frac{TP}{TP+FP} Precision=TP+FPTP
Common scenarios are “ How much information is retrieved that users are really interested in ”.
Use the example of male and female students above to calculate Precision Words ,precision=55/(55+5)=0.917
Recall
Recall rate / Recall rate .
It refers to all positive examples , How many are judged as positive examples by the model , therefore
R e c a l l = T P T P + F N Recall = \frac{TP}{TP+FN} Recall=TP+FNTP
Common scenarios are “ In the information retrieval of all fugitives , How many fugitives can be detected ”.
Use the example of male and female students above to calculate Recall Words ,recall=55/(55+15)=0.786
F1 value
frequently-used F1 The value is calculated as :
F 1 = 2 ∗ P r e c i s i o n ∗ R e c a l l P r e c i s i o n + R e c a l l = 2 ∗ T P sample Ben total Count + T P − T N F1=\frac{2*Precision*Recall}{Precision+Recall}=\frac{2*TP}{ The total number of samples +TP-TN} F1=Precision+Recall2∗Precision∗Recall= sample Ben total Count +TP−TN2∗TP
Use the example of boys and girls above to calculate ,F1=(20.9170.786)/(0.917+0.786)=0.846, Or is it F1=(2*55)/(100+55-25)=0.846
PR curve
PR The curve is based on precision For the vertical axis ,recall Is the horizontal axis , The curve drawn .
PR The larger the area under the curve , The better the performance .
On the curve , When precision And recall When the values of are equal , This point is the equilibrium point (Break-Even Point).
ROC curve
ROC Its full name is “ Work characteristics of subjects ”.
The vertical axis is the true case rate (TPR), The horizontal axis is false positive rate (FPR).
T P R = T P T P + F N TPR= \frac{TP}{TP+FN} TPR=TP+FNTP
F P R = F P T N + F P FPR=\frac{FP}{TN+FP} FPR=TN+FPFP
AUC
Express Area under ROC Cureve, yes ROC The area under the curve .
Larger area , The better the performance .
Cost curve The cost curve
The previous evaluation criteria focus on TP, Actually FP and FN It is also important in some scenarios .
For example, a medical scene : The cancer diagnosis classifier classifies healthy patients into cancer patients , Or classify cancer patients as healthy patients . These two scenarios are the misclassification of the model , But the consequences of the latter situation are more serious .
therefore , In order to better measure the different losses caused by different mistakes , Give fault to “ Unequal costs ”(unequal cost).
At the time of calculation , We need to optimize the overall cost (total cost).
边栏推荐
- Numpy -- epidemic data analysis case
- 持续创作,还得靠它!
- Ue4/ue5 multi thread development attachment plug-in download address
- You Yuxi, coming!
- 融云斩获 2022 中国信创数字化办公门户卓越产品奖!
- 分类模型评价标准(performance measure)
- 深度之眼(六)——矩阵的逆(附:logistic模型一些想法)
- Xcode Revoke certificate
- The "go to definition" in VS2010 does not respond or prompts the solution of "symbol not found"
- Unity3D_ Class fishing project, bullet rebound effect is achieved
猜你喜欢
Three. JS introductory learning notes 07: external model import -c4d to JSON file for web pages -fbx import
numpy--疫情数据分析案例
Unity3D_ Class fishing project, control the distance between collision walls to adapt to different models
通知Notification使用全解析
Numpy -- epidemic data analysis case
It's different for rich people to buy a house
Eye of depth (VII) -- Elementary Transformation of matrix (attachment: explanation of some mathematical models)
Three. JS introductory learning notes 08:orbitcontrols JS plug-in - mouse control model rotation, zoom in, zoom out, translation, etc
Numpy --- basic learning notes
Rongyun won the 2022 China Xinchuang digital office portal excellence product award!
随机推荐
95.(cesium篇)cesium动态单体化-3D建筑物(楼栋)
20th anniversary of agile: a failed uprising
SPI master RX time out interrupt
How does geojson data merge the boundaries of regions?
【花雕体验】15 尝试搭建Beetle ESP32 C3之Arduino开发环境
Continuous creation depends on it!
持续创作,还得靠它!
神经网络c语言中的指针是怎么回事
通知Notification使用全解析
Three. JS introduction learning notes 12: the model moves along any trajectory line
Unity3D_ Class fishing project, control the distance between collision walls to adapt to different models
js中复选框checkbox如何判定为被选中
一个普通人除了去工厂上班赚钱,还能干什么工作?
Apache Doris just "graduated": why should we pay attention to this kind of SQL data warehouse?
The "go to definition" in VS2010 does not respond or prompts the solution of "symbol not found"
Three. JS introductory learning notes 15: threejs frame animation module
Odoo集成Plausible埋码监控平台
Good news! Kelan sundb database and Hongshu technology privacy data protection management software complete compatibility adaptation
Unity3d click events added to 3D objects in the scene
强化实时数据管理,英方软件助力医保平台安全建设