Evaluation index of anomaly detection: rocauc et al. [tips]

Preface

Divided into picture level （ Label the whole picture ） And pixel level （ Label each pixel ） Of .

ROCAUC

roc

from sklearn.metrics import roc_curve
fpr, tpr, thresholds = roc_curve(gt_list, img_scores)

Parameters ： label , Evaluation score （ That is, the probability that each sample is predicted to be positive ）, Positive sample label （ The default is 0）. Return results ：fpr,tpr,thresholds.

fpr： Negative positive rate , The negative category is predicted to be the proportion of the positive category in all negative categories .

tpr： Real class rate , The positive category predicts that the positive category accounts for the proportion of all positive categories .

threshold： threshold

auc

from sklearn.metrics import roc_auc_score
img_roc_auc = roc_auc_score(gt_list, img_scores)

How did it come about

threshold Is the evaluation score of each sample , That is, the probability that each sample is predicted to be positive , There is a probability value , Is this sample a positive sample or a negative sample . So the rules are , When threshold When taking the probability value of one sample , Greater than this threshold The probability value of is defined as positive class , Less than, it is defined as negative class , thus A little reasoning once ,threshold The smaller it is ,fpr,tpr The greater the ,threshold The bigger it is ,fpr,tpr The smaller it is . such , Do the threshold value for the probability values of all samples , With fpr The horizontal axis ,tpr The curve of the vertical axis can be drawn , namely roc curve .

auc Namely roc The area under the curve , From the above ,auc Of course, the closer 1 Better .

Reference resources

ROC Curve and AUC Calculation summary _ Taotie competitive blog -CSDN Blog _roc Curve and aucsklearn Draw in ROC A function of a curve roc_curve() explain _Stephen__W The blog of -CSDN Blog _roc_curveROC Curves and AUC Area understanding _Microstrong0305 The blog of -CSDN Blog _auc curve AUC Detailed explanation _ac7 The blog of -CSDN Blog _auc Detailed explanation

Other evaluation indicators

precision,recall

from sklearn.metrics import precision_recall_curve
precision, recall, thresholds = precision_recall_curve(gt_mask.flatten(), scores.flatten())

  Setting of function parameters , The return of results , The principle of calculation is the same as the above auc be similar , Analogy .

  There is such a confusion matrix ：

actual	forecast
	just	negative
just	TP（ The real class ）	FN（ False negative class ）
negative	FP（ Pseudopositive class ）	TN（ True negative class ）

precision=tp / (tp + fp): Accuracy rate , Positive category prediction is the proportion of positive category in all predicted positive categories .

recall=tp / (tp + fn): Recall rate , That is to say tpr.

The two indicators are often inversely proportional , That is, the greater the accuracy , The smaller the recall rate . It can be understood a little from the definition , High accuracy , Then the probability that the negative category is predicted to be the positive category is low , High recall rate , Then the probability that the positive category is predicted to be the negative category is low , This equates them to the class prediction problem , Further understanding , You know the inverse relationship between the two .

So both cannot be high , Need to find a balance , Finding the balance point needs to be done PRC curve , The horizontal axis recall, The vertical axis precision.F1 This is the common balance point indicator .

F1

2*precision*recall/(precision+recall)

Reference resources

precision,recall and precision-recall curve_tangzy_ The blog of -CSDN Blog