Concepts and differences of PR curve and ROC curve

Hello everyone , I meet you again , I'm your friend, Quan Jun .

Knowledge point

Precision=TP/(TP+FP) Recall=TP/( Total positive samples )=TP/(TP+FN) # At this time TPR=TP/(TP+FN)=Recall # True case rate FPR=FP/(TN+FP) # PR curve cross Recall, longitudinal Precision PR The drawing of curves scene ： Limited sample . Method ： Fixed classification threshold （ such as 0.5）, The prediction results of the classifier are sorted in descending order according to the confidence level , Then predict the samples in this order , The current recall rate can be calculated every time （Recall） And precision （Precision）, Then draw the picture , Draw out P-R curve .（ doubt ？？：P-R The curve is based on a fixed classification threshold , Or according to the watermelon book , In descending order of confidence , Take the samples one by one as positive examples to predict ？？ I do the written test of Shang Tang , Is to adopt the strategy of fixed classification threshold ） ROC curve cross FPR, longitudinal TPR, The ideal situation is TPR=1,FPR=0, Generally speaking ,FPR increase , be TPR It will also increase . ROC The drawing of curves ： scene ： Limited sample . Method ： Card threshold , Sort the prediction results of the learner , At the top of the list are the samples that are most likely to be positive , The last is the most unlikely sample , Then calculate TPR and FPR value , Draw a curve . The card threshold is used as the judgment basis for positive and negative samples , When the threshold is high ,Precision The larger , When the threshold is low ,Recall more .（ Recommended words , Want to Precision more , Users hope that the top recommendations are what they want , Criminal investigation, I hope Recall more , Don't miss a prisoner ） AUC：ROC Area under curve . PR and ROC Application range of curve ： 1. When the proportion of positive and negative samples is similar , There is little difference between the two . 2.PR Curve ratio ROC The curve pays more attention to positive samples , and ROC Both are taken into account . 3.AUC The bigger it is , It reflects that the prediction results of positive samples are higher .（ The recommended samples are more in line with users' preferences ） 4. When the positive and negative samples are out of proportion , For example, positive samples 1 individual , Negative sample 100 individual , be ROC The curve changes little , Use at this time PR The curve can better reflect the performance of the classifier . 5.PR Curves and ROC The method of drawing is different .

PR Curves and ROC Curve difference reference link ：http://www.fullstackdevel.com/computer-tec/data-mining-machine-learning/501.html

attach ： Shang Tang calculation PR topic

#-*-coding:utf-8-*-
def s4j5(a):
    a1=float(int(a*100))
    tag=a*100-a1
    if tag>=0.5:
        return((a1+1)/100)
    else:
        return(a1/100)

N = int(raw_input())
recall = 0.0
labelCon = []
totalPos = 0
for i in range(N):
    temp = [xx for xx in raw_input().strip().split()]
    target, confidence = int(temp[0]), float(temp[0])
    labelCon.append([target, confidence])
    if target==1:
        totalPos+=1
count = 0
posCount = 0
correctCount = 0
FP = 0
FN = 0
someRecallDict = {0.3:0,
            0.4:0,
            0.5:0,
            0.6:0,
            0.7:0,
            0.8:0,
            0.9:0}

someRecall = [0.3,0.4,0.5,0.6,0.7,0.8,0.9]
res = []
posId = 0
predictPos = 0
for target, confidence in labelCon:
    count+=1
    if confidence>=0.5:
        predictPos+=1 #  The number of samples predicted to be positive 
    if target==0:
        if confidence>=0.5:
            FP+=1
    if target==1:
        posCount+=1
        if confidence>=0.5:
            correctCount+=1
        else:
            FN+=1
        recall = float(correctCount)/float(totalPos)
        if posId<=6:
            if recall==someRecall[posId] and someRecallDict[someRecall[posId]]==0:
                someRecallDict[someRecall[posId]]=1
                posId+=1
                precision = int(str(s4j5(float(correctCount)/float(predictPos))*100).split('.')[0])
                res.append(precision)

for i in res:
    print(i)

Copy

Publisher ： Full stack programmer stack length , Reprint please indicate the source ：https://javaforall.cn/148428.html Link to the original text ：https://javaforall.cn