Common evaluation indicators of recommended system: ndcg, recall, precision, hit rate

The evaluation index

NDCG

Normalized Discounted Cumulative Gain( Normalized loss cumulative gain )

NDCG Used as an evaluation index for sorting results , Evaluate the accuracy of sorting .

The recommendation system usually returns a... For a user item list , Suppose the length of the list is K, You can use [email protected] Evaluate the gap between the sorted list and the user's real interaction list .

CG （ Cumulative gain Cumulative Gain）

$$C{G}_K=\sum _{i=1}^{K}re{l}_i$$

Consider a length of K A list of ,$re{l}_i$ representative $i$ The relevance of location items .（ In the recommendation system 0、1）

There is a problem with this evaluation index , The items I recommend are clustered at the end of the list and scored the same as the head , This is not appropriate .

DCG（ Cumulative gain of loss reduction Discounted cumulative gain）

DCG Propose that if the valid results rank low in the list , The list should be penalized for scoring , Punishment is related to the ranking of effective results . So the attenuation factor is added ：
$$DC{G}_p=\sum _{i=1}^p\frac{re{l}_i}{{\log }_2(i+1)}=re{l}_1+\sum _{i=2}^p\frac{re{l}_i}{{\log }_2(i+1)}$$
perhaps
$$DC{G}_p=\sum _{i=1}^p\frac{2^{re{l}_i}-1}{{\log }_2(i+1)}$$
The latter formula is widely used in industry . When the score is 0/1, namely $re{l}_i\in \{0,1\}$ when , The two are equivalent .

NDCG （ Normalized loss cumulative gain Normalized Discounted Cumulative Gain）

DCG Without considering the recommendation list and the really valid results in each search （test items list） The number of , So finally introduce NDCG, It's standardized DCG.
$$NDC{G}_k=\frac{DC{G}_k}{IDC{G}_k}$$
among $IDCG$ Refer to ideal DCG, That is, under the perfect result DCG.

For example, recommend to users 7 movie ：$M_1,M_2,M_3,M_4,M_5,M_6,M_7$

The user's rating for these seven films is ：$5,3,2,1,2,4,0$

Then the perfect recommendation results should be sorted by score ：$5,4,3,2,2,1,0$, So at this time IDCG：
$$\begin{gathered} IDC{G}_5=\frac{2^5-1}{{\log }_{2}2}+\frac{2^4-1}{{\log }_{2}3}+\frac{2^3-1}{{\log }_{2}4}+\frac{2^2-1}{{\log }_{2}5}+\frac{2^2-1}{{\log }_{2}6} \\ =31+9.5+3.5+1.3+1.2=46.5 \end{gathered}$$
therefore $NDCG:$
$$NDC{G}_5=\frac{DC{G}_5}{IDC{G}_5}=\frac{38.5}{46.5}=0.827$$

Precision、Recall

$$Precision=\frac{premeasuringjustindeedOf}{PUSHcommendColumnsurfaceLongdegree}$$

$$Recall=\frac{premeasuringjustindeedOf}{useHouseholdrealOccasionspotblowColumnsurfaceLongdegree}$$

Hit Rate

$$HR=\frac{measuringtrySetinOfitemOutpresentstayTop-NPUSHcommendColumnsurfaceinOfuseHouseholdCountThe\; amount}{useHouseholdtotalCount}$$