SIGIR 2022 | HKU and Wuhan University put forward kgcl: a recommendation system based on knowledge map comparative learning

PaperWeekly original ·  author | Yangyuhao

Company | The university of Hong Kong

Research direction | Recommendation system

Research background

Knowledge map （Knowledge Graphs,KGs） It usually contains rich entity semantic associations , It has been widely used in recommendation systems to improve the quality of user representation learning , And effective additional information to improve the accuracy of recommendations . In these knowledge aware recommendation models , Knowledge map information usually contains the semantic relationship between entities and recommended items . However , The success of these recommendation algorithms largely depends on high-quality knowledge maps , And may be unable to learn high-quality user and product representations because of the following two problems ：

i）Entity The long tailed distribution of results in KG The supervision signal of the object representation becomes sparse ;

The figure above shows the distribution of knowledge map entities collected from three practical application data sets . In the picture , Y The axis represents the number of entities with the corresponding number of exposures , And X The number of entity exposures on the axis corresponds to . Obviously , majority KG Entities all exhibit long tails . Because the knowledge map embedding Of Trans A series of algorithms require each entity to have sufficient triple based （h, r, t） Product connection information for , So as to accurately model semantic transformation , And auxiliary commodities embedding Study . therefore ,KG The problem of long tail distribution brings a challenge to accurately capture the relevance of objects .

ii） The knowledge map in practical application is often noisy , For example, the links of the knowledge map also contain many items and Entity Weak or even less relevant noise connection information .

The above figure shows an example of a news recommendation task , The key entity of news Zack Wheeler He is a famous pitcher of the Phillies in major league baseball . However , We can notice that ,Zack Wheeler It has nothing to do with the semantics of the news itself " The noise " Entities are connected , namely Smyrna, GA and UCL Reconstructive surgery . although Zack Wheeler Was born in Smyrna, And he had accepted UCL Reconstructive surgery , But these two entities are not very relevant to the theme of the sports news itself , Thus, the semantic learning of the news will be deviated .

In this way KG The problem of sparsity and noise makes the entity dependence between objects deviate from the reflection of their real characteristics , This greatly creates a modeling bias , It hinders the accurate learning of user preferences .

Thesis title ：
Knowledge Graph Contrastive Learning for Recommendation

Author of the paper ：

Yangyuhao （ The university of Hong Kong ）, Huang Chao （ The university of Hong Kong ）, Xia lianghao （ The university of Hong Kong ）, Li Chenliang （ Wuhan University ）

Source of the paper ：
https://arxiv.org/abs/2205.00976

Code link ：
https://github.com/yuh-yang/KGCL-SIGIR22

Model is introduced

In view of the above research blank , We propose a knowledge map contrastive learning framework for recommendation systems （KGCL）, To reduce the information noise in the recommendation modeling of knowledge perception . We propose a contrastive learning paradigm based on knowledge map enhancement , In order to suppress the KG The noise , So as to learn more robust knowledge perception representation of objects , relieve KG Long tail and noise problem . Besides , We use KG Enhance additional monitoring signals of the process to guide users across views - Comparative study of object map , Give unbiased users in the gradient of contrast - Greater weight of item interaction , And further alleviate the damage of noise to representation learning .

2.1 Relationship aware knowledge aggregation

First , We design a relational aware knowledge embedding network , In order to reflect the relationship heterogeneity in the structure of knowledge map when aggregating object knowledge . Graph based attention network （GAT） And its variants , our KGCL The model projects the context related to entities and relationships into a specific representation with a parameterized attention matrix . then , An attention based information aggregation mechanism is established between the objects in the knowledge map and their connected entities , An object representation used to generate knowledge perceptions on heterogeneous relational graphs . See the following figure for specific knowledge aggregation mechanism .

2.2 Graph enhancement based on knowledge map

stay KGCL In the frame , We propose to generate different structural views of knowledge maps , In order to carry out the comparative learning of the self distinguishing form of knowledge entities . In particular , We use the enhancement scheme of random discarding relation to generate two contrast views on the input knowledge map . The consistency of the two views reflects the consistency of the knowledge map structure of a single item , To reflect the robustness of objects to knowledge noise disturbance .

2.3 Cross view comparative learning based on knowledge map

We combine graph enhancement on the knowledge map with graph contrast learning paradigm , To improve the robustness of object representation generated by knowledge graph representation learning and knowledge aggregation . meanwhile , In order to effectively transfer the external knowledge of high-quality items to help users learn their preferences , We are for users - The article interaction designs the contrast learning paradigm of knowledge guidance . In this comparative study , The object knowledge of de-noising can be used to guide users and objects' representation learning , And alleviate the sparsity of supervision signals .

say concretely ,KG Items with higher structural consistency scores contain less noise in their knowledge map , And make greater contribution to the prediction of users' real interest modeling . therefore , We're doing user - When comparing and learning the object interaction graph , In the graph enhancement process, these low-noise objects are more likely to be preserved .

Final , We provide enhanced knowledge maps and users - The two views of the article interaction graph are used for knowledge aggregation and collaborative filtering based on graph convolution , And make a comparative study for the representation of each user and item , Calculation InfoNCE Loss function , The gradient descent optimization is carried out together with the loss of the recommended main task .

2.4 The influence of knowledge map contrast learning on model gradient

In this part , From a theoretical point of view, we analyze the effect of knowledge guided comparative learning on users - The influence of object representation learning gradient , And study how this learning process benefits from the comparative learning on the knowledge map . First , Refer to related work （SGL,SIGIR21）, user - The gradient of negative samples in the contrastive learning of object graph can be deduced to be proportional to this function value ：

among Is the cosine similarity value of positive and negative samples . The function image at different temperature coefficients Next is ：

It's not hard to find out , Have high value （ for example ：） The strong negative samples have a higher contribution to the gradient , Can better guide the gradient of comparative learning . our kgcl The model mainly improves the ability to distinguish strong negative samples through the following two processes ：

1）. Objects connected with noise or long tail knowledge entities can enhance the semantic stability of their representation learning through comparative learning on the knowledge map , Can learn more accurate value .

2）. Users related to items that are more affected by the semantic deviation of the knowledge map - Item interactions will be less involved with users - In the process of comparative learning of object map .

For the first point , Consider a set of false strong negative samples , Defined as ：

among It represents the measurement of sample similarity by the model when the semantic deviation of knowledge map is introduced , Express The maximum point of , These samples are Of Near the distance , It makes a great contribution to the gradient of comparative learning . However , When there is no noise and long tail problem in the knowledge map , These samples should be regarded as non strong samples , namely ：

therefore , We think that the noise and long tail problem of knowledge map will make the model treat some ordinary negative samples as strong negative samples , The real strong negative samples are treated as ordinary negative samples , Lead to The curve is offset .KGCL Comparative learning on the proposed knowledge map can correct this problem , The model can better distinguish the strength of negative samples by accurately modeling the sample semantics .

experimental result

3.1 Overall model effect

Let's start with the following research line The comparison model of diversity is selected in ： Tradition CF Model 、 nerve CF Model 、 chart CF Model 、 be based on Embedding Of KG Recommended model 、 Path based KG Recommended model 、 Graph based blending KG Recommendation model and self supervised recommendation model . It can be seen from the table below that ,KGCL The performance of the baseline model is significantly better than that of the baseline model on the three data sets .

3.2 Ablation Experiment

We are respectively right KGCL In the architecture KG Figure enhanced part （KGA） and KG Comparative learning part （KGC） Ablation experiments were performed separately , To explore the impact of these two key components on model performance . As can be seen from the table below , The two modules can obviously promote the learning of the model .

3.3 The advantages of the model in sparse data

KGCL External knowledge is introduced to enhance the representation learning of objects , Again in the user - The object interaction graph adopts the knowledge guided contrastive learning paradigm , So we look forward to presenting KGCL For the user - The sparsity of item interaction has a better mitigation effect . We first divide the objects into five groups according to the sparsity of exposure , And experiment on the user interaction data related to them . The experimental results in the table below show ,KGCL In sparse item groups （ for example 0-2） On , Compared with some of the most advanced baseline models, it has been greatly improved .

Further , We filter out cold start users for the dataset , The number of interactions between these users is less than a specific threshold for a dataset （ for example , stay Yelp2018 On is 20）. Through the experiments on these cold start user generated recommendation lists , We found that KGCL There is also a significant improvement in modeling cold start user tasks ：

3.4 The model is alleviating KG Noise advantage

In order to verify KG The comparative learning on the Internet and the comparative learning guided by knowledge are aimed at KG The advantages of noise and long tail problems , Let's start with KG Join at random 10% The noise of , And compare KGCL With the latest KG Recommended method in noise KG Performance on ：

From the experimental results in the above table, it can be found that ,KGCL Join in the face KG A noisy scene , Only produced 0.58% Performance degradation of , Far better than MVIN、KGIN and KGAT. This illustrates the KGCL   The advanced nature of . Further , We filter out connections to KG Items with medium and long tail entities , And for these users - Experiment with object interaction .

The experimental results above illustrate KGCL In inhibition KG The advantages of the long tail problem compared with other recommended models .

summary

In this work , What we proposed KGCL The framework makes a preliminary attempt to explore how to suppress the noise and long tail distribution of the knowledge map through the comparative learning on the knowledge map . Further , Enhanced by knowledge guided graph data , We were able to estimate the impact of KG Items with semantic deviation due to the influence of problems , And take it as an auxiliary self-monitoring signal , Users who make semantics more explicit - Object interaction can make a greater contribution to the gradient in comparative learning . We hope this work will be KG The enhanced recommendation system opens up a new exploration direction .

reference

[1] Jiancan Wu, Xiang Wang, Fuli Feng, Xiangnan He, Liang Chen, et al. 2021. Self-supervised graph learning for recommendation. In SIGIR. 726–735.
[2] Xiangnan He, Kuan Deng, Xiang Wang, Yan Li, Yongdong Zhang, and Meng Wang. 2020. Lightgcn: Simplifying and powering graph convolution network for recommendation. In SIGIR. 639–648.
[3] Hongwei Wang,Fuzheng Zhang,Jialin Wang,Miao Zhao,Wenjie Li,Xing Xie, and Minyi Guo. 2018. Ripplenet: Propagating user preferences on the knowledge graph for recommender systems. In CIKM. 417–426.
[4] Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, and Lars Schmidt-Thieme. 2009. BPR: Bayesian Personalized Ranking from Implicit Feedback. In UAI. 452–461.

Read more

# cast draft   through Avenue #

  Let your words be seen by more people  

How to make more high-quality content reach the reader group in a shorter path , How about reducing the cost of finding quality content for readers ？ The answer is ： People you don't know .

There are always people you don't know , Know what you want to know .PaperWeekly Maybe it could be a bridge , Push different backgrounds 、 Scholars and academic inspiration in different directions collide with each other , There are more possibilities . 

PaperWeekly Encourage university laboratories or individuals to , Share all kinds of quality content on our platform , It can be Interpretation of the latest paper , It can also be Analysis of academic hot spots 、 Scientific research experience or Competition experience explanation etc. . We have only one purpose , Let knowledge really flow .

  The basic requirements of the manuscript ：

• The article is really personal Original works , Not published in public channels , For example, articles published or to be published on other platforms , Please clearly mark  

• It is suggested that  markdown  Format writing , The pictures are sent as attachments , The picture should be clear , No copyright issues

• PaperWeekly Respect the right of authorship , And will be adopted for each original first manuscript , Provide Competitive remuneration in the industry , Specifically, according to the amount of reading and the quality of the article, the ladder system is used for settlement

  Contribution channel ：

• Send email ：[email protected] 

• Please note your immediate contact information （ WeChat ）, So that we can contact the author as soon as we choose the manuscript

• You can also directly add Xiaobian wechat （pwbot02） Quick contribution , remarks ： full name - contribute

△ Long press add PaperWeekly Small make up

Now? , stay 「 You know 」 We can also be found

Go to Zhihu home page and search 「PaperWeekly」

Click on 「 Focus on 」 Subscribe to our column

·