Www2022 | know your way back: self training method of graph neural network under distribution and migration

Thesis title ：

Confidence May Cheat: Self-Training on Graph Neural Networks under Distribution Shift

meeting ： The WebConf 2022

Address of thesis ：

https://arxiv.org/abs/2201.11349

Generally speaking , In order to filter out false tags as much as possible , The existing graph neural network self training methods will only retain the pseudo tags generated by high confidence prediction . However , The high confidence of prediction means that the model may have learned most of the information contained in this node , Is it really effective to add this node to the training set through self training strategy ？

To answer the above questions , This paper carries out the following two exploratory experiments ：

(1) Explore the relationship between confidence and information gain ： There is an obvious negative correlation between confidence and information gain . in other words , The higher the confidence, the lower the information gain of the node . Considering that the existing graph neural network self-training methods will only retain the pseudo tags generated by high confidence prediction , We believe that these methods are difficult to introduce additional effective supervision information into the model .

(2) Explore the distribution of node embedded representations ： Most low information gain （ high confidence ） The nodes of are distributed far away from the decision boundary . This explains why these nodes have lower information gain , On the other hand, it also implies that most of the nodes concerned by the existing graph training methods are far away from the decision boundary .

In order to solve the above problems , This paper presents a self training method of graph neural network based on distribution restoration DR-GST. First, we analyze the loss function of the neural network self-training method in the following figure under the ideal and distribution migration situation , The theoretical results show that the distribution migration problem can be eliminated as long as appropriate weights are given to each unlabeled node . Based on the analysis of the experimental results , We propose to use the regularized information gain as the above weight . Besides , In order to eliminate the error information that may be introduced by the self training strategy , We introduce the loss correction strategy into the self training method of graph neural network . The final theoretical analysis and experimental verification have proved the effectiveness of our method .