How does GNN generalize? This 135 page PDF doctoral thesis "generalization evaluation and improvement of neurograph reasoning and learning"

PDF Address ：https://atrium.lib.uoguelph.ca/xmlui/handle/10214/26824

This paper has made some contributions in the field of machine learning , Especially in Graph reasoning task . Each article studies and improves the generalization of several graph reasoning applications : Classic graph classification task 、 A new task of combining visual reasoning and neural network graph parameter prediction .

 

In the first article , We study the attention mechanism in graph neural networks . Although the attention is GNN Has been widely studied , However, its effect on generalization to larger noise graphs has not been thoroughly analyzed . We prove , In the composition diagram task , You can initialize it carefully GNN Attention module to improve generalization ability . We also developed a method , It reduces the sensitivity of attention module to initialization , The generalization ability of actual graph tasks is improved .

 

In the second article , We discuss the generalization of the problem to the combination and relationship of rare or invisible objects in the visual scene . Previous work mainly focused on frequent visual composition , Poor generalization ability of composition . To alleviate this problem , We find that it is very important to normalize the loss function with the structure of the scene graph , This makes more efficient use of training tags . Using our loss trained model significantly improves synthesis generalization .

 

In the third article , We will further discuss the generalization of visual synthesis . We consider a data enhancement approach , That is to add rare and invisible components to the training data . We developed a model based on generative countermeasure network , The model generates synthetic visual features based on the rare or invisible scene images obtained by disturbing the real scene images . Our method continuously improves the generalization of synthesis .

 

In the fourth article , We study graph reasoning in a new task of predicting parameters in invisible deep neural structures . The motivation of our task is due to the limitations of the iterative optimization algorithm used to train neural networks . To solve our task , We developed a system based on Graph HyperNetworks Model of , And train it on our neural architecture data set . Our model can predict the invisible depth network in a forward pass ( Such as ResNet-50) Performance parameters of . The model can be used for neural structure search and transfer learning .