当前位置：网站首页>Paper notes: e (n) equivariant graph neural networks

Paper notes: e (n) equivariant graph neural networks

2022-06-29 16:34:00 【UQI-LIUWJ】

0 Introduce

This paper introduces a new model to learn and rotate 、 translation 、 A graph neural network with equivariant reflection and arrangement , be called E(n)- Equivariant graph neural network (EGNN).

Compared with the existing method ,EGNN There is no need to compute expensive high-order representations in the middle layer , While still achieving competitive or better performance . Besides , Although the existing methods are limited to 3 Equivariant of dimensional space , but EGNN It is easy to expand to higher dimensional space .

1 introduction

Although deep learning has largely replaced handmade features , But much progress has been made It depends on the inductive bias in the deep neural network .（inductive bias）

An effective way to limit neural network to a function related to a problem is , Use the symmetry of the problem 、 Transformation equivariant （equivariance）, Simplify the calculation of the current problem by studying a symmetric group .【eg,CNN The convolution of is equivariant 、 Pooling is approximately invariant ;GNN The order of the points of is equivariant （ The arrangement of different points corresponds to different adjacency matrices , But in the end this one graph The message is the same ）】

Many problems show that 3D Translational and rotational symmetry . The set of these symmetric operations is written as SE(3) , If reflection is included , Then the set is written as E(3). It is usually desirable to predict these tasks relative to E(3) Transformation is equivariant or invariant .、

lately , Equal changes have been proposed E(3) or SE(3) Various forms and methods of . Many of these works have achieved innovation in the study of high-order representation types in the intermediate network layer . However , The conversion of these higher-order representations requires the computation of costly coefficients or approximations . Besides , In practice , For many types of data , Input and output are limited to scalar values （ Such as temperature or energy , It is called in the literature type-0） and 3d vector （ Such as velocity or momentum , It is called in the literature type-1）.

This article paper A new architecture is proposed , It is translation 、 Rotation and reflection etc (E(n)), And the permutation of the input point set . The model is simpler than the previous method , At the same time, the equivariant in the model is not limited to 3 Dimensional space , And it can be extended to a larger dimensional space , Without significantly increasing the amount of computation .

2 Background knowledge

2.1 Equivariant

Definition ：

In layman's terms , Pan first / rotate / Permutation remapping , And mapping before translating / rotate / array The effect is the same

2.2 GNN

GNN note ： Message propagation model _UQI-LIUWJ The blog of -CSDN Blog

3 EGNN

Consider a graph $\mathcal{G}=(\mathcal{V,E})$ , among $v_i \in \mathcal{V}, e_{ij} \in \mathcal{E}$ ,
The characteristics of each point embedding yes $h_i \in R^{nf}$ ( and 2.2 Of GNN equally ）[ This nf refer to node feature, Not at all n ride f]【 Physical information without direction , Scalar 】
But on this basis ,EGNN Add one for each point n Coordinates of dimensions $x_i \in R^n$ 【 Geometric information with direction , vector 】

GNN Will be maintained with these coordinates xi Equivariant of rotation and translation , And it will be linked to GNN The same way as the node set V The equivariant of the permutation .

Use the formula to express the l layer EGNN,Equivariant Graph Convolutional Layer (EGCL) Yes ： $h^{l+1},x^{l+1}=EGCL[h^l,x^l,\mathcal{E}]$
And traditional GNN Different places are drawn with green frames
In the equation 3 in ,EGNN The distance between two coordinates is increased As a parameter
In the equation 4 in , According to the meaning of the paper ,xi The position of is updated to the radial vector field , Owned by $(x_i-x_i)_{\forall j}$ Joint decision 【 I don't quite understand here , Why is it radial , Because it's different xi, Its direction is different , $(x_i-x_i)_{\forall j}$ Subtract as a vector , Not necessarily along the radial direction 】
equation 4 The weighting coefficient of each term is determined by the function $\phi_x:R^{nf} \rightarrow R^1$ Calculated
there C=1/(M-1)
——> equation 3&4 It can guarantee equidenaturation
——> meanwhile , Another difference is , All of these are considered here (i,j) Yes , Not just between neighboring points pair, in other words ,embedding mij Can contain information about the full graph

3.1 Translational equivariant

g Is a translation vector ,x（type-1 vector ） Is translational equivariant （equivariant）,h（type-0 vector ） Is translation invariance （invariant）

——> It's not hard to find out ,EGCL The combination of is also equivariant

3.2 expand EGNN

Here to the front EGNN Make minor modifications , So that we can explicitly track the momentum of the particles .

This can not only be used to obtain the estimated velocity of particles in each layer , Momentum can also be introduced

Expressed by formula , Will be

Expressed as

$h^{l+1},x^{l+1},v^{l+1}=EGCL[h^l,x^l,v^{init},\mathcal{E}]$ '

If $v^{init}$ by 0, So the equation 4 And equation 7 It's about one thing

3.3 Get some information

In some cases , We may not always get an adjacency matrix . In these cases , We can assume a full connection graph , Where all nodes exchange messages with each other .

This full connection method may not be well extended to large graphs , We want to consider only the neighbor nodes N(i) Interaction between points of .

The paper uses the following method here

Among them, if (i,j) There is a link , that eij by 1, Otherwise 0

The paper approximates by a function eij : $\phi_{inf}:R^{nf} \rightarrow [0,1]^1$ （ Linear layer +sigmoid Activation function , The input side embedding, Output side value Of soft estimation）