Variational graph auto-encoders (VGAE)

Graph Auto-Encoders (GAE)

Definitions

• Given an undirected graph $\mathcal{G}=(\mathcal{V},\mathcal{E})$,$N=|\mathcal{V}|$ For the top points ,$\mathbf{A}\in {\mathbb{R}}^{N\times N}$ Is the adjacency matrix ( The diagonal element is 1),$\mathbf{X}\in {\mathbb{R}}^{N\times D}$ Is the eigenvector of the node

---

Graph Auto-Encoders

• GAE Medium Encoder by GCN, It is responsible for getting the of each node from the adjacency matrix and node feature coding embedding vector $z_i$ ($i=1,...,N$), They form nodes embedding matrix $\mathbf{Z}\in {\mathbb{R}}^{N\times F}$
• GAE Medium Decoder For a simple inner product decoder, It is responsible for embedding vector $\mathbf{Z}$ To reconstruct the adjacency matrix $\hat{\mathbf{A}}$. It works by calculating $\sigma (z_i^T{z}_j)$ To decide ${\hat{\mathbf{A}}}_{ij}$
  

---

A framework for unsupervised learning on graph-structured data

• GAE Introduce self encoder to process graph data , Sure Unsupervised learning based on graph data

Variational graph auto-encoders (VGAE)

• VGAE stay GAE On the basis of Variational self encoder (VAE) Thought , Yes latent space Regularization is applied to guarantee a regular latent space
  

---

• VGAE hypothesis Prior probability obey Standard normal distribution
  $$p(\mathbf{Z})=\prod _{i}p(z_i)=\prod _i\mathcal{N}(z_i|0,\mathbf{I})$$
• likelihood from Dot product The model gets
  
• Posterior probability from Variational reasoning Approximate result , The probability distribution family is a normal distribution in which the covariance matrix is a diagonal matrix
  among ,$\mu ={\text{GCN}}_{\mu }(\mathbf{X},\mathbf{A})$ by GCN Output posterior probability distribution mean vector ,$\log \sigma ={\text{GCN}}_{\sigma }(\mathbf{X},\mathbf{A})$ by GCN Logarithm of the standard deviation of the output posterior probability distribution .$\text{GCN}$ For a simple 2 layer GCN, Can be expressed as $\text{GCN}(\mathbf{X},\mathbf{A})=\tilde{\mathbf{A}}\text{RELU}(\tilde{\mathbf{A}}\mathbf{X}{\mathbf{W}}_0){\mathbf{W}}_1$, among ${\mathbf{W}}_i$ by MLP Weight matrices ,$\tilde{\mathbf{A}}={\mathbf{D}}^{-\frac{1}{2}}\mathbf{A}{\mathbf{D}}^{-\frac{1}{2}}$ Is the normalized adjacency matrix ,$\mathbf{D}$ Is the degree matrix ( A diagonal matrix , The diagonal element is the degree of each vertex ), Left multiplication ${\mathbf{D}}^{-\frac{1}{2}}$ Will make $\mathbf{A}$ Of the $i$ Row divided by node $i$ The root of the degree , Take the right ${\mathbf{D}}^{-\frac{1}{2}}$ Will make $\mathbf{A}$ Of the $i$ Column divided by node $i$ The root of the degree , therefore ${\tilde{\mathbf{A}}}_{ij}={\mathbf{A}}_{ij}/(\sqrt{{\mathbf{D}}_{ii}{\mathbf{D}}_{jj}})$, So the adjacency matrix is normalized according to the degree .$\tilde{\mathbf{A}}\mathbf{X}=\left[\begin{array}{c}{\tilde{a}}_1^T\mathbf{X}\\ ...\\ {\tilde{a}}_N^T\mathbf{X}\end{array}\right]$ It is the information aggregation of nodes .${\text{GCN}}_{\mu }(\mathbf{X},\mathbf{A})$ and ${\text{GCN}}_{\sigma }(\mathbf{X},\mathbf{A})$ Share the weight of the first layer ${\mathbf{W}}_0$
• Derived from variational reasoning optimization problem To maximize the following formula ：
  

References

• Kipf, Thomas N., and Max Welling. “Variational graph auto-encoders.” arXiv preprint arXiv:1611.07308 (2016).
• Wu, Zonghan, et al. “A comprehensive survey on graph neural networks.” IEEE transactions on neural networks and learning systems 32.1 (2020): 4-24.
• code: https://github.com/DaehanKim/vgae_pytorch