[GNN report] Tencent AI Lab Xu TingYang: graph generation model and its application in molecular generation

Catalog

1、 brief introduction

2、An overview of Graph Generative Models and Their Applications on Molecular Generation

background

  Graph generation learning paradigm

from VAE Angle work

VGAE

 GraphVAE

 JTVAE​ edit

​ edit  Scaffold Hopping Skeleton transition

 GAN Method angle work

 MoIGAN​ edit

 Flow Angle work

 MoFlow

Glow

GraphDF 

Diffusion Angle work

 GraphEBM

 EDM

Summary

3、 Reference resources

---

1、 brief introduction

Guest speaker ： Xu TingYang （ tencent AI lab）

Report title

An overview of Graph Generative Models and Their Applications on Molecular Generation（ Graph generation model and its application in molecular generation ）

Summary of the report

Generating novel graph structures is a fundamental problem for studies in drug discovery, engineering, and social networks. Although a set of different generative models, such as VAE, GAN, Normalizing Flow, and diffusion models, have achieved great successes in image processing, it is still hard to naively apply such generative models directly to graph structures due to the complex distribution, non-unique nature of graph and non-local dependencies between nodes and edges in a graph. In this report, we will review how above generative models are employed into graph generations and their proper applications on the task of molecular generation. We will introduce VGAE, JTVAE, and GraphGMVAE as VAE-based methods; MolGAN and MolAICal as GAN-based methods; MoFlow and GraphDF as Flow-based methods. We will also discuss the current challenges for diffusion models in graph generation.

About the reporter

Tingyang Xu is a Senior researcher of Machine Learning Center in Tencent AI Lab. He obtained the Ph.D. degree from The University of Connecticut in 2017 and the bachelor’s degree from Shanghai Jiaotong University. In Tencent AI Lab, he is working on deep graph learning, graph generations and applying the deep graph learning model to various applications, such as molecular generation and rumor detection. His main research interests include social network analysis, graph neural networks, and graph generations, with particular focus on designing deep and complex graph learning models for molecular generations. He has published several papers on data mining, machine learning top conferences KDD, WWW, NeurIPS, ICLR, CVPR, ICML, etc.

2、An overview of Graph Generative Models and Their Applications on Molecular Generation

background

  The generation model on image and voice is very mature

 GAN Graph generation meaning and learning process

In the molecular diagram 、 Control flow and knowledge map are more meaningful

  stay CV Classical methods of generating models in the field

GAN： Confrontation angle , Put the true and false data into the discriminator , The sum generates false data with a generator

VAE： The variation angle of encoding and decoding , Reconstruction loss after fitting , Gaussian distribution is generally assumed

Flow: Bijection function fx

Diffusion： Through a priori distribution , Gradually add Noise, The sum of the two is pushed back in the opposite direction, and there is no noise before x_0

  Which method is more suitable on the graph ？

The first two are obviously suitable

  Because graph has adjacency matrix , against , again 01 matrix , The latter two methods are difficult to use directly

  Graph generation learning paradigm

1、 Get a hidden space z after , Directly connect the whole... Through a network A The matrix is generated

2、 Generate step by step like a generation sequence

 

from VAE Angle work

VGAE

thought ： Hidden space z Multiply by its transpose to get non 01 Matrix , after sigmoid, Finally, rebuild the loss

Insufficient ： Not satisfied with replacement unchanged

 GraphVAE

It's not a simple calculation of losses , A similarity matrix is used

 

  Molecular generation

The measure ：

1） Legitimacy

2） Uniqueness

3） Novelty

Common data sets ：

1）QM9

2）ZINC

 JTVAE

  Let's first learn about another job

step by step It's done

Do you want to add some ？ Do you want to add a side to the store after adding some ？ Is it necessary to add another point ？ On the basis of two points, do you want to add an edge between these two points

 JTVAE It's different from doing it one by one , And become a structure, a structure generation

  Tree structure

  frame

Although it takes time , But the legitimacy, uniqueness and innovation are great

 Scaffold Hopping Skeleton transition

Making different assumptions in hidden space

  Gaussian mixture distribution

  Take skeleton points near the red part, that is, near the cluster , Even farther Regional mining

  Effect comparison

Activity and structural similarity analysis

 GAN Method angle work

Because the effect is not very good , Just introduce MoIGAN

 MoIGAN

Results contrast

High legitimacy , Low uniqueness and novelty

 Flow Angle work

 MoFlow

There are advantages in refactoring

 

Glow

 MoFlow

Edge matrix calculation

  Comparison of experimental results

GraphDF 

 

  It's just a little bit more / Edge process

Training process

 

Experimental comparison

Diffusion Angle work

 GraphEBM

Gradually add noise , Then a learning process of denoising , Basically in Gaussian noise

 EDM

 

Experimental comparison

 

Summary

 

 

3、 Reference resources

LOGS The first 2022/07/23 period || tencent AI lab Xu TingYang ： Graph generation model and its application in molecular generation _ Bili, Bili _bilibili

 LOGS The first 2022/07/23 period || tencent AI lab Xu TingYang ： Graph generation model and its application in molecular generation (qq.com)

reference

1. Yu, Y., Xu, T., Li, J., Qiu, Y., Rong, Y., Gong, Z., ... & Huang, J. (2021). A novel scalarized scaffold hopping algorithm with graph-based variational autoencoder for discovery of JAK1 inhibitors.ACS omega, 6(35), 22945-22954.

2. Bai, Q., Tan, S., Xu, T., Liu, H., Huang, J., & Yao, X. (2021). MolAICal: a soft tool for 3D drug design of protein targets by artificial intelligence and classical algorithm.Briefings in bioinformatics, 22(3), bbaa161.

3. Bai, Q., Liu, S., Tian, Y., Xu, T., Banegas‐Luna, A. J., Pérez‐Sánchez, H., ... & Yao, X. (2022). Application advances of deep learning methods for de novo drug design and molecular dynamics simulation. Wiley Interdisciplinary Reviews: Computational Molecular Science, 12(3), e1581.