2020 ACM | MoFlow: An Invertible Flow Model for Generating Molecular Graphs

Paper: https://arxiv.org/abs/2006.10137
Code: https://github.com/calvin-zcx/moflow

MoFlow: Generate molecular graph can reverse flow model

Figure generation model usually includes two steps:Learning potential representation and generate molecular figure.然而,Due to the chemical constraints, and combination of molecular figure complexity,From the potential to generate new and effective molecular chemical graph is a very challenging.因此,In this paper, the author puts forward a model based on flow diagram is generatedMoFlow, The model to study reversible mapping between molecular graph and its potential said. MoFlow首先通过基于Glow的模型生成键(边),And then through the flow to generate a given key figure conditions原子(节点),The final assemble them into a chemical effective molecular figure,And to correct the after effect,.Through the four tasks to verify the model:Molecular graph generation and reconstruction、Continuous potential space visualization、性质优化和约束性质优化,Show that this model has potential efficiency and effectiveness of the.

模型

The flow framework
Based on the flow model is designed to study complex high-dimensional data$X\sim p_x(X)$和$Z\sim p_z(Z)$A series of reversible transformation between$f_{\theta }=f_L\circ ...\circ f_1$,The transformation of dimension is the same,潜在分布$P_Z(Z)$It is easy to model(例如,In such a potential space exists strong independence assumption).Potential complex data in original space modeling,其公式$Z=f_{\Theta }(X)$和

The original data sampling：对$X\sim P_x(X)$The sampling is through$Z\sim P_z(Z)$进行的,然后通过$f_{\theta }$Reverse mapping get$X=f_{\theta }^{-1}(Z)$.

设$Z=f_{\theta }(X)=f_L(X)\circ ...\circ f_1(X),H_l=f_l(H_{l-1})$其中$f_l(l=1,\dots L\in N^{+})$是可逆映射.$H_0=X,H_L=Z$和$P_Z(Z)$Follow the independent dimension standard isotropic gaussian distribution.According to the variational formula to get$X$的对数似然:

其中,$P_{Z_i}(Z_i)$为$Z$的第$i$维的概率,KaTeX parse error: Undefined control sequence: \rirc at position 18: …{\theta} = f_L \̲r̲i̲r̲c̲ ̲… \rirc F_1Is a kind of neural network can learn the reversible depth.

Invertible affine coupling layers
Jacobian computation,NICE 和 RealNVP Design the coupling of affine transform：

• 将$X$分成两个部分$X=(X_{1:d},X_{d+1:n})$, The reversible by the：
  
• 在$X_{d+1:n}$The affine transformation in the, Expression ability depends onScale函数$S_{\theta }\to R^{n-d}$和Transformation函数$T_{\theta :R^d\to R^{n-d}}$ Any neural structures of the.
• Effectively calculate jacobian：
  
  Splitting Dimensions
  基于流的RealNVP和GlowModel using extrusion operation,将空间维度$X^{c\times n\times n}$压缩为$X^{(c{h}^2)\times \frac{n}{h}\frac{n}{h}}$In order to obtain more channel,Then the channel is divided into two and a half for coupling layer.A layer of deep flow model to transform a layer of the same dimension before,To maintain all dimension transformation.

Numerical stability by actnorm
In order to ensure the numerical stability of the model based on flow,在GlowThe introduction of the code of conduct layer,This layer through a study of the scale of the affine transformation and deviation of each channel in batch dimension normalization.Each channel in the scale and bias initialized to batch scale standard deviation of the mean and the inverse.

MOFLOW MODEL

Below is the entire model figure,The molecular figureMWith the characteristic matrixA表示原子,Using adjacency tensorB表示键.推论:Atomic flow figure conditions(GCF) $f_{A|B}$将给定的BLatent vector into conditions$Z_{A|B}$,键的Glow $f_B$将$B$Converted to latent vector$Z_B$.Latent space obey gaussian distribution.生成:The generation process is operating in front of the reverse transformation,Then validity is a correct program,To ensure the effectiveness of chemical.

$A\in R^{n\times k}$Atomic matrix,And the atomic matrix has the most$n$个,以及$k$Type of atom.$A(i,k)=1$代表着原子$i$的类型为$k$.
$B\in R^{c\times n\times n}$：Chemical bond matrix.$c$：Represents the number of chemical bond types.$B(c,j,i)=1$代表着原子$i$与原子$j$The type of chemical bond between as$c$
$M=A\times B\in R^{n\times k}\times R^{c\times n\times n}$即矩阵$M$是矩阵$A$和$B$的笛卡尔积.So the molecular structure of figure$M$Can be considered with multiple types of nodes and a variety of types of the edge of the undirected graph.

In order to capture the composite structure of atomic and molecular figure key,The author will the molecule generation model is decomposed into two parts：

参数${\theta }_B$与${\theta }_{A|B}$By maximizing the type to learn：

Atomic flow figure conditions

A given chemical bond matrixB,Atom flowThe goal is to generate the correct atomic matrixA,To generate effective molecularM,有两部分组成.

• B-conditional flow: $Z_{A|B}|B=f_{A|B}(A|B)$Mapping is a reversible and dimension unchanged,Calculate the meetflowThe Jacobi matrix model.则有：
  
• Flow diagram conditions :By introducing a figure convolution structure,The author designed the each figure scale function of coupling layerSΘ和变换函数TΘ.使用多层感知器（MLP）Output layer stack multiplegraphconv->BatchNorm1d->ReLuLayer to build figure proportion functionSΘAnd figure conversion functionTΘ.

Glow for Bonds

Key flow to study reversible mappingfB,According to the change of variable formula,Get the keys of the log probability and tensor by reverse mapping to generate key. Can the key tensor using any flow model,并基于GlowFramework of a variant to build the key stream$f_B$.The author also follow the affine coupling layer scheme to establish the reversible mapping.For each of the affine coupling layer,分割输入$B$沿着通道$c$Dimension is divided into two parts$B=(B_1,B_2)$,得到输出 $Z_B=(Z_{B1},Z_{B2})$.

作者使用多个3×3 conv2d->BatchNorm2d->ReLuTo construct the affine coupling layer.Each affine coupling of jacobian matrix logarithm is：

Effective correction

Molecules must follow each atom valence constraint,But with the generated key tensor matrix assembly a molecular and atomic may cause chemical is invalid.So you need to define a validity checking for each atom：

$B\in \{0,1{\}}^{c\times n\times n}$是 C∈{1,2,3}化学键（单,双,三）On the hotkey tensor alone,ChRepresents a formal charge,The authors consider the role of the formal charge,It may be electrically charged atoms into additional keys.The author here only considerCh=1的,和,And make other atomsCh=0.

实验

The author through the following four aspects has carried on the experiment to evaluateMoFlow模型：
1）Molecular graph generation and reconstruction：First of all is to restore data set all molecules,Effective and generate as much as possible,Does not belong to the data sets of molecular graph;
2）Visual space potential：MoFlowWhether to embed the molecular graph with reasonable chemical similarity in continuous latent space?
3）性质优化：Model can generate molecules with characteristics of optimization figure;
4）约束属性优化：Model can be generated with optimal properties of novel molecular figure,At the same time as far as possible, keep chemical similarity.

数据集

作者在实验中使用了两个数据集QM9和ZINC250K.

评价指标

包括：
1）Validity：Chemical effective molecular percentage in all of the generated molecular;
2）Uniqueness：All generated molecules do not belong to the original data set and to meet the chemical availability ratio of the number of molecules and the molecule number;
3）Novelty：All generated molecules do not belong to the original data set and meet the number of molecules and the effectiveness of the chemical data set contains the ratio of the number of molecules;
4）Reconstruction：The original data set can completely be model reconstruction of the number of molecules of the ratio of the number of molecules with data set contains;
5）N.U.V.：A molecule that meet：Validity,Uniqueness,NoveltyThe number of molecules with the ratio of the number of molecules generated;
6）Validity without check/correction.