2018 arXiv preprint | MolGAN: An implicit generative model for small molecular graphs

Paper: https://arxiv.org/abs/1805.11973
Code: https://github.com/yongqyu/MolGAN-pytorch

MolGAN: Implicit generation model of small molecule graph

The depth generation model of graph structure data provides a new perspective for chemical synthesis : By optimizing the differentiable model , Directly generate molecular diagram , It is possible to avoid expensive search processes in discrete and vast space of chemical structures . So the author puts forward MolGAN, An implicit for small molecule graphs 、 Likelihood free generation model , It avoids the need of expensive graph matching process or node sorting heuristic based on likelihood method . The author uses generative countermeasure network (GANs) Direct operation diagram structure data . This method is combined with reinforcement learning objectives , To encourage the production of molecules with specific desired chemical properties . stay QM9 In the experiment of chemical database , It is proved that the model can produce close 100% Effective compound of .

Introduce

Molecular diagram

lately , Direct operation in graph space has proved to be a feasible alternative to molecular data generation modeling , The added benefit is that all generated output is valid graphics .

Diagram representation ：
Express a molecule as an undirected graph . For one with N A molecular graph of nodes can be represented by the following two matrices :

• Adjacency matrix ： $A\in R^{N\times N\times Y}$.$Y$ It represents that there is Y Covalent bond （ Such as single key 、 Double bond and triple bond ）.$A_{i,j}\in R^Y$ Represents atoms i , j i,ji,j Covalent bond type between （ use one-hot Vector to represent ）.
• $X=[x_1,...x_N{]}^T\in R^{N\times T}$ Is the node characteristic matrix . Each atom corresponds to a node $v_i\in \nu$; This node uses a $T$ Dimensional one-hot vector $x_i$

Implicit generation model and likelihood generation model

The optimization process of likelihood generation model is more stable and simple , But the amount of calculation to explicitly evaluate the possibility of all possible node arrangements is huge , That is, the likelihood method is difficult to meet the permutation invariance .
The method of generating confrontation network completely solves this problem , Because many graph convolution methods are permutation invariant , These graph convolutions can be used to build our discriminator .

Model

Model architecture

• generator ：$N=9$,$T=5$ ,$Y=4$, And first take a sample according to the normal distribution 32 Potential representation of dimension $z\sim N\in (0,1)$. The generator has three layers $MLP[128,256,512]$, And will $tanh$ As the activation function of each layer . The last layer is linearly projected to match X and A dimension , And make use of $softmax$ To regularize .
• Judging device ： Discriminator and feedback network architecture ： Both use two layers $RelationalGCN$, But parameters are not shared . The output of graph network is processed to obtain graph representation vector , And input the obtained graph representation vector into a two-level $MLP[128,1]$ To get the final output .$MLP$ The discriminator adopts $tanh$ As an activation function , The feedback network adopts $sigmoid$ As an activation function .

generator

The objective function of the generator is ：

among $L_{WGAN}$ It's against the loss of training , $RL$ Is the loss of reinforcement learning ,KaTeX parse error: Undefined control sequence: \lamda at position 1: \̲l̲a̲m̲d̲a̲ Is the proportion of two losses .

For the generation process ： sampling $z$ Send it to the generator to generate A And X, Next, in order to get the molecular diagram , Discrete sparseness is adopted to send it into the discriminator and reward network . But this will make the generator lose the gradient used to update parameters . The author adopts three methods to solve this problem ：

• Do not use discrete sparseness , Directly send the generated probability matrix to the discriminator and reward network ;
• take X And A And Gumbel The noise is added and then sent to the discriminator and reward network ;
• utilize link To get the gradient
  $$\begin{gathered} {\tilde{X}}_{i,j}=X_{i,j}+Gumbel(\mu =0,\beta =1) \\ \tilde{A}=A_{ijy}+Gumbel(\mu =0,\beta =1) \end{gathered}$$

Discriminator and reward network

• The reward network is used to approximate the reward function , Reinforcement learning is used to optimize the model in the non differentiable direction . The graph in the data set and the generated graph are the input of the reward network , But unlike the discriminator , The reward network should score each molecule （ For example, how likely are the generated molecules to dissolve in water , The true score is determined by RDKit Provide ）. And when the resulting molecules are chemically ineffective , The whole reward network will output 0.
• The discriminator uses WGAN To train , namely ：
  
• Judging device $D_{\Phi }$ And reward function ${\hat{R}}_{\psi }$ Take the graph structure as input and output a scalar , And both adopt the same architecture , But do not share parameters . First, it is convoluted by a series of graphs Relational GCN:
  
  After several layers of convolution , The graph representation vector is obtained from the following formula ：
  
  What will be obtained $h_g$ As a MLP The input of gets a scalar . The process of forward propagation ends here .

Data sets

• QM9： N=9 Maximum number of nodes ; T = 5 Number of atomic types , namely ：(C, O, N, F, and one padding symbol);Y = 4 Number of edge types , namely ：(single, double, triple and no bond) .

The evaluation index

• Validity： The ratio of the number of molecules conforming to the chemical regulations in the generated molecules to the total generated number ;
• Novelty: The generated effective molecules do not belong to the original data set （ That is, the newly generated ） The ratio of the number of molecules to the total number of effective molecules ;
• Uniqueness: The ratio of the number of molecules generated that are not in the data set to the number of effective molecules . This indicator reflects the degree of change of the sample in the sampling process .
• Druglikeness： How likely are compounds to become drugs
• QED: The expected weighted geometric mean is used to quantify the properties of compounds , Capture the potential data distribution of several drug properties .
• Solubility: Indicates the degree of hydrophilicity of the molecule ; Logarithmic octanol - Water partition coefficient (LogP)： It is defined as the logarithm of the concentration ratio of a solute in two solvents .
• Synthetizability: It can quantify the difficulty of a molecule synthesis
• **SA score: ** It is a method to estimate the comprehensive usability with probability method .

experimental result

MolGAN One limitation of is the sensitivity to model collapse .RL And GAN The objective function of does not encourage the production of diverse non unique outputs , Therefore, the model is pulled to a solution involving only a few sample variability , If the training doesn't stop early , Eventually, only a small number of different molecules will be produced .