Mila, University of Ottawa | molecular geometry pre training with Se (3) invariant denoising distance matching

【 title 】Molecular Geometry Pretraining with SE(3)-Invariant Denoising Distance Matching

【 The author team 】Shengchao Liu, Hongyu Guo, Jian Tang

【 Time of publication 】2021/06/27

【 machine structure 】Mila、 University of Ottawa

【 Thesis link 】https://arxiv.org/pdf/2206.13602v1.pdf

【 Code link 】https://github.com/chao1224/SE3DDM

Due to the limited number of labeled molecules , Pre training molecular characterization is crucial in various applications of drug and material discovery , However, most of the existing work focuses on the pre training of two-dimensional molecular graphs . However , The ability of pre training 3D geometric structure has not been well explored , This is because it is difficult to find a sufficient self supervised pre training task , To effectively extract basic features from geometric structures . Under the excitation of three-dimensional molecular dynamic properties , The continuous motion of molecules in three-dimensional Euclidean space forms a smooth potential energy surface , This paper proposes a three-dimensional coordinate denoising pre training framework to simulate such an energy landscape . utilize SE(3) Invariant fraction matching method , In this paper, SE(3)-DDM, The coordinate denoising task is effectively reduced to the denoising of the distance between paired atoms in molecules . Comprehensive experiments have confirmed the effectiveness and robustness of the method proposed in this paper .

The figure above shows the coordinate geometry of molecules . Molecules are in continuous motion , Form a potential energy surface （PES,Potential Energy Surface）, Each of these three-dimensional coordinates （x Axis ） Corresponding to an energy value （y Axis ）. Molecules provided , Conformation , At the local minimum （g1）. It is often accompanied by noise around the minimum （ Such as statistical and systematic errors or vibration ）, This can be done with perturbed geometry （g2） To capture .

The picture above shows SE(3) Invariant denoising distance matching （SE(3)-DDM）.g1 and g2 Near the same local minimum , But there is coordinate noise disturbance . first , The author wants to do coordinate denoising between these two states . And then according to SE(3)-DDM The advice of , This paper turns it into an equivalent problem , That is, distance denoising . The figure shows three key steps ： Extract the distance from two geometric views , Then the distance disturbance , Finally, denoise the disturbed distance .

The table above shows QM9 Result . You can see , The performance of most of the pre training baseline tested is similar to that of the random initialization baseline （-） Quite a , Even worse than it . The best performing baseline is the representation reconstruction method （RR）, It optimizes coordinate aware MI; It's in QM9 Of 12 Of the tasks 5 A task and MD17 Of 8 Of the tasks 6 Perform better than other baselines in tasks . This means application generativity SSL To maximize this coordinate perception MI It has potential . What this article puts forward SE(3)-DDM stay QM9 and MD17 All of the 20 Continuously improved performance has been achieved on all tasks . All these observations have empirically verified the effectiveness of distance denoising , It simulates the most decisive factor in molecular geometry data .

 

Innovation points

(1) This paper presents a new coordinate denoising method for molecular geometry pre training , As far as this article is concerned , This is the first method to use only three-dimensional molecular data for pre training .

(2) In order to overcome the challenge of achieving the coordinate denoising target , This paper introduces SE(3) Invariant fraction matching strategy , This target is successfully transformed into the denoising of pairwise atomic distance which can be effectively calculated .

(3) This paper proves the SE(3)-DDM The effectiveness and robustness of the method .