2022 ICLR | CONTRASTIVE LEARNING OF IMAGE- AND STRUCTURE BASED REPRESENTATIONS IN DRUG DISCOVERY

CONTRASTIVE LEARNING OF IMAGE- AND STRUCTURE BASED REPRESENTATIONS IN DRUG DISCOVERY

CLOOME: A molecular characterization tool based on multimodal contrastive learning

This article is written by John · Kepler Linz University Ana Sanchez-Fernandez The team recently published on ICLR 2022, Its main content is ： before , Comparative learning methods CLIP and CLOOB It has been proved , When training on multiple modal data , The learned representations can be highly transferred to a large number of different tasks . In the field of drug discovery , Molecular images and chemical structures are similar multimodal datasets , At present, there is no comparative study on the two , This method has great research value in the field of drug discovery with high label cost . Therefore, this work starts with the easily obtained molecular microscopic images and structures , This paper proposes a method based on CLOOB(Contrastive Leave One Out Boost) A new method of contrastive learning ——CLOOME(Contrastive Leave One Out Boost for Molecule Encoders). Through the linear detection of the molecular activity prediction task , It is proved that the method can be used for the transfer characterization , Besides , This characterization can also be used for alternative tasks of biological isomerism .

Method

This work compares and learns the molecular characterization from the microscopic images and chemical structure data of molecules , To obtain a highly transportable molecular encoder ( Pictured 1 Shown ).CLOOME Compared with traditional molecular encoder or manual extraction of molecular features , Its biggest innovation is that it can optimize the molecular characterization without the input of active molecular data or artificial prior knowledge .

Training data from N Microscopic images of disturbed molecular cells and molecular chemical structure composition ：$\{(x_1,z_1),...(x_n,z_N)\}$. Suppose an adaptive image encoder $h^x(.)$ And adaptive structure encoder $h^z(.)$ Images and chemical structures can be mapped to $embedding{x}_n=h^x(x_n)$ and $z_n=h^z(z_n)$. Pictured 1(a), To stack microscopic images embeddings（ That is, the features encoded by the picture encoder ） Write it down as $X=(x_1,...x_N)$, Through the structure encoder embeddings Write it down as $z=\{z_1\dots ,z_N\}$. The goal of contrastive learning is to improve the similarity of matching pairs , Reduce the similarity of mismatched pairs . This goal is usually achieved by minimizing InfoNCE Loss is achieved by maximizing embedded mutual information ：
$$L_{infoNCB}=-\frac{1}{N}\sum _{i=1}^{N}In\frac{exp({\tau }^{-1}{x}_i^T{z}_i)}{{\sum }_{j=1}^{N}exp({\tau }^{-1}{x}_i^T{z}_j)}-\frac{1}{N}\sum _{i=1}^{N}ln\frac{exp({\tau }^{-1}{x}_i^T{z}_i)}{{\sum }_{j=1}^{N}exp({\tau }^{-1}{x}_j^T{z}_i)}$$

But with this InfoLoss It is easy to over present some features , Other features are ignored . Therefore, this work is based on CLOOB To optimize the contrastive learning .

CLOOB Method . First, embed from the stored image $U$ And structural Embeddedness $V$ Retrieval image embedding and structure embedding in ,$U_{x_i}$,$U_{z_i}$; Represent image embedding and structure embedding respectively , And CLOOB similar , utilize modern Hopfield Search through the network ：

$$\begin{gathered} U_{x_i}=Usoftmax(\beta U^T{x}_i) \\ {V}_{x_i}=Vsoftmax(\beta V^T{x}_i) \\ {U}_{z_i}=Usoftmax(\beta U^T{z}_i) \\ {V}_{z_i}=Vsoftmax(\beta V^T{z}_i) \end{gathered}$$
then , take InfoLOOB Loss as objective function :

There are some differences between microscope image and natural image , For example, coloring will affect the number of image channels , All experiments in this paper adopt 5 Of input channels ResNet-50 As an encoder , And reduce the microscope image to 320*320.

Molecular structure encoder CLOOME Use descriptor based fully connected networks . Besides , Graph neural network with proper pooling operation 、 Message passing neural network or sequence based neural network can be used as structural encoder .

result

chart 2. Retrieve task result examples . Given a micrograph ,CLOOME The molecular structure corresponding to the micrograph can be retrieved from several molecular structures ( The blue box in the figure shows the matched molecular structure ).CLOOME It can be used to extract molecules that can produce similar biological effects on treated cells , Bio isomers .