Paper notes: highly accurate protein structure prediction with alphafold (alphafold 2 & appendix)

notes ： This article Nature The paper of , If you just look at the text , It's impossible to understand , It needs to be combined with its supplementary materials

At the same time, this article is too hard to read QAQ, If there is something wrong, you are welcome to correct

0 Preface

• Protein structure prediction ： Given a string of amino acid sequences of a protein , Guess the protein 3D What does the structure look like
• Now biology may take a long time to understand the structure of a protein
  • Let the protein move , Look at its structure with a microscope from different angles
• This paper puts forward AlphaFold 2
  • Previous work AlphaFold 1 Insufficient precision
  • AlphaFold 2 The accuracy can reach the atomic level
    • Measured in the laboratory （ real ） The difference between the position and the predicted position is within the level of atomic size
  • The model uses some knowledge of biology and Physics , Integrated into deep learning

1 The model part

1.1 The whole model

        Transformer part （ That's what we have here encoder） Only play different elements （ Amino acids, ） The role of information integration between , The refining part of real information , Is in decoder Partially implemented

  My thoughts on recycling mechanism ：

This mechanism is a little similar RNN Zhongba hidden state Pass it on to the next round RNN. By constantly using the output learned in the last round , To get better output （ The output of each round may have limited accuracy , Get better results through continuous iteration ）

The difference lies in , Here only reuse the structure , But don't send back the gradient .（ That is to say, the several outputs passed back are detach The output of ）

——> and RNN Compared with the reuse structure , Although there is no difference in calculation time , But there are differences in memory （RNN Words ,“ Comes back ” Of hidden state The gradient of also needs to be recorded in memory , But the output returned here does not need to record the gradient ）

1.2 “encoder” part

1.2.1 The whole model  

 

1.2.2 row-wise gated self-attention with pair bias

 1.2.3 column-wise gated self-attention

General process and 1.2.2 similar , The difference lies in , Here is by column self-attention（ The weight of amino acids at the same position in different proteins ）

 1.2.4 MSA transition

（ Two transition It's the same ）

This is a MLP 

 

1.2.5 Outer product mean

 

 1.2.6 Triangular multiplicative update

 

  Allied , Just turned out

 

  Be careful ： Due to the sequence of out side and in side modules , So the resulting matrix is not necessarily symmetric .

1.2.7 Triangular self-attention

The above figure shows the previous row by row attention, The following figure is here attention, It can be seen that it is very similar

The pseudo code part is basically the same , But the meaning of the paper here is the fifth line （attention Weighted part ） Some trigonometric properties are used

 

 1.3 “decoder” part

1.3.0 How to predict ？

• Express proteins 3D The simplest way to structure is to record each element 3D coordinate .
  • Proteins rotate / Translation does not affect protein structure , But if you use 3D In terms of coordinates , The absolute position will change
  • ——> So the relative position is used here

•   Protein can be thought of as a trunk + Branched chain
• The main point is recorded as x, Then any point on the chain / The latter point of the trunk can be seen as y=Rt+x
  • according to 3*3 Matrix R Make a spin
  • according to x Do translation

1.3.1 The whole model

 

1.3.2 IPA Invariant point attention

 1.3.3 Backbone update

Update the key points s and T

 

Experimental results

.1 Comparison with other projects

•   Each column is a model （ The team ）
• Each bar graph is the corresponding model , The difference between the average predicted location and the real location （ The unit is , namely rice , That is, the size of the atom ）
• You can see AlphaFold 2 The accuracy of has reached atomic accuracy , This is a milestone accuracy

2 AlphaFold The accuracy of the prediction

• The blue one is AlphaFold Predicted results
• Green is the result predicted by the Laboratory
• It can be seen that their error is indeed in the size of an atom （ Black sphere ） within