Deep learning (self supervision: Moco V2) -- improved bases with momentum contractual learning

Preface

This article is an article by Mr. He Kaiming's team , The author found SimCLR As mentioned in the article trick stay MoCo Still valid on .

About MoCo, You can browse my last article ： link

To be specific

• encoder The output of will go through a linear fc Layer handles , Will be fc The layer is replaced by SimCLR Two layers in the MLP（ Hidden layer is 2048 dimension , Use ReLU Activation function ） The specific performance is shown in the figure below
  
  $T$ by InfoNCE Temperature superparameter in , A simple replacement operation can improve performance up to 6%, It's really amazing .
• Use SimCLR Fuzzy data enhancement in ,SimCLR Medium color distortion Data enhancement can damage MoCo Performance of .

After comparative study ,MoCo V2 In image classification 、 The performance of target detection is shown in the following figure

It seems that for comparative learning , some trick Operation can greatly improve the performance of the model .