Four classic training modes in comparative learning

Contrastive learning is a very effective method in unsupervised representation learning , The core idea is to train query and key Of Encoder, Let this Encoder To match query and key The generated coding distance is close to , The mismatched encoding distance is far . Want to make comparative learning effective , A core point is to expand the comparison sample （ Negative sample ） The number of , That is, each time the gradient is updated ,query What you see doesn't match key The number of . The more negative samples , The closer to the actual goal of comparative learning , namely query And all that don't match key All far away .

At present, there are 4 The most typical paradigm , Respectively End-to-End、Memory Bank、Momentum Encoder as well as In-Batch Negtive. The differences of these comparative learning structures are mainly reflected in the treatment of negative samples ,4 The first method is an evolving relationship . This article mainly introduces this 4 The classic work of a comparative learning structure .