Swin Transformer【Backbone】

background

Swin Transformer yes ICCV2021 Best paper .

ViT Give Way transformer from NLP Apply directly to CV There are two direct questions ： Scale problem （ For example, pedestrians , The size of the car is NLP There is no field ）, Sequence problem （ If the basic unit is image pixels , Sequence too large ）.16 individual patch（ Low resolution ） send ViT It may not be suitable for intensive predictive tasks , Global modeling increases the computational complexity by a factor of two .

Swin Transformer Give Way transformer It can also do hierarchical feature extraction , Make the extracted features have the concept of multi-scale . Calculating self attention in the window reduces the sequence length （ Computational complexity increases linearly with image size , Non square growth ）, Moving makes the interaction between two adjacent windows . Semantically similar parts probably appear in adjacent areas , such local The design of is completely adequate ,ViT The overall design of is redundant .

in general , It refers to the combination of window sliding of convolution and the grasp advantage of its own global vision .

Model structure

The initial operation can be compared ViT understand , Suppose the input image [224,224,3], after Patch Partition become involved [56,56,48]（ similar ViT, But the patch The size is 4*4）, after Linear Embedding formation [56,56,96], Change to [3136*96], Then type it into Block The final treatment is [3136*96]. Then type in Patch Merging（ Space size is divided by 2, Multiply the number of channels 2, In order to compare convolutional neural networks, we have such an operation , This operation can be understood as exchanging space for dimension , See the video for reference , Very detailed .） become [28,28,192], Cycle in turn to form the whole Swin Transformer.

Swin Transformer Block

Mentioned earlier ,[56,56,96] After the tensor of input block , It's in 7*7 Self attention in the window of . See the first figure (b) yes Swin Transformer A basic computing unit of （ There are two Block）, First calculate self attention in the window , And then shift The back window does self attention .

How to calculate the self attention after sliding window B Stand on the explanation of God Mu .

Reference resources

B Stand Mu Shen Swin Transformer Intensive reading 【 Intensive reading 】