Swintransformer network architecture

SwinTransformer

1. original text ：

2.

3.

4.
For different versions of Swin Transformer Model , there C It's different

5.
Conduct Linear Embedding after , Each channel is also Layer Norm

6.
Both of these are implemented according to the convolution layer

7.
Here are two Swin Transformer Blocks, But these two are usually used in pairs

8.
The multi-layer perceptron here is Vision Transformer The author said , So I didn't talk about it in this video

9.
This position , For classified networks , In fact, there is a network structure behind it , It's just not drawn here

10.
after Patch Merging Then the length and width of the characteristic matrix will be halved , The number of channels will be doubled


11.

12.
MSA Namely transformer The long attention mechanism in ,MSA That is, every pixel of the characteristic matrix will calculate its Q,K,V, And every pixel will also go with other pixels K Multiply , Calculate the relevant weights , Finally multiply by the relevant V, Finally hungry to the final result .

W-MSA Is to use for each small characteristic matrix MSA, The author did this to reduce the amount of computation .

W-MSA shortcoming ： There is no information exchange between windows , So I can put MDTA and W-MSA Make two branches , Calculate details and global information respectively , Or use convolution layers and W-MSA Make two branches , Convolution layer calculates global information ,W-MSA Calculate local information .
14.

15.
In order to solve W-MSA The disadvantage of not being able to communicate between different windows , Created SW-MSA


16.

Put the matrix above , First move two pixels up , Then move two pixels to the left to get the following characteristic matrix

17.

18.

The final functions of the two structures are the same

21.
With Swin-T give an example