【Attention】Visual Attention Network

Thesis link ：https://arxiv.org/abs/2202.09741
Code link ：https://github.com/Visual-Attention-Network

One 、 background

From the NLP Of self-attention The mechanism is applied to 2D When it comes to images , There are three questions ：
take 2D The result is seen as 1D Sequence , It's going to lose 2D Spatial structure
High resolution images will lead to great computational complexity
General self-attention Are only able to capture spatial correlation , Ignoring the correlation on the channel

Two 、 motivation

In order to solve the above problems , In this paper, we propose a method for visual tasks large kernel attention (LKA), bring self-attention It can adaptively capture long-distance relationships .

3、 ... and 、 Method

LKA Absorbed convolution and self-attention The advantages of —— Including local structure information 、 Long distance dependence 、 Self adaptability

be based on LKA, The author puts forward a new vision backbone——visual attention network（VAN）.

3.1 Large Kernel Attention

Pictured 2, A large convolution can be divided into three parts ：

• Local convolution in space （depth-wise convolution）
• In space long-range convolution（depth-wise dilation convolution）
• Convolution on channels （1x1 convolution）

in other words , One $K\times K$ The convolution of can be split as follows ：

• One $\frac{K}{d} \times \frac{K}{d} $ Of depth-wise dilation convolution,$d$ Is the expansion rate
• One $(2d-1)\times (2d-1)$ Of depth-wise convolution
• One 1x1 Convolution

The split convolution can capture long-distance information , And save computing resources , After getting a long-distance relationship , Be able to estimate the importance of each point , Generate attention map.

Pictured 3a Shown ,LKA Model as follows ：

3.2 VAN

VAN Simple structure , There are four floors , The model structures of different levels are shown in table 2 Shown ：

Pair graph 3d Each of the stage, First, down sample the input , Then follow up .

Complexity analysis ：

The implementation details are as follows , Use the following three convolutions to achieve something similar 21x21 Convolution of

• 1 individual 5x5 depth-wise convolution
• 1 individual 7x7 depth-wise convolution（d=3）
• 1 individual 1x1 Convolution

Four 、 effect

4.1 classification

4.2 object detection

4.3 Semantic segmentation