Pan for in-depth understanding of the attention mechanism in CV

CV Medium Attention Mechanism summary （ 3、 ... and ）：PAN

PAN: Pyramid Attention Network

Thesis link ：《Pyramid Attention Network for Segmantic Segmentation》

One 、 Abstract

PAN Network structure ：

Pyramid Attention Network (PAN) Combine attention mechanism with spatial pyramid to extract dense features more accurately , Improve the accuracy of semantic segmentation . The problem to be solved by this method （motivation） And major contributions include ：

1.1 Put forward Feature Pyramid Attention(FPA) Module

• Motivation

  The existence of objects at multiple scales cause difficulty in classification of categories.
  The different sizes of objects in the same category increase the difficulty of classification , The existing algorithm uses dilated convolution (ASPP) Or pyramids (PSPNet) To increase the accuracy of classification , However, there are problems of grid effect and loss of pixel level location information respectively .

• FPA module

  FPA module performs spatial pyramid attention structure on high-level output and combine global pooling to learn a better feature representation. （ Including spatial pyramids and global convergence , The output acting on the deep network ）

1.2 Put forward Global Attention Upsample(GAU) Module

• Motivation

  high-level features are skilled in making category classification, while weak in restructuring original resolution binary prediction.
  Deep network can extract more accurate category information , But the location information is lost , This is unfavorable for predicting the position of certain objects .

• GAU module

  GAU module acts on each decoder layer to provide global context as a guidance of low-level features to select category localization details, ( The feeling field of deep network is larger , Include more accurate category information , Shallow networks contain more accurate location information , The characteristics of the two can be combined to improve the segmentation effect .

Two 、 Module details

2.1 Feature Pyramid Attention

① Extracting local features （pixel-wise）
FPA Modules are used separately $3\times 3,5\times 5,7\times 7$ The convolution kernel with three sizes further extracts the features contained in the feature graph , Then add and fuse the feature maps of the three sizes in turn ;
② Extract global features （channel-wise）
increase Global Pooling Access Rd , Extract channel features .（ The idea here is similar SE modular , The difference is that only one layer of convolution is used ,SE Two layers were used ）

2.2 Global Attention Upsample

The main idea ：

The main character of decoder module is to repair category pixel localization. Furthermore, high-level features with abundant category information can be used to weight low-level information to select precise resolution details.
“ decode ” The purpose of is to get the pixels contained in each category , Achieve pixel level classification . The output of the deep network contains richer category information , The output of the shallow network contains more prepared location information , therefore , We can use the category features extracted from the deep network to guide the shallow network to achieve more accurate pixel level classification .

Implementation details ：

1. Use $3\times 3$ The convolution of low-level The number of channels of the characteristic graph ;
2. Use global average pooling extract high-level The overall information of ,$1\times 1$ Convolution plus BN Layer and ReLU Activation function ;
3. high-level Output after up sampling and weighting low-level The output characteristic image of is added pixel by pixel .

3、 ... and 、PyTorch Code implementation

Reference resources ：https://github.com/JaveyWang/Pyramid-Attention-Networks-pytorch