Void convolution, deformable convolution, deformable ROI pooling

1. Why mention void convolution 、 Deformable convolution 、 Deformable ROI Pooling ？
   answer ： These techniques are related to the receptive field of convolution kernel , Receptive field refers to ： The mapping area of the feature graph on the original graph , Generally speaking, using a larger convolution kernel and pooling will increase the size of the receptive field （ Or use convolution + Pool operation ）, But this improvement is limited or will lose accuracy . Cavity convolution 、 Deformable convolution 、 Deformable ROI Pooling can be done without losing much accuracy , Effectively increase receptive field .

Cavity convolution （Dilated Convolution）

• The basic principle ：Dilated/Atrous Convolution( It is called void convolution or dilation convolution in Chinese ) Or is it Convolution with holes It's easy to understand literally , It's in the standard convolution map Inject a hole into the , In order to increase reception field. Compared with the original normal convolution,dilated convolution One more. hyper-parameter be called dilation rate, refer to kernel Number of intervals (e.g. natural convolution yes dilatation rate 1).
  

• The receptive field formula

• PyTorch Realization ： stay Conv2d Parameters used in dilation To achieve

• The benefits of empty convolution ：

  • Expand the feeling field ： stay deep net In order to increase the receptive field and reduce the amount of calculation , Always downsampling (pooling or s2/conv), Although this can increase the receptive field , But the spatial resolution is reduced . In order not to lose resolution （ Be skeptical ）, And still expand the receptive field , Void convolution can be used . This is testing , Useful in split tasks . On the one hand, when the receptive field is large, it can detect and segment large targets , On the other hand, with high resolution, it can accurately locate the target .
  • Capture multi-scale context information ： Cavity convolution has a parameter that can be set dilation rate, The specific meaning is to fill in the convolution kernel dilation rate-1 individual 0, therefore , When the settings are different dilation rate when , The feeling field will be different , That is, multi-scale information is obtained .

• The disadvantage of empty convolution ：

  • Local information loss The Gridding Effect： Because the calculation method of void convolution is similar to the chessboard format , The convolution result of a certain layer , A separate collection from the upper layer , No interdependence , Therefore, there is no correlation between the convolution results of this layer , That is, local information is lost .
  • Information obtained from a distance is not relevant Long-ranged information might be not relevant： Sparse sampled input signal due to hole convolution , So that there is no correlation between the information obtained by long-distance convolution , Affect the classification results .

• Solution ：Hybrid Dilated Convolution (HDC)、Atrous Spatial Pyramid Pooling (ASPP)

Deformable convolution （Deformable Convolution）

• The basic principle ：

  • Without additional monitoring , Use additional offsets to increase the spatial sampling location in the module , And learn the offset from the target task . New modules can easily replace existing cnn Common modules in , And simple end-to-end training can be carried out through standard back propagation , Thus a deformable convolution network is generated .
  • The dimension of the offset matrix is hw2N, Why hw？ After convolution, the two sizes are at most the same , So each location stores twice the size of the convolution kernel 2N=2kk Offset
    

• PyTorch：torchvision.ops.deform_conv2d

• Be careful ： Here is a very, very confusing point , So-called deformable, to the end deformable Where is the ？ Many people may think that deformable conv Learning is deformable kernel, In fact, it's not ！ This article is not about kernel Study offset It's right feature Learn one for each position of offset.

Deformable ROI Pooling

• The basic principle
  

reference

[1] https://zhuanlan.zhihu.com/p/43784441
[2] https://blog.csdn.net/qq_41076797/article/details/114593840
[3] https://www.zhihu.com/question/54149221
[4] https://zhuanlan.zhihu.com/p/138886700
[5] https://blog.csdn.net/C1nDeRainBo0M/article/details/123104016?ops_request_misc=%257B%2522request%255Fid%2522%253A%2522165608484916782248594568%2522%252C%2522scm%2522%253A%252220140713.130102334…%2522%257D&request_id=165608484916782248594568&biz_id=0&utm_medium=distribute.pc_search_result.none-task-blog-2_allsobaiduend~default-1-123104016-null-null.142^v21pc_rank_34,157^v15new_3&utm_term=torchvision.ops.deform_conv2d&spm=1018.2226.3001.4187
[6] https://blog.csdn.net/qq_45122568/article/details/124190576