FPN characteristic pyramid network

improvement Convolutional neural networks object detection , Extract multi-scale feature information for fusion , Improve the accuracy of target detection  

R-CNN Series is to extract features from the last feature map , So as to carry out target recognition ,

The top-level features may ignore some information of small objects in the process of continuous convolution pooling

Only the top-level features are used for target recognition , Can not completely reflect the information of small target objects .

If you can combine multi-level features , The accuracy of multi-scale detection can be greatly improved .  

FPN Sure Widely used Aimed at Detection of small object On

Image pyramid

It mainly uses manual extraction of the features of images with different scales

  Traditional methods of extracting multi-level features , Multi resolution to interpret images

   Using convolutional neural network in Picture pyramid （a Left ） Feature extraction on , You can build Characteristic pyramid （a Right ）

          

  The problem is that it takes too much time , High computing power required

SSD（Single Shot Detector）

CNN There is a multi-level characteristic graph in the calculation , And the scale of the characteristic map of different layers is different , Like a pyramid structure .

If we use this pyramid structure for target detection , Extract features of different scales from different layers of the network for prediction ,

Not only does it not add extra computational work , Low level features can also be used .

SSD（Single Shot Detector） This is the way to do it .

 SSD The algorithm does not use enough low-level features （ stay SSD in , The lowest layer is characterized by VGG Online conv4_3）

FPN Algorithm

Put low resolution 、 High level features of high semantic information

And high resolution 、 Low level features of low semantic information  

Make side connection from top to bottom , It makes the features of all scales have rich semantic information .

Forward process of convolution neural network （ Above the left ）, Top down process （ Top right ） And the side connection between features .

In the forward process , The size of the feature map changes after passing through certain layers , And it doesn't change when you pass through other layers , The layers that do not change the size of the feature map are classified as one stage , Therefore, each feature extracted is the output of the last layer of each stage , So you can build a pyramid of features .

A top-down process Yes Characteristics of figure Conduct   On the sampling . Make the feature map after upper sampling have the same size as the feature map of the next layer .

The process of lateral connection between the sides is shown in the following figure .

The result of the up sampling   and   The feature map generated from bottom to top is fused .

Convolution neural network will be generated in   Characteristic diagram of corresponding layer   Conduct 1×1 Convolution operation of ,

Compare it with the past On the sampling Of Feature map fusion , A new feature graph is obtained , This feature map combines features of different layers , With more information .

here 1×1 The purpose of convolution is to change channels, Request and the next layer of channels identical .

After the fusion, it will be used again 3*3 Each fusion result is convoluted , The aim is to eliminate the aliasing effect of up sampling ,

So we get a new feature map . And then iterate layer by layer , We can get many new feature maps .

The result of the generated characteristic graph is P2,P3,P4,P5, And the original bottom-up convolution result C2,C3,C4,C5 One-to-one correspondence . All levels in the pyramid structure share the classification layer （ Regression layer ）.

On the sampling 、 Down sampling

Main purpose

Zoom out the image （ Or subsampling （subsampled） Or downsampling （downsampled））： Match the size of the display area

Zoom in on the image （ Or called upsampling （upsampling） Or image interpolation （interpolating））： Enlarge the original image , Improve the quality

principle

Down sampling ： Images I Size is M*N, On the s Double down sampling , Or get (M/s)*(N/s) The size of the resolution image ,

（s yes M and N The common factor of ）,

If it's a matrix image ,s*s The image in the window becomes a pixel , The value of this pixel is the average value of all pixels in the window ：

On the sampling ： Almost all of the image zooming is done by interpolation , A new element is inserted between pixels using an appropriate interpolation algorithm .

Zoom image （ Down sampling ）、 Zoom in on the image （ On the sampling ）, There are many sampling methods . Such as nearest neighbor interpolation , Bilinear interpolation , Mean interpolation , Median interpolation and other methods

Reference link ： Image upsampling （upsampling） And down sampling （subsampled）_ Yi Dafei's blog -CSDN Blog _ Sampling on image

FPN Characteristic pyramid network -- Interpretation of the thesis _ Happy Dragon Fruit blog -CSDN Blog _ Characteristic pyramid network

FPN（ Characteristic graph pyramid network ） Theoretical basis and concrete implementation _coordinate_blog The blog of -CSDN Blog _fpn structure

Address of thesis ：Feature Pyramid Networks for Object Detection