Deep learning classification network -- Network in network

Deep learning classification network Summary

1. AlexNet
2. VGGNet
3. Network in Network

---

Preface

Network in Network It puts forward two important ideas , Respectively ：

• 1×1 Convolution realizes cross channel information interaction , Add nonlinear expression
• Combined with enhanced local modeling , You can replace the full connection layer with global average pooling , More explanatory , Reduce over fitting

These two points were later Inception Series network 、ResNet and CAM Visualization technology .

1. Network structure

The network structure is relatively simple , By three mlpconv Layer and a global average pool layer . So-called mlpconv When it is implemented 3×3 Convolution + Two 1×1 Convolution .

2. The main points of

2.1 mlpconv

ZFNet The visualization technology used in this paper reveals some potential concepts corresponding to the activation value in the characteristic graph （ Category information ）, The higher the activation value , Corresponding input patch The greater the probability of containing potential concepts . When the underlying concept is linearly separable , Traditional linear convolution + The nonlinear activation mode can identify , however The confidence level of the potential concept is usually a highly nonlinear function of the input . therefore , The use of micro network Instead of linear convolution + Nonlinear activation , To enhance local potential conceptual modeling . The author uses multi-layer perceptron （MLP） As micro network Instantiation , because MLP It is a general function approximator and a neural network that can be trained by back propagation , be based on MLP The establishment of a micro network Was named mlpconv.

Here's a quote This blog A picture in , Be clear at a glance , I think it can explain clearly Mlpconv layer .

2.2 Global average pooling

stay NIN The traditional full connection layer is not used for classification , Instead, the last one is output directly through the global average pooling layer mlpconv The average value of the feature map of the layer is used as the confidence of the category , Then input the obtained vector softmax layer . There are two advantages to doing so ：

• More interpretable ： The category information is directly obtained from the global average pool of characteristic graphs , Strengthen the correspondence between feature mapping and categories ;(CAM Visualization technique This idea is used for reference )
• Avoid overfitting ： Global average pooling itself is a structural regularizer , It can prevent over fitting of the overall structure .

Reference material

[1] Lin M, Chen Q, Yan S. Network in network[J]. arXiv preprint arXiv:1312.4400, 2013.
[2] 【 Intensive reading 】Network In Network（1*1 Convolution layer instead of FC layer global average pooling）