【Mixed Pooling】《Mixed Pooling for Convolutional Neural Networks》

RSKT-2014
International conference on rough sets and knowledge technology

---

---

1 Background and Motivation

The effect of pooling layer （ Read all the in-depth learning in one article 9 A pooling method ！）

• Increase the network receptive field
• Suppress noise , Reduce information redundancy
• Reduce the amount of model calculation , Reduce the difficulty of network optimization , Prevent over fitting of network
• It makes the model more robust to the change of feature position in the input image

Author for max and ave pooling The shortcomings of ,

Put forward mix pooling——randomly employs the local max pooling and average pooling methods when training CNNs

2 Review of Convolutional Neural Networks

• Convolutional Layer, Including convolution operation and activation function
• Non-linear Transformation Layer, That is to say normalization layer , What's more popular now is BN etc. , It used to be LCN（local contrast normalization） and AlexNet Of LRN（the local response normalization） etc. ,PS: In the paper LCN There is something wrong with the formula ,LRN There is also a gap between the details and the original paper , The form is basically the same
• Feature Pooling Layer

3 Advantages / Contributions

reference dropout, blend max and ave Pooling , Put forward mixed pooling

4 Method

1）mixed pooling The formula

$\lambda$ is a random value being either 0 or 1

2）mixed pooling Back propagation

Have a look first max and ave pooling Back propagation of

max pooling

（ This is from the Internet , Invasion and deletion ！！！）

ave pooling

（ This is from the Internet , Invasion and deletion ！！！）

mixed pooling

I have to record it $\lambda$ The value of , In order to correctly back spread

the pooling history about the random value $\lambda$ in Eq. must be recorded during forward propagation.

3）Pooling at Test Time

Statistics training time pooling use max and ave Frequency of $F_{max}^k$ and $F_{ave}^k$, Whose frequency is high? When testing there pooling Just use who , Start metaphysics, right , Ha ha ha ha

5 Experiments

5.1 Datasets

• CIFAR-10
• CIFAR-100
• SVHN

5.2 Experimental Results

1）CIFAR-10

train error high ,acc high

The author explains This indicates that the proposed mixed pooling outperforms max pooling and average pooling to address the over-fitting problem

Visualization results

It can be seen that mixed pooling Contains more information

2）CIFAR-100

3）SVHN

4）Time Performance

6 Conclusion（own） / Future work

LRN

$k,n,\alpha ,\beta$ It's all super parameters ,$a,b$ Input and output characteristic diagram ,$x,y$ Space location ,$i$ Channel location

The following is from Local response normalization of deep learning LRN(Local Response Normalization) understand

import tensorflow as tf
import numpy as np
x = np.array([i for i in range(1,33)]).reshape([2,2,2,4])
y = tf.nn.lrn(input=x,depth_radius=2,bias=0,alpha=1,beta=1)
 
with tf.Session() as sess:
    print(x)
    print('#############')
    print(y.eval())

---

LCN

《What is the best multi-stage architecture for object recognition?》