当前位置:网站首页>【Mixed Pooling】《Mixed Pooling for Convolutional Neural Networks》
【Mixed Pooling】《Mixed Pooling for Convolutional Neural Networks》
2022-07-02 06:26:00 【bryant_meng】
RSKT-2014
International conference on rough sets and knowledge technology
文章目录
1 Background and Motivation
池化层的作用(一文看尽深度学习中的9种池化方法!)
- 增大网络感受野
- 抑制噪声,降低信息冗余
- 降低模型计算量,降低网络优化难度,防止网络过拟合
- 使模型对输入图像中的特征位置变化更加鲁棒
作者针对 max 和 ave pooling 的缺点,
提出了 mix pooling——randomly employs the local max pooling and average pooling methods when training CNNs
2 Review of Convolutional Neural Networks
- Convolutional Layer,包括卷积操作和 activation function
- Non-linear Transformation Layer,也即 normalization 层,现在比较流行的是 BN 等,以前的是 LCN(local contrast normalization) 和 AlexNet 的 LRN(the local response normalization) 等,PS:论文中 LCN 的公式感觉有问题,LRN 细节也原版论文也有差距,形式基本一致
- Feature Pooling Layer
3 Advantages / Contributions
借鉴 dropout, 混合max 和 ave 池化,提出 mixed pooling
4 Method
1)mixed pooling 公式
λ \lambda λ is a random value being either 0 or 1
2)mixed pooling 反向传播
先看看 max 和 ave pooling 的反向传播
max pooling
(图片来源于网络,侵删!!!)
ave pooling
(图片来源于网络,侵删!!!)
mixed pooling
得记录下 λ \lambda λ 的取值,才能正确反向传播
the pooling history about the random value λ \lambda λ in Eq. must be recorded during forward propagation.
3)Pooling at Test Time
统计训练时某次 pooling 采用 max 和 ave 的频次 F m a x k F_{max}^{k} Fmaxk 和 F a v e k F_{ave}^{k} Favek,谁的频次高测试的时候该处的 pooling 就用谁,开始玄学了是吧,哈哈哈哈
5 Experiments
5.1 Datasets
- CIFAR-10
- CIFAR-100
- SVHN
5.2 Experimental Results
1)CIFAR-10
train error 高,acc 高
作者解释 This indicates that the proposed mixed pooling outperforms max pooling and average pooling to address the over-fitting problem
可视化结果
可以看出 mixed pooling 包含更多的信息
2)CIFAR-100
3)SVHN
4)Time Performance
6 Conclusion(own) / Future work
LRN
k , n , α , β k, n, \alpha, \beta k,n,α,β 都是超参数, a , b a,b a,b 输入输出特征图, x , y x,y x,y 空间位置, i i i 通道位置
以下内容来自 深度学习的局部响应归一化LRN(Local Response Normalization)理解
import tensorflow as tf
import numpy as np
x = np.array([i for i in range(1,33)]).reshape([2,2,2,4])
y = tf.nn.lrn(input=x,depth_radius=2,bias=0,alpha=1,beta=1)
with tf.Session() as sess:
print(x)
print('#############')
print(y.eval())
LCN
《What is the best multi-stage architecture for object recognition?》
边栏推荐
- 使用Matlab实现:Jacobi、Gauss-Seidel迭代
- @Transitional step pit
- 【信息检索导论】第六章 词项权重及向量空间模型
- 【Torch】最简洁logging使用指南
- Calculate the difference in days, months, and years between two dates in PHP
- 使用Matlab实现:幂法、反幂法(原点位移)
- [medical] participants to medical ontologies: Content Selection for Clinical Abstract Summarization
- Generate random 6-bit invitation code in PHP
- ABM论文翻译
- 程序的执行
猜你喜欢
ModuleNotFoundError: No module named ‘pytest‘
使用百度网盘上传数据到服务器上
程序的内存模型
[medical] participants to medical ontologies: Content Selection for Clinical Abstract Summarization
Faster-ILOD、maskrcnn_ Benchmark trains its own VOC data set and problem summary
自然辩证辨析题整理
使用MAME32K进行联机游戏
ABM thesis translation
@Transitional step pit
【信息检索导论】第二章 词项词典与倒排记录表
随机推荐
Using MATLAB to realize: Jacobi, Gauss Seidel iteration
Drawing mechanism of view (3)
【AutoAugment】《AutoAugment:Learning Augmentation Policies from Data》
【Paper Reading】
Get the uppercase initials of Chinese Pinyin in PHP
【Wing Loss】《Wing Loss for Robust Facial Landmark Localisation with Convolutional Neural Networks》
华为机试题-20190417
The difference and understanding between generative model and discriminant model
Faster-ILOD、maskrcnn_ Benchmark trains its own VOC data set and problem summary
Common CNN network innovations
程序的内存模型
Interpretation of ernie1.0 and ernie2.0 papers
【Mixup】《Mixup:Beyond Empirical Risk Minimization》
Faster-ILOD、maskrcnn_benchmark训练coco数据集及问题汇总
CPU的寄存器
Generate random 6-bit invitation code in PHP
【MEDICAL】Attend to Medical Ontologies: Content Selection for Clinical Abstractive Summarization
[introduction to information retrieval] Chapter 7 scoring calculation in search system
Proof and understanding of pointnet principle
【信息检索导论】第七章搜索系统中的评分计算