【Cutout】《Improved Regularization of Convolutional Neural Networks with Cutout》

arXiv-2017

---

---

1 Background and Motivation

With the development of deep learning technology ,CNN Emerging in many computer vision tasks , but increased representational power also comes increased probability of overfitting, leading to poor generalization.

To improve the generalization performance of the model , simulation object occlusion, The author puts forward Cutout Methods of data enhancement ——randomly masking out square regions of input during training,take more of the image context into consideration when making decisions.

This technique encourages the network to better utilize the full context of the image, rather than relying on the presence of a small set of specific visual features（which may not always be present）.

2 Related Work

• Data Augmentation for Images
• Dropout in Convolutional Neural Networks
• Denoising Autoencoders & Context Encoders（self-supervised, Dig out some , Network supplement , To strengthen the characteristics ）

3 Advantages / Contributions

It is proposed in supervised learning Cutout Data enhancement method （dropout A form of , There are similar methods in self-monitoring ）

4 Method

Initial edition ：remove maximally activated features

The final version ： Random center point , The square blocks （ Can be outside the picture , After being intercepted by the image boundary, it is not square ）

It needs to be centralized when using （ That is, subtract the mean ）

the dataset should be normalized about zero so that modified images will not have a large effect on the expected batch statistics.

5 Experiments

5.1 Datasets and Metrics

• CIFAR-10（32x32）
• CIFAR-100（32x32）
• SVHN（Street View House Numbers,32x32）
• STL-10（96x96）
  
  The evaluation index is top1 error

5.2 Experiments

1）CIFAR10 and CIFAR100

Individual experiments are repeated 5 Time ,±x

The following figure explores cutout Different from patch length Influence ,

2）STL-10

3）Analysis of Cutout’s Effect on Activations

introduce cutout After shallow activation, both are improved , Deep level in the tail end of the distribution.

The latter observation illustrates that cutout is indeed encouraging the network to take into account a wider variety of features when making predictions, rather than relying on the presence of a smaller number of features

Then focus on the single sample

6 Conclusion（own） / Future work

• code：https://github.com/uoguelph-mlrg/Cutout

• memory footprint Memory footprint

• Related work introduction drop out when , This sentence appears in the article ：All activations are kept when evaluating the network, but the resulting output is scaled according to the dropout probability

  dropout How to deal with it during the test ？
  

• dropout It works on FC The effect is better than Conv Excellent , The author's explanation is ：1）convolutional layers already have much fewer parameters than fully-connected layers; 2）neighbouring pixels in images share much of the same information（ It's harmless to lose some ）

• cutout—— The continuous area only acts on the input layer dropout technology

  Dropout Technology in ： Visual interpretation and in DNN/CNN/RNN Application in