Brief introduction of 228 dropout methods of pytorch and fast implementation of dropblock with 4 lines of code based on dropout

since dropout After the operation is proposed , Played tricks by the great gods . Specific have 8 Kind of , This article first introduces 8 Kind of Dropout Method , The last part is based on dropout use 4 Line code implementation DropBlock. The core principle is just one sentence ：dropout Pixels will be lost randomly , But yes. dropout After up sampling the result of, the pixel will become a pixel block . Based on the data after up sampling dropout result , We can do that DropBlock. In this paper, we realize pytorch Edition and paddle Version of DropBlock.

In this article, we mainly talk about dropout Realization DropBlock The implementation of the , Because bloggers find a lot of implementations in browsing DropBlock The code is from the implementation probability p To keep_ratio transformation , Then based on Bernoulli distribution, we get mask, The code is rather dull . therefore , Based on the framework's own dropout Method , Fast implementation DropBlock（ Core only 4 Line code ）. So , We need to understand dropout Operation mechanism of , Has been based on the baptism of this thought dropout Variant operation .

chart 1 DropBlock stay feature map Schematic diagram

1、Dropout Variant series of

Dropout： Random by pixel 0   In the process of training , Using samples from the Bernoulli distribution , With probability p Randomly zeroing some elements of the input tensor . Origin theory Improving neural networks by preventing co-adaptation of feature detectors [2012.7.3