The contribution of the article can be summarized as the following points ：
（1） The existing methods can increase the convolution kernel to 31x31. But on a larger convolution , for example 51x51 and 61x61, Obvious drop points begin to appear .
（2） classical CNN The Internet , Such as ResNet and ConvNeXt,stem cell Have adopted 4× Downsampling . So for typical 224×224 ImageNet Come on ,51×51 The extreme kernel of has been roughly equal to the global convolution . The same as the global attention mechanism , I speculate that global convolution also has the problem of insufficient ability to capture local low-level features .
（3） Based on this observation , I propose a set of methods for training extreme convolution kernels recipe, It can increase the convolution kernel to 61x61, And further improve the performance of the model . My methodology is mainly based on the widespread sparsity in human visual system . At the micro level , I will 1 A square large convolution kernel is decomposed into 2 With dynamic sparse structure , Parallel rectangular convolution kernel , Used to improve the scalability of large convolution ; At the macro level , I built a pure sparse network , It can maintain the same parameters and... As dense networks while improving network capacity FLOPs.
（4） According to this recipe, I have constructed a new network structure Sparse Large Kernel Network, abbreviation SLaK.SLaK Carrying the largest 51x51 Convolution kernel , Can be used in similar parameter quantities and FLOPs Under the condition of , Get more advanced than the latest ConvNeXt,Swin Transformer and RepLKNet Better performance .
（5） Last , The author believes that the most important contribution of this paper is sparsity, Usually compressed as a model “ old man ”,can be a promising tool to boost neural network scaling.