Tsinghua University product: penalty gradient norm improves generalization of deep learning model

The structure of neural network is simple , Insufficient training sample size , It will lead to the low classification accuracy of the trained model ; The structure of neural network is complex , Training sample size is too large , It will lead to over fitting of the model , Therefore, how to train neural network to improve the generalization of model is a very core problem in the field of artificial intelligence . I recently read an article related to this problem , In this paper, the author improves the generalization of the deep learning model by adding the constraint of the gradient norm of the regularization term in the loss function . The author expounds and verifies the methods in this paper in detail from two aspects of principle and experiment .L i p s c h i t z \mathrm{Lipschitz}Lipschitz Continuous learning is a very important and common mathematical tool in the theoretical analysis of deep learning , The loss function of neural network is L i p s c h i t z yes \mathrm{Lipschitz} yes Lipschitz Mathematical derivation with continuity as the starting point . In order to facilitate readers to more smoothly appreciate the author's beautiful mathematical proof ideas and processes , This paper supplements the details of mathematical proof that is not carried out in the paper .

————————————————

Copyright notice ： This paper is about CSDN Blogger 「 sorcery 2022」 The original article of , follow CC 4.0 BY-SA Copyright agreement , For reprint, please attach the original source link and this statement .

Link to the original text ：https://blog.csdn.net/qq_38406029/article/details/122851202

Produced by Tsinghua University ： Penalty gradient norm improves generalization of deep learning model _ Welcome to the world of Tao -CSDN Blog .pdf