[basic knowledge of deep learning - 38] the difference between L1 regularization and L2 regularization

Concept

• L1 Regularization means that the regularization term added to the loss function is the model parameter L1 norm .
• L2 Regularization means that the regularization term added to the loss function is the model parameter L2 norm .

Purpose

• By adding a weight regularization term to the loss function , It can prevent the model from becoming too complex or the parameters of the model from being too large , So as to improve the generalization performance of the model .

L1 Regularization and L2 Difference of regularization

• add L1 Norm is easy to get the coefficient solution of the characteristic , That is, many parameters are 0, add L2 The characteristic solution obtained by norm is relatively smooth , But it can also ensure that the solution is close to 0 More of them . From the perspective of reciprocal ,L1 The norm has a mutation at zero , That is, the minimum point , Therefore, it is easy to optimize to this extreme point .

Bloggers will continue to update some basic knowledge related to in-depth learning, as well as problems and insights encountered in work , Please pay attention if you like 、 give the thumbs-up 、 Collection .