Icml2022 | sharp maml: model independent meta learning for sharpness perception

Thesis link ：https://arxiv.org/abs/2206.03996

Model has nothing to do with meta learning (Model-agnostic meta learning, MAML) It is one of the mainstream methods of small sample meta learning . Even though MAML It works , But because of MAML Inherent double-layer problem structure , Its optimization is challenging . To be specific ,MAML The loss situation is much more complicated than the empirical risk minimization method , It may contain more saddle points and local minimization points . To meet this challenge , We use the recently invented sharpness perception to minimize (sharp -aware minimization) Method , Propose a sharpness aware MAML Method (Sharp-MAML). Experimental results show that ,Sharp-MAML The performance of its efficient computing variant is better than that of the existing mainstream MAML The benchmark ( for example , stay Mini-Imagenet The accuracy on is +12%). We use the convergence rate to analyze and Sharp-MAML To complement empirical research . As far as we know , This is the first empirical and theoretical study on the minimization of sharpness perception in the context of two-level learning . The code can be in https://github.com/mominabbass/Sharp-MAML On find .