The latest 2022 research review of "continuous learning, CL"

In recent years , With the continuous development of information technology , All kinds of data show explosive growth , The traditional machine learning algorithm only when the distribution of test data is similar to that of training data , Learning algorithm can achieve better performance , let me put it another way , They cannot learn continuously and adaptively in a dynamic environment , However , This ability of adaptive learning is a characteristic of any intelligent system . Deep neural networks show the best learning ability in many applications , However , When using this method to perform incremental update learning on data , Will face catastrophic interference or forgetting problems , The model forgets how to solve the old task after learning the new task .  Continuous learning (continual learning, CL) This problem has been alleviated by our research . Continuous learning simulates the learning process of the brain , For continuous non independent identically distributed in a certain order (independently and identically distributed, IID) Stream data for learning , Then the model is updated incrementally according to the execution results of the task . The significance of continuous learning lies in the efficient transformation and use of the learned knowledge to complete the learning of new tasks , And can greatly reduce the problems caused by forgetting . The study of continuous learning is of great significance for intelligent computing systems to adapt to the changing environment .  Based on this , This paper systematically summarizes the research progress of continuous learning , Firstly, the definition of continuous learning is summarized , Introduces non forgetting learning 、 Elastic weight integration and gradient episodic memory 3 A typical continuous learning model , It also introduces the key problems and solutions of continuous learning , And then based on regularization 、 Dynamic structure and memory playback complementary learning system 3 Class continuous learning model is classified and described , At last, it points out the problems to be solved in the further research of continuous learning and the possible development direction in the future .

Address of thesis ：https://crad.ict.ac.cn/CN/10.7544/issn1000-1239.20201058#1