A survey on model compression for natural language processing (NLP model compression overview)

author ：Canwen Xu, Julian McAuley

brief introduction ： With Transformer  And pre training technology , natural language processing （NLP） Great progress has been made in the application of . However ,Transformer High energy consumption and long reasoning delay hinder NLP Into a broader scene , Including edge and mobile computing . Effective NLP The purpose of the study is to comprehensively consider the calculation , The whole life cycle of time and carbon emissions NLP, Including data preparation , Model training and reasoning . In this review , The author focuses on the reasoning stage , And review NLP Current situation of model compression , Including benchmark 、 Indicators and methods , The last author also The current obstacles and future research directions are summarized .

Paper download ：https://arxiv.org/pdf/2202.07105