Naacl-22 | introduce the setting of migration learning on the prompt based text generation task

The pre training language model has made significant progress in text generation tasks through fine-tuning , But in the scenario of sparse data , It is usually impossible to fine tune directly . therefore , In this paper, based on prompt Setting of transfer learning . The author first learns one for different tasks in the source domain prompt, Thus construct prompt pool , Then migrate in the target task . In order to consider both task level and instance level information , The author designed an adaptive attention mechanism , For each instance sample in the target task , The model will select the most relevant source task for it prompt. The author has carried out experiments on various generation tasks and data sets , The results show that the migration method proposed by the author can improve the generation effect on the target task very well .

Paper title ：

Learning to Transfer Prompts for Text Generation    

Thesis link : 

https://arxiv.org/abs/2205.01543