International School of Digital Economics, South China Institute of technology 𞓜 unified Bert for few shot natural language understanding

author ：JunYu Lu, Ping Yang, JiaXing Zhang, etc.

brief introduction ： This paper studies the pre training model Bert In language understanding (NLU) The unified framework of the field and the practice under the small sample scenario . Even if the pre - trained language model shares a semantic coder , Natural language understanding is also influenced by many output modes . In this paper , The author puts forward a method based on BERT A unified two-way language understanding model UBERT frame , We can use the double affine network to solve different problems NLU General modeling of the training object of the task . say concretely ,UBERT Encode prior knowledge from all aspects , Across multiple NLU Task unified construction learning representation , It helps to enhance the ability to capture common semantic understanding . Use double affine to model the fractional pairs of the starting and ending positions of the original text , Various classification and extraction structures can be transformed into general span decoding methods . Experiments show that ：UBERT stay 7 individual NLU Mission 、14 The most advanced performance is achieved on the small sample and zero sample settings of data sets , It also realizes the unification of extensive information extraction and language reasoning tasks .

 

 

Paper download ：https://arxiv.org/pdf/2206.12094.pdf