Surpass palm! Peking University Master proposed diverse to comprehensively refresh the NLP reasoning ranking

lately , Researchers from Peking University and Microsoft based on self consistent new methods DiVeRSe, There are three main innovations , It further improves the reasoning ability of the model .

Thesis link ：https://arxiv.org/abs/2206.02336

Code link ：https://github.com/microsoft/DiVeRSe

 

First of all , Be self consistent 「 Different ideas , The answer is the same 」 Inspired by the , That is, sampling different reasoning paths from the language model ,DiVeRSe Go further in diversity , according to 「 All roads lead to Rome 」 Idea , The use of multiple prompt Generate answers , It can generate more complete 、 Complementary answers .

 

second , When generating reasoning paths , There is no mechanism in the language model to correct errors in previous steps , It may cause confusion in the final prediction results .DiVeRSe reference verifier Thought , Verify the correctness of each reasoning path to guide the voting mechanism . in other words , Not all reasoning mechanisms are equally important or good .

 

Third , Because the answer is based on multi-step reasoning , When a path generates a correct answer , It can be considered that all the steps have contributed to the final correctness . However , When a wrong answer is generated , This does not mean that all steps are wrong or contribute to the error .