Large language models teach agents to evolve. Openai reveals the complementary relationship between the two

In a recent paper , come from OpenAI Of researchers have explored the possibility of complementary models . They studied the potential significance of large language models in gene programming and openness , Found a synergy between the two .

Thesis link ： https://arxiv.org/abs/2206.08896

especially , This new big model evolves （ELM） In the method , A large language model trained in code （LLM） Can provide intelligent 「 variation 」, So as to significantly promote more efficient mutation operators , Avoided many of the challenging evolutionary processes that existed before . Interestingly ,ELM The benefits of this in turn affect deep learning ： adopt LLM The generated sample set can eventually form a new training set in a new field , And then fine tune LLM Can get good performance in new fields , This is a new data generation process . Besides , This approach is ultimately enhanced only by self generated data LLM Generation capacity of , Thus opening up new opportunities in the pursuit of openness .

Nearly period of time ,LLM A series of achievements have been made in automatic code generation , These models derive guidance from human knowledge , By learning from very large data sets , To achieve universal programming capabilities .

in fact , This bootstrap （bootstrapping） Possibility and GP It's related. , After all GP It is actually a generative programming method . Although at first glance , LLM May surpass or contain GP, But actually GP Search for specific categories of programs that are targeted away from LLM Training distribution ( Even a total lack of ) Still have an advantage in the case of . under these circumstances ,LLM Provide limited dependencies ( Learn about a whole new field prompt Engineering is very difficult ) , and GP In principle, it can evolve in any space ( Although in practice , The amount of variation required to adaptively obtain a consistent signal , Some spaces can be difficult to handle ).

Interestingly , The best combination of the two is easy to achieve ： Just hint LLM Make a difference ,LLM It can be embedded into a global evolutionary algorithm as a highly complex mutation operator . thus , Whether it is the evolution of conventional mutation operators or LLM The evolution of itself , Can not produce any result close to the solution space , however LLM Combining with evolution can lead them to the correct region of the solution space .

actually , Using a LLM Perturbed program evolution is bridging the gap between evolutionary algorithms and those that run at the level of human thought . in other words ,LLM Training can be used to estimate how humans consciously change programs , Keep on top of multiple functions at the same time . Besides , In this way LLM It can further fine tune the purpose of successful variation , Make self-improvement , Finally, iterate over a new technology to enhance ELM Performance of .

In short , The main contributions of this paper include ：

1. Put forward to pass LLM Of the highly efficient evolutionary program ELM Method ;
2. A method based on LLM Mutation operator to improve ELM Technology for searching capabilities over time ;
3. It shows LLM In areas not covered by training data ELM;
4. The verification passed ELM The generated data can lead to enhanced LLM, This provides a new way to openness .