LM NAV: robot navigation method based on large models of language, vision and behavior

【 Abstract 】 It can train the target condition strategy of robot navigation 、 Uncommented dataset , Provide a good generalization of real-world settings . However , Especially in the vision based setting of specifying the image required by the target , This will cause the interface to be unnatural . Language provides more convenient ways to communicate with robots , But modern methods often require expensive supervision , Describe in formal language with annotated tracks . We propose a system ,LM-Nav, For robot navigation , Enjoy the benefits of training on large uncommented track data sets , At the same time, it still provides users with an advanced interface . Instead of using tags according to the instructions of the dataset , We show that such a system can be completely controlled by a pre trained navigation model (ViNG)、 Image Language Association (CLIP) And language modeling (GPT-3) structure , Robot data without any fine tuning or language annotation . We instantiate on real-world mobile devices LM-Nav robot , And demonstrate long-range navigation in a complex outdoor environment according to natural language instructions .