University of Oslo: Li Meng | deep reinforcement learning based on swing transformer

【 title 】Deep Reinforcement Learning with Swin Transformer

【 The author team 】Li Meng, Morten Goodwin, Anis Yazidi, Paal Engelstad

【 Date of publication 】2022.6.30

【 Thesis link 】https://arxiv.org/pdf/2206.15269.pdf

【 Recommended reasons 】Transformers It is a neural network model using multilayer self attention head . Focus on transformers Achieve in “key” and “query” Context embedding .Transformers In recent years, it has shown excellent performance in natural language processing tasks .Swin-Transformer Divide the image pixels into small pieces , And in a fixed size （ displacement ） Apply local self attention operation in the window . Decision converter has been successfully applied to offline reinforcement learning , And indicate from Atari The random walk sample of the game is enough for agents to learn to optimize behavior . However, online reinforcement learning and transformers Combined, it is more challenging . This paper discusses reinforcement learning strategies without modification , Instead, only the convolutional neural network architecture is replaced by Swin Transformer The possibility of self attention architecture . The goal is to change the way agents view the world , Not the way agents plan the world . And in the arcade learning environment 49 Experiments were carried out on three games . It turns out that , Use in reinforcement learning Swin Transformer Significantly higher evaluation scores were achieved in most games in the arcade learning environment . Therefore, online reinforcement learning can benefit from self attention embedded by spatial tokens .