ICML 2022:ufrgs | optimistic linear support and subsequent features as the basis for optimal strategy transfer

【 title 】Optimistic Linear Support and Successor Features as a Basis for Optimal Policy Transfer

【 The author team 】Lucas N. Alegre, Ana L. C. Bazzan, Bruno C. da Silva

【 Date of publication 】2022.6.22

【 Thesis link 】https://arxiv.org/pdf/2206.11326.pdf

【 Recommended reasons 】 In many real-world applications , Reinforcement learning (RL) An agent may have to solve multiple tasks , Each task is usually modeled by a reward function . If the reward function is linear , And the agent has learned a set of strategies for different tasks , Then you can take advantage of the following features (SFs) To combine these strategies , And find reasonable solutions to new problems . However , The identified solution is not guaranteed to be optimal . This paper introduces a new algorithm to solve this limitation . It allows the RL Agents combine existing strategies , And directly determine the best strategy for any new problem , Without any further interaction with the environment . This paper first proves that the transfer learning problem solved by systemic functional language learners is equivalent to that solved in RL Learning to optimize multi-objective problems . then , In this paper, we introduce an optimistic linear support algorithm based on SF To learn a set of strategies , These strategies are SF Form a convex covering set . Experiments show that this method is superior to the most advanced competitive algorithm in both discrete and continuous fields .