POSTECH | option compatible reward reverse reinforcement learning

【 title 】Option compatible reward inverse reinforcement learning

【 The author team 】Rakhoon Hwang, Hanjin Lee, Hyung Ju Hwang

【 Date of publication 】2022.1.24

【 Thesis link 】https://www.sciencedirect.com/science/article/pii/S0167865522000241#!

【 Recommended reasons 】 Reinforcement learning in complex environment is a challenging problem . especially , The success of reinforcement learning algorithm depends on the well-designed reward function . Reverse reinforcement learning (IRL) Solved the problem of restoring reward function from expert demonstration . This paper solves the hierarchical inverse reinforcement learning problem within the option framework , Thus, the intrinsic motivation of expert demonstration can be used . The gradient method of parametric options is used to derive Q The definition equation of feature space , So as to obtain the reward feature space . Second order optimal conditions using option parameters , Select the optimal reward function . The experimental results in discrete domain and continuous domain confirm , The reward usage time abstraction recovered in this article provides IRL Solutions to problems , This in turn effectively accelerates the transfer learning task , And this method is robust to the noise contained in the expert demonstration .