University of Toronto: Anthony coach | the conditions of deep reinforcement learning can induce dynamic risk measurement

【 title 】Conditionally Elicitable Dynamic Risk Measures for Deep Reinforcement Learning

【 The author team 】Anthony Coache, Sebastian Jaimungal, Álvaro Cartea

【 Date of publication 】2022.6.29

【 Thesis link 】https://arxiv.org/pdf/2206.14666.pdf

【 Recommended reasons 】 This paper proposes a new framework to solve the problem of risk sensitive reinforcement learning （RL） problem , The agent optimizes the time consistent dynamic spectrum risk measurement . Based on the concept of conditional inducement , This method constructs （ Strictly consistent ） Scoring function , Used as a penalty factor in the estimation process . There are three main contributions ：（i） Designed an effective method , Using depth neural network to estimate a class of dynamic spectral risk measures ,（ii） It is proved that these dynamic spectral risk measures can be approximated to any arbitrary accuracy by deep neural network , as well as （iii） Develop a risk sensitive actor - Critic algorithm , The algorithm uses a complete series and does not require any additional nested transformations . The conceptually improved reinforcement learning algorithm is compared with the nested simulation method , And its performance is explained in two cases ： Statistical arbitrage and portfolio allocation on simulated and real data .