Hong Kong Polytechnic University | data efficient reinforcement learning and adaptive optimal perimeter control of network traffic dynamics

【 title 】Data efficient reinforcement learning and adaptive optimal perimeter control of network traffic dynamics

【 The author team 】C. Chen, Y.P. Huang, W.H.K. Lam, T.L. Pan, S.C. Hsu, A. Sumalee, R.X. Zhong

【 Date of publication 】2022.6.28

【 Thesis link 】https://www.sciencedirect.com/sdfe/reader/pii/S0968090X22001929/pdf

【 Recommended reasons 】 The existing data-driven and feedback flow control strategies do not consider the heterogeneity of real-time data measurement . Besides , Traditional traffic control reinforcement learning （RL） Due to the lack of data efficiency , Usually slow convergence . Moreover, the traditional optimal perimeter control scheme needs to accurately understand the system dynamics , Therefore, they are vulnerable to endogenous uncertainty . In this paper, we propose a holistic reinforcement learning (IRL) To learn macro traffic dynamics , To achieve adaptive optimal perimeter control . The main contribution of this paper is ：（a） Continuous time control with discrete gain update is developed , To adapt to discrete-time sensor data .(b) In order to reduce sampling complexity and use available data more effectively , Replay experience (ER) Technology introduction IRL Algorithm .(c) The proposed method is based on “ No model ” The method relaxes the requirements for model calibration , Through data-driven RL The algorithm achieves robustness to modeling uncertainty and improves real-time performance .(d) be based on IRL The convergence of the algorithm and the stability of the controlled traffic dynamics are proved theoretically . The optimal control law is parameterized , Then through neural network (NN) Approaching , This reduces the computational complexity .