4、策略學習

Policy Gradient with Baseline

Policy Gradient

• BaseLine
  
  
• Monte Carlo Approximation
  
  
  
• Choices of Baselines
  Choice 1: b=0
  
  **Choice 2:b is state-value **
• b = VΠ（St）

• Policy Network
  
• Value Network
  
• Parameter Sharing
  

Reinforce with Baseline

• Updating the policy network
  
  
  

Advantage Actor-Critic（A2C）

Reinforce versus A2C

兩者網絡結構幾乎一致，價值網絡不同

A2C with Multi-Step TD Target

one tep


Multi step

Reinforce with Baseline

versus