Sarsa notes

The first thing to say is ,sarsa It's also based on TD The algorithm of , He and Q-LEARNING The only difference is that target_policy, Or update Q The way of value , Before Q-learning Is the next... After selecting the action state, At present Q The largest value in the table .
and Sarsa He and Q-Learning The difference is that it is also based on epilson-greedy Select the next action to be executed , And put this action Q Value into the formula to update Q surface .
It's worth noting that , This action was not performed , He is also Q The estimated value in the table , It's just that the selected actions are different , When I watched the Mo fan video, it said sarsa It's action , I thought it was sarsa It's carried out action, Is based on MC Methods , It turned out that I was wrong , This may also be a mistake of my own .
Look at it like this sarsa The algorithm flow should not be too simple ：