Reinforcement learning: from entry to pit to shit

The information in this article comes from the learning video: Reinforcement Learning Method Summary (Reinforcement Learning) _ beep mile _bilibili

1. What is reinforcement learning

Reinforcement Learning (RL) also has many other names, such as Reinforcement Learning, Reinforcement Learning, and Evaluation Learning. It is one of the paradigms and methodologies of machine learning, which is used to describe and solve the interaction process of the agent with the environment., the problem of learning strategies to maximize returns or achieve specific goals.

What we generally call reinforcement learning is actually deep reinforcement learning (Deep Reinforcement Learning DRL). Deep reinforcement learning is the result of the combination of reinforcement learning and deep learning.As the name implies, it is to use deep learning to complete a certain part of traditional reinforcement learning.

The above picture is a classic reinforcement learning structure diagram. As can be seen from the picture, the reinforcement learning process is mainly composed of four parts: agent, observed state (observation/state), reward (reward) andaction.

In the process of continuously interacting with the environment, the agent will retain the experience learned last time. When interacting with the environment in the next round, it will choose behaviors with greater rewards.choose the best behavior through decision-making".

Differences from other machine learning methods

The other machine learning methods here are mainly supervised learning and unsupervised learning, and it is also where we are most prone to confusion in the process of understanding reinforcement learning.

Supervised learning is the most researched method in the field of machine learning, and it is very mature. In the training set of supervised learning, each sample contains a label. Ideally, this label usually refers tocorrect result.The task of supervised learning is to allow the system to infer the appropriate feedback mechanism on the training set according to the label corresponding to each sample, and then to calculate a result as accurate as possible on samples with unknown labels, such as the familiar classificationwith regression problems.In the interaction problem in reinforcement learning, there is no such a universally correct "label", and the agent can only learn from its own experience.

However, reinforcement learning is not the same as unsupervised learning, which is also unlabeled. Unsupervised learning is to discover hidden structures from unlabeled data sets. A typical example is the clustering problem.However, the goal of reinforcement learning is to maximize the reward rather than finding the hidden data set structure. Although using unsupervised learning to find the internal structure of the data can help the reinforcement learning task, it does not fundamentally solve the problem of maximizing the reward..

Therefore, reinforcement learning is the third paradigm of machine learning besides supervised and unsupervised learning.

2. Classification method

Algorithms

Category 1

Probability-based method selection, not necessarily the one with the highest probability (applicable to continuous action values)

A value-based approach selects the highest value action (more firm decision-making) (does not apply to continuous action values)

Category 2

Category 3

Category 4