Machine learning 06: Decision Tree Learning

Decision tree learning basics

Basic algorithm of decision tree learning

The above is a recursive call process

Common problems in decision tree learning

The measurement problem of optimal partition

Choose the attribute that can minimize entropy

Combine the image to understand the three indexes

entropy x$\in$[0, 1]：

gini index x$\in$[0, 1]：

Make an auxiliary line and you will find ： The closer to the middle , The longer the red line , It shows that our entropy is larger

Misclassification rate x$\in$[0, 1]：

C4.5 Decision tree algorithm

It can be understood from the image ： The more evenly distributed the attributes ,SplitInfo The bigger it is



Add the following six statistical information gain measurement methods , There is a total 9 Kind of







Cousins can be of different types

Deal with the problem of exact attribute value

Dealing with continuous attribute values

discretization

Leaf node determination problem

How to solve the fitting problem

Classification of samples to be tested

Decision tree learning, understanding and interpretation