AI operation ch8

1

•	[ Decision tree ]  Based on information gain , Build a decision tree for the following data sets , Describe the process 
 The data needed for a decision-making classification of glasses , Data set containing 4 attribute ：
    age
    astigmatism
    trear-prod-rate For input features ,
    contact-lenses Is a decision attribute .

The first feature

We can consider the following formula
$$G(D,a)=H(D)-\sum _{v=1}^V\frac{|D^v|}{D}H(D^v)$$

$H(D)$ It has been decided when the data is confirmed , So we only need to consider the second half ${\sum }_{v=1}^V\frac{|D^v|}{D}$

Consider three eigenvalues first

1. For age

The eigenvalue	soft	hard	none	sum
young	1	1	1	3
pre-prebyopic	1	1	3	5
prebyopic	0	1	3	4

It is not difficult to get through the formula
$$\begin{array}{rl}age=& -[\frac{3}{12}(\frac{1}{3}lo{g}_2\frac{1}{3}+\frac{1}{3}lo{g}_2\frac{1}{3}+\frac{1}{3}lo{g}_2\frac{1}{3})\\ & +\frac{5}{12}(\frac{1}{5}lo{g}_2\frac{1}{5}+\frac{1}{5}lo{g}_2\frac{1}{5}+\frac{3}{5}lo{g}_2\frac{3}{5})\\ & +\frac{4}{12}(\frac{1}{4}lo{g}_2\frac{1}{4}+\frac{3}{4}lo{g}_2\frac{3}{4})]=1.238\end{array}$$

2. For astigmatism

The eigenvalue	soft	hard	none	sum
yes	0	3	4	7
no	1	1	3	5

Generation into the formula

$$astigmatism=0.979$$

3. Tear production rate

The eigenvalue	soft	hard	none	sum
reduced	0	0	4	4
normal	2	3	3	8

Generation into the formula

$$tear\_production\_rate=1.041$$

So we First take astigmatism You can maximize the function

Second feature

Then consider the remaining features

First be based on Yes situation Input characteristics under

The eigenvalue	soft	hard	none	sum
young	0	1	1	2
pre-prebyopic	0	1	2	3
prebyopic	0	1	1	2
reduced	0	0	2	2
normal	0	3	2	5

$$\begin{array}{rl}age=& -[\frac{2}{7}(\frac{1}{2}lo{g}_2\frac{1}{2}+\frac{1}{2}lo{g}_2\frac{1}{2})\\ & +\frac{3}{7}(\frac{1}{3}lo{g}_2\frac{1}{3}+\frac{1}{3}lo{g}_2\frac{1}{3}+\frac{1}{3}lo{g}_2\frac{1}{3})\\ & +\frac{2}{7}(\frac{1}{2}lo{g}_2\frac{1}{2}+\frac{1}{2}lo{g}_2\frac{1}{2})]=0.965\end{array}$$
$$tear\_production\_rate=0.694$$
** take yes When to choose tear
**
be based on No The situation of

The eigenvalue	soft	hard	none	sum
young	1	0	0	1
pre-prebyopic	1	0	1	2
prebyopic	0	0	2	2
reduced	0	0	2	2
normal	2	0	1	3

$$age=0.4$$
$$tear=0.551$$

take no You should choose age

The following decision tree can be obtained

2.

[ Linear classification ] The following is derived logit function and logistic function Equivalent ：
$$p(X)=\frac{e^{{\beta }_0+{\beta }_{1}X}}{1+e^{{\beta }_0+{\beta }_{1}X}}\frac{p(X)}{1-p(X)}=e^{{\beta }_0+{\beta }_{1}X}$$
Exchange element , Make $f(X)=\frac{p(X)}{1-p(X)},\frac{f(X)}{1-f(X)}=p(X)$

$$\begin{array}{rl}\frac{p(X)}{1-p(X)}=f(X)& =\frac{\frac{e^{{\beta }_0+{\beta }_{1}X}}{1+e^{{\beta }_0+{\beta }_{1}X}}}{1-\frac{e^{{\beta }_0+{\beta }_{1}X}}{1+e^{{\beta }_0+{\beta }_{1}X}}}\\ & \\ & =\frac{e^{{\beta }_0+{\beta }_{1}X}}{1+e^{{\beta }_0+{\beta }_{1}X}-(e^{{\beta }_0+{\beta }_{1}X})}\\ & =e^{{\beta }_0+{\beta }_{1}X}\\ & =\frac{f(X)}{1-f(X)}=p(X)\end{array}$$

To sum up, it is equivalent