The basic steps of using information theory to deal with scientific problems are

1、 Build state space ;

There are many ways to build state space ; etc. count、 Equal width ,...and so on;

2、 Calculate the probability of different states ;

With frequency , Approximate substitution probability , Therefore, the number of experiments should be as much as possible ;

3、 Calculation entropy 、 Or mutual information ;

------

Understanding of basic concepts ：

1、 Entropy is The average number of problems needed to determine the status ;

2、 Two meanings of information ：

（1）、 Reduction of uncertainty ;

（2）、 Between two probability distributions KL The divergence ;

3、 What is mutual information ,

         You know ： A measure of the degree of interdependence between random variables .

        I understand it ： How much information does a variable carry about other variables ？

        Or say , The interaction between two variables , How big is the ？

4、 Conditional mutual information , The existence of the third variable , Mutual information between the first two variables can be increased ;