Relevant knowledge of information entropy

Information entropy related knowledge

Information entropy ： Describe the uncertainty of the information source , It can be considered as the order of information , More orderly information , The smaller the information entropy , The more disordered the information is , Then the greater the information entropy ,（ The average uncertainty of the source should be the uncertainty of a single symbol -logPi The statistical average of , It is called information entropy ）

The relationship between probability and information redundancy is clarified

Information redundancy ： Information surplus , The difference between the maximum amount of information and the actual amount of information , Source coding is to reduce the information redundancy in the signal , The purpose is to improve the effectiveness of the communication system ; Channel coding is to improve the information redundancy in the signal , The purpose is to improve the reliability of the communication system .

Source coding uses finite length bit, To transmit the most information , The channel coding is to strengthen the transmission channel bit Transmit to the receiving end , A more correct code was received .

in summary , You can know , When increasing relevance , Delete the incorrect information , Leave something that is highly relevant , Information entropy is used to filter data to encode the channel , In this way, the information transmitted will be more accurate at the receiver , Data with high correlation should be consolidated , Delete the data with small correlation , Forecast data can be more accurate .

3 Rules ： A set of data subject to normal distribution , In less than -3 Or greater than 3 when , For outliers , Delete outliers ,

Median elimination ： Sort a set of data by size , Find the median , And keep (0.8middle_value-1.2middle_value) The data of , Eliminate other outliers , This method is applicable to big data values with little data change .