Variance and covariance

Covariance matrix can be seen everywhere in statistics and machine learning , generally speaking , It can be regarded as variance and covariance Two parts , That is, the variance constitutes the elements on the diagonal , Covariance constitutes the non diagonal element . This paper aims to introduce the well-known covariance matrix from a geometric point of view .

1. Definition of variance and covariance

In statistics , Variance is used to measure the dispersion of a single random variable , Covariance is generally used to describe the similarity of two random variables , among , The calculation formula of variance is as follows ：

among , Represents the sample size , Symbol Represents the mean value of the observed sample , This definition has been in contact since junior middle school .

On this basis , covariance The calculation formula of is defined as :

In the formula , Symbol and Respectively represent the mean value of the observation sample corresponding to the two random variables , Accordingly , We found that ： variance It can be regarded as a random variable About its own covariance .

Baidu Encyclopedia explains it as follows ：

Covariance represents Two The overall error of variables , This is different from the variance that represents only one variable error .

• If the trends of the two variables are the same , That is, if one of them is greater than its own expectations , The other is also greater than one's own expectations , Then the covariance between the two variables is positive .
• If the change trend of two variables is opposite , That is, one of them is greater than its own expectation , The other is less than its own expectations , Then the covariance between the two variables is negative .