Collinearity problem

introduction

Traditional modeling and analysis , When there are many dependent variables entered , Collinearity analysis is required .
What is collinearity analysis ？ It is called multicollinearity in statistics , It is not accurate to say whether there is a high correlation between multiple dependent variables , If there is , Then only some variables can be reserved . For example, analyze precipitation 、 Sunshine duration 、 temperature 、 slope 、 Slope direction 、 Elevation and other dependent variables and vegetation growth （ Such as NDVI Equal independent variable ） The coupling of , It is necessary to perform collinearity analysis on dependent variables , Based on common sense , There is a high correlation between slope and elevation , So two are at the same time with NDVI It is unreasonable to do correlation analysis .

Why is it unreasonable

As mentioned in the introduction, if there is a strong multicollinearity between slope and elevation , The result may be unreasonable . See below for details ：
Let's assume a simplified relational model

If the following relationship exists , among $x_i,i\subset (1....3)$ Is the above slope 、 Elevation and other dependent variables ,$y$ by NDVI Equal independent variable .
$$y=a\cdot x_1+b\cdot x_2+c\cdot x_3（1）$$
We can $a、b、c$ Consider each dependent variable pair NDVI Contribution rate of .
If there is no multicollinearity , Then the above relationship is clear .

however ！

If x1,x2 There is multicollinearity , And x1 And x2 The correlation of 0.9, be x1,x2 The relation of can be simply expressed as
$x_1=A\cdot x_2+B,R^2=0.9$
With a small loss of accuracy , type （1） Can be expressed as
$$y=a\cdot (A\cdot x2+B)+b\cdot x_2+c\cdot x_3=(a\cdot A+b)\cdot x_2+c\cdot x_3+D=d\cdot x_2+c\cdot x_3$$
Then we found out , The model can eliminate redundancy with little loss of accuracy , Besides , We can also see the coefficient $d$ by $(a\cdot A+b)$ The sum of the . that a、b You can take any value , As long as the sum of the two is a constant value d that will do , such as A=1 when ,a=-10,b=20, perhaps a=20,b=-10. At this time, the model results cannot be explained .

How to eliminate the influence of collinearity ？

It's simple , It's to eliminate . Eliminate redundant dependent variables , Before that, we can try to explain the effects of different dependent variables in the model , One of them is preferred . For example, slope 、 Slope direction 、 The elevation has multicollinearity , Then the slope is divided into 、 Slope direction 、 Elevation input model , Choose the one with the best effect .
In addition, some bloggers say that increasing the sample size can eliminate the collinearity effect , It must be good to increase the sample size , But I doubt whether collinearity can be eliminated , You are welcome to provide documentary evidence .

This article is written according to my own understanding , Mistakes and omissions are inevitable , Welcome criticism .

---

Information about collinearity ：https://zhuanlan.zhihu.com/p/88025370
Academician Li Xiaowen talked about ill conditioned inversion , I feel that the idea behind it is essentially consistent with collinearity ：https://blog.sciencenet.cn/blog-2984-20778.html