[linear algebra] understand eigenvalues and eigenvectors

1 Popular explanation

   Definition ： about Any reversible square matrix , There is a vector , Multiply the matrix by the vector , The size of the vector changes but the direction does not change . in other words , about $n\times n$ matrix $M$, There is a non $0$ Of $n$ Dimension vector $V_1,V_2,......V_n$ Let's set up the following formula ：
$$M{V}_i={\lambda }_i{V}_i$$
among , ratio ${\lambda }_i$ Become a matrix $M$ The eigenvalues of the , vector $V_i$ Become the eigenvector corresponding to the eigenvalue .
For a reversible square matrix, there can be a set of eigenvalues and eigenvectors . Simplify the above formula to ：
$$Ax=\lambda x$$
among $A$ It's a matrix $x$ Value eigenvector $\lambda$ It's characteristic value . The eigenvalue is a number , And vector $\lambda x$ Number multiplication is essentially the scaling of a vector . Such as $\lambda =2$, $x=[2,3{]}^T$, be $\lambda x=[4,6{]}^T$. The transformed vector is compared with the original vector $x$ The size of the has doubled and the direction has not changed . And since the above two formulas are equal to each other , Therefore, the effect of multiplying a matrix by a vector is to make the vector stretch in a constant direction .【 reference 1】
So the popular explanation of eigenvalues and eigenvectors is :

1. A matrix is a transformation of a vector .
2. An eigenvector is a vector whose direction is invariant after a matrix transformation .
3. The eigenvalue $\lambda$ Is a multiple of expansion .

Here we add the properties of eigenvalues and eigenvectors ：
The eigenvalue ：$A$ yes $n$ Order matrix ${\lambda }_1,{\lambda }_2,{\lambda }_3......{\lambda }_n$ yes $A$ Of $n$ Eigenvalues have ：
$$\begin{gathered} \sum _i^n{\lambda }_i={\lambda }_1+{\lambda }_2+{\lambda }_3+......+{\lambda }_n=a_{11}+a_{22}+a_{33}+......+a_{nn}=tr(A) \\ \prod _{i=1}^n{\lambda }_1{\lambda }_2......{\lambda }_n=|A| \end{gathered}$$
Eigenvector :$n$ Order matrix $A$ Unequal eigenvalues of ${\lambda }_1,{\lambda }_2,{\lambda }_3......{\lambda }_n$ The corresponding eigenvectors $x_1,x_2,......,x_n$ Linearly independent . Be careful ： The eigenvectors of symmetric matrix with unequal eigenvalues are orthogonal .

2 The matrix is understood from the perspective of motion

   If you have read the matrix （ One ）（ Two ）（ 3、 ... and ） Series of articles , Eigenvalues and eigenvectors can be understood from the perspective of transformation . With $Ma=b$ Introduce matrix as an example $M$ The meaning of ：

• From the perspective of transformation , matrix $M$ It can be understood as a pair of vectors $a$ Make a transformation and get $b$
• From the point of view of the coordinate system ,$M$ It can be understood as a coordinate system （ Commonly used coordinates are Cartesian coordinates , namely $I$）, vector $a$ Is in the $M$ The coordinates in this coordinate system ,$a$ Corresponding to $I$ The coordinates in the coordinate system are vectors $b$.

   What do eigenvalues and eigenvectors mean ？
   Let's assume the matrix $A$ A characteristic value of is $m_1$, The corresponding eigenvector is $x_1$. According to the definition and the above understanding of the matrix, we can know ,$x_1$ In order to $A$ Is the coordinate vector of the coordinate system , Transform it to with $I$ Is the coordinate vector obtained after the coordinate system And Its original coordinate vector There will always be one $m_1$ The scaling relation of times .
   For convenience of understanding, give a simple example , If the matrix $A$ as follows , You can see that its characteristic values are $2$ individual , Namely $1,100$, They correspond to each other $2$ A special eigenvector , namely $[1,0],[0,1]$.
$$A=\left[\begin{array}{cc}1& 0\\ 0& 100\end{array}\right]$$
   So the matrix $A$ Multiply left by any vector $x$, In fact, it can be understood as a vector $x$ Along this $2$ The direction of the two eigenvectors is expanded , The scaling ratio is the corresponding eigenvalue . You can see this $2$ The difference between the two eigenvalues is very large , The smallest is $1$, The largest eigenvalue is $100$.

The picture is from 【 reference 3】

3 The meaning of eigenvalues and eigenvectors

   The point is that if we know the magnitude of the eigenvalue , Sometimes in order to reduce the calculation , We can keep only those with large eigenvalues , For example, in the picture above , We can see the transformed vector $x$ The shaft fits the same , and $y$ The axis direction is stretched $100$ times , So usually in order to implement the compression algorithm , We can just keep $y$ The axis direction can be changed . It is similar to the high-dimensional case , Multidimensional matrices stretch vectors in multiple directions , Some directions may stretch very little , And some are big , We only need to keep a large range to achieve the purpose of compression .【 reference 3】

4 Understand from a computational point of view

   for instance ： matrix $A$ The eigenvalue of is $2,1$, The eigenvector is $[1,1{]}^{T}and[2,3{]}^T$.
$$A=\left[\begin{array}{ll}4& -2\\ 3& -1\end{array}\right]$$
Suppose there is a vector $x=[1,2{]}^T,bey=Axby[0,1{]}^T$. The following uses another method to calculate ： First of all, will $x$ Expressed as a linear combination of eigenvectors
$$x=\left(\begin{array}{l}1\\ 2\end{array}\right)=-1\ast \left(\begin{array}{l}1\\ 1\end{array}\right)+1\ast \left(\begin{array}{l}2\\ 3\end{array}\right)$$
then , Multiply the eigenvalue by the corresponding coefficient , obtain :
$$y=-1\ast 2\ast \left(\begin{array}{l}1\\ 1\end{array}\right)+1\ast 1\ast \left(\begin{array}{l}2\\ 3\end{array}\right)=-2\ast \left(\begin{array}{l}1\\ 1\end{array}\right)+1\ast \left(\begin{array}{l}2\\ 3\end{array}\right)$$
obviously $y=[0,1{]}^T$.（ Understand well ）
   So far , Let's summarize the previous conclusion again ：

• Matrix multiplication can be understood as the transformation of the coordinate system of the corresponding vector （ Understand from the perspective of coordinate system ）
• From the properties of eigenvectors , A set of eigenvectors corresponding to a matrix is linearly independent , So it can be used as a group The base .

   The key is coming. , From the above calculation, we can see that , The result of a matrix left multiplying by a vector is equivalent to the representation of the corresponding vector by the expansion of the linear combination of the eigenvectors of the matrix （ Understand this sentence well ）. That is to say, the corresponding vector expands and contracts in the coordinate system based on the matrix eigenvector . It can also be understood as , The mapping that the matrix acts as , It's actually scaling the eigenvector , The scaling degree of each eigenvector is the eigenvalue .【 reference 5】 It can be understood that eigenvalues and eigenvectors are attributes of the matrix itself .

5 Understand other conclusions

5.1 Diagonalization decomposition

reference 【5】 I don't understand it very thoroughly , When you understand it, you can add

6 reference

[1] A popular explanation of eigenvalues and eigenvectors
[2]Python Calculate eigenvalues and eigenvectors
[3] What are eigenvalues and eigenvectors ？ What can it do ？
[4] How to understand matrix eigenvalues and eigenvectors ？
[5] Understanding of eigenvalues and eigenvectors simple There must be a harvest
[6] Symmetric matrix The eigenvectors are orthogonal