[basic knowledge of deep learning - 50] PCA dimensionality reduction principal component analysis

PCA The principle of dimension reduction

• PCA High dimensional variables that may have correlation can be synthesized into low dimensional variables that are linearly independent , It's called the main component （ principal components）. The new low dimensional data set preserves the variables of the original data as much as possible .
• The way of dimensionality reduction is to analyze the principal components of data , Without losing too much information , Through mapping, high-dimensional data is projected into lower latitude data .

Calculation of principal components

• The principal component of the matrix is the eigenvector of its covariance matrix , Sorted according to the corresponding eigenvalue size . The largest eigenvalue is the first principal component , The second largest eigenvalue is the second principal component , And so on .

Variance and covariance

• variance ： Used to measure the dispersion of a set of data , Is the mean of the square of the difference between each sample and the sample mean . The formula is as follows ：
  $$s^2=\frac{{\sum }_{i=1}^n(X_i-\overline{X})}{n-1}$$
• covariance ： Measure the degree of linear correlation between two variables . If the covariance is zero , It means that the two are linearly independent （ Not completely independent , Just no linear correlation ） Covariance greater than zero means that one variable increases and the other increases , That is, positive correlation . Covariance less than zero means that one variable increases and the other decreases , Negative correlation .
  $$conv(X,Y)=\frac{{\sum }_{i=1}^n(X_i-\overline{X})(Y_i-\overline{Y})}{n-1}$$
• Covariance matrix ： It consists of the covariance of two variables in the data set . In matrix $(i,j)$ The first element in the data set is $i$ And the $j$ Covariance of elements .

Eigenvectors and eigenvalues

• The eigenvector is equivalent to the coordinate axis , Eigenvalues are equivalent to coordinates .
• The eigenvector is a non-zero vector obtained by satisfying the following matrix ：
  $$A\vec{v}=\lambda \vec{v}$$
• among $A$ It's a matrix , $\vec{v}$ It's the eigenvector ,$\lambda$ It's characteristic value .

PCA The role of dimension reduction

Dimensionality reduction is committed to solving three types of problems .

1. Dimensionality reduction can alleviate the problem of dimensionality disaster ;
2. Dimensionality reduction can compress data while minimizing information loss ;
3. It's difficult to understand the structure of hundreds of dimensions , The data of two or three dimensions is easier to understand through visualization .

Bloggers will continue to update some basic knowledge related to in-depth learning, as well as problems and insights encountered in work , Please pay attention if you like 、 give the thumbs-up 、 Collection .