Principal component analysis (matlab)

  This article draws on mathematical modeling Teacher Qingfeng's courseware and ideas , You can click the view link to view the video explanation of teacher Qingfeng ： Qingfeng mathematical modeling ：https://www.bilibili.com/video/BV1DW411s7wi

  One 、 Preface

 

  Two 、 Principal component analysis thought

 

3、 ... and 、 Calculation steps of principal component analysis

The calculation steps can be put into the paper .

step1: Data pair x Standardize to X

 step2: Calculate the sample covariance matrix （ correlation matrix ）

The correlation coefficient matrix can be beautified and put into the paper , See video for details 17：40 .

notes ： The first and second steps are MATLAB It can be achieved directly in one step . 

  step3: Calculation R Eigenvalues and eigenvectors

   step4： Calculate the principal component contribution rate and cumulative contribution rate

    step5： Write the principal components , Take the cumulative contribution rate of more than 80% The principal component corresponding to the eigenvalue of

     step6：  Analyze the significance of principal component representation according to the coefficient

      step7：   Use the results of principal components for subsequent analysis

  Cannot be used to evaluate model reasons ： Dimensionality reduction is to reduce the indicators with data , Since there is data, why not Topsis？

Four 、 Example explanation

  This question goes directly to the second step , Not in practice .

 

  5、 ... and 、 Principal component analysis program （MATLAB）

This data is clustering data

clear;clc
 load data1.mat   %  Principal component clustering 
%  load data2.mat   %  Principal component regression 

%  Be careful , Here you can make descriptive statistics on the data first 
%  For the content of descriptive statistics, see page 5 speak . The correlation coefficient 
[n,p] = size(x);  % n It's the number of samples ,p Is the number of indicators 

%%  First step ： Data pair x Standardize to X
X=zscore(x);   % matlab Built in standardized functions （x-mean(x)）/std(x)

%%  The second step ： Calculate the sample covariance matrix 
R = cov(X);

%%  Be careful ： The above two steps can be combined into the next step ： Directly calculate the sample correlation coefficient matrix 
R = corrcoef(x);
disp(' The sample correlation coefficient matrix is ：')
disp(R)

%%  The third step ： Calculation R Eigenvalues and eigenvectors 
%  Be careful ：R It's a positive semidefinite matrix , So its eigenvalue is not negative 
% R It's also a symmetric matrix ,Matlab When calculating the symmetric matrix , The eigenvalues will be arranged from small to large 
% eig See the video of the first lecture on analytic hierarchy process for a detailed explanation of the function 
[V,D] = eig(R);  % V  Eigenvector matrix   D  Diagonal matrix composed of eigenvalues 


%%  Step four ： Calculate the principal component contribution rate and cumulative contribution rate 
lambda = diag(D);  % diag The function is used to get the value of the main diagonal element of a matrix ( What is returned is the column vector )
lambda = lambda(end:-1:1);  %  because lambda Vectors are sorted from small to large , Let's turn it around 
contribution_rate = lambda / sum(lambda);  %  Calculate the contribution rate 
cum_contribution_rate = cumsum(lambda)/ sum(lambda);   %  Calculate the cumulative contribution rate   cumsum Is a function of the cumulative value 
disp(' The characteristic value is ：')
disp(lambda')  %  Transpose to row vector , Easy to show 
disp(' The contribution rate is ：')
disp(contribution_rate')
disp(' The cumulative contribution rate is ：')
disp(cum_contribution_rate')
disp(' The eigenvector matrix corresponding to the eigenvalue is ：')
%  Be careful ： The eigenvector here should correspond to the eigenvalue one by one , The previous eigenvalue is equivalent to reversing , Therefore, the columns of the eigenvector need to be reversed 
%  rot90 Function to rotate a matrix counterclockwise 90 degree , Then transpose , You can achieve the effect of reversing the columns of the matrix 
V=rot90(V)';
disp(V)


%%  Calculate the value of the principal component we need 
m =input(' Please enter the number of principal components to be saved :  ');
F = zeros(n,m);  % Initialize the matrix that holds the principal components （ Each column is a principal component ）
for i = 1:m
    ai = V(:,i)';   %  Will be the first i A feature vector is extracted , And transpose it to a row vector 
    Ai = repmat(ai,n,1);   %  Repeat this row vector n Time , Constitute a n*p Matrix 
    F(:, i) = sum(Ai .* X, 2);  %  Be careful , Calculate the sum of each row after calculating the weight of the standardized data 
end

Code result analysis ：

  6、 ... and 、 Principal component analysis is used for clustering （SPSS）

The clustering here and the regression below can be seen in detail in the video .

  The clustering here is hierarchical clustering

7、 ... and 、 Principal component analysis is used for clustering （Stata）

 

  8、 ... and 、 other