Machine learning PCA - Experimental Report

---

〇、 Experimental report pdf It can be downloaded from this website

Machine learning experiment 1 ：PCA
This requires points to download （ Because the background of the experiment report checks the duplicate , It is not recommended to go whoring for nothing ）.
I suggest you read the blog , There will be many experimental reports in the blog, which will be used in the future 【…】 Bold notes .

One 、 The purpose and requirements of the experiment

1. Realization PCA Face reconstruction algorithm , The box 20,40,60,80,…,160 A projection to reconstruct the effect of the image ;
2. Realization PCA Face recognition based on Algorithm , give 10,20,30,…,160 Face recognition rate of dimension ;
3. use PCA Used for face image dimensionality reduction , Realization 3 2-D and 3-D spatial reality of multiple subsets of different data sets
   Visualization of existing data ;
4. At the same time, design a new PCA Algorithm , The content is briefly written in the experiment report , And with the classic PCA Compare .（ detailed
   The detailed content can be written in another paper and submitted to “ Paper submission office ”.）;
5. All experiments must be done independently , Never copy experimental reports and codes ！！ We have a duplicate check function in the background ,
   COPY Once the course is found to be 0 branch . Don't take the garbage grade that you did in your freshman year PCA The project report is handed in again , I want to check the repetition rate .

Two 、 Experiment content and method

2.0 PCA Algorithm learning and review

PCA It is one of the most commonly used dimensionality reduction methods , In teacher Zhou Zhihua's 《 machine learning 》 Two properties are given in the book ：

• Recently refactoring ： The sample point is close enough to the hyperplane ;
• Maximum separability ： The projections of sample points on this hyperplane are separated as far as possible .
  The book gives a clear derivation , Based on the recent reconfigurability and maximum separability , The principal component analysis can be obtained separately
  Two equivalent derivations .

2.0.1 PCA Derived optimization problem

2.0.2 The solution of the optimization problem

2.0.3 Algorithm flow

2.1 Face data set

The main face data sets used in this experiment are ：

ORL56_46 Face data set , This dataset has 40 personal , everyone 10 A picture . The pixel size of each picture is 56×46. In this experiment, all photos of the data set are used .

AR Face data set , The database consists of 3 More than one database ;126 Facial frontal images of subjects 200 A color image . Each theme has 26 Different pictures . For each subject , These images are recorded in two different periods , Two weeks apart , Each period consists of 13 It's made up of images . All images are taken by the same camera under strictly controlled lighting and viewpoint conditions . Every image in the database is 768×576 Pixel size , Each pixel consists of 24 position RGB The color value indicates . Before this experiment 15 A collection of personal human faces , Everyone only uses the former 7 An unobstructed face image .

FERET Face data set ,200 people , Everyone 7 Zhang , Classified , grayscale ,80x80 Pixels . The first 1 This is a standard unchanged image , The first 2,5 This is an image of a large range of attitude changes , The first 3,4 This is an image of small amplitude attitude change . The first 7 This is an image of illumination change .

2.2 Experimental flow chart

3、 ... and 、 Experimental steps and processes

3.1 Experimental process

In feature extraction , Principal component analysis is an important linear transformation method , This is through the right ORL Data sets are experimented to illustrate PCA Face feature extraction and face reconstruction using these features . Final presentation 20,40,60,80,…,160 A projection to reconstruct the effect of the image .

1） First, import. 40×10=400 Zhang Renren , And pull it into a column vector , Make a human face matrix faces;

%---------- Import a class of face samples ----------
faces=zeros(face_length*face_width,sample*class);
for i=1:class
    for j=1:sample
    p=(i-1)*10+j;
    facepath = strcat('C:\Users\Vermouth\Desktop\ML\ Homework \ORL56_46\orl',num2str(i),'_',num2str(j),'.bmp');
    face=imread(facepath);
    faces(:,p)=face(:); % Pull the face into a column vector 
    end
end

2） Then the face is decentralized , Calculate covariance faces_cov, Eigenvalues and eigenvectors are obtained by eigenvalue decomposition of covariance matrix , Calculated basis base;

%----------pca Algorithm ----------
faces_average=mean(faces,2); % Average face 
faces_diff=faces-repmat(faces_average,1,sample*class); % Subtract the average face 
% Calculate covariance 
faces_cov=faces_diff'*faces_diff;
% Eigenvalue decomposition 
[V,D]=eig(faces_cov);
D1=diag(D);
D_sort = flipud(D1);
V_sort = fliplr(V);
base = faces_diff*V_sort(:,:)*diag(D_sort(:).^(-1/2)); % The base 

3） Then take out the first picture of the first person （orl1_1.bmp） refactoring ;

%---------- Reconstruct the first picture ----------
img1=strcat('C:\Users\Vermouth\Desktop\ML\ Homework \ORL56_46\orl1_',num2str(1),'.bmp');
test1=imread(img1);
combined1(:,1)=test1(1:face_length*face_width);
combined1=double(combined1);
dis1=combined1-faces_average; % Subtract the average face 
factor1=base'*dis1;

4） Normalization of projection vector + Result visualization .

figure
for t=20:20:160
    base_temp = base(:,1:t)*factor1(1:t);
    base_temp = base_temp + faces_average;
    base_temp = reshape(base_temp, face_length,face_width);
    MAX=max(max(base_temp));                 % Find out the maximum gray value in the image matrix 
    MIN=min(min(base_temp));                 % Find the minimum value of gray value in the image matrix 
    base_temp(:,:)=(base_temp(:,:)-MIN)/(MAX-MIN)*255;  % Normalized and mapped to 0-255
    base_temp=uint8(base_temp);
    subplot(2,4,t/20);
    imshow(base_temp);
end

【 It is not recommended to paste code on the test report , The following experimental reports will not paste the code directly 】

3.1.2 Experimental effect picture

For the projection matrix ,orl The dataset yields the following results ：

about orl1_1.bmp Face image , The following reconstruction results are obtained ：

about orl2_1.bmp Face image , The following reconstruction results are obtained ：

3.2 PCA Algorithm for face image dimensionality reduction

3.2.1 Face image dimensionality reduction steps

3.2.2 Three data sets are reduced to 2,3 Visualization of dimension

The following shows 3 Two dimensional and three-dimensional space of multiple subsets of different data sets to realize the visualization effect of data ：
1） ORL Face data set （200 Face images ）

• Goal dimension ：2 dimension
  

• Goal dimension ：3 dimension
  
  2） FERET_80 Face data set

• Goal dimension ：2
  

• Goal dimension ：3
  
  3） AR Face data set ：

• Goal dimension ：2
  

• Goal dimension ：3
  

3.3 PCA+KNN Face recognition

This experiment mainly aims at the reconstruction threshold of dimension reduction , The proportion of the training set and the test set KNN Of k Set for consideration . This experiment uses FERET_80 Face data set .

3.3.1 Reconstruction threshold for dimensionality reduction

3.3.2 Face recognition effect data display

3.4 Design a new PCA Algorithm

【 Come on ！！！ I will not make a public show of shame ！！！】

Four 、 Experimental conclusion or experience

Freshmen may be more “ Try to understand PCA And complete the experiment ”, To sophomore year , And has a period of scientific research experience , I can expand my thinking PCA What are the advantages of , Why is there such an advantage , What other algorithms can be optimized with this advantage ,PCA What's the problem? , So how can I optimize it .

For the typesetting and writing of the experimental report , After a year and a half of study , Also gradually formed their own style . It will be more rigorous and standardized than that of my freshman year .

Thinking PCA When optimizing , I think about kernel functions , Image entropy is considered , Even consider putting PCA And LBP combination , There are also integrated learning methods to improve the recognition rate , However, a good recognition effect has not been achieved . Although this left and right faces to optimize PCA The idea is relatively simple , But I have tried many methods and achieved better results , And the complexity is reduced .

When there are more operations MATLAB No more use a,b,c Wait for the variable name , When I was a freshman , I remember I went on stage to share PCA Code time , The code is really “ ugly ”, Now think about it. , This kind of code thinking has been greatly improved .

This time PCA The experimental code is more detailed , The problems to be discussed in this experiment should also be as detailed as possible , I hope we can hold the scientific research scholars behind us “ Ingenuity attitude ” Treat everything . This experiment is compared with that of freshmen PCA Experimentally speaking , More is in a kind of own understanding and exploration .