Face detection and recognition system based on mtcnn+facenet

Recently, I sorted out the face recognition reproduction I did last semester , The source code in git Little change on .
Github link ：https://github.com/ALittleLeo/FaceRcognization/tree/master

introduction

With the rapid development of Internet technology , The Internet has also entered the era of massive data . In the world , About every day 2EB The data is generated . In various types of data information , Face digital image data is the most important data type . So more and more new application scenarios need to use the information in the image , As an intuitive and efficient information carrier, face image data , It contains complex and diverse content information .
In recent years , Based on deep learning OCR Technology has significantly improved the accuracy and efficiency of face recognition . At present, the mainstream face recognition is mainly divided into two parts , Face detection and face recognition .1） In face detection , At present, there are relatively mature research results in traditional single dimension face detection . Texture feature extraction algorithm LBP It also has a certain good effect on multi-dimensional face detection .2） In face recognition , The mainstream is CNN and transformer Model . In addition, there are still teams that will Attention The mechanism is applied to face recognition , Strengthen the ability of network mining feature information . At present, the accuracy of face recognition is still not high , This project will build on the existing deep learning model , Use MCTNN+facenet Combined approach , In order to achieve better recognition effect .
Face detection and recognition is an important key technology in the field of artificial intelligence , And deep learning is the research hotspot of the rise of this technology , This technology can improve the accuracy of face detection and recognition technology and data security . This passage MCTNN+facenet The method analyzes the detection technology principle of deep learning , Organically combine the key points of this technology , Finally, face detection module technology based on deep learning is formed .

MCTNN+facenet

MTCNN A framework for deep convolution multitasking , This framework takes advantage of the inherent relationship between detection and alignment to enhance their performance . Use here MTCNN Face detection , On the one hand, its detection accuracy is really good , On the other hand facenet In Engineering , Has provided... For face detection mtcnn Interface .MTCNN It's multitasking cascading CNN Deep learning model of face detection based on , In this model, the boundary regression and key point detection are considered . especially , When predicting faces and facial markers , adopt 3 individual CNN Cascade processing of tasks from coarse to fine .
Stage 1： Use P-Net It's a full convolution network , Use to generate candidate window and border regression vectors (bounding box regression vectors). Use Bounding box regression To correct these candidate windows , Use non maximum suppression （NMS） Merge overlapping candidate boxes . Total convolution networks and Faster R-CNN Medium RPN In line .
Stage 2： Use R-Net Improve the candidate window . Will pass through P-Net The candidate window input of R-Net in , Reject most of false The window of , Continue to use Bounding box regression and NMS Merge .
Stage 3： Finally using O-Net Output the final face frame and feature point position . Similar to the second step , But the difference is to generate 5 Feature point locations .

experimental result

Dataset analysis

In this paper AgeDB-30 (570 ids/12,240 images/6K pairs) Cross age face recognition dataset ,AgeDB contain 16,488 Images of celebrities , according to 4:1 The proportion is divided into model training set and test set . The specific content includes actors , The writer , scientists , Politician , Each image is identified , Age and gender attributes . Co exist 568 Different subjects . The average number of images per subject is 29. The minimum and maximum ages are 1 and 101. The average age range for each subject is 50.3 year .

Experimental development environment

The experimental development environment of this paper is Intel i7-9750h,GPU by NVIDIA GTX1050
tensorflow-gpu 1.8.0
tensorflow-tensorboard 0.4.0
numpy 1.16.2
scipy 1.4.1
matplotlib 3.3.3
six 1.14.0
sklearn 0.0

Experimental results and Analysis

System implementation process ：
1、 Build a face database
2、 build MTCNN The Internet
3、 utilize MTCNN The network transforms the face data image database into the face feature database （embedding library ）
4、 Use MTCNN The network extracts the features of the face image to be detected
5、 Use facenet Compare the face to be recognized with the features of the face database , Drawing pictures , Get the final result

Directory description ：
align Folder contains three mtcnn Model files to be used , And build mtcnn Network files detect_face.py, The things in this are facenet Can be found in all projects
dataset The folder mainly stores data , Include images（ Face data Gallery ）,emb（ Face feature database ）,test_images（ The test image ）
models In the folder facenet Pre training model
utils It is a tool folder , For file reading and writing , Picture related operations .
predict.py It is the entrance to face recognition
create_dataset.py It is used to convert face data library into face feature library .

MTCNN from 3 Network structure （P-Net,R-Net,O-Net）
facenet Face recognition
Method ：facenet adopt CNN The face is mapped to the eigenvector of Euclidean space , Calculate the distance of facial features in different pictures , The distance between faces passing through the same individual is always smaller than that of different individuals Face · Distance of this prior knowledge training network （ Introduce the following supplements in detail ）.
Code implementation , function predict.py. technological process ： Load the face feature database , Extract the features of the face to be recognized and compare them with the information in the face database one by one .

The picture is the result of the real machine experiment , You can see in the training process of the model , The photos of Hu Ge and Jay Chou are in the face recognition database , The features of human face are stored in the training model , Can be detected （ Green frame out ） And was identified as Hu Ge and Jay Chou （ The red mark ）. Although Sabine's photos can be detected as human figures , However, due to the lack of photos in the data set, they are not trained and cannot be recognized .

Model evaluation , function evaluation_test.py, This file will draw the test file ROC curve , The optimal threshold is given , And use sklearn.metric obtain FPR, TPR, AUC Equal parameter . For datasets agedb_30 To test , This dataset has 12000 A picture , It is divided into 6000 There is one for each pair of photos label The value is 1 or 0 It is used to indicate whether the two pictures are one person , Is the same as True Different for False, Test results through this data set , Draw the current model ROC curve , Available roc_auc:0.8308236111111111,optimal_idx :1190,best_threshold :
1.1180874109268188, That is to say, it is the best to divide the two classifications by this threshold .

Face detection and recognition is a general recognition technology , In recent years, it has become a research hotspot in the direction of deep learning computer vision . Nowadays, the traditional face recognition technology has been relatively mature , However, face recognition technologies such as multi-dimensional recognition and human recognition still have room for improvement . This paper uses MCTNN and facenet Combined with the method of experiment , The operation procedure is simple , High feasibility . The experimental test pictures are from the published training set and the network , The experimental results show that the method used in this paper has strong adaptability , High accuracy .

Tradition LBP Algorithm

LBP Is a texture feature extraction algorithm ,LBP By extraction LBP Operator for image detection and recognition .
The original LBP The operator is defined as in 33 In the window of , Take the window center pixel as the threshold , Will be adjacent to 8 Compare the gray value of pixels with it , If the surrounding pixel value is greater than the central pixel value , Then the position of the pixel is marked as 1, Otherwise 0. From this 33 In the neighborhood 8 A point comparison can be obtained 8 Bit binary number , Then it is converted to decimal number type, that is 256 in LBP code , Through calculation, we can get the LBP value , And use this value to reflect the texture information of the region . As shown in the figure below ：

from LBP Definition available ,LBP The operator only satisfies the gray level invariance, but does not satisfy the rotation invariance . The rotated image will get different LBP value . So Maenpaa People will LBP Operator improvements are extended , Improved LBP Is rotation invariant LBP operator , After continuously rotating the circular neighborhood, a series of initial definitions are obtained LBP value , Take the minimum of these values as the LBP value .
LBP Its main application fields are face recognition analysis and texture analysis ,LBP Atlas is generally not used as a feature vector for classification and recognition , Instead, it's made up of LBP Statistical histogram generated by characteristic spectrum , And then used for classification and recognition .
As can be seen from the above , This feature is closely related to the location information of the picture . Directly extract features from two images without preprocessing , And analyze and classify , It will produce large errors due to the position deviation, which is difficult to identify . After researchers' improvement, they found , Divide the picture into several sub areas , Every pixel in each sub region is extracted LBP features , then , Create... In each sub region LBP Statistical histogram of features . Thus, the whole picture can be subdivided into multiple sub areas , Each sub region is described by a statistical histogram ; The whole picture consists of several statistical histograms .

Relevant mainstream neural network algorithms

The current neural network algorithm is a nonlinear dynamic system , Compared with the traditional texture feature detection algorithm, it has good self-organization 、 Adaptive ability . At present, the research of neural network method in face recognition is in the ascendant .Valentin Put forward a way , Firstly, the face is extracted 50 Principal elements , Then the autocorrelation neural network is used to map it to 5 In the dimension , Then an ordinary multi-layer perceptron is used for discrimination , For some simple test images, the effect is better ;Intrator A hybrid neural network is proposed for face recognition , Unsupervised neural network is used for feature extraction , Supervised neural networks are used for classification .Lee The characteristics of human face are described by six rules , Then, according to these six rules, the five senses are positioned , The geometric distance between facial features is input into fuzzy neural network for recognition , The effect is much better than the general method based on Euclidean distance ,Laurence Convolution neural network method is used for face recognition , Because the convolution neural network integrates the correlation knowledge between adjacent pixels , Thus, the image translation is obtained to a certain extent 、 Invariance of rotation and local deformation , Therefore, very ideal recognition results are obtained ,Lin A neural network method based on probabilistic decision-making is proposed (PDBNN), Its main idea is to use virtual ( Positive and negative examples ) Samples are used for reinforcement and anti reinforcement learning , Thus, an ideal probability estimation result is obtained , The modular network structure is adopted (OCON) Speed up network learning . This method is used in face detection 、 Face location and face recognition have been well applied in all steps , Other studies include :Dai Etc Hopfield Network for low resolution face Association and recognition ,Gutta Wait to propose RBF A hybrid classifier model combined with tree classifier for face recognition ,Phillips People will MatchingPursuit Filter for face recognition , In China, support vector machine (SVM) in statistical learning theory is used for face classification .
The application of neural network method in face recognition has certain advantages over the above-mentioned methods , Because it is quite difficult to explicitly describe many laws or rules of face recognition , The neural network method can obtain the implicit expression of these laws and rules through the process of learning , It is more adaptable , Generally, it is easy to realize . Therefore, the recognition speed of artificial neural network is fast , But the recognition rate is low . Neural network methods usually need to input the face as a one-dimensional vector , So the input node is huge , An important goal of its recognition is dimension reduction .

Conclusion

This research topic is face detection and recognition , This topic is still worthy of in-depth discussion in today's era , In terms of the achievements of face detection under current research , The application of computers in this field is still challenging . Based on the background of multi-dimensional small samples , Compared with traditional face detection and recognition, it will have more interference factors and have a stronger impact on the experimental results . Although there are higher difficulties , But there is inevitably more room for reflection to be investigated and studied , And the development direction of future science and technology application technology .
In this study , Collect data and analyze results through independent experiments , The point is faster-R-CNN + CRNN Model combination method for deep face learning . This research enables us to have a deeper understanding and understanding of this knowledge , At the same time, the research on face detection also has some basic knowledge and personal opinions . Last , I would like to thank the teachers and students who helped us in the process of our research and the authors of our references .