Reading guide ： Verification code as the first barrier of network security , Its importance is self-evident . At present , With the rapid development of convolutional neural network, the security of many verification codes is greatly reduced , Some new verification codes even choose to sacrifice availability to ensure security . Research on Countermeasure sample technology , It has brought new opportunities to the field of verification code , It has been applied to the reverse identification of verification code , It has injected new vitality into this protracted offensive and defensive confrontation .
The sharing content includes three aspects ：
- Introduction to countermeasure samples
- Exploration and application of polar counter sample technology
- Follow up work and thinking
--
01 Introduction to countermeasure samples
1. What is a counterexample
Counter samples ( Adversarial Examples ) The first concept of Christian Szegedy（ Christian · Segdi ） stay ICLR2014 （ International Conference on learning characterization ） Put forward on the , That is, input samples are formed by deliberately adding subtle non random interference to the data set , The input after interference causes the model to give a wrong output with high confidence .
Pictured above （ Left ） The original image is in 57% The confidence of is judged as “ Panda ”, But with a little interference , When the human eye cannot see the difference at all , The model is based on 99% Confidence output of “ Gibbons ”.
Of course , The confrontation sample will not only appear in the picture , voice 、 There will also be confrontation samples on the text , Add a slightly imperceptible background sound to a speech , It can make the speech recognition model output wrong speech content ; Replace with synonyms in a paragraph of text , You can also construct a countermeasure sample , Misleading language models .
So why does the deep neural network confront samples ？
At present, it is generally accepted that 15 Goodfellow's view in , It is the high-dimensional linearity of deep neural network that leads to the emergence of countermeasure samples .
Intuitive understanding , When performing a high-dimensional linear operation , Every dimension makes some minor changes , It will make a huge change in the output results . Pictured above , The original input is x, The weight of the linear operation is w, At this point, classify the samples into categories 1 Is the probability that 5%, But we will change every dimension we enter 0.5, At this point, classify the samples into categories 1 The probability becomes 88%.
The above are some simple definitions of countermeasure samples and the reasons why they are widely accepted at present .
2. Why do we need to fight against the sample
since 2012 year AlexNet Since the birth of , Deep neural network ushered in a period of explosive development , And it is widely used in automatic driving 、 Medical care 、 Finance 、 Security and other fields . It can be said that the deep neural network model has gone deep into every aspect of our life . It is an objective fact to confront the threat of samples to these models , If you will STOP Add some small disturbance to the traffic signs , It will be recognized as deceleration by the test model . Pedestrians wear clothes with trained mosaic patterns , In the sight of intelligent monitoring model “ stealth ”.
therefore , We have made great efforts to study the counter sample technology , On the one hand, the security of deep neural network is explored by using countermeasure samples , On the other hand, the use of counter sample defense AI Abuse , Such as automatic verification code identification , Misuse of face recognition models , Automated spear fishing attacks .
3. Development history and research trend of countermeasure samples
Szegedy stay 2014 The concept of countersample was proposed at the International Conference on representational learning in , He believes that this is caused by high-dimensional nonlinearity , An optimization method based on L-BFGS Method . In the second year Goodfellow wait forsomeone , It is proved that the appearance of counter samples is the result of high-dimensional linearity , A fast gradient symbol method based on gradient is proposed . after , All kinds of FGSM The methods of generating countermeasure samples based on are emerging one after another , One of the most representative is I_FGSM, It's in FGSM Several iterations are performed on the basis of .
stay L-BFGS On the basis of C&W, This strange name is the initials of the two authors .
2018 Xiao Chaowei put forward advGAN, Enter the original clean picture , Use the generated countermeasure network to generate countermeasure pictures .
Then various attack methods based on gradient iteration , Optimization based attack method and GAN Methods are gradually enriched . also , Counter sample in computer vision 、nlp、 Applications in various fields such as speech recognition have been gradually excavated .
stay dblp Search for adversarial example You can find , from 14 Since then , The number of papers related to counter samples is also increasing , Antagonism sample has become a hot research field .
--
02 Exploration and application of polar counter sample technology
Above, we have a preliminary understanding of the counter sample technology , Now let's introduce our exploration of polar test in counter sample technology and the application of counter sample technology in verification code .
1. Crack method of verification code
Extreme experience 2012 The puzzle verification code was developed in , Later, the Jiugongge verification code and text Click verification code were launched . We have also studied various methods to crack the verification code on the market .
In the picture are the verification code of extreme test and the text Click verification code .
For the Jiugongge verification code , Our cracking method and process are as follows ： Extract the above prompt words from the verification code , Get into CRNN The Internet , Output the contents of this prompt word . Then pull out 9 A little picture , Get into CNN, Predict the category of each picture . Combine the content of the prompt words given with the category of each small picture , Get the final answer .
Cracking of text Click verification code , The same is to pick out the prompt words , Enter into CRNN. But for the recognition of text position , We need to use a target detection model to detect the position of the text , Then pick out each word in the picture according to its position , The subsequent process is the same as the cracking method of the Jiugongge verification code .
Insight into the mainstream Jiugongge verification code and the cracking method of text Click verification code , We can generate countermeasure samples for each link of model recognition . For example, the confrontation against prompt words , Confrontation against image classification , The antagonism of text location for target detection .
2. Geometric perception antagonism sample generation framework
- Black dots mean clean sample
- Two dashed lines f,h They are training model and verification model , Solid line g test model
What the whole framework does is , Iteratively in the model f Up operation attack Algorithm , Until you get adv image In validating the model h The probability of the correct category of output on is less than a certain threshold , And in the process of iteration, gradually release the anti-interference L-p The limitation of norm .
Why divide models into training models 、 Validation model and test model ？ It is mainly to improve the transfer rate of counter samples , in other words , In the model A The confrontation samples obtained from the training , In the model B It can also have a good effect . This aspect can be compared to improving the generalization of a model on a data set that has not been seen .
such as , The yellow diamond indicates f The optimal countermeasure sample , If you do not validate the model h Make a constraint , Use only the training model f Generate countermeasure samples , Then it is likely that the iteration will stop at this position , So the counter sample is in the training model f On , Over fitting .
As for why we should gradually release the restrictions against interference , It is mainly to ensure that when the confrontation effect is achieved , It can make the confrontation picture relative to the original picture , It looks similar .
such as , For a picture , First, set the constraint value of interference as 6, If you update it several times, you can achieve the effect of attack , So the iteration stops ; If multiple updates still fail to achieve the attack effect , Then increase the constraint value to 10, Continue to update .
The rightmost sample in the figure above , after 2 Increase the constraint value a second time , To achieve the effect of the attack , The leftmost sample , After 4 Time , To achieve the effect .
3. Specific attack methods
The internal attack method is relatively simple , It's using FGSM And some of its variations that add tricks , It is mainly to improve the antagonism and transfer power of the antagonism samples .
FGSM, Is the steepest gradient sign method , The idea of the algorithm is very simple ：
- Increase when training the model loss function , Need to compute loss The gradient of the function to the disturbing noise , At this point, we update the parameter in the positive direction of the gradient , Can make loss The function increases .
- FGSM Update parameters only once ,I-FGSM Iterate many times .
- MI-FGSM Introduce momentum m, Retain historical information about short-term gradients , Improve stability .
- DI-FGSM With a certain probability p, Diversify the input , It can be understood as a small data augmentaion.
- TI-FGSM Use a pre-defined kernel, Convolution smoothing is performed on the gradient of disturbance parameters , It is also to improve the transfer rate of confrontation .
The above are some methods of internal attack of our entire architecture .
4. Preliminary effect against the sample
Here we use in imagenet Parameters of pre training , Fine tune out 10 A model for identifying the verification code of the nine squares , this 10 The models have different structures —— Yes vision transformer、resnet、inception As the foundation block Deep neural network based on . The classification accuracy of these models on clean samples has reached 98% above .
With model7 As a target ,model1 As a training model ,model0 As a validation model , The maximum allowable interference pixel value is 64（ This parameter will affect the quality of the generated countermeasure sample image ）, And the confidence of the counter sample to the correct category in the validation model is less than 0.01.
The experimental conclusion ：
Pictured above （ Top left ） The bar graph is the classification accuracy of each model on clean samples and confrontation samples , From the results of the experiment , Counter jamming trained only on a single model , It can reduce the classification accuracy of other unknown models to 20% following .
Pictured above （ Right ） Thermogram , Reflect each category 、 The classification accuracy of each model on the confrontation samples . The antagonism sample has ideal antagonism effect on all kinds of pictures .
Pictured above （ The lower left ） Distribution of maximum disturbed pixel values , The maximum perturbed pixel value of most anti image is 64, It can be predicted that these confrontation pictures are quite different from the original pictures .
Pictured , The above figure is the verification code generated against the image , The following figure shows the verification code generated by the clean image ; It can be seen that the confrontation picture is blurred , There is a big gap with the original drawing , Some pictures can only barely distinguish the content of the picture by the outline of the object （ Like a chair 、 The kettle 、 Axe, etc ）, Some pictures are even completely unrecognizable （ Like the rocket on the right ）.
This is the result of our preliminary experiment . The antagonism and usability of the model may require a series of tradeoffs , Finally achieve a balanced effect .
5. Weigh the quality of the picture against the effect of the confrontation
Set the maximum interference pixel value to 64, Verify the model confidence threshold 0.01 when , The image interference of the countermeasure sample obtained from the training is too serious , Make the picture more snowflakes 、 It's fuzzy , Low picture availability .
Gradually reduce the maximum interference pixels , Increase the confidence threshold of the validation model , At the same time, the training model and the verification model are configured as the Ensemble, This makes the countermeasure sample without losing too much antagonism , Improve picture quality .
With model7 As a target ,model 01239 As a training model ,model 489 As a validation model , The maximum allowable interference pixel value is 20, And the confidence of the counter sample to the correct category in the validation model is less than 0.05.
The classification accuracy of the antagonism image on the target model is 30% about , At the same time, the accuracy of other models has also been improved 30%-40%, Compared with the previous training settings, the confrontation effect is reduced .
however , Analyze the distribution of the maximum interference pixels , Most of the pictures increase to... At the pixel threshold 10 After that, the confrontation effect will be achieved , And stop the iteration . At this point, the image quality of the counter image can guarantee the usability .
Although there are still a few fuzzy pictures in the picture , Like the stethoscope in the upper right corner , But most of the pictures are not very different from the original pictures . In real use , We can use an algorithm to evaluate the image quality to remove these blurred images .
6. On the confrontation of word order recognition
In addition to the cracking model of the Jiugongge verification code, it generates defensive countermeasures , For word order recognition crnn And the object detection model for character position recognition .
It's also used I_FGSM Attack methods , Iterate to make crnn Model ctc loss Growing , The generated countermeasure samples are training crnn Output wrong results on the model , Such as “ Deep fried pumpkin slices ” Was identified as “ Yinan honey is expensive ”,“ Yanxi Hall ” Was identified as “ Fried Yuanbao ”. But to the naked eye , There is little difference between the counter sample and the original clean sample .
7. The confrontation of target recognition
Target detection counter sample , We tried to target yolov3 The attack of , Initialization interference , Add to the original picture to enter yolov3 Test model , Iterative update interference , Minimize the predictive confidence of real text position .
The result of the attack , Pictured above yolov3 Click on the text and the text in the verification code cannot be completely located , And the naked eye can hardly distinguish the difference between the resistant sample and the clean sample .
8. The engineering of nine palaces against samples
Let's crawl through various categories of graphs , Then update each model in the model library , Enter the frame of geometric perception just mentioned , Train to get pictures of the confrontation samples , After another round of image filtering , You get a static repository .
When generating the verification code of the ninetown grid , Extract images from the static Repository , Generate a picture of the Jiugong verification code , Then generate clean prompt words , Then use CRNN The attack of , Generate a cue word for confrontation , Finally, the counter samples of the nine grid verification code are combined .
--
03 Follow up work and thinking
The above is our current work and Exploration on the application of countermeasure samples in the field of verification codes , We also have some planning and thinking about the follow-up work .
1. be based on GAN Generation technology of AdvGAN
We used FGSM This relatively simple method is used to generate countermeasure samples , There is no way to separate the generation process from the model training process , In this way, there is no way to completely transplant the framework of the model to target detection and prompt word attacks . We want to change the framework , Based on GAN Generation technology of , such as AdvGAN.
be based on GAN How to generate confrontation samples , The advantage is that it is directly in G In the process of model training , Store the anti-jamming features in G In the parameters of the model , Decouple the process of model training and countermeasure sample generation . Against the generation process of samples , No more iterations or optimizations , Improve the efficiency of generation , It also reduces the cost of deployment .
Today's sharing is here , Thank you. .
This article was first published on WeChat public “DataFunTalk”.
