Judgment of deep learning experiment results

The results of deep learning are inconsistent each time

It is better to use fixed value random seed

Random number seed setting ？ There are three random number seeds ,random,numpy And deep learning library gpu seeds , But I have set up all three , stay gpu The result will be different ,cpu It's the same every time , I am using dynet.
You can directly take the best value for reporting , More rigorous words do 10 Average the times .

Summary of deep learning experiment

The result of the experiment is not good

First, fit a sample or a subset , If the training loss decreases reasonably , Then consider the generalization .

1. Make sure the code is correct （ footstone ）
2. Look at the data , Quantity problem , The distribution of data （ Take a look at both the training set and the test set ）, Whether the data is balanced ;
3. See if the objective function is compatible with the business objective （ The wrong direction , The effect is not good ）;
4. Visualize the training process and the intermediate results of the prediction （ Locate the problem ）;
5. The last is the parameter adjustment

Adjustable parameter ：

1. Loss function loss No drop ： Whether the data is read correctly 、 Whether the network depth is appropriate 、 Whether there is a problem in the selection of loss function, etc ;
2. The accuracy of training set is high train acc、 The accuracy of the test set is low val acc： Whether the network has been fitted 、 If over fitting is required dropout Or other regularization strategies 、 Is there a problem with the selection of test set and training set 、 Whether the early stop strategy can be adopted ;
3. The loss function converges too slowly or does not decrease after reaching a certain value ： Expand the learning rate 、 Adopt learning rate attenuation strategy 、SGD、Adam、RMSProp Try it later ;

Of course , There's a lot more , I will not introduce . All in all , I think the result of the experiment is not good , If time permits , You shouldn't give up , In this way, we can “ By adjusting parameters ” Better understand the difficulties of the problem . Although tuning is only part of deep learning , But this part is the basis of metaphysics .

Over fitting

train loss When it comes down ,val acc If it also drops , Generally, it is over fitting . Over fitting usually occurs when the number of samples in the training set is small and the model is complex . It's usually just the beginning of the training set loss falling 、 Verification set accuracy increases , Train to a certain extent , Training set loss To descend slowly or no longer , The accuracy of the validation set decreases . You can add dropout、 Regular items or extended data sets .

loss The solution of not falling

Reference link 1
Reference link 2
Reference link 3
loss There are generally three types of non decline , namely ： On the training set loss No drop , On validation set loss No drop , And test set loss No drop .