CNN optimized trick

in the light of CNN Optimize

• Use similar 1*1 Network structure pre training RGB data
• Use about 128（0.005） To 256 （0.01） Of mini-batch size . If this is for your GPU Too big for , Reduce the learning rate to this ratio
• Convolution layer replaces FC, Global average pooling is used to predict
• Research to increase data set size , Consider data distribution
• If you cannot increase the size of the input image , But you can reduce the stride on subsequent layers

Training depth neural network trick

• every last epoch Both shuffle
• Extended dataset ： Small data sets are easy to over fit .
• Before training the entire dataset , First, it is trained and fitted on a very small sub data set , In this way, we can know whether the network can converge .
• stay FC layer channel>256 Should be used appropriately dropout
• Avoid hyperbolic activation functions sigmoid or tanh
• Don't use... Until you maximize pooling relu, Instead, use it after saving the calculation
• Try not to use relu, Poor initialization results in lost disk . May adopt PRelu, On the left *0.1
• Batch normalization is often used
• Modify the model , Use whenever possible 1*1 Of CNN layer

Ideas for improving algorithm performance

• Data improves performance
• Algorithm improves performance
• Adjust parameters to improve performance

Data improves performance

1. Get more data
2. Create more data
3. rescale data
   Replay the reduced data to the active function boundary
4. Data transformation

Algorithm improves performance

1. Sampling survey on Algorithm
2. Learn from the existing literature
3. Resampling method

Adjust parameters to improve performance

1. The diagnosis
   Judge model Whether it is over fitted or under fitted
2. Weight initialization
3. Learning rate
4. Activation function
5. Change the network structure
6. batchsize and epoch
7. Regularization
8. Optimizer and loss function
9. early stopping