Why should model.eval() be added to the pytorch test?

Many machine learning tutorials have mentioned that when using pytorch for training and testing, you must specify eval for the instantiated model, so why do you need to set model.eval() when pytorch is testing?What is the function of model.eval()?The next article tells you.

When using PyTorch for training and testing, be sure to specify train/eval for the instantiated model. When eval(), the framework will automatically fix BN and DropOut.It will not take the average, but use the trained value, otherwise, once the batch_size of the test is too small, it will easily be caused by the BN layer to generate a large color distortion of the image!!!!!!

The difference between model.eval() and with torch.no_grad()

When validating in PyTorch, model.eval() is used to switch to test mode, in which mode,

Mainly used to notify the dropout layer and batchnorm layer to switch between train and val modes

In the train mode, the dropout network layer will set the probability of retaining the activation unit according to the set parameter p (retention probability = p); the batchnorm layer will continue to calculate and update the parameters such as the mean and var of the data.

In val mode, the dropout layer will allow all activation units to pass through, while the batchnorm layer will stop computing and update mean and var, directly using what has been learned in the training phaseout the mean and var values.

This mode does not affect the gradient calculation behavior of each layer, that is, the gradient calculation and storage are the same as the training mode, but no backprobagation is performed

And with torch.no_grad() is mainly used to stop the work of the autograd module to accelerate and save video memory. The specific behavior is to stop the gradient calculation, thereby saving GPU computing power and video memory, but it does notAffects the behavior of dropout and batchnorm layers.

I don't understand why there is a difference between model.eval() and model.train() in the training and testing functions. After reviewing, I will make the following arrangements

In general, our training process is as follows:

1. Train after getting the data. During the training process, use

model.train(): Tell our network that this stage is used for training and can update parameters.

2. Prediction after training is completed. During the prediction process, use

model.eval() : Tell our network that this stage is used for testing, so the parameters of the model are not updated in this stage.