Random seed torch in deep learning manual_ seed(number)、torch. cuda. manual_ seed(number)

During the training model , You will encounter a lot of randomness settings , The results of setting randomization and multiple experiments are more convincing . But now more and more papers require the reproducibility of the model , At this time, we have to control the randomness of the code
And the initial weight is the same every time , It is beneficial to the comparison and improvement of the experiment

instructions ： There is no direct relationship between random seed and neural network training , The function of random seeds is to produce random numbers with weights as initial conditions . The effect of neural network directly depends on the learning rate and the number of iterations .

To put it simply , The process of generating random numbers in a computer is not random , But its initial number （ seeds ） Is random . In deep learning ,（ For example, deep neural network ） We often need to set the initial value of the superparameter in the network , Such as weight , Here we need to use some functions that generate random numbers , These functions are usually seeded manually , If the seed is set to the same , Then the initial weight is the same .

The best random seed should not be found , The existence of randomness is just used to evaluate the robustness of the model . An excellent model , Not because the random initial position is slightly different , And can't find the best location . This is the work that the model itself should resolve , Instead of choosing a random number .

Correlation function ：

• torch.manual_seed(number)： by CPU Set seed in , Generate random number ;
• torch.cuda.manual_seed(number)： For specific GPU Set seeds , Generate random number ;
• torch.cuda.manual_seed_all(number)： For all GPU Set seeds , Generate random number ;

torch.manual_seed(1) It's about setting up CPU The random number of is fixed , Make the same run immediately .py Of documents rand（） function ==【 Random function 】== The generated values are fixed and random ！

But after setting random seeds , Is every run test.py The output of the file is the same , Instead of the same result every time a random function is generated ：

# test.py
import torch
torch.manual_seed(0)
print(torch.rand(1))
print(torch.rand(1))

Output ：

tensor([0.4963])
tensor([0.7682])

If you just want to run a random function every time, the result will be the same , As like as two peas, you can set the same random seeds before each random function. ：

# test.py
import torch
torch.manual_seed(0)
print(torch.rand(1))
torch.manual_seed(0)
print(torch.rand(1))

Output ：

tensor([0.4963])
tensor([0.4963])

Reference resources ：【PyTorch】torch.manual_seed() Detailed explanation