Understanding of the detach() function of pytorch

https://blog.csdn.net/qq_27825451/article/details/95498211

https://blog.csdn.net/weixin_33913332/article/details/93300411?utm_medium=distribute.pc_relevant.none-task-blog-2%7Edefault%7ECTRLIST%7Edefault-4.no_search_link&depth_1-utm_source=distribute.pc_relevant.none-task-blog-2%7Edefault%7ECTRLIST%7Edefault-4.no_search_link

detach() & data()

** detach()** Back to a new tensor, It is separated from the current calculation diagram , But it still points to the storage location of the original variable , Its grad_fn=None And requires_grad=False, I got this tensor It is never necessary to calculate its gradient , Without gradient grad, Even if it's later re requires_grad Set as true, It also doesn't have gradients grad.

** Be careful ：** Back to tensor And the original tensor Share the same memory data .in-place The function will be modified in two tensor At the same time ( Because they share memory data ), At this point, when you want to call backward() May cause errors .

** data() Functions and detach() ** Same function , But it does not guarantee in-place The security of .

**In-place Correctness check ：** be-all tensor Will record what's used on them in-place operations. If pytorch detected tensor In a Function Has been saved for backward, But then it was in-place operations modify . When that happens , stay backward When ,pytorch You're going to report a mistake . This mechanism guarantees , If you use in-place operations, But in backward There is no error reported in the process , Then the gradient calculation is correct .

detach_()

** detach_()** take tensor From the... That created it graph In the separation of , Take it as a leaf node , Its grad_fn=None And requires_grad=False. Same as detach(), but detach_() Is a modification of itself （in-place operation ）, and detach() A new tensor.