1. Introduction to pytorch

One . Development

2002 year Torch
2011 year Torch7
2016.10 Release PyTorch0.1,THNN Back end
2018.12 Release 1.0,CAFFE2 Back end
2019.5 Release 1.1

Two . Homogeneous framework

Google：keras,TensorFlow
Amazon：MXNet
Facebook：Caffe2,PyTorch

PyTorch and TensorFlow difference

1. Dynamic graph

Similar to human thinking mode , Calculate directly with specific values each time and get the results .
PyTorch It's a motion picture .

2. Static diagram

It's like defining a formula first , Every time we will give a specific value again , The calculation result of the formula will be substituted in the runtime , Create formula and run two separate segments , Static expression cannot be changed after the formula is created .
TensorFlow1 It's a static graph ,TensorFlow2 It is the combination of dynamic graph and static graph .

3. summary

As a researcher or novice , recommend PyTorch, Easy to use , Focus more on algorithms .
As an engineer , recommend TensorFLow2, Better industrial layout .

3、 ... and .PyTorch Simple features

1.GPU Speed up

import torch
import time
print(torch.torch_version)
print(torch.cuda.is_available())

# Two matrices 
a = torch.randn(10000,1000)
b = torch.randn(1000,2000)

# Use cpu Calculate the multiplication of the matrix , And record the calculation time 0
t0 = time.time()
c = torch.matmul(a,b)
t1 = time.time()
print(a.device,t1-t0,c.norm(2))

# Use GPU Calculation 
device = torch.device('cuda')
a = a.to(device)
b = b.to(device)

t0 = time.time()
c = torch.matmul(a,b)
t1 = time.time()
print(a.device,t1-t0,c.norm(2))

# Calculate the verification result again 
t0 = time.time()
c = torch.matmul(a,b)
t1 = time.time()
print(a.device,t1-t0,c.norm(2))

use CPU The running time is 0.1456127166748047, use GPU The first run time is 0.38901543617248535, Than CPU Also big , Because it is the first time to adopt CUDA Will initialize the environment , This time is an inaccurate calculation of time , The second time may be because the calculation is too fast , Direct display 0.

Make the matrix bigger ：

a = torch.randn(10000,1000)
b = torch.randn(1000,8000)

The result is ：

2. Automatic derivation

import torch
from torch import autograd

x = torch.tensor(1.)
a = torch.tensor(1.,requires_grad=True)
b = torch.tensor(2.,requires_grad=True)
c = torch.tensor(3.,requires_grad=True)

y = a**2 * x + b * x + c

print('before:',a.grad,b.grad,c.grad)
# Start derivation 
grads = autograd.grad(y,[a,b,c])
print('after:',grads[0],grads[1],grads[2])

For this formula y, Yes a To find a partial derivative 2ax, Yes b To find a partial derivative x, Yes c To find a partial derivative 1.

3. Common network layer

nn.Linear
nn.Conv2d
nn.LSTM

nn.ReLU
nn.Sigmod

nn.Softmax
nn.CrossEntropyLoss
nn.MSE