Pytorch Basics (Introductory)

PyTorch Basics

1. The core is tensor Tensor, A kind of Multidimensional Mathematical object of data .
2. Use torch Package creation tensor , The concrete is very simple , Not much said .
   （1） If you want to use a specific value fill tensor , have access to fill_() Method .【 Any underlined (_) Of PyTorch Methods all refer to in situ operation , That is, there is no need to create new objects Modify in place Content 】
   （2） When using torch.Tensor Constructor , The default tensor type is torch.FloatTensor. You can perform type conversion when using or utilize torch.tensor() Medium dtype Parameters .
3. The operations of addition, subtraction, multiplication and division are similar to others .
4. torch.transpose()： Transpose operation , It's simple , May refer to https://blog.csdn.net/qq_50001789/article/details/120451717
5. General indexing and slicing ： Basically the same as numpy.
6. Complex index ：
   （1） Discontinuous index of tensor

def describe(x):
    print("Type: {}".format(x.type()))
    print("Shape/size: {}".format(x.shape))
    print("Values: \n{}".format(x))

import torch

#  Create a 2D tensor 
x = torch.arange(6).view(2, 3)
describe(x)
# 1、 obtain 2D First order of tensor 2 Dimension and index number is 0 and 2 A quantum set of 
indices = torch.LongTensor([0, 2])
describe(torch.index_select(x, dim=1, index=indices))

# 2、 obtain 2D First order of tensor 1 Dimension and index number is 0 and 0 A quantum set of 
indices = torch.LongTensor([0, 0])
describe(torch.index_select(x, dim=0, index=indices))

# 3、row_indices The result is a tensor([0., 1.]), then col_indices yes tensor([1, 2])
row_indices = torch.arange(2).long()
col_indices = torch.LongTensor([1, 2])
describe(x[row_indices, col_indices])

Output results ：

Type: torch.LongTensor
Shape/size: torch.Size([2, 3])
Values: 
tensor([[0, 1, 2],
        [3, 4, 5]])
Type: torch.LongTensor
Shape/size: torch.Size([2, 2])
Values: 
tensor([[0, 2],
        [3, 5]])
Type: torch.LongTensor
Shape/size: torch.Size([2, 3])
Values: 
tensor([[0, 1, 2],
        [0, 1, 2]])
Type: torch.LongTensor
Shape/size: torch.Size([2])
Values: 
tensor([1, 5])

Process finished with exit code 0

Explain the last operation （3）,
（2） Connection tensor

import torch
x = torch.arange(6).view(2, 3)
describe(x)

describe(torch.cat([x, x], dim=0))

describe(torch.cat([x, x], dim=1))

describe(torch.stack([x, x]))

result ：

Type: torch.LongTensor
Shape/size: torch.Size([2, 3])
Values: 
tensor([[0, 1, 2],
        [3, 4, 5]])
Type: torch.LongTensor
Shape/size: torch.Size([4, 3])
Values: 
tensor([[0, 1, 2],
        [3, 4, 5],
        [0, 1, 2],
        [3, 4, 5]])
Type: torch.LongTensor
Shape/size: torch.Size([2, 6])
Values: 
tensor([[0, 1, 2, 0, 1, 2],
        [3, 4, 5, 3, 4, 5]])
Type: torch.LongTensor
Shape/size: torch.Size([2, 2, 3])
Values: 
tensor([[[0, 1, 2],
         [3, 4, 5]],

        [[0, 1, 2],
         [3, 4, 5]]])

Process finished with exit code 0

7. Tensor requires_grad=True when , You can track gradient tensors and gradient functions . These two things need gradient based learning “ Supervised learning paradigm ”.

Create tensors and calculate gradients ：

import torch
x = torch.ones(2, 2, requires_grad=True)
describe(x)
print(x.grad is None)

y = (x+2)*(x+5)+3
describe(y)
print(x.grad is None)

z = y.mean()
describe(z)
z.backward()
print(x.grad is None)

result ：

Type: torch.FloatTensor
Shape/size: torch.Size([2, 2])
Values: 
tensor([[1., 1.],
        [1., 1.]], requires_grad=True)
True
Type: torch.FloatTensor
Shape/size: torch.Size([2, 2])
Values: 
tensor([[21., 21.],
        [21., 21.]], grad_fn=<AddBackward0>)
True
Type: torch.FloatTensor
Shape/size: torch.Size([])
Values: 
21.0
False

Process finished with exit code 0

Gradient is a value （ The slope of function output to function input ）.

8. CUDA tensor

（1） establish cuda tensor

import torch

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
print(device)

x = torch.rand(3, 3).to(device)
describe(x)

result ：

cuda
Type: torch.cuda.FloatTensor
Shape/size: torch.Size([3, 3])
Values: 
tensor([[0.2369, 0.9929, 0.6972],
        [0.1366, 0.0594, 0.0726],
        [0.4803, 0.1209, 0.6055]], device='cuda:0')

（2）CUDA Tensor and CPU Binding tensor mixing

#  take CUDA Tensor sum CPU Binding tensor mixing （CUDA He Fei CUDA object ）,y stay CPU On ,x stay GPU On . In this case, an error will be reported 
y = torch.rand(3, 3)
x + y

The result is wrong ：

Traceback (most recent call last):
  File "C:/Users/27692/PycharmProjects/Quinn/test/test.py", line 60, in <module>
    x + y
RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu!

If you want to be right cuda He Fei cuda object To operate , It must be ensured that they are Same device On ：

cpu_device = torch.device("cpu")
y = y.to(cpu_device)
x = x.to(cpu_device)
print(x + y)

result ：

tensor([[0.9204, 0.8360, 0.6289],
        [1.2421, 1.4353, 1.2174],
        [0.9342, 1.0427, 1.1978]])

Process finished with exit code 0

【 Practice expanding 】

• unsqueeze() Functions and squeeze() function ：

a = torch.rand(2, 3)
print(describe(a))

#  In the 0 Dimension increase 1 Dimensions 
b = a.unsqueeze(0)
print(describe(b))

#  In the 1 Dimension increase 1 Dimensions 
b = a.unsqueeze(1)
print(describe(b))

#  In the last place 2 Dimensions increase 1 Dimensions 
b = a.unsqueeze(-2)
print(describe(b))

Pay attention to the output ：！！！ Tensor shape （torch.size）

Type: torch.FloatTensor
Shape/size: torch.Size([2, 3])
Values: 
tensor([[0.2643, 0.1925, 0.2562],
        [0.7674, 0.9930, 0.2341]])
None
Type: torch.FloatTensor
Shape/size: torch.Size([1, 2, 3])
Values: 
tensor([[[0.2643, 0.1925, 0.2562],
         [0.7674, 0.9930, 0.2341]]])
None
Type: torch.FloatTensor
Shape/size: torch.Size([2, 1, 3])
Values: 
tensor([[[0.2643, 0.1925, 0.2562]],

        [[0.7674, 0.9930, 0.2341]]])
None
Type: torch.FloatTensor
Shape/size: torch.Size([2, 1, 3])
Values: 
tensor([[[0.2643, 0.1925, 0.2562]],

        [[0.7674, 0.9930, 0.2341]]])
None

Process finished with exit code 0

# ======squeeze()====== 
#  Above is insert / increase , This is deleted / Get rid of 
a = torch.rand(2, 1, 3)
print(describe(a))

#  Will be the first 1 Dimension out of 
b = a.squeeze(1)  # a.squeeze(-2) It has the same effect   second to last 
print(describe(b))

#  try -3
b = a.squeeze(-3)   #  This result remains unchanged , Because only the dimension is 1 Will be removed when 
print(describe(b))

Output results ：

Type: torch.FloatTensor
Shape/size: torch.Size([2, 1, 3])
Values: 
tensor([[[0.4349, 0.0568, 0.0304]],

        [[0.2712, 0.3612, 0.3238]]])
None
Type: torch.FloatTensor
Shape/size: torch.Size([2, 3])
Values: 
tensor([[0.4349, 0.0568, 0.0304],
        [0.2712, 0.3612, 0.3238]])
None
Type: torch.FloatTensor
Shape/size: torch.Size([2, 1, 3])
Values: 
tensor([[[0.4349, 0.0568, 0.0304]],

        [[0.2712, 0.3612, 0.3238]]])
None

Process finished with exit code 0

• Create a tensor with normal distribution ：

a = torch.rand(3,3)
a.normal_()