Pytorch learning notes 2 - about tensor

Tensor yes PyTorch A special matrix storage structure in , And numpy similar , It's just tensors have access to GPU Speed up the operation

1. Tensor Initialization

You can initialize in the following ways tensor：

1. From data

data = [[1, 2], [3, 4]]
x_data = torch.tensor(data)

---

2. From numpy array

np_array = np.array(data)
x_data = torch.from_numpy(np_array)

---

3. From another tensor

x_ones = torch.rand_like(x_data, dtype=torch.float)
print(x_data)

---

4. Rand or const tensor

rand_tensor = torch.rand(size=(2, 3))
ones_tensor = torch.ones(size=(2, 3))
zeros_tensor = torch.zeros(size=(2, 3))
print(rand_tensor)
print(ones_tensor)
print(zeros_tensor)

tensor & numpy

CPU Upper tensors And Numpy The matrix physically shares storage units , And through tensor.numpy() and torch.from_numpy() Interconversion ：

# convert torch to numpy
t = torch.ones(5)
n = t.numpy()
print(t)	# tensor([1., 1., 1., 1., 1.])
print(n)	# [1. 1. 1. 1. 1.]

# torch and numpy array share the same underlying memory
t.add_(3)
print(t)	# tensor([4., 4., 4., 4., 4.])
print(n)	# [4. 4. 4. 4. 4.]

# convert numpy to torch
n = np.array([1, 1, 1])
t = torch.from_numpy(n)
print(t)	# tensor([1, 1, 1], dtype=torch.int32)
print(n)	# [1. 1. 1.]

2. Tensor Attributes

Tensor Attributes Can describe tensor Of shape,datatype And the stored device：

tensor = torch.randn(size=(2, 3), requires_grad=True, device='cuda')
print(f'Shape: {
      tensor.shape}')
print(f'Datatype: {
      tensor.dtype}')
print(f'Device tensor stored on: {
      tensor.device}')

# Shape: torch.Size([2, 3])
# Datatype: torch.float32
# Device tensor stored on: cuda:0

3. Tensor Operations

Over 100 tensor operations, including transposing, indexing, slicing, mathematical operations, linear algebra, random sampling, and more are comprehensively described here.

Each of them can be run on the GPU (at typically higher speeds than on a CPU). If you’re using Colab, allocate a GPU by going to Edit > Notebook Settings.

Use GPU operation tensors：

device = 'cuda' if torch.cuda.is_available() else 'cpu'
tensor1 = torch.rand(size=(2, 3))
tensor2 = tensor1.to(device)
print(f'Device tensor stored on: {
      tensor2.device}')
# Device tensor stored on: cuda:0

1. Standard numpy-like indexing and slicing

take index=1 The column of is set to zero ：

tensor = torch.ones(3, 3)
tensor[:, 1] = 0
print(tensor)

---

2. Joining tensors

adopt torch.cat([t1, t2, t3], dim=1) By column (dim=1 Specify to merge by the second dimension column ,dim=0 To merge by the first dimension row ) Merge multiple tensors

tensor = torch.ones(3, 3)
t1 = torch.cat([tensor, tensor], dim=0)
print(t1.shape)		# torch.Size([6, 3])
print(t1)

t2 = torch.cat([tensor, tensor, tensor], dim=1)
print(t2.shape)		# torch.Size([3, 9])

---

3. Multiplying tensors

Multiplication includes element multiplication and matrix multiplication ：

• Element multiplication ：*,t1.mul(t2)
• Matrix multiplication ：t1.matmul(t2),t1 @ t2

# element-wise product
t1 = torch.ones(2, 2)
print(t1.mul(t1))
print(t1 * t1)

# matrix multiplication
t1 = torch.ones(size=(2, 3))
print(t1.matmul(t1.T))
print(t1 @ t1.T)

among ,T Transposition of representative matrix

Be careful ：numpy in , Multiply the element by *, Matrix multiply by np.dot(A, B),A.dot(B) and np.matmul(A, B)

---

4. In-place operations

with _ The suffix operator is in-place Operator ：

tensor = torch.rand(2, 2)
tensor.add_(1)					# add
print(tensor)
tensor.copy_(torch.rand(2, 2))	# copy
print(tensor)

That's all ~

More reference PyTorch Learning notes

---

REFERENCE: Tensors