当前位置:网站首页>"Hands on learning in depth" Chapter 2 - preparatory knowledge_ 2.5 automatic differentiation_ Learning thinking and exercise answers
"Hands on learning in depth" Chapter 2 - preparatory knowledge_ 2.5 automatic differentiation_ Learning thinking and exercise answers
2022-07-06 02:31:00 【coder_ sure】
List of articles
2.5. Automatic differentiation
author github link : github link
practice
- Prove a matrix A \mathbf{A} A The transpose of is A \mathbf{A} A, namely ( A ⊤ ) ⊤ = A (\mathbf{A}^\top)^\top = \mathbf{A} (A⊤)⊤=A.
- Two matrices are given A \mathbf{A} A and B \mathbf{B} B, prove “ They are transposed and ” be equal to “ They are transposed with ”, namely A ⊤ + B ⊤ = ( A + B ) ⊤ \mathbf{A}^\top + \mathbf{B}^\top = (\mathbf{A} + \mathbf{B})^\top A⊤+B⊤=(A+B)⊤.
- Given an arbitrary square matrix A \mathbf{A} A, A + A ⊤ \mathbf{A} + \mathbf{A}^\top A+A⊤ Is it always symmetrical ? Why? ?
- We define shapes in this section ( 2 , 3 , 4 ) (2,3,4) (2,3,4) Tensor
X
.len(X)
What is the output of ? - For tensors of arbitrary shape
X
,len(X)
Whether it always corresponds toX
The length of a particular axis ? What is this axis ? - function
A/A.sum(axis=1)
, See what happens . Can you analyze the reason ? - Consider a with a shape ( 2 , 3 , 4 ) (2,3,4) (2,3,4) Tensor , In the shaft 0、1、2 What shape is the summation output on ?
- by
linalg.norm
Function provides 3 Tensors of one or more axes , And observe its output . For tensors of any shape, what does this function calculate ?
Practice reference answers
- Why is it more expensive to calculate the second derivative than the first derivative ?
Because the second derivative is based on the calculation of the first derivative , Therefore, the cost of calculating the second derivative must be greater than that of the first derivative - After running the back propagation function , Run it again now , See what happens .
The complains , about Pytorch Come on , The forward process establishes a calculation diagram , Release after back propagation . Because the intermediate result of the calculation graph has been released , So the second run of back propagation will make an error . At this moment in backward Add parameters to the function retain_graph=True, You can run back propagation twice . - In the case of control flow , We calculated
d
Abouta
The derivative of , If we put variablesa
Change to random vector or matrix , What's going to happen ?
A runtime error has occurred , stay Pytorch in , Don't let the tensor derive from the tensor , Only scalar derivatives of tensors are allowed . If you want to call on a non scalar backward(), You need to pass in a gradient Parameters . - Redesign an example of finding the gradient of control flow , Run and analyze the results .
# When a The norm of is greater than 10 when , Gradient for all elements is 1 Vector ; When a The gradient of is not greater than 10 when , Gradient for all elements is 2 Vector .
def f(a):
if a.norm() > 10:
b = a
else:
b = 2 * a
return b.sum()
a = torch.tensor([1.0, 2.0, 3.0, 4.0, 5.0], requires_grad=True)
d = f(a)
d.backward()
- send f ( x ) = sin ( x ) f(x)=\sin(x) f(x)=sin(x), draw f ( x ) f(x) f(x) and d f ( x ) d x \frac{df(x)}{dx} dxdf(x) Image , The latter does not use f ′ ( x ) = cos ( x ) f'(x)=\cos(x) f′(x)=cos(x).
The latter does not use f ′ ( x ) = cos ( x ) f'(x)=\cos(x) f′(x)=cos(x), The original intention of this problem is to save all the derivative values obtained from the derivation of the function , According to these saved df
value , Draw f ′ ( x ) f'(x) f′(x)
# Import the corresponding library
import numpy as np
import torch
import matplotlib.pyplot as plt
# Make some definitions
x = np.arange(-5, 5, 0.02)# Define the argument in [5,5] Between , Every number interval 0.02
f = np.sin(x)
df = []
for i in x:
# Yes x Find the derivative for every value of
v = torch.tensor(i,requires_grad=True)
y = torch.sin(v)
y.backward()
df.append(v.grad)
# The drawing part
# Create plots with pre-defined labels.
fig, ax = plt.subplots()
ax.plot(x, f, 'k', label='f(x)')
ax.plot(x, df, 'k*', label='df(x)')
legend = ax.legend(loc='upper left', shadow=True, fontsize='x-large')
# Put a nicer background color on the legend.
legend.get_frame().set_facecolor('C0')
plt.show()
Reference material
边栏推荐
- 从顶会论文看2022年推荐系统序列建模的趋势
- PAT甲级 1033 To Fill or Not to Fill
- 更换gcc版本后,编译出现make[1]: cc: Command not found
- [Yunju entrepreneurial foundation notes] Chapter II entrepreneur test 15
- Keyword static
- 我把驱动换成了5.1.35,但是还是一样的错误,我现在是能连成功,但是我每做一次sql操作都会报这个
- Accident index statistics
- 2022 edition illustrated network pdf
- Six stone management: why should leaders ignore product quality
- Template_ Quick sort_ Double pointer
猜你喜欢
力扣今日題-729. 我的日程安排錶 I
Keyword static
[Digital IC manual tearing code] Verilog asynchronous reset synchronous release | topic | principle | design | simulation
Crawler (9) - scrape framework (1) | scrape asynchronous web crawler framework
Ue4- how to make a simple TPS role (II) - realize the basic movement of the role
[Yunju entrepreneurial foundation notes] Chapter II entrepreneur test 16
Httprunnermanager installation (III) - configuring myql Database & initialization data under Linux
UE4 - how to make a simple TPS role (I) - create a basic role
[solution] add multiple directories in different parts of the same word document
3D drawing ()
随机推荐
The ECU of 21 Audi q5l 45tfsi brushes is upgraded to master special adjustment, and the horsepower is safely and stably increased to 305 horsepower
高数_向量代数_单位向量_向量与坐标轴的夹角
[untitled] a query SQL execution process in the database
【无标题】数据库中一条查询SQL执行的过程
[Yunju entrepreneurial foundation notes] Chapter II entrepreneur test 6
爬虫(9) - Scrapy框架(1) | Scrapy 异步网络爬虫框架
729. 我的日程安排表 I / 剑指 Offer II 106. 二分图
RDD partition rules of spark
Crawler (9) - scrape framework (1) | scrape asynchronous web crawler framework
Initial understanding of pointer variables
Minecraft 1.16.5 生化8 模组 2.0版本 故事书+更多枪械
Looking at the trend of sequence modeling of recommended systems in 2022 from the top paper
[Yunju entrepreneurial foundation notes] Chapter II entrepreneur test 14
RDD creation method of spark
2022.02.13
Exness: Mercedes Benz's profits exceed expectations, and it is predicted that there will be a supply chain shortage in 2022
Dachang image library
Building the prototype of library functions -- refer to the manual of wildfire
Sword finger offer 29 Print matrix clockwise
[Yunju entrepreneurial foundation notes] Chapter II entrepreneur test 11