当前位置:网站首页>"Hands on learning in depth" Chapter 2 - preparatory knowledge_ 2.5 automatic differentiation_ Learning thinking and exercise answers
"Hands on learning in depth" Chapter 2 - preparatory knowledge_ 2.5 automatic differentiation_ Learning thinking and exercise answers
2022-07-06 02:31:00 【coder_ sure】
List of articles
2.5. Automatic differentiation
author github link : github link
practice
- Prove a matrix A \mathbf{A} A The transpose of is A \mathbf{A} A, namely ( A ⊤ ) ⊤ = A (\mathbf{A}^\top)^\top = \mathbf{A} (A⊤)⊤=A.
- Two matrices are given A \mathbf{A} A and B \mathbf{B} B, prove “ They are transposed and ” be equal to “ They are transposed with ”, namely A ⊤ + B ⊤ = ( A + B ) ⊤ \mathbf{A}^\top + \mathbf{B}^\top = (\mathbf{A} + \mathbf{B})^\top A⊤+B⊤=(A+B)⊤.
- Given an arbitrary square matrix A \mathbf{A} A, A + A ⊤ \mathbf{A} + \mathbf{A}^\top A+A⊤ Is it always symmetrical ? Why? ?
- We define shapes in this section ( 2 , 3 , 4 ) (2,3,4) (2,3,4) Tensor
X
.len(X)
What is the output of ? - For tensors of arbitrary shape
X
,len(X)
Whether it always corresponds toX
The length of a particular axis ? What is this axis ? - function
A/A.sum(axis=1)
, See what happens . Can you analyze the reason ? - Consider a with a shape ( 2 , 3 , 4 ) (2,3,4) (2,3,4) Tensor , In the shaft 0、1、2 What shape is the summation output on ?
- by
linalg.norm
Function provides 3 Tensors of one or more axes , And observe its output . For tensors of any shape, what does this function calculate ?
Practice reference answers
- Why is it more expensive to calculate the second derivative than the first derivative ?
Because the second derivative is based on the calculation of the first derivative , Therefore, the cost of calculating the second derivative must be greater than that of the first derivative - After running the back propagation function , Run it again now , See what happens .
The complains , about Pytorch Come on , The forward process establishes a calculation diagram , Release after back propagation . Because the intermediate result of the calculation graph has been released , So the second run of back propagation will make an error . At this moment in backward Add parameters to the function retain_graph=True, You can run back propagation twice . - In the case of control flow , We calculated
d
Abouta
The derivative of , If we put variablesa
Change to random vector or matrix , What's going to happen ?
A runtime error has occurred , stay Pytorch in , Don't let the tensor derive from the tensor , Only scalar derivatives of tensors are allowed . If you want to call on a non scalar backward(), You need to pass in a gradient Parameters . - Redesign an example of finding the gradient of control flow , Run and analyze the results .
# When a The norm of is greater than 10 when , Gradient for all elements is 1 Vector ; When a The gradient of is not greater than 10 when , Gradient for all elements is 2 Vector .
def f(a):
if a.norm() > 10:
b = a
else:
b = 2 * a
return b.sum()
a = torch.tensor([1.0, 2.0, 3.0, 4.0, 5.0], requires_grad=True)
d = f(a)
d.backward()
- send f ( x ) = sin ( x ) f(x)=\sin(x) f(x)=sin(x), draw f ( x ) f(x) f(x) and d f ( x ) d x \frac{df(x)}{dx} dxdf(x) Image , The latter does not use f ′ ( x ) = cos ( x ) f'(x)=\cos(x) f′(x)=cos(x).
The latter does not use f ′ ( x ) = cos ( x ) f'(x)=\cos(x) f′(x)=cos(x), The original intention of this problem is to save all the derivative values obtained from the derivation of the function , According to these saved df
value , Draw f ′ ( x ) f'(x) f′(x)
# Import the corresponding library
import numpy as np
import torch
import matplotlib.pyplot as plt
# Make some definitions
x = np.arange(-5, 5, 0.02)# Define the argument in [5,5] Between , Every number interval 0.02
f = np.sin(x)
df = []
for i in x:
# Yes x Find the derivative for every value of
v = torch.tensor(i,requires_grad=True)
y = torch.sin(v)
y.backward()
df.append(v.grad)
# The drawing part
# Create plots with pre-defined labels.
fig, ax = plt.subplots()
ax.plot(x, f, 'k', label='f(x)')
ax.plot(x, df, 'k*', label='df(x)')
legend = ax.legend(loc='upper left', shadow=True, fontsize='x-large')
# Put a nicer background color on the legend.
legend.get_frame().set_facecolor('C0')
plt.show()
Reference material
边栏推荐
- Global and Chinese market of wheelchair climbing machines 2022-2028: Research Report on technology, participants, trends, market size and share
- I changed the driver to 5.1.35, but it is still the same error. I can succeed even now, but I will report this every time I do an SQL operation
- How to generate rich text online
- VIM usage guide
- [Yunju entrepreneurial foundation notes] Chapter II entrepreneur test 12
- 力扣今日题-729. 我的日程安排表 I
- 【机器人库】 awesome-robotics-libraries
- Minecraft 1.16.5 biochemical 8 module version 2.0 storybook + more guns
- [postgraduate entrance examination English] prepare for 2023, learn list5 words
- 2022年版图解网络PDF
猜你喜欢
从顶会论文看2022年推荐系统序列建模的趋势
[Digital IC manual tearing code] Verilog asynchronous reset synchronous release | topic | principle | design | simulation
【无标题】数据库中一条查询SQL执行的过程
RDD conversion operator of spark
Keyword static
Crawler (9) - scrape framework (1) | scrape asynchronous web crawler framework
2022年版图解网络PDF
Minecraft 1.16.5 生化8 模组 2.0版本 故事书+更多枪械
2022 eye health exhibition, vision rehabilitation exhibition, optometry equipment exhibition, eye care products exhibition, eye mask Exhibition
构建库函数的雏形——参照野火的手册
随机推荐
Prepare for the autumn face-to-face test questions
有没有sqlcdc监控多张表 再关联后 sink到另外一张表的案例啊?全部在 mysql中操作
Global and Chinese markets of general purpose centrifuges 2022-2028: Research Report on technology, participants, trends, market size and share
[Yunju entrepreneurial foundation notes] Chapter II entrepreneur test 15
Dachang image library
Minecraft 1.16.5 生化8 模组 2.0版本 故事书+更多枪械
The ECU of 21 Audi q5l 45tfsi brushes is upgraded to master special adjustment, and the horsepower is safely and stably increased to 305 horsepower
Redis delete policy
Zero basic self-study STM32 wildfire review of GPIO use absolute address to operate GPIO
Blue Bridge Cup group B provincial preliminaries first question 2013 (Gauss Diary)
Global and Chinese markets of nasal oxygen tubes 2022-2028: Research Report on technology, participants, trends, market size and share
爬虫(9) - Scrapy框架(1) | Scrapy 异步网络爬虫框架
Crawler (9) - scrape framework (1) | scrape asynchronous web crawler framework
inherited constructors
Httprunnermanager installation (III) - configuring myql Database & initialization data under Linux
[robot hand eye calibration] eye in hand
VIM usage guide
2022年版图解网络PDF
[Yunju entrepreneurial foundation notes] Chapter II entrepreneur test 20
Shell脚本更新存储过程到数据库