当前位置:网站首页>Complete linear regression manually based on pytoch framework
Complete linear regression manually based on pytoch framework
2022-07-07 08:07:00 【Students who don't want to be bald】
Pytorch Complete linear regression
hello, Dear friends, I haven't seen you for a long time , I am busy with the final exam recently , Continue to update our after now Pytorch Frame learning notes
The goal is
- know
requires_grad
The role of - Know how to use
backward
- Know how to do linear regression manually
1. Forward calculation
about pytorch One of them tensor, If you set its properties .requires_grad
by True
, Then it will track all operations on the tensor . Or it can be understood as , This tensor It's a parameter , The gradient will be calculated later , Update this parameter .
1.1 The calculation process
Suppose the following conditions (1/4 Mean value ,xi There is 4 Number ), Use torch The process of completing its forward calculation
KaTeX parse error: No such environment: align* at position 8: \begin{̲a̲l̲i̲g̲n̲*̲}̲ &o = \frac{1}{…
If x Is the parameter , The gradient needs to be calculated and updated
that , Set randomly at the beginning x The value of , Need to set his requires_grad The attribute is True, Its The default value is False
import torch
x = torch.ones(2, 2, requires_grad=True) # Initialize parameters x And set up requires_grad=True Used to track its computing history
print(x)
#tensor([[1., 1.],
# [1., 1.]], requires_grad=True)
y = x+2
print(y)
#tensor([[3., 3.],
# [3., 3.]], grad_fn=<AddBackward0>)
z = y*y*3 # square x3
print(x)
#tensor([[27., 27.],
# [27., 27.]], grad_fn=<MulBackward0>)
out = z.mean() # Calculating mean
print(out)
#tensor(27., grad_fn=<MeanBackward0>)
As can be seen from the above code :
- x Of requires_grad The attribute is True
- Each subsequent calculation will modify its
grad_fn
attribute , Used to record operations done- Through this function and grad_fn It can form a calculation diagram similar to the previous section
1.2 requires_grad and grad_fn
a = torch.randn(2, 2)
a = ((a * 3) / (a - 1))
print(a.requires_grad) #False
a.requires_grad_(True) # Modify in place
print(a.requires_grad) #True
b = (a * a).sum()
print(b.grad_fn) # <SumBackward0 object at 0x4e2b14345d21>
with torch.no_gard():
c = (a * a).sum() #tensor(151.6830), here c No, gard_fn
print(c.requires_grad) #False
Be careful :
To prevent tracking history ( And using memory ), You can wrap code blocks in with torch.no_grad():
in . Especially useful when evaluating models , Because the model may have requires_grad = True
Trainable parameters , But we don't need to calculate the gradient of them in the process .
2. Gradient calculation
about 1.1 Medium out for , We can use backward
Method for back propagation , Calculate the gradient
out.backward()
, Then we can find the derivative d o u t d x \frac{d out}{dx} dxdout, call x.gard
Can get the derivative value
obtain
tensor([[4.5000, 4.5000],
[4.5000, 4.5000]])
because :
d ( O ) d ( x i ) = 3 2 ( x i + 2 ) \frac{d(O)}{d(x_i)} = \frac{3}{2}(x_i+2) d(xi)d(O)=23(xi+2)
stay x i x_i xi be equal to 1 The value is 4.5
Be careful : When the output is a scalar , We can call the output tensor
Of backword()
Method , But when the data is a vector , call backward()
You also need to pass in other parameters .
Many times our loss function is a scalar , So we won't introduce the case where the loss is a vector .
loss.backward()
Is based on the loss function , For parameters (requires_grad=True) To calculate his gradient , And add it up and save it to x.gard
, Its gradient has not been updated at this time
Be careful :
tensor.data
:stay tensor Of require_grad=False,tensor.data and tensor Equivalent
require_grad=True when ,tensor.data Just to get tensor Data in
tensor.numpy()
:require_grad=True
Cannot convert directly , Need to usetensor.detach().numpy()
3. Linear regression realizes
below , We use a custom data , To use torch Implement a simple linear regression
Suppose our basic model is y = wx+b
, among w and b All parameters , We use y = 3x+0.8
To construct data x、y, So finally, through the model, we should be able to get w and b Should be close to 3 and 0.8
- Prepare the data
- Calculate the predicted value
- Calculate the loss , Set the gradient of the parameter to 0, Back propagation
- Update parameters
import torch
import numpy as np
from matplotlib import pyplot as plt
#1. Prepare the data y = 3x+0.8, Prepare parameters
x = torch.rand([50])
y = 3*x + 0.8
w = torch.rand(1,requires_grad=True)
b = torch.rand(1,requires_grad=True)
def loss_fn(y,y_predict):
loss = (y_predict-y).pow(2).mean()
for i in [w,b]:
# Set the gradient to... Before each back propagation 0
if i.grad is not None:
i.grad.data.zero_()
# [i.grad.data.zero_() for i in [w,b] if i.grad is not None]
loss.backward()
return loss.data
def optimize(learning_rate):
# print(w.grad.data,w.data,b.data)
w.data -= learning_rate* w.grad.data
b.data -= learning_rate* b.grad.data
for i in range(3000):
#2. Calculate the predicted value
y_predict = x*w + b
#3. Calculate the loss , Set the gradient of the parameter to 0, Back propagation
loss = loss_fn(y,y_predict)
if i%500 == 0:
print(i,loss)
#4. Update parameters w and b
optimize(0.01)
# Drawing graphics , Observe the predicted value and real value at the end of training
predict = x*w + b # Use the trained w and b Calculate the predicted value
plt.scatter(x.data.numpy(), y.data.numpy(),c = "r")
plt.plot(x.data.numpy(), predict.data.numpy())
plt.show()
print("w",w)
print("b",b)
The graphic effect is as follows :
[ Failed to transfer the external chain picture , The origin station may have anti-theft chain mechanism , It is suggested to save the pictures and upload them directly (img-EPoO4Eha-1656763156468)(…/images/1.2/ Linear regression 1.png)]
Print w and b, Can have
w tensor([2.9280], requires_grad=True)
b tensor([0.8372], requires_grad=True)
You know ,w and b Already very close to the original preset 3 and 0.8
The graphic effect is as follows :
[ Outside the chain picture transfer in ...(img-EPoO4Eha-1656763156468)]
Print w and b, Can have
```python
w tensor([2.9280], requires_grad=True)
b tensor([0.8372], requires_grad=True)
You know ,w and b Already very close to the original preset 3 and 0.8
边栏推荐
- LeetCode简单题之字符串中最大的 3 位相同数字
- Cnopendata list data of Chinese colleges and Universities
- [UVM basics] summary of important knowledge points of "UVM practice" (continuous update...)
- DNS server configuration
- 【数字IC验证快速入门】14、SystemVerilog学习之基本语法1(数组、队列、结构体、枚举、字符串...内含实践练习)
- Cnopendata American Golden Globe Award winning data
- 青龙面板--花花阅读
- Niu Mei's mathematical problem --- combinatorial number
- 2022茶艺师(初级)考试题模拟考试题库及在线模拟考试
- Rust versus go (which is my preferred language?)
猜你喜欢
【数字IC验证快速入门】13、SystemVerilog interface 和 program 学习
让Livelink初始Pose与动捕演员一致
The charm of SQL optimization! From 30248s to 0.001s
Implementation of replacement function of shell script
Record a stroke skin bone error of the skirt
Network learning (II) -- Introduction to socket
Custom class loader loads network class
Most elements
The zblog plug-in supports the plug-in pushed by Baidu Sogou 360
青龙面板-今日头条
随机推荐
【数字IC验证快速入门】12、SystemVerilog TestBench(SVTB)入门
[matlab] when matrix multiplication in Simulink user-defined function does not work properly, matrix multiplication module in module library can be used instead
QT learning 26 integrated example of layout management
让Livelink初始Pose与动捕演员一致
海信电视开启开发者模式
[quick start of Digital IC Verification] 15. Basic syntax of SystemVerilog learning 2 (operators, type conversion, loops, task/function... Including practical exercises)
Visualization Document Feb 12 16:42
【数字IC验证快速入门】11、Verilog TestBench(VTB)入门
【数字IC验证快速入门】14、SystemVerilog学习之基本语法1(数组、队列、结构体、枚举、字符串...内含实践练习)
The principle and implementation of buffer playback of large video files
Custom class loader loads network class
Explore dry goods! Apifox construction ideas
太真实了,原来自己一直没有富裕起来是有原因的
Roulette chart 2 - writing of roulette chart code
ZCMU--1492: Problem D(C语言)
Paddlepaddle 29 dynamically modify the network structure without model definition code (relu changes to prelu, conv2d changes to conv3d, 2D semantic segmentation model changes to 3D semantic segmentat
【数字IC验证快速入门】17、SystemVerilog学习之基本语法4(随机化Randomization)
2022 recurrent training question bank and answers of refrigeration and air conditioning equipment operation
快解析内网穿透为文档加密行业保驾护航
2022 Inner Mongolia latest advanced fire facility operator simulation examination question bank and answers