当前位置：网站首页>Backpropagation formula derivation [Li Hongyi deep learning version]

Backpropagation formula derivation [Li Hongyi deep learning version]

2022-07-04 02:27:00 【Zi Yan Ruoshui】

Introducing problems

The parameters in general neural network $w^l_{i,j}$ To many, many , Calculate each... One by one $\partial C^r/w^l_{i,j}$ , unrealistic .

Therefore, an efficient method for calculating the partial differential of each parameter is introduced ,--Backpropogation.

Rely on your mathematical knowledge

The derivative target is ：

according to chain rule, Split the derivative target into two parts of the product

The first part ：

Record the output activation result of each layer through forward propagation , You can find out. .

The second part ： hold $\partial C^r/\partial{z^l_i}$ Planning $\delta ^l_i$ , Put... On the same floor $\delta ^l_i$ Put together as a vector $\delta ^l$ , In the middle of the $\delta ^l$ Can be determined by the last $\delta ^L$ , and $\delta ^l$ And $\delta ^{l+1}$ The relationship between .

Next, please $\delta ^L$ ：

Next, please $\delta ^l$ And $\delta ^{l+1}$ The relationship between ：

Original video link ：http://speech.ee.ntu.edu.tw/~tlkagk/courses/MLDS_2015_2/Lecture/DNN%20backprop.ecm.mp4/index.html

版权声明
本文为[Zi Yan Ruoshui]所创，转载请带上原文链接，感谢
https://yzsam.com/2022/02/202202141826277099.html

边栏推荐

猜你喜欢

随机推荐