当前位置:网站首页>Let's see how to realize BP neural network in Matlab toolbox
Let's see how to realize BP neural network in Matlab toolbox
2022-07-07 01:30:00 【Old cake explanation BP neural network】
Catalog
One 、 Source code reproduction effect
3、 ... and 、 Analysis of the source of effect difference
Four 、 Comparison of the effects of different training methods
Original article , Reprint please indicate from 《 Lao Bing explains -BP neural network 》bp.bbbdata.com
If we directly use the gradient descent method to solve BP neural network ,
Often there is no matlab The effect of the toolbox is so good .
This problem has troubled the author for a long time ,
Then we might as well dig out the source code and have a look ,matlab How the toolbox is implemented BP Neural network ,
Why is there no toolbox for our self written training effect BP Neural network is good .
One 、 Source code reproduction effect
Pick out matlab Toolbox gradient drops traingd Algorithm source code , After sorting out the algorithm process ,
One obtained by writing code 2 Cryptic layer BP The weight of the neural network
Call the toolbox newff The weight obtained :
You can see , The two results are the same , It shows that you fully understand and reproduce the BP Neural network training logic .
Two 、 Training mainstream
BP The main process of neural network gradient descent method is as follows
First initialize the weight threshold ,
Then iterate the weight threshold with gradient ,
If the termination conditions are met, quit training
The termination condition is : The error has reached the requirement 、 The gradient is too small or reaches the maximum number
The code is as follows :
function [W,B] = traingdBPNet(X,y,hnn,goal,maxStep)
%------ Variable pre calculation and parameter setting -----------
lr = 0.01; % Learning rate
min_grad = 1.0e-5; % Minimum gradient
%--------- initialization WB-------------------
[W,B] = initWB(X,y,hnn); % initialization W,B
%--------- Start training --------------------
for t = 1:maxStep
% Calculate the current gradient
[py,layerVal] = predictBpNet(W,B,X); % Calculate the predicted value of the network
[E2,E] = calMSE(y,py); % Calculation error
[gW,gB] = calGrad(W,layerVal,E); % Calculate the gradient
%------- Check whether the exit conditions are met ----------
gradLen = calGradLen(gW,gB); % Calculate the gradient value
% If the error has reached the requirement , Or the gradient is too small , Then quit training
if E2 < goal || gradLen <=min_grad
break;
end
%---- Update weight threshold -----
for i = 1:size(W,2)-1
W{i,i+1} = W{i,i+1} + lr * gW{i,i+1};% Update gradient
B{i+1} = B{i+1} + lr * gB{i+1};% Update threshold
end
end
end
( In the code reproduction here, we shield the normalization processing 、 Generalization verifies the general operation of these two algorithms )
3、 ... and 、 Analysis of the source of effect difference
Source of effect difference
There is no difference between the main process and the conventional algorithm tutorial ,
So why matlab The result will be better ,
The main reason is initialization ,
A lot of tutorials , Random initialization is recommended ,
But in fact ,matlab The tool box uses nguyen_Widrow Method to initialize
nguyen_Widrow Law
nguyen_Widrow The idea of method initialization is as follows :
Take the single input network as an example , It initializes the network in the following form :
Its purpose is to make all hidden nodes evenly distributed in the range of input data .
The reason is , If BP Finally, every neuron of neural network is fully utilized ,
Then it should be more similar to the above distribution ( Full coverage of input range 、 Every neuron is fully utilized ),
Instead of randomly initializing and slowly adjusting , It's better to give such an initialization at the beginning .
The original text of this method is :
Derrick Nguyen and Bernard Widrow Of 《Improving the learning Speed of 2-Layer Neural Networks by Choosing Initial Values of the Adaptive Weights 》
Four 、 The effect of different training methods is different
Effect comparison
And the author uses traingd、traingda、trainlm Compare the effect ,
Find the same problem ,
traingd Training will not come out ,traingda Can train ,
and traingda Training will not come out ,trainlm Can train again .
That is, in the training effect
traingd< traingda < trainlm
that , If we directly use the gradient descent method written by ourselves ,
It is still far inferior to what we use matlab The tool box works well .
matlab Of BP The default neural network is trainlm Algorithm
A brief account of the cause
that traingda Why is it better than traingd Strong ,trainlm Why is it better than that traingda Strong ?
After source code analysis , Mainly traingda Adaptive learning rate is added in ,
and trainlm It uses the information of the second derivative , Make learning faster .
5、 ... and 、 Related articles
See :
《 rewrite traingd Code ( Gradient descent method )》
See :
This is it. matlab The algorithm logic of gradient descent method in neural network toolbox , It's so simple ~!
边栏推荐
猜你喜欢
Make Jar, Not War
字节P7专业级讲解:接口测试常用工具及测试方法,福利文
2022 Google CTF SEGFAULT LABYRINTH wp
Yunna - work order management system and process, work order management specification
Send template message via wechat official account
微信公众号发送模板消息
对C语言数组的再认识
云呐-工单管理制度及流程,工单管理规范
How to manage distributed teams?
子网划分、构造超网 典型题
随机推荐
搭建【Redis in CentOS7.x】
Neon Optimization: an optimization case of log10 function
[advanced C language] 8 written questions of pointer
Instructions for using the domain analysis tool bloodhound
JTAG debugging experience of arm bare board debugging
JS reverse -- ob confusion and accelerated music that poked the [hornet's nest]
uva 1401 dp+Trie
Make Jar, Not War
Dark horse notes - exception handling
Start from the bottom structure to learn the customization and testing of fpga---- FIFO IP
云呐-工单管理制度及流程,工单管理规范
Make Jar, Not War
NEON优化:log10函数的优化案例
数据手册中的词汇
身体质量指数程序,入门写死的小程序项目
Neon Optimization: summary of performance optimization experience
go-zero微服务实战系列(九、极致优化秒杀性能)
C# 计算农历日期方法 2022
Typical problems of subnet division and super network construction
C language instance_ four