Hebb and delta learning rules

Link to the original text ：https://blog.csdn.net/u012562273/article/details/56297648

Unsupervised learning rules

Donald · Herb （1904-1985） He is a famous physiological psychologist in Canada .Hebb Learning rules and “ Conditioned reflex ” The mechanism is consistent , And it has been confirmed by the theory of nerve cells .
  Pavlov's conditioned reflex experiment ： Ring the bell before feeding the dog , A long time , The dog will associate the bell with food . In the future, if you ring the bell but don't give food , Dogs drool .
  Inspired by the experiment ,Hebb The theory is that the connection between neurons excited at the same time will be strengthened . such as , When the bell rings, a neuron is activated , At the same time, the presence of food triggers another neuron nearby , Then the connection between the two neurons will be strengthened , So remember that there is a connection between the two things . contrary , If two neurons don't always fire synchronously , Then the connection between them will be weaker and weaker .
 Hebb The law of learning can be expressed as ：
Wij(t+1)=Wij(t)+a⋅yi⋅yj

among WijWij Represents a neuron jj To neurons ii The right to connect ,yiyi And yjyj Represents the output of two neurons ,aa Is a constant representing the learning rate , If yiyi And yjyj Activated at the same time , namely yiyi And yjyj At the same time, it is positive , that wijwij Will increase . If yiyi To be activated , and yjyj In a suppressed state , namely yiyi Being positive yjyj Negative , that wijwij Will be smaller .

Supervised learning rules

Delta Learning rules are a simple supervised learning algorithm , The algorithm adjusts the connection weight according to the difference between the actual output and the expected output of the neuron , Its mathematical expression is as follows ：
Wij(t+1)=Wij(t)+a⋅(di−yi)xj(t)

among WijWij Represents a neuron j To neurons i The right to connect ,didi It's neurons ii Expected output ,yiyi It's neurons i Actual output of ,xjxj Represents a neuron jj state , If neurons j If it is active xjxj by 1, If it is in the inhibited state, then xjxj by 0 or -1（ Depending on the activation function ）.aa Is a constant representing the learning speed . hypothesis xixi by 1, if didi Than yiyi Big , that WijWij Will increase , if didi Than yiyi Small , that WijWij Will be smaller .
 Detla The rule is simply ： If the actual output of the neuron is greater than the expected output , Then reduce the weight of the connection whose input is positive , Increase the weight of all connections with negative input . conversely , Then increase the weight of all connection weights with positive input , Reduce the weight of all negative input connection weights .

————————————————
Copyright notice ： This paper is about CSDN Blogger 「 Smear 」 The original article of , follow CC 4.0 BY-SA Copyright agreement , For reprint, please attach the original source link and this statement .
Link to the original text ：https://blog.csdn.net/u012562273/article/details/56297648