11. Recurrent neural network RNN

RNN(Recurrent Neural Network)

RNN It is called cyclic neural network or recurrent neural network . In the past few years RNN In language recognition , natural language processing , Translation and image description have very good applications .
When dealing with image classification , You can put pictures one by one into the classifier to judge independently . But when dealing with voice and text , You can't pronounce independently , Nor can the text be independent , It needs to be connected and analyzed . Traditional neural networks can't do this .

RNN There is a feedback loop , This feedback loop will send the output information of the last time , As the input of the next time .

The Problem of Long-Term Dependencies:
RNN An important usage is to use previous information to decide the current problem .
Example 1： There is a cloud floating （）
Example 2： I grew up in China ... I can speak fluently （）.

To the rear , The smaller the impact of the front on the back prediction .

Long and short term memory network LSTM

LSTM(Long Short Term Memory):


LSTM The Internet is a special kind of RNN, It just has a more complex structure , stay LSTM The network uses the block Replaced the neurons of the original hidden layer .

The input of each door comes from different signals , Complex composition .

This is an improved LSTM The structure of the network , Generally, it is similar to the previous version , The main reason is that there is an extra forgetting door . Forgetting that the gate operates directly on the cell body , Originally, the I / O gate only controls the inflow and outflow of signals , With this forgetting gate, we can control the value in the cell body , In order to control us, we need to remember this signal , Or slowly forget this signal

The input of each door is the same , But they play different roles , The input gate is responsible for controlling the input signal , The output gate is responsible for the output signal , The forgetting gate is used to decide whether to forget the signal .
In a word ,LSTM More than usual RNN strong , Can achieve better results .