RNN recurrent neural network

Timing data

Give a more strict definition ： Time series data are data columns recorded in time sequence by the same unified index . Data in the same column must be of the same caliber , It requires comparability . Time series data can be periods , You can also count time . The purpose of time series analysis is to find out the statistical characteristics and development regularity of time series in samples , Building time series models , Make out of sample prediction .
Generally speaking , Time series data is the data with sequential relationship ：

1. Specific time point （ moment ） Not important ;
2. Focus only on its successively Relationship

Historical information

How to record historical information is a key problem of time series data .
Remember that all States from the beginning to the present are impossible .

Hidden Markov hypothesis

Simply speaking , The current status is related to the previous status , It has nothing to do with the previous state . That is, the previous state can perfectly record all the previous information .
generally speaking , This assumption is not tenable , The first few or dozens of states of the current state are often used to represent all the previous information . Through this assumption , You can greatly reduce the number of states you need to remember .

Introduce the state function

Another way to reduce what needs to be recorded .
Suppose the last state function is $H_{t-1}$, The current input is $X_t$.
that $H_t=f(H_{t-1},X_t)$.
It's not hard to see. ,$H_t$ In fact, it is constantly $X_t$ Accumulate , In a sense , It records all the previous information .

• This is exactly RNN The basic idea of .

RNN neural network

There are two inputs $X_t,H_{t-1}$ It means that the current time is data and historical information .
There are two outputs $Y_t,H_t$ It respectively represents the output of the current time and the historical information containing the current time information .

Training

Here we need to use the idea of hidden Markov .
A segment is randomly intercepted from the time series to train the model . This sounds a little strange , But it is very feasible in practice . Because there are many rounds of training , One session at a time , Eventually, the whole sequence will be trained .
In contrast, every time you train from scratch , This greatly reduces the number of samples , It is very easy to cause over fitting .

forecast

Give me the first few X, that will do , The initial state is set to 0.