[ESN] learning echo state network

•   The input can be one-dimensional or multi-dimensional
• Symbolic representation :
• Training
• 
• forecast

 

•   The reserve pool has two main functions ： Nonlinear high-dimensional transformations and input data ut The memory of （ It has memory function for historical time related data ）.
• There is a similar relationship between echo state network and kernel function method

  ---

Parameters of echo state network ： Number of neurons in the reserve pool n, Sparsity of the reserve pool ,wr The distribution of non-zero data ,wr The spectral radius of ,win Zoom ratio and leakage rate γ

Parameter selection

• Dimension of reserve pool （ Number of neurons ）n： Consider the number of independent data that the reserve pool must remember to estimate the dimension of the reserve pool n The lower limit of , Independent data refers to the data that the echo state network needs to remember in order to complete the task successfully .. For input data with independent identically distributed , The number of independent data is roughly estimated by multiplying the dimension of the input mutual data by the time step of memory required to solve the problem
• Sparsity of the reserve pool ： Gaussian distribution is widely used to generate the connection weight matrix of the reserve pool wr
• Enter the connection matrix win Scale factor of ：win Subject to uniform distribution , Suppose the scale factor is defined as α, that α∈[-a,a],win From the interval [-a,a] sampling ; if win It follows a normal distribution , Then choose the standard deviation as the scale factor .

（ The reserve pool tends to be eliminated xt in ut The spectrum of the principal component of , So the data is input into the echo state network , You need to choose the correct representation of data or preprocess . If you enter a smaller principal component, it does not carry useful information , They may be removed from the data by principal component analysis before they are input into the reserve pool , Otherwise they will be magnified in the reserve pool ）

• Retention rate γ Is to determine the state of the reserve pool xt Update speed is an important parameter ,γ It can be seen as the time interval discretized between two continuous time steps .
• The connection weight matrix of the reserve pool wr The spectral radius of ： The spectral radius determines how quickly the influence of input on the state of the reserve pool disappears with time , And the stability of the reserve pool . Spectral radius <1 It is not a necessary condition for the echo state attribute , The optimal spectral radius value may sometimes be significantly greater than 1

（ The reserve pool should meet the so-called echo state characteristics ： Status of the reserve pool xt You should enter ut The only certainty . let me put it another way ： For long enough input ut, The final state of the reserve pool xt It should not be related to the initial conditions before input .）