The BP neural network

1, role

BP neural network is a multi-layer feedforward network trained by error back-propagation algorithm. It is one of the most widely used neural network models.The learning rule of the bp neural network is to use the steepest descent method to continuously adjust the weights and thresholds of the network through backpropagation to minimize the sum of squared errors of the network.

2, input and output description

Input: The independent variable X is one or more categorical or quantitative variables, and the dependent variable Y is a quantitative variable.
Output: The result value of the model output and the model regression effect.

3, case example

There is a batch of Iris flowers. It is known that this batch of Iris flowers can be divided into 3 varieties, which need to be classified.According to the data of sepal length, sepal width, petal length, petal width.Train a neural network to use as a classifier with existing data.

4, modeling steps

BP neural network is a multi-layer feed-forward neural network, its main feature is: the signal is forward-propagated, and the error is back-propagated.Specifically, for the following neural network model with only one hidden layer:
The process of BP neural network is mainly divided into two stages. The first stage is the forward propagation of the signal, from the input layer through the hidden layer,Finally, it reaches the output layer; the second stage is the back-propagation of the error, from the output layer to the hidden layer, and finally to the input layer, adjusting the weights and biases from the hidden layer to the output layer, and the weights from the input layer to the hidden layer in turnand bias.

Take a three-layer BP neural network as an example

The output of the hidden layer is set to Fj, the output of the output layer to m is set to Ok, the excitation function of the system is set to G, and the learning rate is set to β, then the three layers have the following mathematicsRelationship:

If the expected output of the system is set as Tk, the error E of the system can be represented by the variance of the actual output value and the expected target value. The specific relationship is expressed as follows:

And let , using the gradient descent principle, the update formulas of the system weights and biases are as follows: