Neural network

人工神经网络

Artificial neurons are the basic building blocks of artificial neural network elements,如下图所示,$={\left[x_1,x_2,x_3...x_m\right]}^T,W={\left[w_1,w_2,...w_m\right]}^T$For the connection right,So the network input$u={\sum }_{i=1}^m{w}_i{x}_i$,其向量形式为$u=W^{T}X$

The picture above for single-layer perceptron neural model,其中mAs the number of neurons in input
$v={\sum }_{i=1}^m{w}_i{x}_i,y=\{\begin{array}{l}1v\ge 0\\ 0v<0\end{array}$

Activation function also by excitation function,活化函数,Used to perform the neurons of network input transformation,一般有以下四种：

1. 线性函数$f(u)=ku+c$

2. 非线性斜面函数$$f\left(u\right)=\{\begin{array}{l}\gamma ,u\ge \theta \\ ku,\left|u\right|<\theta \\ -\gamma ,u\le -\theta \end{array}$$
   其中$\theta ,\gamma$为非负实数,$\gamma$Referred to as the saturation value,即$\gamma$As the biggest output neurons

3. 阈值函数/阶跃函数
   $$f\left(u\right)=\{\begin{array}{l}\beta ,u>\theta \\ -\gamma ,u\le \theta \end{array}$$

4. 在logisticReturn to have been introduced insigmoid函数,This function will range minus infinity to infinite is mapped to the（0,1）
   sigmoid函数的公式为：
   $$f\left(u\right)=\frac{1}{1+e^{-u}}$$

5. tanhFunction about educationsigmoid函数要常见一些,This function will range minus infinity to infinite is mapped to the（-1,1）,其公式为：
   $$f\left(u\right)=\frac{e^u-e^u}{e^u+e^{-u}}$$

例：
Using a single perceptron neural solve the problem of the classification of simple：There are two types of four input vector,Two vector corresponding to the target value as1,The other two vector corresponding to the target value as0,The input vector matrix：
$$\left[\begin{array}{cccc}-0.5& -0.5& 0.3& 0\\ -0.5& 0.5& -0.5& 1\end{array}\right]$$
其中每一列1Column is the value of an input,Target classification and vectorT=[1,1,0,0].Try to predict new input vector$p={\left[-0.5,0.2\right]}^T$的目标值:

from sklearn.linear_model import Perceptron
import numpy as np
x0=np.array([[-0.5,-0.5,0.3,0.0],[-0.5,0.5,-0.5,1.0]]).T
y0=np.array([1,1,0,0])
md=Perceptron().fit(x0,y0)#Construction and fitting model
print("Model coefficients and constant term respectively：",md.coef_,",",md.intercept_)
print("模型精度：",md.score(x0,y0))#模型检验
print("预测值为：",md.predict([[-0.5,0.2]]))

Remember two indicators variables respectively$x_1,x_2$,For the classification function was obtained$v=-1.3x_1-0.5x_2$.New input vectorp的目标值为1

BP神经网络

BPNeural network is the biggest advantage is with strong nonlinear mapping ability,He is mainly used in the following four aspects:

1. 函数逼近.With the input vector and the corresponding output vector training a network to approximate a function
2. 模式识别
3. 预测
4. 数据压缩

BPNeural network specific process is as follows：
（1） 初始化,To the connection weight and threshold is given[-1,1]的随机值
（2） Choose a random pattern of$X_0=\left[x_1^0,x_2^0....x_n^0\right],Y_0=\left[y_1^0,y_2^0,...y_n^0\right]$提供给网络

(3)用输入模式、连接权,And the value,计算中间层各单元的输入,然后用sThe middle layer through the calculation of living function of each unit output.
(4)With the output of the middle layer;连接权.And the broad terms of input output layer units,然后用.After activation function calculating the response of the output layer unitsd.
(5)With hope output mode、Network actual output calculation of generalization error output layer units..
(6)With connection powerg、Output layer of generalization error、中间层输出,Calculate the middle tier units generalization error
Yuan generalization errorc、The middle tier units output,修正连接权u,和阈值.Every unit input,Fixed connection weight and threshold.
（7）重新从mA learning model of randomly selected from a pattern of,Namely return step3,Until the network global error functionE小于预先设定的一个极小值,即网络收敛;Or study number is greater than the preset value,The network can't convergence