Parameters of convolutional neural network

The error direction propagation master proposed in order to deal with the huge amount of calculation of partial derivatives . But the gradient descent method is still the foundation .

One 、 Parameters required by convolution neural network

Filter example of convolution layer ：

Unified offset of convolution layer ： $b_{}^{F1}$
Output layer weight ： $w_{1-11}^{O1}$
Output layer offset ： $b_1^{O1}$
The basic formula of gradient descent method ：$$\left(\Delta w_{11}^{F1},\cdots ,\Delta w_{1-11}^{O1},\cdots ,\Delta b_1^2,\cdots \right)=-\eta \left(\frac{\partial C_T}{\partial w_{11}^{F1}},\cdots ,\frac{\partial C_T}{\partial w_{1-11}^{O1}},\cdots ,\frac{\partial C_T}{\partial b^{F1}},\cdots ,\frac{\partial C_T}{\partial b_1^O},\cdots \right)$$

Two 、 Using neural unit error ${\delta }_j^l$ To represent the gradient component of each parameter

1、 Output layer error
$$\frac{\partial C}{\partial w_{k-ij}^{O_n}}={\delta }_{ij}^O{a}_{ij}^{Pk},\frac{\partial C}{\partial b_n^O}={\delta }_n^O$$

among n Label the neural unit of the output layer ,k Number the sublayer of the pool layer ,i、j Is the row of the filter 、 Column number . first ： Represents the output layer n Number of neurons k Layer pool layer i-j Neural unit error of output neuron weight . the second ： The first n Neural unit error of bias of neurons
2、 Error of convolution layer
A、 Convolution layer filter weight
$$\frac{\partial C}{\partial w_{ij}^{F_k}}={\delta }_{11}^{F_k}{x}_{ij}+{\delta }_{12}^{F_k}{x}_{ij+1}+\cdots +{\delta }_{44}^{F_k}{x}_{i+3j+3}$$
This is the number of pixels 6x6、 The filter for 3x3 The relation of . In other cases , We need to make corresponding changes according to the actual situation . It means the first one k The first layer of convolution i-j Neural unit error .
B、 Unified offset weight of convolution layer
$$\frac{\partial C}{\partial b_{}^{F_k}}={\delta }_{11}^F+{\delta }_{12}^{F_k}+\cdots +{\delta }_{33}^{F_k}+\cdots +{\delta }_{44}^{F_k}$$

There is only one uniform offset for each convolution . It means the first one k The offset of the convolution layer obtained by the layer filter layer .

3、 ... and 、 How to calculate the output layer ${\delta }_j^l$

The activation function of the output layer is a(z),n Number the neural units of this layer .
KaTeX parse error: Expected group after '^' at position 56: …O}-t_n\text{）}a^̲'\left( z_{n}^{…
Represents the output layer n Neural unit error of neurons .