当前位置：网站首页>Parameter calculation of deep learning convolution neural network

Parameter calculation of deep learning convolution neural network

2022-06-30 07:26:00 【Three stinky ginger】

Deep learning convolution neural network parameter calculation

List of articles

Deep learning convolution neural network parameter calculation

This article refers to this article ：https://www.cnblogs.com/touch-skyer/p/9150039.html

（ I didn't want to write it myself , But the formula behind this article is not very clear ）

This article will be based on LeNet-5 This simplest model is demonstrated step by step , And implement it in code , Verify whether the output of the network structure is consistent with the calculated results .

LeNet5 The Internet Yes 7 layer ：

1. The first 1 layer ： Convolution layer

Input ： Original image pixel matrix （ length 、 Width 、 The channel number ）, The size is 32×32×1;

Parameters ： The filter size is 5×5, Depth is 6, Don't use all 0 fill , In steps of 1;

Output ： Characteristics of figure , The size is 28×28×6.

analysis ： Because I didn't use all 0 fill , So the output size = 32 - 5 + 1 = 28, The depth is consistent with the filter depth , by 6.

2. The first 2 layer ： Pooling layer

Input ： Characteristics of figure , The size is 28×28×6;

Parameters ： The filter size is 2×2, In steps of 2;

Output ： Characteristics of figure , The size is 14×14×6.

3. The first 3 layer ： Convolution layer

Input ： Characteristics of figure , The size is 14×14×6;

Parameters ： The filter size is 5×5, Depth is 16, Don't use all 0 fill , In steps of 1;

Output ： Characteristics of figure , The size is 10×10×16.

analysis ： Because I didn't use all 0 fill , So the output size = 14 - 5 + 1 = 10, The depth is consistent with the filter depth , by 16.

4. The first 4 layer ： Pooling layer

Input ： Characteristics of figure , The size is 10×10×16;

Parameters ： The filter size is 2×2, In steps of 2;

Output ： Characteristics of figure , The size is 5×5×6.

5. The first 5 layer ： Fully connected layer

Number of input nodes ：5×5×16 = 400;

Number of parameters ：5×5×16×120+120 = 48120;

Number of output nodes ：120.

6. The first 6 layer ： Fully connected layer

Number of input nodes ：120;

Number of parameters ：120×84+84 = 10164;

Number of output nodes ：84.

7. The first 7 layer ： Fully connected layer

Number of input nodes ：84;

Number of parameters ：84×10+10 = 850;

Number of output nodes ：10.

Because of the use of MNIST Data sets , The picture size is 28×28×1 Of ,1 Represents the number of channels , That is, grayscale images , So the following code is implemented with 28×28 And explain .

It doesn't matter if you don't understand it , Here comes the formula .

One 、 Convolution layer image output size

The definition is as follows ：

O = The size of the output image .

I = Enter the size of the image .

K = The kernel size of convolution layer

N = Nuclear quantity

S = Move step

P = Number of fillings

The formula ：

( Numbers 0 And letters O It's like , Please ignore this little problem )

Example ：MNIST The size of the handwritten digital picture is 28×28×1 Of ,LeNet5 The number of convolution kernels in the first layer is 5, Therefore, the output image size is ：

Therefore, the output image size is ：24×24×6. （ A convolution kernel corresponds to a channel ）

Two 、 Pool layer image output size

The definition is as follows ：

O= The size of the output image .

I= Enter the size of the image .

S= Move step

P_S= Size of pool layer

The formula ：

Example ： The output of the first layer is 24×24×6, Therefore, the output image size is ：

Therefore, the output image size is ：12×12×6. （ The pool layer does not change the number of channels ）

3、 ... and 、 Full connection layer output size

The length of the output vector of the full connection layer is equal to the number of neurons .

Four 、 Number of convolution layer parameters

stay CNN in , Each layer has two types of parameters : The weight weights And bias terms biases. The total number of parameters is all weights and biases The sum of .

The definition is as follows :

W_C = Convolutional weights Number

B_C = Convolutional biases Number

P_C = Number of all parameters

K = Nuclear size

N = Nuclear quantity

C = Number of input image channels

In the convolution layer , The depth of the core is equal to the number of channels of the input image . So each core has K*K Parameters . And there are N A nuclear . The following formula is obtained ：

Example ：LeNet5 The first convolution , The convolution kernel size is 5*5, And there are 6 individual , The input image size is 28×28×1, namely K = 5, C = 1, N = 6, There are ：

So the total parameter quantity of the first layer is ：156.

The pool layer does not calculate parameters .

5、 ... and 、 Number of full connection layer parameters

stay CNN There are two types of full connection layers in . The first 1 One is connected to the last 1 Convolution layers , in addition 1 Species FC Layers are connected to other FC layer . The two cases are discussed separately .

① Connect to the last convolution ：

The definition is as follows :

Wcf= weights The number of

Bcf= biases The number of

O= The size of the output image of the front convolution layer

N = The number of nuclei in the preconvolution

F = Number of neurons in the whole connective layer

The formula ：

Example ： With MNIST Of 28×28×1 Take the input picture of as an example , stay LeNet5 In the first full connection layer of , The output image size of the previous layer is ：4×4×16, The full connection layer has 120 Neurons , namely O = 4, N = 16 , F = 120, There are ：

So there are a total of 30840 Parameters .

② Connect to the previous full connection layer ：

The definition is as follows :

W_ff= weights The number of

B_ff= biases The number of

P_ff= Number of total parameters

F= At present FC The number of neurons in this layer

F_-1 = front FC The number of neurons in this layer

The formula ：

Example ：LeNet5 In the full connection layer 2 layer , The number of neurons in the previous fully connected layer is F_-1 = 120, The number of neurons in the current layer is F = 84, Then there are ：
Insert picture description here

6、 ... and 、 Code implementation and verification

be based on TensorFlow The code implementation is as follows ：

# LeNet5 A network model 
model = tf.keras.Sequential([
    #  The first 1 layer - Convolution layer   Filter size 5*5,6 individual 
    keras.layers.Conv2D(6,5),
    #  The first 2 layer - Pooling layer , The filter size is 2×2, In steps of 2
    keras.layers.MaxPooling2D(pool_size=2,strides=2),
    keras.layers.ReLU(),
    #  The first 3 layer - Convolution layer   Filter size 5*5,16 individual 
    keras.layers.Conv2D(16,5),
    #  The first 4 layer - Pooling layer , The filter size is 2×2, In steps of 2
    keras.layers.MaxPooling2D(pool_size=2,strides=2),
    keras.layers.ReLU(),
    keras.layers.Flatten(), #  After leveling, it is convenient to make full connection 
    #  The first 5 layer - Fully connected layer 
    keras.layers.Dense(120,activation='relu'),
    #  The first 6 layer - Fully connected layer 
    keras.layers.Dense(84,activation='relu'),
    #  The first 7 layer - Fully connected layer 
    keras.layers.Dense(10,activation='softmax') #  The final output 10 class ,0-9 The number of 
])