9. Introduction to convolutional neural network

Convolutional neural networks

summary

Convolutional neural network is developed in recent years , It is widely used in image processing and NLP And other fields .

Tradition BP Problems in image processing :

• 1. Too many weights , Too much computation

Suppose you use 100X100 Train with pictures of , that 100X100 The size pictures are 10000 Pixels , Then the input of building the network needs 10000 Neurons , The number of hidden layer neurons in the network is similar to that of input layer neurons , In this case , There will be a lot of weights to be trained and updated . The amount of calculation is very large .

• 2. Too many weights , A large number of samples are needed for training

The more neurons , The more parameters , When you have more parameters , Just like solving equations, the more unknown parameters , The more data is needed to solve the unknown parameters .

Since there are these problems, what should we do ？
CNN adopt Local receptive field and Weight sharing The number of parameters needed to be trained in neural network is reduced .

The first picture above shows a fully connected neural network , The second picture above and the first picture below are locally connected neural networks . Locally connected neural networks have fewer parameters than fully connected neural networks .

1. Convolution kernel

filter , The function of convolution kernel can be understood as extracting some different features of pictures , Different convolution kernels can extract different features .

2. Pooling

Pooling Three common ways ：

• max-pooling
  -

• meaning-pooling
  

• stochastic pooling
  

3. Convolution Padding

SAME PADDING
Supplement the outside of the plane 0, After convolution window sampling, a plane with the same size as the original is obtained

VALID PADDING
Not beyond the plane , After convolution window sampling, a plane smaller than the original plane is obtained

4. Pooling Padding

SAME PADDING： May be added to the outside of the plane 0
VALID PADDING： Not beyond the plane

• If there is one 28*28 The plane of the , use 2*2 In steps of 2 Window to it pooling operation ：
  Use SAME PADDING The way , obtain 14*14 The plane of the ;
  Use VALID PADDING The way , obtain 14*14 The plane of the .

• If there is one 2*3 The plane of the , use 2*2 In steps of 2 Window to it pooling operation ：
  
  Use SAME PADDING The way , obtain 1*2 The plane of the ;
  Use VALID PADDING The way , obtain 1*1 The plane of the .

5.LeNET-5 Introduce

LeNET-5 It is one of the earliest convolutional neural networks , Once widely used in Bank of America . The recognition accuracy of handwritten digits is 99% above .