Second week of postgraduate freshman training: convolutional neural network foundation

part 1： Video learning

        Learning content specialized courses 《 The basis of convolutional neural network 》, The main contents include CNN Basic structure and typical neural network structure

One 、 The basic structure of convolutional neural network

        1. Convolution layer

        Basic concepts defined in convolution layer ：

        

         Convolution neural network operation ：

      

        Convolution layer in CNN Its main function is feature extraction of local relations .

        2. Pooling layer

        The role of pooling layer in convolutional neural network ： Retain the main features while reducing parameters and computation , Prevent over fitting , Improve the model generalization ability

        

        Pooling The type of ：

                （1）MaxPooling    (2)AveragePooling

        3. Fully connected layer

        

        The full connection layer is in CNN It mainly plays a role in learning and classifying the local features extracted from the convolution layer .

Two 、 Classic structure of convolutional neural network ：

        1. The history of convolutional neural networks

        

        2.AlexNet

        

                 AlexNet yes CNN The foundation work of , Lead CNN And the upsurge of in-depth learning and research

                Two pieces are used in the network GPU Parallel operation of data , Finally through FC Layer integration

        3.VGG

        

                Big depth , Number of large parameters , Neural network structure of large operation

        4.GoogleNet

                

          introduce inception block , Multiple convolution kernels are used for feature extraction

        in addition , The full connection layer is not used , The number of parameters in the neural network is greatly reduced

        5.ResNet

        Introduce residual thought , The effect is excellent in training deep level network

                

PART 2： Code experiments

One 、 utilize CNN Realization MNIST Classification of handwritten data sets ：

        1. Neural network construction

        Fully connected layer ：

        

        Convolution layer ：

        

        2. Training results of fully connected neural network and convolutional neural network ：

 

        CNN The accuracy of image recognition is much higher than that of fully connected neural network

        3. Recognize the image data after shuffling

        Scrambled image data ：

         

        4. Training results of fully connected neural network and convolutional neural network ：

         

         CNN Unable to extract the local features of the image , The classification result is not good .

Two 、 Use CNN Yes CIFAR10 Data sets are classified

        1. To build the network ：

        

        2. Training models

        

        3. Picture classification results

         

3、 ... and 、 Use VGG16 Yes CIFAR10 classification

        1. Load data set

        

          Different from Experiment 2 , Here the RandomCrop(),RandomHorizontalFlip(), Preprocess the data

        2. Build a neural network model

        

        3. model training

        

        4. Test set classification accuracy

        

        Classification accuracy is improved 20 Multiple percentage points , Simple VGG The effect of network model is far better than that of simple LeNetCNN A network model .

PART 3： I want to answer

        1.dataloader Inside shuffle What's the difference between taking different values ？

        shuffle Can be taken as "true" perhaps "false", Indicates whether to shuffle the data , You usually need to shuffle the training set , The test set can be undisturbed

        2.transform  in , Different values are taken , What's the difference between this ？

        transform It is a commonly used image preprocessing method , Including normalization 、 Random clipping, etc , Improve the generalization ability . Code exercises use ：

        transforms.ToTensor() The function is to convert into tensor Format , This format can be directly input into the neural network ;
        transforms.Normalize() Is to normalize the pixel value , Make the data obey the mean value of 0, The standard deviation is 1 The distribution of ;
        transforms.RandomCrop() It is to crop the image at a random position and return a new image ;
        transforms.RandomHorizontalFlip() It's based on probability p Flip the picture horizontally ,p Default 0.5;
        transforms.Compose() Is a series of transforms Orderly combination , When implementing, operate the image in sequence according to these methods .

        3.epoch and batch The difference between ？

        epoch and batch They are all super parameters in neural network training ,epoch Represents the number of rounds of neural network training ,batch Used to define the number of samples to be processed before updating internal model parameters , once epoch At least one must be trained batch.

        4.1x1 Convolution sum of FC What's the difference? ？ What role does it play ？

        FC It can be regarded as global convolution ,1*1 Convolution can replace FC,FC It mainly plays the role of linear change and classification ,1*1 Usually used to achieve dimensionality reduction , Used as nonlinear variation

        5.residual leanring Why can we improve the accuracy ？

        Solved the problem of gradient disappearance  

        6. Code exercise 2 , The Internet and 1989 year Lecun Proposed LeNet What's the difference? ？

         The activation function in code exercise 2 is ReLU, and LeNet The activation function is Sigmoid. The network structure in code 2 is usually = It's called LeNet-5

        7. Code exercise 2 , After convolution feature map It's going to get smaller , How to apply Residual Learning?

         The original image is reduced to sum by linear transformation feature map Images of the same size . When the input and output dimensions rise, there are two methods ： The first is to still use identity mapping , The extra channels are filled with zero matrix , The advantage of this is that it will not bring additional parameters ; The second is to add transformation equations , Generally speaking, I will use 1*1 Convolution to complete dimension upgrading

        8. What methods can further improve the accuracy ？

        （1） Improve the network structure         （2） Select the appropriate optimizer         （3） Choose the appropriate loss function         （4） Select the appropriate activation function         （5） Increase network depth         （6） Increase the number of training rounds         （7） Use a larger data set

                

 

 

 

        

 

 

 

 

        

 

 

        

    

        2.