The third week of postgraduate freshman training: resnet+resnext

PART 1： Video learning

One 、ResNet

        1.ResNet Network innovation

        2.ResNet The question of the outcome

        3.ResNet-34 Network structure

        4.ResNet Network structure

        5.Residual structure

        In the lower layers ResNet And deep ResNet The network structure used

        Residual Calculation principle ：

        

         6. Network training results

        In general neural networks , The deeper the network, the worse the effect

        ResNet:

        

Two 、Batch Nomalization        

        BN Its function is to accelerate the convergence of the network and improve the accuracy , The purpose is to make every batch Of feature map The mean is 0, The variance of 1 Distribution law of    

3、 ... and 、 The migration study

        1. Advantages of transfer learning

     

        2. Common transfer learning methods

Four 、ResNeXt

        1. Group convolution

        The main difference is ResNet The grouping convolution is added on the basis of , The parameter quantity is further reduced

                2. Network structure

        Convolution block structure :

                

PART 2： Code experiments

One 、 Experimental process

        1. Introduce modules

        2. Super parameter setting

        3. Data preprocessing

         4. Load data

        5. Instantiation model

        6. Training , test

        Training ：

          test ：

        Learning rate adjustment ：

        Training ：

        Output csv file ：

Two 、 Experimental questions

        When doing neural network training, it is mainly built on this machine pytorch platform , Because the function of computer graphics card is not ideal ,batch_size and epoch And other parameters cannot be set too large , Large data sets and large parameter model training often explode .

The experimental effect is not ideal , It has not been used due to network problems colab, Next week, we will focus on solving the problem of the experimental platform , Retraining ResNet The Internet , Get better experimental results .

PART 3： I want to answer

        1.Residual learning

                Solve the gradient explosion in deep network / Gradient disappears and accuracy decreases （ Focus on training ） The problem of , Deepen the depth of neural network .

        2.Batch Normalization

                BN Its function is to accelerate the convergence of the network and improve the accuracy , The purpose is to make every batch Of feature map The mean is 0, The variance of 1 Distribution law of .

        3. Why can grouping convolution improve accuracy ？ That is, grouping convolution can improve the accuracy , At the same time, it can also reduce the amount of calculation , Can't you try to have as many scores as possible ？

                (1) Reduce the number of parameters , Amount of computation , It can train deeper Networks

                (2) Group as many as possible into DW Convolution , It is easy to split the feature relationship in each feature map , The result is not good

                (3) Appropriate convolution mode and parameter setting should be selected according to the model and calculation amount