Does neural network sound tall? Take you to train a network from scratch (based on MNIST)

1 What is neural network ？

We know that there are three main schools in the development of artificial intelligence ：

• Semiotic school
• The connectionist school
• Behaviorism school

The connectionist school believes that ： Trillions of neurons in the human brain are intricately interconnected , It is the source of intelligence . The core of connectionism is bionics and neuroscience , It focuses on the connection mechanism and learning algorithm between neural networks , Dedicated to representing a large number of neurons through computers , To simulate the intelligence of the brain . This is the origin of neural networks , Its formal definition is as follows ：

Artificial neural network (Artificial Neural Networks,ANNs) Also referred to as neural network or connection model , It's a way of mimicking the behavioral characteristics of animal neural networks , Mathematical model of distributed parallel information processing algorithm . This network depends on the complexity of the system , By adjusting the interconnecting relationships between a large number of internal nodes , In order to achieve the purpose of processing information .

As shown in the figure below, it is a convolutional neural network framework for graphics and image processing , Today, start training from scratch on the following network .

2 Convolutional neural networks

Convolutional neural networks (Convolutional Neural Network, CNN) Feature extraction layer is added on the basis of fully connected neural network , It is mainly used in the field of computer vision , Processing pattern recognition 、 Image classification 、 Target detection and so on .

CNN comparison FCNN The reason why it is more suitable for visual tasks is , Its implementation Aggregation and compression of high-dimensional information .

For example , Take the pixels of a two-dimensional picture as input , be FCNN The number of neurons in the input layer will be very large , Then consider that each neuron is connected to all neurons in the adjacent layer , Therefore, as the optimization goal, the connection weight matrix increases exponentially , Bring unacceptable learning time complexity . and CNN Through the feature screening of image information , Filter out a large amount of redundant information mixed with pictures , Then map to the output space through a simple fully connected network , It will greatly reduce the complexity .

3 Experimental process

This question is aimed at classic MNIST Handwritten numeral classification experiment , be based on Pytorch Framework independent design neural network , Test network performance , And carry out some visual analysis . The experimental process is as follows ：

1. Build convolutional neural network ;
2. Load data set . download MNIST Handwritten digital datasets , Divide the training set 、 Validation set and test set , And encapsulated as an iteratable data loader object ;
3. Training models . Define loss function and optimization method , Calculate the loss through forward propagation , Then based on the back propagation optimization model parameters , After iterating until the training error converges, save the model locally ;

3.1 Building neural networks

As shown below , Build convolutional neural network . among

• Conv2d： Convolution layer
• MaxPool2d： Pooling layer
• ReLu： Activation function

The specific principles and functions of these neural network components will be analyzed in another article , This chapter focuses on application practice .

class CNN(nn.Module):
    ''' * @breif:  Convolutional neural networks  '''    
    def __init__(self):
        super().__init__()
        self.convPoolLayer_1 = nn.Sequential(
            nn.Conv2d(in_channels=1, out_channels=10, kernel_size=5),
            nn.MaxPool2d(kernel_size=2),
            nn.ReLU()
        )
        self.convPoolLayer_2 = nn.Sequential(
            nn.Conv2d(in_channels=10, out_channels=20, kernel_size=5),
            nn.MaxPool2d(kernel_size=2),
            nn.ReLU()
        )
        self.fcLayer = nn.Linear(320, 10)

    def forward(self, x):
        batchSize = x.size(0)
        x = self.convPoolLayer_1(x)
        x = self.convPoolLayer_2(x)
        x = x.reshape(batchSize, -1)
        x = self.fcLayer(x)
        return x

3.2 Load data set

Use pytorch Provided Dataset Class to load and preview datasets

from abc import abstractmethod
import numpy as np
from torchvision.datasets import mnist
from torch.utils.data import Dataset
from PIL import Image

class mnistData(Dataset):
    ''' * @breif: MNIST Dataset abstract interface  * @param[in]: dataPath ->  Data set storage path  * @param[in]: transforms ->  Data set transformation  '''    
    def __init__(self, dataPath: str, transforms=None) -> None:
        super().__init__()
        self.dataPath = dataPath
        self.transforms = transforms
        self.data, self.label = [], []

    def __len__(self) -> int:
        return len(self.label)

    def __getitem__(self, idx: int):
        img = self.data[idx]
        if self.transforms:
            img = self.transforms(img)
        return img, self.label[idx]

    def loadData(self, train: bool) -> list:
        ''' * @breif:  Download and load datasets  * @param[in]: train ->  Whether it is a training set  * @retval:  List of data and labels  '''    
        #  If there is no dataset under the specified directory, download 
        dataSet   = mnist.MNIST(self.dataPath, train=train, download=True)
        #  Initialize data and labels 
        data  = [ i[0] for i in dataSet ]
        label = [ i[1] for i in dataSet ]
        return data, label

3.3 Training models

Considering that this practice is a multi classification problem , Therefore, the output of the final network is a ten dimensional vector and passes through softmax Into a probability distribution , The loss function is designed as cross entropy , The optimization method is random gradient descent algorithm .

for images, labels in trainBar:
    images, labels = images.to(config.device), labels.to(config.device)
    #  Gradient clear 
    opt.zero_grad()
    #  Positive communication 
    outputs = model(images)
    #  Calculate the loss 
    loss = F.cross_entropy(outputs, labels)
    #  Back propagation 
    loss.backward()
    #  Model update 
    opt.step()

The training process is as follows ：

4 Algorithm analysis

After testing , At the same learning rate ,CNN stay 20 Generation has converged , but FCNN stay 20 It's only around the generation that it begins to converge . The generalization error of the test set shows CNN The accuracy of prediction reaches 95%, but FCNN Only 70%, So in the problem of image classification ,CNN Its efficiency and accuracy are far higher than FCNN. therefore CNN Strong learning and generalization ability , Just use the known pattern to CNN Train , The network has the ability to map and express the input and output .

---

More wonderful Columns ：

• 《 Robot principle and technology 》
• 《ROS From entry to mastery 》
• 《 Computer vision course 》
• 《 machine learning 》
• 《 Numerical optimization method 》
• …

Private messages enter AI Technology exchange group , Whoring for nothing 50G E-books and teaching resources , Regularly release AI Knowledge dry goods 、 Free technology books and other benefits ！

Matching code · Quality experience · System knowledge Please pay attention to