MLP - Multilayer Perceptron

MLP- Multilayer perceptron

Also known as Artificial neural network , It is a generalization of perceptron . The main solution is the weakness of linear indivisibility for data recognition . Such as XOR operation
MLP In addition to the input and output layers, there are hidden layers , The hidden layer can be multi-layer or one layer . Let's take the simplest example ：

Each layer is fully linked to the next layer , Full link here every value is connected with every value of the next layer , obtain f(wi*xi+b),f Is the activation function , The main function of the activation function is to find , In fact, each neuron in the hidden layer is characterized by input x The linear combination of . However, if it is only a linear combination , So no matter how many layers this neural network has , The results will be linearly related to the characteristics . So we are behind each neuron , Add an activation function (Activation Function), Change the linear rule . Common activation functions are ：sigmoid,relu etc. .

such , From the hidden layer to the input layer, we get an input-output relationship . In the forward propagation of multilayer perceptron , We are due to features xi And weight wi Do matrix operations between , No matter how the position of features is sorted , Will not affect the results . In the field of point cloud processing , It solves the problem of point cloud disorder ,pointnet Used max function .

We judge the quality of the model through the loss function , The commonly used loss function is the mean square , there xi For real value ,yi For the predicted value ,N by batch_size The number of loss functions of .

In the classification problem, we often use the cross function ,y hat Represents the predicted value obtained by the model ,y For real value , It is also the value given during our training .

In order to find the best model , We need to find a way to minimize the loss function wi and bi, Commonly used optimization methods are
BD Algorithm （ Back propagation ） To train the model .
BD The algorithm is mainly divided into two parts , Part is the forward propagation mentioned above , The other part is back propagation , The core is the chain rule .
according to BP Algorithm to determine the gradient , After that, we continue to optimize the model . Get the best model