In depth learning report (1)

Catalog

Chapter one Basic concepts of neural network

Chapter two Dataset processing

The third chapter Network construction

---

Chapter one Basic concepts of neural network

Through the study of Chapter 1 , I understand the basic concepts of Neural Networks , Including neurons 、 neural network 、 perceptron 、 The basic composition and thoughts of networks such as activation function and loss function . Also master the training methods of the network, including forward propagation and error back propagation algorithms . And the two most commonly used neural networks , Convolutional neural networks and cyclic neural networks .

The artificial neuron unit is composed of linear function and activation function , And according to the connection between different neurons , Neural networks can be divided into feedforward networks 、 Feedback network and graph network .

Perceptron is one of the simplest Artificial Neural Networks , A single-layer perceptron is essentially a two classifier , It can only deal with linear classification problems, but cannot deal with nonlinear data . Adding hidden layers can solve nonlinear problems , More hidden layers , The stronger the resolution of neural network , The stronger the ability of feature expression .

The function of activation function is to introduce nonlinearity . Common activation functions are Sigmoid function 、Tanh function 、Softsign function 、ReLU function 、Softplus Functions and Softmax Functions, etc . The nonlinearity of the function should be considered when selecting the activation function 、 Continuous differentiability 、 Boundedness 、 Monotonicity and smoothness .

The transmission of input signals from the input layer to the output layer is called forward propagation . The loss function is used to evaluate the inconsistency between the predicted value and the real value of the model , It's a non negative real value function , The smaller the loss function , The better the robustness of the model . Common loss functions include mean square error and cross error , Generally, the mean square error is more used for regression problems , Cross entropy error is more used in classification problems , Finally, they are transformed into solving extreme value problems . Gradient descent method is often used to solve extreme value , Gradient descent method includes global gradient descent method 、 Random gradient descent method 、 Small batch gradient descent method .

Convolution neural network includes convolution layer 、 Pool layer and full connection layer , Its core idea is local perception and parameter sharing . Cyclic neural network can classify serialized data .

---

Chapter two Dataset processing

Through the study of Chapter 2 , I learned about common data types , Master the principle of data set division and the method of dealing with model variance and deviation .

Data types include images 、 video 、 voice 、 Text and timing data . A data set is a set of data used in machine learning tasks , Data sets are divided into training sets 、 Validation set and test set .

In data set segmentation , Use the set aside method 、K- Crossover verification 、Holdout Cross validation 、 Self help method, etc , Make the training set and the test set equally distributed . If the training set and the test set are not identically distributed , The attributes of data distribution of training set and test set should be consistent as far as possible , Find more training set data that match the test set samples better .

Avoid high deviations , Try a bigger model , Extend the training time , Try a new model architecture , Reduce regularization ; Avoid high variance , More data is available , Add regularization , Try to stop training in advance , Try a new model architecture ; Avoid training test set mismatch , More data similar to the test set can be obtained .

---

The third chapter Network construction

Through the study of the third chapter , I have mastered how to choose the right neural network and build it , Understand what a super parameter is and how to set it , Learn the influence of activation function and loss function on the construction of network model .

There are generally two parameters in the learning model , One parameter can be obtained from learning , There is another kind that cannot be obtained by data , It can only be set by human experience , It's called super parameter .

When dealing with different problems , Sometimes standard neural networks are not used , We need to choose convolutional neural network and cyclic neural network , Different networks should use appropriate activation functions and loss functions .