Deep learning notes (constantly updating...)

   

Data preprocessing One-Hot - You know (zhihu.com)

            A person without touching each other , In a special way , Have a series of questions and answers with each other ． If for a long time , He can't judge whether the other party is a person or a computer based on these questions , Then it can be considered that the computer is intelligent .                    

                                                                                                                                                        —— Alan · Turing （Alan Turing）

                                                                                                                                   《Computing Machinery and Intelligence 》

Knowledge system

Why does deep learning mainly use neural networks ？

The main reason is that the neural network model can use error back propagation BP Algorithm , So we can better solve the problem of contribution distribution

What neural networks need to learn ： The weight parameter of the connection between each neuron

Why use activation functions ： If you don't use the activation function , Then the output of each layer is the linear function of the input of the previous layer , Then eventually, the whole network is linear , That is, the most primitive perceptron (Perceptron）, Its presentation ability is poor

Neuron

Neural networks are composed of Neuron Composed of , The structure of each neuron is as follows

Activation function

among f Is the activation function , common Activation function as follows

Sigmoid function

 ReLu function

 Swish function

Network structure

         There are three common network structures , Feedforward networks FNN Similar to a directed acyclic graph , Information can only spread in one direction , It's simpler . Memory network MN Similar to a directed cyclic graph , Information can be transmitted in one direction or two directions , More complicated . Figure network GN It is the generalization of feedforward network and memory network , Each square is one or a group of neurons

Reference material

 《 Neural networks and deep learning 》-- Professor Qiu Xipeng

《 Hands-on deep learning 》Pytorch Open source -- Li Mu

《2022 Deep learning and machine learning 》-- Wu enda