CV learning notes - deep learning

Deep learning

1. neural network

1. summary

Cited example ： Mechanism of biological neural network

The basic working principle of biological neural network ：
There are multiple dendrites at the input of a neuron , It is mainly used to receive input information . Input information is processed by synapse , The letter to be entered
Interest accumulation , When the processed input information is greater than a specific threshold , It will transmit information through axons , At this time, it is called neuron
To be activated . contrary , When the processed input information is less than the threshold , Neurons are in a state of inhibition , It doesn't transmit like other neurons
Information . Or send a small message .

Artificial neural network ：

Artificial neural network is divided into two stages ：

• Receive from others n Signals from neurons , These input signals are matched with corresponding weights
  The weighted sum is passed to the next stage .（ Pre activation phase ） It's the sum
• Pass the weighted result of pre activation to the activation function It's the f

The concepts of artificial neural network and biological neural network correspond ：

Biological neural networks	Artificial neural network
nucleus	Neuron
Dendrites	Input
axon	Output
synaptic	The weight

Artificial neuron ：

Input ：$x_1,x_2,x_3$

Output ：output

simplified model ： Agree that there are only two possibilities for each input （1 or 0）

• All inputs are 1, Indicates that the input conditions are all true , Output is 1
• All inputs are 0, Indicates that none of the input conditions is true , Output is 0

【 example 】 Judge whether watermelon is good or bad

Input ：[ Color : dark green , roots : Curl up , Knock sound : Murmur ]

Output ： Good melon (1)

neural network ： It is formed by interconnected neurons , These neurons have weights and errors during network training
To update the deviation , The goal is to find an approximation of an unknown function . Its principle is influenced by the physiological structure of our brain —— Cross connected neurons inspire . But unlike a neuron in the brain that can connect to any neuron within a certain distance ,
Artificial neural networks have discrete layers 、 Direction of connectivity and data dissemination .

2. Types of neural networks

3. Neuron

Neuron is the most basic unit of neural network , It originated from the human body , Imitate human neurons , Function is also related to the God of the human body
Jing Yuan is consistent , Get the input of the signal , After data processing , Then give a result as output or as the next neuron
Input .

4. Parameters in neural networks

Input ： It's the eigenvector . The eigenvector represents the direction of change . Or say , Is the most representative of the characteristics of this thing
towards .

The weight （ A weight ）： It's characteristic value . There are positive and negative , Strengthen or suppress , Same as eigenvalue . The absolute value of the weight , generation
The influence of input signal on neurons is shown in table .

The essence ： When the vector is n Dimensional space time , That is, the input vector and the weight vector are n dimension , That is to say n Inputs 、 Weight time , Yes
$$h=(x_1,x_2,...)({\omega }_1,{\omega }_2,...{)}^T+b$$
Neurons are when h Greater than 0 Time output 1,h Less than 0 Time output 0 Such a model , Its essence is to divide the feature space into two parts , Think that the two halves belong to two classes respectively .

5. Multilayer neural network

Because neurons can only divide the feature space into two , In the face of complex problems , This method obviously cannot meet the demand , Therefore, a multilayer neural network is proposed .

Neural network is a kind of operation model , By a large number of nodes （ Neuron ） And the mutual connection between . The connection between each two nodes represents a weighted value for the signal passing through the connection , Call it weight , This is equivalent to the memory of the artificial neural network .
The output of the network depends on the connection mode of the network , The weight value is different from the excitation function . The network itself is usually an approximation to some algorithm or function in nature , It could also be an expression of a logical strategy .

Single layer neural network （ perceptron ）

Multilayer neural network ： Neural network is composed of multiple neurons , The result of the former neuron is the input of the latter neuron , Combined in turn . The neural network is generally divided into three layers , The first layer acts as the input layer , The last layer is the output layer , All in the middle are hidden layers .

Theoretical proof , Any multilayer network can be approximately represented by a three-layer network .
Generally, experience is used to determine how many nodes the hidden layer should have , In the process of testing, you can also constantly adjust the number of nodes to achieve the best results .

6. Feedforward neural networks

The artificial neural network model mainly considers the topology of network links 、 Neuron characteristics 、 Learn the rules, etc .

Feedforward neural networks ： Feedforward neural networks (FNN) It is the first type of simple artificial neural network invented in the field of artificial intelligence . Inside it , Parameters propagate unidirectionally from the input layer through the hidden layer to the output layer . Unlike Recurrent Neural Networks , It will not form a directed ring inside . The following figure shows a simple feedforward neural network ：

FNN By an input layer 、 One （ Shallow networks ） Or more （ Deep networks , Therefore, it is called deep learning ） Hidden layer , And an output layer . Every layer （ Except for the output layer ） Connect with the next layer . This connection is FNN The key to architecture , It has two main characteristics ： Weighted average and activation function .

Classification of Feedforward Neural Networks ：

• （ monolayer ） perceptron

There is only a single-layer neural network , It is regarded as the simplest form of feedforward neural network , It is a binary linear classifier .

• Multilayer perceptron

It is an artificial neural network with forward structure , It can be seen as a directed graph , It consists of multiple node layers , Each layer is fully connected to the next layer . In addition to the input node , Each node is a neuron with nonlinear activation function （ Or processing unit ）, It is the generalization of single-layer perceptron , It overcomes the weakness that the perceptron can not recognize the linear inseparable data .

7. Design Neural Networks

1. Before using neural network training data , The number of layers of neural network must be determined , And the number of units per level
2. When the feature vector is introduced into the input layer, it is usually standardized to 0-1 Between （ To speed up the learning process ）
3. Discrete variables can be encoded into the possible values of an eigenvalue corresponding to each input unit
   such as ： The eigenvalue A I could take three values $(a_0,a_1,a_2)$, have access to 3 The input units represent A.
   If A=$a_0$, So on behalf of $a_0$ The cell value of 1, Other take 0;1,0,0
   If A=$a_1$, So on behalf of $a_1$ The cell value of 1, Other take 0, And so on 0,1,0
4. Neural network can be used for classification （classification） problem , You can also solve regression （regression） problem
   • For the classification problem , If it is 2 class , It can be represented by an output unit （0 and 1 Represent the 2 class ）; If more than 2 class , Then every
     A category is represented by an output unit Such as 001
   • There are no clear rules to design how many hidden layers are best , It can be tested and improved according to the experimental test, error and accuracy .

8. Perceptual knowledge of the hidden layer

For example, face recognition ：

Each of the above subnetworks , It can also be further decomposed into smaller problems , For example, the question of whether the upper left is an eye , Can be divided into
The solution is ：

• Do you have eyeballs ？
• Do you have eyelashes ？
• Do you have iris ？
• …

This sub network can be further decomposed ,. Decompose layer after layer , Until the answer is simple enough to be answered on a single neuron .

2. Activation function

1. summary

The state of neurons ： In the neural network , Neurons in active state are called active state , Neurons that are inactive are called inhibitory . Activation function endows neurons with the ability of self-learning and adaptation .

The function of activation ： The function of activation function is to introduce nonlinear learning and processing ability into neural network . Activation function is a core unit of neural network design .

With sigmoid Activation function $y=\frac{1}{1+e^{-x}}$ For example ：

2. Commonly used activation functions

There are three conditions for activating a function

• nonlinear
• It's very small
• monotonous

1. sigmoid function

$$y=logsig(x)=\frac{1}{1+e^{-x}}$$

sigmoid There are two main disadvantages :

• Gradient saturated , Look at the picture , The gradient of values on both sides is 0;
• The average value of the results is not 0, This is what we don't want , Because this causes the input of neurons in the posterior layer to be true or false
  0 Mean signal , This will have an impact on the gradient .

2. tanh function

$$y=tansig(x)=\frac{e^x-e^{-x}}{e^x+e^{-x}}$$

$$tanh(x)=2\sigma (2x)-1$$

3. ReLU function ( Linear rectifier layer )

$$f(x)=max(0,x)$$

3. Comparison of activation functions

4. Neurons are sparse

Unilateral inhibition ：ReLU A function is actually a piecewise linear function , Change all negative values to 0, And the positive value remains the same , This operation is called unilateral inhibition .

Because of this unilateral inhibition , So that the neurons in the neural network also have Sparse activation .（ Simulate the characteristics of the human brain , At the same time , There are only neurons in human brain 1%~4% To be activated , Most of them are in a state of inhibition ）

When the model increases N After the layer , Theoretically ReLU The activation rate of neurons will decrease 2 Of N Times the power .

3. Deep learning

machine learning & Artificial intelligence & Deep learning ：

Machine learning is the core of artificial intelligence , A discipline that studies how to use machines to simulate human learning activities .
Deep learning （ Multilayer artificial neural network ） It's a branch of machine learning , The method of representational learning of data .

Deep neural network & Deep learning ： The traditional neural network has developed to the case of multiple hidden layers , Neural networks with multiple hidden layers are called deep neural networks , The research of machine learning based on deep neural network is called deep learning . If you need to refine and differentiate , that , Deep neural network can be understood as the structure of traditional multilayer network 、 Optimization of methods .

Deep learning , Multilayer artificial neural network

---

Personal study notes , Just learn to communicate , Reprint please indicate the source ！