Introduction to deep learning in machine learning

1. Deep learning
   1. 1. Deep learning Introduction
   2. 2. The principle of deep learning
   3. 3. Deep learning enables

Deep learning

1. Deep learning Introduction

Deep learning （Deep learning） It is a branch of machine learning , It comes from labor
The study of neural networks .
Deep learning is widely used in computer vision , Audio processing , Naturallanguageprocessing, etc
Domain .

Artificial neural network （Artificial Neural Network）, A mathematical model is used to simulate nerve
Meta activity , It is an information processing system based on imitating the structure and function of brain neural network
system .

Biological neural cell structure

Perceptron model

2. The principle of deep learning

Forward propagation （Forward Propagation）, It is calculated from the input through a layer of hidden layers
Process to output , That is, weighted summation , Then through an activation function .

Back propagation （Backward Propagation） Is consistent with the calculation direction of forward propagation
back , It is the error between the calculated output value and the real value , Calculate each by network reverse flow
Gradient of a layer of parameters （ Partial derivative ）, To update training parameters .

1） Commonly used activation functions
1、Sigmoid function

derivative

2、tanh function

derivative

3、Relu function

derivative

4、Leaky Relu function

derivative

2） Loss functions are commonly used
1、 Mean square loss function

2、 Cross entropy loss function

（3） Common optimization functions
1、SGD： Random gradient descent optimizer

2、Momentum： Random gradient descent with momentum

3、Nesterov Accelerated Gradient： Newton's acceleration gradient drops

4、Adagrad（Adaptive gradient）: Adaptive gradient descent
5、Adadelta

6、RMSprop

6、Adam：Adaptive Moment Estimation

3. Deep learning enables

Deep learning network realizes regression
Deep learning network realizes classification