Novice entry depth learning | 3-6: optimizer optimizers

• Running environment ：python3
• author ：K Students
• Select columns ：《 Deep learning 100 example 》
• From column ：《 Novice introduction and deep learning 》
• Recommendation column ：《Matplotlib course 》
• 🧿 Excellent column ：《Python introduction 100 topic 》

---

The optimizer is in the process of deep learning back propagation , Guide the loss function （ Objective function ） The parameters of are updated in the correct direction and the appropriate size , Make the updated parameters let the loss function （ Objective function ） The value keeps approaching the global minimum .

I wrote a lot of things , But I feel that talking too much will be counterproductive ,

The choice of model optimization algorithm is directly related to the performance of the final model . Sometimes it doesn't work , It's not necessarily about features or model design , It's probably the problem of optimizing the algorithm .

Deep learning optimization algorithm has probably experienced SGD -> SGDM -> NAG ->Adagrad -> A