Machine learning by Li Hongyi 2. Regression

Author：AXYZdong

Automation   Engineering Male

A little bit of thinking , Have a little idea , A little bit rational ！

Set a small goal , Try to be a habit ！ Meet a better self in the most beautiful years ！

More wonderful articles go to ： 

Personal home page

One 、 Return to （Regression）

Return to （Regression）： Find a function Function, By entering a feature  

, Output a value  

 .

Application, for example,

• Stock market forecast （Stock market forecast）

• Autopilot （Self-driving Car）

• recommendation （Recommendation）

• Pokemon Elf attack power prediction （Combat Power of a pokemon）：

Two 、 Model steps

2.1  The model assumes  -  Linear model

• Univariate linear model （ A single feature ）

Model to represent ：

• Multivariate linear model （ Multiple features ）

Model to represent ：

• 
  
  ： All kinds of characteristics (fetrure)

• 
  
  ： The weight of each feature

• b： Offset

2.2  Model to evaluate  -  Loss function

A single feature ：

.

Definition  

  It's pre evolutionary CP value , 

  For the evolved CP value ,

  What it represents is the real value .

collect 10 Group true value , With these real data , How do we measure the quality of the model ？ Mathematically speaking , We use distance . seek 【 After evolution CP value 】 And 【 The model predicted CP value 】 Bad , To determine the quality of the model . That is, using the loss function （Loss function）  To measure the quality of the model .

take  

  and  

  Show... In two-dimensional coordinates

• Each point in the figure represents the corresponding point of a model  
  
  
    and  
  
  
  ;

• The darker the color, the better the model .

2.3  Model optimization  -  gradient descent

A single feature ：

.

How to select the optimal model （ Find out what makes  Loss Function  The smallest  

  and  

）

• From the simplest, there is only one parameter  
  
  
    Starting with , Definition  
  
  

step 1： Choose one at random  

step 2： Calculate the differential , That is, the current slope , Determine the direction of movement according to the slope

• Greater than 0 To the right （ increase ww）

• Less than 0 Move to the left （ Reduce ww）

step 3： Move according to the learning rate

Repeat step 2 And steps 3, Until you find the lowest point

• For two parameters  
  
  
    and  
  
  
  , The procedure is similar to one of the above parameters , What we need to do is partial differential .

The challenge of gradient descent algorithm in the real world

• problem 1： The best at the moment （Stuck at local minima）

• problem 2： be equal to 0（Stuck at saddle point）

• problem 3： Tend to be 0（Very slow at the plateau）

In the linear model, it is the shape of a bowl （ Valley shape ）, Gradient descent can basically find the best , But in other more complex models , Will meet   problem 2  and   problem 3 .

Verify whether the model is good or bad

Use

Average error of training set and test set

To verify the quality of the model .

3、 ... and 、 Over fitting （Overfitting）

Based on a simple model , It can be optimized , Choose a more complex model （ One yuan N Sublinear model ）, Or use

polynomial

Fitting .

If we choose a higher power model , Better models in the training set , On the test set, the effect may be worse . This is the problem of over fitting the model on the training set .

Four 、 Regularization （Regularization）

For more features , But the weight  

  The weight of some features may be too high , Still lead to overfitting, You can add regularization .

5、 ... and 、 summary

Datawhale Team learning , Li Hongyi 《 machine learning 》Task2. Regression（ Return to ）, It mainly includes the definition of regression 、 To create a model 、 How to optimize the model 、 Possible problems in the process of optimizing the model and using regularization to solve the problem of over fitting .

This sharing is here

If my article helps you 、 If you like the content of my article , please  

“ give the thumbs-up ” “ Comment on ” “ Collection ”

  One button, three links ！

hear

   give the thumbs-up   

It's not that bad luck for people who are , Every day will be full of vitality ！^ _ ^

It's not easy to code words , Everyone's support is my driving force to stick to it . Don't forget to like it  

Focus on

  I oh ！

If any of the above is inaccurate , Welcome to leave a message below . Or you have a better idea , Welcome to exchange and study together ~~~