13. Logistic regression

Logical regression logistic regression The concept of has been classification Instead of .
For a question whether it is logistic regression or classification It doesn't matter anymore .

Regression and classification comparison

1. The return question

The goal is ： bring pred=y
Method ： Reduce pred and y The gap between （pred and y The square difference of ）

2. classification

The goal is ： Maximize benchmar or accuracy etc.
Method 1： Reduce dist（pθ（y|x）,pr（y|x））, stay θ Conditionally y And the conditional probability distribution in the real case , The closer the two are, the better
Method 2： Reduce divergence diviergence（pθ（y|x）,pr（y|x））

problem

Question 1 ： Why not maximize directly accuracy

Directly maximize accuracy There are two questions

problem 1： The gradient of 0, When the value weights When the change ,accuracy There is no change .
For classification , There's a threshold , Suppose the threshold is 0.5 when , Even if your prediction starts from 0.4 Rise to 0.49, Closer to the real value , But because the threshold is not exceeded 0.5, So the last accuracy The value is still unchanged . amount to x Changed ,y No change ,y Yes x The gradient is 0.
problem 2： Gradient discontinuity , When the correct number is discontinuous .
When weights When small changes occur , It may also cross the threshold , Lead to accuracy mutation , Make the gradient unstable or explode .

Question two ： Why is it called logical regression

Use sigmod
controversy ：（ Teacher long Liangqu concluded ）
MSE->regression
cross entropy->classification
Use two different methods to solve loss, It can be seen as a different problem