[note] logistic regression

One 、 Overview of logistic regression

（1） The purpose of logistic regression ： classification .

Logistic regression often solves classification problems , Especially the binary classification problem .

（2） The process of logistic regression ： Return to .

The result is 0~1 Continuous values between , Represents the possibility of occurrence （ Class probability ）.

（3） threshold ： Complete the classification through the comparison of possibility and threshold .

Such as ： Calculate the possibility of default , If it is greater than 0.5, The borrower is classified as a bad customer .

Two 、 Logistic regression model

Because in the second classification problem , Tags are only yes or no （1 and 0）, If linear regression is used for fitting , The range of linear regression is not 0 ~ 1 Between , The prediction result is difficult to be output as possibility . And if we use piecewise function to fit , Because the piecewise function is not continuous , The predicted result is not the possibility we hope 0 ~ 1 Continuous values of .

The solution is to combine linear regression with sigmoid Functions together , That is to form a nested function .

sigmoid Function image ：

Nested functions composed of ：

3、 ... and 、 The loss function of logistic regression

If we directly combine $\hat{y}$ Replace with sigmoid Function as the loss function of logistic regression to find the minimum value , You will find that this function is not convex , Therefore, other loss functions are used .

（1） In the problem of classification y Probability

Due to agreement $\hat{y}=P(y=1|x)$ , therefore ：

$y=1$ when , $P(y|x)=\hat{y}$

$y=0$ when , $P(y|x)=1-\hat{y}$

Merge to get ：

$P(y|x)={\hat{y}}^y(1-\hat{y}{)}^{(1-y)}$

（2） Using the maximum likelihood estimation method to estimate the model parameters

The likelihood function is ：

The log likelihood function is ：

Find the maximum value of the maximum likelihood function , The most ideal parameter value can be obtained . So for the whole training set , The cost function can be defined as ：