Maximum likelihood method, likelihood function and log likelihood function

Likelihood function

In Statistics , Likelihood function Is a function of statistical model parameters . Represents... In model parameters Likelihood .

Definition ： Given the output x when , About parameters θ Likelihood function of L(θ|x)（ In numerical terms ） Equal to the given parameter θ Post variable X Probability ：

among , Small x Refers to the joint sample random variable X Value taken .θ Refers to unknown parameters , It belongs to parameter space .

p(x|θ) It can be regarded as a function with two variables .
When θ Set to constant , Then you will get an answer about x The probability function of (probability function), For different sample points x, What is the probability of its occurrence ;
When x Set to constant , You will get information about θ Likelihood function of (likelihood function), For different parameters θ, appear x What's the probability of this sample point .

  example ：
 -  Toss a homogeneous coin , throw 10 Time ,6 How likely is the secondary positive ？  This is probability .
 -  Flip a coin , throw 10 Time , The result is 6 Second, face up , And how likely is it to be homogeneous ？ This is likelihood , Find parameters .
 
  notes ： The possibility of homogeneity means that it is possible to face up and face up , Are all 0.5. So the result is the same last time .

How to understand the maximum likelihood function ？

Maximum likelihood estimation refers to It is known that a random sample satisfies a certain probability distribution , Use the result to deduce the parameter value that leads to the result .

example ： Flip a coin , throw 10 Time , The result is 6 Sub head up , What is the biggest parameter ？
notes ： It can be understood as how likely it is to throw it face up once , Throwing 10 Next time , The result is 6 The probability of facing up is the greatest .

Steps of maximum likelihood method ：

1. Write the likelihood function .
2. If you can't find the derivative directly , Log the likelihood function .
3. Find the derivative , Let the derivative be 0, Get the likelihood equation .
4. solve equations , Get the parameter result .

Why use log likelihood function ？

Solving the maximization of a function often requires solving the partial derivative of the function with respect to unknown parameters , But direct derivation will make the calculation more complicated . So with the help of log likelihood function . Because the logarithmic function is monotonically increasing , So the maximum point will be the same .

notes ： Probability value is decimal , In the case of multiple consecutive rides , The result will be close to 0, At this time, take the negative number of logarithm for the likelihood function , To minimize the log likelihood function .