maximum likelihood estimation

Probability and likelihood

Quote Wikipedia's description of probability and likelihood

• In mathematical statistics , Likelihood function is a function of the parameters in a statistical model , Indicates the likelihood in the model parameters . Likelihood function plays an important role in statistical inference , Such as the application of maximum likelihood estimation and Fisher information and so on .“ Likelihood ” And “ Probability ” or “ probability ” Similar meaning , It's all about the possibility of something happening

• In statistics ,“ Likelihood ” and “ Probability ” or “ probability ” There is a clear distinction . Probability is used when some parameters are known , Predict the results of the next Observations , The likelihood is used when the results of some observations are known , To estimate the parameters of the nature of something .

The above paragraph is still quite abstract ," probability " and " likelihood " In fact, it looks at the events from different perspectives

possibility

. Take tossing a coin as an example ( Random variables are discrete ), sample space

、 For random variables

, When

" just " when , Observation value

; When

" back " There are observations from time to time

 、 

, There is the following distribution law ( It's easier to understand probability function )

Is a functional model ,

Represents a specific sample observation ;

Represent the parameters of the model .

• If
  
  
  It's known and certain ( Homogenization ),
  
  
  It's a variable. , This function is called probability function (probability function), It describes whether there is an appearance or a reversal , It appears
  possibility
  (
  probability
  ) How much is the .

• If
  
  
  It's known and certain ( face up ),
  
  
  It's a variable. ( The mass distribution of coins is not determined , Different distributions may make it easier for coins to face up or reverse up ), This function is called
  Likelihood function
  (likelihood function),  It describes for different mass distributions , There is a downward upward situation
  possibility
  (
  likelihood
  ) How much is the .

Probability is given a certain parameter value , Find the possibility of a certain result ;  Likelihood is a given result , Possibility of finding a certain parameter value .

Likelihood function

Likelihood function is a kind of statistical model

Function of parameter

, On the whole

The case of discrete distribution and continuous distribution shows the form of likelihood function , set up

It's taken from the whole

A sample of

Set the overall

It is a discrete distribution , Its distribution law is

among

Is an unknown parameter , Observe the value of a given sample

, Make

If the population is continuously distributed , Its density function is

, among

Is an unknown parameter , Observe the value of a given sample

( What has happened , affirmatory ), Make

It can be seen that , Likelihood function

It's an unknown parameter

Function of , It reflects the observed value of a given sample

possibility

( For different

The possibility of being taken is different ), Our concern is , Under what circumstances

Most likely

Fetch , When the likelihood function value is maximum, it means that the probability is maximum , At this time, you will also get a specific parameter

, If you use

Express , Then call it

by

Of

Maximum likelihood estimation

, namely

Maximum likelihood estimation is based on the results of known samples , Reverse guess the parameter value of the model that is most likely to get such a result .

Understand the significance of maximum likelihood estimation , The following is the mathematical problem of finding the maximum , It is usually solved by deriving .

Example calculation

First, let's agree on an expression

, It said ： The mass distribution of coins is

when , At this time, the probability of a random throw is positive

If we know that coins are homogeneous (

), that " throw 10 Time to get 6 Second, face up " The probability is about

0.205

problem ： If you don't know the mass distribution of coins

In the case of 10 trials , Finally, I get 6 Secondary positive results (1,0,1,1,0,0,0,1,1,1), Then can we get the most likely mass distribution of coins ???

1. Find the law of distribution

Coin tossing test obeys binomial distribution , The whole can be regarded as infinite ( Throw it down endlessly ), When the mass distribution is unknown , The probability of positive rise is p, Then the probability of reverse upward is 1-p, The number of times below and up

Is a binomial random variable , Other values may be ：0,1,2,...,k,...,n; There is the following distribution law

among n It means to throw n Time ,k It means to get k Second, face up

2. Write the likelihood function

3. Find the derivative

Take logarithm of the above formula to get likelihood equation

Right again p To find the derivative

Solution

, It means , The mass distribution of coins is

when , Most likely “ throw 10 Time to get 6 Second, face up ” Such result

If the solution of maximum likelihood estimation is not used , Take various mass distributions to get “ throw 10 Time to get 6 Second, face up ” List the possibilities of this result , You can see

Most likely , This result occurs and the coin is homogeneous (

) The possibility is 0.205078125

Reference resources

1.《 Probability theory and mathematical statistics 》  Hao Zhifeng   Xie Guorui   Wang Guoqiang

2.  probability （probability)、 likelihood （likelihood)、 Maximum likelihood method ：http://blog.sina.com.cn/s/blog_e8ef033d0101oa4k.html)

3.  likelihood （likelihood） And probability （probability） The difference between ：http://yangfangs.github.io/2018/04/06/the-different-of-likelihood-and-probability/