Mathematics in machine learning -- point estimation (I): basic knowledge

Set the overall $X$ The distribution form of is known , But one or more of its parameters is unknown , With the help of The overall $X$ The problem of estimating the total unknown parameter value from a sample of is called the point estimation of the parameter . Set the overall $X\sim f(x;\theta )$, among $f$ The form of is known ,$\theta$ It's an unknown parameter ． for example , The overall $X\sim B(1,p)$, among $p$ Unknown , This $p$ This is the unknown parameter of the marker population distribution , Short for overall parameters . Although the overall parameters are unknown , But the range of its possible values is known ． The value range of the overall parameters is called the parameter space , Write it down as $\Theta$． for example , Known population $X\sim N(\mu ,{\sigma }^2)$, among $\mu$ and ${\sigma }^2$ It's all unknown , Parameter space $\Theta =(\mu ,{\sigma }^2):-\infty <\mu <\infty ,{\sigma }^2>0$.

set up $(x_1,x_2,\cdots ,x_n)$ It's taken from the whole $X$ A sample of , If a statistic is used $\hat{\theta }=\hat{\theta }(x_1,x_2,\cdots ,x_n)$ To estimate $\theta$, said $\hat{\theta }$ Is the parameter $\theta$ A point estimator of .$\hat{\theta }$ Is the parameter $\theta$ An estimate of , Then there are $g(\hat{\theta })$ Is the parameter $g(\theta )$ An estimate of ． ad locum , Construct Statistics $\hat{\theta }$ Commonly used methods are moment estimation 、 Maximum likelihood estimation and maximum a posteriori estimation .