Parameter estimation -- Chapter 7 study report of probability theory and mathematical statistics (point estimation)

Parameter estimation ——《 Probability theory and its mathematical statistics 》 Chapter VII study report （ Point estimation ）

Preface

It's been a long time , I have been busy with other subjects recently ,emmm Make a study report on Chapter 7 before the end of the term .

Because of the setting of teaching , This time, only the point estimation in Chapter 7 parameter estimation has been done , A full version will be released if there is a chance in the future .

The tutorial is the same as before , The fourth edition of Zhejiang University + The fifth edition .

MindMap

The point estimate is divided into Moment estimation method and Maximum likelihood estimation Two kinds of .

Moment estimation method

We learned a little about moments in Chapter 4 , For example, the origin moment 、 Central moment, etc . ad locum , We use Sample moment To estimate Total moment , Then the estimation of relevant parameters .

We use random variables X The type of , That is, discrete type or continuous type , To divide .

if X by discrete ：
$$\begin{gathered} branchclothLawby：P\{X=x0\}=p(x;{\theta }_1,{\theta }_2,...,{\theta }_k),thisinOf{\theta }_i,JustyesIPeoplewantseekOfstayEstimatemeterginsengCount. \\ have\; toTototalbodyXOffrontkrankMoment：{\mu }_l=E(X^l)=\sum _{x\in R_x}{x}^{l}p(x;{\theta }_1,{\theta }_2,...,{\theta }_k) \end{gathered}$$
if Continuous type ：
$$\begin{gathered} GeneralrateThe\; secretdegree：f(x;{\theta }_1,{\theta }_2,...,{\theta }_k) \\ {\mu }_l=E(X^l)={\int }_{-\infty }^{\infty }{x}^{l}f(x;{\theta }_1,{\theta }_2,...,{\theta }_k)dx \end{gathered}$$
The sample moment is
$$A_l=\frac{1}{n}\sum _{i=1}^n{X}_i^l$$
The so-called moment estimation method Is to use the sample moment as the total moment An estimate .

Solution steps

1. Let's start with the moment , The number of moments listed here depends on the number of parameters we are going to estimate .
2. Then we calculate the formula of the parameter about the moment .
3. Replace the above total moment with the sample moment .
4. Finally, remember to add a sharp corner mark on the estimated parameters .

Maximum likelihood estimation

Let's discuss it separately and continuously .

discrete

Distribution law
$$\begin{gathered} P\{X=x\}=p(x;\theta ),\theta \in \Theta \\ set\; up{X}_1,X_2,...,X_{n}OfOneindividualsampleBenvalue,IPeoplecanWithknowAvenue{X}_i,i\in [1,k]OfGeneralrate \end{gathered}$$
You can get
$$P\{X_1=x_1,X_2=x_2,...,X_n=x_n\}=L(\theta )=L(x_1,x_2,...,x_n;\theta )=\prod _{i=1}^{n}p(x_i;\theta ),\theta \in \Theta$$
This function L Namely Of the sample Likelihood function .

The reason why this method is called Maximum likelihood estimation , That's what we took θ Estimated parameter value of , Is a maximum parameter value
$$L(x_1,x_2,...,x_n;\hat{\theta })=\underset{\theta \in \Theta }{\max }L(x_1,x_2,...,x_n;\theta )$$

Continuous type

Probability density
$$\prod _{i=1}^{n}f(x_i;\theta )$$
Likelihood function
$$L(\theta )=L(x_1,x_2,...,x_n;\hat{\theta })=\prod _{i=1}^{n}f(x_i;\theta )$$
If we take a logarithm and then take the derivative , You can get
$$\frac{d}{d\theta }lnL(\theta )=0$$
You get Log likelihood equation .

Why do you take derivatives ？ Because you want to take the maximum , So the derivative is 0 The situation of , Of course, minima and bounds are ignored here .

Basically, there is little difference between the steps and the moment estimation , It's also calculation , It's just more about deriving and solving equations .

In the case of multiple parameters , Just find the partial derivative .