Notes on probability theory

probability theory

Chapter one Random events and probabilities

Section 1 Relationship and operation of events

$A\subset B\Rightarrow A-B=A\bar{B}=\varnothing$
Duality rate ：$\overline{\underset{i=1}{\overset{n}{\cap }}{A}_i}=\underset{i=1}{\overset{n}{\cup }}\overline{A_i},\overline{\underset{i=1}{\overset{n}{\cup }}{A}_i}=\underset{i=1}{\overset{n}{\cap }}\overline{A_i}$

In the second quarter Probability and formula

Conditional probability

$P(B|A)=\frac{P(AB)}{P(A)}$

Independent

$P(AB)=P(A)P(B)namelyP(A|B)=P(A|\bar{B})$

A class

$P(A\cup B)=P(A)+P(B)-P(AB)$
$P(A\cup B\cup C)=P(A)+P(B)+P(C)-P(AB)-P(AC)-P(BC)+P(ABC)$

Multiplication

$P(AB)=P(A)P(B|A)$
$P(A_1{A}_2{A}_n)=P(A_1)P(A_2|A_1)\dots P(A_n|A_1{A}_2\dots A_{n-1})$

All probability formula

$P(A)=\sum _{i=1}^{n}P(B_i)P(A|B_i)$

Bayes' formula

$P(B_k|A)=\frac{P(B_k)P(A|B_k)}{\sum _{i=1}^{n}P(B_i)P(A|B_i)}$

Classical concept and Bernoulli

$P(A)=\frac{n_A}{n}=\frac{AthepackagecontainOfsamplespot}{sampleBenspottotalCount}$

Only $A,\overline{A}$ , be called Bernoulli's test , repeat $n$ The second is n Heavy Bernoulli experiment ,$P(A)=p$ , happen k The probability of times is $P=C_n^k{p}^k(1-p{)}^{n-k}$

Add

$C_{n+1}^k=C_n^k+C_n^{k-1}$

n+1 Choose a few k individual = The current number is not selected （ Choose k-1）+ Select the current number （ Choose k）

$C_{n+m}^k=\sum _{i=1}^k{C}_n^i\ast C_m^{k-i}$

n+m Number selected k individual = from n Number selected i individual * from m Number selected k-i individual

Chapter two Random variables and their probability distributions

Section 1 Random variables and their distribution functions

$F(x)=P(X<x)$

If there is a non negative integrable function $f(x)$, Make any $x$, There are $$F(x)={\int }_{-\infty }^{x}f(t)\mathrm{d}t,-\infty <x<+\infty$$, said X by Continuous random variables f(x) Is the probability density function

In the second quarter Common distribution

$0-1$ Distribution

$E(X)=p,D(X)=p(1-p)$

The binomial distribution （ Bernoulli

$X\sim B(n,p)$
$P(X=k)=C_n^k{p}^k(1-p{)}^{n-k}$
$E(X)=np,D(X)=np(1-p)$

Hypergeometric distribution

$P(X=k)=\frac{C_N^k{C}_M^{n-k}}{C_{N+M}^n}$

Geometric distribution

$P(X=k)=(1-p{)}^{k-1}p$
$E(X)=\frac{1}{p},D(X)=\frac{1-p}{p^2}$

Poisson distribution

$X\sim P(\lambda ),\lambda >0$
$P(X=k)=\frac{{\lambda }^k}{k!}{e}^{-\lambda },k=0,1,2,3,..$
$E(X)=\lambda ,D(X)=\lambda$
Poisson's theorem ： Bernoulli experiment $p_n$ representative A The probability of occurrence in an experiment , It is related to the total number of experiments $n$ of , With $n$ The increase of ,$p_n$ It's diminishing , If $\underset{n\to \infty }{\lim }n{p}_n=\lambda \Rightarrow \underset{n\to \infty }{\lim }{C}_n^k{p}^k(1-p{)}^{n-k}=\frac{{\lambda }^k}{k!}{e}^{-\lambda }$

Uniform distribution

$X\sim [a,b]orpersonX\sim (a,b)$
$f(x)=\{\begin{array}{rlr}& \frac{1}{b-a},& a<x<b\\ & 0,& ItsHe\end{array}$
$F(x)=\{\begin{array}{rlr}& 0,& b\le x\\ & \frac{x-a}{b-a},& a\le x<b\\ & 1,& b\le x\end{array}$

$E(X)=\frac{a+b}{2},D(X)=\frac{a+b}{12}$

An index distribution

$X\sim E(\lambda )$
among $\lambda >0$
$f(x)=\{\begin{array}{rlr}& \lambda e^{-\lambda x}& ,x>0\\ & 0& ,x\le 0\end{array}$
$F(x)=\{\begin{array}{rlr}& 1-e^{-\lambda x}& ,x>0\\ & 0& ,x\le 0\end{array}$

$E(X)=\frac{1}{\lambda },D(X)=\frac{1}{{\lambda }^2}$

Normal distribution

$X\sim N(\mu ,{\sigma }^2)$
$f(x)=\frac{1}{\sqrt{2\pi }\sigma }{e}^{-\frac{(x-\mu {)}^2}{2{\sigma }^2}}$
$F(x)=\frac{1}{\sqrt{2\pi }\sigma }{\int }_{-\infty }^x{e}^{-\frac{(t-\mu {)}^2}{2{\sigma }^2}}\mathrm{d}t$

$E(X)=\mu ,D(X)={\sigma }^2$

Standard normal distribution

Make $\mu =0,\sigma =1$
$\phi (x)=\frac{1}{\sqrt{2\pi }}{e}^{-\frac{x^2}{2}}$
$\Phi (x)=\frac{1}{\sqrt{2\pi }}{\int }_{-\infty }^x{e}^{-\frac{t^2}{2}}\mathrm{d}t$

$E(X)=0,D(X)=1$

In the third quarter Random variable function distribution

$Y=g(X)$
X Is a discrete random variable Y It is also a discrete random variable

X Is a continuous random variable Y It is also a continuous random variable

• Formula method
  $y=g(x)$ It's a monotone function ,$h(y)$ Is its inverse function
  $f_Y(y)=\{\begin{array}{rlr}& |h^{\prime }(y)|f_X(h(y)),& a<y<b\\ & 0,& ItsHe\end{array}$
• Definition
  $F_Y(y)=P(Y\le y)=P(g(X)\le y)=\underset{g(x)\le y}{\int }{f}_X(x)\mathrm{d}x$
  then $f_Y(y)=F_Y^{\prime }(y)$

The third chapter Multidimensional random variables and their distribution

Section 1 Two dimensional random variables and their distribution

Two dimensional continuous random variable

$F(x,y)={\int }_{-\infty }^x{\int }_{-\infty }^{y}f(u,v)\mathrm{d}u\mathrm{d}v$
$F_X(x)=F(x,+\infty )=F(x,y)={\int }_{-\infty }^x{\int }_{-\infty }^{+\infty }f(u,v)\mathrm{d}u\mathrm{d}v$
$f_{X|Y}(x|y)=\frac{f(x,y)}{f_Y(y)},f_Y(y)>0$

In the second quarter The independence of random variables

Definition ：
$X、Yphasemutualsinglestate\iff renIt\; meansx,y\{\begin{array}{rl}& P\{X\le x,Y\le y\}=P\{X\le x\}=P\{Y\le y\}\\ & F(x,y)=F_X(x)F_Y(y)\end{array}$

$leavescatteredtypealong\; withmachinechangeThe\; amountX、Yphasemutualsinglestate\iff P\{X=x_i,Y=y_i\}=P\{X=x_i\}P\{Y=y_i\}namely{p}_{ij}=p_i{p}_j$
$evenTo\; continuetypealong\; withmachinechangeThe\; amountX、Yphasemutualsinglestate\iff f(x,y)=f_X(x)f_Y(y)$

In the third quarter Two dimensional uniform distribution and two-dimensional normal distribution

Two dimensional uniform distribution

$DistrictDomainGOfNoodlesproductbyA$
$f(x,y)=\{\begin{array}{rlr}& \frac{1}{A}& ,(x,y)\in G\\ & 0& ,ItsHe\end{array}$

Two dimensional normal distribution

$X\sim N({\mu }_1,{\sigma }_1^2),Y\sim N({\mu }_2,{\sigma }_2^2)$
$(X,Y)\sim N({\mu }_1,{\mu }_2,{\sigma }_1^2,{\sigma }_2^2,\rho )$
$f(x,y)=\frac{1}{2\pi {\sigma }_1{\sigma }_2\sqrt{1-{\rho }^2}}exp[-\frac{1}{2(1-{\rho }^2)}(\frac{(x-{\mu }_1{)}^2}{2{\sigma }_1^2}+\frac{(y-{\mu }_2{)}^2}{2{\sigma }_2^2})-\frac{2\rho (x-{\mu }_1)(y-{\mu }_2)}{{\sigma }_1{\sigma }_2}]$

$exp(t)surfacein{e}^t$
$X、Yphasemutualsinglestate\iff \rho =0$

The fourth quarter, Two random variable functions $Z=g(X,Y)$ The distribution of

The core formula

$\begin{array}{rl}{F}_Z(z)& =P\{Z\le z\}\\ & =P\{g(X,Y)\le z\}\\ & =\underset{g(x,y)\le z}{\iint }f(x,y)\mathrm{d}x\mathrm{d}y\end{array}$

$Z=X+Y$ The distribution of

$\begin{array}{rl}{F}_Z(z)& =\underset{x+y\le z}{\iint }f(x,y)\mathrm{d}x\mathrm{d}y\\ & ={\int }_{-\infty }^{+\infty }\mathrm{d}x{\int }_{-\infty }^{z-x}f(x,y)\mathrm{d}y={\int }_{-\infty }^{+\infty }\mathrm{d}y{\int }_{-\infty }^{z-y}f(x,y)\mathrm{d}x\\ f_Z(z)& ={\int }_{-\infty }^{+\infty }f(x,z-x)\mathrm{d}x={\int }_{-\infty }^{+\infty }f(z-y,y)\mathrm{d}y\end{array}$
When $X$ and $Y$ When they are independent of each other
$f_Z(z)={\int }_{-\infty }^{+\infty }{f}_X(x)f_Y(z-x)\mathrm{d}x={\int }_{-\infty }^{+\infty }{f}_X(z-y)f_Y(y)\mathrm{d}y$
It is called convolution formula , Write it down as $f_Z=f_X\ast f_Y$

$Z=\max (X,Y),\min (X,Y)$ The calculation of

$Z=\max (X,Y)\le z\Rightarrow X\le z\bigcap Y\le z$
$\begin{array}{rl}{F}_Z(z)& =P\{Z\le z\}\\ & =P\{X\le z\}P\{Y\le z\}\\ & =F_X(x)F_Y(y)\end{array}$


$Z=\min (X,Y)\le z\Rightarrow X\le z\bigcup Y\le z$
$\begin{array}{rl}{F}_Z(z)& =P\{Z\le z\}\\ & =F_X(x)+F_Y(y)-F_X(x)F_Y(y)\end{array}$
perhaps

$X\le z\bigcup Y\le z\Rightarrow Union-(X>z\bigcap Y>z)$
$\begin{array}{rl}{F}_Z(z)& =P\{Z\le z\}\\ & =1-P\{X>z\}P\{Y>z\}\\ & =1-(1-F_X(x))(1-F_Y(y))=Same\; asOn\end{array}$

Chapter four The numerical characteristics of random variables

Section 1 Mathematical expectation and variance of random variables

expect

Definition

Continuous type
$E(X)={\int }_{-\infty }^{+\infty }xf(x)dx$
$E(Y)=E[g(X)]={\int }_{-\infty }^{+\infty }g(x)f(x)dx$

nature

$E(aX+b)=aE(X)+b$
$E(X\pm Y)=E(X)\pm E(Y)$
$X,Ysinglestate\Rightarrow E(XY)=E(X)E(Y)$

variance

Definition

$D(X)=E\{[X-E(X){]}^2\}$
Standard deviation （ Mean square error ）$\sigma (X)=\sqrt{D(X)}$

nature

$D(aX+b)=a^{2}D(X)$
$X,Ysinglestate\Rightarrow D(X\pm Y)=D(X)\pm D(Y)$

$\begin{array}{rl}D(X)& =E\{[X-E(X){]}^2\}\\ & =E\{[X^2-2XE(X)+E^2(X)]\}\\ & =E(X^2)-2E[XE(X)]+E^2(X)\\ & =E(X^2)-2E^2(X)+E^2(X)\\ & =E(X^2)-E^2(X)\end{array}$
Available $E(X^2)-E^2(X)\ge 0$

Moment 、 covariance and correlation coefficient

Definition

$XOfkrankprimaryspotMoment：E(X^k)$
$XOfkrankinheartMoment：E\{[X-E(X){]}^k\}$
$XandYOfk+lmixedcloseMoment：E(X^k{Y}^l)$
$XandYOfk+lmixedcloseinheartMoment：E\{[X-E(X){]}^k[Y-E(Y){]}^l\}$
$AssociationFangBadCov(X,Y)=E\{[X-E(X)][Y-E(Y)]\}$

$phaseTurn\; offsystemCount{\rho }_{XY}=\frac{Cov(X,Y)}{\sqrt{D(X)}\sqrt{D(Y)}}$

nature

$Cov(X,Y)=E(XY)-E(X)E(Y)$
$D(X\pm Y)=D(X)+D(Y)\pm 2Cov(X,Y)$
$Cov(aX,bY)=abCov(X,Y)$
$Cov(X_1+X_2,Y)=Cov(X_1,Y)+Cov(X_2,Y)$

$Cov(X,X)=E(X^2)-E(X)E(X)=D(X)$
$D(X)D(Y)=0\Rightarrow {\rho }_{XY}=0$
${\rho }_{XY}=0\Rightarrow X,YNophaseTurn\; off$
$|{\rho }_{XY}|=1\Rightarrow Y=aX+b$

Independent and irrelevant

• $X,Yphasemutualsinglestate\Rightarrow X,YNophaseTurn\; off$
• $Twodimensionjuststatealong\; withmachinechangeThe\; amount(X,Y),X、Yphasemutualsinglestate\iff \rho =0$
• $Twodimensionjuststatealong\; withmachinechangeThe\; amount(X,Y),X、Yphasemutualsinglestate\iff X,YNophaseTurn\; off$

The fifth chapter The law of large numbers and the central limit theorem

Chebyshev inequality

$E(X)andD(x)savestay,YesOnrenIt\; meansOf\epsilon >0,Yes$$$P\{|X-E(X)|\ge \epsilon \}\le \frac{D(X)}{{\epsilon }^2}$$

convergence

$X_1,X_2,\dots ,X_n,\dots yesOneindividualalong\; withmachinechangeThe\; amountorderColumn,AyesOneindividualoftenCount,Such\; asfruitYesOnrenIt\; meansOf\epsilon >0Yes$
$$\underset{n\to +\infty }{\lim }P\{|X_n-A|<\epsilon \}=1$$$becall{X}_1,X_2,\dots ,X_n,\dots \text{Converges to a constant in probability}A,rememberdo{X}_n\stackrel{P}{*}A$

Chebyshev's law of large numbers

$X_1,X_2,\dots ,X_n,\dots yesOneindividualtwotwoNophaseTurn\; offOfalong\; withmachinechangeThe\; amountorderColumn,D(X_i)\le C(CyesOneindividualoftenCount,i=1,2,\dots ),beYesOnrenIt\; meansOf\epsilon >0Yes$
$$\underset{n\to +\infty }{\lim }P\{|\frac{1}{n}\sum _{i=1}^n{X}_i-\frac{1}{n}\sum _{i=1}^{n}E(X_i)|<\epsilon \}=1$$

Bernoulli's law of large numbers

$X_n\sim B(n,p),n=1,2,...,beYesOnrenIt\; meansOf\epsilon >0Yes$$$\underset{n\to +\infty }{\lim }P\{|\frac{X_n}{n}-p|<\epsilon \}=1$$

Sinchin's law of large numbers

$X_1,X_2,\dots ,X_n,\dots yessinglestateSame\; asbranchcloth,E(X_i)\le \mu (i=1,2,\dots ),beYesOnrenIt\; meansOf\epsilon >0Yes$
$$\underset{n\to +\infty }{\lim }P\{|\frac{1}{n}\sum _{i=1}^n{X}_i-\mu |<\epsilon \}=1$$

Di morph - Laplace central limit theorem

$X\sim B(n,p),be$
$$\underset{n\to +\infty }{\lim }P\{\frac{X_n-np}{\sqrt{np(1-p)}}\le x\}=\Phi (x)$$$,\Phi (x)yesjuststatebranchclothLetterCount$

Levy - Lindbergh's central limit theorem

$X_1,X_2,\dots ,X_n,\dots yessinglestateSame\; asbranchcloth,E(X_i)=\mu ,D(X_i)={\sigma }^2(i=1,2,\dots ),beYesOnrenIt\; meansOf\epsilon >0Yes$
$$\underset{n\to +\infty }{\lim }P\{\frac{\sum _{i=1}^n{X}_i-n\mu }{\sqrt{n}\sigma }\le x\}=\Phi (x)$$$,\Phi (x)yesjuststatebranchclothLetterCount$

Chapter six Basic concepts of mathematical statistics

Section 1 The overall 、 sample 、 Statistics and sample number characteristics

$f_n(x_1,x_2,...,x_n)=\prod _i^{n}f(x_i)$

sample

1. Sample mean $\overline{X}=\frac{1}{n}\sum _{i=1}^n{X}_i$
2. Sample variance $S^2=\frac{1}{n-1}\sum _{i=1}^n(X_i-\overline{X}{)}^2$
   $D(\overline{X})=\frac{1}{n}D(X)$

$\begin{array}{rl}(n-1)D(X)& =nD(X)-D(X)\\ & =nD(X)-nD(\overline{X})\\ & =[\sum _{i=1}^n{X}_i^2-n{E}^2(X)]-[\sum _{i=1}^n\overline{X}-n{E}^2({\overline{X}}^2)]\\ & =\sum _{i=1}^n{X}_i^2-\sum _{i=1}^n{\overline{X}}^2\\ & =\sum _{i=1}^n{X}_i^2-2\sum _{i=1}^n{X}_i^2{\overline{X}}^2+\sum _{i=1}^n{\overline{X}}^2\\ & =\sum _{i=1}^n(X_i-\overline{X}{)}^2\end{array}$
namely $D(X)=\frac{\sum _{i=1}^n(X_i-\overline{X}{)}^2}{n-1}$

3. sample $k$ Moment of origin of order $A_k=\frac{1}{n}\sum _{i=1}^n{X}_i^k,A_1=\overline{X}$
4. sample $k$ Moment of order center $B_k=\frac{1}{n}\sum _{i=1}^n(X_i-\overline{X}{)}^k,B_2=\frac{n-1}{n}{S}^2$

In the second quarter Commonly used statistical sampling distribution

${\chi }^2$ Distribution

Definition
$sincefromdegreebynOf{\chi }^{2}branchcloth:$
${\chi }^2\sim {\chi }^2(n)$
${\chi }^2=\sum _{i=1}^n{X}_i^2,X_i\sim N(0,1)Andphasemutualsinglestate$
nature

1. $P\{{\chi }^2>{\chi }_{\alpha }^2(n)\}={\int }_{{\chi }_{\alpha }^2(n)}^{+\infty }f(x)dx=\alpha$
2. $E({\chi }^2)=n,D({\chi }^2)=2n$

t Distribution

Definition
$sincefromdegreebynOftbranchcloth:$
$T\sim t(n)$
$X\sim N(0,1),Y\sim {\chi }^2(n),X、Ysinglestate$
$T=\frac{X}{\sqrt{\frac{Y}{n}}}$

nature

1. $f(x)=f(-x),nenoughBigwhen,TrendnearOnN(0,1)$
2. $P\{T>t_{\alpha }(n)\}={\int }_{t_{\alpha }(n)}^{+\infty }f(x)dx=\alpha$
3. $t_{\alpha }(n)=-t_{1-\alpha }$ To be continued
4. $P\{|T|>t_{\frac{\alpha }{2}}(n)\}=\alpha$

F Distribution

Definition
$sincefromdegreeby(n_1,n_2)OfFbranchcloth:$
$F\sim F(n_1,n_2)$
$X\sim {\chi }^2(n_1),Y\sim {\chi }^2(n_2),X、Ysinglestate$
$F=\frac{\frac{X}{n_1}}{\frac{Y}{n_2}}$

nature

1. $P\{F>F_{\alpha }(n_1,n_2)\}={\int }_{F_{\alpha }(n_1,n_2)}^{+\infty }f(x)dx=\alpha$
2. $\frac{1}{F}\sim F(n_2,n_1)$
3. $F_{\alpha }(n_1,n_2)\sim \frac{1}{F_{1-\alpha }(n_2,n_1)}$

The sampling distribution of a normal population

A normal population

$X_i\sim N(\mu ,{\sigma }^2)$
$allvalue\overline{X},sampleBenFangBad{S}^2$

1. $\overline{X}\sim N(\mu ,\frac{{\sigma }^2}{n}),U=\frac{\overline{X}-\mu }{{\sigma }^2}\sim N(0,1)$
2. $\overline{X}and{S}^{2}phasemutualsinglestate,{\chi }^2=\frac{(n-1)S^2}{{\sigma }^2}=\frac{\sum _{i=1}^n(X_i-\overline{X}{)}^2}{{\sigma }^2}\sim {\chi }^2(n-1)$
3. $T=\frac{\overline{X}-\mu }{\frac{S}{\sqrt{n}}}\sim t(n-1)$
4. ${\chi }^2=\frac{\sum _{i=1}^n(X_i-\mu {)}^2}{{\sigma }^2}\sim {\chi }^2(n)$

Two normal populations

$X_i\sim N({\mu }_1,{\sigma }_1^2),Y_j\sim N({\mu }_2,{\sigma }_2^2),1\le i\le n_1,1\le j\le n_2$
$allvalue\overline{X}and\overline{Y},sampleBenFangBad{S}_1^{2}and{S}_2^2$

1. $\overline{X}-\overline{Y}\sim N({\mu }_1-{\mu }_2,\frac{{\sigma }_1^2}{n_1}+\frac{{\sigma }_2^2}{n_2})$
   $U=\frac{(\overline{X}-\overline{Y})-({\mu }_1-{\mu }_2)}{\sqrt{\frac{{\sigma }_1^2}{n_1}+\frac{{\sigma }_2^2}{n_2}}}$
2. If ${\sigma }_1^2={\sigma }_2^2$ be
   $\{\begin{array}{rl}& T=\frac{(\overline{X}-\overline{Y})-({\mu }_1-{\mu }_2)}{S_w\sqrt{\frac{1}{n_1}+\frac{1}{n_2}}}\sim t(n_1+n_2-2)\\ & S_w=\frac{(n_1-1)S_1^2+(n_2-1)S_2^2}{n_1+n_2-2}\end{array}$
3. $F=\frac{\frac{S_1^2}{{\sigma }_1^2}}{\frac{S_2^2}{{\sigma }_2^2}}\sim F(n_1-1,n_2-1)$

Chapter vii. Parameter estimation

Section 1 Point estimation

$notknowginsengCount\theta ,\hat{\theta }(X_1,X_2,\dots ,X_n)callby\text{An estimate}$
$ifE(\hat{\theta })=\theta ,\hat{\theta }yes\theta Of\text{Unbiased estimator}$
$\{\begin{array}{rl}& E({\hat{\theta }}_1)=E({\hat{\theta }}_2)=\theta \\ & D({\hat{\theta }}_1)\le D({\hat{\theta }}_2)\end{array}\Rightarrow {\hat{\theta }}_{1}Than{\hat{\theta }}_2\text{More effective}$

$Such\; asfruit\hat{\theta }(X_1,X_2,\dots ,X_n)In\; accordance\; with\; theGeneralrateclosedConvergenceOn\theta ,becall\hat{\theta }yes\theta Of\text{Consistent estimator}$

In the second quarter Estimation and interval estimation

Moment estimation method

Definition ： The sample moments are used to estimate the corresponding population moments , Use the function of sample moment to estimate the corresponding function of population moment , Then find the parameters to be estimated
step ： Set the overall X The distribution of contains unknown parameters $\theta_1,\theta_2,\dots,\theta _k,\alpha _k = E(X^l) $ There is , ${\alpha }_{l}yesTurn\; offOn{\theta }_1,{\theta }_2,\dots ,{\theta }_k$ Function of , Write it down as ${\alpha }_l({\theta }_1,{\theta }_2,\dots ,{\theta }_k),l=1,2,\dots ,k$ . Of the sample $l$ The moment of origin of the order is $A_l=\frac{\sum _{i=1}^n{X}_i^l}{n}.$ Make ${\alpha }_l({\theta }_1,{\theta }_2,\dots ,{\theta }_k)=A_l,l=1,2,\dots ,k\Rightarrow$ It can be solved that ${\theta }_1,{\theta }_2,\dots ,{\theta }_k$

set up $g(a_1,a_2)$ Is the first moment $a_1$ And the second moment $a_2$ Function of , and ${\hat{a}}_1、{\hat{a}}_{2}yes{a}_1、a_2$ The moment estimate of , be $g({\hat{a}}_1,{\hat{a}}_2)$ Namely $g(a_1,a_2)$ The moment estimate of

Maximum likelihood estimation

$X_{i}yesXOfsampleBen,x_{i}yessampleBenvalue,\theta yesgenerationEstimateLetterCount$
Parameters $\theta$ Likelihood function of
discrete ：$L(\theta )=L(x_1,x_2,\dots ,x_n;\theta )=\prod _{i=1}^{n}p(x_i;\theta )$
Continuous type ：$L(\theta )=L(x_1,x_2,\dots ,x_n;\theta )=\prod _{i=1}^{n}f(x_i;\theta )$

Maximum likelihood estimation
Definition
For a given sample $(x_1,x_2,\dots ,x_n)$, Let the likelihood function $L(x_1,x_2,\dots ,x_n;\theta )$ The parameter that reaches the maximum value $\hat{\theta }=\hat{\theta }(x_1,x2,\dots ,x_n)$ It is called unknown parameter $\theta$ Maximum likelihood estimate of , The corresponding likelihood function $L(X_1,X_2,\dots ,X_n;\theta )$ The parameter value that reaches the maximum value $\hat{\theta }=\hat{\theta }(X_1,X_2,\dots ,X_n)$ be called $\theta$ Maximum likelihood estimator of . Collectively referred to as $\theta$ Maximum likelihood estimation of
step
seek $\hat{\theta }$, The likelihood equation can be used $$\frac{\mathrm{d}L(\theta )}{\mathrm{d}\theta }=0or\frac{\mathrm{d}\ln L(\theta )}{\mathrm{d}\theta }=0$$

Suppose the parameter to be estimated is ${\theta }_{1}And{\theta }_2$, Then the likelihood equation can be obtained
$$\{\begin{array}{rl}& \frac{\partial L(\theta )}{\partial {\theta }_1}=0\\ & \frac{\partial L(\theta )}{\partial {\theta }_2}=0\end{array}or\{\begin{array}{rl}& \frac{\partial \ln L(\theta )}{\partial {\theta }_1}=0\\ & \frac{\partial \ln L(\theta )}{\partial {\theta }_2}=0\end{array}$$

interval estimation

confidence interval
Definition of confidence interval ： set up $\theta$ Is the overall $X$ Unknown parameter for ,$X_1,X_2,...,X_n$ It's a sample from the population , For a given $a(0<a<1)$, If two statistics satisfy $$P({\theta }_1<\theta <{\theta }_2)=1-a$$ It is called random interval $({\theta }_1,{\theta }_2)$ Is the parameter $\theta$ The confidence level of is $1-a$ The confidence interval of

Interval estimation of a normal population parameter

Interval estimation of two normal population parameters

Chapter viii.

Common formula

${\int }_0^{+\infty }{x}^n{e}^{-x}dx=n!$

${\int }_0^{+\infty }{e}^{-\frac{x^2}{2}}dx=\frac{\sqrt{\pi }}{\sqrt{2}},{\int }_0^{+\infty }x{e}^{-\frac{x^2}{2}}dx=1$

${\int }_0^{+\infty }tx{e}^{-tx}dx=1$

$\sum _{i=1}^n(X_i-\overline{X}{)}^2=\sum _{i=1}^n{X}_i^2-\sum _{i=1}^n{\overline{X}}^2=\sum _{i=1}^n{X}_i^2-n{\overline{X}}^2$
$\sum _{i=1}^n(X_i-\overline{X}{)}^2=\sum _{i=1}^n(X_i-\mu {)}^2-n(\overline{X}-\mu {)}^2$