1.17 daily improvement of winter vacation learning (frequency school and Bayesian school) and maximum likelihood estimation

1.17 Make a little progress every day during the winter vacation

Fundamentals of machine learning

Frequency school and Bayes school

For a dataset $X:data\to X(x_1,x_2,\dots ,x_n{)}^T\theta :parameter$

$x-p(x|\theta )$

Frequency school （ Statistical machine learning ）

$\theta$： Unknown constant ,x Obey probability distribution

${\theta }_{mle}=argma{x}_{\theta }logP(x|\theta )$

optimization problem

1. Model
2. loss function
3. The algorithm calculates the loss

Bayesian school ( Probability graph model )

$\theta$ Obey probability distribution ,$\theta -p(x|\theta )$

Bayesian formula derivation
$$p(\theta |x)=\frac{p(x|\theta )p(\theta )}{p(x)}$$
$p(\theta )$： Prior probability $p(\theta |x)$： Posterior probability

Bayesian estimation

$p(\theta |x)=\frac{p(x|\theta )p(\theta )}{{\int }_{\theta }p(x|\theta )p(\theta )d\theta }$

Bayesian prediction

KaTeX parse error: Expected 'EOF', got '&' at position 17: …p(\tilde{x}|X) &̲=\int_\theta p(…

The problem of integral

Gaussian distribution

Normal distribution

For a dataset $X:data\to X(x_1,x_2,\dots ,x_n{)}^T$

Ask for his maximum likelihood estimation

${\theta }_{mle}=argma{x}_{\theta }logP(x|\theta )$

solve $\mu$（ Unbiased estimate ）

$$\begin{gathered} logP(x|\theta )=\log \prod _{i=1}^{N}p(x_i|\theta )=\sum _{i=1}^N\log p(x_i|\theta ) \\ {\mu }_{MLE}=\arg ma{x}_{\mu }logP(x|\theta ) \\ =argma{x}_{\mu }(\sum _{i=1}^N-\frac{(x_i-\mu {)}^2}{2{\sigma }^2}) \\ =argmi{n}_{\mu }min\sum _{i=1}^N(x_i-\mu {)}^2 \\ partialguideCount=0 \\ {\mu }_i=\frac{1}{N}\sum _{i=1}^N{x}_i \\ E({\mu }_{MLE})=\frac{1}{N}\sum _{i=1}^{N}E[x_i]=\mu \end{gathered}$$

solve $\sigma$（ A biased estimate ）

KaTeX parse error: Expected group after '_' at position 80: …\ Partial derivative =0 \\ \sum_̲\limits{i=1}^N[…