[LDA] rough version notes of EM variational reasoning [to be improved

The boss said it in detail :csdn_ Potential Dirichlet assignment in machine learning (LDA) Variation EM Algorithm and python Realization

In a nutshell ：
thought ：（ Excerpt from the original ：）
The hypothetical model is Joint probability distribution p(x,z), among Observation variables x , Hidden variables z , Including the parameters .
The goal is to learn about the model Posterior probability p ( z ∣ x ).[tip: It is known that x, Implicit variable z]
But the distribution is complicated You can't solve it directly , So consider using A probability distribution q ( z ) Come on The approximate Conditional probability distribution p ( z ∣ x ),
After use KL The divergence KL(q(z)||p(z|x)) Calculate the similarity between the two ,q(z) It's called the variational distribution .

KL Form of divergence ：
because KL Divergence greater than 0, Get the following formula

So the left side of the unequal sign is called ： evidence evidence
The right side of the unequal sign is called ： The lower bound of evidence ELBO

Here on the left logp(x) Is the fixed value , When the right side is bigger , The smaller the divergence , The more similar the two are ,（ q ( z ) and p(z∣x) The closer the distribution of ）, So maximize the lower bound of evidence .

Then we have to calculate the lower bound of the evidence ELBO（q）, You have to write the joint probability density p(x,z) and q ( z ), Then seek log, Then expect .

** But behind lda I didn't understand the part of