About LDA model

Recently, I am studying recommendation system , There is a model of argot meaning LDA. Read a lot of blogs , Information , The literature , For a person with a bad background in mathematics, I know a little about what this is . Make a note of , First, induction and summary has always been a better way of learning , I hope it can enlighten the latecomers even a little understanding .

Aside from boring Mathematics , What can this model be used for , I found that many materials and blog introductions are relatively general , After reading it, my mind is full of complex and unintelligible mathematical formulas , Even the purpose of this model is not clear . Here to talk about my own understanding . For computer programs , It is clear from the input intermediate process to the output . The input of this model is some corpus （ This abstract word can be embodied into several documents , For example, a few txt file , Each of these files contains an article ）+ Initial parameter setting of the model , The intermediate process is to train through model corpus LDA Parameters of the model , Well trained LDA The model can be applied to a document , Output the probability that this document belongs to a topic . So just like other supervised statistical learning methods ,LDA The model should be trained first , Then use the trained model for analysis and prediction .

Although the above process is not complicated , But the whole model involves a lot of mathematical knowledge , Especially if you don't understand the function of these knowledge clearly, it's easy to make people dizzy . These include Dirichlet+ Multinomial conjugate distribution ,LDA Model process and   Gibbs sampling method . Quote LDA Probability graph model ：

  

From this probability diagram , You can imagine such a process , Join a strange writer to write an article , When it writes each word, it must first determine a theme for the word . The determination of this topic is determined by a multinomial distribution （ It can be thought of as a polyhedral dice , Each side of the dice is a theme ）, But this multinomial distribution is composed of a Dirichlet Distribution to determine . This is the upper part of the figure , First of all Dirichlet Distribution determines multinomial distribution , Then the topic is determined by the distribution of multiple items Zm,n. There is also such a process when determining words by topic , First, by a Dirichlet Distribution determines a multinomial distribution , Then the word to be written is determined by the distribution of multiple items .

The above is even LDA Model , Next is gibbs abstract ,gibbs In fact, the function of sampling is to train the values of intermediate hidden parameters according to the input corpus . Because this whole LDA In the process , The input includes two Dirichlet Parameters of the distribution alpha,beta And corpus and topic The number of . Based on these inputs gibbs sampling method , Calculate the middle three parameters （ I don't know how to type these letters , Namely Zm,n And two parameters of multinomial distribution ）

       LDA After training , For new documents , To know its theme distribution , Can be used again gibbs sampling method , Fix , obtain that will do .

        The whole blog post does not involve any mathematical derivation , You need to know more about Daniel's special blog http://blog.csdn.net/yangliuy/article/details/8302599, If there is a mistake , Welcome criticism and correction