The difference and understanding between generative model and discriminant model

One 、 Discrimination method and generation method

Supervised learning methods can be divided into discrimination methods and generation methods .

Discrimination method （Discriminative approach）

Learning decision function directly from data Y=f(X) Or conditional probability distribution P(Y|X) As a model of prediction , The discriminant model . The discrimination method is concerned with the given input X, What kind of output should be predicted Y. The basic idea is to establish the discriminant function under the condition of limited samples , Regardless of the sample generation model , Study the prediction model directly . Typical discriminant models include k a near neighbor , Perception level , Decision tree , Support vector machine, etc .

Generation method （Generative approach）

Learning joint probability density distribution from data P(X,Y), Then from P(Y|X)=P(X,Y)/P(X) Find the conditional probability distribution P(Y|X) As a model of prediction , It's a generative model . This method represents a given input X And generate output Y The generative relationship of . The basic idea is to first establish the joint probability density model of samples P(X,Y), And then we get the posterior probability P(Y|X), Use it again to classify . In this process, we need to calculate the probability distribution of the training data P(X), So as long as there are a lot of data samples , Got P(X) In order to well describe the real distribution of training data 【 Like tossing coins 】. Generation model is used to model randomly generated observations , Especially when some hidden parameters are given . Typical generation models are ： Naive Bayes and hidden Markov models .

Two 、 Discriminant model and generative model

1. Intuitively ：

Generate models ： Source oriented , How to generate when focusing on data , And then classify a signal .（ Signal input , The generation model determines which category is most likely to generate this signal , Then this signal belongs to which category .

Discriminant model ： Results oriented , Focus on the differences between categories , Don't care about how the sample data is generated , According to the “ Demarcation line " To simply classify a given sample .

2. From the formula we know ：

Generate models ： When you study, you get P（X,Y), According to the formula, we get P（Y|X); The maximum a posteriori probability method is used in prediction （MAP） Get the forecast category Y.

Discriminant model ： Learn directly to get P（Y|X), recycling MAP obtain Y; Or learn a mapping function directly Y=F(X）

3. The difference and connection between generative model and discriminant model