Naive bayes

12. Naive Bayes

12.1 Introduction

Categories

• 

• 

12.2 Basics of Probability

12.2.1 Definition of probability

• The likelihood of an event happening
• P(X) : value in [0, 1]

12.2.2 Joint Probabilities

• Contains multiple conditions, and the probability that all conditions are met at the same time
• Denoted as: P(A,B)

12.2.3 Conditional Probabilities

• Probability of event A when another event B has already occurred
• Denoted as: P(A|B)

12.2.4 independent of each other

• If P(A, B) = P(A)P(B), then event A and event B are said to be independent of each other

12.2.5 Bayesian Formula

• 

12.2.6 Naive Bayes

• A Bayesian formula that assumes independence between features

12.3 API

sklearn.naive_bayes.MultinomialNB(alpha = 1.0)

• Naive Bayesian Classification
• alpha: Laplace smoothing coefficient

12.4 Algorithm Summary

12.4.1 Advantages

• Naive Bayesian model originated from classical mathematical theory and has stable classification efficiency
• It is less sensitive to missing data and the algorithm is relatively simple, which is often used for text classification
• High classification accuracy and fast speed

12.4.2 Disadvantages

• It does not work well if the feature attributes are correlated due to the assumption of sample attribute independence
• The prior probability needs to be calculated, and the prior probability often depends on the hypothesis. There can be many kinds of hypothetical models, so in some cases, the prediction effect will be poor due to the hypothesized prior model.

12.4.3 Principles of NB

• Naive Bayes is a classification method based on Bayes' theorem and the assumption of feature condition independence

  • For a given item xx to be classified, calculate the posterior probability distribution through the learned model,
  • That is: the probability of each target category appearing under the condition that this item appears, and the category with the largest posterior probability is taken as the category to which xx belongs.

12.4.4 Why Simple

• When calculating the conditional probability distribution P(X=x∣Y=c_k), NB introduces a strong conditional independence assumption, that is, when Y is determined, the values ​​of each feature component of X are independent of each other

12.4.5 Why is the conditional independence assumption introduced

• In order to avoid the problem of combinatorial explosion and sample sparseness when solving Bayes' theorem

12.4.6 What should I do if the probability is 0 when estimating the conditional probability P(X∣Y)

• Introduce λ

  • When λ=0, it is an ordinary maximum likelihood estimation
  • When λ=1, it is called Laplace smoothing

12.4.7 Difference between Naive Bayes and LR

• One

  • Naive Bayes is a generative model
  • LR is a discriminant model

• Two

  • Naive Bayes is based on a strong assumption of conditional independence (under the condition that the classification Y is known, the values ​​of each feature variable are independent of each other)
  • LR does not require this

• Three

  • Naive Bayes is suitable for small datasets
  • LR is suitable for large datasets