Bayes' law

1. probability theory

First review some probability theory .

joint probability ： event A And events B Probability of simultaneous occurrence ; Also called product rule .

$P(A,B)=P(A\cap B)=P(A|B)P(B)=P(B|A)P(A)$

Summation rule ： event A and event B The probability of different occurrences .

$P(A\cup B)=P(A)+P(B)-P(A\cap B)$

If A and B Are mutually exclusive ：

$P(A\cup B)=P(A)+P(B)$

Total probability ： If the event A The occurrence of may be caused by many possible event B Lead to .
$$P(A)=\sum _i^{n}P(A|B_i)P(B_i)$$

Conditional probability ： Given event B event A Probability of occurrence .

$$P(A|B)=\frac{P(A,B)}{P(B)}$$

2. Bayes' law

In machine learning , Given the observed training data B, We are often interested in finding the best hypothesis space A.

The best hypothetical space is the most possible hypothetical space , That is, given training data B, Put all kinds of training data B In hypothetical space A Medium Prior probability Add up .

According to the above definition , Finding hypothesis space A The probability is as follows ：
$$P(A)=\sum _{n}P(A|B_i)P(B_i)$$
Is that familiar ？

This is actually All probability formula , event A The occurrence of may be caused by data $B_1$, $B_2$… …$B_n$
Many reasons lead to .

For a given training data B, Finding hypothesis space A Probability , Bayesian theorem provides a more direct method .

Bayesian law uses ：

• Hypothetical space A Of Prior probability $P(A)$
• And observation data Prior probability probability $P(B)$
• Given a hypothetical space A, Observation data B Probability $P(B|A)$

Find the given observation data B, Finding hypothesis space A Probability $P(A|B)$, Also known as Posterior probability , Because it reflects the given data B, For hypothetical space A The influence of probability .

Contrary to a priori probability , P(A) And B It's independent .

Bayes' formula ：
$$P(A|B)=\frac{P(B|A)P(A)}{P(B)}$$

The derivation of Bayesian formula is also very simple , Combining the conditional probability and joint probability mentioned in the first part, we can find .

Conditional probability ：
$$P(A|B)=\frac{P(A,B)}{P(B)}$$
joint probability ：
$$P(A,B)=P(B|A)P(A)$$

3. Maximum posterior probability MAP

Sometimes , Given data B, Want to ask for hypothetical space A The most likely assumption in is called Maximum posterior probability MAP（Maximum a Posteriori）.

$$A_{MAP}=argmaxP(A|B)$$

That is o ：

$$=argmax\frac{P(B|A)P(A)}{P(B)}$$

Get rid of $P(B)$ Because it is related to the assumption A yes independent .
$$=argmaxP(B|A)P(A)$$