当前位置：网站首页>13 probability distributions that must be understood in deep learning

13 probability distributions that must be understood in deep learning

2022-07-28 10:04:00 【Big data V】

Reading guide ： As a machine learning practitioner , You need to know about probability distribution . Here is a tutorial on the most common basic probability distribution , Most and use Python Library for deep learning .

source ：AI developer

Link to the original text ：

https://github.com/graykode/distribution-is-all-you-need

Overview of probability distribution ：

Conjugation means that it has a conjugate distribution relationship .
In Bayesian probability theory , If posterior distribution p（θx） And a priori probability distribution p（θ） In the same probability distribution family , Then a priori and a posteriori are called conjugate distributions , The prior is called the conjugate prior of the likelihood function . Conjugate a priori Wikipedia is here
https://en.wikipedia.org/wiki/Conjugate_prior
Multi classification means that the random variance is greater than 2.
n Second means that we also consider a priori probability p（x）.
To further understand probability , I suggest reading [pattern recognition and machine learning,Bishop 2006].

Distribution probability and characteristics ：

01 Uniform distribution （ continuity ）

Code ：
https://github.com/graykode/distribution-is-all-you-need/blob/master/uniform.py

Evenly distributed in [a,b] Have the same probability value on , It's a simple probability distribution .

02 Bernoulli distribution （ discrete ）

Code ：
https://github.com/graykode/distribution-is-all-you-need/blob/master/bernoulli.py

Prior probability p（x） Without considering Bernoulli distribution . therefore , If we optimize the maximum likelihood , Then we can easily be overfitted .
Binary cross entropy is used to classify binomial classification . Its form is the same as the negative logarithm of Bernoulli distribution .

03 The binomial distribution （ discrete ）

Code ：
https://github.com/graykode/distribution-is-all-you-need/blob/master/binomial.py

Parameter is n and p The binomial distribution of is a series of n Discrete probability distribution of success times in independent experiments .
Binomial distribution refers to a distribution that considers a priori probability by specifying the number to be selected in advance .

04 Dobbernoulli distribution , Distribution by category （ discrete ）

Code ：
https://github.com/graykode/distribution-is-all-you-need/blob/master/categorical.py

Dobenuli calls it the taxonomic distribution .
The cross entropy has the same form as the dobbernoulli distribution with negative logarithm .

05 Polynomial distribution （ discrete ）

Code ：
https://github.com/graykode/distribution-is-all-you-need/blob/master/multinomial.py

The relationship between polynomial distribution and classification distribution is the same as that between bernoulle distribution and binomial distribution .

06 β Distribution （ continuity ）

Code ：
https://github.com/graykode/distribution-is-all-you-need/blob/master/beta.py

β The distribution is conjugate with binomial distribution and Bernoulli distribution .
Using conjugation , Using the known prior distribution, the posterior distribution can be obtained more easily .
When β The distribution satisfies the special case （α=1,β=1） when , The uniform distribution is the same .

07 Dirichlet Distribution （ continuity ）

Code ：
https://github.com/graykode/distribution-is-all-you-need/blob/master/dirichlet.py

dirichlet Distribution and polynomial distribution are conjugate .
If k=2, Then for β Distribution .

08 Gamma distribution （ continuity ）

Code ：
https://github.com/graykode/distribution-is-all-you-need/blob/master/gamma.py

If gamma（a,1）/gamma（a,1）+gamma（b,1） And beta（a,b） identical , be gamma Distribution is β Distribution .
Exponential distribution and chi square distribution are special cases of gamma distribution .

09 An index distribution （ continuity ）

Code ：
https://github.com/graykode/distribution-is-all-you-need/blob/master/exponential.py

The exponential distribution is α by 1 when γ A special case of distribution .

10 Gaussian distribution （ continuity ）

Code ：
https://github.com/graykode/distribution-is-all-you-need/blob/master/gaussian.py

Gaussian distribution is a very common continuous probability distribution .

11 Normal distribution （ continuity ）

Code ：
https://github.com/graykode/distribution-is-all-you-need/blob/master/normal.py

The normal distribution is the standard Gaussian distribution , The average value is 0, The standard deviation is 1.

12 Chi square distribution （ continuity ）

Code ：
https://github.com/graykode/distribution-is-all-you-need/blob/master/chi-squared.py

k The chi square distribution of degrees of freedom is k Distribution of the sum of squares of independent standard normal random variables .
The chi square distribution is β A special case of distribution

13 t Distribution （ continuity ）

Code ：
https://github.com/graykode/distribution-is-all-you-need/blob/master/student-t.py

t The distribution is a symmetrical bell distribution , Similar to the normal distribution , But the tail is heavy , This means that it is more likely to produce values much lower than the average .