Data type case of machine learning -- using data to distinguish men and women based on Naive Bayesian method

Author's brief introduction ： The most important part of the whole building is the foundation , The foundation is unstable , The earth trembled and the mountains swayed . And to learn technology, we should lay a solid foundation , Pay attention to me , Take you to firm the foundation of the neighborhood of each plate .
Blog home page ： Qigui's blog
Included column ：《 Statistical learning method 》 The second edition —— Personal notes
From the south to the North , Don't miss it , Miss this article ,“ wonderful ” May miss you la
Triple attack( Three strikes in a row ):Comment,Like and Collect—>Attention

Write it at the front

stay 《 Statistical learning method 》 In the fourth chapter of , The author only describes the simple Bayes method roughly , Because the theory is too difficult to understand , With probability ！ But it doesn't matter , Theory , I think it's ok if I don't understand it for the first time , I still don't understand it the second time , It's okay. , The third time, I still don't understand , Nothing more ... Maybe you'll think I'm teasing you ？ exactly ,, Of course not . The author suggests remembering the formula . When the , You must be asking again ： So many formulas , How to remember ？ Which ones should I remember ？ Then I can only tell you ： The Buddha is . Don't talk nonsense , Let's see the actual battle ！

Naive Bayes

• Naive Bayes is a simple but powerful predictive modeling algorithm , Simplicity lies in the assumption that each feature is independent .

• Naive Bayesian model consists of two types of probability ：

  • 1、 The probability of each category ——$P(C_j)$
  • 2、 Conditional probability of each attribute ——$P(A_i|C_j)$

• Bayes' formula ：

  • $P(C_j|A_i)=\frac{P(A_i|C_j)P(C_j)}{P(A_i)}$

• Naive Bayes classifier ：

  • $y=f(x)=argmax\frac{P(C_j){\prod }_i^{}P(A_i|C_j)}{{\sum }_j^{}P(C_j){\prod }_i^{}P(A_i|C_j)}$
  • among , The full probability formula is ：
    • $P(A_i)={\sum }_j^{}P(C_j){\prod }_i^{}P(A_i|C_j)$

• Denominator to all $C_j$ All the same ：

  • $y=argmaxP(C_j){\prod }_i^{}P(A_i|C_j)$

How naive Bayesian classification works

Discrete data cases

• The data are as follows ：
  

next , Determine the following characteristics ：

• height
• weight
• shoe size

also , Set goals as follows ：

• Gender
  • C1： male
  • C2： Woman
  • Cj： Unknown

demand ：

• Solve whether the gender of the following characteristic data is male or female
  • A1： height = high
  • A2： weight = in
  • A3： shoe size = in

Realization ：

• Simply speaking , We only need the probability that the solution is male under this feature , Then solve the probability of being female under this feature , And then compare them , Choose the most likely as the result

$P(C_j|A_1{A}_2{A}_3)=\frac{P(A_1{A}_2{A}_3|C_j)P(C_j)}{P(A_1{A}_2{A}_3)}$

• Again because $A_i$ Independent of each other , So it can be converted as follows ：

$P(A_1{A}_2{A}_3|C_j)=P(A_1|C_j)P(A_2|C_j)P(A_3|C_j)$

• P(Cj|A1A2A3) = (P(A1A2A3|Cj) * P(Cj)) / P(A1A2A3)
• P(A1A2A3|Cj) = P(A1|Cj) * P(A2|Cj) *P(A3|Cj)

- C1 Attribute probability under category condition ：
	- P(A1|C1) = 2/4 = 1/2 
	- P(A2|C1) = 1/2 
	- P(A3|C1) = 1/4
- C2 Attribute probability under category condition ：
	- P(A1|C2) = 0 
	- P(A2|C2) =1/2 
	- P(A3|C2) = 1/2

• P(A1A2A3|C1) = 1/16
• P(A1A2A3|C2) = 0

therefore , regard as P(A1A2A3|C1) > P(A1A2A3|C2), Should be C1 Category , For men .

Continuous data cases

• The data are as follows ：
  

demand ：

• Solve whether the gender of the following characteristic data is male or female

  • height ：180
  • weight ：120
  • shoe size ：41

• The formula is still the above formula , The difficulty here is , Because of height 、 weight 、 Shoe sizes are continuous variables , The method of discrete variables cannot be used to calculate probability . And because there are too few samples , Therefore, it cannot be divided into intervals .

• What shall I do? ？ At this time , Sure Suppose the height of men and women 、 weight 、 Shoe sizes are normally distributed , Calculate the mean and variance from the sample

• That is to say, we get the density function of normal distribution . With the density function , You can substitute the value into , Calculate the value of the density function at a point

• such as , The height of men is the average 179.5、 The standard deviation is 3.697 Is a normal distribution . So the height of men is 180 The probability of is 0.1069. How to calculate it ?

from scipy import stats 

male_high = stats.norm.pdf(180,male_high_mean,male_high _var) 
male_weight = stats.norm.pdf(120, male_weight_mean, male_weight_var)
male_code = stats.norm.pdf(41, male_code_mean, male_code_var)

Realization ：

• Suppose the height of men and women 、 weight 、 Shoe sizes are normally distributed
• Calculate the mean and variance from the sample , That is to say, we get the density function of normal distribution
• With the density function , You can substitute the value into , Calculate the value of the density function at a point

import numpy as np
import pandas as pd

#  Import data 
df = pd.read_excel('table_data.xlsx', sheet_name="Sheet3", index_col=0)
df2 = df.groupby(" Gender ").agg([np.mean, np.var])

#  men   Of all characteristics   mean value   And   variance 
male_high_mean = df2.loc[" male ", " height "]["mean"]
male_high_var = df2.loc[" male ", " height "]["var"]

male_weight_mean = df2.loc[" male ", " weight "]["mean"]
male_weight_var = df2.loc[" male ", " weight "]["var"]

male_code_mean = df2.loc[" male ", " shoe size "]["mean"]
male_code_var = df2.loc[" male ", " shoe size "]["var"]

from scipy import stats

male_high_p = stats.norm.pdf(180, male_high_mean, male_high_var)
male_weight_p = stats.norm.pdf(120, male_weight_mean, male_weight_var)
male_code_p = stats.norm.pdf(41, male_code_mean, male_code_var)

print(' men ：', male_high_p * male_weight_p * male_code_p)

#  women   Of all characteristics   mean value   And   variance 
female_high_mean = df2.loc[" Woman ", " height "]["mean"]
female_high_var = df2.loc[" Woman ", " height "]["var"]

female_weight_mean = df2.loc[" Woman ", " weight "]["mean"]
female_weight_var = df2.loc[" Woman ", " weight "]["var"]

female_code_mean = df2.loc[" Woman ", " shoe size "]["mean"]
female_code_var = df2.loc[" Woman ", " shoe size "]["var"]

from scipy import stats

female_high_p = stats.norm.pdf(180, female_high_mean, female_high_var)
female_weight_p = stats.norm.pdf(120, female_weight_mean, female_weight_var)
female_code_p = stats.norm.pdf(41, female_code_mean, female_code_var)

print(' women ：', female_high_p * female_weight_p * female_code_p)

print(male_high_p*male_weight_p*male_code_p > female_high_p*female_weight_p*female_code_p)

Written in the back

This is a relatively simple case , Understand that naive Bayes is a generative model , It is a learning process , Constantly adjust cognitive probability .（ The generation model is ： The result given is not of type , It's probability ） Further understand Bayesian formula , It is helpful to better understand naive Bayesian model .