[basic data mining technology] KNN simple clustering

KNN Clustering technology

The picture shows age and income , Will you buy magazines

KNN Is to choose one K As the radius of , Circle with sample as origin , If there are more categories in the circle , Then we will divide the sample into this category .K Is a super parameter , Because we are sure .

KNN Theoretical basis ： Customers in the same cluster will show the same behavior .

So the cluster is the same as the adjacent customers , It is not a machine learning method

Inferiority ： inefficiency , Because I'm not sure K So try many times .

It is difficult to explain why KNN Clustering effect will be better than naïve prediction Good prediction .

KNN And Naïve Prediction Result probability comparison ：

We found that the correct probability is indeed much higher .

Practical application KNN Of 3 A step

Data preprocessing （Data Preprocessing） Guaranteed attributes （age vs Income） The measurement （ The proportion ）scale No problem . Remember to standardize

Calculation of distance （Distance Caculation）： Choose which distance calculation formula

Calculation of prediction probability （Predicted Probability）

step 1： Standardization

We use extreme positive programming here （Min-max Normalization）【0,1】

The transformation effect is as follows ：

step 2 Several distance calculations ：

Manhattan distance （ First power ）, This is the street distance , Not a straight distance

Among them R The formula is as follows ：

Euclidean distance （ A quadratic ）

It's our classic distance formula , Linear distance

The difference between the two is p The final formula of the control is

When p be equal to 1 Manhattan distance

When p be equal to 2 Time is European distance

Python In the accessories, it is to change p To change the distance formula

step 3：

For example, there is a score 3 Class distance , The test data sample is T,k=5. give the result as follows

The latest goal is A class

The second most recent target attribute is B class

The third recent target attribute is A class

The fourth recent target attribute is C class

The fifth recent target attribute is A class

Then we predict that the target attribute value is A, Accuracy rate is 3/5

Case study 1 Give diagnostic data for the following diseases , Field 1 is the patient code , The following input fields （ sore throat 、 Have a fever 、 Swollen lymph glands 、 congestion , Have a headache ） And the target field （ The diagnosis ）

 

utilize KNN Predict the diagnostic results of the following patients （K=3）

Distance（Yes,No）=1

DIStance（YES,YES）=0

Distance（No,No）=0

Two customers distance The calculation method adopts interception distance