[machine learning] first day of introduction

The overview

Algorithm principle

K- Steps of nearest neighbor algorithm

1. Calculate the distance between a point in a given category dataset and the current point
2. Sort in the order of increasing distance
3. Select the front with the smallest distance k A little bit
4. Before finding out k The point with the highest frequency of category occurrence among the points
5. Select this category as the prediction category of test data

python Basic code knowledge

1.np.tile(inX,(dataSetSize,1)) Express the inX Repeat in the column vector direction 1 Time , Row vector repeat dataSize Time , In this case, i.e inX Constructed into and dataSet A matrix of the same order , To facilitate the subtraction operation .
2.array.argsort() That is, the array array Sort from small to large and return the sorted subscript value .
3.dict.get(key,default=None) Dictionary get() Method , Returns the value of the specified key , If it is not in the dictionary, the default value is returned None, The default value returned in this program is set to 0.
4.key=operator.itemgetter(1) Sort by dictionary value
5.key=operator.itemgetter(0) Sort according to the key value of the dictionary

Complete code

import numpy as np
import operator

#  Create a dataset 
def createDataSet():
    group = np.array([[1, 101], [5, 89], [108, 5], [115, 8]])
    #  label 
    labels = [' Love story ', ' Love story ', ' Action movies ', ' Action movies ']
    return group, labels

''' K-NN Algorithm  Paraments： inX  Data for classification （ Test set ） dataSet  Data for training （ Training set ） labes  Category labels  K K-NN Choose the front... With the smallest distance K A little bit  Returns： sortedClassCount[0][0]  Classification results  '''
def K_NN(inX,dataSet,labes,k):
    # dataSet The number of rows 
    dataSetSize=dataSet.shape[0]
    #  Repeat on column vector discovery inX common 1 Time （ The transverse ）, The direction of the line vector is repeated dataSetSize Time （ The longitudinal ）, That means to put inX Convert to order and dataSet The same matrix 
    diffMat=np.tile(inX,(dataSetSize,1))-dataSet
    sqDiffMat=diffMat**2 # Two dimensional vectors are subtracted and squared 
    #sum() All the elements add up ,sum(0) Column addition ,sum(1) Add lines 
    sqDistances=sqDiffMat.sum(axis=1)

    #  Calculate the distance by square root 
    distances=sqDistances**0.5
    #  return distances The index value of elements sorted from small to large in , Put back the original subscript of the sorted data 
    sortedDistIndices=distances.argsort()

    #  Define a dictionary that records the number of times of a category 
    classCount={
    }
    for i in range(k):
        #  Before removal K A category of elements , That is, take out the sorted tag names in turn 
        votellabel=labels[sortedDistIndices[i]]
        # dict.get(key,default=None) Dictionary get() Method , Returns the value of the specified key , If it is not in the dictionary, the default value is returned None, The default value returned in this program is set to 0
        classCount[votellabel]=classCount.get(votellabel,0)+1

                
    sortedClassCount=sorted(classCount.items(),key=operator.itemgetter(1),reverse=True)
    #  Put back the most frequent tags 
    return sortedClassCount[0][0]




if __name__=='__main__':
    group,labels=createDataSet()
    print(group)
    print(labels)

    test=[101,20]

    test_class=K_NN(test,group,labels,3)
    print(test_class)

[1] Reference boss Jack-Cui