Machine learning interview preparation (I) KNN

Catalog

One 、 Briefly KNN Principle of algorithm ?

  Two 、KNN The three elements of the algorithm

3、 ... and 、 How to understand K Choice of value

Four 、kd What is a tree and its function ？

5、 ... and 、KNN Advantages and disadvantages

---

One 、 Briefly KNN Principle of algorithm ?

KNN（K-Nearest Neighbor） It is a basic classification and regression method , Given a training data set , For new input instances , Find the nearest... To the instance in the dataset K An example , If this K Most of the instances belong to a class , Put the input instance into this category .

KNN Learning process without display , In fact, it uses the training data set to divide the feature space , And as a model for its classification .

 

---

  Two 、KNN The three elements of the algorithm

Distance metric ： It is generally used LP distance .（P=1, Manhattan distance ;P=2, Euclidean distance ;P=∞, Chebyshev distance ）

K It's worth choosing ： Usually choose a small number , Usually, cross validation is used to select the best K value .

Classification decision rules ： The majority voting rule is generally adopted  

---

3、 ... and 、 How to understand K Choice of value

If K Smaller value selection , It is equivalent to using training examples in smaller fields to predict ,“ Study ” The approximate error of will be reduced , But the estimation error will increase , If the adjacent point is noise , The prediction will go wrong . in addition ,K A decrease in value means that the overall model becomes more complex , It's easy to get fitted .

If K It's worth choosing more , It is equivalent to using training examples in a larger field to predict , The estimation error of learning will be reduced , The approximation error will increase , This is far from the input instance （ Dissimilar ） Examples can also play a role in Forecasting , Make a prediction wrong ,K Increasing the value means that the overall model becomes simpler .

In the application , Usually choose a small number , Usually, cross validation is used to select the best K value .

---

Four 、kd What is a tree and its function ？

KNN The simplest way to do this is to do a linear scan , Because the distance is calculated for each instance , So when the training set is large , The training instance dimension is very high , The calculation will be very time-consuming . In order to improve the KNN Search efficiency , May adopt kd Tree to store data

 

---

5、 ... and 、KNN Advantages and disadvantages

 

---

Reference resources ：

《 Statistical learning method 》

Dr. Jane's personal space _ Bili, Bili _Bilibili 

---