[data mining] nearest neighbor and Bayesian classifier

data mining

1. Nearest neighbor classifier

1.1 The evolution of learning methods

• Active learning methods
• Negative learning methods
  • rote classifier
  • Nearest neighbor classifier

1.2 The basic idea

If it walks like a duck , Bark like a duck , Then it's probably a duck .

Calculate the k Recent training examples , according to k Determine the category of test samples by the category of the nearest neighbor training samples .

• Elements of nearest neighbor classifier ：
  • The training sample
  • Distance metric
  • k Value selection

1.3 Features of nearest neighbor classifier

• Case based learning , There is no need to learn the model
• Testing samples is expensive ,
• k The value is too small to be disturbed by noise ,k It's easy if the value is too big
• The classification boundary is like a honeycomb

1.4 Advantages and disadvantages of nearest neighbor classifier

• advantage
  • Nearest neighbor is a nonparametric classification technique , Simple and intuitive , Easy to implement
  • It's an online technology , New data can be added directly to the data set without retraining
• shortcoming
  • When the sample is unbalanced

1.5 Improvement of nearest neighbor classifier ：K-D Trees