What is data leakage

Here is a concept ：

data leakage（ The data reveal that ）

. Here, data leakage does not mean the leakage of privacy data , It refers specifically to data in machine learning （ Training data and validation data ） There are anomalies .

1. What is? data leakage

Kaggle Some players in the competition will find some magical characteristics （magic feature） And win the championship ,magic feature The existence of may indicate the existence of competition data data leakage.

data leakage The external manifestation of ：  The accuracy of your training model is exceptionally good , For example, to achieve 100%. But the effect of actual online prediction is very poor .

So when your model performs beyond your expectations , Don't be happy too early , Please analyze your data carefully

.

data leakage The classification of ：

• forecast data leakage
   

•  
  data validation data leakage
  .

2.  forecast data leakage

Model prediction is to build a cause to effect mapping process , such leakage Generally, the data in the model is used by 【 fruit 】 Some characteristics after occurrence , These characteristics and 【 fruit 】 There is a strong correlation , The predicted model will become

【 fruit 】 To 【 fruit 】

Mapping . for example , We need to predict 【 Is it a diabetes patient 】 This 【 fruit 】, Degree of obesity 、 Degree of physical activity 、 The diet 、 Virus infection and so on may be normal 【 because 】. But if you use 【 Have you ever been insulin 】 This as a 【 because 】, This is what happens leakage,  Because patients with diabetes will take insulin after diagnosis , There is

put the cart before the horse

The phenomenon of . When you deploy the model, you will find that you can't get these features at all .

3. data validation data leakage

such leakage It is generally due to the split training set and test （ verification ） Set exists due to improper processing

.

For example, you are doing preprocessing （ Such as normalization , Missing value supplement, etc ） Then split the data , At this time, there may be some correlation between the training set and the verification set , To some extent, the verification set lacks the assumption of complete independence , The trained model may have a good score in the validation set , But online deployment is not .

4. How to avoid data leakage

If the performance of your model is surprisingly good , Stop and analyze the data to see if it exists data leakage

• Carefully analyze the characteristics that are strongly related to the prediction target

• Try to ensure the independence of validation set and model training , Especially when cross validation .

Reference resources ：https://www.kaggle.com/dansbecker/data-leakage