[distributed external detection] Odin ICLR '18

《Enhancing the reliability of out-of-distribution image detection in neural networks.》 ICLR’18

OOD classic baseline.

ODIN The idea is very simple , Since the model is right ID and OOD Sample output softmax The predicted probability distribution is different , So can we make their distribution more different ？ This makes it easier to detect OOD sample . This paper presents two methods to assist ：Temperature scaling and Input Preprocessing：

Compared with baseline, Use Temperature scaling and Input Preprocessing Can effectively make ID and OOD sample softmax The difference of distribution probability becomes larger .

Temperature Scaling

$$p_i(\mathbf{x};T)=\frac{\exp \left(f_i(\mathbf{x})/T\right)}{{\sum }_{j=1}^N\exp \left(f_j(\mathbf{x})/T\right)}$$

Softmax Of Temperature Scaling In fact, it is used in many places , Such as Hinton Of Knowledge Distillation, Use a large enough $T$ value , You can make softmax The score of is close to $\frac{1}{N}$, The author also proved that this method can effectively separate ID and OOD Of the sample softmax Distribution .

Input Preprocessing

$$\tilde{\mathbf{x}}=\mathbf{x}-\epsilon \mathrm{sign}\left(-{\nabla }_{\mathbf{x}}\log p_{\hat{y}}(\mathbf{x};T)\right)$$

Inspired by the confrontation sample . The purpose of the counter sample is to move in the opposite direction of the real label , Yes $x$ Add a small disturbance ; It's the opposite here , Towards the real label $x$ Add a small disturbance , Through this operation, we can get $\tilde{x}$ Close to its real category , Corresponding category softmax The probability of prediction increases , So all of ID The sample will be closer to its true category , Further separation ID and OOD sample softmax Probability distribution of .

I don't quite agree with this , Yes, in the training set ID Data is disturbed , Then when testing, we encounter ID data , But this data has not been disturbed , It will also appear that it deviates from the original distribution ？