[interpretable] | axiomatic attribute for deep networks

Axiomatic Attribution for Deep Networks, ICML 2017

The problem of attributing the prediction of deep network to its input characteristics is studied , Simply put, it is to study the relationship between input and output , To understand the input of the model - Output behavior .

And define what attribution should satisfy 2 Basic axioms , Sensitivity and implementation invariance

The author finds that in other literatures on attribution methods of characteristics , about 2 This is an axiom , At least one is unsatisfied .
These documents include
DeepLift (Shrikumar et al., 2016; 2017), Layer-wise relevance propagation (LRP) (Binder et al., 2016), Deconvolutional networks (Zeiler & Fergus, 2014), and Guided back-propagation (Springenberg et al., 2014).

Based on these two axioms , The author puts forward a new attribution method , Integral gradient .

axiom ： Sensitivity
An attribution method should satisfy sensitivity for all inputs and benchmark inputs , That is, for different input characteristics , When different prediction results are produced , The attribution of this different characteristic （ attribute ） Not 0.

axiom ： Implement invariance
If two completely different ways of implementing the network for all inputs , All outputs are equal , Then the two networks are functionally equivalent . Attribution methods should meet the requirements of realizing invariance , about 2 A network with exactly the same functions , Attribution should be consistent .

The integral gradient method starts by integrating the gradient along different paths , It is expected that the contribution of non-zero gradient in unsaturated region to the importance of decision-making . When scaling along this path , Which pixels add the most to the correct category of network output ？ By integrating on the path , Integral gradient avoids the problem of local gradient saturation .

The original integral gradient method uses pure black pictures , Noise picture as integral baseline .Distill tried 4 Different integration baselines . The integral path is generally chosen as linear interpolation , I don't know whether people have considered different interpolation paths .

ref
https://mp.weixin.qq.com/s?__biz=MzU0NjgzMDIxMQ==&mid=2247590903&idx=4&sn=8b2cda04da6ed8b3761e63d4abf3865e&chksm=fb54811bcc23080d6b43608cc2c43cb3be4c838b7623bbdf145789e6127d0af774089ecb3c3c&scene=126&&sessionid=0

https://blog.csdn.net/c9Yv2cf9I06K2A9E/article/details/107828644

https://distill.pub/2020/attribution-baselines/