Select the modules to which the bug refers:

• [X] Attributions Methods
• [ ] Feature Visualization
• [ ] Concepts
• [ ] Metrics
• [ ] Documentation

Describe the bug Some returned explanations are 4 dimensional (batch dim + hwc images), which does not work with the plot_attributions method in xplique/plots/image.py as the method is suited to handle only 2D images with no channel dimension. The bug affects the following methods, which seem to return 4-dimensional explanations, which have explanations of shape (6, 224, 224, 3) as opposed to (6, 224, 224): GradientInput, GuidedBackprop, IntegratedGradients, SmoothGrad, SquareGrad, VarGrad

Screenshots Stack trace:

[/usr/local/lib/python3.7/dist-packages/xplique/plots/image.py](https://localhost:8080/#) in plot_attributions(explanations, images, cmap, alpha, clip_percentile, absolute_value, cols, img_size, **plot_kwargs)
    154     rows = ceil(len(explanations) / cols)
    155     # get width and height of our images
--> 156     l_width, l_height = explanations.shape[1:]
    157 
    158     # define the figure margin, width, height in inch

ValueError: too many values to unpack (expected 2)

Desktop (please complete the following information): Using default Google Collab notebook.

To Reproduce Simply run all in Getting_started.ipynb

Expected behavior plot_attributions should be able to handle images with multiple channels to produce the visualizations.

Additional context None

It would be a good practice to have a template when people submit issues. It could even be different template depending on the issue: do we need to reproduce the error, should we have the machine configuration, is it a comfort issue (e.g I can have a hotfix but it would be a nice enhancement that I do not use it), etc....

@justinplakoo do you have any idea how we should manage that ? I think we can add an issue template, what do you think ?

Hey, Is it possible to run GradCam visuals on a semantic segmentation model using this library? If yes, kindly help me out with how I can proceed.

Specifically, I have trained a FCN segmentation model using VGG-19 as the backbone, in Keras, and have the trained model, and wish to interpret the segmentation behaviour using gradcam/gradcam plus etc. (my model, has only two categories, background and foreground)

Any help is appreciated, Thank you.

name: Bug report about: attributions of sklearn wrapped models title: "[BUG] - attributions of sklearn wrapped models are incoherent" labels: ''wrapper", "bug", "attributions" assignees: ''

Select the modules to which the bug refers:

• [x] Attributions Methods
• [ ] Feature Visualization
• [ ] Concepts
• [ ] Metrics
• [ ] Documentation

Describe the bug When a wrapper is used on regression model from sklearn, the obtained attributions are not coherent. (Problem can be larger).

Screenshots

Desktop:

• OS - windows
• Python version - 3.7.12
• Sklearn - 1.0.2
• Xplique - 0.2.6

To Reproduce

import numpy as np
from numpy.random import seed, normal
import sklearn
import xplique
from sklearn.linear_model import LinearRegression
from sklearn.model_selection import train_test_split

seed(0)

# dataset parameters
features_coef = np.array([1, -2, -3, 4, 0, 0, 0, 0])
nb_samples = 1000
nb_features = len(features_coef)


# create dataset
dataset = normal(0, 1, (nb_samples, nb_features))
noise = normal(0, 1, nb_samples)
target = dataset.dot(features_coef) + noise

# split dataset
X_train, X_test, y_train, y_test = train_test_split(dataset, target, test_size=0.05, shuffle=True)


# train model
sk_linear = LinearRegression().fit(X_train, y_train)

# Create the wrapper class
class Wrapper():
    # The init method is necessary for every class
    def __init__(self, model):
        self.model = model

    # The call method calls the predict method
    def __call__(self, inputs):
        return self.model.predict(inputs)

# wrap model
sk_model = Wrapper(sk_linear)

# adapt inputs/outputs
inputs_tf = tf.cast(X_test, tf.float32)
targets_tf = tf.ones((X_test.shape[0], 1))

# instanciate explainer
explainer = KernelShap(
    model,
    nb_samples=200,  # 2000
    ref_value=0.0
)

# compute explanations
explanations = abs(explainer(inputs_tf, targets_tf))

print(np.array(explanations).mean(axis=0))

# [4.328179, 4.357149, 4.6055717, 5.554367, 3.5661576, 4.1552, 3.5590754, 4.7494626]

To ease the debugging, here is a minimal example notebook (There are several lines for the visualization, but you can jump to the end of the notebook to see the different attributions) : https://colab.research.google.com/drive/1zvt4I9lVpvzg1oWPUNZoFs39w8MPSz_b?usp=sharing

Expected behavior The 4 last attributions values should be close to 0, far inferior to the 4 first.

Additional context _

Add causal fidelity metrics for tabular data

The subject of this pull request is to add causal fidelity metrics for tabular data (deletion and insertion):

• Add metrics 318a019
• Add docs 7809cd8
• Add unit tests 98f7245

Other contribution

• Correct xplique/plots/tabular.py to avoid warnings b193084

Plots for metrics and timeseries

For metrics, the plots are added in 996fad3 and tested in d755524. For timeseries, the plots are added in 70ded0d and tested in be8ff60.

Metrics

Those are two plots that I often use. I though they may need to be integrated in Xplique.

I also made is so that each method can be assign a color and match between both plots.

Metrics histograms

This one to compare the attribution methods through several metrics:

Fidelity curves

This plot show the evolution of the score depending on the number of features perturbed. It is also used to compare methods. It can easily be done thank to the detailed_evaluate() method of CausalFidelity and CausalFidelity introduced in pull request #70.

Note that the lines on this plot are just here for testing the function.

Timeseries

The following plots come from the same function, they both output heatmaps. But for the first, a numpy array is given while the second receives a dictionary of numpy arrays.

One explanation heatmap

Several explanations heatmaps

This plot should adapt the arrangement of the subplots based on the heatmap shape and their number.

class AverageStability(ExplainerMetric): """ Used to compute the average sensitivity metric (or stability). This metric ensure that close inputs with similar predictions yields similar explanations. For each inputs we randomly sample noise to add to the inputs and compute the explanation for the noisy inputs. We then get the average distance between the original explanations and the noisy explanations. Ref. Bhatt & al., Evaluating and Aggregating Feature-based Model Explanations (2020). https://arxiv.org/abs/2005.00631 (def. 2) Parameters ---------- model Model used for computing metric. inputs Input samples under study. targets One-hot encoded labels or regression target (e.g {+1, -1}), one for each sample. batch_size Number of samples to explain at once, if None compute all at once. radius Radius defining the neighborhood of the inputs with respect to l1 distance. distance Distance metric between the explanations. nb_samples Number of different neighbors points to try on each input to measure the stability. """

def __init__(self,
             model: Callable,
             inputs: Union[tf.data.Dataset, tf.Tensor, np.ndarray],
             targets: Optional[Union[tf.Tensor, np.ndarray]] = None,
             batch_size: Optional[int] = 64,
             radius: float = 0.1,
             distance: Union[str, Callable] = 'l2',
             nb_samples: int = 20):
    # pylint: disable=R0913
    super().__init__(model, inputs, targets, batch_size)
    self.nb_samples = nb_samples

    if distance == 'l1':
        self.distance = lambda x, y: tf.reduce_sum(tf.abs(x - y))
    elif distance == 'l2':
        self.distance = lambda x, y: tf.reduce_sum((x-y)**2.0)
    elif hasattr(distance, '__call__'):
        self.distance = distance
    else:
        raise ValueError(f"{distance} is not a valid distance.")

    # prepare the noisy masks that will be used to generate the neighbors
    nb_variables = np.prod(inputs.shape[1:])
    if distance == 'l1':
        self.noisy_masks = tf.random.uniform((nb_samples, *inputs.shape[1:]), 0, radius/nb_variables)
    elif distance == 'l2':
        self.noisy_masks = tf.random.uniform((nb_samples, *inputs.shape[1:]), 0, np.sqrt(radius/nb_variables))
    elif hasattr(distance, '__call__'):
        # find the right radius by evaluating the distance when selecting inputs and by repeating the process
        epsilon = 1e-6
        radius_tp = radius
        radius_min = 0.
        radius_max = 10 * radius
        while np.array(self.distance(radius_max, np.zeros(nb_variables))) <= radius:
            radius_max *= 10
        while (np.abs(self.distance(radius_tp, np.zeros(nb_variables)) - radius) > epsilon) and (radius_max - radius_min > epsilon):
            if np.array(self.distance(radius_tp, np.zeros(nb_variables))) > radius:
                radius_max = radius_tp
                radius_tp = (radius_tp - radius_min)/2
            else:
                radius_min = radius_tp
                radius_tp = (radius_max - radius_tp)/2 + radius_min
        radius = radius_tp
        self.noisy_masks = tf.random.uniform((nb_samples, *inputs.shape[1:]), 0, radius)

def evaluate(self,

Update Python version for CI

Commit 217fb77 adress #120.

Docs

27d48d3 allow to justify content in the generated documentation

Templates

4412551 modify issues template for a more user-friendly one

Rebase develop into master.

This PR includes:

#114 fix a missing module in the init of plot. #115 which enhance documentation and smoothgrad batching mechanism. #117 which fix a bug in the plot module and another one concerning the miniature of the feature viz.

I will push the v0.4.2 on pypi this afternoon. Congrats everyone (@TeodorChiaburu, @AntoninPoche, @lucashervier) 🎉

I was not able to import plot_attribution (the one without the 's' at the end, used for plotting single images). I just added the function's name to the __init__ in xplique.plots. Now it can be imported.

this PR introduces a refactoring in the Sobol module henceforth called Global Sensitivity Analysis so that code can be shared with the HSIC method soon to be introduced.

As a matter of fact, the estimators, sampler and the core of the method can be shared.

Fix

9c44321ff8aac46c7e454371774d5a1531fa163c reduce drastically the number of forward during sanity check tests (when we test the method can be called with correct args, should return a correct shape...) to speed up the tests.

ef79ee5115d7f2aa398f39ccbeb33822b5fe766f relax the strict equality constraint on an almost equal to take into account numerical issue when doing the tests on gpu (e.g on my new mac haha).

Preparing HSIC

We will soon introduce the new HSIC Attribution method. In order to avoid code redondance with Sobol as they are both Global Sensitivity Analysis methods, I propose an abstraction called GSABaseAttributionMethod that will be parametrized by an estimator and a sampler and take care of doing the forward and apply the perturbations.

But first, we need to rename the sobol module into global_sensitivity_analysis b797b3fa1c71c1dddb41331d6f4664cb96b76eaa.

The sampler module now need to be more specific as for Sobol estimation we use a specific sampling mechanism (replicated design), ac2677365032ae807ee3bccb199bad4a050ebcb9 rename and specify each XXXsampler into XXXsamplerRS for replicated design.

Similarly, the estimator module is now the sobol_estimatormodule 37b59f88f93bf2624573d967e682339ed266d9e2.

Now the big part, I abstracted the forward / perturbation mechanism of the SobolAttributionMethod and created a new base explainer for GSA based attribution 35e4f3a3015b9e86984975c9bcd2efda47c07508.

See how SobolAttributionMethod is super slim ! Also, good news It would be the same with HSIC ! ;)

Select the modules to which the bug refers:

• [ ] Attributions Methods
• [ ] Feature Visualization
• [ ] Concepts
• [x] Metrics
• [ ] Documentation

Describe the bug While running the Average Stability metric, I receive the unhashable type error regardless of whether I pass the images and labels as tensor or numpy array.

Screenshots

Desktop (please complete the following information):

• Windows 10
• Python version: 3.8.10
• Tensorflow version: 2.10.1
• Packages used version: numpy: 1.23.5

To Reproduce Run AverageStability metric with np array of images and labels array of the same length.

Expected behavior I expected to just receive a score just like with Deletion and Insertion metrics, which worked right away.

Is your feature request related to a problem? Please describe. Workflow dependant on Python 3.6 is deprecated. We should however cover Python 3.9 and maybe 3.10

Describe the solution you'd like Update workflow

Select the modules to which the bug refers:

• [ ] Metrics

Describe the bug Passing a tf.data.Dataset to metrics constructors derives into the program incorrectly accessing an element_spec of the aforementioned dataset as if it were a list when it is not the case.

Screenshots

2022-03-17 11:44:57.703227: W tensorflow/stream_executor/platform/default/dso_loader.cc:60] Could not load dynamic library 'cudart64_110.dll'; dlerror: cudart64_110.dll not found
2022-03-17 11:44:57.704477: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
2022-03-17 11:45:01.816089: I tensorflow/compiler/jit/xla_cpu_device.cc:41] Not creating XLA devices, tf_xla_enable_xla_devices not set
2022-03-17 11:45:01.817565: W tensorflow/stream_executor/platform/default/dso_loader.cc:60] Could not load dynamic library 'nvcuda.dll'; dlerror: nvcuda.dll not found
2022-03-17 11:45:01.817889: W tensorflow/stream_executor/cuda/cuda_driver.cc:326] failed call to cuInit: UNKNOWN ERROR (303)
2022-03-17 11:45:01.822735: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:169] retrieving CUDA diagnostic information for host: B201801349
2022-03-17 11:45:01.823202: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:176] hostname: B201801349
2022-03-17 11:45:01.824379: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2022-03-17 11:45:01.826257: I tensorflow/compiler/jit/xla_gpu_device.cc:99] Not creating XLA devices, tf_xla_enable_xla_devices not set
WARNING:tensorflow:Model was constructed with shape (1, 32, 32, 3) for input KerasTensor(type_spec=TensorSpec(shape=(1, 32, 32, 3), dtype=tf.float32, name='conv2d_input'), name='conv2d_input', description="created by layer 'conv2d_input'"), but it was called on an input with incompatible shape (64, 32, 32, 3).
2022-03-17 11:45:03.985713: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:116] None of the MLIR optimization passes are enabled (registered 2)
WARNING:tensorflow:Model was constructed with shape (1, 32, 32, 3) for input KerasTensor(type_spec=TensorSpec(shape=(1, 32, 32, 3), dtype=tf.float32, name='conv2d_input'), name='conv2d_input', description="created by layer 'conv2d_input'"), but it was called on an input with incompatible shape (56, 32, 32, 3).
WARNING:tensorflow:AutoGraph could not transform <function <lambda> at 0x000001D6E9D67D30> and will run it as-is.
Cause: could not parse the source code of <function <lambda> at 0x000001D6E9D67D30>: found multiple definitions with identical signatures at the location. This error may be avoided by defining each lambda on a single line and with unique argument names.
Match 0:
(lambda z, w: z)

Match 1:
(lambda z, w: w)

To silence this warning, decorate the function with @tf.autograph.experimental.do_not_convert
WARNING:tensorflow:AutoGraph could not transform <function <lambda> at 0x000001D6E9D67E50> and will run it as-is.
Cause: could not parse the source code of <function <lambda> at 0x000001D6E9D67E50>: found multiple definitions with identical signatures at the location. This error may be avoided by defining each lambda on a single line and with unique argument names.
Match 0:
(lambda z, w: z)

Match 1:
(lambda z, w: w)

To silence this warning, decorate the function with @tf.autograph.experimental.do_not_convert
Traceback (most recent call last):
  File "C:/Users/a-m.picard/AppData/Roaming/JetBrains/PyCharmCE2021.3/scratches/scratch_1.py", line 24, in <module>
    metric = Deletion(model, dataset.map(lambda z, w: z), dataset.map(lambda z, w: w))
  File "C:\Users\a-m.picard\Anaconda3\envs\scouter-tf2\lib\site-packages\xplique\metrics\fidelity.py", line 324, in __init__
    super().__init__(model, inputs, targets, batch_size, "deletion",
  File "C:\Users\a-m.picard\Anaconda3\envs\scouter-tf2\lib\site-packages\xplique\metrics\fidelity.py", line 174, in __init__
    super().__init__(model, inputs, targets, batch_size)
  File "C:\Users\a-m.picard\Anaconda3\envs\scouter-tf2\lib\site-packages\xplique\metrics\base.py", line 36, in __init__
    self.inputs, self.targets = numpy_sanitize(inputs, targets)
  File "C:\Users\a-m.picard\Anaconda3\envs\scouter-tf2\lib\site-packages\xplique\commons\data_conversion.py", line 69, in numpy_sanitize
    inputs, targets = tensor_sanitize(inputs, targets)
  File "C:\Users\a-m.picard\Anaconda3\envs\scouter-tf2\lib\site-packages\xplique\commons\data_conversion.py", line 35, in tensor_sanitize
    dataset_shape = inputs.element_spec[0].shape
TypeError: 'TensorSpec' object is not subscriptable

Desktop (please complete the following information):

• OS: Windows, Linux
• Python version: 3.9.5
• Tensorflow version: 2.4.1 - 2.7

To Reproduce

import tensorflow as tf
from tensorflow.keras import Sequential
from tensorflow.keras.layers import Conv2D, GlobalAveragePooling2D, Dense
from xplique.attributions import Saliency
from xplique.metrics import Deletion

x = tf.random.normal((120, 32, 32, 3))
y = tf.random.normal((120, 10))
dataset = tf.data.Dataset.from_tensor_slices((x, y))
model = Sequential([
    Conv2D(8, 3, strides=2, padding='same'),
    Conv2D(8, 3, strides=2, padding='same'),
    GlobalAveragePooling2D(),
    Dense(10)
])
model(tf.random.normal((1, 32, 32, 3)))
explainer = Saliency(model, -1, batch_size=64)
explanations = []
for batch_x, batch_y in dataset.batch(64):
    exp = explainer(batch_x, batch_y)
    explanations.extend(exp)
metric = Deletion(model, dataset.map(lambda z, w: z), dataset.map(lambda z, w: w))
metric_score = metric(explanations)

Expected behavior I would expect the metric to be computed, but whilst instantiating the Deletion object, the constructor attempts to figure out the shapes of the inputs of the individual points of a tf.data.Dataset. Whilst typically straight-forward, a simple subscript bug appears, probably stemming from a incorrectly treated edge case.

Additional context The property element_spec of a dataset is not a list, and thus, the subscript is unnecessary for the desired function.

The methods Lime and KernelShap need a map_to_interpret_space function. By default, the function is the quickshift segmentation algorithm. For better results, watershed function on images have to be used. This algorithm, in the skimage, package needs a parameter (makers) which value is dedicated to the image treated and not for all images of the dataset. The current API of Lime and KernelShap cannot allowed such configuration (function + dedicated value).

A solution could be to have a parameter for the image maps computed in advance instead of compute them every time in the explicability method function.

As hotfix for the issue regarding the order of input tensor, I created a model that is an aggregation of two models. In the first one I added a permute layer to handle the channel order. The hotfix is working well in forward pass (tested). Unfortunately following error is raised by xplique. I think aggregation of models is not handled by Xplique.

ValueError                                Traceback (most recent call last)
/tmp/ipykernel_8110/184323267.py in <module>
      7 
      8 # create an explainer and generate explanations
----> 9 explainer = GradCAM(modper)
     10 explanations = explainer(X_preprocessed, Y) # `explainer.explain(inputs, labels)` also works
     11 

~/.local/lib/python3.7/site-packages/xplique/attributions/grad_cam.py in __init__(self, model, output_layer, batch_size, conv_layer)
     41                  batch_size: Optional[int] = 32,
     42                  conv_layer: Optional[Union[str, int]] = None):
---> 43         super().__init__(model, output_layer, batch_size)
     44 
     45         # find the layer to apply grad-cam

~/.local/lib/python3.7/site-packages/xplique/attributions/base.py in __init__(self, model, output_layer, batch_size)
    123             # reconfigure the model (e.g skip softmax to target logits)
    124             target_layer = find_layer(model, output_layer)
--> 125             model = tf.keras.Model(model.input, target_layer.output)
    126 
    127             # sanity check, output layer before softmax

/opt/conda/lib/python3.7/site-packages/tensorflow/python/training/tracking/base.py in _method_wrapper(self, *args, **kwargs)
    528     self._self_setattr_tracking = False  # pylint: disable=protected-access
    529     try:
--> 530       result = method(self, *args, **kwargs)
    531     finally:
    532       self._self_setattr_tracking = previous_value  # pylint: disable=protected-access

/opt/conda/lib/python3.7/site-packages/keras/engine/functional.py in __init__(self, inputs, outputs, name, trainable, **kwargs)
    107     generic_utils.validate_kwargs(kwargs, {})
    108     super(Functional, self).__init__(name=name, trainable=trainable)
--> 109     self._init_graph_network(inputs, outputs)
    110 
    111   @tf.__internal__.tracking.no_automatic_dependency_tracking

/opt/conda/lib/python3.7/site-packages/tensorflow/python/training/tracking/base.py in _method_wrapper(self, *args, **kwargs)
    528     self._self_setattr_tracking = False  # pylint: disable=protected-access
    529     try:
--> 530       result = method(self, *args, **kwargs)
    531     finally:
    532       self._self_setattr_tracking = previous_value  # pylint: disable=protected-access

/opt/conda/lib/python3.7/site-packages/keras/engine/functional.py in _init_graph_network(self, inputs, outputs)
    191     # Keep track of the network's nodes and layers.
    192     nodes, nodes_by_depth, layers, _ = _map_graph_network(
--> 193         self.inputs, self.outputs)
    194     self._network_nodes = nodes
    195     self._nodes_by_depth = nodes_by_depth

/opt/conda/lib/python3.7/site-packages/keras/engine/functional.py in _map_graph_network(inputs, outputs)
    982                              'The following previous layers '
    983                              'were accessed without issue: ' +
--> 984                              str(layers_with_complete_input))
    985         for x in tf.nest.flatten(node.outputs):
    986           computable_tensors.add(id(x))

ValueError: Graph disconnected: cannot obtain value for tensor KerasTensor(type_spec=TensorSpec(shape=(None, 3, 512, 612), dtype=tf.float32, name='input.1'), name='input.1', description="created by layer 'input.1'") at layer "87_pad". The following previous layers were accessed without issue: []

The explaination does not work for models that ask for the channel as the first dimension of the input tensor (channel, height, width. When such a model is provided, the output shape of the explanation is (channel x height) instead of (height x width)