I am using the HELOC Dataset and trying to explain a single test instance using prototypes from my training subset using below code:

explainer = ProtodashExplainer() (W, S, _) = explainer.explain(dfTrain.to_numpy(), dfTest.iloc[0:1,:].to_numpy(), m=2)

However, I am getting below error:

Is this intentional? Please help.

Thank you

Includes the Ripper algorithm and TRXF ruleset exchange format.

The rule_induction directory will eventually contain a set of closely related algorithms (drop-in replacements) that are used to induce and export rule set in the common TRXF format for consumption by AIMEE, ADS, and RedHat Decision Manager, etc. This originates from the internal aix360i:ripper branch, and we intend to migrate parts of that code as the quality bar is achieved.

In particular, as previously discussed with @vijay-arya, the reason for this migration to the public repo is to provide the more technical ADS (CP4BA) clients with the means to programmatically generate their own rule sets without relying on the AIMEE GUI.

Required dependencies:

• numpy
• pandas
• sklearn
• nyoka
• xmltodict
• numba

If I change the name:

ValueError Traceback (most recent call last) in 2 from aix360.algorithms.rbm import FeatureBinarizer 3 fb = FeatureBinarizer(negations=True, returnOrd=True) ----> 4 dfTrain, dfTrainStd = fb.fit_transform(dfTrain) 5 dfTest, dfTestStd = fb.transform(dfTest) 6 dfTrain['MostRecentBillAmountRaw'].head()

~/opt/anaconda3/envs/aix360/lib/python3.6/site-packages/sklearn/base.py in fit_transform(self, X, y, **fit_params) 697 if y is None: 698 # fit method of arity 1 (unsupervised transformation) --> 699 return self.fit(X, **fit_params).transform(X) 700 else: 701 # fit method of arity 2 (supervised transformation)

~/PycharmProjects/AIX360/aix360/algorithms/rbm/features.py in fit(self, X) 111 self.ordinal = ordinal 112 # Fit StandardScaler to ordinal features --> 113 self.scaler = StandardScaler().fit(data[ordinal]) 114 return self 115

~/opt/anaconda3/envs/aix360/lib/python3.6/site-packages/sklearn/preprocessing/_data.py in fit(self, X, y, sample_weight) 728 # Reset internal state before fitting 729 self._reset() --> 730 return self.partial_fit(X, y, sample_weight) 731 732 def partial_fit(self, X, y=None, sample_weight=None):

~/opt/anaconda3/envs/aix360/lib/python3.6/site-packages/sklearn/preprocessing/_data.py in partial_fit(self, X, y, sample_weight) 766 X = self._validate_data(X, accept_sparse=('csr', 'csc'), 767 estimator=self, dtype=FLOAT_DTYPES, --> 768 force_all_finite='allow-nan', reset=first_call) 769 n_features = X.shape[1] 770

~/opt/anaconda3/envs/aix360/lib/python3.6/site-packages/sklearn/base.py in _validate_data(self, X, y, reset, validate_separately, **check_params) 419 out = X 420 elif isinstance(y, str) and y == 'no_validation': --> 421 X = check_array(X, **check_params) 422 out = X 423 else:

~/opt/anaconda3/envs/aix360/lib/python3.6/site-packages/sklearn/utils/validation.py in inner_f(*args, **kwargs) 61 extra_args = len(args) - len(all_args) 62 if extra_args <= 0: ---> 63 return f(*args, **kwargs) 64 65 # extra_args > 0

~/opt/anaconda3/envs/aix360/lib/python3.6/site-packages/sklearn/utils/validation.py in check_array(array, accept_sparse, accept_large_sparse, dtype, order, copy, force_all_finite, ensure_2d, allow_nd, ensure_min_samples, ensure_min_features, estimator) 538 539 if all(isinstance(dtype, np.dtype) for dtype in dtypes_orig): --> 540 dtype_orig = np.result_type(*dtypes_orig) 541 542 if dtype_numeric:

<array_function internals> in result_type(*args, **kwargs)

ValueError: at least one array or dtype is required

With pandas version > 1.1.0, the line 148 (145, and 150) return an error: ValueError: cannot reindex from a duplicate axis. Locally, locally I just added '.values'. Example of line 148: colKeep[i[0]] = ((Xp[i[0]].columns.get_level_values(0) == '<=') & (thresh > i[2])).values

Issue: #112

I've just noticed that the FeatureBinarizer, when including the negated columns as well, does not work when using a dataset where there is a binary categorical feature. ~~That's probably another Pandas version error, where the 1.0.0 or newer Pandas versions work significantly different than they previously did.~~ (Got the error using Pandas 0.25.3)

When calling fb.fit_transform(<dataset_with_binary_category>, negations=True) The error message was: TypeError: unsupported operand type(s) for -: 'int' and 'Categorical'
At line 142. in function transform(): A[(str(c), 'not', '')] = 1 - A[(str(c), '', '')]
where A[(str(c), '', '')] = data[c].map(maps[c]) and c is a specific column

At that line the substraction does not work, because the Series A[(str(c), '', '')] is categorical.

Solution:
For a solution just convert the type of A[(str(c), '', '')] to integer as A[(str(c), '', '')] = data[c].map(maps[c]).astype(int). Although it could be solvable in many formats, I've seen the pattern astype(int) elsewhere in the codebase, so I hope that the solution is satisfactory.

Hi there,

I've actually copied the code (did no modification at all) from the BRCG part of the "Credit Approval Tutorial" code and ran into errors. I'm quite sure that the dataset was loaded appropriately, as I have also trained a scikit learn Decision Tree Classifier on it with no problem and in the same notebook.

Can someone help me with this issue? Am I missing something or is it an internal problem?

Thanks in advance!

Here is the code and the output. It was run on google colab, with pandas 1.1.2 and the latest aix360 release, which is 0.2.0.

Copied code

import warnings
warnings.filterwarnings('ignore')

# Load FICO HELOC data with special values converted to np.nan
from aix360.datasets.heloc_dataset import HELOCDataset, nan_preprocessing
data = HELOCDataset(custom_preprocessing=nan_preprocessing).data()
# Separate target variable
y = data.pop('RiskPerformance')

# Split data into training and test sets using fixed random seed
from sklearn.model_selection import train_test_split
dfTrain, dfTest, yTrain, yTest = train_test_split(data, y, random_state=0, stratify=y)
dfTrain.head().transpose()

# Binarize data and also return standardized ordinal features
from aix360.algorithms.rbm import FeatureBinarizer
fb = FeatureBinarizer(negations=True, returnOrd=True)
dfTrain, dfTrainStd = fb.fit_transform(dfTrain)
dfTest, dfTestStd = fb.transform(dfTest)
dfTrain['ExternalRiskEstimate'].head()

# Instantiate BRCG with small complexity penalty and large beam search width
from aix360.algorithms.rbm import BooleanRuleCG
br = BooleanRuleCG(lambda0=1e-3, lambda1=1e-3, CNF=True)

# Train, print, and evaluate model
br.fit(dfTrain, yTrain)
from sklearn.metrics import accuracy_score
print('Training accuracy:', accuracy_score(yTrain, br.predict(dfTrain)))
print('Test accuracy:', accuracy_score(yTest, br.predict(dfTest)))
print('Predict Y=0 if ANY of the following rules are satisfied, otherwise Y=1:')
print(br.explain()['rules'])

Output

Learning CNF rule with complexity parameters lambda0=0.001, lambda1=0.001
Initial LP solved
---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
/usr/local/lib/python3.6/dist-packages/pandas/core/series.py in __setitem__(self, key, value)
   1001         try:
-> 1002             self._set_with_engine(key, value)
   1003         except (KeyError, ValueError):

/usr/local/lib/python3.6/dist-packages/pandas/core/series.py in _set_with_engine(self, key, value)
   1032         # fails with AttributeError for IntervalIndex
-> 1033         loc = self.index._engine.get_loc(key)
   1034         validate_numeric_casting(self.dtype, value)

pandas/_libs/index.pyx in pandas._libs.index.BaseMultiIndexCodesEngine.get_loc()

KeyError: 'ExternalRiskEstimate'

During handling of the above exception, another exception occurred:

ValueError                                Traceback (most recent call last)
<ipython-input-98-8d81fbd6c0e1> in <module>()
     26 
     27 # Train, print, and evaluate model
---> 28 br.fit(dfTrain, yTrain)
     29 from sklearn.metrics import accuracy_score
     30 print('Training accuracy:', accuracy_score(yTrain, br.predict(dfTrain)))

/usr/local/lib/python3.6/dist-packages/aix360/algorithms/rbm/boolean_rule_cg.py in fit(self, X, y)
    118         UB = min(UB.min(), 0)
    119         v, zNew, Anew = beam_search(r, X, self.lambda0, self.lambda1,
--> 120                                     K=self.K, UB=UB, D=self.D, B=self.B, eps=self.eps)
    121 
    122         while (v < -self.eps).any() and (self.it < self.iterMax):

/usr/local/lib/python3.6/dist-packages/aix360/algorithms/rbm/beam_search.py in beam_search(r, X, lambda0, lambda1, K, UB, D, B, wLB, eps, stopEarly)
    285             if i[1] == '<=':
    286                 thresh = Xp[i[0]].columns.get_level_values(1).to_series().replace('NaN', np.nan)
--> 287                 colKeep[i[0]] = (Xp[i[0]].columns.get_level_values(0) == '>') & (thresh < i[2])
    288             elif i[1] == '>':
    289                 thresh = Xp[i[0]].columns.get_level_values(1).to_series().replace('NaN', np.nan)

/usr/local/lib/python3.6/dist-packages/pandas/core/series.py in __setitem__(self, key, value)
   1008             else:
   1009                 # GH#12862 adding an new key to the Series
-> 1010                 self.loc[key] = value
   1011 
   1012         except TypeError as e:

/usr/local/lib/python3.6/dist-packages/pandas/core/indexing.py in __setitem__(self, key, value)
    668 
    669         iloc = self if self.name == "iloc" else self.obj.iloc
--> 670         iloc._setitem_with_indexer(indexer, value)
    671 
    672     def _validate_key(self, key, axis: int):

/usr/local/lib/python3.6/dist-packages/pandas/core/indexing.py in _setitem_with_indexer(self, indexer, value)
   1790                 # setting for extensionarrays that store dicts. Need to decide
   1791                 # if it's worth supporting that.
-> 1792                 value = self._align_series(indexer, Series(value))
   1793 
   1794             elif isinstance(value, ABCDataFrame):

/usr/local/lib/python3.6/dist-packages/pandas/core/indexing.py in _align_series(self, indexer, ser, multiindex_indexer)
   1909             # series, so need to broadcast (see GH5206)
   1910             if sum_aligners == self.ndim and all(is_sequence(_) for _ in indexer):
-> 1911                 ser = ser.reindex(obj.axes[0][indexer[0]], copy=True)._values
   1912 
   1913                 # single indexer

/usr/local/lib/python3.6/dist-packages/pandas/core/series.py in reindex(self, index, **kwargs)
   4397     )
   4398     def reindex(self, index=None, **kwargs):
-> 4399         return super().reindex(index=index, **kwargs)
   4400 
   4401     def drop(

/usr/local/lib/python3.6/dist-packages/pandas/core/generic.py in reindex(self, *args, **kwargs)
   4457         # perform the reindex on the axes
   4458         return self._reindex_axes(
-> 4459             axes, level, limit, tolerance, method, fill_value, copy
   4460         ).__finalize__(self, method="reindex")
   4461 

/usr/local/lib/python3.6/dist-packages/pandas/core/generic.py in _reindex_axes(self, axes, level, limit, tolerance, method, fill_value, copy)
   4480                 fill_value=fill_value,
   4481                 copy=copy,
-> 4482                 allow_dups=False,
   4483             )
   4484 

/usr/local/lib/python3.6/dist-packages/pandas/core/generic.py in _reindex_with_indexers(self, reindexers, fill_value, copy, allow_dups)
   4525                 fill_value=fill_value,
   4526                 allow_dups=allow_dups,
-> 4527                 copy=copy,
   4528             )
   4529             # If we've made a copy once, no need to make another one

/usr/local/lib/python3.6/dist-packages/pandas/core/internals/managers.py in reindex_indexer(self, new_axis, indexer, axis, fill_value, allow_dups, copy, consolidate)
   1274         # some axes don't allow reindexing with dups
   1275         if not allow_dups:
-> 1276             self.axes[axis]._can_reindex(indexer)
   1277 
   1278         if axis >= self.ndim:

/usr/local/lib/python3.6/dist-packages/pandas/core/indexes/base.py in _can_reindex(self, indexer)
   3283         # trying to reindex on an axis with duplicates
   3284         if not self.is_unique and len(indexer):
-> 3285             raise ValueError("cannot reindex from a duplicate axis")
   3286 
   3287     def reindex(self, target, method=None, level=None, limit=None, tolerance=None):

ValueError: cannot reindex from a duplicate axis

Hi!

I'm encountering an error with a simple use case of ProtoDash to get prototypes of a given dataset. Here's an example that triggers the error:

import pandas as pd
from sklearn import datasets
from aix360.algorithms.protodash import PDASH

# Load Iris
X, y = datasets.load_iris(True)
df = pd.DataFrame(X, columns=range(X.shape[1]))
df['y'] = y

tmp = df[df['y'] == 0].drop('y', axis=1).values
X_1 = PDASH.HeuristicSetSelection(X=tmp, Y=tmp, m=10, kernelType='gaussian', sigma=2)

# This generates an error:
# ---------------------------------------------------------------------------
# ValueError                                Traceback (most recent call last)
# <ipython-input-48-e631ba33f62a> in <module>
#      1 tmp = df[df['y'] == 0].drop('y', axis=1).values
# ----> 2 X_1 = PDASH.HeuristicSetSelection(X=tmp, Y=tmp, m=10, kernelType='gaussian', sigma=2)
#
# c:\users\pc\aix\aix360\aix360\algorithms\protodash\PDASH_utils.py in HeuristicSetSelection(X, Y, m, kernelType, sigma)
#    267             currK = K2
#    268             if maxGradient <= 0:
#--> 269                 newCurrOptw = np.vstack((currOptw[:], np.array([0])))
#    270                 newCurrSetValue = currSetValue
#    271             else:
#
#~\AppData\Local\Continuum\anaconda3\envs\aix360\lib\site-packages\numpy\core\shape_base.py in vstack(tup)
#    281     """
#    282     _warn_for_nonsequence(tup)
#--> 283     return _nx.concatenate([atleast_2d(_m) for _m in tup], 0)
#    284 
#    285 
#
#ValueError: all the input array dimensions except for the concatenation axis must match exactly

Interestingly, the error does not pop up for m < 10.

Is this a bug or am I using it incorrectly?

Thanks,

Hi,

I tried to run an example notebook via Docker.

I followed the instructions to build the Docker image and run the Jupyter server here

However, when I run the notebook examples/rule_induction/brcg_demo.ipynb

Importing the dependent libs simply gave:

---------------------------------------------------------------------------
ModuleNotFoundError                       Traceback (most recent call last)
<ipython-input-3-33e99aef874b> in <module>
      3 from sklearn.model_selection import train_test_split
      4 from sklearn.metrics import precision_score, recall_score, accuracy_score, balanced_accuracy_score
----> 5 from aix360.algorithms.rule_induction.rbm.boolean_rule_cg import BooleanRuleCG as BRCG
      6 from aix360.algorithms.rbm import FeatureBinarizer
      7 import time

ModuleNotFoundError: No module named 'aix360.algorithms.rule_induction'

Hello! I've got a problem you might be able to help me with.

I'm trying to use CEM to explain scikit-learn binary classification models. I'm using RandomForestClassifier & KNeighborsClassifier to be more exact.

Here's a snipet using K-Neighbors:

from sklearn.neighbors import KNeighborsClassifier

model_kneighbors = KNeighborsClassifier()

fit_kneighbors = model_kneighbors.fit(X_train, y_train_)
y_pred_kneighbors = model_kneighbors.predict(X_test)

cem_kneighbors_clas = CEMExplainer(model_kneighbors)

arg_mode = 'PN'             # Find pertinent negatives
arg_max_iter = 1000         # Maximum number of iterations to search for the optimal PN for given parameter settings
arg_init_const = 10.0       # Initial coefficient value for main loss term that encourages class change
arg_b = 9                   # No. of updates to the coefficient of the main loss term
arg_kappa = 0.2             # Minimum confidence gap between the PNs (changed) class probability and original class' probability
arg_beta = 1e-1             # Controls sparsity of the solution (L1 loss)
arg_gamma = 100             # Controls how much to adhere to a (optionally trained) auto-encoder
my_AE_model = None          # Pointer to an auto-encoder
arg_alpha = 0.01            # Penalizes L2 norm of the solution
arg_threshold = 1.          # Automatically turn off features <= arg_threshold if arg_threshold < 1
arg_offset = 0.5            # the model assumes classifier trained on data normalized
                            # in [-arg_offset, arg_offset] range, where arg_offset is 0 or 0.5

(adv_pn, delta_pn, info_pn) = cem_kneighbors_clas.explain_instance(
    input_X=X_to_explain_clas,      # input_X (numpy.ndarray) – input instance to be explained
    arg_mode=arg_mode,              # arg_mode (str) – ‘PP’ or ‘PN’
    AE_model=my_AE_model,           # AE_model – Auto-encoder model
    arg_kappa=arg_kappa,            # arg_kappa (double) – Confidence gap between desired class and other classes
    arg_b=arg_b,                    # arg_b (double) – Number of different weightings of loss function to try
    arg_max_iter=arg_max_iter,      # arg_max_iter (int) – For each weighting of loss function number of iterations to search
    arg_init_const=arg_init_const,  # arg_init_const (double) – Initial weighting of loss function
    arg_beta=arg_beta,              # arg_beta (double) – Weighting of L1 loss
    arg_gamma=arg_gamma             # arg_gamma (double) – Weighting of auto-encoder                  
)

When I try to run it I get: 'KNeighborsClassifier' object has no attribute 'predict_long'. I checked out the CEM implementation and found the predict_long call. I also tried the HELOC tutorial example and it all works well, but with a classifier named KerasClassifier, that I think has been deprecated? I can't find documentation for it anywhere.

Based on this, I've got a few questions:

1. What estimators are compatible with CEM? The other classes are compatible with scikit-learn, but perhaps CEM is for NNs or TensorFlow only? I skimmed the paper but didn't quite catch that. I know there's a variation that deals with images, but "regular" CEM should work for tabular data, correct? Since ProtoDash is compatible with these estimators, I assumed CEM was too.
2. Perhaps I simply need to downgrade the package? I didn't install anything in particular in my Jupyter Notebook, just aix360 (didn't specify a version either, just ran pip install aix360).
3. If I were to "tweak" the implementation and change predict_long for the usual scikit-learn predict method, would it work? I don't fully understand the code, to be honest.
4. If scikit-learn estimators aren't supported for CEM at the moment, is there an implementation planned for the future?

Superb work with the library, by the way. I'm a huge fan of IBM Research and the amazing work you all do.

Regards from a fellow IBMer!

Kindly, Josefina

• Categorical datafields with str type now includes the list of possible/legal values in the PMML file.
• Fixed one test that was failing in python 3.8 in rule induction

More details in the individual commit messages.

Edit: It was 3.8 not 3.7

Summary

Adding a White Box Explainer for Matchings, as described in the upcoming ICML 2022 publication.

• the below algorithm is implemented in a python package, which is imported into AIX360

Fabian Lim, Laura Wynter, Shiau Hong Lim. 2022. "Order Constraints in Optimal Transport". https://arxiv.org/abs/2110.07275.

Algorithm

Given a matching, provide an explaination in terms of returning alternate matchings that each focus on a sparse set of salient matches.

• inherit from the LocalWBExplainer since it requires access to the internal coefficients of a matching (white box) and does not require retraining over a dataset (locality)

Package Dependencies

In setup.py we have the following dependency that is installed as an egg

• 'otoc @ git+https://github.com/IBM/[email protected]#egg=otoc'

In examples/matching/matching-pairs-of-sentences.ipynb we ask the user to install the below packages in order to execute the demo

• POT==0.7.0

Example

An NLP-based example inspired from one of the figures in the paper is provided

TODO

• [x] algorithm has been added in here.
• [x] semantic dataset used in examples has been included in here.
• [x] examples has been included in here. With the following 2 subdirectories
  • data: storing NLP embeddings that will be downloaded for the demo
  • models: contains code for an NLP embedding model used in demo
  • utils: utility functions for the demo
• [x] docs have been updated. Sphinx has been run locally and tested.
• [x] Update READMEs in examples, top-level directory, examples, etc
• [x] tests has been included in here.
  • data: each test case is specified in a .json file and stored here.
• [x] Update setup.py. The python package is hosted publically at https://github.com/IBM/otoc.
  • [x] This repo is placed in install_requires as an egg link
  • [x] Update the GitHub link in the top-level README
• [ ] Discuss with @vijay-arya on where to move the below items
  • examples/matching/data
  • examples/matching/models
  • tests/matching/data

When trying to execute protodash, getting the error: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison. Any reason why?

Hi Trusted-AI/AIX360!

This is a one-off automatically generated pull request from LGTM.com :robot:. You might have heard that we’ve integrated LGTM’s underlying CodeQL analysis engine natively into GitHub. The result is GitHub code scanning!

With LGTM fully integrated into code scanning, we are focused on improving CodeQL within the native GitHub code scanning experience. In order to take advantage of current and future improvements to our analysis capabilities, we suggest you enable code scanning on your repository. Please take a look at our blog post for more information.

This pull request enables code scanning by adding an auto-generated codeql.yml workflow file for GitHub Actions to your repository — take a look! We tested it before opening this pull request, so all should be working :heavy_check_mark:. In fact, you might already have seen some alerts appear on this pull request!

Where needed and if possible, we’ve adjusted the configuration to the needs of your particular repository. But of course, you should feel free to tweak it further! Check this page for detailed documentation.

Questions? Check out the FAQ below!

FAQ

Implemented the reader portion of the scorecard pmml export functionality. Together with the writer portion https://github.com/Trusted-AI/AIX360/pull/166, exporting TRXF scorecards to pmml is possible.

I found a strange phenomenon. For the same model, the same training sample and test sample, other operations are identical. Theoretically, the values obtained by using the XAI method (like Saliency) to evaluate the interpretability of the model should be the same. However, I retrained a new model, and the interpretability values obtained are completely different from those obtained from the previous model. Does anyone know why this happens? The interpretability value is completely unstable, and the results cannot be reproduced. Unless I completely save this model after training it, and then reload this parameter, the results will be the same. Does anyone know why

Ripper algorithm recognizes timestamp features (e.g. 2022-06-14-19.39.35.929641) as integers, and thus encodes them to categorical features. The resulting rules are in terms of equality predicates (e.g. timestamp == 2022-06-14-19.39.35.929641) instead of intervals/inequalities as one would expect.

Proper timestamp type support for Ripper would be nice.