Version 1 of the shap evaluator completed.

Functionality:

1. Output of summary statistics, currently mean and mean(|x|)
2. Optional output of shap values for specific samples in the dataset

Change description

Extended NLPGeneratorAnalyzer for flexible text attributes assessment Debugged data paths Upgraded the demo

Type of change

• [x] Bug fix (fixes an issue)
• [x] New feature (adds functionality)

Related issues

Fix #1

Checklists

Development

• [ ] Lint rules pass locally
• [ ] Application changes have been tested thoroughly
• [ ] Automated tests covering modified code pass

Security

• [ ] Security impact of change has been considered
• [ ] Code follows company security practices and guidelines

Code review

• [ ] Pull request has a descriptive title and context useful to a reviewer. Screenshots or screencasts are attached as necessary
• [ ] "Ready for review" label attached and reviewers assigned
• [ ] Changes have been reviewed by at least one other contributor
• [ ] Pull request linked to task tracker where applicable

Added evaluator schema + evaluators pages

• Restructured the docs directory to improve tidyness
• Evaluator schema/how to: docs/pages/evaluiators/make_your_own.rst
• Each evaluator has an autogenerated page, code for autogeneration: docs/autogeneration/pages/evaluators.py

• Enabled multiple models run support and more customizability
• Added prompts datasets with standard schema
• Enabled reporting
• Updated demo and documentations

Describe your changes

test-reports.yml workflow will run on pushes to develop and main. It can also be triggered for other branches but the badges will not be updated. Badges are updated by pushing a generated badge to S3 that is referenced in the README.md. The README.md references the badges for the main branch but the URL to retrieve the badges for develop can be discovered easily.

The permissions to write to the S3 bucket are limited to this workflow running on the develop and main branch.

Examples of the badges generated for the last run on this branch before lockdown in AWS:

Issue ticket number and link

This was a request in Slack.

Known outstanding issues that are not fully accounted for

Checklist before requesting a review

• [x] I have performed a self-review of my code
• [x] I have built basic tests for new functionality (particularly new evaluators)
• [ ] If new libraries have been added, I have checked that readthedocs API documentation is constructed correctly
• [ ] Will this be part of a major product update? If yes, please write one phrase about this update.

Extra-mile Checklist

• [ ] I have thought expansively about edge cases and written tests for them

Describe your changes

Setting up model types to enable multiclass classification metric routing.

Checklist before requesting a review

• [x] I have performed a self-review of my code
• [ ] I have built basic tests for new functionality (particularly new evaluators)
• [ ] If new libraries have been added, I have checked that readthedocs API documentation is constructed correctly
• [ ] Will this be part of a major product update? If yes, please write one phrase about this update.

Extra-mile Checklist

• [ ] I have thought expansively about edge cases and written tests for them

Prototype for Prism object

Overall folder/file structure

• comparators: Contains the basic logic to compare different type of results. Comparators can be a function of the result container type, and potentially the evaluators

  • comparator.py: abstract class
  • metric_comparator.py: takes a list of Lens objects, extract metric containers and compares metrics across all the Lens objects. Comparisons are done respect to a reference Lens object. Currently the reference is defined by model name, to be changed in the future if necessary. @esherman-credo I simplified the logic, the all vs all comparison produced a lot of results that were probably not going to get used, we can reinstate it if needs be. I have added the possibility to select different operations.

• compare.py: this is a type of task that prism can orchestrate. ATM it's only calling metric_comparator, in the future compare.Compare will instantiate the necessary comparators (table, metric, etc...) depending on the existing results containers.

• prism.py: the orchestrator, this is the equivalent of Lens but at a higher level. The purpose of Prism is to coordinate the various tasks. Currently when the user instantiates it, they can pass a dictionary of tasks to perform on the Lens objects. The dictionary is in the format {'task_name:{'parameter':value}}, where parameters are specific to the type of task to perform.

Missing

• Docstrings in general
• Abstract classes for task objects (like compare.Compare). I haven't created an abstract class for this yet, I am wondering if there is a need for it, or if the variety of the object will be too big for an abstract class to be useful.
• Tests

To test the run you can use the following code:

from credoai.lens import Lens
from credoai.artifacts import TabularData, ClassificationModel
from credoai.evaluators import *
from credoai.governance import Governance
import numpy as np


# imports for example data and model training
from credoai.datasets import fetch_creditdefault
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import train_test_split
from xgboost import XGBClassifier

data = fetch_creditdefault()
df = data["data"]
df["target"] = data["target"].astype(int)

# fit model
model1 = RandomForestClassifier(random_state=42)
X = df.drop(columns=["SEX", "target"])
y = df["target"]
sensitive_features = df["SEX"]
(
    X_train,
    X_test,
    y_train,
    y_test,
    sensitive_features_train,
    sensitive_features_test,
) = train_test_split(X, y, sensitive_features, random_state=42)
model1.fit(X_train, y_train)

model2 = XGBClassifier(tree_method='hist', random_state=42, n_estimators=2, enable_categorical=True)
model2.fit(X_train, y_train)



# ## Set up artifacts

credo_RF1 = ClassificationModel(
    'credit_RF1_classifier',
    model1
)

credo_XGB = ClassificationModel(
    'credit_XGB_classifier',
    model2
)

train_data = TabularData(
    name = "UCI-credit-default-train",
    X=X_train,
    y=y_train,
    sensitive_features=sensitive_features_train
)
test_data = TabularData(
    name='UCI-credit-default-test',
    X=X_test,
    y=y_test,
    sensitive_features=sensitive_features_test
)


# pipeline scan be specifed using a sklearn-like style
metrics = ['accuracy_score', 'roc_auc_score']
pipeline = [
    (Performance(metrics)),
    (ModelFairness(metrics)),
]

pipeline2 = [
    (Performance(metrics)),
    (ModelFairness(metrics)),
]


# and added to lens in one fell swoop
lens_rf1 = Lens(
    model=credo_RF1,
    assessment_data=test_data,
    # training_data=train_data,
    pipeline=pipeline
)



lens_xgb = Lens(
    model=credo_XGB,
    assessment_data=test_data,
    # training_data=train_data,
    pipeline=pipeline2
)


from credoai.prism import Prism

prism_test = Prism(
    [lens_rf1, lens_xgb], 
    tasks={'compare':{'ref':'credit_RF1_classifier'}}
)

prism_test.execute()

prism_test.get_results()

Expected Behavior

Lens should return results for each specified dataset, and for each evaluator (e.g. 2 data sets + 1 evaluator --> 2 sets of results).

Actual Behavior

Lens is currently only returning one result set when both assessment_data and training_data are passed to the Lens() constructor. It appears that only the results from the training_data are processed (likely because that parameter is listed 2nd?)

Creating data objects as follows:

credo_model = ClassificationModel(
    'credit_default_classifier',
    model
)
train_data = TabularData(
    name = "UCI-credit-default-train",
    X=X_train,
    y=y_train,
    sensitive_features=sensitive_features_train
)
test_data = TabularData(
    name='UCI-credit-default-test',
    X=X_test,
    y=y_test,
    sensitive_features=sensitive_features_test
)

Creating pipeline and Lens object as follows

# pipeline scan be specifed using a sklearn-like style
metrics = ['accuracy_score', 'roc_auc_score']
pipeline = [
    (Performance(metrics), 'Performance Assessment'),
    (ModelFairness(metrics), "Fairness Assessment"),
]


# and added to lens in one fell swoop
lens = Lens(
    model=credo_model,
    assessment_data=test_data,
    training_data=train_data,
    pipeline=pipeline
)

lens.run()

Expected Results: dictionary with 2 sets each of Performance and ModelFairness results. Running as above yields results that look like this (not enough entries and entries are clearly on training_data since performance is so strong):

Commenting out the line that passes train_data to Lens() constructor yields performance that looks like this:

This PR aims to enable support of multiple sensitive features in Lens. This is expected to require updates across the module and this working PR covers these updates as they are implemented.

Change description

• Air gapped documentation added to governance notebook.
• Increased functionality of CredoGovernance object. It now can:
  • Handle registration of models and projects
  • Handle retrieving spec from governance platform
  • Handle retrieving spec from file
• Documentation for explicit registration of models updated to reflect use of credogovernance
• Better warning, exception and verbose functionality for Lens
• dataset registration

Describe your changes

Extracted run_cv and get demo_parity from data fairness

Issue ticket number and link

Known outstanding issues that are not fully accounted for

Checklist before requesting a review

• [x] I have performed a self-review of my code
• [ ] I have built basic tests for new functionality (particularly new evaluators)
• [ ] If new libraries have been added, I have checked that readthedocs API documentation is constructed correctly
• [ ] Will this be part of a major product update? If yes, please write one phrase about this update.

Extra-mile Checklist

• [ ] I have thought expansively about edge cases and written tests for them

Refactored Ranking Fairness evaluators

• Generic metrics moved to metrics_credoai.py
• FINS metrics refactored to helper function
• Reformatted some codes and documentations

Describe your changes

This is a feature branch implementing support for tensor-based data and neural network models. For Credo-internal developers, the proposed changes are detailed here.

Summary of Changes (in progress, more to come):

• BaseModel
  • Group init functionality in process_model function
  • Split validation functionality into 2 functions:
    1. Checks that required callable functions are present (existing functionality)
    2. Checks that various details of the framework match up with the model wrapper type; throws warnings rather than errors
• ClassificationModel
  • Add support for Keras-style classifiers (highly restrictive, w/ warnings if assumptions are not met)
    • Assume Sequential-style model w/ Dense last (or 2nd to last) layer + (softmax or sigmoid)
    • Implement __post_init__() functionality for keras predictions
      • Depends on sigmoid (label outputs; no predict_proba) vs. softmax (probability outputs; predict_proba by default -> can infer predict) -DummyClassifier
  • Passing X + y artifacts to data wrapper when they're already wrapped
  • If X and y are separate in the user's non-Lens workflow, passing these to Lens will be done in the same way as for TabularData
  • If X and y are wrapped in one object (such as a tf.data or keras.utils.Sequence object) then passing them to the Lens wrapper requires separating them
  • Change: Add ability to pass model_like to DummyClassifier
    • This does not fully address usability issue above (e.g., if user wants to run dataset assessments on X and y)
    • Allows user to avoid excessive computation by Lens, while still retaining model details for, e.g., the ModelProfiler evaluator
• Base Data
  • Converted process functions to non-abstract (committed)
    • Can support tensor data (from Keras + TF, at least; see supported inputs to predict here) with base Data class
    • Working example of data-on-disk model fitting + Lens evaluation exists (not pushed to GH) Evaluator Validation
• Security evaluator needed to be refactored (a bit) to use ART's TensorflowV2Classifier rather than their KerasClassifier
  • The latter does not support eager execution
  • The former is ART's current workaround -> eager execution is important for overall TF/Keras support (allows running things on-demand rather than as a batched graph execution)
  • Tests passing

Lens Validation Validation of Model + Data --> At Lens init stage, we now verify that predict, predict_proba, and compare (whichever are relevant to the provided model) works for the provided data. Throws an error and prevents instantiating/running evaluators without first checking Model + Data compatibility.

Evaluator Validation Established a starting point for streamlining/unifying artifact checking. Converted check_artifact_for_nulls to check_data... (name reflects what it's doing) and added options to only check some parts of the artifact (i.e., a subset of X, y, and sensitive_features) for nulls rather than checking all parts. Functionality doesn't fundamentally change but makes requirements more explicit from function arguments: check_X, check_y, and check_sens (all boolean).

• If datatype is completely arbitrary or generator-like, we have no way of checking; need the user to do this before wrapping
• This revised null-checker expands capabilities to several non-Pandas types

Issue ticket number and link

https://credo-ai.atlassian.net/browse/DSP-344

Known outstanding issues that are not fully accounted for

• Need to fix/merge the Keras support in the ModelProfiler
• Tests! Tests! Tests!
• Confirm documentation is properly built

Checklist before requesting a review

• [x] I have performed a self-review of my code
• [ ] I have built basic tests for new functionality (particularly new evaluators)
• [ ] If new libraries have been added, I have checked that readthedocs API documentation is constructed correctly
• [x] Will this be part of a major product update? If yes, please write one phrase about this update.
  • See above. This expands Lens functionality to support Keras models (and, by default, some TF models). This remains relatively experimental -> the space of models and data that "work" is likely much larger than what we have explicitly developed for. We have tried to implement warnings where possible. Some growing pains are likely.

Extra-mile Checklist

• [ ] I have thought expansively about edge cases and written tests for them

Describe your changes

Modified DataFairness and Privacy evaluators to create results as several Metric containers (one for each row in pre-results DataFrame). This allows passing subtype information as an additional label to the Container constructor (and subsequently the Evidence constructor).

Opening the PR now, as this fixes the two evaluators where this is known to be an issue. ~Going to look at other evaluators to see whether or not this issue is restricted to DataFairness + Privacy.~

UPDATE: This or a similar issue also applies to ~IdentityVerification, and RankingFairness.~ It is possible (have not confirmed, but I think it's unlikely) that it also applies to ~Equity and~ SurvivalFairness.

Security has a subtype thing going on, but the metrics are uniquely identified by the metric_type field...there are only 2 metrics. Equity is fine. IdentityVerification is confirmed to be fine. All metrics/tables created by the evaluator lead to unique labels without further mangling. RankingFairness seems fine for now.

Final update: no existing test cases for Survival. I'm not going to test this right now to see what the label outputs look like Other evaluators named above have been run & checked to determine what they're labeling scheme looks like.

Issue ticket number and link

Fixes https://github.com/credo-ai/credoai_lens/issues/265

Known outstanding issues that are not fully accounted for

See above comment about other evaluators. Reviewer can refuse to approve until confirmation that no other evaluators are subject to this issue.

Checklist before requesting a review

• [x] I have performed a self-review of my code
• [ ] I have built basic tests for new functionality (particularly new evaluators) N/A
• [ ] If new libraries have been added, I have checked that readthedocs API documentation is constructed correctly N/A
• [ ] Will this be part of a major product update? If yes, please write one phrase about this update. No

Extra-mile Checklist

• [x] I have thought expansively about edge cases and written tests for them No new tests. This bug was identified as part of implementation of broader Platform+Lens+Connect integration test, which will catch this sort of bug once online.

Expected Behavior

When Lens is matching evidences to requirements, it checks the label of the Lens-generate evidence against the label specified by a policy pack. Lens' labeling should, theoretically, be unique such that a proper labeling in the policy pack will correspond to Lens outputs.

Actual Behavior

Evaluators output unique evidence, however sometimes the uniqueness is encoded in evidence metadata rather than in the label directly. This causes Lens error messages saying multiple evidences satisfy a given requirement.

For example, consider the following error message:

This corresponds to the following evidences in the outputted JSON file:

{
          "type": "metric",
          "label": {
            "metric_type": "MembershipInference",
            "evaluator": "Privacy"
          },
          "data": {
            "value": 0.5922,
            "confidence_interval": null,
            "confidence_level": null
          },
          "generated_at": "2022-11-30T17:01:02.771421",
          "metadata": {
            "subtype": "BlackBoxRuleBased",
            "evaluator": "Privacy",
            "model_name": "credit_default_classifier",
            "assessment_dataset_name": "UCI-credit-test",
            "training_dataset_name": "UCI-credit-train"
          }
        },
        {
          "type": "metric",
          "label": {
            "metric_type": "MembershipInference",
            "evaluator": "Privacy"
          },
          "data": {
            "value": 0.5674666666666667,
            "confidence_interval": null,
            "confidence_level": null
          },
          "generated_at": "2022-11-30T17:01:02.771446",
          "metadata": {
            "subtype": "BlackBox",
            "evaluator": "Privacy",
            "model_name": "credit_default_classifier",
            "assessment_dataset_name": "UCI-credit-test",
            "training_dataset_name": "UCI-credit-train"
          }
        },
        {
          "type": "metric",
          "label": {
            "metric_type": "MembershipInference",
            "evaluator": "Privacy"
          },
          "data": {
            "value": 0.5922,
            "confidence_interval": null,
            "confidence_level": null
          },
          "generated_at": "2022-11-30T17:01:02.771467",
          "metadata": {
            "subtype": "attack_score",
            "evaluator": "Privacy",
            "model_name": "credit_default_classifier",
            "assessment_dataset_name": "UCI-credit-test",
            "training_dataset_name": "UCI-credit-train"
          }
        },

The subtype field in the metadata is sufficient to uniquely identify the 3 different evidences, while the information in the label field is not. (It's not clear that it will ALWAYS be the case that looking at subtype + label is enough to identify, there might be other fields...)

This appears to be an issue for the Privacy and DataFairness evaluators since these inherently calculate several metrics by default and they're stored in results dataframes where there is a column for the subtype.

Potential solutions:

1. Add subtype to the label by pulling from the metadata object (where it currently lives) here https://github.com/credo-ai/credoai_connect/blob/develop/connect/evidence/evidence.py#L95 <- Bad solution since this is inconsistent across evaluators...unless we want the entire label to be [present label] (union) [metadata]...still probably a bad idea.
2. Pass extra labeling information to the evidence object when it's created here https://github.com/credo-ai/credoai_lens/blob/develop/credoai/evaluators/privacy.py#L169 <- better, but requires us to go check every evaluator for this sort of behavior and to remember to do this for any future evaluators
3. Enable specifying evidence requirements in terms of the entire evidence object, not just the label. This would entail something like (on PP Editor) specifying labels: [metric_type: ..., evaluator: ...], metadata: [subtype: ...] <- this might be the best long-term solution since it provides a lot of flexibility for the policy pack creator to match arbitrary outputs from Lens (as long as the outputs from Lens are unique, which they presently are). Likely much more involved to implement.

Solution 2 is probably the best for right now. Solution 3 should be on the radar as a future feature.

Change description

Description here

Type of change

• [ ] Bug fix (fixes an issue)
• [ ] New feature (adds functionality)

Related issues

Fix #1

Checklists

Development

• [ ] Lint rules pass locally
• [ ] Application changes have been tested thoroughly
• [ ] Automated tests covering modified code pass

Security

• [ ] Security impact of change has been considered
• [ ] Code follows company security practices and guidelines

Code review

• [ ] Pull request has a descriptive title and context useful to a reviewer. Screenshots or screencasts are attached as necessary
• [ ] "Ready for review" label attached and reviewers assigned
• [ ] Changes have been reviewed by at least one other contributor
• [ ] Pull request linked to task tracker where applicable

Desired Behavior

Depending on the pipeline, lens can create many results which are hard to visualize. I would like two things:

1. Some basic printout that is easy to read
2. A summary perspective that prioritizes certain metrics that anyone who used an evaluator would care about. E.g., most people probably care about top-line performance, disaggregated performance, and parity metrics. Could we create a summary function?