LIMEcraft
LIMEcraft: Handcrafted superpixel selectionand inspection for Visual eXplanations
The LIMEcraft algorithm is an explanatory method based on image perturbations. Its prototype was the LIME algorithm - but not without significant drawbacks, especially in the context of difficult cases and medical imaging. The superiority of the LIMEcraft algorithm is primarily the ability to freely select the areas that we want to analyze, as well as the aforementioned perturbations, thanks to which we can compare with the original image, and thus better understand which features have a significant impact on the prediction of the model.
About the User Interface
The interface is displayed in the browser, where the user can upload the image and the superpixel mask from their own computer. They can also manually mark interesting areas in the image with the help of a computer mouse. Then, they choose the number of superpixels the areas selected by them are divided into. The same procedure is also applied to the areas uploaded in the mask and, independently, to the areas outside. Then, they just need to confirm their actions and wait for the result of the LIMEcraft algorithm. The user receives the model prediction results expressed as the percentage for both the class originally predicted and the class after image editing.
Functionalities:
- upload image
- upload mask
- manually select mask
- change number of superpixels inside and outside the mask
- show how the prediction changed
- change color - RGB
- change shape - power expansion
- rotate - degrees
- shift - left/right and down/up
- remove object by shifting it
- generate report
How to run the code
git clone https://github.com/MI2DataLab/LIMEcraft.git
cd LIMEcraft
virtualenv venv
source venv/bin/activate
pip install -r requirements.txt
In case of problems with library versions, try to install the latest
git submodule update --init
if previous instruction does not work, use:
git submodule add --force https://github.com/marcotcr/lime.git code/lime_library
type jupyter notebook in the console
go to code/dashboard_LIMEcraft.ipynb
run the whole notebook
type http://127.0.0.1:8001/ in the web browser
How to test own model?
go to web browser and download full_skin_cancer_model.h5 from https://www.kaggle.com/kmader/deep-learning-skin-lesion-classification/data
put the model in the folder code
change selected model in code/dashboard_LIMEcraft.ipynb in section "Choose model"
Reference
Paper for this work is avaliable at: https://arxiv.org/abs/2111.08094
If you find our work useful, please cite our paper:
@misc{Hryniewska2021LIMEcraft,
title={{LIMEcraft: Handcrafted superpixel selection and inspection for Visual eXplanations}},
author={Weronika Hryniewska and Adrianna Grudzień and Przemysław Biecek},
year={2021},
eprint={2111.08094},
archivePrefix={arXiv},
primaryClass={cs.CV}
keywords = {Explainable AI, superpixels, LIME, image features, interactive User Interface},
howpublished = {\url{https://arxiv.org/abs/2111.08094}},
}
293 Dec 20, 2022
![[CVPR 2021] Involution: Inverting the Inherence of Convolution for Visual Recognition, a brand new neural operator](https://github.com/d-li14/involution/raw/main/fig/involution.png)
1.3k Dec 28, 2022
189 Nov 22, 2022
![git git《Transformer Meets Tracker: Exploiting Temporal Context for Robust Visual Tracking》(CVPR 2021) GitHub:git2] 《Masksembles for Uncertainty Estimation》(CVPR 2021) GitHub:git3]](https://github.com/594422814/TransformerTrack/raw/main/TransformerTracker.png){kind=tracking}
236 Dec 22, 2022
484 Dec 29, 2022
49 Jan 2, 2023
100 Dec 1, 2022
628 Dec 28, 2022
105 Nov 7, 2022
4.2k Dec 29, 2022
169 Dec 02, 2022
6 Oct 10, 2022
85 Dec 22, 2022
441 Jan 05, 2023
75 Nov 17, 2022
10 Sep 12, 2022
3 Jan 10, 2022
6 Nov 15, 2022
545 Dec 28, 2022
77 Dec 03, 2022
62 Dec 14, 2022
26 Dec 19, 2022
684 Nov 02, 2022
91 Dec 26, 2022
94 Jan 06, 2023
12 May 29, 2021
1.6k Jan 05, 2023
410 Jan 03, 2023
208 Nov 25, 2022