Easy and comprehensive assessment of predictive power, with support for neuroimaging features

Related tags

Deep Learningmachine-learningscikit-learncross-validationreporteasy-to-useneuroimagingpattern-recognitionnilearntractographystructural-imaginganatomical-mrifunctional-connectivitytract-based-statistics
Overview

docs/logo_neuropredict.png

https://landscape.io/github/raamana/neuropredict/master/landscape.svg?style=flat https://api.codacy.com/project/badge/Grade/501e560b8a424562a1b8f7cd2f3cadfe https://travis-ci.org/raamana/neuropredict.svg?branch=master

Documentation: https://raamana.github.io/neuropredict/

News

  • As of v0.6, neuropredict now supports regression applications i.e. predicting continuous targets (in addition to categorical classes), as well as allow you to regress out covariates / confounds within the nested-CV (following all the best practices). Utilizing this feature requires the input datasets be specified in the pyradigm data structures: code @ https://github.com/raamana/pyradigm, docs @ https://raamana.github.io/pyradigm/. Check the changelog below for more details.

Older news

  • neuropredict can handle missing data now (that are encoded with numpy.NaN). This is done respecting the cross-validation splits without any data leakage.

Overview

On a high level,

roleofneuropredict

On a more detailed level,

roleofneuropredict

Long term goals

neuropredict, the tool, is part of a broader initiative described below to develop easy, comprehensive and standardized predictive analysis:

longtermgoals

Citation

If neuropredict helped you in your research in one way or another, please consider citing one or more of the following, which were essential building blocks of neuropredict: - Pradeep Reddy Raamana. (2017, November 18). neuropredict: easy machine learning and standardized predictive analysis of biomarkers (Version 0.4.5). Zenodo. http://doi.org/10.5281/zenodo.1058993 - Raamana et al, (2017), Python class defining a machine learning dataset ensuring key-based correspondence and maintaining integrity, Journal of Open Source Software, 2(17), 382, doi:10.21105/joss.00382

Change Log - version 0.6

  • Major feature: Ability to predict continuous variables (regression)
  • Major feature: Ability to handle confounds (regress them out, augmenting etc)
  • Redesigned the internal structure for easier extensibility
  • New CVResults class for easier management of a wealth of outputs generated in the Classification and Regression workflows
  • API access is refreshed and easier

Change Log - version 0.5.2

  • Imputation of missing values
  • Additional classifiers such as XGBoost, Decision Trees
  • Better internal code structure
  • Lot more tests
  • More precise tests, as we vary number of classes wildly in test suites
  • several bug fixes and enhancements
  • More cmd line options such as --print_options from a previous run
Comments
  • ImportError: cannot import name 'MultiDatasetClassify'

    ImportError: cannot import name 'MultiDatasetClassify'

    Hi,

    When I try to run neuropredict on the command line using neuropredict -m ~/NeuroPredict/meta_data.csv -o ~/NeuroPredict/results/ --user_feature_paths ~/NeuroPredict/face/ ~/NeuroPredict/cnlg/ ~/NeuroPredict/glmn/ ~/NeuroPredict/cnsr/ ~/NeuroPredict/bimndir/ ~/NeuroPredict/bimnemo/ ~/NeuroPredict/msep/ -t 0.8 -n 20 -g 'light' I get the error that ImportError: cannot import name 'MultiDatasetClassify' from 'pyradigm.multiple' It appears that the multiple.py file that I've downloaded does not match the current version that is in your pyradigm repository. I installed it just now though, using pip3 install neuropredict. Do you know why I am getting this problem?

    Thanks

    opened by cpappas18 13
  • Add new classifier: need of probability output?

    Add new classifier: need of probability output?

    Hi @raamana ,

    I would like to add LinearSVC classifier based on liblinear implementation. Does the current implementation of neuropredict need that predictions are based on probability values? Because, LinearSVC doesn't allow prediction of probabilities.

    opened by mattvan83 12
  • Implement option to let user select number of features to select

    Implement option to let user select number of features to select

    To do:

    • [x] add an argument: --num_features
    • [x] describe the argument thoroughly in help text
    • [x] checks on the input types
    • [x] checks on the input ranges
    • [ ] checks on whether all the feature sets have the same number of features
    • [ ] make the new argument interface with all the other methods, if need be.
    • [ ] add option to pass on subset of feature names to visualize.feature_importance_map

    We can add/remove as we realize them.

    enhancement 
    opened by raamana 12
  • How to add multiple features?

    How to add multiple features?

    Hi Pradeep,

    I can see the file structure for adding a single feature is:

    subject1/features.txt subject2/features.txt subject3/features.txt subject4/features.txt

    Where the feature is a vector in each case. To clarify, if I wanted to add a second feature, would this be a second column in the features.txt file of each participant?

    Thanks,

    John

    question 
    opened by johnaeanderson 10
  • Classification blocked with multiple 1-dimensional features

    Classification blocked with multiple 1-dimensional features

    Hello,

    I have problems when using multiple 1-D features with the following command:

          neuropredict -m meta_data.csv \
      -d mask_WMpet_av45_early.ero1.25mm.csv mask_WMpet_av45_early.ero1.5mm.csv \
      -o outdir -t 0.8 -n 250 -k 'all' \
     --gs_level 'exhaustive' --classifier "LinearSVC"

    The CV trial were well launched for the first subgroup CN,MCI for both 1-D features but it seemed that at the end of CV trials the process got stuck at this stage:

    >

>  Python 3.7.3
> > SGE recognized, job set up with 40 slots.
> > Positive class inferred for AUC calculation: CN
> > Running neuropredict 0.5+34.g220af55.dirty
> > 
> > Requested features for analysis:
> > get_data_matrix from /netapp/vol2_agewell/pro/IMAP/imap_mvh/CAT12/pet/Analyses/ML/All/CN_vs_MCI+AD/av45_early/pons/start0_dur4/mask_WMpet_av45_early.ero1.25mm.csv
> > get_data_matrix from /netapp/vol2_agewell/pro/IMAP/imap_mvh/CAT12/pet/Analyses/ML/All/CN_vs_MCI+AD/av45_early/pons/start0_dur4/mask_WMpet_av45_early.ero1.5mm.csv
> > Ignoring imputation strategy chosen, as no missing data were found!
> > 
> > Data import is done.
> > 
> > 
> > Requested processing for the following subgroups:
> > DIS,CN
> > 
> > --------------------------------------------------------------------------------
> > Processing subgroup : DIS,CN (1/1)
> > --------------------------------------------------------------------------------
> > SGE recognized, job set up with 40 slots.
> > Training percentage      : 0.8
> > Number of CV repetitions : 250
> > Classifier chosen        : linearsvc
> > Feature selection chosen : variancethreshold
> > Level of grid search     : exhaustive
> > Number of processors     : 40
> > Saving the results to 
> >   /netapp/vol2_agewell/pro/IMAP/imap_mvh/CAT12/pet/Analyses/ML/All/CN_vs_MCI+AD/av45_early/pons/start0_dur4/linearsvc/binary_WMmasks/CN_DIS
> > 
> > -------------------------
> > All datasets contain:
> >  
> > 101 samples, 2 classes, 1 features
> > Class  CN : 71 samples
> > Class DIS : 30 samples
> > -------------------------
> > 
> > Estimated chance accuracy : 0.500
> > 
> > Different classes in the training set are stratified to match the smallest class!
> > Parallelizing the repetitions of CV with 40 processes ...
> > CV trial 52     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6525 	 weighted AUC: 0.7482
> > CV trial 78     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6631 	 weighted AUC: 0.8227
> > CV trial 50     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.5993 	 weighted AUC: 0.6596
> > CV trial 72     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6082 	 weighted AUC: 0.6596
> > CV trial 46     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7252 	 weighted AUC: 0.7021
> > CV trial 58     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8085 	 weighted AUC: 0.8475
> > CV trial 38     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6933 	 weighted AUC: 0.7518
> > CV trial 16     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7358 	 weighted AUC: 0.7801
> > CV trial 12     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7660 	 weighted AUC: 0.8794
> > CV trial 0      feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6738 	 weighted AUC: 0.6950
> > CV trial 68     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7057 	 weighted AUC: 0.8369
> > CV trial 14     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7979 	 weighted AUC: 0.8794
> > CV trial 22     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7677 	 weighted AUC: 0.8333
> > CV trial 6      feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6631 	 weighted AUC: 0.7234
> > CV trial 48     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7571 	 weighted AUC: 0.8298
> > CV trial 10     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6720 	 weighted AUC: 0.7624
> > CV trial 64     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8085 	 weighted AUC: 0.8191
> > CV trial 20     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7677 	 weighted AUC: 0.8688
> > CV trial 4      feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8191 	 weighted AUC: 0.8688
> > CV trial 56     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7553 	 weighted AUC: 0.8262
> > CV trial 70     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8191 	 weighted AUC: 0.8440
> > CV trial 74     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7145 	 weighted AUC: 0.7553
> > CV trial 36     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7465 	 weighted AUC: 0.8050
> > CV trial 2      feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7872 	 weighted AUC: 0.9043
> > CV trial 54     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8191 	 weighted AUC: 0.8830
> > CV trial 24     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7447 	 weighted AUC: 0.9043
> > CV trial 76     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7340 	 weighted AUC: 0.9220
> > CV trial 42     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7553 	 weighted AUC: 0.8759
> > CV trial 26     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7872 	 weighted AUC: 0.9078
> > CV trial 62     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7571 	 weighted AUC: 0.8103
> > CV trial 66     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6418 	 weighted AUC: 0.7624
> > CV trial 60     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8511 	 weighted AUC: 0.8972
> > CV trial 72     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6294 	 weighted AUC: 0.6418
> > CV trial 40     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7660 	 weighted AUC: 0.9149
> > CV trial 52     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6720 	 weighted AUC: 0.7340
> > CV trial 16     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7252 	 weighted AUC: 0.7411
> > CV trial 46     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6720 	 weighted AUC: 0.6489
> > CV trial 38     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6507 	 weighted AUC: 0.7376
> > CV trial 78     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7358 	 weighted AUC: 0.8014
> > CV trial 50     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6099 	 weighted AUC: 0.6454
> > CV trial 58     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.8191 	 weighted AUC: 0.8298
> > CV trial 6      feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7145 	 weighted AUC: 0.6879
> > CV trial 14     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7766 	 weighted AUC: 0.8688
> > CV trial 36     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6844 	 weighted AUC: 0.8156
> > CV trial 18     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7465 	 weighted AUC: 0.8014
> > CV trial 0      feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.5904 	 weighted AUC: 0.6454
> > CV trial 20     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7571 	 weighted AUC: 0.8582
> > CV trial 68     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7677 	 weighted AUC: 0.7979
> > CV trial 4      feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.8511 	 weighted AUC: 0.8440
> > CV trial 74     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7660 	 weighted AUC: 0.7358
> > CV trial 56     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7447 	 weighted AUC: 0.8050
> > CV trial 34     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.5816 	 weighted AUC: 0.7518
> > CV trial 12     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7447 	 weighted AUC: 0.8723
> > CV trial 66     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7252 	 weighted AUC: 0.7376
> > CV trial 70     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6844 	 weighted AUC: 0.8227
> > CV trial 64     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7358 	 weighted AUC: 0.8121
> > CV trial 24     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7872 	 weighted AUC: 0.8794
> > CV trial 76     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7234 	 weighted AUC: 0.9007
> > CV trial 79     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.5904 	 weighted AUC: 0.7021
> > CV trial 53     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6720 	 weighted AUC: 0.7766
> > CV trial 2      feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7766 	 weighted AUC: 0.8936
> > CV trial 73     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7766 	 weighted AUC: 0.9255
> > CV trial 54     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7571 	 weighted AUC: 0.8759
> > CV trial 62     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7571 	 weighted AUC: 0.8014
> > CV trial 17     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7358 	 weighted AUC: 0.7624
> > CV trial 22     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7677 	 weighted AUC: 0.8191
> > CV trial 39     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6631 	 weighted AUC: 0.7695
> > CV trial 60     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.8191 	 weighted AUC: 0.9007
> > CV trial 51     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8404 	 weighted AUC: 0.8227
> > CV trial 26     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7872 	 weighted AUC: 0.9007
> > CV trial 42     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7979 	 weighted AUC: 0.8546
> > CV trial 48     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7571 	 weighted AUC: 0.8121
> > CV trial 59     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7766 	 weighted AUC: 0.8014
> > CV trial 69     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7145 	 weighted AUC: 0.7695
> > CV trial 47     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7039 	 weighted AUC: 0.7695
> > CV trial 21     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7553 	 weighted AUC: 0.9220
> > CV trial 28     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8316 	 weighted AUC: 0.9007
> > CV trial 37     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7660 	 weighted AUC: 0.8475
> > CV trial 57     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7660 	 weighted AUC: 0.8582
> > CV trial 1      feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7979 	 weighted AUC: 0.8440
> > CV trial 40     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7447 	 weighted AUC: 0.9149
> > CV trial 30     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8085 	 weighted AUC: 0.8582
> > CV trial 18     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7358 	 weighted AUC: 0.7589
> > CV trial 65     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7145 	 weighted AUC: 0.7837
> > CV trial 75     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7872 	 weighted AUC: 0.8617
> > CV trial 25     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6720 	 weighted AUC: 0.7943
> > CV trial 13     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7766 	 weighted AUC: 0.8121
> > CV trial 79     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6525 	 weighted AUC: 0.6915
> > CV trial 61     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7553 	 weighted AUC: 0.7766
> > CV trial 71     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7979 	 weighted AUC: 0.8156
> > CV trial 32     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8316 	 weighted AUC: 0.9291
> > CV trial 3      feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7766 	 weighted AUC: 0.8440
> > CV trial 53     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7039 	 weighted AUC: 0.7411
> > CV trial 55     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7766 	 weighted AUC: 0.8298
> > CV trial 15     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6933 	 weighted AUC: 0.8582
> > CV trial 23     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7145 	 weighted AUC: 0.7872
> > CV trial 34     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.5284 	 weighted AUC: 0.7376
> > CV trial 59     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7447 	 weighted AUC: 0.7713
> > CV trial 43     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7465 	 weighted AUC: 0.7766
> > CV trial 77     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7252 	 weighted AUC: 0.7305
> > CV trial 49     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7039 	 weighted AUC: 0.8528
> > CV trial 27     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7766 	 weighted AUC: 0.8191
> > CV trial 73     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7340 	 weighted AUC: 0.8936
> > CV trial 39     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.5798 	 weighted AUC: 0.7305
> > CV trial 47     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7145 	 weighted AUC: 0.7234
> > CV trial 69     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7145 	 weighted AUC: 0.7199
> > CV trial 21     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7021 	 weighted AUC: 0.9149
> > CV trial 44     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7677 	 weighted AUC: 0.8369
> > CV trial 65     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7447 	 weighted AUC: 0.7943
> > CV trial 41     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6826 	 weighted AUC: 0.7092
> > CV trial 80     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6011 	 weighted AUC: 0.7305
> > CV trial 51     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6738 	 weighted AUC: 0.8121
> > CV trial 57     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7128 	 weighted AUC: 0.8369
> > CV trial 75     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7979 	 weighted AUC: 0.8156
> > CV trial 1      feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7872 	 weighted AUC: 0.8191
> > CV trial 63     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7340 	 weighted AUC: 0.8582
> > CV trial 82     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7872 	 weighted AUC: 0.8475
> > CV trial 3      feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7766 	 weighted AUC: 0.8050
> > CV trial 23     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7039 	 weighted AUC: 0.7270
> > CV trial 61     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6933 	 weighted AUC: 0.7553
> > CV trial 55     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7340 	 weighted AUC: 0.8014
> > CV trial 43     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6418 	 weighted AUC: 0.7270
> > CV trial 77     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7145 	 weighted AUC: 0.7234
> > CV trial 25     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6613 	 weighted AUC: 0.7872
> > CV trial 84     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7766 	 weighted AUC: 0.9362
> > CV trial 30     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7358 	 weighted AUC: 0.8333
> > CV trial 19     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8298 	 weighted AUC: 0.8298
> > CV trial 37     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7872 	 weighted AUC: 0.8298
> > CV trial 88     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7872 	 weighted AUC: 0.8121
> > CV trial 96     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6507 	 weighted AUC: 0.7270
> > CV trial 86     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7979 	 weighted AUC: 0.8759
> > CV trial 13     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7766 	 weighted AUC: 0.7837
> > CV trial 41     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6206 	 weighted AUC: 0.6809
> > CV trial 5      feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8617 	 weighted AUC: 0.9043
> > CV trial 102    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6933 	 weighted AUC: 0.8333
> > CV trial 80     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6011 	 weighted AUC: 0.7163
> > CV trial 92     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7872 	 weighted AUC: 0.9220
> > CV trial 15     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7252 	 weighted AUC: 0.8227
> > CV trial 104    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6436 	 weighted AUC: 0.7660
> > CV trial 35     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7979 	 weighted AUC: 0.8333
> > CV trial 106    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.5585 	 weighted AUC: 0.7163
> > CV trial 108    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6418 	 weighted AUC: 0.7163
> > CV trial 110    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7145 	 weighted AUC: 0.6844
> > CV trial 94     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8617 	 weighted AUC: 0.9255
> > CV trial 98     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7447 	 weighted AUC: 0.8652
> > CV trial 82     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7766 	 weighted AUC: 0.8546
> > CV trial 100    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7890 	 weighted AUC: 0.8652
> > CV trial 49     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7872 	 weighted AUC: 0.8440
> > CV trial 63     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7234 	 weighted AUC: 0.8475
> > CV trial 112    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7766 	 weighted AUC: 0.8475
> > CV trial 90     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8511 	 weighted AUC: 0.8511
> > CV trial 118    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6525 	 weighted AUC: 0.7411
> > CV trial 27     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7979 	 weighted AUC: 0.7943
> > CV trial 44     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7571 	 weighted AUC: 0.8014
> > CV trial 124    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7465 	 weighted AUC: 0.8085
> > CV trial 84     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7553 	 weighted AUC: 0.9255
> > CV trial 110    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.4645 	 weighted AUC: 0.6454
> > CV trial 31     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7465 	 weighted AUC: 0.9149
> > CV trial 96     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6401 	 weighted AUC: 0.7021
> > CV trial 108    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7039 	 weighted AUC: 0.6738
> > CV trial 126    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7979 	 weighted AUC: 0.8582
> > CV trial 81     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6631 	 weighted AUC: 0.7589
> > CV trial 116    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8085 	 weighted AUC: 0.9184
> > CV trial 104    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7145 	 weighted AUC: 0.7163
> > CV trial 106    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.5585 	 weighted AUC: 0.7163
> > CV trial 67     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7996 	 weighted AUC: 0.9007
> > CV trial 92     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7766 	 weighted AUC: 0.9043
> > CV trial 88     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7145 	 weighted AUC: 0.7695
> > CV trial 120    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7553 	 weighted AUC: 0.7695
> > CV trial 122    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7358 	 weighted AUC: 0.8511
> > CV trial 130    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6933 	 weighted AUC: 0.7376
> > CV trial 19     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.8298 	 weighted AUC: 0.8085
> > CV trial 102    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7252 	 weighted AUC: 0.8262
> > CV trial 98     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7872 	 weighted AUC: 0.8688
> > CV trial 112    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7234 	 weighted AUC: 0.8298
> > CV trial 114    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7571 	 weighted AUC: 0.8652
> > CV trial 86     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7872 	 weighted AUC: 0.8475
> > CV trial 35     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7872 	 weighted AUC: 0.8121
> > CV trial 100    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7057 	 weighted AUC: 0.8475
> > CV trial 71     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7252 	 weighted AUC: 0.7996
> > CV trial 118    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6525 	 weighted AUC: 0.6986
> > CV trial 94     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7784 	 weighted AUC: 0.8901
> > CV trial 83     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6738 	 weighted AUC: 0.8475
> > CV trial 128    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7872 	 weighted AUC: 0.8369
> > CV trial 85     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6826 	 weighted AUC: 0.7695
> > CV trial 17     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6631 	 weighted AUC: 0.7447
> > CV trial 90     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.8404 	 weighted AUC: 0.8511
> > CV trial 45     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8617 	 weighted AUC: 0.8191
> > CV trial 111    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6188 	 weighted AUC: 0.8333
> > CV trial 124    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6933 	 weighted AUC: 0.7713
> > CV trial 97     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7571 	 weighted AUC: 0.8582
> > CV trial 109    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7979 	 weighted AUC: 0.8652
> > CV trial 105    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7039 	 weighted AUC: 0.8369
> > CV trial 120    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7660 	 weighted AUC: 0.7340
> > CV trial 113    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7447 	 weighted AUC: 0.6702
> > CV trial 87     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7660 	 weighted AUC: 0.9362
> > CV trial 81     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6011 	 weighted AUC: 0.7305
> > CV trial 136    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.5904 	 weighted AUC: 0.8050
> > CV trial 132    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6738 	 weighted AUC: 0.7589
> > CV trial 116    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7766 	 weighted AUC: 0.8901
> > CV trial 99     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8298 	 weighted AUC: 0.8972
> > CV trial 119    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7021 	 weighted AUC: 0.8759
> > CV trial 103    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8191 	 weighted AUC: 0.9113
> > CV trial 95     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7145 	 weighted AUC: 0.6879
> > CV trial 134    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6720 	 weighted AUC: 0.8014
> > CV trial 130    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7660 	 weighted AUC: 0.6986
> > CV trial 93     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7677 	 weighted AUC: 0.8121
> > CV trial 125    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6933 	 weighted AUC: 0.7482
> > CV trial 101    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7784 	 weighted AUC: 0.8652
> > CV trial 91     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7021 	 weighted AUC: 0.9255
> > CV trial 85     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7039 	 weighted AUC: 0.7482
> > CV trial 128    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7872 	 weighted AUC: 0.8262
> > CV trial 107    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6826 	 weighted AUC: 0.7801
> > CV trial 111    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6188 	 weighted AUC: 0.7979
> > CV trial 5      feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7784 	 weighted AUC: 0.8865
> > CV trial 31     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7766 	 weighted AUC: 0.9220
> > CV trial 138    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7784 	 weighted AUC: 0.8901
> > CV trial 114    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7784 	 weighted AUC: 0.8333
> > CV trial 121    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7979 	 weighted AUC: 0.8546
> > CV trial 113    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6933 	 weighted AUC: 0.6312
> > CV trial 97     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7252 	 weighted AUC: 0.8546
> > CV trial 28     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.8528 	 weighted AUC: 0.8617
> > CV trial 89     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6418 	 weighted AUC: 0.6702
> > CV trial 109    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7872 	 weighted AUC: 0.8688
> > CV trial 122    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7660 	 weighted AUC: 0.8422
> > CV trial 87     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7872 	 weighted AUC: 0.9149
> > CV trial 105    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6933 	 weighted AUC: 0.8191
> > CV trial 83     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7571 	 weighted AUC: 0.8475
> > CV trial 95     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6826 	 weighted AUC: 0.6879
> > CV trial 140    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7979 	 weighted AUC: 0.8546
> > CV trial 7      feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8191 	 weighted AUC: 0.8830
> > CV trial 125    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7039 	 weighted AUC: 0.6844
> > CV trial 142    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8085 	 weighted AUC: 0.8723
> > CV trial 136    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7358 	 weighted AUC: 0.7908
> > CV trial 103    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7252 	 weighted AUC: 0.8759
> > CV trial 119    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7021 	 weighted AUC: 0.8262
> > CV trial 99     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.8085 	 weighted AUC: 0.8759
> > CV trial 132    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6436 	 weighted AUC: 0.7234
> > CV trial 129    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7145 	 weighted AUC: 0.7021
> > CV trial 91     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6915 	 weighted AUC: 0.9184
> > CV trial 126    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7979 	 weighted AUC: 0.8493
> > CV trial 150    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7660 	 weighted AUC: 0.7376
> > CV trial 93     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6011 	 weighted AUC: 0.8014
> > CV trial 144    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7872 	 weighted AUC: 0.7872
> > CV trial 148    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7465 	 weighted AUC: 0.7323
> > CV trial 146    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7872 	 weighted AUC: 0.7624
> > CV trial 107    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6294 	 weighted AUC: 0.7411
> > CV trial 152    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7447 	 weighted AUC: 0.8723
> > CV trial 101    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6950 	 weighted AUC: 0.8617
> > CV trial 115    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7234 	 weighted AUC: 0.9007
> > CV trial 154    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7979 	 weighted AUC: 0.8759
> > CV trial 134    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6401 	 weighted AUC: 0.8014
> > CV trial 29     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7979 	 weighted AUC: 0.8582
> > CV trial 158    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7128 	 weighted AUC: 0.8121
> > CV trial 166    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7340 	 weighted AUC: 0.7872
> > CV trial 137    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7145 	 weighted AUC: 0.7234
> > CV trial 168    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7252 	 weighted AUC: 0.7057
> > CV trial 160    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8191 	 weighted AUC: 0.8440
> > CV trial 123    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7872 	 weighted AUC: 0.8121
> > CV trial 117    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7465 	 weighted AUC: 0.8652
> > CV trial 164    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7677 	 weighted AUC: 0.8369
> > CV trial 45     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6950 	 weighted AUC: 0.7979
> > CV trial 170    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6011 	 weighted AUC: 0.7766
> > CV trial 150    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7447 	 weighted AUC: 0.7057
> > CV trial 7      feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7447 	 weighted AUC: 0.8723
> > CV trial 121    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7979 	 weighted AUC: 0.8050
> > CV trial 174    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7784 	 weighted AUC: 0.8298
> > CV trial 176    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7677 	 weighted AUC: 0.8369
> > CV trial 162    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6826 	 weighted AUC: 0.7660
> > CV trial 156    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7465 	 weighted AUC: 0.7908
> > CV trial 127    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6933 	 weighted AUC: 0.8014
> > CV trial 8      feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8723 	 weighted AUC: 0.9326
> > CV trial 172    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6117 	 weighted AUC: 0.7589
> > CV trial 144    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7553 	 weighted AUC: 0.7589
> > CV trial 148    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.5691 	 weighted AUC: 0.7021
> > CV trial 129    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.4965 	 weighted AUC: 0.6809
> > CV trial 180    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7145 	 weighted AUC: 0.8121
> > CV trial 140    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7979 	 weighted AUC: 0.8191
> > CV trial 146    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7660 	 weighted AUC: 0.7624
> > CV trial 142    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.8191 	 weighted AUC: 0.8652
> > CV trial 154    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7872 	 weighted AUC: 0.8475
> > CV trial 115    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6915 	 weighted AUC: 0.8865
> > CV trial 29     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7340 	 weighted AUC: 0.8227
> > CV trial 166    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7021 	 weighted AUC: 0.7553
> > CV trial 89     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6418 	 weighted AUC: 0.6348
> > CV trial 133    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7465 	 weighted AUC: 0.8298
> > CV trial 168    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7252 	 weighted AUC: 0.6702
> > CV trial 184    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.5780 	 weighted AUC: 0.7589
> > CV trial 137    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6418 	 weighted AUC: 0.6950
> > CV trial 158    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7447 	 weighted AUC: 0.7837
> > CV trial 152    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7234 	 weighted AUC: 0.8262
> > CV trial 151    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.5479 	 weighted AUC: 0.7305
> > CV trial 131    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8191 	 weighted AUC: 0.9255
> > CV trial 160    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.8085 	 weighted AUC: 0.8227
> > CV trial 123    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7358 	 weighted AUC: 0.7766
> > CV trial 32     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7979 	 weighted AUC: 0.0709
> > CV trial 135    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7979 	 weighted AUC: 0.7713
> > CV trial 170    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6011 	 weighted AUC: 0.7270
> > CV trial 188    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7145 	 weighted AUC: 0.6454
> > CV trial 145    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7234 	 weighted AUC: 0.7553
> > CV trial 117    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7677 	 weighted AUC: 0.8369
> > CV trial 156    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7979 	 weighted AUC: 0.7837
> > CV trial 127    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7039 	 weighted AUC: 0.7890
> > CV trial 186    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6826 	 weighted AUC: 0.7979
> > CV trial 178    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7465 	 weighted AUC: 0.7624
> > CV trial 141    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7340 	 weighted AUC: 0.8511
> > CV trial 147    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7039 	 weighted AUC: 0.8759
> > CV trial 180    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6418 	 weighted AUC: 0.7979
> > CV trial 174    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7784 	 weighted AUC: 0.8085
> > CV trial 149    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8085 	 weighted AUC: 0.8582
> > CV trial 162    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6507 	 weighted AUC: 0.7163
> > CV trial 172    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.5904 	 weighted AUC: 0.7270
> > CV trial 184    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.5674 	 weighted AUC: 0.7340
> > CV trial 151    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6826 	 weighted AUC: 0.7163
> > CV trial 176    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7677 	 weighted AUC: 0.7872
> > CV trial 194    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6525 	 weighted AUC: 0.7092
> > CV trial 188    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.4645 	 weighted AUC: 0.5993
> > CV trial 169    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7553 	 weighted AUC: 0.8688
> > CV trial 155    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7571 	 weighted AUC: 0.7872
> > CV trial 171    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6826 	 weighted AUC: 0.7766
> > CV trial 159    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6223 	 weighted AUC: 0.7482
> > CV trial 167    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7872 	 weighted AUC: 0.8511
> > CV trial 153    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7766 	 weighted AUC: 0.8652
> > CV trial 133    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7465 	 weighted AUC: 0.7766
> > CV trial 143    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7660 	 weighted AUC: 0.8617
> > CV trial 198    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7766 	 weighted AUC: 0.8333
> > CV trial 161    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7766 	 weighted AUC: 0.8475
> > CV trial 202    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7145 	 weighted AUC: 0.6950
> > CV trial 182    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7145 	 weighted AUC: 0.8333
> > CV trial 164    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6844 	 weighted AUC: 0.8191
> > CV trial 186    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6401 	 weighted AUC: 0.7411
> > CV trial 8      feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.8830 	 weighted AUC: 0.0674
> > CV trial 157    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7447 	 weighted AUC: 0.9574
> > CV trial 141    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7021 	 weighted AUC: 0.8617
> > CV trial 135    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6631 	 weighted AUC: 0.7695
> > CV trial 145    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7340 	 weighted AUC: 0.7199
> > CV trial 190    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7039 	 weighted AUC: 0.7553
> > CV trial 181    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7766 	 weighted AUC: 0.7979
> > CV trial 196    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7145 	 weighted AUC: 0.7766
> > CV trial 33     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8103 	 weighted AUC: 0.8936
> > CV trial 192    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8422 	 weighted AUC: 0.8546
> > CV trial 138    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7784 	 weighted AUC: 0.8901
> > CV trial 200    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8085 	 weighted AUC: 0.9468
> > CV trial 149    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7979 	 weighted AUC: 0.8546
> > CV trial 173    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7872 	 weighted AUC: 0.8475
> > CV trial 177    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7979 	 weighted AUC: 0.8014
> > CV trial 178    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7465 	 weighted AUC: 0.7340
> > CV trial 204    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7979 	 weighted AUC: 0.8404
> > CV trial 206    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7979 	 weighted AUC: 0.7872
> > CV trial 163    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7465 	 weighted AUC: 0.8085
> > CV trial 185    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7872 	 weighted AUC: 0.8298
> > CV trial 171    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7039 	 weighted AUC: 0.7801
> > CV trial 202    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6206 	 weighted AUC: 0.6950
> > CV trial 169    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7447 	 weighted AUC: 0.8440
> > CV trial 167    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7766 	 weighted AUC: 0.8369
> > CV trial 153    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7447 	 weighted AUC: 0.8369
> > CV trial 131    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.8298 	 weighted AUC: 0.9326
> > CV trial 159    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6950 	 weighted AUC: 0.7340
> > CV trial 194    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6418 	 weighted AUC: 0.6879
> > CV trial 187    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7571 	 weighted AUC: 0.7553
> > CV trial 155    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6738 	 weighted AUC: 0.7660
> > CV trial 212    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7145 	 weighted AUC: 0.8546
> > CV trial 198    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6826 	 weighted AUC: 0.8191
> > CV trial 189    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7979 	 weighted AUC: 0.8794
> > CV trial 165    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7872 	 weighted AUC: 0.8652
> > CV trial 157    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7340 	 weighted AUC: 0.9397
> > CV trial 208    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7979 	 weighted AUC: 0.8546
> > CV trial 139    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6720 	 weighted AUC: 0.7855
> > CV trial 190    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7039 	 weighted AUC: 0.7305
> > CV trial 214    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.5780 	 weighted AUC: 0.6879
> > CV trial 181    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7128 	 weighted AUC: 0.7660
> > CV trial 210    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7465 	 weighted AUC: 0.8121
> > CV trial 143    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7553 	 weighted AUC: 0.8404
> > CV trial 9      feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7039 	 weighted AUC: 0.8511
> > CV trial 196    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6738 	 weighted AUC: 0.7518
> > CV trial 182    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6933 	 weighted AUC: 0.8227
> > CV trial 147    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7039 	 weighted AUC: 0.8582
> > CV trial 175    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8298 	 weighted AUC: 0.9113
> > CV trial 216    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7252 	 weighted AUC: 0.7943
> > CV trial 173    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7766 	 weighted AUC: 0.8440
> > CV trial 177    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7872 	 weighted AUC: 0.7979
> > CV trial 206    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.8085 	 weighted AUC: 0.7518
> > CV trial 179    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7571 	 weighted AUC: 0.8121
> > CV trial 204    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.8085 	 weighted AUC: 0.8440
> > CV trial 218    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8298 	 weighted AUC: 0.8759
> > CV trial 185    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7660 	 weighted AUC: 0.8014
> > CV trial 226    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6312 	 weighted AUC: 0.8227
> > CV trial 224    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7872 	 weighted AUC: 0.8475
> > CV trial 220    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7872 	 weighted AUC: 0.7801
> > CV trial 195    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7145 	 weighted AUC: 0.7872
> > CV trial 200    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.8511 	 weighted AUC: 0.9291
> > CV trial 163    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7465 	 weighted AUC: 0.7908
> > CV trial 161    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7979 	 weighted AUC: 0.8156
> > CV trial 230    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6401 	 weighted AUC: 0.8227
> > CV trial 199    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7447 	 weighted AUC: 0.8050
> > CV trial 212    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6613 	 weighted AUC: 0.8262
> > CV trial 187    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7145 	 weighted AUC: 0.7447
> > CV trial 232    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6844 	 weighted AUC: 0.7837
> > CV trial 207    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.4752 	 weighted AUC: 0.6064
> > CV trial 33     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7979 	 weighted AUC: 0.0975
> > CV trial 228    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7145 	 weighted AUC: 0.8440
> > CV trial 203    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7358 	 weighted AUC: 0.7766
> > CV trial 139    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6525 	 weighted AUC: 0.7624
> > CV trial 183    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7358 	 weighted AUC: 0.8262
> > CV trial 208    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7252 	 weighted AUC: 0.8404
> > CV trial 165    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7979 	 weighted AUC: 0.8191
> > CV trial 234    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8298 	 weighted AUC: 0.9468
> > CV trial 214    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6206 	 weighted AUC: 0.6631
> > CV trial 216    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7039 	 weighted AUC: 0.7589
> > CV trial 191    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8298 	 weighted AUC: 0.8511
> > CV trial 240    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7660 	 weighted AUC: 0.8440
> > CV trial 236    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7145 	 weighted AUC: 0.8262
> > CV trial 210    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7571 	 weighted AUC: 0.7943
> > CV trial 242    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8298 	 weighted AUC: 0.8582
> > CV trial 224    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7553 	 weighted AUC: 0.7943
> > CV trial 189    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7979 	 weighted AUC: 0.8387
> > CV trial 220    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7252 	 weighted AUC: 0.7163
> > CV trial 213    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6950 	 weighted AUC: 0.8156
> > CV trial 238    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6844 	 weighted AUC: 0.8085
> > CV trial 218    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7766 	 weighted AUC: 0.8582
> > CV trial 230    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6401 	 weighted AUC: 0.7730
> > CV trial 226    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6312 	 weighted AUC: 0.8014
> > CV trial 179    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7252 	 weighted AUC: 0.8014
> > CV trial 201    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6507 	 weighted AUC: 0.8298
> > CV trial 207    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.4645 	 weighted AUC: 0.5816
> > CV trial 246    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6720 	 weighted AUC: 0.8121
> > CV trial 248    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6117 	 weighted AUC: 0.7163
> > CV trial 195    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7252 	 weighted AUC: 0.7518
> > CV trial 199    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7766 	 weighted AUC: 0.7837
> > CV trial 183    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6525 	 weighted AUC: 0.8298
> > CV trial 244    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7358 	 weighted AUC: 0.8298
> > CV trial 203    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7571 	 weighted AUC: 0.7270
> > CV trial 209    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7340 	 weighted AUC: 0.8511
> > CV trial 205    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.5691 	 weighted AUC: 0.7801
> > CV trial 217    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8085 	 weighted AUC: 0.9149
> > CV trial 240    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7766 	 weighted AUC: 0.8191
> > CV trial 221    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7660 	 weighted AUC: 0.9291
> > CV trial 225    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7358 	 weighted AUC: 0.8262
> > CV trial 232    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6436 	 weighted AUC: 0.7660
> > CV trial 211    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7252 	 weighted AUC: 0.8262
> > CV trial 234    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.8298 	 weighted AUC: 0.9326
> > CV trial 236    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7145 	 weighted AUC: 0.7943
> > CV trial 246    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6720 	 weighted AUC: 0.7730
> > CV trial 213    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.8404 	 weighted AUC: 0.8050
> > CV trial 209    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7447 	 weighted AUC: 0.8316
> > CV trial 219    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6631 	 weighted AUC: 0.7305
> > CV trial 248    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.5071 	 weighted AUC: 0.7092
> > CV trial 201    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6613 	 weighted AUC: 0.7979
> > CV trial 67     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7163 	 weighted AUC: 0.8617
> > CV trial 228    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6933 	 weighted AUC: 0.8121
> > CV trial 9      feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6933 	 weighted AUC: 0.8262
> > CV trial 217    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.8085 	 weighted AUC: 0.8830
> > CV trial 215    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7766 	 weighted AUC: 0.9255
> > CV trial 244    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.8085 	 weighted AUC: 0.8121
> > CV trial 175    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.8298 	 weighted AUC: 0.8688
> > CV trial 231    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7145 	 weighted AUC: 0.7979
> > CV trial 227    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8085 	 weighted AUC: 0.8369
> > CV trial 241    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7465 	 weighted AUC: 0.8652
> > CV trial 211    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7571 	 weighted AUC: 0.8014
> > CV trial 205    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7039 	 weighted AUC: 0.7660
> > CV trial 235    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6933 	 weighted AUC: 0.8156
> > CV trial 191    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.8298 	 weighted AUC: 0.8475
> > CV trial 249    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6826 	 weighted AUC: 0.6773
> > CV trial 233    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6206 	 weighted AUC: 0.7234
> > CV trial 225    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7358 	 weighted AUC: 0.7908
> > CV trial 247    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7021 	 weighted AUC: 0.8865
> > CV trial 237    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7872 	 weighted AUC: 0.8652
> > CV trial 242    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.8298 	 weighted AUC: 0.8333
> > CV trial 229    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6720 	 weighted AUC: 0.7872
> > CV trial 219    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.5904 	 weighted AUC: 0.7199
> > CV trial 221    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7660 	 weighted AUC: 0.8901
> > CV trial 238    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6011 	 weighted AUC: 0.7943
> > CV trial 245    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8298 	 weighted AUC: 0.8528
> > CV trial 222    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8191 	 weighted AUC: 0.8794
> > CV trial 215    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7660 	 weighted AUC: 0.9078
> > CV trial 249    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6933 	 weighted AUC: 0.6312
> > CV trial 231    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7145 	 weighted AUC: 0.7447
> > CV trial 235    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6720 	 weighted AUC: 0.7943
> > CV trial 241    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.8191 	 weighted AUC: 0.8582
> > CV trial 247    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6915 	 weighted AUC: 0.8511
> > CV trial 229    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6933 	 weighted AUC: 0.7624
> > CV trial 233    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6206 	 weighted AUC: 0.6915
> > CV trial 237    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7872 	 weighted AUC: 0.8546
> > CV trial 227    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7660 	 weighted AUC: 0.8262
> > CV trial 239    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7252 	 weighted AUC: 0.8901
> > CV trial 245    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.8404 	 weighted AUC: 0.8121
> > CV trial 243    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7996 	 weighted AUC: 0.8688
> > CV trial 222    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.8085 	 weighted AUC: 0.8723
> > CV trial 239    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7252 	 weighted AUC: 0.8901
> > CV trial 192    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.8316 	 weighted AUC: 0.8298
> > CV trial 197    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8191 	 weighted AUC: 0.9539
> > CV trial 197    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7766 	 weighted AUC: 0.9433
> > CV trial 223    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.5709 	 weighted AUC: 0.7908
> > CV trial 243    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7890 	 weighted AUC: 0.8617
> > CV trial 193    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8191 	 weighted AUC: 0.9255
> > CV trial 223    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.4982 	 weighted AUC: 0.7943
> > CV trial 193    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.8191 	 weighted AUC: 0.9397
>

    If I launch the previous command line with only one 1-D feature as follows, everything is going well:

          neuropredict -m meta_data.csv \
      -d mask_WMpet_av45_early.ero1.25mm.csv \
      -o outdir -t 0.8 -n 250 -k 'all' \
      --gs_level 'exhaustive' --classifier "LinearSVC"

    Do you have any idea?

    Best,

    Matthieu

    Sources.zip

    opened by mattvan83 6
  • IO error, unable to load features

    IO error, unable to load features

    Hi Pradeep,

    I finally got neuropredict installed with the dependencies using python 2.7.13! I'm having some issues with running the command, see the error message below,

    Cheers,

    John

    (py2713) Johns-MacBook-Pro-2:KNN John$ neuropredict -m /Users/John/Desktop/KNN/METADATAFILE.csv -o /Users/John/Desktop/KNN/features/results -u /Users/John/Desktop/KNN/features

    Requested features for analysis: get_dir_of_dirs from /Users/John/Desktop/KNN/features Traceback (most recent call last): File "/Users/John/anaconda3/envs/py2713/lib/python2.7/site-packages/neuropredict/neuropredict.py", line 333, in getfeatures data, feat_names = getmethod(featdir, subjid) File "/Users/John/anaconda3/envs/py2713/lib/python2.7/site-packages/neuropredict/neuropredict.py", line 274, in get_dir_of_dirs raise IOError('Unable to load features from \n{}'.format(featfile))

    opened by johnaeanderson 5
  • Unable to launch multi-class classifier

    Unable to launch multi-class classifier

    Hi Pradeep,

    When trying to launch multi-class classifier (whatever the classifier chosen) on a 3-class (CN, MCI, AD) problem, I got the following problem:

    Traceback (most recent call last):
  File "/homes_unix/mvanhoutte/Soft/anaconda3/envs/neuropredict/bin/neuropredict", line 8, in <module>
    sys.exit(main())
  File "/homes_unix/mvanhoutte/Soft/anaconda3/envs/neuropredict/lib/python3.7/site-packages/neuropredict/__main__.py", line 11, in main
    run_workflow.cli()
  File "/homes_unix/mvanhoutte/Soft/anaconda3/envs/neuropredict/lib/python3.7/site-packages/neuropredict/run_workflow.py", line 969, in cli
    grid_search_level, classifier, feat_select_method = parse_args()
  File "/homes_unix/mvanhoutte/Soft/anaconda3/envs/neuropredict/lib/python3.7/site-packages/neuropredict/run_workflow.py", line 525, in parse_args
    class_set, subgroups, positive_class = validate_class_set(classes, user_args.sub_groups, user_args.positive_class)
  File "/homes_unix/mvanhoutte/Soft/anaconda3/envs/neuropredict/lib/python3.7/site-packages/neuropredict/run_workflow.py", line 659, in validate_class_set
    ''.format(comb))
ValueError: Subgroup all does not contain 2 unique classes! Each subgroup must contain atleast two classes for classification experiments.

    I don't understand since I have well defined 3-class in the metadata.csv file and used the following command line:

      neuropredict -m meta_data.csv \
  -d metaROI.csv metaROI_split.csv HCP_parcellation.csv \
  -t 0.8 -n 250 -k 'all' --sub_groups 'all' --gs_level 'exhaustive' \
  --feat_select_method 'variancethreshold' --classifier ${classifier} --make_vis ./visu \
  --num_procs ${NCPU} --print_options ./options

    Could you help me? Best, Matthieu

    opened by mattvan83 4
  • Regression output visualizations: residual plots

    Regression output visualizations: residual plots

    A good output in the regression version would be residual plots

    Good example is from yellowbrick: http://www.scikit-yb.org/en/latest/api/regressor/residuals.html

    opened by raamana 2
  • CLI flag to print chosen options from an existing run

    CLI flag to print chosen options from an existing run

    normal run would print only the most important options such as which classifier, cross-validation details etc, whereas verbose option would print which subjects and more info into feature sets etc.

    opened by raamana 1
  • Add documentation to help interpret the results and visualizations

    Add documentation to help interpret the results and visualizations

    Have a separate markdown file

    • [ ] describe each output figure comprehensively
    • [ ] describe each output CSV file
    • [ ] And what else they can do with results
    enhancement help wanted 
    opened by raamana 1
  • feature request: covariates and ability to regress them

    feature request: covariates and ability to regress them

    few ideas:

    CSV input

    • [ ] have column names, specify which columns are class, id, covariates and feature anmes!
    • [ ] get the class ID from a specified colum (e.g. first, or specified)
    • [ ] get the subject ID from a specified column (or the second or specified)
    sbuject_id,class,feature1,feature2,age,sex
acbde_01,healthy,1.2,2.3,3.4,74,female
fghijk_01,healthy,1.2,2.3,3.4,76,male
...
lmnop_01,alzheimer,1.2,2.3,3.4,70,female
...
qrstu_01,depression,1.2,2.3,3.4,71,male

    Directory of directories input

    • [ ] covariates.txt in addition to features.txt within each subject folder
    enhancement Hackathon project 
    opened by raamana 1
  • Enabling plug-in user-chosen models or hyper-parameters

    Enabling plug-in user-chosen models or hyper-parameters

    Sometimes some advanced users would like to choose their model or hyper-parameters for many good reasons including model not existing in sklearn etc or needing deep control over them etc, then having the API or CLI allowing users to do it would be great!

    opened by raamana 0
  • Differences betw. frequency misclassified vs frequently correctly classified

    Differences betw. frequency misclassified vs frequently correctly classified

    besides the misclassification frequency plots (which can be a helpful diagnostic), it may be useful to offer an option to enable the user to compare and study the differences characteristics between frequency misclassified vs. frequently correctly classified..

    Characteristics could be the input features themselves, or the other associated meta data and attributes (e.g. provided as part of pyradigm)

    examples include: those frequently misclassified were mostly the older patients , and those frequently correctly classified has "property X" etc.

    opened by raamana 0
  • Add docs for all API, esp. CVResults

    Add docs for all API, esp. CVResults

    CVResults might be individually even more useful than others in the API. So document it thoroughly align with the rest with a tutorial or two for use by dev and reuse by the replicator or meta-analytic researcher.

    Hackathon project 
    opened by raamana 0
  • update all docs and graphics to indicate users can try different models also

    update all docs and graphics to indicate users can try different models also

    not just compare different feature sets with a fixed builtin model, but users can also input a model of their own choice. it does not limit exploration of new models or pipelines - they can use implementation of best practices while evaluating the such new models on features of their choice

    opened by raamana 1
  • Attribute-contrained performance estimates

    Attribute-contrained performance estimates

    Often, performance is estimated in aggregate on the entire test-set (in a split), regardless of their covariate- or attribute characteristics.. hence looking into performance estimates segregated by certain characteristics of covariates would reveal additional insights - such as difference in AUC in men vs women, or across different age bins, or between sites etc.

    Hackathon project 
    opened by raamana 2
Releases(0.6)

  • 0.6(Jun 8, 2020)

    • Major feature: Ability to predict continuous variables (regression)
    • Major feature: Ability to handle confounds (regress them out, augmenting etc)
    • Redesigned the internal structure for easier extensibility
    • New CVResults class for easier management of a wealth of outputs generated in the Classification and Regression workflows
    • API access is refreshed and easier
    Source code(tar.gz)
    Source code(zip)

  • 0.5(Jan 31, 2019)

    • Good news: neuropredict can handle missing data now (that are encoded with numpy.NaN). This is done respecting the cross-validation splits without any data leakage.
    • Few other useful utilities
    Source code(tar.gz)
    Source code(zip)

  • 0.4.5(Nov 18, 2017)

    • new classifier : SVM
    • new flag to choose a feature selection method
    • user chosen options now saved to disk, to better handle complex interactions between options
    • code clean up and faster tests
    Source code(tar.gz)
    Source code(zip)

  • 0.4.1(Nov 6, 2017)

    • Parallelizing the main the CV loop, leading to great reduction in total time for report generation!
    • More options, including choice of different classifiers (Random Forest and Extra Trees classifiers)
    • Support for dataset in Weka's ARFF format
    • Better visualizations (handling small/nan values in feature importance, layouts and design)
    • auto versioning!
    • Ability to read meta data from pyradigms or ARFF files, without having to specify that separately.
    • Dropping support for Python 2.7 :(
    Source code(tar.gz)
    Source code(zip)

  • v0.3-DOI(Sep 26, 2017)

Owner
Pradeep Reddy Raamana
Neuroscientist trying to bridge the gap between clinic & computer science. Interests: Machine learning, Neuroimaging, Brain disorders, Informatics, Open science
Pradeep Reddy Raamana
GitHub Repository https://raamana.github.io/neuropredict/
Genshin-assets - 👧 Public documentation & static assets for Genshin Impact data.

genshin-assets This repo provides easy access to the Genshin Impact assets, primarily for use on static sites. Sources Genshin Optimizer - An Artifact

Zerite Development 5 Nov 22, 2022
Learning to Reach Goals via Iterated Supervised Learning

Vanilla GCSL This repository contains a vanilla implementation of "Learning to Reach Goals via Iterated Supervised Learning" proposed by Dibya Gosh et

Christoph Heindl 4 Aug 10, 2022
Imaginaire - NVIDIA's Deep Imagination Team's PyTorch Library

Imaginaire Docs | License | Installation | Model Zoo Imaginaire is a pytorch library that contains optimized implementation of several image and video

NVIDIA Research Projects 3.6k Dec 29, 2022
Code for "PV-RAFT: Point-Voxel Correlation Fields for Scene Flow Estimation of Point Clouds", CVPR 2021

PV-RAFT This repository contains the PyTorch implementation for paper "PV-RAFT: Point-Voxel Correlation Fields for Scene Flow Estimation of Point Clou

Yi Wei 43 Dec 05, 2022
CCPD: a diverse and well-annotated dataset for license plate detection and recognition

CCPD (Chinese City Parking Dataset, ECCV) UPdate on 10/03/2019. CCPD Dataset is now updated. We are confident that images in subsets of CCPD is much m

detectRecog 1.8k Dec 30, 2022
GNN-based Recommendation Benchmark

GRecX A Fair Benchmark for GNN-based Recommendation Homepage and Documentation Homepage: Documentation: Paper: GRecX: An Efficient and Unified Benchma

73 Oct 17, 2022
Converts geometry node attributes to built-in attributes

Attribute Converter Simplifies converting attributes created by geometry nodes to built-in attributes like UVs or vertex colors, as a single click ope

Ivan Notaros 12 Dec 22, 2022
Make differentially private training of transformers easy for everyone

private-transformers This codebase facilitates fast experimentation of differentially private training of Hugging Face transformers. What is this? Why

Xuechen Li 73 Dec 28, 2022
Keepsake is a Python library that uploads files and metadata (like hyperparameters) to Amazon S3 or Google Cloud Storage

Keepsake Version control for machine learning. Keepsake is a Python library that uploads files and metadata (like hyperparameters) to Amazon S3 or Goo

Replicate 1.6k Dec 29, 2022
RIFE - Real-Time Intermediate Flow Estimation for Video Frame Interpolation

RIFE - Real-Time Intermediate Flow Estimation for Video Frame Interpolation YouTube | BiliBili 16X interpolation results from two input images: Introd

旷视天元 MegEngine 28 Dec 09, 2022
The repo of the preprinting paper "Labels Are Not Perfect: Inferring Spatial Uncertainty in Object Detection"

Inferring Spatial Uncertainty in Object Detection A teaser version of the code for the paper Labels Are Not Perfect: Inferring Spatial Uncertainty in

ZINING WANG 21 Mar 03, 2022
External Attention Network

Beyond Self-attention: External Attention using Two Linear Layers for Visual Tasks paper : https://arxiv.org/abs/2105.02358 EAMLP will come soon Jitto

MenghaoGuo 357 Dec 11, 2022
Semi-Supervised Graph Prototypical Networks for Hyperspectral Image Classification, IGARSS, 2021.

Semi-Supervised Graph Prototypical Networks for Hyperspectral Image Classification, IGARSS, 2021. Bobo Xi, Jiaojiao Li, Yunsong Li and Qian Du. Code f

Bobo Xi 7 Nov 03, 2022
A repository for the updated version of CoinRun used to collect MUGEN, a multimodal video-audio-text dataset.

A repository for the updated version of CoinRun used to collect MUGEN, a multimodal video-audio-text dataset. This repo contains scripts to train RL agents to navigate the closed world and collect vi

MUGEN 11 Oct 22, 2022
PyTorch code for JEREX: Joint Entity-Level Relation Extractor

JEREX: "Joint Entity-Level Relation Extractor" PyTorch code for JEREX: "Joint Entity-Level Relation Extractor". For a description of the model and exp

LAVIS - NLP Working Group 50 Dec 01, 2022
Traditional deepdream with VQGAN+CLIP and optical flow. Ready to use in Google Colab

VQGAN-CLIP-Video cat.mp4 policeman.mp4 schoolboy.mp4 forsenBOG.mp4

23 Oct 26, 2022
Interactive Visualization to empower domain experts to align ML model behaviors with their knowledge.

An interactive visualization system designed to helps domain experts responsibly edit Generalized Additive Models (GAMs). For more information, check

InterpretML 83 Jan 04, 2023
TSIT: A Simple and Versatile Framework for Image-to-Image Translation

TSIT: A Simple and Versatile Framework for Image-to-Image Translation This repository provides the official PyTorch implementation for the following p

Liming Jiang 255 Nov 23, 2022
MAU: A Motion-Aware Unit for Video Prediction and Beyond, NeurIPS2021

MAU (NeurIPS2021) Zheng Chang, Xinfeng Zhang, Shanshe Wang, Siwei Ma, Yan Ye, Xinguang Xiang, Wen GAo. Official PyTorch Code for "MAU: A Motion-Aware

ZhengChang 20 Nov 25, 2022
Another pytorch implementation of FCN (Fully Convolutional Networks)

FCN-pytorch-easiest Trying to be the easiest FCN pytorch implementation and just in a get and use fashion Here I use a handbag semantic segmentation f

Y. Dong 158 Dec 21, 2022