This code is for eCaReNet: explainable Cancer Relapse Prediction Network.

Related tags

Deep Learningecarenet
Overview

eCaReNet

This code is for eCaReNet: explainable Cancer Relapse Prediction Network. (Towards Explainable End-to-End Prostate Cancer Relapse Prediction from H&E Images Combining Self-Attention MultipleInstance Learning with a Recurrent Neural Network, Dietrich, E., Fuhlert, P., Ernst, A., Sauter, G., Lennartz, M., Stiehl, H. S., Zimmermann, M., Bonn, S. - ML4H 2021)

eCaReNet takes histopathology images (TMA spots) as input and predicts a survival curve and a risk score for individual patients. The network consists of an optional self-attention layer, an RNN and an attention based Mulitple Instance Learning module for explainability. To increase model performance, we suggest to include a binary prediction of a relapse as input to the model. alt text

TL;DR

  • store your dataset information in a .csv file

  • make your own my_config.yaml, following the example in config.yaml

  • run $ python train_model.py with my_config.yaml

Requirements and Installation

  • Python and Tensorflow
  • npm install -g omniboard to view results in browser and inspect experiments

Data preprocessing

All annotations of your images need to be stored in a .csv file with the image path and annotations as columns. You need separate csv files for your training, validation and test sets. Here is an example:

img_path censored relapse_time survived_2years ISUP_score
img1.png 0 80.3 1 3

The columns can be named as you wish, you need to tell python which columns to use in the config file ↓

config file

The config file (config.yaml) is needed to define the directories where the images and training, validation and test .csv files are stored. Further, you can choose whether to train a classification (for M_ISUP or M_Bin) or the survival model eCaReNet, which loss function and optimizer to use. Also the preprocessing is defined here (patching, resizing, ...) Details are found in config.yaml. It is best to create a custom my_config.yaml file and run the code as

$ python train_model.py with my_config.yaml 

You can also change single parameters in the command line like

$ python train_model.py with my_config.yaml general.seed="13" 

Training procedure

As recommended in our paper, we suggest to first train M_ISUP with config_isup.yaml. So train your base network (Inception or other) on a classification task as a transfer learning task. Second, train a binary classifier with config_bin.yaml, choose an appropriate time point to base the decision on. Here, you need to load the pretrained model from step one, do not load Inception or other keras models. For the third step the prediction from model two M_Bin are needed, so please store the information in the .csv file. Then again, load model from step one, and this time include the predictions as additional input and train.

Unittests

For most functions, a unittest is given in the test folder. This can be used to test if the function works correctly after adapting it (e.g. add more functionality or speed up). Further, it can be used for debugging to find errors or to find out what the function is actually doing. This is faster than running the whole code.

Docker

In the docker_context folder, the Dockerfile and requirements are stored. To build a docker image run

$ "docker build -t IMAGE_NAME:DATE docker_context"
$ docker build  -t ecarenet_docker:2021_10 docker_context

To use the image, run something like the following. Needs to be adapted to your own paths and resources

$ docker run --gpus=all --cpuset-cpus=5,6 --user `id --user`:`id --group` -it --rm -v /home/UNAME/PycharmProjects/ecarenet:/opt/project -v /PATH/TO/DATA:/data --network NETWORK_NAME--name MY_DOCKER_CONTAINER ecarenet_docker:2021_10 bash

More information on docker can be found here: https://docs.docker.com/get-started/

sacred

We use sacred (https://sacred.readthedocs.io/en/stable/) and a MongoDB (https://sacred.readthedocs.io/en/stable/observers.html https://www.mongodb.com/) to store all experiments. For each training run, a folder with an increasing id will be created automatically and all information about the run will be stored in that folder, like resulting weights, plots, metrics and losses. Which folder this is, is written in settings/default_settings and in the config in training.model_save_path.

The code works without mongodb, if you want. All results will be stored. If you do want to use the mongodb, you need to run a docker container with a mongoDB:

$ docker run -d -p PORTNUMBER:27017 -v ./my_data_folder:/data/db --name name_of_my_mongodb mongo

Create a network:

$ docker network create NETWORK_NAME

Attach container to network

$ docker network connect NETWORK_NAME name_of_my_mongodb

Then during training, the --network NETWORK_NAME property needs to be set. Use omniboard to inspect results:

$ omniboard -m localhost:PORTNUMBER:sacred

Tensorflow

Using tensorflow tf.data speeds up the data generation and preprocessing steps. If the dataset is very large, it can be cached with tf.cache() either in memory or to a file with tf.cache('/path/to/folder/plus/filename') [in dataset_creation/dataset_main.py]. Using tensorflow, it is also best to not overly use numpy functions or to decorate them with tf.function. Functions decorated with @tf.function will be included in the tensorflow graph in the first step and not be created again and again. For debugging, you need to remove the tf.function decorator, because otherwise the function (and breakpoints inside) will be skipped.

Owner
Institute of Medical Systems Biology
Institute of Medical Systems Biology
Pyramid addon for OpenAPI3 validation of requests and responses.

Validate Pyramid views against an OpenAPI 3.0 document Peace of Mind The reason this package exists is to give you peace of mind when providing a REST

Pylons Project 79 Dec 30, 2022
A Python library created to assist programmers with complex mathematical functions

libmaths libmaths was created not only as a learning experience for me, but as a way to make mathematical models in seconds for Python users using mat

Simple 73 Oct 02, 2022
AI Summer's complete catalog of articles

Learn Deep Learning with AI Summer A collection of all articles (almost 100) written for the AI Summer blog organized by topic. Deep Learning Theory M

AI Summer 95 Dec 29, 2022
Utilizes Pose Estimation to offer sprinters cues based on an image of their running form.

Running-Form-Correction Utilizes Pose Estimation to offer sprinters cues based on an image of their running form. How to Run Dependencies You will nee

3 Nov 08, 2022
PyTorch implementation for paper StARformer: Transformer with State-Action-Reward Representations.

StARformer This repository contains the PyTorch implementation for our paper titled StARformer: Transformer with State-Action-Reward Representations.

Jinghuan Shang 14 Dec 09, 2022
Python package for covariance matrices manipulation and Biosignal classification with application in Brain Computer interface

pyRiemann pyRiemann is a python package for covariance matrices manipulation and classification through Riemannian geometry. The primary target is cla

447 Jan 05, 2023
Implementation of UNet on the Joey ML framework

Independent Research Project - Code Joey can be cloned from here https://github.com/devitocodes/joey/. Devito and other dependencies such as PyTorch a

Navjot Kukreja 1 Oct 21, 2021
A scikit-learn-compatible module for estimating prediction intervals.

MAPIE - Model Agnostic Prediction Interval Estimator MAPIE allows you to easily estimate prediction intervals (or prediction sets) using your favourit

588 Jan 04, 2023
Weakly-supervised object detection.

Wetectron Wetectron is a software system that implements state-of-the-art weakly-supervised object detection algorithms. Project CVPR'20, ECCV'20 | Pa

NVIDIA Research Projects 342 Jan 05, 2023
A DCGAN to generate anime faces using custom mined dataset

Anime-Face-GAN-Keras A DCGAN to generate anime faces using custom dataset in Keras. Dataset The dataset is created by crawling anime database websites

Pavitrakumar P 190 Jan 03, 2023
Fake videos detection by tracing the source using video hashing retrieval.

Vision Transformer Based Video Hashing Retrieval for Tracing the Source of Fake Videos 🎉️ 📜 Directory Introduction VTL Trace Samples and Acc of Hash

56 Dec 22, 2022
[ICCV 2021] Encoder-decoder with Multi-level Attention for 3D Human Shape and Pose Estimation

MAED: Encoder-decoder with Multi-level Attention for 3D Human Shape and Pose Estimation Getting Started Our codes are implemented and tested with pyth

ZiNiU WaN 176 Dec 15, 2022
Assessing the Influence of Models on the Performance of Reinforcement Learning Algorithms applied on Continuous Control Tasks

Assessing the Influence of Models on the Performance of Reinforcement Learning Algorithms applied on Continuous Control Tasks This is the master thesi

Giacomo Arcieri 1 Mar 21, 2022
Improving Transferability of Representations via Augmentation-Aware Self-Supervision

Improving Transferability of Representations via Augmentation-Aware Self-Supervision Accepted to NeurIPS 2021 TL;DR: Learning augmentation-aware infor

hankook 38 Sep 16, 2022
Solving Zero-Shot Learning in Named Entity Recognition with Common Sense Knowledge

Zero-Shot Learning in Named Entity Recognition with Common Sense Knowledge Associated code for the paper Zero-Shot Learning in Named Entity Recognitio

Søren Hougaard Mulvad 13 Dec 25, 2022
This program was designed to detect whether someone is wearing a facemask through a live video stream.

This program was designed to detect whether someone is wearing a facemask through a live video stream. A custom lightweight CNN trained with TensorFlow on a public dataset provided by Kaggle is used

0 Apr 02, 2022
Have you ever wondered how cool it would be to have your own A.I

Have you ever wondered how cool it would be to have your own A.I. assistant Imagine how easier it would be to send emails without typing a single word, doing Wikipedia searches without opening web br

Harsh Gupta 1 Nov 09, 2021
Code for Mesh Convolution Using a Learned Kernel Basis

Mesh Convolution This repository contains the implementation (in PyTorch) of the paper FULLY CONVOLUTIONAL MESH AUTOENCODER USING EFFICIENT SPATIALLY

Yi_Zhou 35 Jan 03, 2023
Calibrated Hyperspectral Image Reconstruction via Graph-based Self-Tuning Network.

mask-uncertainty-in-HSI This repository contains the testing code and pre-trained models for the paper Calibrated Hyperspectral Image Reconstruction v

JIAMIAN WANG 9 Dec 29, 2022
A compendium of useful, interesting, inspirational usage of pandas functions, each example will be an ipynb file

Pandas_by_examples A compendium of useful/interesting/inspirational usage of pandas functions, each example will be an ipynb file What is this reposit

Guangyuan(Frank) Li 32 Nov 20, 2022