Mae segmentation - Reproduction of semantic segmentation using masked autoencoder (mae)

Last update: Dec 17, 2022

Overview

ADE20k Semantic segmentation with MAE

Getting started

Install the mmsegmentation library and some required packages.

pip install mmcv-full==1.3.0 mmsegmentation==0.11.0
pip install scipy timm==0.3.2

Install apex for mixed-precision training

git clone https://github.com/NVIDIA/apex
cd apex
pip install -v --disable-pip-version-check --no-cache-dir --global-option="--cpp_ext" --global-option="--cuda_ext" ./

Follow the guide in mmseg to prepare the ADE20k dataset.

Fine-tuning for Reproducing Results of MAE ViT-Base

Command:

tools/dist_train.sh configs/mae/upernet_mae_base_12_512_slide_160k_ade20k.py 8 --seed 0  --options model.pretrained=https://dl.fbaipublicfiles.com/mae/pretrain/mae_pretrain_vit_base.pth

Expected results log(paper results: 48.1 mIoU):

+--------+-------+-------+-------+
| Scope  | mIoU  | mAcc  | aAcc  |
+--------+-------+-------+-------+
| global | 48.15 | 58.99 | 83.05 |
+--------+-------+-------+-------+

Evaluation

Command format:

tools/dist_test.sh  <CONFIG_PATH> <CHECKPOINT_PATH> <NUM_GPUS> --eval mIoU

Acknowledgment

This code is built using the mmsegmentation library, Timm library, the Swin repository, XCiT, SETR, BEiT and the MAE repository.

Mae segmentation - Reproduction of semantic segmentation using masked autoencoder (mae)

Related tags

Overview

ADE20k Semantic segmentation with MAE

Getting started

Fine-tuning for Reproducing Results of MAE ViT-Base

Evaluation

Acknowledgment

Owner

CS50's Introduction to Artificial Intelligence Test Scripts

Code for models used in Bashiri et al., "A Flow-based latent state generative model of neural population responses to natural images".

Robust Lane Detection via Expanded Self Attention (WACV 2022)

deep learning for image processing including classification and object-detection etc.

High-performance moving least squares material point method (MLS-MPM) solver.

Pytorch implementation of face attention network

Official implementation of VQ-Diffusion

The official implementation of NeurIPS 2021 paper: Finding Optimal Tangent Points for Reducing Distortions of Hard-label Attacks

Official code for MPG2: Multi-attribute Pizza Generator: Cross-domain Attribute Control with Conditional StyleGAN

This is a pytorch implementation for the BST model from Alibaba https://arxiv.org/pdf/1905.06874.pdf

Conditional Generative Adversarial Networks (CGAN) for Mobility Data Fusion

Image Segmentation with U-Net Algorithm on Carvana Dataset using AWS Sagemaker

This is the repo for our work "Towards Persona-Based Empathetic Conversational Models" (EMNLP 2020)

Prompts - Read a textfile of prompts and import into anki via ankiconnect

Code for SIMMC 2.0: A Task-oriented Dialog Dataset for Immersive Multimodal Conversations

Generalized Decision Transformer for Offline Hindsight Information Matching

Vertex AI: Serverless framework for MLOPs (ESP / ENG)

Supervised 3D Pre-training on Large-scale 2D Natural Image Datasets for 3D Medical Image Analysis

codes for IKM (arXiv2021, Submitted to IEEE Trans)

Contrastive Language-Image Pretraining