RVT: Robust Vision Transformers

This repository contains PyTorch code for Robust Vision Transformers.

For details see Rethinking the Design Principles of Robust Vision Transformer by Xiaofeng Mao, Gege Qi, Yuefeng Chen, Yuan He and Hui Xue.

Usage

First, clone the repository locally:

git clone https://github.com/vtddggg/Robust-Vision-Transformer.git

Then, install PyTorch 1.7.0+ and torchvision 0.8.1+ and pytorch-image-models 0.3.2:

conda install -c pytorch pytorch torchvision
pip install timm==0.3.2

We use 4 nodes with 8 gpus to train RVT-Ti, RVT-S and RVT-B:

Training RVT-Ti

python -m torch.distributed.launch --nproc_per_node=8 --nnodes=4 main.py --model rvt_tiny --data-path /path/to/imagenet --output_dir output --dist-eval

Training RVT-S

python -m torch.distributed.launch --nproc_per_node=8 --nnodes=4 main.py --model rvt_small --data-path /path/to/imagenet --output_dir output --dist-eval

Training RVT-B

python -m torch.distributed.launch --nproc_per_node=8 --nnodes=4 main.py --model rvt_base --data-path /path/to/imagenet --output_dir output --batch-size 32 --dist-eval

If you want to train RVT-Ti*, RVT-S* or RVT-B*, simply add --use_mask and --use_patch_aug to enable positon-aware attention scaling and patch-wise augmentation.

This repository contains PyTorch code for Robust Vision Transformers.

Related tags

Overview

RVT: Robust Vision Transformers

Usage

Training RVT-Ti

Training RVT-S

Training RVT-B

Owner

Danfeng Hong, Lianru Gao, Jing Yao, Bing Zhang, Antonio Plaza, Jocelyn Chanussot. Graph Convolutional Networks for Hyperspectral Image Classification, IEEE TGRS, 2021.

Wordle-solver - Wordle answer generation program in python

Yolo Traffic Light Detection With Python

VIMPAC: Video Pre-Training via Masked Token Prediction and Contrastive Learning

Makes patches from huge resolution .svs slide files using openslide

SHRIMP: Sparser Random Feature Models via Iterative Magnitude Pruning

LeViT a Vision Transformer in ConvNet's Clothing for Faster Inference

Use .csv files to record, play and evaluate motion capture data.

Multi-View Radar Semantic Segmentation

JFB: Jacobian-Free Backpropagation for Implicit Models

Implementation of Lie Transformer, Equivariant Self-Attention, in Pytorch

PyTorch Live is an easy to use library of tools for creating on-device ML demos on Android and iOS.

A Python multilingual toolkit for Sentiment Analysis and Social NLP tasks

A PyTorch implementation of Mugs proposed by our paper "Mugs: A Multi-Granular Self-Supervised Learning Framework".

Pervasive Attention: 2D Convolutional Networks for Sequence-to-Sequence Prediction

(CVPR2021) DANNet: A One-Stage Domain Adaptation Network for Unsupervised Nighttime Semantic Segmentation

RCD: Relation Map Driven Cognitive Diagnosis for Intelligent Education Systems

Annotated, understandable, and visually interpretable PyTorch implementations of: VAE, BIRVAE, NSGAN, MMGAN, WGAN, WGANGP, LSGAN, DRAGAN, BEGAN, RaGAN, InfoGAN, fGAN, FisherGAN

For visualizing the dair-v2x-i dataset

A PyTorch implementation of unsupervised SimCSE