STAR

Official implementation of Sparse Transformer-based Action Recognition

Dataset

download NTU RGB+D 60 action recognition of 2D/3D skeleton from http://rose1.ntu.edu.sg/datasets/actionRecognition.asp

or use google drive

NTU60 NTU120

uzip data as the following file structure: $(project_folder)/raw/.\*skeleton or $(project_folder)/dataset/raw/.\*skeleton (create "raw" folder under $(project_folder) or $(project_folder)/dataset then put raw skeleton files under "raw" folder)

run the code below to generate dataset:

python datagen.py

Training

git fetch and checkout to "distributed" branch

python train_dist.py -#distributed training

Configuration

parser.set_defaults(gpu=True,
                    batch_size=128,
                    dataset_name='NTU',
                    dataset_root=osp.join(os.getcwd()),  # or dataset_root=osp.join(os.getcwd(), 'dataset')
                    load_model=False,
                    in_channels=9,
                    num_enc_layers=5,
                    num_conv_layers=2,
                    weight_decay=4e-5,
                    drop_rate=[0.4, 0.4, 0.4, 0.4],  # linear_attention, sparse_attention, add_norm, ffn
                    hid_channels=64,
                    out_channels=64,
                    heads=8,
                    data_parallel=False,
                    cross_k=5,
                    mlp_head_hidden=128)

parser.set_defaults(gpu=True,
                    batch_size=128,
                    dataset_name='NTU',
                    dataset_root=osp.join(os.getcwd()),
                    load_model=False,
                    in_channels=9,
                    num_enc_layers=5,
                    num_conv_layers=2,
                    weight_decay=4e-5,
                    drop_rate=[0.4, 0.4, 0.4, 0.4],  # linear_attention, sparse_attention, add_norm, ffn
                    hid_channels=128,
                    out_channels=128,
                    heads=8,
                    data_parallel=False,
                    cross_k=5,
                    mlp_head_hidden=128)

Official implementation of Sparse Transformer-based Action Recognition

Related tags

Overview

STAR

Dataset

Training

Configuration

Owner

Chonghan_Lee

A Shading-Guided Generative Implicit Model for Shape-Accurate 3D-Aware Image Synthesis

A Re-implementation of the paper "A Deep Learning Framework for Character Motion Synthesis and Editing"

Cross-lingual Transfer for Speech Processing using Acoustic Language Similarity

Oscar and VinVL

Implementation of the master's thesis "Temporal copying and local hallucination for video inpainting".

EZ graph is an easy to use AI solution that allows you to make and train your neural networks without a single line of code.

An efficient PyTorch implementation of the evaluation metrics in recommender systems.

The DL Streamer Pipeline Zoo is a catalog of optimized media and media analytics pipelines.

Optimized code based on M2 for faster image captioning training

Snscrape-jsonl-urls-extractor - Extracts urls from jsonl produced by snscrape

Tensorflow/Keras Plug-N-Play Deep Learning Models Compilation

RRL: Resnet as representation for Reinforcement Learning

PatchMatch-RL: Deep MVS with Pixelwise Depth, Normal, and Visibility

Implementation of the Triangle Multiplicative module, used in Alphafold2 as an efficient way to mix rows or columns of a 2d feature map, as a standalone package for Pytorch

InvTorch: memory-efficient models with invertible functions

Official PyTorch implementation of "BlendGAN: Implicitly GAN Blending for Arbitrary Stylized Face Generation" (NeurIPS 2021)

Flexible-Modal Face Anti-Spoofing: A Benchmark

An NVDA add-on to split screen reader and audio from other programs to different sound channels

Enhancing Knowledge Tracing via Adversarial Training

Exploiting Robust Unsupervised Video Person Re-identification