Implementation of Geometric Vector Perceptron, a simple circuit for 3d rotation equivariance for learning over large biomolecules, in Pytorch. Idea proposed and accepted at ICLR 2021

Last update: Nov 24, 2022

Overview

Geometric Vector Perceptron

Implementation of Geometric Vector Perceptron, a simple circuit with 3d rotation equivariance for learning over large biomolecules, in Pytorch. The repository may also contain experimentation to see if this could be easily extended to self-attention.

Install

$ pip install geometric-vector-perceptron

Functionality

GVP: Implementing the basic geometric vector perceptron.
GVPDropout: Adapted dropout for GVP in MPNN context
GVPLayerNorm: Adapted LayerNorm for GVP in MPNN context
GVP_MPNN: Adapted instance of Message Passing class from torch-geometric package. Still not tested.

Usage

import torch
from geometric_vector_perceptron import GVP

model = GVP(
    dim_vectors_in = 1024,
    dim_feats_in = 512,
    dim_vectors_out = 256,
    dim_feats_out = 512
)

feats, vectors = (torch.randn(1, 512), torch.randn(1, 1024, 3))

feats_out, vectors_out = model( (feats, vectors) ) # (1, 256), (1, 512, 3)

With the specialized dropout and layernorm as described in the paper

import torch
from torch import nn
from geometric_vector_perceptron import GVP, GVPDropout, GVPLayerNorm

model = GVP(
    dim_vectors_in = 1024,
    dim_feats_in = 512,
    dim_vectors_out = 256,
    dim_feats_out = 512
)

dropout = GVPDropout(0.2)
norm = GVPLayerNorm(512)

feats, vectors = (torch.randn(1, 512), torch.randn(1, 1024, 3))

feats, vectors = model( (feats, vectors) )
feats, vectors = dropout(feats, vectors)
feats, vectors = norm(feats, vectors)  # (1, 256), (1, 512, 3)

TF implementation:

The original implementation in TF by the paper authors can be found here: https://github.com/drorlab/gvp/

Citations

@inproceedings{
    anonymous2021learning,
    title={Learning from Protein Structure with Geometric Vector Perceptrons},
    author={Anonymous},
    booktitle={Submitted to International Conference on Learning Representations},
    year={2021},
    url={https://openreview.net/forum?id=1YLJDvSx6J4},
    note={under review}
}

Comments

Adds MPNN wrapper

Could you double check it?

I've tried to follow the original implementation ( https://github.com/drorlab/gvp/ see src/models.py ) but it's weird since they seem to mix the edge_attrs and the node features and return a mix of them, but later they seem to discard them so :man_shrugging: (whereas i only return node features - bc idk how to modify edge_attrs in an easy way -> the aggregation is always node-based)

opened by hypnopump 3
Fixes edge issues and distinguishes between covalent and proximity bonds
Gets all covalent bonds for a protein conformation (different than proximity)

Reduces unused computations

Fixes edges (now undirected graph)

Adapts denoising example accordingly (not in reconstruction example yet)
opened by hypnopump 0

PyTorch reimplementation of the Smooth ReLU activation function proposed in the paper "Real World Large Scale Recommendation Systems Reproducibility and Smooth Activations" [arXiv 2022].

Smooth ReLU in PyTorch Unofficial PyTorch reimplementation of the Smooth ReLU (SmeLU) activation function proposed in the paper Real World Large Scale

10 Jan 2, 2023

Seach Losses of our paper 'Loss Function Discovery for Object Detection via Convergence-Simulation Driven Search', accepted by ICLR 2021.

CSE-Autoloss Designing proper loss functions for vision tasks has been a long-standing research direction to advance the capability of existing models

54 Dec 17, 2022

Neuralnetwork - Basic Multilayer Perceptron Neural Network for deep learning

Neural Network Just a basic Neural Network module Usage Example Importing Module

0 Nov 1, 2022

This repo contains the implementation of the algorithm proposed in Off-Belief Learning, ICML 2021.

Off-Belief Learning Introduction This repo contains the implementation of the algorithm proposed in Off-Belief Learning, ICML 2021. Environment Setup

32 Jan 5, 2023

From Perceptron model to Deep Neural Network from scratch in Python.

Neural-Network-Basics Aim of this Repository: From Perceptron model to Deep Neural Network (from scratch) in Python. ** Currently working on a basic N

1 Jan 14, 2022

[ICLR 2021, Spotlight] Large Scale Image Completion via Co-Modulated Generative Adversarial Networks

Large Scale Image Completion via Co-Modulated Generative Adversarial Networks, ICLR 2021 (Spotlight) Demo | Paper [NEW!] Time to play with our interac

373 Jan 2, 2023

A PyTorch implementation of Mugs proposed by our paper "Mugs: A Multi-Granular Self-Supervised Learning Framework".

Mugs: A Multi-Granular Self-Supervised Learning Framework This is a PyTorch implementation of Mugs proposed by our paper "Mugs: A Multi-Granular Self-

62 Nov 8, 2022

An attempt at the implementation of Glom, Geoffrey Hinton's new idea that integrates neural fields, predictive coding, top-down-bottom-up, and attention (consensus between columns)

GLOM - Pytorch (wip) An attempt at the implementation of Glom, Geoffrey Hinton's new idea that integrates neural fields, predictive coding,

173 Dec 14, 2022

Official Pytorch implementation of 6DRepNet: 6D Rotation representation for unconstrained head pose estimation.

6D Rotation Representation for Unconstrained Head Pose Estimation (Pytorch) Paper Thorsten Hempel and Ahmed A. Abdelrahman and Ayoub Al-Hamadi, "6D Ro

284 Dec 23, 2022

Implementation of Geometric Vector Perceptron, a simple circuit for 3d rotation equivariance for learning over large biomolecules, in Pytorch. Idea proposed and accepted at ICLR 2021

Related tags

Overview

Geometric Vector Perceptron

Install

Functionality

Usage

TF implementation:

Citations

You might also like...

PyTorch reimplementation of the Smooth ReLU activation function proposed in the paper "Real World Large Scale Recommendation Systems Reproducibility and Smooth Activations" [arXiv 2022].

Seach Losses of our paper 'Loss Function Discovery for Object Detection via Convergence-Simulation Driven Search', accepted by ICLR 2021.

Neuralnetwork - Basic Multilayer Perceptron Neural Network for deep learning

This repo contains the implementation of the algorithm proposed in Off-Belief Learning, ICML 2021.

From Perceptron model to Deep Neural Network from scratch in Python.

[ICLR 2021, Spotlight] Large Scale Image Completion via Co-Modulated Generative Adversarial Networks

A PyTorch implementation of Mugs proposed by our paper "Mugs: A Multi-Granular Self-Supervised Learning Framework".

An attempt at the implementation of Glom, Geoffrey Hinton's new idea that integrates neural fields, predictive coding, top-down-bottom-up, and attention (consensus between columns)

Official Pytorch implementation of 6DRepNet: 6D Rotation representation for unconstrained head pose estimation.

Comments

Adds MPNN wrapper

Fixes edge issues and distinguishes between covalent and proximity bonds

Releases(0.0.14)

0.0.14(Jun 8, 2021)

0.0.12(Mar 7, 2021)

0.0.11(Mar 5, 2021)

0.0.10(Mar 3, 2021)

0.0.9(Mar 1, 2021)

0.0.8(Feb 24, 2021)

0.0.7(Feb 21, 2021)

0.0.6(Jan 27, 2021)

0.0.5(Jan 24, 2021)

0.0.3(Jan 24, 2021)

0.0.2(Jan 14, 2021)

0.0.1(Jan 13, 2021)

Owner

Phil Wang

A customisable game where you have to quickly click on black tiles in order of appearance while avoiding clicking on white squares.

PyTorch code for BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation

Autoregressive Predictive Coding: An unsupervised autoregressive model for speech representation learning

Random Walk Graph Neural Networks

CLIP (Contrastive Language–Image Pre-training) trained on Indonesian data

Real-time ground filtering algorithm of cloud points acquired using Terrestrial Laser Scanner (TLS)

A Dying Light 2 (DL2) PAKFile Utility for Modders and Mod Makers.

StyleGAN2-ada for practice

Multi-Scale Vision Longformer: A New Vision Transformer for High-Resolution Image Encoding

Implementation of Kalman Filter in Python

This repository contains the code for our fast polygonal building extraction from overhead images pipeline.

Evaluating saliency methods on artificial data with different background types

Deep-learning-roadmap - All You Need to Know About Deep Learning - A kick-starter

Code for One-shot Talking Face Generation from Single-speaker Audio-Visual Correlation Learning (AAAI 2022)

Sequence Modeling with Structured State Spaces

Just Go with the Flow: Self-Supervised Scene Flow Estimation

WPPNets: Unsupervised CNN Training with Wasserstein Patch Priors for Image Superresolution

My take on a practical implementation of Linformer for Pytorch.

aka "Bayesian Methods for Hackers": An introduction to Bayesian methods + probabilistic programming with a computation/understanding-first, mathematics-second point of view. All in pure Python ;)

Self-supervised spatio-spectro-temporal represenation learning for EEG analysis