The Implicit Bias of Gradient Descent on Generalized Gated Linear Networks

Last update: Feb 05, 2022

Related tags

Overview

The Implicit Bias of Gradient Descent on Generalized Gated Linear Networks

This folder contains the code to reproduce the data in "The Implicit Bias of Gradient Descent on Generalized Gated Linear Networks."

To reproduce the 'data' folder:

Activate the environment provided as the 'environment.yml' file.
Run 'shallow_gln.sh', 'shallow_gln_cp.sh', 'shallow_gln_cp_2.sh', 'deep_gln.sh', 'deep_gln_cp.sh', 'relu_net.sh', and 'relu_net_cp.sh'. The convex optimization scripts (end in cp) must be run after their gradient descent counterparts (do not end in cp).

We then created an R package to package the cleaned datasets ('glnsanalysis', available in the folder 'implicit-bias-glns-analysis'). To access the datasets, build the packages. The notebooks in the folder 'vignettes' reproduce the figures.

Owner

Samuel Lippl

Neuroscience, statistics, mathematics

GitHub Repository

Implementation of the Paper: "Parameterized Hypercomplex Graph Neural Networks for Graph Classification" by Tuan Le, Marco Bertolini, Frank Noé and Djork-Arné Clevert

Parameterized Hypercomplex Graph Neural Networks (PHC-GNNs) PHC-GNNs (Le et al., 2021): https://arxiv.org/abs/2103.16584 PHM Linear Layer Illustration

26 Aug 11, 2022

NHL 94 AI contests

nhl94-ai The end goals of this project is to: Train Models that play NHL 94 Support AI vs AI contests in NHL 94 Provide an improved AI opponent for NH

2 Dec 06, 2021

A lightweight python AUTOmatic-arRAY library.

A lightweight python AUTOmatic-arRAY library. Write numeric code that works for: numpy cupy dask autograd jax mars tensorflow pytorch ... and indeed a

62 Dec 27, 2022

DeepSpeed is a deep learning optimization library that makes distributed training easy, efficient, and effective.

8.4k Jan 01, 2023

An evaluation toolkit for voice conversion models.

Voice-conversion-evaluation An evaluation toolkit for voice conversion models. Sample test pair Generate the metadata for evaluating models. The direc

30 Aug 29, 2022

OHLC Average Prediction of Apple Inc. Using LSTM Recurrent Neural Network

Stock Price Prediction of Apple Inc. Using Recurrent Neural Network OHLC Average Prediction of Apple Inc. Using LSTM Recurrent Neural Network Dataset:

410 Jan 05, 2023

NeRD: Neural Reflectance Decomposition from Image Collections

NeRD: Neural Reflectance Decomposition from Image Collections Project Page | Video | Paper | Dataset Implementation for NeRD. A novel method which dec

195 Dec 29, 2022

Official Pytorch and JAX implementation of "Efficient-VDVAE: Less is more"

The Official Pytorch and JAX implementation of "Efficient-VDVAE: Less is more" Arxiv preprint Louay Hazami · Rayhane Mama · Ragavan Thurairatn

144 Dec 23, 2022

YOLOX-CondInst - Implement CondInst which is a instances segmentation method on YOLOX

YOLOX CondInst -- YOLOX 实例分割前言本项目是自己学习实例分割时，复现的代码. 通过自己编程，让自己对实例分割有更进一步的了解。若想

16 Nov 18, 2022

A Planar RGB-D SLAM which utilizes Manhattan World structure to provide optimal camera pose trajectory while also providing a sparse reconstruction containing points, lines and planes, and a dense surfel-based reconstruction.

ManhattanSLAM Authors: Raza Yunus, Yanyan Li and Federico Tombari ManhattanSLAM is a real-time SLAM library for RGB-D cameras that computes the camera

117 Dec 28, 2022

Codes and models of NeurIPS2021 paper - DominoSearch: Find layer-wise fine-grained N:M sparse schemes from dense neural networks

DominoSearch This is repository for codes and models of NeurIPS2021 paper - DominoSearch: Find layer-wise fine-grained N:M sparse schemes from dense n

11 Sep 10, 2022

Autonomous Robots Kalman Filters

Autonomous Robots Kalman Filters The Kalman Filter is an easy topic. However, ma

20 Jul 18, 2022

The Implicit Bias of Gradient Descent on Generalized Gated Linear Networks

Related tags

Overview

The Implicit Bias of Gradient Descent on Generalized Gated Linear Networks

Owner

Samuel Lippl

Implementation of the Paper: "Parameterized Hypercomplex Graph Neural Networks for Graph Classification" by Tuan Le, Marco Bertolini, Frank Noé and Djork-Arné Clevert

NHL 94 AI contests

A lightweight python AUTOmatic-arRAY library.

DeepSpeed is a deep learning optimization library that makes distributed training easy, efficient, and effective.

An evaluation toolkit for voice conversion models.

OHLC Average Prediction of Apple Inc. Using LSTM Recurrent Neural Network

NeRD: Neural Reflectance Decomposition from Image Collections

Official Pytorch and JAX implementation of "Efficient-VDVAE: Less is more"

YOLOX-CondInst - Implement CondInst which is a instances segmentation method on YOLOX

A Planar RGB-D SLAM which utilizes Manhattan World structure to provide optimal camera pose trajectory while also providing a sparse reconstruction containing points, lines and planes, and a dense surfel-based reconstruction.

DI-smartcross - Decision Intelligence Platform for Traffic Crossing Signal Control

Official implementation of "Learning Proposals for Practical Energy-Based Regression", 2021.

UI2I via StyleGAN2 - Unsupervised image-to-image translation method via pre-trained StyleGAN2 network

Anomaly Transformer: Time Series Anomaly Detection with Association Discrepancy" (ICLR 2022 Spotlight)

Implementation of "Meta-rPPG: Remote Heart Rate Estimation Using a Transductive Meta-Learner"

A synthetic texture-invariant dataset for object detection of UAVs

Fast mesh denoising with data driven normal filtering using deep variational autoencoders

RL-GAN: Transfer Learning for Related Reinforcement Learning Tasks via Image-to-Image Translation

Codes and models of NeurIPS2021 paper - DominoSearch: Find layer-wise fine-grained N:M sparse schemes from dense neural networks

Autonomous Robots Kalman Filters