Awesome Pruning

A curated list of neural network pruning and related resources. Inspired by awesome-deep-vision, awesome-adversarial-machine-learning, awesome-deep-learning-papers and Awesome-NAS.

Please feel free to pull requests or open an issue to add papers.

Type of Pruning
2020 Venues
2019 Venues
2018 Venues
2017 Venues
2016 Venues
2015 Venues

Type of Pruning

Type	`F`	`W`	`Other`
Explanation	Filter pruning	Weight pruning	other types

2020

Title	Venue	Type	Code
HYDRA: Pruning Adversarially Robust Neural Networks	NeurIPS	`W`	PyTorch(Author)
Logarithmic Pruning is All You Need	NeurIPS	`W`	-
Directional Pruning of Deep Neural Networks	NeurIPS	`W`	-
Movement Pruning: Adaptive Sparsity by Fine-Tuning	NeurIPS	`W`	PyTorch(Author)
Sanity-Checking Pruning Methods: Random Tickets can Win the Jackpot	NeurIPS	`W`	PyTorch(Author)
Neuron Merging: Compensating for Pruned Neurons	NeurIPS	`F`	PyTorch(Author)
Neuron-level Structured Pruning using Polarization Regularizer	NeurIPS	`F`	PyTorch(Author)
SCOP: Scientific Control for Reliable Neural Network Pruning	NeurIPS	`F`	-
Storage Efficient and Dynamic Flexible Runtime Channel Pruning via Deep Reinforcement Learning	NeurIPS	`F`	-
The Generalization-Stability Tradeoff In Neural Network Pruning	NeurIPS	`F`	PyTorch(Author)
Pruning Filter in Filter	NeurIPS	`Other`	PyTorch(Author)
Position-based Scaled Gradient for Model Quantization and Pruning	NeurIPS	`Other`	PyTorch(Author)
Bayesian Bits: Unifying Quantization and Pruning	NeurIPS	`Other`	-
Pruning neural networks without any data by iteratively conserving synaptic flow	NeurIPS	`Other`	PyTorch(Author)
EagleEye: Fast Sub-net Evaluation for Efficient Neural Network Pruning	ECCV (Oral)	`F`	PyTorch(Author)
DSA: More Efficient Budgeted Pruning via Differentiable Sparsity Allocation	ECCV	`F`	-
DHP: Differentiable Meta Pruning via HyperNetworks	ECCV	`F`	PyTorch(Author)
Meta-Learning with Network Pruning	ECCV	`W`	-
Accelerating CNN Training by Pruning Activation Gradients	ECCV	`W`	-
DA-NAS: Data Adapted Pruning for Efficient Neural Architecture Search	ECCV	`Other`	-
Differentiable Joint Pruning and Quantization for Hardware Efficiency	ECCV	`Other`	-
Channel Pruning via Automatic Structure Search	IJCAI	`F`	PyTorch(Author)
Adversarial Neural Pruning with Latent Vulnerability Suppression	ICML	`W`	-
Proving the Lottery Ticket Hypothesis: Pruning is All You Need	ICML	`W`	-
Soft Threshold Weight Reparameterization for Learnable Sparsity	ICML	`WF`	Pytorch(Author)
Network Pruning by Greedy Subnetwork Selection	ICML	`F`	-
Operation-Aware Soft Channel Pruning using Differentiable Masks	ICML	`F`	-
DropNet: Reducing Neural Network Complexity via Iterative Pruning	ICML	`F`	-
Towards Efficient Model Compression via Learned Global Ranking	CVPR (Oral)	`F`	Pytorch(Author)
HRank: Filter Pruning using High-Rank Feature Map	CVPR (Oral)	`F`	Pytorch(Author)
Neural Network Pruning with Residual-Connections and Limited-Data	CVPR (Oral)	`F`	-
Multi-Dimensional Pruning: A Unified Framework for Model Compression	CVPR (Oral)	`WF`	-
DMCP: Differentiable Markov Channel Pruning for Neural Networks	CVPR (Oral)	`F`	TensorFlow(Author)
Group Sparsity: The Hinge Between Filter Pruning and Decomposition for Network Compression	CVPR	`F`	PyTorch(Author)
Few Sample Knowledge Distillation for Efficient Network Compression	CVPR	`F`	-
Discrete Model Compression With Resource Constraint for Deep Neural Networks	CVPR	`F`	-
Structured Compression by Weight Encryption for Unstructured Pruning and Quantization	CVPR	`W`	-
Learning Filter Pruning Criteria for Deep Convolutional Neural Networks Acceleration	CVPR	`F`	-
APQ: Joint Search for Network Architecture, Pruning and Quantization Policy	CVPR	`F`	-
Comparing Rewinding and Fine-tuning in Neural Network Pruning	ICLR (Oral)	`WF`	TensorFlow(Author)
A Signal Propagation Perspective for Pruning Neural Networks at Initialization	ICLR (Spotlight)	`W`	-
ProxSGD: Training Structured Neural Networks under Regularization and Constraints	ICLR	`W`	TF+PT(Author)
One-Shot Pruning of Recurrent Neural Networks by Jacobian Spectrum Evaluation	ICLR	`W`	-
Lookahead: A Far-sighted Alternative of Magnitude-based Pruning	ICLR	`W`	PyTorch(Author)
Dynamic Model Pruning with Feedback	ICLR	`WF`	-
Provable Filter Pruning for Efficient Neural Networks	ICLR	`F`	-
Data-Independent Neural Pruning via Coresets	ICLR	`W`	-
AutoCompress: An Automatic DNN Structured Pruning Framework for Ultra-High Compression Rates	AAAI	`F`	-
DARB: A Density-Aware Regular-Block Pruning for Deep Neural Networks	AAAI	`Other`	-
Pruning from Scratch	AAAI	`Other`	-

2019

Title	Venue	Type	Code
Network Pruning via Transformable Architecture Search	NeurIPS	`F`	PyTorch(Author)
Gate Decorator: Global Filter Pruning Method for Accelerating Deep Convolutional Neural Networks	NeurIPS	`F`	PyTorch(Author)
Deconstructing Lottery Tickets: Zeros, Signs, and the Supermask	NeurIPS	`W`	TensorFlow(Author)
One ticket to win them all: generalizing lottery ticket initializations across datasets and optimizers	NeurIPS	`W`	-
Global Sparse Momentum SGD for Pruning Very Deep Neural Networks	NeurIPS	`W`	PyTorch(Author)
AutoPrune: Automatic Network Pruning by Regularizing Auxiliary Parameters	NeurIPS	`W`	-
Model Compression with Adversarial Robustness: A Unified Optimization Framework	NeurIPS	`Other`	PyTorch(Author)
MetaPruning: Meta Learning for Automatic Neural Network Channel Pruning	ICCV	`F`	PyTorch(Author)
Accelerate CNN via Recursive Bayesian Pruning	ICCV	`F`	-
Adversarial Robustness vs Model Compression, or Both?	ICCV	`W`	PyTorch(Author)
Learning Filter Basis for Convolutional Neural Network Compression	ICCV	`Other`	-
Filter Pruning via Geometric Median for Deep Convolutional Neural Networks Acceleration	CVPR (Oral)	`F`	PyTorch(Author)
Towards Optimal Structured CNN Pruning via Generative Adversarial Learning	CVPR	`F`	PyTorch(Author)
Centripetal SGD for Pruning Very Deep Convolutional Networks with Complicated Structure	CVPR	`F`	PyTorch(Author)
On Implicit Filter Level Sparsity in Convolutional Neural Networks, Extension1, Extension2	CVPR	`F`	PyTorch(Author)
Structured Pruning of Neural Networks with Budget-Aware Regularization	CVPR	`F`	-
Importance Estimation for Neural Network Pruning	CVPR	`F`	PyTorch(Author)
OICSR: Out-In-Channel Sparsity Regularization for Compact Deep Neural Networks	CVPR	`F`	-
Partial Order Pruning: for Best Speed/Accuracy Trade-off in Neural Architecture Search	CVPR	`Other`	TensorFlow(Author)
Variational Convolutional Neural Network Pruning	CVPR	-	-
The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks	ICLR (Best)	`W`	TensorFlow(Author)
Rethinking the Value of Network Pruning	ICLR	`F`	PyTorch(Author)
Dynamic Channel Pruning: Feature Boosting and Suppression	ICLR	`F`	TensorFlow(Author)
SNIP: Single-shot Network Pruning based on Connection Sensitivity	ICLR	`W`	TensorFLow(Author)
Dynamic Sparse Graph for Efficient Deep Learning	ICLR	`F`	CUDA(3rd)
Collaborative Channel Pruning for Deep Networks	ICML	`F`	-
Approximated Oracle Filter Pruning for Destructive CNN Width Optimization github	ICML	`F`	-
EigenDamage: Structured Pruning in the Kronecker-Factored Eigenbasis4	ICML	`W`	PyTorch(Author)
COP: Customized Deep Model Compression via Regularized Correlation-Based Filter-Level Pruning	IJCAI	`F`	Tensorflow(Author)

2018

Title	Venue	Type	Code
Rethinking the Smaller-Norm-Less-Informative Assumption in Channel Pruning of Convolution Layers	ICLR	`F`	TensorFlow(Author), PyTorch(3rd)
To prune, or not to prune: exploring the efficacy of pruning for model compression	ICLR	`W`	-
Discrimination-aware Channel Pruning for Deep Neural Networks	NeurIPS	`F`	TensorFlow(Author)
Frequency-Domain Dynamic Pruning for Convolutional Neural Networks	NeurIPS	`W`	-
Learning Sparse Neural Networks via Sensitivity-Driven Regularization	NeurIPS	`WF`	-
Amc: Automl for model compression and acceleration on mobile devices	ECCV	`F`	TensorFlow(3rd)
Data-Driven Sparse Structure Selection for Deep Neural Networks	ECCV	`F`	MXNet(Author)
Coreset-Based Neural Network Compression	ECCV	`F`	PyTorch(Author)
Constraint-Aware Deep Neural Network Compression	ECCV	`W`	SkimCaffe(Author)
A Systematic DNN Weight Pruning Framework using Alternating Direction Method of Multipliers	ECCV	`W`	Caffe(Author)
PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruning	CVPR	`F`	PyTorch(Author)
NISP: Pruning Networks using Neuron Importance Score Propagation	CVPR	`F`	-
CLIP-Q: Deep Network Compression Learning by In-Parallel Pruning-Quantization	CVPR	`W`	-
“Learning-Compression” Algorithms for Neural Net Pruning	CVPR	`W`	-
Soft Filter Pruning for Accelerating Deep Convolutional Neural Networks	IJCAI	`F`	PyTorch(Author)
Accelerating Convolutional Networks via Global & Dynamic Filter Pruning	IJCAI	`F`	-

2017

Title	Venue	Type	Code
Pruning Filters for Efficient ConvNets	ICLR	`F`	PyTorch(3rd)
Pruning Convolutional Neural Networks for Resource Efficient Inference	ICLR	`F`	TensorFlow(3rd)
Net-Trim: Convex Pruning of Deep Neural Networks with Performance Guarantee	NeurIPS	`W`	TensorFlow(Author)
Learning to Prune Deep Neural Networks via Layer-wise Optimal Brain Surgeon	NeurIPS	`W`	PyTorch(Author)
Runtime Neural Pruning	NeurIPS	`F`	-
Designing Energy-Efficient Convolutional Neural Networks using Energy-Aware Pruning	CVPR	`F`	-
ThiNet: A Filter Level Pruning Method for Deep Neural Network Compression	ICCV	`F`	Caffe(Author), PyTorch(3rd)
Channel pruning for accelerating very deep neural networks	ICCV	`F`	Caffe(Author)
Learning Efficient Convolutional Networks Through Network Slimming	ICCV	`F`	PyTorch(Author)

2016

Title	Venue	Type	Code
Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding	ICLR (Best)	`W`	Caffe(Author)
Dynamic Network Surgery for Efficient DNNs	NeurIPS	`W`	Caffe(Author)

2015

Title	Venue	Type	Code
Learning both Weights and Connections for Efficient Neural Networks	NeurIPS	`W`	PyTorch(3rd)

EfficientDNNs

Embedded-Neural-Network

awesome-AutoML-and-Lightweight-Models

Model-Compression-Papers

knowledge-distillation-papers

Network-Speed-and-Compression

A curated list of neural network pruning resources.

Related tags

Overview

Awesome Pruning

Table of Contents

Type of Pruning

2020

2019

2018

2017

2016

2015

Related Repo

Owner

Yang He

PyTorch implementation for the paper Pseudo Numerical Methods for Diffusion Models on Manifolds

A neuroanatomy-based augmented reality experience powered by computer vision. Features 3D visuals of the Atlas Brain Map slices.

Python scripts for performing stereo depth estimation using the HITNET Tensorflow model.

Source code for our Paper "Learning in High-Dimensional Feature Spaces Using ANOVA-Based Matrix-Vector Multiplication"

This repository contains the code for TACL2021 paper: SummaC: Re-Visiting NLI-based Models for Inconsistency Detection in Summarization

Code for SIMMC 2.0: A Task-oriented Dialog Dataset for Immersive Multimodal Conversations

Code for the SIGIR 2022 paper "Hybrid Transformer with Multi-level Fusion for Multimodal Knowledge Graph Completion"

Lightwood is Legos for Machine Learning.

Official codebase for "B-Pref: Benchmarking Preference-BasedReinforcement Learning" contains scripts to reproduce experiments.

This is the official implementation for "Do Transformers Really Perform Bad for Graph Representation?".

Image Classification - A research on image classification and auto insurance claim prediction, a systematic experiments on modeling techniques and approaches

Array Camera Ptychography

CAPRI: Context-Aware Interpretable Point-of-Interest Recommendation Framework

EMNLP 2021 - Frustratingly Simple Pretraining Alternatives to Masked Language Modeling

The World of an Octopus: How Reporting Bias Influences a Language Model's Perception of Color

DeepLab resnet v2 model in pytorch

Visual odometry package based on hardware-accelerated NVIDIA Elbrus library with world class quality and performance.

Code for "Learning Graph Cellular Automata"

A Python training and inference implementation of Yolov5 helmet detection in Jetson Xavier nx and Jetson nano

2021-MICCAI-Progressively Normalized Self-Attention Network for Video Polyp Segmentation