Awesome Active Learning 
Background
(image source: Settles, Burr)
What is Active Learning?
Active learning is a special case of machine learning in which a learning algorithm can interactively query a oracle (or some other information source) to label new data points with the desired outputs.
(image source: Settles, Burr)
There are situations in which unlabeled data is abundant but manual labeling is expensive. In such a scenario, learning algorithms can actively query the oracle for labels. This type of iterative supervised learning is called active learning. Since the learner chooses the examples, the number of examples to learn a concept can often be much lower than the number required in normal supervised learning. With this approach, there is a risk that the algorithm is overwhelmed by uninformative examples. Recent developments are dedicated to multi-label active learning, hybrid active learning and active learning in a single-pass (on-line) context, combining concepts from the field of machine learning (e.g. conflict and ignorance) with adaptive, incremental learning policies in the field of online machine learning.
(source: Wikipedia)
Contributing
If you find the awesome paper/code/book/tutorial or have some suggestions, please feel free to pull requests or contact [email protected] to add papers using the following Markdown format:
Year | Paper Name | Conference | [Paper](link) | [Code](link) | Tags | Notes |
Thanks for your valuable contribution to the research community.
Table of Contents
Books
- Active Learning. Burr Settles. (CMU, 2012)
Surveys
- Active Learning Literature Survey. Settles, Burr. (2009)
- A Survey of Deep Active Learning. Pengzhen Ren et al. (2020)
- From Model-driven to Data-driven: A Survey on Active Deep Learning. Peng Liu et al. (2021)
Papers
Tags
Sur.: survey | Cri.: critics | Pool.: pool-based sampling | Str.: stream-based sampling | Syn.: membership query synthesize | Meta.: meta learning | SSL.: semi-supervised learning | RL.: reinforcement learning | FS.: few-shot learning | SS.: self-supervised |
Before 2017
| Year | Title | Conf | Paper | Code | Tags | Notes |
|---|---|---|---|---|---|---|
| 1994 | Improving Generalization with Active Learning | Machine Learning | paper | |||
| 2007 | Discriminative Batch Mode Active Learning | NIPS | paper | |||
| 2008 | Active Learning with Direct Query Construction | KDD | paper | |||
| 2008 | An Analysis of Active Learning Strategies for Sequence Labeling Tasks | EMNLP | paper | |||
| 2008 | Hierarchical Sampling for Active Learning | ICML | paper | |||
| 2010 | Active Instance Sampling via Matrix Partition | NIPS | paper | |||
| 2011 | Ask Me Better Questions: Active Learning Queries Based on Rule Induction | KDD | paper | |||
| 2011 | Active Learning from Crowds | ICML | paper | |||
| 2011 | Bayesian Active Learning for Classification and Preference Learning | CoRR | paper | |||
| 2011 | Active Learning Using On-line Algorithms | KDD | paper | |||
| 2012 | Bayesian Optimal Active Search and Surveying | ICML | paper | |||
| 2012 | Batch Active Learning via Coordinated Matching | ICML | paper | |||
| 2013 | Active Learning for Multi-Objective Optimization | ICML | paper | |||
| 2013 | Active Learning for Probabilistic Hypotheses Usingthe Maximum Gibbs Error Criterion | NIPS | paper | |||
| 2014 | Active Semi-Supervised Learning Using Sampling Theory for Graph Signals | KDD | paper | |||
| 2014 | Beyond Disagreement-based Agnostic Active Learning | NIPS | paper | |||
| 2016 | Cost-Effective Active Learning for Deep Image Classification | TCSVT | paper | |||
| 2016 | Active Image Segmentation Propagation | CVPR | paper |
2017
| Title | Conf | Paper | Code | Tags | Notes |
|---|---|---|---|---|---|
| Active Decision Boundary Annotation with Deep Generative Models | ICCV | paper | |||
| Active One-shot Learning | CoRR | paper | code | Str. RL. FS. |
|
| A Meta-Learning Approach to One-Step Active-Learning | [email protected]/ECML | paper | Pool. Meta. |
||
| Generative Adversarial Active Learning | arXiv | paper | Pool. Syn. |
||
| Active Learning from Peers | NIPS | paper | |||
| Learning Active Learning from Data | NIPS | paper | code | Pool. |
|
| Learning Algorithms for Active Learning | ICML | paper | |||
| Deep Bayesian Active Learning with Image Data | ICML | paper | code | Pool. |
2018
| Title | Conf | Paper | Code | Tags | Notes |
|---|---|---|---|---|---|
| The Power of Ensembles for Active Learning in Image Classification | CVPR | paper | |||
| Adversarial Learning for Semi-Supervised Semantic Segmentation | BMVC | paper | code | Pool. SSL. |
|
| A Variance Maximization Criterion for Active Learning | Pattern Recognition | paper | |||
| Meta-Learning Transferable Active Learning Policies by Deep Reinforcement Learning | ICLR-WS | paper | Pool. Meta. RL. |
||
| Active Learning for Convolutional Neural Networks: A Core-Set Approach | ICLR | paper | |||
| Adversarial Active Learning for Sequence Labeling and Generation | IJCAI | paper | |||
| Meta-Learning for Batch Mode Active Learning | ICLR-WS | paper |
2019
| Title | Conf | Paper | Code | Tags | Notes |
|---|---|---|---|---|---|
| ViewAL: Active Learning with Viewpoint Entropy for Semantic Segmentation | CVPR | paper | Pool. |
||
| Bayesian Generative Active Deep Learning | ICML | paper | code | Pool. Semi. |
|
| Variational Adversarial Active Learning | ICCV | paper | code | Pool. SSL. |
|
| Integrating Bayesian and Discriminative Sparse Kernel Machines for Multi-class Active Learning | NeurIPS | paper | |||
| Active Learning via Membership Query Synthesisfor Semi-supervised Sentence Classification | CoNLL | paper | |||
| Discriminative Active Learning | arXiv | paper | |||
| Semantic Redundancies in Image-Classification Datasets: The 10% You Don’t Need | arXiv | paper | |||
| Bayesian Batch Active Learning as Sparse Subset Approximation | NIPS | paper | |||
| Learning Loss for Active Learning | CVPR | paper | code | Pool. |
|
| Rapid Performance Gain through Active Model Reuse | IJCAI | paper | |||
| Parting with Illusions about Deep Active Learning | arXiv | paper | Cri. |
||
| BatchBALD: Efficient and Diverse Batch Acquisition for Deep Bayesian Active Learning | NIPS | paper |
2020
| Title | Conf | Paper | Code | Tags | Notes |
|---|---|---|---|---|---|
| Reinforced active learning for image segmentation | ICLR | paper | code | Pool. RL. |
|
| [BADGE] Batch Active learning by Diverse Gradient Embeddings | ICLR | paper | code | Pool. |
|
| Adversarial Sampling for Active Learning | WACV | paper | Pool. |
||
| Online Active Learning of Reject Option Classifiers | AAAI | paper | |||
| Deep Active Learning for Biased Datasets via Fisher Kernel Self-Supervision | CVPR | paper | |||
| Deep Reinforcement Active Learning for Medical Image Classification | MICCAI | paper | Pool. RL. |
||
| State-Relabeling Adversarial Active Learning | CVPR | paper | code | Pool. |
|
| Towards Robust and Reproducible Active Learning Using Neural Networks | arXiv | paper | Cri. |
||
| Consistency-Based Semi-supervised Active Learning: Towards Minimizing Labeling Cost | ECCV | paper | Pool. SSL. |
2021
| Title | Conf | Paper | Code | Tags | Notes |
|---|---|---|---|---|---|
| MedSelect: Selective Labeling for Medical Image Classification Combining Meta-Learning with Deep Reinforcement Learning | arXiv | paper | Pool. Meta. RL. |
||
| Can Active Learning Preemptively Mitigate Fairness Issues | ICLR-RAI | paper | code | Pool. |
Thinking fairness issues |
| Sequential Graph Convolutional Network for Active Learning | CVPR | paper | code | Pool. |
|
| Task-Aware Variational Adversarial Active Learning | CVPR | paper | code | Pool. |
|
| Effective Evaluation of Deep Active Learning on Image Classification Tasks | arXiv | paper | Cri. |
||
| Semi-Supervised Active Learning for Semi-Supervised Models: Exploit Adversarial Examples With Graph-Based Virtual Labels | ICCV | paper | Pool. SSL. |
||
| Contrastive Coding for Active Learning under Class Distribution Mismatch | ICCV | paper | code | Pool. |
Defines a good question |
| Mind Your Outliers! Investigating the Negative Impact of Outliers on Active Learning for Visual Question Answering | ACL-IJCNLP | paper | code | Pool. |
Thinking about outliers |
| LADA: Look-Ahead Data Acquisition via Augmentation for Active Learning | NeurIPS | paper | Pool. |
||
| Multi-Anchor Active Domain Adaptation for Semantic Segmentation | ICCV | paper | code | Pool. |
|
| Active Learning for Lane Detection: A Knowledge Distillation Approach | ICCV | paper | Pool. |
||
| Active Contrastive Learning of Audio-Visual Video Representations | ICLR | paper | code | Pool. |
|
| Multiple instance active learning for object detection | CVPR | paper | code | Pool. |
|
| SEAL: Self-supervised Embodied Active Learning using Exploration and 3D Consistency | NeurIPS | paper | Robot exploration | ||
| Influence Selection for Active Learning | ICCV | paper | code | Pool. |
|
| Reducing Label Effort: Self-Supervised meets Active Learning | arXiv | paper | Pool. SS. Cri. |
A meaningful attempt on the combination of SS & AL |
Turtorials
- Overview of Active Learning for Deep Learning. Jacob Gildenblat.
- Active Learning from Theory to Practice. Steve Hanneke, Robert Nowak. (ICML, 2019)
Tools
- ALiPy: Active Learning in Python. Ying-Peng Tang, Guo-Xiang Li, Sheng-Jun Huang. (NUAA, 2019)
- [BAAL] A Bayesian Active Learning Library. Atighehchian, Parmida and Branchaud-Charron, Frederic and Freyberg, Jan and Pardinas, Rafael and Schell, Lorne. (2019)
73 Dec 27, 2022
8 Mar 11, 2022
40 Jan 04, 2023
50 Dec 21, 2022
4 Jun 18, 2022
1.1k Jan 01, 2023
1 Feb 17, 2022
8 Nov 24, 2022
121 Dec 14, 2022
1 Aug 18, 2022
41 Dec 06, 2022
2 Nov 26, 2021
1.5k Dec 27, 2022
1.3k Jan 02, 2023
131 Jan 02, 2023
99 Dec 30, 2022
406 Dec 12, 2022
23 Nov 29, 2022
226 Jan 03, 2023
23 Dec 20, 2022