2022.7.9 quick view of papers

Factorizing Knowledge in Neural Networks （ECCV2022）

The concept proposed in this article is very interesting ,mark Look in this direction ：Disentangled representation learning. Here are some definitions ：

Representation Factorization
Apart from the functionality disentanglement, we hope that learned representations of the factor networks are statistically independent as well, so that each sub-network master’s task-wise disentangled knowledge. This means task-specific features should only contain minimal information only related to a certain task, while the common representation contains as much information as possible.

Structural Factorization
The goal of structural factorization is to endow different sub-networks with functional distinctions. Each factor network is expected to inherit only a portion of the knowledge from the teacher and specializes in an individual task.

The difference between the method proposed in this paper and the existing multi-objective knowledge distillation method ：

The implementation principle of the method proposed in this paper ：

Not All Models Are Equal: Predicting Model Transferability in a Self-challenging Fisher Space（ECCV2022）

It gives a common way to quantify mobility , Calculate correlation .

Comparison between some existing methods and the proposed methods ：