CVPR 2022 | greatly reduce the manual annotation required for zero sample learning. Mapuosuo and Beiyou proposed category semantic embedding rich in visual information

Zero sample learning aims to imitate human reasoning process , Using knowledge of visible categories , Identify invisible categories without training samples . Category embedding （class embeddings） It is a vector that describes category semantics and visual features , It can realize the transfer of knowledge between categories , Therefore, it plays an irreplaceable role in zero sample learning .

Zero sample classification diagram

As shown in the figure above , Because of attributes （attributes） Can be shared by different categories , Promote the transfer of knowledge between categories , Therefore, it is the most widely used category embedding . And in other computer vision tasks （ Such as face recognition 、 Fine grained classification 、 Fashion trend forecast ） Is widely used as auxiliary information . 

However, the process of attribute annotation requires a lot of manpower input and expert knowledge , It limits the expansion of zero sample learning on new data sets . Besides , Limited by human cognitive limitations , Its labeled attributes cannot traverse the visual space , Therefore, some distinguishing features in the image cannot be captured by attributes , Result in poor learning effect of zero samples .

For the above problems , From Beijing University of Posts and telecommunications 、 Researchers from Mapu Institute and other institutions have proposed category embedding mining network （Visually-Grounded Semantic Embedding Network, VGSE）, This article mainly answers two questions ：（1） How to automatically discover category embedding with semantic and visual features from visible class images ;（2） How to... Without training samples , Forecast category embedding for invisible categories .

• Thesis link : https://arxiv.org/abs/2203.10444

• Code link : https://github.com/wenjiaXu/VGSE

VGSE Model structure