"Actbert" Baidu & Sydney University of technology proposed actbert to learn the global and local video text representation, which is effective in five video text tasks!

Thesis link ：https://arxiv.org/abs/2011.07231

In this paper , The author puts forward ActBERT, Self supervised learning for joint video text representation in unlabeled data . First , The author uses global action information to analyze the interaction between language text and local area objects . It reveals global and local visual cues from paired video sequences and text descriptions , For detailed visual and text relationship modeling .

then , The author introduces a TaNgled Transformer block（TNT） To encode three information sources , Global behavior 、 Local area objects and language descriptions . overall situation - Local correspondence is found by clues extracted from context information . It enforces joint video text representation , To understand fine-grained objects and global human intentions . The author proves that ActBERT Generalization ability in downstream video and language tasks , Text video clip retrieval 、 Video subtitles 、 Video Q & A 、 Action segmentation and action step location .ActBERT The performance of is obviously superior to the most advanced technology , It shows its advantages in video text representation learning .