CVPR 2022 | Tsinghua & bytek & JD put forward BRT: Bridging Transformer for vision and point cloud 3D target detection

In this paper, BrT(Bridged Transformer)： One for 3D End to end architecture for target detection , Simple and effective , It learns to recognize from points and image blocks 3D and 2D Target bounding box , Point clouds can be seamlessly fused with multi view images .

Thesis link ：https://link.zhihu.com/?target=https%3A//openaccess.thecvf.com/content/CVPR2022/html/Wang_Bridged_Transformer_for_Vision_and_Point_Cloud_3D_Object_Detection_CVPR_2022_paper.html

near , There is a tendency to utilize multiple input data sources , For example, use a..., which usually has more colors and less noise 2D Image to complement 3D Point cloud . However , because 2D and 3D Represents the heterogeneous geometry of , It prevents us from using ready-made neural networks to realize multimodal fusion .

So , We propose bridging Transformer (BrT), It's a way to 3D End to end architecture for target detection .

BrT Simple and effective , It learns to recognize from points and image blocks 3D and 2D Target bounding box . BrT A key element of is the use of object queries to bridge 3D and 2D Space , This is in Transformer Different data representation sources are unified in . We adopt a feature aggregation form realized by point to block projection , Further enhance the interaction between images and points . Besides ,BrT Point clouds can be seamlessly fused with multi view images .