[Detr for 3D object detection] detr3d: 3D object detection from multi view images via 3D-to-2D queries

Brief introduction of the paper ：

This paper introduces a framework for multi camera 3D target detection . The existing work is to estimate the 3D bounding box directly from monocular images , Or use the depth prediction network to generate the input of three-dimensional target detection from two-dimensional information , Unlike the , The method in this paper operates prediction directly in three-dimensional space .

DETR3D Extract two-dimensional features from multiple camera images , Then use a sparse set of 3D Object query to index these two-dimensional features , Use the camera conversion matrix to 3D The location is linked to the multi view image , Then the bounding box prediction is performed for each object query , Use the set to set loss to measure the difference between the ground truth and the prediction .

This top-down approach is better than the bottom-up approach , That is, the object boundary box prediction follows the depth estimation per pixel , Because it is not affected by the composite error introduced by the depth prediction model . Besides , This method does not require post-processing , If not the maximum inhibition , Significantly improve the reasoning speed , And in nuScenes The self driving benchmark has achieved the most advanced performance .

Contribution of thesis ：

• The original name is based on RGB 3D object detection model of image . Different from the existing work ,DETR3D At the last stage