[Point Cloud] M3DeTR: Multi-representation, Multi-scale, Mutual-relation 3D Object Detection with Transformers

Introduction to the paper:

This paper proposes a new 3D object detection architecture, M3DETR, which combines different point cloud representations (raw, voxel, bird's-eye view) with different feature scales based on a multi-scale feature pyramid.M3DETR is the first method to use Transformers to simultaneously unify multiple point cloud representations, feature scales, and model the interrelationships between point clouds.

The authors conduct extensive ablation experiments, emphasizing the benefits of fusing different representations and scales, and modeling the relationship.The method achieves state-of-the-art performance on the KITTI 3D object detection dataset and the Waymo open dataset.The results show that M3DETR significantly improves mAP by 1.48% over the baseline for all classes on the Waymo Open Dataset, and ranks first on the KITTI 3D detection benchmark for the car and bicycle classes, on the Waymo Open Dataset with a single-frame point cloud inputranked first.

Basic idea:

There are two key limitations of 3D object detection methods based on different networks:

• Invalid point cloud representation: The three main techniques used to process point clouds are voxel based, raw point cloud and bird's eye view.Each representation has a unique advantage, and it has been shown that combining these representations can improve detection accuracy.However, fusing these representations is not straightforward.First, the corresponding structures of the three types of neural networks are different.In addition, when