Iros 2021 | new idea of laser vision fusion? Lidar intensity diagram +vpr

Paper reading 《Visual PlaceRecognition using LiDAR Intensity Information》

This article is mainly a work of experimental analysis , The article does not propose a new method , But there is a hypothesis ： Lidar intensity images can be used for VPR, Then the feasibility of this assumption is proved by the fusion of the existing methods . On the premise that this assumption holds , Laser vision fusion may have new ideas .

Motivation

VPR(Visual placerecognition) Is a common task , But there has never been a way to apply an image of point cloud intensity to VPR On the way , Considering the density of point cloud intensity map after cylindrical projection , The author thinks this idea is feasible . So in this work, the author analyzed VPR How to communicate with LiDAR Using data in combination , Different data sets are evaluated . It turns out that , This method is indeed an effective way to determine the closed loop .

Contribution

1、 Analyze and evaluate the application of existing visual position recognition technology in 3D LiDAR The performance of the scanner when prompted by its strength .

2、 Use 3D LiDAR Tested on several robot datasets VPR Several variants of the method . Experiments show that , To the existing VPR Direct adjustment of technology can produce reliable loopback , So as to realize large-scale laser only LiDAR SLAM

Content

1、 Laser radar

In a way , Lidar intensity is a function of reflection range and wavelength ：

A Represents the beam aperture measured at a fixed angle ,β Is the reflectivity of the object ,θ It is the absorption of the medium , Reflectivity β Acceptor component 、 The influence of roughness and moisture content as well as the incident angle of the beam hitting the surface , The specific diagram is as follows

The clock accuracy of the first return of the measurement signal limits the accuracy of distance measurement . However, an additional accuracy gain can be obtained by determining the phase difference between the transmitted and received signals . Scanners that target higher accuracy send multiple pulses , This in turn limits the frequency of distance measurements . Now the measurement frequency of a single sensor fm May reach 50 kHz. By selecting the rotation frequency of the beam fr, The angular resolution is directly 2πfm fr. Mechanical considerations limit the rotational speed of the sensor . stay 2D Under the circumstances , The beam can be deflected through a relatively small rotating mirror , stay 3D Under the circumstances , Sensors can carry up to 128 A measuring unit .

The scanner also measures intensity information . The intensity information is normalized and discretized into 8 or 16 A value . Strength depends on surface properties , There are several other factors that affect the measurement . The above strength formula is continuous , So when measuring the same 3D a.m. , A slight change in viewpoint will produce a slight change in intensity .

I was aiming at SLAM And position recognition 3D-LiDAR Most of our work focuses on the use of distance measurement , Ignore other potentially valuable information . This may also be because the intensity information of the old lidar used in robotics is not so accurate . Current 3D-LiDAR Performance is very good . When the lidar intensity is projected into a panoramic image , Lidar intensity is reminiscent of the intensity obtained with a grayscale camera . without doubt ,LiDAR The quality of intensity image is still low compared with the image obtained by passive sensors such as cameras . however , In Robotics , Especially in VPR Tasks ,3D-LiDAR The intensity image generated by scanning has advantages such as being unaffected by external lighting conditions and shadows .

The popularity of self driving and autonomous vehicle has promoted 3D-LiDAR Improvement . In this application field , Their main purpose is to provide local 3D Reconstruction and obstacle information . Traditionally , Use LiDAR Go global 3D Reconstruction is a major challenge for the closed loop . And at present, there are relatively few loopback work based on lidar . by comparison ,CV The community has invested a lot of energy in the task of location identification and loopback , Impressive results have been achieved . therefore , The purpose of this article is to analyze common VPR Methods and LiDAR The feasibility of combining intensity information

 2.   Applied to visual position recognition of cylindrical lidar intensity image (VPR)

This part mainly introduces how the author deals with image formation from laser scanning , Then review VPR The general structure of .

A.  Image formation

Perform cylindrical projection , Convert LIDAR Points from Cartesian coordinate system to spherical coordinate system ：

Suppose the beam is uniformly distributed , You can calculate (u,v) as follows

If multiple points fall in the same image pixel , Then only the value with the nearest distance is retained . In each pixel of the created image , Store intensity values instead of distance .

The uneven distribution of the vertical beam may cause white space in the generated image , Usually the whole horizontal line . This problem can be solved a posteriori by reducing the vertical resolution or performing interpolation . In this experiment , The author adopts the second method . To remove blank lines from the panoramic image , First, binary thresholds and horizontal kernels as wide as the image are used to detect them . Then the interpolation of each pixel in the blank row is calculated by bilinear interpolation . The following figure shows the results of this process

B.  feature extraction

Experiments were carried out to test ORB,BRISK,SURF,Superpoints Other methods , See the experimental part for the specific results .C. VPR

The method tested by the author in the experiment is based on tree HBST And word bag based DBoW2, See the experimental part for the specific results .

3.   experiment

The main experiments are 5 Combined configurations :

FAST - ORB – HBST 、FAST - BRISK – HBST、FAST - ORB - DBoW2、Superpoint - DBoW2、FAST - SURF - DBoW2

The experimental data set is configured as follows :

Use FAST As a key detector ,Superpoint Output pairs of keys and descriptors directly for all experiments . For each data set , The following descriptors are extracted ：ORB、BRISK As BIN,Superpoint and SURF As a floating point . For retrieval methods , Use HBST Parameters δmax = 0.1 and Nmax = 50 To get BIN features , The detailed parameters are as follows :

For the above five configurations , Tested F1 indicators , The overall method accuracy is feasible ：

PR The curve index test is as follows ：

The time consumption is as follows :

In general , Among the methods of testing ,ORB And HBST and DBoW2 The combination results in the best .Superpoint-DBoW2 The performance of the combination is also good . But floating-point descriptors are also relatively expensive to compute . stay The Newer College The accuracy of the data set is not as good as that in IPB Car Accuracy obtained on the self recording data set of . Because these data are recorded while walking on campus , So the viewpoint changes obviously .

Conclusion

What this article says , It is mainly a work of experimental analysis . Firstly, the author proposes that the lidar intensity image can be used for VPR Assumptions , Then the method of using lidar intensity image in closed loop is analyzed through the existing methods VPR Performance of , The experiment was tested on four different data sets , The experimental results show that the lidar intensity image is applied to VPR It's very feasible . Although this article does not propose new methods , But this work is of great enlightening significance for the fusion of lidar and vision , For example, laser vision fusion loop .

This article is only for academic sharing , If there is any infringement , Please contact to delete .

3D Recommended visual quality courses ：

1. Multi sensor data fusion technology for automatic driving field

2. For the field of automatic driving 3D Whole stack learning route of point cloud target detection ！( Single mode + Multimodal / data + Code )
3. Thoroughly understand the visual three-dimensional reconstruction ： Principle analysis 、 Code explanation 、 Optimization and improvement
4. China's first point cloud processing course for industrial practice
5. laser - Vision -IMU-GPS The fusion SLAM Algorithm sorting and code explanation
6. Thoroughly understand the vision - inertia SLAM： be based on VINS-Fusion The class officially started
7. Thoroughly understand based on LOAM Framework of the 3D laser SLAM: Source code analysis to algorithm optimization
8. Thorough analysis of indoor 、 Outdoor laser SLAM Key algorithm principle 、 Code and actual combat (cartographer+LOAM +LIO-SAM)

9. Build a set of structured light from zero 3D Rebuild the system [ theory + Source code + practice ]

10. Monocular depth estimation method ： Algorithm sorting and code implementation

11. Deployment of deep learning model in autopilot

12. Camera model and calibration ( Monocular + Binocular + fisheye ）

13. blockbuster ！ Four rotor aircraft ： Algorithm and practice

14.ROS2 From entry to mastery ： Theory and practice

blockbuster ！3DCVer- Academic paper writing contribution   Communication group Established

Scan the code to add a little assistant wechat , can Apply to join 3D Visual workshop - Academic paper writing and contribution   WeChat ac group , The purpose is to communicate with each other 、 Top issue 、SCI、EI And so on .

meanwhile You can also apply to join our subdivided direction communication group , At present, there are mainly 3D Vision 、CV& Deep learning 、SLAM、 Three dimensional reconstruction 、 Point cloud post processing 、 Autopilot 、 Multi-sensor fusion 、CV introduction 、 Three dimensional measurement 、VR/AR、3D Face recognition 、 Medical imaging 、 defect detection 、 Pedestrian recognition 、 Target tracking 、 Visual products landing 、 The visual contest 、 License plate recognition 、 Hardware selection 、 Academic exchange 、 Job exchange 、ORB-SLAM Series source code exchange 、 Depth estimation Wait for wechat group .

Be sure to note ： Research direction + School / company + nickname , for example ：”3D Vision  + Shanghai Jiaotong University + quietly “. Please note... According to the format , Can be quickly passed and invited into the group . Original contribution Please also contact .

▲ Long press and add wechat group or contribute

▲ The official account of long click attention

3D Vision goes from entry to mastery of knowledge ： in the light of 3D In the field of vision Video Course cheng （ 3D reconstruction series 、 3D point cloud series 、 Structured light series 、 Hand eye calibration 、 Camera calibration 、 laser / Vision SLAM、 Automatically Driving, etc ）、 Summary of knowledge points 、 Introduction advanced learning route 、 newest paper Share 、 Question answer Carry out deep cultivation in five aspects , There are also algorithm engineers from various large factories to provide technical guidance . meanwhile , The planet will be jointly released by well-known enterprises 3D Vision related algorithm development positions and project docking information , Create a set of technology and employment as one of the iron fans gathering area , near 4000 Planet members create better AI The world is making progress together , Knowledge planet portal ：

Study 3D Visual core technology , Scan to see the introduction ,3 Unconditional refund within days

  There are high quality tutorial materials in the circle 、 Answer questions and solve doubts 、 Help you solve problems efficiently

Feel useful , Please give me a compliment ~