Camera and lidar calibration: gazebo simulation livox_ camera_ lidar_ Calibration ---- external parameter calibration calculation and result verification

Preface

ROS Function pack :livox_camera_lidar_calibration A manual calibration is provided Livox The method of external parameters between radar and camera , Already in Mid-40,Horizon and Tele-15 Verified on . It contains the calculation of camera internal parameters , Get the calibration data , Optimize and calculate code related to radar camera fusion application . In this scheme, the corner of the calibration plate is used as the calibration target , because Livox Characteristics of radar non repetitive scanning , The point cloud has a high density , It is easier to find the exact position of the corner in the radar point cloud . The calibration and fusion of camera and radar can also get good results .

In the previous article link : Camera and lidar calibration :gazebo Simulation livox_camera_lidar_calibration— Calibration data acquisition and processing

Finally, two files of calibration data are obtained :

• corner_lidar.txt
• corner_photo.txt

Each row of the two data corresponds to the same corner .

With so many groups of pixels with the same name and three-dimensional coordinate positions , Next, we can use the idea of optimization , To solve the external parameters of two sensors .

This blog mainly introduces the calibration calculation of external parameters , Verify with the results

External parameter calculation

The external parameter calculation node will read the previous data/corner_photo.txt and data/corner_lidar.txt To calculate the external parameters , The data needs to be saved in a specific format before it can be correctly read by the external parameter calculation node .

When the program reads data , The setting is like this :
Each row of data only has more than 10 An alphabetic program will read it as a calculated parameter , So the data above is used for numbering 1 2 3 4 And the title ,test0 test1 Will not be read .
When the program reads an empty line, it will stop reading parameters and start calculation , So don't leave blank lines when saving .

Before calculation getExt1.launch Configure the initial value of external parameters in the file ( The default initial value in the program is based on Livox Lidar's own coordinate system , The relative position of radar and camera is set , It should be modified according to the situation . If the initial value is very poor, it may lead to bad optimization results .)

<?xml version="1.0" encoding="UTF-8"?>
<launch>
  
    <rosparam param="init_value">  [0.0, -1.0, 0.0, 0.2,
                                    0.0, 0.0, -1.0, 0.0,
                                    1.0, 0.0, 0.0,  0.0] </rosparam>  <!-- init value of roatation matrix(3*3 on the left) and the translation(3*1 vector on the right) -->

    <param name="intrinsic_path"    value="$(find camera_lidar_calibration)/data/parameters/intrinsic.txt" />  <!-- intrinsic file -->
    <param name="extrinsic_path"    value="$(find camera_lidar_calibration)/data/parameters/extrinsic.txt" />  <!-- extrinsic file -->
    <param name="input_lidar_path"  value="$(find camera_lidar_calibration)/data/corner_lidar.txt" />  <!-- get the lidar corner data -->
    <param name="input_photo_path"  value="$(find camera_lidar_calibration)/data/corner_photo.txt" />  <!-- get the photo corner data -->
    <param name="error_threshold"    type="int" value="12" />  <!-- the threshold of the reprojection error -->

    <node pkg="camera_lidar_calibration" name="getExt1" type="getExt1" output="screen"></node>

</launch>

Input instructions to start calculating external parameters

roslaunch camera_lidar_calibration getExt1.launch

The terminal will print ceres Iterative process of

You can see from the terminal
Initial cost yes 3.496e4, Optimized to 5.749e1

Optimize camera internal and external parameters at the same time

getExt1 Nodes only optimize external parameters , and getExt2 When calculating, the node will optimize the internal parameter calculated at the beginning as the initial value and external parameter . The input instruction program will get a new internal and external parameter , And the new parameters are used to verify the re projection .

roslaunch camera_lidar_calibration getExt2.launch

In general use getExt1 The node can be , If the initial value of the external parameter has been verified , And after the outliers have been eliminated , Optimization still has a large residual , Then you can use getExt2 Have a try . The premise of use needs to ensure a large amount of calibration data , And fully verify the results .

If verified getExt2 The result of calculation is really better , Then update the new internal parameters in data/parameters/intrinsic.txt in .

The results verify that

After obtaining external parameters, we can use two common applications to see the effect of fusion . The first is to project the point cloud onto the photo , The second is the coloring of point clouds

On camera and livox Put a big object in front of the radar , Look at the final effect
I put a house in front , The photo is shown below :

The point cloud is shown in the figure below :

Project point clouds onto photos

Image before optimization :

Optimized image :

The optimization effect can be seen in the upper right corner

Point cloud shading

The following is the comparison of point cloud shading before and after optimization

Point cloud before optimization :

Optimized point cloud :

Look at the left side of the roof , The effect is obvious .