SST-Calib: A lidar-visual extrinsic parameter calibration method combining semantics and VO for spatiotemporal synchronization calibration (ITSC 2022)

原文链接： SST-Calib：Combined with the semantic andVOSpace-time simultaneous calibration oflidar-visualOutside parameter calibration method(ITSC 2022)

论文阅读：SST-Calib: Simultaneous Spatial-Temporal Parameter Calibration between LIDAR and Camera(ITSC 2022)

(注：ITSCIn intelligent transportation field event)

Motivation

对于大多数Visual和lidarFusion algorithm for,The parameter calibration will greatly affect performance.具体而言,Sensor fusion algorithms need to be very accurate parameter calibration and time synchronization between sensors.So can a joint estimationvisual-lidarOutside and correct geometry and time parameters of the algorithm is valuable.另外,Considering the experience when the vehicle vibration or collision,Manual calibration outside the cords will failure.Therefore automatic parameter calibration function have also is very important.

Recently rise forecast based on studying the sensor parameter method is often a data-driven,Need to tag data as supervision to forecast in advance,However true value calibration data is difficult to get,And these based on the study of work they would not be used in other data sets the generalization ability of,So a regression forecast based on semantics of different sensor input parameter method can to some extent the problems.

最后,At present a lot of data set including the calibration of sensor fusion methods ignore the time problem,Most of the sensor fusion method based on the synchronization time hypothesis,But this assumes that it is difficult to completely set up,So the time delay between the dynamic calibration sensor is necessary.

Contribution

1. 针对lidar和visual融合,This paper proposes a joint space-time calibration algorithm.

2. Designed a two-way loss function in geometric parameters regression to obtain more stable performance.

3. Will participate in timevisual odometryTo estimate the time delay between sensor combining.

Content

1. 系统概述

Calibration process below.Calibration including used in space and the initial calibration parameters calibration module and static space for double parameter estimation of joint space-time parameters calibration module.

Input of the algorithm consists of alidarPoint cloud scanning frameAnd two consecutiveRGB图像,Algorithm's goal is to estimate the6dofThe outside and refs与之间的时间延迟.

General practice is to first by the semantic segmentation method of arbitrary point cloud and image out of their semanticmask 与.And then by parameter calibration initial value outsideAnd camera known inside,将lidarPoint cloud onto the camera image plane,At the same time through from point to the pixels and the pixels from point-to-point nearest neighbor search,After and calculate their Euclidean distance used as the optimal cost function.

首先,优化迭代（Static space parameter calibration）Will be in default of speed is about0On the frame,从而可以得到,And use this as a joint space-time calibration after initial value and regularization item reference.然后,在动态场景中,通过visual odometryEstimated time of adjacent frames, information and speed.Under the condition of no external force,And between the translation can be defined as,And then through a narrow optimization to calculate and.

2.语义分割

理论上讲,This article can use any semantic network segmentation,The author USES the semantic network segmentation are respectivelySqueezeSegV3(点云)与SDC-net(图像),Besides considering the characteristics of the urban environment,Use only the vehicles such asmask的输出,最终可以得到mask输出与.

3.点云投影

Will belong to each point cloudmaskPoint projection on the camera image：

4.Two-way loss

定义For after the projection in camerafovWithin the scope of the laser point set,For each of the laser projection point,Is the nearest pixels belonging to the same category,所以,单向（激光点-像素点）Semantic alignment loss in framekOn can be defined as：

The loss calculation process of the diagram below.

But when outside and initial value with theground truthClearly not at the same time,The nearest neighbor matching can not bring the right match,So some important information of pixel may be discarded.因此,One-way loss function into the local minimum problem.为了避免这个问题,Two-way loss function is filed as improvement,具体过程如下图：

Two-way loss function is actually based on one-way loss function adds a pixel point-The loss of the laser spot,像素点-Laser spot losses are defined as follows：

Thus finally two-way loss function in the firstlIteration can be expressed as：

其中,为正则化项,为迭代次数l所对应的权重,When the smaller,On behalf of the bias towards the laser spot projection to the image plane,Whereas means towards willvisual maskThe projection point cloud cluster,Through adjustments to avoid falling into local minimum loss,Finally the optimized bidirectional,Can produce a better outside the static parameter calibration value：

5.Joint space-time calibration

In the joint space and time calibration before,通过visual odometryExtraction of two successive framesBetween the speed of movement of the,This paper usedVOIs based on sparse optical flowFASTPoint tracking and use5点法RANSACEstimate the eigen matrix.

Under the condition of the vehicle movement,Based on static parameter of camera and the point cloud projection is hard to image alignment,So in order to compensate time delay,Amend the projection matrix as follows：

其中,与Laser projection are the coordinates of points in the camera coordinate system with compensation,And then modify the bidirectional loss function,Can make it at the same time as space and time calibration parameters,The modified two-way loss function is as follows：

其中,是正则化项,The purpose is to make the parameter estimates are closer to the initial value,And is used for translation and rotation of the regularization coefficient

6.实验

采用的KITTI 00序列,For static correction,应用在前20次的迭代,Applications for the next30次迭代,应用在最后·10次迭代,That is to say, optimization of60次迭代次数.For joint optimization of time and space,迭代次数设置为20次,应用,正则化项和

Using quaternions Angle difference(QAD)And euler Angle difference(AEAD)To measure rotation error

Using the average translational error(ATD)To review translation error

1）Verify the joint space-time calibration

Using asynchronous data,Algorithm to estimate the time delay for103.4 ms,With true value only differ3.4 ms.External calibration precision and record in the table below,和baselineThan also showed the significant advantage.

2）鲁棒性验证(After the noise involved in calibration)

To the outside and pan joined distribution in的噪声,To the outside and rotate to join和的噪声,After joining the performance andbaseline比,In the case of don't need pretreatment goes into better accuracy.

类似的,Give delay also joined the noise,Further evidence that the robustness of the system

3）消融实验

Verified in this paper, the effectiveness of the proposed two-way loss function

Conclusion

This article proposed a joint space-time calibration onlinelidar-visual算法.Designed a two-way semantic lossh函数,结合了来自VOThe time delay estimation of,At the same time estimate parameter and time delay.在KITTIData set to justifying the its validity and robustness,It is a good parameter calibration work outside.

本文仅做学术分享,如有侵权,请联系删文.

独家重磅课程官网：cvlife.net

全国最大的机器人SLAM开发者社区

技术交流群

— 版权声明 —

本公众号原创内容版权属计算机视觉life所有;从公开渠道收集、整理及授权转载的非原创文字、图片和音视频资料,版权属原作者.如果侵权,请联系我们,会及时删除.