A Collection of LiDAR-Camera-Calibration Papers, Toolboxes and Notes

Overview

Awesome-LiDAR-Camera-Calibration

Awesome

A Collection of LiDAR-Camera-Calibration Papers, Toolboxes and Notes.

Outline

0. Introduction

For applications such as autonomous driving, robotics, navigation systems, and 3-D scene reconstruction, data of the same scene is often captured using both lidar and camera sensors. To accurately interpret the objects in a scene, it is necessary to fuse the lidar and the camera outputs together. Lidar camera calibration estimates a rigid transformation matrix (extrinsics, rotation+translation, 6 DoF) that establishes the correspondences between the points in the 3-D lidar plane and the pixels in the image plane.

Example

1. Target-based methods

Paper Target Feature Optimization Toolbox Note
Extrinsic Calibration of a Camera and Laser Range Finder (improves camera calibration), 2004 checkerboard C:Plane (a), L: pts in plane (m) point-to-plane CamLaserCalibraTool CN
Fast Extrinsic Calibration of a Laser Rangefinder to a Camera, 2005 checkerboard C: Plane (a), L: Plane (m) plane(n/d) correspondence, point-to-plane LCCT *
Extrinsic calibration of a 3D laser scanner and an omnidirectional camera, 2010 checkerboard C: plane (a), L: pts in plane (m) point-to-plane cam_lidar_calib *
LiDAR-Camera Calibration using 3D-3D Point correspondences, 2017 cardboard + ArUco C: 3D corners (a), L: 3D corners (m) ICP lidar_camera_calibration *
Reflectance Intensity Assisted Automatic and Accurate Extrinsic Calibration of 3D LiDAR and Panoramic Camera Using a Printed Chessboard, 2017 checkerboard C: 2D corners (a), L: 3D corners (a) PnP, angle difference ILCC *
Extrinsic Calibration of Lidar and Camera with Polygon, 2018 regular cardboard C: 2D edge, corners (a), L: 3D edge, pts in plane (a) point-to-line, point-inside-polygon ram-lab/plycal *
Automatic Extrinsic Calibration of a Camera and a 3D LiDAR using Line and Plane Correspondences, 2018 checkerboard C: 3D edge, plane(a), L: 3D edge, pts in plane (a) direcion/normal, point-to-line, point-to-plane Matlab LiDAR Toolbox *
Improvements to Target-Based 3D LiDAR to Camera Calibration, 2020 cardboard with ArUco C: 2d corners (a), L: 3D corners (a) PnP, IOU github *
ACSC: Automatic Calibration for Non-repetitive Scanning Solid-State LiDAR and Camera Systems, 2020 checkerboard C: 2D corners (a), L: 3D corners (a) PnP ACSC *
Automatic Extrinsic Calibration Method for LiDAR and Camera Sensor Setups, 2021 cardboard with circle & Aruco C: 3D points (a), L: 3D points (a) ICP velo2cam_ calibration *

C: camera, L: LiDAR, a: automaic, m: manual

2. Targetless methods

2.1. Motion-based methods

Paper Feature Optimization Toolbox Note
LiDAR and Camera Calibration Using Motions Estimated by Sensor Fusion Odometry, 2018 C: motion (ICP), L: motion (VO) hand-eye calibration * *

2.2. Scene-based methods

2.2.1. Traditional methods

Paper Feature Optimization Toolbox Note
Automatic Targetless Extrinsic Calibration of a 3D Lidar and Camera by Maximizing Mutual Information, 2012 C:grayscale, L: reflectivity mutual information, BB steepest gradient ascent Extrinsic Calib *
Automatic Calibration of Lidar and Camera Images using Normalized Mutual Information, 2013 C:grayscale, L: reflectivity, noraml normalized MI, particle swarm * *
Automatic Online Calibration of Cameras and Lasers, 2013 C: Canny edge, L: depth-discontinuous edge correlation, grid search * *
SOIC: Semantic Online Initialization and Calibration for LiDAR and Camera, 2020 semantic centroid PnP * *
A Low-cost and Accurate Lidar-assisted Visual SLAM System, 2021 C: edge(grayscale), L: edge (reflectivity, depth projection) ICP, coordinate descent algorithms CamVox *
Pixel-level Extrinsic Self Calibration of High Resolution LiDAR and Camera in Targetless Environments,2021 C:Canny edge(grayscale), L: depth-continuous edge point-to-line, Gaussian-Newton livox_camera_calib *
CRLF: Automatic Calibration and Refinement based on Line Feature for LiDAR and Camera in Road Scenes, 2021 C:straight line, L: straight line perspective3-lines (P3L) * CN

2.2.2. Deep-learning methods

Pape Toolbox Note
RegNet: Multimodal sensor registration using deep neural networks, 2017,IV regnet *
CalibNet: Geometrically supervised extrinsic calibration using 3d spatial transformer networks,2018,IROS CalibNet *

3. Other toolboxes

Toolbox Introduction Note
Apollo sensor calibration tools targetless method, no source code CN
Autoware camera lidar calibrator pick points mannually, PnP *
Autoware calibration camera lidar checkerboard, similar to LCCT CN
livox_camera_lidar_calibration pick points mannually, PnP *
Stock-history-display - something like a easy yearly review for your stock performance

Stock History Display Available on Heroku: https://stock-history-display.herokua

LiaoJJ 1 Jan 07, 2022
Code from PropMix, accepted at BMVC'21

PropMix: Hard Sample Filtering and Proportional MixUp for Learning with Noisy Labels This repository is the official implementation of Hard Sample Fil

6 Dec 21, 2022
MAVE: : A Product Dataset for Multi-source Attribute Value Extraction

MAVE: : A Product Dataset for Multi-source Attribute Value Extraction The dataset contains 3 million attribute-value annotations across 1257 unique ca

Google Research Datasets 89 Jan 08, 2023
FCOSR: A Simple Anchor-free Rotated Detector for Aerial Object Detection

FCOSR: A Simple Anchor-free Rotated Detector for Aerial Object Detection FCOSR: A Simple Anchor-free Rotated Detector for Aerial Object Detection arXi

59 Nov 29, 2022
This code is an unofficial implementation of HiFiSinger.

HiFiSinger This code is an unofficial implementation of HiFiSinger. The algorithm is based on the following papers: Chen, J., Tan, X., Luan, J., Qin,

Heejo You 87 Dec 23, 2022
Main Results on ImageNet with Pretrained Models

This repository contains Pytorch evaluation code, training code and pretrained models for the following projects: SPACH (A Battle of Network Structure

Microsoft 151 Dec 14, 2022
Writeups for the challenges from DownUnderCTF 2021

cloud Challenge Author Difficulty Release Round Bad Bucket Blue Alder easy round 1 Not as Bad Bucket Blue Alder easy round 1 Lost n Found Blue Alder m

DownUnderCTF 161 Dec 31, 2022
Deep Multi-Magnification Network for multi-class tissue segmentation of whole slide images

Deep Multi-Magnification Network This repository provides training and inference codes for Deep Multi-Magnification Network published here. Deep Multi

Computational Pathology 12 Aug 06, 2022
Code for the paper "Jukebox: A Generative Model for Music"

Status: Archive (code is provided as-is, no updates expected) Jukebox Code for "Jukebox: A Generative Model for Music" Paper Blog Explorer Colab Insta

OpenAI 6k Jan 02, 2023
A pure PyTorch batched computation implementation of "CIF: Continuous Integrate-and-Fire for End-to-End Speech Recognition"

A pure PyTorch batched computation implementation of "CIF: Continuous Integrate-and-Fire for End-to-End Speech Recognition"

張致強 14 Dec 02, 2022
PyBullet CartPole and Quadrotor environments—with CasADi symbolic a priori dynamics—for learning-based control and reinforcement learning

safe-control-gym Physics-based CartPole and Quadrotor Gym environments (using PyBullet) with symbolic a priori dynamics (using CasADi) for learning-ba

Dynamic Systems Lab 300 Dec 28, 2022
performing moving objects segmentation using image processing techniques with opencv and numpy

Moving Objects Segmentation On this project I tried to perform moving objects segmentation using background subtraction technique. the introduced meth

Mohamed Magdy 15 Dec 12, 2022
Official repository for "Action-Based Conversations Dataset: A Corpus for Building More In-Depth Task-Oriented Dialogue Systems"

Action-Based Conversations Dataset (ABCD) This respository contains the code and data for ABCD (Chen et al., 2021) Introduction Whereas existing goal-

ASAPP Research 49 Oct 09, 2022
Code for Talk-to-Edit (ICCV2021). Paper: Talk-to-Edit: Fine-Grained Facial Editing via Dialog.

Talk-to-Edit (ICCV2021) This repository contains the implementation of the following paper: Talk-to-Edit: Fine-Grained Facial Editing via Dialog Yumin

Yuming Jiang 221 Jan 07, 2023
A robust pointcloud registration pipeline based on correlation.

PHASER: A Robust and Correspondence-Free Global Pointcloud Registration Ubuntu 18.04+ROS Melodic: Overview Pointcloud registration using correspondenc

ETHZ ASL 101 Dec 01, 2022
Volumetric parameterization of the placenta to a flattened template

placenta-flattening A MATLAB algorithm for volumetric mesh parameterization. Developed for mapping a placenta segmentation derived from an MRI image t

Mazdak Abulnaga 12 Mar 14, 2022
ProjectOxford-ClientSDK - This repo has moved :house: Visit our website for the latest SDKs & Samples

This project has moved 🏠 We heard your feedback! This repo has been deprecated and each project has moved to a new home in a repo scoped by API and p

Microsoft 970 Nov 28, 2022
Toolbox of models, callbacks, and datasets for AI/ML researchers.

Pretrained SOTA Deep Learning models, callbacks and more for research and production with PyTorch Lightning and PyTorch Website • Installation • Main

Pytorch Lightning 1.4k Dec 30, 2022
OpenMMLab's Next Generation Video Understanding Toolbox and Benchmark

Introduction English | 简体中文 MMAction2 is an open-source toolbox for video understanding based on PyTorch. It is a part of the OpenMMLab project. The m

OpenMMLab 2.7k Jan 07, 2023
Pyramid Grafting Network for One-Stage High Resolution Saliency Detection. CVPR 2022

PGNet Pyramid Grafting Network for One-Stage High Resolution Saliency Detection. CVPR 2022, CVPR 2022 (arXiv 2204.05041) Abstract Recent salient objec

CVTEAM 109 Dec 05, 2022