当前位置:网站首页>Introduction Guide to stereo vision (5): dual camera calibration [no more collection, I charge ~]
Introduction Guide to stereo vision (5): dual camera calibration [no more collection, I charge ~]
2022-07-05 08:56:00 【Li Yingsong~】
Dear students , Our world is 3D The world , Our eyes can observe three-dimensional information , Help us perceive distance , Navigation obstacle avoidance , So as to soar between heaven and earth . Today's world is an intelligent world , Our scientists explore a variety of machine intelligence technologies , Let the machine have the three-dimensional perception of human beings , And hope to surpass human beings in speed and accuracy , For example, positioning navigation in autopilot navigation , Automatic obstacle avoidance of UAV , Three dimensional scanning in the measuring instrument, etc , High intelligent machine intelligence technology is 3D Visual realization .
Stereo vision is an important direction in the field of 3D reconstruction , It simulates the structure of the human eye and simulates the binocular with two cameras , Project in Perspective 、 Based on triangulation , Through the logical complex homonymous point search algorithm , Restore the 3D information in the scene . It is widely used , Autopilot 、 Navigation obstacle avoidance 、 Cultural relic reconstruction 、 Face recognition and many other high-tech applications have its key figure .
This course will help you to understand the theoretical and practical knowledge of stereo vision from simple to profound . We will talk about the coordinate system and camera calibration , From passive stereoscopic to active stereoscopic , Even from deep recovery to grid construction and processing , Interested students , Come and explore the charm of stereovision with me !
This course is an electronic resource , So there will not be too many restrictions in the writing , But it will be logically clear 、 Easy to understand as the goal , Level co., LTD. , If there are deficiencies , Please give me some advice !
Personal wechat :EthanYs6, Add me to the technical exchange group StereoV3D, Talk about technology together .
CSDN Search for :Ethan Li Li Yingsong , see Web based courses .
Lesson code , Upload to github On , Address :StereoV3DCode:https://github.com/ethan-li-coding/StereoV3DCode
The first two bloggers introduced two single camera calibration methods , Some students will have doubts , This series is clearly stereo vision , Why introduce single camera calibration instead of dual camera calibration ? The reason is simple , That is, single camera calibration is the basis of dual camera calibration , say concretely , Dual camera calibration is to complete the calibration of two single cameras respectively , Then conduct the overall calibration . And listen to me ~
List of articles
Binocular stereo
Have a common sense cognition , That is, stereo vision generally refers to a stereo system composed of two cameras , Just like the human eye . When there is an overlapping area between the views taken by two cameras , They form a basic binocular stereo .
Double view through epipolar correction and stereo matching , You can get three-dimensional points through triangular relations ,
But the premise of all this :
- To accurately know the position relationship between two cameras , That is, the rotation of one camera to the other R R R Move peacefully t t t, namely The external parameters of the camera
- To accurately know the internal equipment parameters of dual cameras , For example, like the main point ( x 0 , y 0 ) (x_0,y_0) (x0,y0)、 The focal length f f f、 Distortion parameters, etc , namely The internal parameters of the camera
Introduction to internal and external parameters , Please see the previous article : Introduction to stereo vision (1): Coordinate system and camera parameters
Dual camera calibration
In general , We have seen the most binocular stereoscopic , Fixed on a rigid structure by two different cameras , Form a stable binocular stereo system . In use , There will be no obvious mutual movement between cameras , The stable dual view structure can complete the acquisition of single frame depth information at any time . The following figure shows some examples :
We said earlier , The camera internal parameter matrix can be obtained by a single camera K K K、 External parameter matrix R , t R,t R,t, Compared with single camera calibration, dual camera calibration
- One more relative external parameter , That is, the rotation matrix of the right camera relative to the left camera R R R Vector of peaceful shift t t t.
- Two absolute external parameters are missing , That is, the external parameter matrix calibrated by the single camera itself is no longer needed , Just keep 1 The relative external parameters obtained , It is called the external parameter of stereo vision system .( In fact, absolute external parameters are generally not required for single camera calibration , It is only natural output as calibrated output , In dual camera calibration , Absolute external parameters can be used as intermediate variables to calculate relative external parameters )
We said at the beginning of the article , Dual camera calibration is based on single camera calibration , I think smart students have come up with some ideas , First, we use the same shooting method to collect the calibration pattern , Most of the operation methods and precautions are the same , See Introduction to stereo vision (3): Zhang calibration method of camera calibration 【 Super detailed, worth collecting 】
The difference is this :
- Every time you shoot , Pay attention to whether the overlapping area of two cameras is enough , Generally more than 1/2 The image area of , Of course, the bigger the better , At some relatively large angles , A certain degree of overlap can be sacrificed .
- Note that the two cameras try to keep the same distance to collect , This is to ensure that the scale and clarity of the two views are consistent , Reduce the error caused by inconsistency .
Sketch Map :
The algorithm part
After image acquisition , Enter the algorithm section , As mentioned earlier , The first step is to calibrate the two cameras individually , Get the internal parameter matrix of each of the two cameras K、 Absolute external reference R , t R,t R,t( That is, the external parameters of the camera relative to the calibration plate at each acquisition ), And distortion coefficient d d d( there d d d Is the set of all distortion coefficients )
1. Do single camera calibration for two cameras separately , Get the internal parameter matrix of each of the two cameras K、 Absolute external reference R , t R,t R,t, And distortion coefficient d d d
Then some students may think of , According to the absolute external parameters collected each time , That is, the relative external parameters can be calculated , But obviously we will generate N Relative external parameters (N Is the number of image pairs collected , That is, the number of acquisition positions ), Which one is better ? Take the mean ? Or take the median ?
Experienced students will find out , Neither of these is reliable , Taking the mean value first has no robustness to significant outliers , As long as the absolute external parameter of a pair is wrong , The average value has a great influence ; Second, take the median because only one pair , The error average characteristic of multiple measurements is not used .
So the more reliable way is , Calculate the relative external parameters of each acquisition , And take the median value as the initial value , Based on minimizing the re projection error , Nonlinear iterative optimization obtains the optimal solution . Under this scheme , The internal parameters of the camera can be used as fixed values , The unknown number is N Set the absolute external parameters of the left camera ( R i , t i ) , i = 1 , . . n (R_i,t_i),i=1,..n (Ri,ti),i=1,..n、1 Group right relative relative relative external parameters relative to left camera R ′ , t ′ R',t' R′,t′. The equation of minimum re projection error is the same as that of single camera calibration , Only the absolute external parameters of the right camera are calculated by the absolute external parameters and relative external parameters of the left camera .
2. Calculate the relative external parameters of each acquisition , And take the median value as the initial value , Based on minimizing the re projection error , Nonlinear iterative optimization obtains the optimal solution
Because the second iteration optimization , Camera internal parameters are fixed , The number of external parameters is also reduced, and there is a more accurate initial value , So iteration will be relatively easier to converge and more accurate .
summary
Stereo vision is the premise of calculating depth through stereo matching :
- To accurately know the position relationship between cameras , That is, the rotation of one camera to the other R R R Move peacefully t t t, namely The external parameters of the camera
- Know the equipment parameters inside the camera accurately , For example, like the main point ( x 0 , y 0 ) (x_0,y_0) (x0,y0)、 The focal length f f f、 Distortion parameters, etc , namely The internal parameters of the camera
Compared with single camera calibration, dual camera calibration :
- One more relative external parameter , That is, the rotation matrix of the right camera relative to the left camera R R R Vector of peaceful shift t t t.
- Two absolute external parameters are missing , That is, the external parameter matrix calibrated by the single camera itself is no longer needed , Just keep 1 The relative external parameters obtained , It is called the external parameter of stereo vision system .
The difference between dual camera image acquisition and single camera image acquisition lies in :
- Every time you shoot , Pay attention to whether the overlapping area of two cameras is enough , Generally more than 1/2 The image area of , Of course, the bigger the better , At some relatively large angles , A certain degree of overlap can be sacrificed .
- Note that the two cameras try to keep the same distance to collect , This is to ensure that the scale and clarity of the two views are consistent , Reduce the error caused by inconsistency .
Steps of dual camera calibration algorithm :
- Do single camera calibration for two cameras separately , Get the internal parameter matrix of each of the two cameras K、 Absolute external reference R , t R,t R,t, And distortion coefficient d d d
- Calculate the relative external parameters of each acquisition , And take the median value as the initial value , Based on minimizing the re projection error , Nonlinear iterative optimization obtains the optimal solution
So you
Homework
Here are some exercises for you , We can deepen our understanding through practice :
1 adopt opencv The interface provided by the open source library completes the calibration of dual cameras .
2 Higher order is , You can be independent opencv Library to write a set of dual camera calibration algorithm ? Or just use opencv To detect the corner coordinates , Other steps are implemented by yourself .
Please refer to the address :https://github.com/ethan-li-coding/StereoV3DCode [ Sorry to let it go , The code has not been updated for a long time , But do you believe that I will be more right one day ?]
边栏推荐
- [daily training -- Tencent selected 50] 557 Reverse word III in string
- [code practice] [stereo matching series] Classic ad census: (5) scan line optimization
- Multiple linear regression (sklearn method)
- 什么是防火墙?防火墙基础知识讲解
- Meta标签详解
- Huber Loss
- Basic number theory - fast power
- Latex improve
- TF coordinate transformation of common components of ros-9 ROS
- Use and programming method of ros-8 parameters
猜你喜欢
Halcon snap, get the area and position of coins
Programming implementation of ROS learning 5-client node
The combination of deep learning model and wet experiment is expected to be used for metabolic flux analysis
TF coordinate transformation of common components of ros-9 ROS
Guess riddles (5)
Confusing basic concepts member variables local variables global variables
Ros- learn basic knowledge of 0 ROS - nodes, running ROS nodes, topics, services, etc
RT-Thread内核快速入门,内核实现与应用开发学习随笔记
Guess riddles (8)
Guess riddles (10)
随机推荐
Hello everyone, welcome to my CSDN blog!
Digital analog 1: linear programming
Halcon Chinese character recognition
Yolov4 target detection backbone
ORACLE进阶(三)数据字典详解
C [essential skills] use of configurationmanager class (use of file app.config)
MPSoC QSPI Flash 升级办法
ROS learning 1- create workspaces and function packs
How many checks does kubedm series-01-preflight have
[牛客网刷题 Day4] JZ32 从上往下打印二叉树
Summary of "reversal" problem in challenge Programming Competition
Basic number theory - factors
Guess riddles (5)
我从技术到产品经理的几点体会
[daiy4] jz32 print binary tree from top to bottom
Guess riddles (3)
优先级队列(堆)
Add discount recharge and discount shadow ticket plug-ins to the resource realization applet
696. Count binary substring
Array,Date,String 对象方法