【双目视觉】双目矫正

一、双目标定

双目标定需要获取到两个相机的内参以及变换矩阵。可参照链接：
https://blog.csdn.net/qq_38236355/article/details/89280633
https://blog.csdn.net/qingfengxiaosong/article/details/109897053
或者自行百度
建议使用Matlab工具箱做标定，其中建议勾选3 Coefficients。
输出Matlab的数据之后，可用一下脚本提取数据：

rowName = cell(1,10);
rowName{1,1} = '平移矩阵';
rowName{1,2} = '旋转矩阵';
rowName{1,3} = '相机1内参矩阵';
rowName{1,4} = '相机1径向畸变';
rowName{1,5} = '相机1切向畸变';
rowName{1,6} = '相机2内参矩阵';
rowName{1,7} = '相机2径向畸变';
rowName{1,8} = '相机2切向畸变';
rowName{1,9} = '相机1畸变向量';
rowName{1,10} = '相机2畸变向量';
xlswrite('out.xlsx',rowName(1,1),1,'A1');
xlswrite('out.xlsx',rowName(1,2),1,'A2');
xlswrite('out.xlsx',rowName(1,3),1,'A5');
xlswrite('out.xlsx',rowName(1,4),1,'A8');
xlswrite('out.xlsx',rowName(1,5),1,'A9');
xlswrite('out.xlsx',rowName(1,6),1,'A10');
xlswrite('out.xlsx',rowName(1,7),1,'A13');
xlswrite('out.xlsx',rowName(1,8),1,'A14');
xlswrite('out.xlsx',rowName(1,9),1,'A15');
xlswrite('out.xlsx',rowName(1,10),1,'A16');
xlswrite('out.xlsx',stereoParams.TranslationOfCamera2,1,'B1');  % 平移矩阵
xlswrite('out.xlsx',stereoParams.RotationOfCamera2.',1,'B2');  % 旋转矩阵
xlswrite('out.xlsx',stereoParams.CameraParameters1.IntrinsicMatrix.',1,'B5');  % 相机1内参矩阵
xlswrite('out.xlsx',stereoParams.CameraParameters1.RadialDistortion,1,'B8');  % 相机1径向畸变(1,2,5)
xlswrite('out.xlsx',stereoParams.CameraParameters1.TangentialDistortion,1,'B9');  % 相机1切向畸变(3,4)
xlswrite('out.xlsx',stereoParams.CameraParameters2.IntrinsicMatrix.',1,'B10');  % 相机2内参矩阵
xlswrite('out.xlsx',stereoParams.CameraParameters2.RadialDistortion,1,'B13');  % 相机2径向畸变(1,2,5)
xlswrite('out.xlsx',stereoParams.CameraParameters2.TangentialDistortion,1,'B14');  % 相机2切向畸变(3,4)
xlswrite('out.xlsx',[stereoParams.CameraParameters1.RadialDistortion(1:2), stereoParams.CameraParameters1.TangentialDistortion,...
    stereoParams.CameraParameters1.RadialDistortion(3)],1,'B15');  % 相机1畸变向量
xlswrite('out.xlsx',[stereoParams.CameraParameters2.RadialDistortion(1:2), stereoParams.CameraParameters2.TangentialDistortion,...
    stereoParams.CameraParameters2.RadialDistortion(3)],1,'B16');  % 相机2畸变向量

Python双目矫正

新建一个python脚本，输入以下代码：

import cv2
import numpy as np
# 左目内参
left_camera_matrix = np.array([[443.305413261701, 0., 473.481578105186],
                               [0., 445.685585080218, 481.627083907456],
                               [0., 0., 1.]])
#左目畸变
#k1 k2 p1 p2 k3
left_distortion = np.array([[-0.261575534517449, 0.0622298171820726, 0., 0., -0.00638628534161724]])
# 右目内参
right_camera_matrix = np.array([[441.452616156177,0., 484.276702473006],
                                [0., 444.350924943458, 465.054536507021],
                                [0., 0., 1.]])
# 右目畸变
right_distortion = np.array([[-0.257761221642368, 0.0592089672793365, 0., 0., -0.00576090991058531]])
# 旋转矩阵
R = np.matrix([
    [0.999837210893742, -0.00477934325693493, 0.017398551383822],
    [0.00490062605211919, 0.999963944810228, -0.0069349076319899],
    [-0.0173647797717217, 0.00701904249875521, 0.999824583347439]
])
# 平移矩阵
T = np.array([-71.0439056359403, -0.474467959947789, -0.27989811881883]) # 平移关系向量

size = (960, 960) # 图像尺寸

# 进行立体更正
R1, R2, P1, P2, Q, validPixROI1, validPixROI2 = cv2.stereoRectify(left_camera_matrix, left_distortion,
                                                                  right_camera_matrix, right_distortion, size, R,
                                                                  T)
# 计算更正map
left_map1, left_map2 = cv2.initUndistortRectifyMap(left_camera_matrix, left_distortion, R1, P1, size, cv2.CV_16SC2)
right_map1, right_map2 = cv2.initUndistortRectifyMap(right_camera_matrix, right_distortion, R2, P2, size, cv2.CV_16SC2)

参数需要换成自己实际的参数。
接下来随便写一个脚本测试一下更正结果：

import cv2
import numpy as np
import camera_config

w = 1920
h = 960
cap = cv2.VideoCapture(0)
cap.set(cv2.CAP_PROP_FRAME_WIDTH, w)
cap.set(cv2.CAP_PROP_FRAME_HEIGHT, h)

key = ""
ww = int(w/2)

jiange = int(h/10)

while key!=27:
    ret, img = cap.read()
    if ret:
        imgLeft = img[:, :ww]
        imgRight = img[:, ww:w]

        left_remap = cv2.remap(imgLeft, camera_config.left_map1, camera_config.left_map2, cv2.INTER_LINEAR)
        right_remap = cv2.remap(imgRight, camera_config.right_map1, camera_config.right_map2, cv2.INTER_LINEAR)

        out = np.hstack([left_remap, right_remap])
        for i in range(10):
            cv2.line(out, (0, jiange*i), (w, jiange*i), (255, 0, 0), 2)

        cv2.imshow("frame", out)
        key = cv2.waitKey(10)
    
cap.release()
cv2.destroyAllWindows()

即可看到效果：
校正前（很差的相机，鱼眼效果，不适合用于实际使用）：

校正后：