根據變換矩陣進行圖像拼接

根據變換矩陣進行圖像拼接

描述：在多相機組合拍攝時，相機的比特置相對固定。通常進行圖像拼接時，需要尋找兩副圖像間的特征點，計算出轉化矩陣，再進行圖像拼接。在尋找特征點和生成轉化矩陣這一過程中，會消耗較多的計算量，導致速度較慢。對於固定的相機組合來說，其轉化矩陣計算一次即可，後續直接讀取矩陣內容就行。

一、獲取轉化矩陣

直接貼代碼main.py

from stitcher import stitcher
import cv2
import time
# 讀取拼接圖片

t0 = time.time()
imageA = cv2.imread("left_01.png")
imageB = cv2.imread("right_01.png")

# 把圖片拼接成全景圖
stitcher = stitcher()
(result, vis) = stitcher.stitch([imageA, imageB], showMatches=True)
print('Use time is : ',time.time()-t0)

# 顯示所有圖片
cv2.imshow("Image A", imageA)
cv2.imshow("Image B", imageB)
cv2.imwrite('vis.jpg',vis)
cv2.imshow("Keypoint Matches", vis)
cv2.imwrite('result.jpg',result)
cv2.imshow("Result", result)
cv2.waitKey(0)
cv2.destroyAllWindows()

stitcher.py的代碼

import numpy as np
import cv2


class stitcher:

    def stitch(self, images, ratio=0.75, reprojThresh=4.0, showMatches=False):
        (imageA, imageB) = images

        # 找特征點
        (kpsA, npkpsA, feasA) = self.find_kps_feas(imageA)
        (kpsB, npkpsB, feasB) = self.find_kps_feas(imageB)

        M = self.matchKeypoints(npkpsA, npkpsB, feasA, feasB, ratio, reprojThresh)
        (good, H, status) = M
        np.savetxt("data.txt", H)
        result = cv2.warpPerspective(imageB, H, (imageA.shape[1] + imageB.shape[1], imageB.shape[0]))
        #self.cv_show('result1', result)
        result[0:imageA.shape[0], 0:imageA.shape[1]] = imageA
        #self.cv_show('result2', result)

        img = cv2.drawMatchesKnn(imageB, kpsB, imageA, kpsA, good, None, flags=2)
        #self.cv_show('result', img)

        return result, img

    def find_kps_feas(self, image):
        # 建立SIFT生成器
        descriptor = cv2.xfeatures2d.SIFT_create()
        # 檢測SIFT特征點
        (kps, features) = descriptor.detectAndCompute(image, None)

        # 將結果轉換成NumPy數組
        npkps = np.float32([kp.pt for kp in kps])

        # 返回特征點集，及對應的描述特征
        return (kps, npkps, features)

    def matchKeypoints(self, kpsA, kpsB, featuresA, featuresB, ratio, reprojThresh):
        bf = cv2.BFMatcher()
        allMatches = bf.knnMatch(featuresB, featuresA, k=2)
        matches = []
        good = []
        for m, n in allMatches:
            if m.distance < ratio * n.distance:
                matches.append((m.trainIdx, m.queryIdx))
                good.append([m])

        if len(matches) > 4:
            ptsA = np.float32([kpsA[i] for (i, _) in matches])
            ptsB = np.float32([kpsB[i] for (_, i) in matches])

            (H, status) = cv2.findHomography(ptsB, ptsA, cv2.RANSAC, reprojThresh)

        return (good, H, status)

    def cv_show(self, name, img):
        cv2.imshow(name, img)
        cv2.waitKey(0)
        cv2.destroyAllWindows()

在上述代碼中，主要將特征矩陣保存為data.txt。方便後續的數據讀取。經過測試，上述代碼用時大概70ms。

二、直接讀取轉化矩陣進行圖像拼接

1.python版本
代碼如下：

import cv2
import time
import numpy as np
# 讀取拼接圖片

t0 = time.time()
imageA = cv2.imread("left_01.png")
imageB = cv2.imread("right_01.png")


A = np.zeros((3, 3), dtype=float)

f = open('/home/cj/work/python/proc_image/concatimg/2/data.txt')  # 打開數據文件文件
lines = f.readlines()  # 把全部數據文件讀到一個列錶lines中
A_row = 0  # 錶示矩陣的行，從0行開始
for line in lines:  # 把lines中的數據逐行讀取出來
    list = line.strip('\n').split(' ')  # 處理逐行數據：strip錶示把頭尾的'\n'去掉，split錶示以空格來分割行數據，然後把處理後的行數據返回到list列錶中
    A[A_row:] = list[0:3]  # 把處理後的數據放到方陣A中。list[0:4]錶示列錶的0,1,2,3列數據放到矩陣A中的A_row行
    A_row += 1  # 然後方陣A的下一行接著讀

result = cv2.warpPerspective(imageB, A, (imageA.shape[1] + imageB.shape[1], imageB.shape[0]))
result[0:imageA.shape[0], 0:imageA.shape[1]] = imageA
print("Use calculated matrix operation me is : ",time.time()-t0)
cv2.imwrite('myresult.jpg',result)

直接讀取矩陣進行圖像拼接，大概用時11ms，相對來說快了很多了！
2.C++版本
代碼如下

#include <iostream>
#include <opencv2/opencv.hpp>

#include <stdio.h>
#include<time.h>

using namespace cv;

using namespace std;

int main()
{
    
    clock_t start_time,stop_time;//增加時間起始
    start_time = clock();
    //讀取變換矩陣
    float A[3][3];
    //Mat A;
    FILE* fpread;
        fpread = fopen("/home/cj/work/code/C++/concatimg/2/untitled/data.txt", "r");
        if (fpread == NULL)
        {
    
            printf("file is error.");
            return -1;
        }
        for (int i = 0; i < 3; i++)
        {
    
            for (int j = 0; j < 3; j++)
            {
    
                fscanf(fpread, "%e", &A[i][j]);
            }
        }
        fclose(fpread);
        for (int i = 0; i < 3; i++)
            {
    
                for (int j = 0; j < 3; j++)
                {
    
                    printf("%e\t",A[i][j]);
                }
                printf("\n");
             }


    Mat imageA=imread("/home/cj/work/code/C++/concatimg/2/untitled/left_01.png");
    Mat imageB=imread("/home/cj/work/code/C++/concatimg/2/untitled/right_01.png");
    int W_imageA = imageA.cols;
    int H_imageA = imageA.rows;
    int W_imageB = imageB.cols;
    int H_imageB = imageB.rows;
    int new_w = W_imageA+W_imageB;

    imshow("img",imageA);
    Mat H(3,3,CV_32FC1,A);//轉化為Mat矩陣
    cout<<"H is :"<<H;


    //dst = cv::warpPerspective(imageB, A, dsize[, dst[, flags[, borderMode[, borderValue]]]])
    Mat dst;
    //cv::warpPerspective(imageB,dst,H,imageB.size());
    cv::warpPerspective(imageB,dst,H,cv::Size(new_w,H_imageB));//透視變換
    cout<<"Size of imageB is :\n"<<imageB.size();
    cv::imwrite("dst.jpg",dst);

    cv::Rect roi_rect = cv::Rect(0, 0, imageA.cols, imageA.rows);

    imageA.copyTo(dst(roi_rect));
    stop_time = clock();
    printf("Use Time : %f s\n",(double)(stop_time-start_time)/CLOCKS_PER_SEC);
    cv::imwrite("dst_result.jpg",dst);


    waitKey(2000);

    destroyAllWindows();
}

C++運行終端顯示內容為：

效果