[300 opencv routines] 240. Shi Tomas corner detection in opencv

『youcans Of OpenCV routine 300 piece - General catalogue 』

【youcans Of OpenCV routine 300 piece 】240. OpenCV Medium Shi-Tomas Corner detection

An angle is a rapid change in the direction of a straight line . A corner is usually defined as the intersection of two edges , In other words, the neighborhood of the corner should have the boundaries of two different regions in different directions .

Angle is a highly effective feature . Corner detection （Corner Detection） It is widely used in motion detection 、 Image matching 、 Video tracking 、 3D reconstruction and target recognition .

Shi-Tomas The algorithm is right Harris Improvement of corner detection algorithm , General comparison Harris The algorithm gets better corners .

6.1 Harris - Stephens angle detector （Harris）

Harris The principle of corner detection algorithm is , Move on the image through the detection window , Calculate the gray changes of pixels in the window before and after moving . A corner is the intersection of two sides , Its feature is that moving the detection window in any direction will lead to significant changes in gray .

For point $(x,y)$, Make $w(x,y)$ Rectangular detection window or Gaussian detection window ,$I(x,y)$ Is the detection window at point $(x,y)$ Gray value ,$I(x+u,y+v)$ Is the detection window sliding $(u,v)$ Gray value after distance .

Calculation window $w(x,y)$ slide $(u,v)$ Gray change after distance ：
$$E(u,v)=\sum _{x,y}w(x,y)[I(x+u,y+v)-I(x,y){]}^2$$
For very small displacements $(u,v)$, Use first-order Taylor expansion for simplification ：
$$\begin{array}{rl}E(u,v)=& \sum _{x,y}w(x,y)[I(x,y)+u{I}_x+v{I}_y+O(u^2,v^2)-I(x,y){]}^2\\ \approx & \sum _{x,y}w(x,y)[u^2{I}_x^2+2uv{I}_x{I}_y+v^2{I}_y^2]\\ \approx & [u,v]\sum _{x,y}w(x,y)\left[\begin{array}{cc}{I}_x{I}_x& I_x{I}_y\\ I_x{I}_y& I_y{I}_y\end{array}\right]\left[\begin{array}{c}u\\ v\end{array}\right]\end{array}$$
$I_x,I_y$ Respectively in x, y Derivative of direction , Can be Sobel Gradient operator finds .

remember M Is the covariance matrix of the gradient ：
$$M=\sum _{x,y}w(x,y)\left[\begin{array}{cc}{I}_x{I}_x& I_x{I}_y\\ I_x{I}_y& I_y{I}_y\end{array}\right]$$

By judging the size of two eigenvalues in the geometric model , To determine the attributes of pixels . matrix M yes $I_x,I_y$ The quadratic function of , It can be expressed as an elliptical shape , The major and minor semiaxes of an ellipse are determined by the matrix M The eigenvalues of the ${\lambda }_1,{\lambda }_2$ decision , The direction is determined by the eigenvector .

Define the corner response function R：
$$\begin{array}{rl}R=& det(M)-k[trace(M){]}^2\\ & det(M)={\lambda }_1\ast {\lambda }_2\\ & trace(M)={\lambda }_1+{\lambda }_2\end{array}$$
k Is the adjustment parameter （ Usually take 0.04~0.06）.

Corner response function R Only with matrix M The eigenvalues of the ${\lambda }_1,{\lambda }_2$ of , It can be used to judge whether the area is a corner 、 The edge is still flat ：

• ${\lambda }_1,{\lambda }_2$ More hours , $|R|$ smaller , That is, the gray level in all directions is basically unchanged , It indicates that the detector is in a flat area ;
• ${\lambda }_1>>{\lambda }_2$ or $ \lambda _2 >> \lambda _1$ when , $R<0$ , That is, the gray changes in a certain direction , But it does not change in its orthogonal direction , It indicates that the detector is in the edge area ;
• ${\lambda }_1,{\lambda }_2$ Large and ${\lambda }_1\sim {\lambda }_2$ when , $|R|$ It's big , That is, the grayscale changes significantly in all directions , Indicates that the detector contains corners （ Or isolated point .

6.3 OpenCV Medium Shi-Tomas Angle detector

Shi-Tomas The algorithm is right Harris Improvement of corner detection algorithm , General comparison Harris The algorithm gets better corners .

Harris The corner response function in the algorithm is ：
$$R={\lambda }_1{\lambda }_2-k({\lambda }_1+{\lambda }_2{)}^2$$
Shi-Tomas The algorithm modifies the corner response function to ：
$$R=min({\lambda }_1,{\lambda }_2)$$

Only when M The eigenvalues of a matrix $ \lambda _1,\lambda _2$ Are greater than the minimum $\lambda min$ when , Just think of it as a corner .

OpenCV Provided in Shi-Tomas Corner detection function cv.goodFeaturesToTrack() .

Function description ：

cv.goodFeaturesToTrack(image, maxCorners, qualityLevel, minDistance[, corners=None, mask=Mat(), blockSize=3, useHarrisDetector=false, k=0.04]) → corners

function cv.goodFeaturesToTrack adopt Shi-Tomasi Methods to find the most prominent in the image N Corner points .

Parameter description ：

• src： The input image , Single channel 8 Bit image or 32 Bit floating point number image
• corners： Output vector , Corner detected
• maxCorners： The upper threshold of the number of corners obtained
• qualityLevel： Minimum threshold of acceptable corner quality , Value range 0～1
• minDistance： The minimum Euclidean distance between acceptable corners
• mask： Mask area , Detect corners only in the mask area
• blockSize： neighborhood size , The default value is 3
• k：Harris Detector adjustment parameters , The default value is 0.04
• useHarrisDetector ： The default value is False, Use cornerMinEigenVal Parameters of ,True Said the use of Harris detector

matters needing attention ：

1. Function USES cornerMinEigenVal or cornerHarris Calculate the corner response result of each pixel .
2. With qualityLevel Multiply with the maximum corner value as the minimum eigenvalue （cornerMinEigenVal） Or response function value （cornerHarris）, Corners smaller than this value are rejected . for example , The maximum response value is 1500, The coefficient is 0.1, Then the response value is less than 150 The corners of are abandoned .
3. Less than maxDistance The corner of the distance is rejected , To avoid getting adjacent feature points .

routine 14.21： Feature detection Shi-tomas Angle detector

    # 14.21  Feature detection  Shi-tomas  Angle detector 
    img = cv2.imread("../images/sign04.png", flags=1)  # (300, 300, 3)
    gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)

    # Harris  Check corner 
    dst = cv2.cornerHarris(gray, 2, 3, k=0.04)
    # Harris[dst > 0.01*dst.max()] = [0, 0, 255] #  Filter corners , The red mark 
    corners = np.column_stack(np.where(dst>0.1*dst.max()))  #  Filter and return corner coordinates  (y,x)
    corners = corners.astype(np.int)  #  Point set of detected corners  (y,x), (92, 2)
    imgHarris = np.copy(img)
    for point in corners:  #  Pay attention to the coordinate order 
        cv2.circle(imgHarris, (point[1], point, 4, (0,0,255), 2)  # #  At point  (x,y)  Draw a circle at 

    # Shi-tomas  Check corner 
    corners = cv2.goodFeaturesToTrack(gray, 30, 0.3, 5)  # (30, 1, 2)
    corners = np.squeeze(corners).astype(np.int)  # (30, 1, 2)->(30,2)  Corner coordinates  (x,y)
    imgShiTomas = np.copy(img)
    for point in corners:
        cv2.circle(imgShiTomas, point, 4, (0,0,255), 2)  # #  At point  (x,y)  Draw a circle at 

    plt.figure(figsize=(9, 6))
    plt.subplot(131), plt.axis('off'), plt.title("Origin")
    plt.imshow(cv2.cvtColor(img, cv2.COLOR_BGR2RGB))
    plt.subplot(132), plt.axis('off'), plt.title("Harris corners")
    plt.imshow(cv2.cvtColor(imgHarris, cv2.COLOR_BGR2RGB))
    plt.subplot(133), plt.axis('off'), plt.title("Shi-tomas corners")
    plt.imshow(cv2.cvtColor(imgShiTomas, cv2.COLOR_BGR2RGB))
    plt.tight_layout()
    plt.show()

【 At the end of this section 】

Copyright notice ：
[email protected] Original works , Reprint must be marked with the original link ：(https://blog.csdn.net/youcans/article/details/125828053)
Copyright 2022 youcans, XUPT
Crated：2022-7-15

238. OpenCV Medium Harris Corner detection
239. Harris Accurate positioning of corner detection （cornerSubPix）
240. OpenCV Medium Shi-Tomas Corner detection