当前位置：网站首页>Palm detection and finger counting based on OpenCV

Palm detection and finger counting based on OpenCV

2022-06-10 16:31:00 【AI technology base camp】

author | Nuby

source | Xiaobai studies vision

Use the cosine theorem to use OpenCV-Python Realize finger counting and palm detection .

Hand detection and finger counting

Next, let's explore how the following functions are implemented .

OpenCV

OpenCV（ Open source computer vision library ） Is an open source computer vision and machine learning software library .OpenCV Is built to provide a common infrastructure for computer vision applications , And accelerate the use of machine awareness in commercial products .

Import library

• cv2： opencv [pip install opencv]

• numpy： Used to handle arrays and math [pip install numpy]

import cv2 as cv
import numpy as np

Import images

img_path = "data/palm.jpg"
img = cv.imread(img_path)
cv.imshow('palm image',img)

Palm image

The skin Mask

• Used to highlight a specific color on an image .

• hsvim： take BGR（ Blue , green , Red ） The image is changed to HSV（ Hue , saturation , value ）.

• The lower ：HSV The range of skin color in the skin is small .

• upper：HSV The upper limit of skin color .

• skinRegionHSV： stay HSV The skin is detected within the upper and lower pixel values of the color space .

• Fuzzy ： Blur the image to improve the mask .

• threshing ： threshing .

hsvim = cv.cvtColor(img, cv.COLOR_BGR2HSV)
lower = np.array([0, 48, 80], dtype = "uint8")
upper = np.array([20, 255, 255], dtype = "uint8")
skinRegionHSV = cv.inRange(hsvim, lower, upper)
blurred = cv.blur(skinRegionHSV, (2,2))
ret,thresh = cv.threshold(blurred,0,255,cv.THRESH_BINARY)
cv.imshow("thresh", thresh)

Processing results

Contour drawing

Now let's find the outline on the image .

contours, hierarchy = cv.findContours(thresh, cv.RETR_TREE, cv.CHAIN_APPROX_SIMPLE)
contours = max(contours, key=lambda x: cv.contourArea(x))
cv.drawContours(img, [contours], -1, (255,255,0), 2)
cv.imshow("contours", img)

Palm contour

Convex hull detection

hull = cv.convexHull(contours)
cv.drawContours(img, [hull], -1, (0, 255, 255), 2)
cv.imshow("hull", img)

detection result

Convex defect detection

Any deviation between the palm and the convex hull detection contour line can be regarded as a convex defect .

hull = cv.convexHull(contours, returnPoints=False)
defects = cv.convexityDefects(contours, hull)

Examples of convex defects

Count fingers

Convex defect returns an array , Each of these lines contains the following values ：

• The starting point

• End

• Farthest point

• The approximate distance to the farthest point

Through this , We can easily conclude that Sides：a,b,c（ Please see the CODE）, And according to the cosine theorem , We can also get gamma or angle between two fingers . As mentioned earlier , If gamma is less than 90 degree , We'll think of it as a finger . After knowing gamma , We just need to draw a radius of 4 The circle of , The approximate distance to the farthest point . After simply putting the text into the image , We mean the number of fingers （cnt）.

if defects is not None:
cnt = 0
for i in range(defects.shape[0]): # calculate the angle
s, e, f, d = defects[i][0]
start = tuple(contours[s][0])
end = tuple(contours[e][0])
far = tuple(contours[f][0])
a = np.sqrt((end[0] - start[0]) ** 2 + (end[1] - start[1]) ** 2)
b = np.sqrt((far[0] - start[0]) ** 2 + (far[1] - start[1]) ** 2)
c = np.sqrt((end[0] - far[0]) ** 2 + (end[1] - far[1]) ** 2)
angle = np.arccos((b ** 2 + c ** 2 - a ** 2) / (2 * b * c)) # cosine theorem
if angle <= np.pi / 2: # angle less than 90 degree, treat as fingers
cnt += 1
cv.circle(img, far, 4, [0, 0, 255], -1)
if cnt > 0:
cnt = cnt+1
cv.putText(img, str(cnt), (0, 50), cv.FONT_HERSHEY_SIMPLEX,1, (255, 0, 0) , 2, cv.LINE_AA)

Let's look at the final result