当前位置：网站首页>Dlib detects blink times based on video stream

Dlib detects blink times based on video stream

2022-07-06 15:02:00 【gmHappy】


Eye aspect ratio (EAR)

In the discussion EAR Before , Have a look first 68 Personal face feature points ： 

dlib Detect blink times based on camera stream _ Blink times

The algorithm of facial feature point detection itself is very complex ,dlib The related implementation is given in .

Each eye consists of 6 individual （x,y） Coordinate representation , Start at the left corner of the eye , Then it displays clockwise around the rest of the area ：

dlib Detect blink times based on camera stream _ide_02

Based on this description , We should get to the point ： Of these coordinates Width and Height There is a relationship between them .

Soukupová and Čech In its 2016 Year paper “​ ​ Real time eye blink detection using facial signs ” The job of ​​, We can deduce the equation reflecting this relationship , be called Eye aspect ratio （EAR）：

dlib Detect blink times based on camera stream _ide_03

among p1,...,p6 yes 2D Location of facial landmarks .

The molecule of this equation is to calculate the distance between vertical eye marks , And the denominator is to calculate the distance between horizontal eye marks , Because only One Group horizontal point , however Yes Two sets of vertical points , So the weighted denominator .

Why is this equation so interesting ？

We will find out , The aspect ratio of the eyes is roughly constant when the eyes are open , But in the blink of an eye, it will quickly fall to zero .

Use this simple equation , We It can avoid using image processing technology , Simply rely on Eye landmark distance leave Of The proportion To determine whether a person blinks .

To make it clear , Look at the picture below ：

dlib Detect blink times based on camera stream _dlib_04

stay Bottom In the figure, the curve of eye aspect ratio over time is drawn . As we can see , The aspect ratio of the eyes is constant , Then it drops rapidly to near zero , And then add , Indicates that a single blink has occurred .

Code implementation 

       

        # -*- coding: utf-8 -*-
        
# import the necessary packages
        
from scipy.spatial import distance as dist
        
from imutils.video import FileVideoStream
        
from imutils.video import VideoStream
        
from imutils import face_utils
        
import numpy as np
        
import argparse
        
import imutils
        
import time
        
import dlib
        
import cv2
        

        

        
def eye_aspect_ratio(eye):
        
    #  Calculate the Euclidean distance between two sets 
        
    #  Vertical eye sign （X,Y） coordinate 
        
    A = dist.euclidean(eye[1], eye[5])
        
    B = dist.euclidean(eye[2], eye[4])
        
    #  Calculate the Euclidean distance between levels 
        
    #  Horizontal eye mark （X,Y） coordinate 
        
    C = dist.euclidean(eye[0], eye[3])
        
    #  Calculation of eye aspect ratio 
        
    ear = (A + B) / (2.0 * C)
        
    #  Return to the aspect ratio of glasses 
        
    return ear
        

        

        
#  Define two constants 
        
#  Eye aspect ratio 
        
#  Flicker threshold 
        
EYE_AR_THRESH = 0.2
        
EYE_AR_CONSEC_FRAMES = 3
        
#  Initialize the frame counter and total blinks 
        
COUNTER = 0
        
TOTAL = 0
        

        
#  initialization DLIB Face detector （HOG）, Then create a facial marker prediction 
        
print("[INFO] loading facial landmark predictor...")
        
detector = dlib.get_frontal_face_detector()
        
predictor = dlib.shape_predictor('shape_predictor_68_face_landmarks.dat')
        

        
#  Get the indexes of left and right eye facial signs respectively 
        
(lStart, lEnd) = face_utils.FACIAL_LANDMARKS_IDXS["left_eye"]
        
(rStart, rEnd) = face_utils.FACIAL_LANDMARKS_IDXS["right_eye"]
        

        
cap = cv2.VideoCapture(1)
        

        
#  Loop frames from video stream 
        
while True:
        
    ret, frame = cap.read()
        

        
    gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
        
    #  Detect faces in gray frames 
        
    rects = detector(gray, 0)
        

        
    #  Face detection cycle 
        
    for rect in rects:
        
        #  Determine the facial signs of the face area , Then put the facial logo （x,y） Coordinate conversion to digital array 
        
        shape = predictor(gray, rect)
        
        shape = face_utils.shape_to_np(shape)
        
        #  Extract left eye and right eye coordinates , Then use coordinates to calculate the eye aspect ratio of both eyes 
        
        leftEye = shape[lStart:lEnd]
        
        rightEye = shape[rStart:rEnd]
        
        leftEAR = eye_aspect_ratio(leftEye)
        
        rightEAR = eye_aspect_ratio(rightEye)
        
        #  Average aspect ratio of both eyes 
        
        ear = (leftEAR + rightEAR) / 2.0
        

        
        #  Calculate the marker points of the left eye and the right eye and draw 
        
        leftEyeHull = cv2.convexHull(leftEye)
        
        rightEyeHull = cv2.convexHull(rightEye)
        
        cv2.drawContours(frame, [leftEyeHull], -1, (0, 255, 0), 1)
        
        cv2.drawContours(frame, [rightEyeHull], -1, (0, 255, 0), 1)
        

        
        #  Mark the face in the picture , And display 
        
        left = rect.left()
        
        top = rect.top()
        
        right = rect.right()
        
        bottom = rect.bottom()
        
        cv2.rectangle(frame, (left, top), (right, bottom), (0, 255, 0), 3)    
        

        
        '''
        
                         The first step is to check whether the aspect ratio of the eye is lower than our blink threshold , If it is , We increment the number of consecutive frames indicating that blinking is occurring . otherwise , We will deal with cases where the aspect ratio of the eye is not lower than the blink threshold , We check it ,
        
                         See if a sufficient number of consecutive frames contain blink rates below our predefined threshold . If the inspection passes , We increase the total number of flashes . Then we reset the number of consecutive flashes  COUNTER.
        
        '''
        
        if ear < EYE_AR_THRESH:
        
            COUNTER += 1
        
        else:
        
            #  If the eyes are closed enough , Then the total number of blinks increases 
        
            if COUNTER >= EYE_AR_CONSEC_FRAMES:
        
                TOTAL += 1
        
            #  Reset eye frame counter 
        
            COUNTER = 0
        

        
        for (x, y) in shape:
        
            cv2.circle(frame, (x, y), 1, (0, 0, 255), -1)
        

        
        cv2.putText(frame, "COUNTER: {}".format(COUNTER), (150, 30),
        
        cv2.FONT_HERSHEY_SIMPLEX, 0.7, (0, 0, 255), 2)    
        
        cv2.putText(frame, "Blinks: {}".format(TOTAL), (10, 30),
        
        cv2.FONT_HERSHEY_SIMPLEX, 0.7, (0, 0, 255), 2)
        
        cv2.putText(frame, "EAR: {:.2f}".format(ear), (300, 30),
        
        cv2.FONT_HERSHEY_SIMPLEX, 0.7, (0, 0, 255), 2)
        

        
    cv2.imshow("Frame", frame)
        
    print(len(rects))   
        
    # if the `q` key was pressed, break from the loop
        
    if cv2.waitKey(1) & 0xFF == ord('q'):
        
        break
        

        
# do a bit of cleanup
        
cv2.destroyAllWindows()
       

       
    
        
  • 1.
    • 
        
  • 2.
    • 
        
  • 3.
    • 
        
  • 4.
    • 
        
  • 5.
    • 
        
  • 6.
    • 
        
  • 7.
    • 
        
  • 8.
    • 
        
  • 9.
    • 
        
  • 10.
    • 
        
  • 11.
    • 
        
  • 12.
    • 
        
  • 13.
    • 
        
  • 14.
    • 
        
  • 15.
    • 
        
  • 16.
    • 
        
  • 17.
    • 
        
  • 18.
    • 
        
  • 19.
    • 
        
  • 20.
    • 
        
  • 21.
    • 
        
  • 22.
    • 
        
  • 23.
    • 
        
  • 24.
    • 
        
  • 25.
    • 
        
  • 26.
    • 
        
  • 27.
    • 
        
  • 28.
    • 
        
  • 29.
    • 
        
  • 30.
    • 
        
  • 31.
    • 
        
  • 32.
    • 
        
  • 33.
    • 
        
  • 34.
    • 
        
  • 35.
    • 
        
  • 36.
    • 
        
  • 37.
    • 
        
  • 38.
    • 
        
  • 39.
    • 
        
  • 40.
    • 
        
  • 41.
    • 
        
  • 42.
    • 
        
  • 43.
    • 
        
  • 44.
    • 
        
  • 45.
    • 
        
  • 46.
    • 
        
  • 47.
    • 
        
  • 48.
    • 
        
  • 49.
    • 
        
  • 50.
    • 
        
  • 51.
    • 
        
  • 52.
    • 
        
  • 53.
    • 
        
  • 54.
    • 
        
  • 55.
    • 
        
  • 56.
    • 
        
  • 57.
    • 
        
  • 58.
    • 
        
  • 59.
    • 
        
  • 60.
    • 
        
  • 61.
    • 
        
  • 62.
    • 
        
  • 63.
    • 
        
  • 64.
    • 
        
  • 65.
    • 
        
  • 66.
    • 
        
  • 67.
    • 
        
  • 68.
    • 
        
  • 69.
    • 
        
  • 70.
    • 
        
  • 71.
    • 
        
  • 72.
    • 
        
  • 73.
    • 
        
  • 74.
    • 
        
  • 75.
    • 
        
  • 76.
    • 
        
  • 77.
    • 
        
  • 78.
    • 
        
  • 79.
    • 
        
  • 80.
    • 
        
  • 81.
    • 
        
  • 82.
    • 
        
  • 83.
    • 
        
  • 84.
    • 
        
  • 85.
    • 
        
  • 86.
    • 
        
  • 87.
    • 
        
  • 88.
    • 
        
  • 89.
    • 
        
  • 90.
    • 
        
  • 91.
    • 
        
  • 92.
    • 
        
  • 93.
    • 
        
  • 94.
    • 
        
  • 95.
    • 
        
  • 96.
    • 
        
  • 97.
    • 
        
  • 98.
    • 
        
  • 99.
    • 
        
  • 100.
    • 
        
  • 101.
    • 
        
  • 102.
    • 
        
  • 103.
    • 
        
  • 104.
    • 
        
  • 105.
    • 
        
  • 106.
    • 
        
  • 107.
    • 
        
  • 108.
    • 
        
  • 109.
    • 
        
  • 110.
    • 
        
  • 111.
    • 
        
  • 112.
    • 
        
  • 113.
    •

effect

 dlib Detect blink times based on camera stream _ide_05


原网站

版权声明
本文为[gmHappy]所创，转载请带上原文链接，感谢
https://yzsam.com/2022/02/202202131326172763.html

边栏推荐

猜你喜欢

随机推荐