当前位置：网站首页>Dlib detects blink times based on video stream

Dlib detects blink times based on video stream

2022-07-06 15:02:00 【gmHappy】

Eye aspect ratio (EAR)

In the discussion EAR Before , Have a look first 68 Personal face feature points ：

dlib Detect blink times based on camera stream _ Blink times

The algorithm of facial feature point detection itself is very complex ,dlib The related implementation is given in .

Each eye consists of 6 individual （x,y） Coordinate representation , Start at the left corner of the eye , Then it displays clockwise around the rest of the area ：

dlib Detect blink times based on camera stream _ide_02

Based on this description , We should get to the point ： Of these coordinates Width and Height There is a relationship between them .

Soukupová and Čech In its 2016 Year paper “ Real time eye blink detection using facial signs ” The job of , We can deduce the equation reflecting this relationship , be called Eye aspect ratio （EAR）：

dlib Detect blink times based on camera stream _ide_03

among p1,...,p6 yes 2D Location of facial landmarks .

The molecule of this equation is to calculate the distance between vertical eye marks , And the denominator is to calculate the distance between horizontal eye marks , Because only One Group horizontal point , however Yes Two sets of vertical points , So the weighted denominator .

Why is this equation so interesting ？

We will find out , The aspect ratio of the eyes is roughly constant when the eyes are open , But in the blink of an eye, it will quickly fall to zero .

Use this simple equation , We It can avoid using image processing technology , Simply rely on Eye landmark distance leave Of The proportion To determine whether a person blinks .

To make it clear , Look at the picture below ：

dlib Detect blink times based on camera stream _dlib_04

stay Bottom In the figure, the curve of eye aspect ratio over time is drawn . As we can see , The aspect ratio of the eyes is constant , Then it drops rapidly to near zero , And then add , Indicates that a single blink has occurred .

Code implementation

       
        # -*- coding: utf-8 -*-
        
# import the necessary packages
        
from scipy.spatial import distance as dist
        
from imutils.video import FileVideoStream
        
from imutils.video import VideoStream
        
from imutils import face_utils
        
import numpy as np
        
import argparse
        
import imutils
        
import time
        
import dlib
        
import cv2
        

        

        
def eye_aspect_ratio(eye):
        
    #  Calculate the Euclidean distance between two sets 
        
    #  Vertical eye sign （X,Y） coordinate 
        
    A = dist.euclidean(eye[1], eye[5])
        
    B = dist.euclidean(eye[2], eye[4])
        
    #  Calculate the Euclidean distance between levels 
        
    #  Horizontal eye mark （X,Y） coordinate 
        
    C = dist.euclidean(eye[0], eye[3])
        
    #  Calculation of eye aspect ratio 
        
    ear = (A + B) / (2.0 * C)
        
    #  Return to the aspect ratio of glasses 
        
    return ear
        

        

        
#  Define two constants 
        
#  Eye aspect ratio 
        
#  Flicker threshold 
        
EYE_AR_THRESH = 0.2
        
EYE_AR_CONSEC_FRAMES = 3
        
#  Initialize the frame counter and total blinks 
        
COUNTER = 0
        
TOTAL = 0
        

        
#  initialization DLIB Face detector （HOG）, Then create a facial marker prediction 
        
print("[INFO] loading facial landmark predictor...")
        
detector = dlib.get_frontal_face_detector()
        
predictor = dlib.shape_predictor('shape_predictor_68_face_landmarks.dat')
        

        
#  Get the indexes of left and right eye facial signs respectively 
        
(lStart, lEnd) = face_utils.FACIAL_LANDMARKS_IDXS["left_eye"]
        
(rStart, rEnd) = face_utils.FACIAL_LANDMARKS_IDXS["right_eye"]
        

        
cap = cv2.VideoCapture(1)
        

        
#  Loop frames from video stream 
        
while True:
        
    ret, frame = cap.read()
        

        
    gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
        
    #  Detect faces in gray frames 
        
    rects = detector(gray, 0)
        

        
    #  Face detection cycle 
        
    for rect in rects:
        
        #  Determine the facial signs of the face area , Then put the facial logo （x,y） Coordinate conversion to digital array 
        
        shape = predictor(gray, rect)
        
        shape = face_utils.shape_to_np(shape)
        
        #  Extract left eye and right eye coordinates , Then use coordinates to calculate the eye aspect ratio of both eyes 
        
        leftEye = shape[lStart:lEnd]
        
        rightEye = shape[rStart:rEnd]
        
        leftEAR = eye_aspect_ratio(leftEye)
        
        rightEAR = eye_aspect_ratio(rightEye)
        
        #  Average aspect ratio of both eyes 
        
        ear = (leftEAR + rightEAR) / 2.0
        

        
        #  Calculate the marker points of the left eye and the right eye and draw 
        
        leftEyeHull = cv2.convexHull(leftEye)
        
        rightEyeHull = cv2.convexHull(rightEye)
        
        cv2.drawContours(frame, [leftEyeHull], -1, (0, 255, 0), 1)
        
        cv2.drawContours(frame, [rightEyeHull], -1, (0, 255, 0), 1)
        

        
        #  Mark the face in the picture , And display 
        
        left = rect.left()
        
        top = rect.top()
        
        right = rect.right()
        
        bottom = rect.bottom()
        
        cv2.rectangle(frame, (left, top), (right, bottom), (0, 255, 0), 3)    
        

        
        '''
        
                         The first step is to check whether the aspect ratio of the eye is lower than our blink threshold , If it is , We increment the number of consecutive frames indicating that blinking is occurring . otherwise , We will deal with cases where the aspect ratio of the eye is not lower than the blink threshold , We check it ,
        
                         See if a sufficient number of consecutive frames contain blink rates below our predefined threshold . If the inspection passes , We increase the total number of flashes . Then we reset the number of consecutive flashes  COUNTER.
        
        '''
        
        if ear < EYE_AR_THRESH:
        
            COUNTER += 1
        
        else:
        
            #  If the eyes are closed enough , Then the total number of blinks increases 
        
            if COUNTER >= EYE_AR_CONSEC_FRAMES:
        
                TOTAL += 1
        
            #  Reset eye frame counter 
        
            COUNTER = 0
        

        
        for (x, y) in shape:
        
            cv2.circle(frame, (x, y), 1, (0, 0, 255), -1)
        

        
        cv2.putText(frame, "COUNTER: {}".format(COUNTER), (150, 30),
        
        cv2.FONT_HERSHEY_SIMPLEX, 0.7, (0, 0, 255), 2)    
        
        cv2.putText(frame, "Blinks: {}".format(TOTAL), (10, 30),
        
        cv2.FONT_HERSHEY_SIMPLEX, 0.7, (0, 0, 255), 2)
        
        cv2.putText(frame, "EAR: {:.2f}".format(ear), (300, 30),
        
        cv2.FONT_HERSHEY_SIMPLEX, 0.7, (0, 0, 255), 2)
        

        
    cv2.imshow("Frame", frame)
        
    print(len(rects))   
        
    # if the `q` key was pressed, break from the loop
        
    if cv2.waitKey(1) & 0xFF == ord('q'):
        
        break
        

        
# do a bit of cleanup
        
cv2.destroyAllWindows()
       
       
        1.
        2.
        3.
        4.
        5.
        6.
        7.
        8.
        9.
        10.
        11.
        12.
        13.
        14.
        15.
        16.
        17.
        18.
        19.
        20.
        21.
        22.
        23.
        24.
        25.
        26.
        27.
        28.
        29.
        30.
        31.
        32.
        33.
        34.
        35.
        36.
        37.
        38.
        39.
        40.
        41.
        42.
        43.
        44.
        45.
        46.
        47.
        48.
        49.
        50.
        51.
        52.
        53.
        54.
        55.
        56.
        57.
        58.
        59.
        60.
        61.
        62.
        63.
        64.
        65.
        66.
        67.
        68.
        69.
        70.
        71.
        72.
        73.
        74.
        75.
        76.
        77.
        78.
        79.
        80.
        81.
        82.
        83.
        84.
        85.
        86.
        87.
        88.
        89.
        90.
        91.
        92.
        93.
        94.
        95.
        96.
        97.
        98.
        99.
        100.
        101.
        102.
        103.
        104.
        105.
        106.
        107.
        108.
        109.
        110.
        111.
        112.
        113.