Xiaobai tutorial: Raspberry pie 3b+onnxruntime+scrfd+flask to realize public face detection system

Antecedents feed ; Recently, a new face detection framework has been developed scrfd,scrfd My paper is in 5 month 10 The sun hung on Akai , Interested students can go and have a look

https://arxiv.org/abs/2105.04714

new scrfd It aims to achieve the ultimate balance of model effectiveness ：（Extensive experiments conducted on WIDER FACE demonstrate the state-of-the-art efficiency-accuracy trade-off for the proposed \scrfd family across a wide range of compute regimes, The paper says ）

Comparative experiments also seem to illustrate this , The smallest scrfd-0.5GF The detection speed is faster than before Retinaface-mb0.25 Twice as fast , Detection performance has also achieved a comprehensive breakthrough ：

But what's the effect , Still need down Under the source code run Again , Can be verified .

1、inter Card Run Library test effect

What's the actual effect , Still need to go through git clone Come down and test , Pull the official code ;

git clone https://github.com/deepinsight/insightface.git

take scrfd The source code of is proposed separately , Others can be deleted , It does not affect the running of the Library

Make sure the documents are complete , Install the libraries required by the environment （ The details are written in requirements In the folder ）, stay Inter Core i5-4210M Test on the processor , The test results are as follows ：

The following points are added ：

Single frame of the test Infer Time= Reason for a minute / Number of frames processed in this minute

• In fact, it doesn't need to be so high in actual production input_size（ Main survey 2-3m Detection performance in ）
• This test uses the transformed onnx Model （ I really don't want to call other miscellaneous libraries ）
• The above tests are CPU Proceed under , Want to test GPU Please install by yourself onnxruntime-gpu
• Different input_size The time consumption is as follows （ With 500m The model, for example ）：
  
  Use 500m Model in input_size=320*320 Reasoning under the condition of highly dense face pictures ：
  

2、 Dynamic dimensions onnx Model extraction

• install mmcv-full

Specific installation method can refer to mm Its official website :
https://link.zhihu.com/?target=https%3A//github.com/open-mmlab/mmcv

pip install mmcv-full -f https://download.openmmlab.com/mmcv/dist/cu102/torch1.6.0/index.html

Add a few holes you've stepped on

• Blogger's cuda=10.2,pytorch=1.6.0, According to the command line given by the official website , Generally, there is no mistake
• Actually measured window The compilation of mmcv-full There will be a lack of libraries , And 1.3.4 Above version window Can't install
• Be sure to install the latest mmcv-full, Otherwise, some function interfaces will be missing , The blogger's version is 1.3.5
• Don't be greedy for convenient installation mmcv Reduced version of , Missing Library

Install and compile mmdet

• pip install -r requirements/build.txt
• pip install -v -e .# or “python setup.py develop”

Check after completion list Are there three libraries ：

If there is , You can start extracting onnx Model , Official baseline as follows ：
github.com/deepinsight/insightface

Look at the model you need , Click on download You can download it. , Use the following command to export onnx Model ：

python tools/scrfd2onnx.py configs/scrfd/scrfd_500m.py weights/scrfd_500m.pth --shape -1 -1 --input-img face.jpg

among

• configs It stores model parameter information , Please contact model Good pairing
• Please put shape Designated as -1
• Check the picture randomly , It is recommended to include pictures of multiple faces After extraction onnx It can be used onnxsim Brush once
• Containing key points bnkps The model does not support dynamic dimension input , Extract with fixed size
  

3、flask web Realize streaming

There are two ways to push streams on the public network ：

① Use ECS as the mediation server

② Buy the domain name used for intranet puncture

Both methods have been tried

The first one is that 9.9 Monthly student cloud resources , The bandwidth and processing power are far from the throughput requirements required by the model , This caused serious delays , Bad experience （ Or the problem of soft money ....）

The second is to puncture the intranet through software , There are many specific online materials （ Refer to the quotation 【2】, It's very detailed ）, The advantage is that it saves time and effort, and it can also go whoring for nothing

Whatever it is , Thinking is nothing more than borrowing the public network IP To carry out interworking under different LANs

After intranet puncture , You can use flask Push the detected frame to the public network , The code is as follows ：

from flask import Flask, render_template, Response
import argparse
from tools.scrfd import *
import datetime
import cv2

class VideoCamera(object):
    def __init__(self):
        #  adopt opencv Get a real-time video stream 
        self.video = cv2.VideoCapture(0)
    def __del__(self):
        self.video.release()
    def get_frame(self):
        success, image = self.video.read()
        #  because opencv The picture read is not jpeg Format , So use motion JPEG Mode needs to transcode the image to jpg Format picture 
        # ret, jpeg = cv2.imencode('.jpg', image)
        # return jpeg.tobytes()
        return image

app = Flask(__name__)

@app.route('/')  #  Home page 
def index():
    #  The specific format is saved in index.html In file 
    return render_template('index.html')

def scrfd(camera):
    detector = SCRFD(model_file='onnx/scrfd_500m_bnkps_shape160x160.onnx')
    while True:
        frame = camera.get_frame()
        # cv2.imshow('fourcc', frame)
        # img = cv2.imread(frame)

        for _ in range(1):
            ta = datetime.datetime.now()
            bboxes, kpss = detector.detect(frame, 0.5, input_size = (160, 160))
            tb = datetime.datetime.now()
            print('all cost:', (tb - ta).total_seconds() * 1000)
        for i in range(bboxes.shape[0]):
            bbox = bboxes[i]
            x1, y1, x2, y2, score = bbox.astype(np.int)
            cv2.rectangle(frame, (x1, y1), (x2, y2), (255, 0, 0), 2)
            if kpss is not None:
                kps = kpss[i]
                for kp in kps:
                    kp = kp.astype(np.int)
                    cv2.circle(frame, tuple(kp), 1, (0, 0, 255), 2)
        ret, jpeg = cv2.imencode('.jpg', frame)
        frame = jpeg.tobytes()
        yield (b'--frame\r\n'
               b'Content-Type: image/jpeg\r\n\r\n' + frame + b'\r\n\r\n')

@app.route('/video_feed')  #  This address returns the video stream response 
def video_feed():
    if model == 'scrfd':
        return Response(scrfd(VideoCamera()),
                        mimetype='multipart/x-mixed-replace; boundary=frame')

if __name__ == '__main__':
    parser = argparse.ArgumentParser(description='Object Detection using YOLO-Fastest in OPENCV')
    parser.add_argument('--model', type=str, default='scrfd')
    args = parser.parse_args()
    model = args.model
    app.run(host='0.0.0.0', debug=True, port=1938)

Run the command ：

python flask_api.py

Push the test results to the web page , At this time, according to the network segment number w400985k15.wicp.vip:54551 Log in to the web page to view （w400985k15.wicp.vip For the complimentary domain name ,54551 The port assigned to , primary host Designated as native ip, Automatically jump to the public domain name after mapping ,port It's going to change ）, At this time, raspberry pie and computer belong to different LAN ：

More Than This , You can also check it with your mobile phone , No matter how far away it is .

• The actual measurement uses flask It will be about 0.5s about , Later, you can use fastapi perhaps node.js Try
• It is recommended to use raspberry pie 4B Experience ,3B The board is older , Some performance cannot keep up
• Overclocking is recommended , Reference resources How to Overclock Raspberry Pi 4
  to 2.0 GHz - CNX Softwarey-pi-4/

Code with all onnx Put the model on this connection

pengtougu/onnx-scrfd-flask
​
Last , Put on a song I like to listen to

Reference resources ：

【1】nihui： Record in detail insightface Of SCRFD Face detection ncnn Realization

【2】 Raspberry pie remote monitoring - web Remote monitoring

Welcome to join the deep learning exchange group :696654483
There are many graduate leaders and bigwigs from all walks of life ～