Face++ realizes face detection by flow

We use face detection sdk The process of face recognition can be roughly divided into the following steps ：

1. load model File and initialize the interface
2. Get authorization
3. Call face detection to get key points
4. Notice the key points of the drawing
5. Release interface resources

Here are the steps 3 and 4 On receiving camera Execute repeatedly after data . We can abstract these steps into data flow for encapsulation .face++ Face coordinates are generated continuously after face detection , So the whole detection process is the process of face coordinate data production , Here with flow The concept of corresponds to .

face++ Need to deal with camera Data to get face key points , therefore camera again face++ Face detection api Data providers for . The following figure shows the process of data flow ：

 

Seeing this, people also think that we passed flow It is more reasonable to realize this process ！ Let's implement it step by step detect flow The creation of .

The first step is to pass flow Of builder Create a flow：

 private val imageChannel = Channel<FacePPImage>()

flow {
            while (currentCoroutineContext().isActive) {
                emit(imageChannel.receive())
            }
        }

Use here Channel To receive image data and transmit . stay detect flow We also need some initialization before processing image data face++ The job of , These work in onStart In dealing with .

flow.onStart {
            var ret = -1
            context.assets.open("megviifacepp_model").use { ios ->
                modelBuffer = ByteArray(ios.available())
                ios.read(modelBuffer)
                FaceppApi.getInstance().setLogLevel(4)
                ret = FaceppApi.getInstance().initHandle(modelBuffer)
            }
            if (ret != 0) {
                Log.d("dragon_debug", " onStart open failed!")
                throw RuntimeException("init")
            }
            if (requestTakeLicense && modelBuffer != null) {
                Log.d("dragon_debug", " onStart takeLicense")
                ret = takeLicense(context, modelBuffer!!)
            }
            if (ret != 0) {
                Log.d("dragon_debug", " onStart takeLicense failed!")
                throw RuntimeException("takeLicense")
            }
            ret = FaceDetectApi.getInstance().initFaceDetect()
            DLmkDetectApi.getInstance().initDLmkDetect()
            if (ret != 0) {
                if (requestTakeLicense) {
                    Log.d("dragon_debug", " onStart initFaceDetect error")
                    throw RuntimeException("error")
                }
                requestTakeLicense = true
                Log.d("dragon_debug", " onStart initFaceDetect retry exception")
                throw RuntimeException("initFace")
            }
            val config = FaceDetectApi.getInstance().faceppConfig
            config.face_confidence_filter = 0.6f
            config.detectionMode = FaceDetectApi.FaceppConfig.DETECTION_MODE_TRACKING
            FaceDetectApi.getInstance().faceppConfig = config
        }

face++ The initialization operation includes loading the model 、 Detect interface authorization 、 Face detection interface initialization and other steps .

The detection interface authorization processing is somewhat special ,face++ The interface authorization of is only executed when the authorization expires . So here we go flow Of retryWhen To handle expired authorizations .

flow.retryWhen { cause, attempt ->
            Log.d("dragon_debug", " retryWhen $cause attempt $attempt")
            if (attempt > 1) {
                false
            } else {
                (cause as? RuntimeException)?.message?.equals("initFace") ?: false
            }
        }

First try to initialize face++ Interface , If initialization fails , An exception will be thrown here RuntimeException("init").retryWhen Catch this exception and initiate retry processing , When onStart When executed again , We will try to obtain the interface permission .

face++ After the interface authorization is initialized successfully , We can use face++ Face detection interface . Use here map Converting image data into face coordinate data , Conversion operation by face++ Face detection interface is completed .

flow.map { image ->
            val faces = FaceDetectApi.getInstance().detectFace(image)
            faces.forEach { face ->
                FaceDetectApi.getInstance().getLandmark(face, FaceDetectApi.LMK_84, true)
            }
            block.invoke(faces)
            faces
        }

The obtained face coordinate data passes through block Callback notification screen drawing .

face++ The release operation of the interface is in onCompletion in ,

flow.onCompletion {
            Log.d("dragon_debug", " onCompletion ")
            FaceppApi.getInstance().ReleaseHandle()
            DLmkDetectApi.getInstance().releaseDlmDetect()
        }

Complete code ：GitHub - mjlong123123/FaceDetectorFlow