Android NDK development and actual combat WeChat official account 2-D code detection

Want to make APP Same thing as WeChat , Can run small programs smoothly ？ | Experience will send you to Xinjiang 、 Huawei 、 Cherry keyboard ！>>>

On two dimensional code recognition , We usually use them Zxing perhaps Zbar , But their recognition rate is not very high , In some cases, it fails , Take the following two pictures for example ：

 

 

Using open source libraries Zxing Scan the above two QR codes , There's a dead or alive one . Wechat is OK , You can try Alipay. （ no way ）, What should we do in this situation ？ ha-ha , This time, it has a place , We're trying to optimize it .

We used this function in WeChat official account. , Press a picture , If the picture contains a QR code , The QR code in the identification map will pop up , If the picture does not contain a QR code , The option to identify the QR code will not pop up . At this point, we should know , There are two steps to recognize QR code , The first step is to find the intercepted QR code area , The second step is to identify the intercepted QR code area . that zxing What is the problem with Alipay? ？ First of all, let's take a look at the first step to find the intercepted QR code region .

QR code example

The above figure is a common example of two-dimensional code , There are three important areas , It's the top left , Top right and bottom left , We just need to find these three areas , It can be determined that there is a QR code in the picture . Next, let's analyze the ideas ：

1. Find the contour of it
2. The initial filtering is carried out for the found contour
3. Judge whether it conforms to the feature rules of QR code
4. Intercept QR code area
5. Identify QR code

//   Judge  X  Is the direction in line with the rules 
bool isXVerify(const Mat& qrROI){
    ...  Code ellipsis 
    //  Judge  x  Direction left to right pixel scale 
    //  black ： white ： black ： white ： black  = 1:1:3:1:1
}

//   Judge  Y  Is the direction in line with the rules 
bool isYVerify(const Mat& qrROI){
    ...  Code ellipsis 
    // y  You can also follow the  isXVerify  Methods to judge 
    //  But we can also write it simply 
    //  White pixels  * 2 <  Black pixels  &&  Black image  < 4 *  White pixels  
}

int main(){
    Mat src = imread("C:/Users/hcDarren/Desktop/android/code1.png");

    if (!src.data){
        printf("imread error!");
        return -1;
    }
    imshow("src", src);

    //  Gray scale conversion of image 
    Mat gary;
    cvtColor(src, gary, COLOR_BGR2GRAY);
    //  Two valued 
    threshold(gary, gary, 0, 255, THRESH_BINARY | THRESH_OTSU);
    imshow("threshold", gary);
    // 1.  Find the contour of it 
    vector<vector<Point> > contours;
    findContours(gary, contours, RETR_LIST, CHAIN_APPROX_SIMPLE);

    for (int i = 0; i < contours.size(); i++)
    {
        // 2.  The initial filtering is carried out for the found contour 
        double area = contourArea(contours[i]);
        // 2.1  Preliminary filtration area  7*7 = 49
        if (area < 49){
            continue;
        }
        
        RotatedRect rRect = minAreaRect(contours[i]);
        float w = rRect.size.width;
        float h = rRect.size.height;
        float ratio = min(w, h) / max(w, h);
        // 2.2  Preliminary filter aspect ratio 
        if (ratio > 0.9 && w< gary.cols/2 && h< gary.rows/2){
            Mat qrROI = warpTransfrom(gary, rRect);
            // 3.  Judge whether it conforms to the feature rules of QR code 
            if (isYVerify(qrROI) && isXVerify(qrROI)) {
                drawContours(src, contours, i, Scalar(0, 0, 255), 4);
            }
        }
    }

    imshow("src", src);
    imwrite("C:/Users/hcDarren/Desktop/android/code_result.jpg", src);

    waitKey(0);
    return 0;
}

Processing results

The code is very simple , The key is that we should be good at learning to analyze , Develop problem solving skills , As long as you know how to realize it , Nothing else is a problem . So here comes the interesting one , When scanning the second image , We found that we couldn't recognize life or death . So careful students may understand , The code above is identified according to the characteristics of the square , And the second picture is the feature of a circle , therefore Zxing It's normal to be unrecognized , Because we didn't think about it when we wrote the code . So how can we recognize the features of a circle ？ It's time to test us , We can think of three solutions ：

1. Write another set of code to recognize circular features
2. Draw lessons from the scheme of face recognition , Training samples are used to identify
3. Change the inspection plan , Write only one set of code

Face recognition will be written in the next article , The way of training samples is more troublesome , If you haven't touched it before , So it takes a certain time cost , But it should be the best . Write another set of Circle Recognition code , Feel difficult to maintain , As an engineer with a soul, I always feel uncomfortable . So here we'll take the third option , In fact, there are so many knowledge points , Or that sentence Cultivate our ability to analyze and solve problems .

Let's watch carefully , They still have a lot in common , When we filter the contour, we will find that , It's a big outline with two small outlines inside . The specific process is as follows ：

1. Find the contour of it
2. The initial filtering is carried out for the found contour
3. Judge whether it is a large profile with two small profiles and conform to the feature rules ( Area proportion judgment )
4. Intercept QR code area
5. Identify QR code

extern "C"
JNIEXPORT jobject JNICALL
Java_com_darren_ndk_day76_MainActivity_clipQrBitmap(JNIEnv *env, jobject instance, jobject bitmap) {
    Mat src;
    cv_helper::bitmap2mat(env, bitmap, src);

    //  Gray scale conversion of image 
    Mat gary;
    cvtColor(src, gary, COLOR_BGR2GRAY);

    //  Two valued 
    threshold(gary, gary, 0, 255, THRESH_BINARY | THRESH_OTSU);

    // 1.  Find the contour of it 
    vector<Vec4i> hierarchy;
    vector<vector<Point> > contours;
    vector<vector<Point> > contoursRes;
    /*
      Parameter description ：https://blog.csdn.net/guduruyu/article/details/69220296
         The input image image Must be a 2 Value single channel image 
        contours The parameter is the detected Contour array , Each profile uses one point Type of vector Express 
        hiararchy The number of parameters and contours is the same , Every contour contours[ i ] Corresponding 4 individual hierarchy Elements hierarchy[ i ][ 0 ] ~hierarchy[ i ][ 3 ],
             Each represents the next contour 、 The previous profile 、 The outline of the father 、 Index number of the embedded contour , If there is no counterpart , The value is set to a negative number .
        mode Represents the retrieval mode of the contour 
            CV_RETR_EXTERNAL  Indicates that only the outer contour is detected 
            CV_RETR_LIST  The detected contour does not establish a hierarchical relationship 
            CV_RETR_CCOMP  Build two levels of outline , The upper layer is the outer boundary , The inner layer is the boundary information of the inner hole . If there is a connected object in the inner hole , The boundary of this object is also at the top .
            CV_RETR_TREE  Create a hierarchical tree structure outline . Specific reference contours.c This demo
        method An approximation of the outline 
            CV_CHAIN_APPROX_NONE  Store all contour points , The pixel position difference between two adjacent points shall not exceed 1, namely max（abs（x1-x2）,abs（y2-y1））==1
            CV_CHAIN_APPROX_SIMPLE  Compress the horizontal direction , vertical direction , Diagonal elements , Only the coordinates of the end point in this direction are reserved , For example, a rectangular outline only needs 4 Points to save profile information 
            CV_CHAIN_APPROX_TC89_L1,CV_CHAIN_APPROX_TC89_KCOS  Use teh-Chinl chain  The approximate algorithm 
        offset Represents the offset representing the contour point , Can be set to any value . Yes ROI Outline found in image , And to be analyzed in the whole image , This parameter is still useful .
     */
    findContours(gary, contours, hierarchy, CV_RETR_TREE, CHAIN_APPROX_NONE, Point(0, 0));
    int tCC = 0; //  Sub contour counter for temporary accumulation 
    int pId = -1;//  Parent profile  index
    for (int i = 0; i < contours.size(); i++) {
        if (hierarchy[i][2] != -1 && tCC == 0) {
            pId = i;
            tCC++;
        } else if (hierarchy[i][2] != -1) {//  Parent profile 
            tCC++;
        } else if (hierarchy[i][2] == -1) {//  There is no parent profile 
            tCC = 0;
            pId = -1;
        }
        //  Two sub contours were found 
        if (tCC >= 2) {
            contoursRes.push_back(contours[pId]);
            tCC = 0;
            pId = -1;
        }
    }
    //  Too many matching feature contours were found , Filter them 
    if (contoursRes.size() > FEATURE_NUMBER) {
        contoursRes = filterContours(gary, contoursRes);
    }

    //  No matching conditions were found 
    if (contoursRes.size() < FEATURE_NUMBER) {
        return NULL;
    }
    
    for (int i = 0; i < contoursRes.size(); ++i) {
        drawContours(src, contoursRes, i, Scalar(255, 0, 0), 2);
    }

    //  Cut QR code , hand  zxing  perhaps  zbar  Processing can be 
    
    cv_helper::mat2bitmap(env, src, bitmap);

    return bitmap;
}

Processing results

What we like most in development is to take it and use it directly , But it's better to understand the principle , Because we can't tell what's going to happen in development . Big companies like wechat naturally have their own way , In fact, a good framework can be taken to optimize , I think it's almost the same . Of course, the above method is used in some specific situations , There may still be some loopholes , This depends on our constant thinking and optimization .

 

PS： About me

---

I am a Have 6 Years of experience in development Android Siege lions , Remember to read a little like it , Develop habits , Search on wechat 「 Program ape Development Center 」 Focus on this programmer who likes to write dry goods .

in addition It took two years Sorting and collecting Android Interview for large factories PDF Baked , Information 【 Full version 】 Updated on my 【Github】, Yes Friends for interview We can refer to it , If it helps you , You can order Star Oh ！

Address ：【https://github.com/733gh/xiongfan】