Yolo opencv scale identification scale reading identification water gauge identification water level identification source code

Scale identification items

• • brief introduction
  • Realization effect
  • Implementation process
  • Realization principle
  • • Calibration of the scale
    • • Pointer position calibration
      • Reading position calibration
      • Scale mark calibration
    • Digital detection
    • Digital segmentation
    • Number recognition
  • web Deploy
  • appendix

brief introduction

it's been a long time , I looked at my recent blog and now it has 3 It hasn't been updated for months . This is because I have been busy with my graduation project recently , So there has been no time to update . I have gained a lot in the past three months , I will share with you through my blog recently . Now let's write an article on the water . If you have seen my EAST Text detector application The words of this blog , May be curious about my last gif demo How it was done . That was actually one of my junior students srp project . Now let me introduce it .

Realization effect

Need not be wordy , Just look at the effect , Actually, I did it quite simply , There are many things that can be improved .

1. demo1

2. demo2

Implementation process

I have completed the above basic functions, mainly including the following steps .

• Calibration of the scale
• Digital position detection and segmentation
• Number recognition
• Result visualization
• Deploy to web

among , It is necessary , The rest are personalized options .

Realization principle

Calibration of the scale

The calibration of the scale is a very important link . Considering that the position of the scale is relatively fixed , Therefore, the pointer can be known through a simple manual calibration process 、 Relative position of scale and number . Of course, we can also use today's very powerful target detection technology based on deep learning , You can play by yourself .

The following is a schematic diagram of the calibration ：

from demo It can be seen that the whole reading system can be pictographically made into the above figure . What we need to pay attention to ：1） The position of the pointer ;2） The position of the scale reading , Let us assume that we can frame the numbers , Then the central coordinate of the frame is the position of the reading ;3） The distance from the reading to the position indicated by the pointer ; First of all, there is a thick black solid line at the beginning of the number , Then the pixel distance from the solid line to the pointer needs to be converted into the actual scale distance .

Pointer position calibration

Pointer position calibration is very simple , Because it usually presents isosceles triangle , Let's draw its edges , Then find its angular bisector , You will get the point of the pointer .

My calibration program needs to specify three red dots from top to bottom , Automatically connect the green line after specifying （ Pointer edge ）, Then automatically generate its angular bisector （ The blue line ）, The whole program is interactive , Prompt during execution , The final calibration file will be saved automatically , Load the calibration file for the next use .

Reading position calibration

Suppose we get the central pixel coordinates of the number closest to the pointer position , But we also need to know how far this pixel coordinate is from the thick black solid line below it . ad locum , I assume that the center coordinate is y coordinate y c y_c yc​ And the thick solid line y coordinate y b y_b yb​ There is a relation of quadratic function , This is not really strict , It's just a feeling . So there is ：
y b = a y c 2 + b y c + c y_b = ay_c^2 + by_c + c yb​=ayc2​+byc​+c
such , We demarcate multiple positions of multiple pictures ( y c , y b ) (y_c, y_b) (yc​,yb​), We can fit the coefficients .

The program needs to manually frame all the numbers , It will automatically return to the red center point ; Then manually point out the black solid line （ Blue dot ）, After the program collects enough groups of samples, it can fit , Generate fit curve ：

The fitting relationship is good .

Scale mark calibration

We can now derive the coordinates of the starting thick solid line , We need to know ：1） The pointer points to the intersection with the scale （ End coordinates ）;2） Conversion from pixel distance to scale distance . First , Let's calibrate the scale first , As shown below ：

Tick the scales from bottom to top or in reverse order , And try to make the points line up roughly , This is because after collecting multiple images , The program will fit this line . The fitted scale line , Find the intersection point with the angular bisector of the pointer calibrated at the beginning , Got y The axis direction coordinate is the end coordinate .

There are start coordinates and end coordinates , There is only the problem of distance conversion . Here I also assume that , any y Axis coordinates y 0 y_0 y0​, The pixel increments of the previous scale and the next scale δ y ↑ , δ y ↓ \delta y_{\uparrow}, \delta y_{\downarrow} δy↑​,δy↓​ There are also quadratic functions , namely ：
δ y ↑ = a ↑ y 0 2 + b ↑ y 0 + c ↑ δ y ↓ = a ↓ y 0 2 + b ↓ y 0 + c ↓ \delta y_{\uparrow} = a_{\uparrow}y_0^2 + b_{\uparrow}y_0 + c_{\uparrow} \\ \delta y_{\downarrow} = a_{\downarrow}y_0^2 + b_{\downarrow}y_0 + c_{\downarrow} \\ δy↑​=a↑​y02​+b↑​y0​+c↑​δy↓​=a↓​y02​+b↓​y0​+c↓​
Same as above , After collecting enough samples, fit , The fitting effect is as follows ：

The visible effect is OK . After fitting , Suppose our starting and ending coordinates are y s , y e y_s,y_e ys​,ye​. Then we can find y s y_s ys​ Pixel coordinates of the next scale ：
y s , 1 = y s + δ y s = a ↓ y s 2 + ( b ↓ + 1 ) y s + c ↓ y_{s,1} = y_s + \delta y_s = a_{\downarrow}y_s^2 + (b_{\downarrow}+1)y_s + c_{\downarrow} ys,1​=ys​+δys​=a↓​ys2​+(b↓​+1)ys​+c↓​
Keep repeating the process , We can get a millimeter reading y s , k ≤ y e ≤ y s , k + 1 y_{s,k} \le y_e \le y_{s,k+1} ys,k​≤ye​≤ys,k+1​. If you need to take two decimal places , Then it is OK to take an equal proportion in this interval .

Digital detection

Digital detection uses what I mentioned in my previous blog EAST Model , It details how to configure and use , Model input original picture , Returns the center coordinate of a number .

Digital segmentation

I use the same traditional method for digital segmentation . First , Convert the image to grayscale , And then use OTSU Method to binarize the picture . Last , Use for binary images MSER Algorithm , This algorithm can effectively extract the connected domain . In this way, we will get a single divided number .

mser = cv2.MSER_create(_min_area=min_area, _max_area=max_area)
img = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)		#  Turn to grayscale 
_, img = cv2.threshold(img, 0, 255, cv2.THRESH_BINARY_INV + cv2.THRESH_OTSU)	#  Two valued 
_, boxes = self.mser.detectRegions(img)		# MSER Division 

Number recognition

Easy to find , All the numbers we need to identify are printed . therefore , Data sets require only printed numbers 0~9 that will do , You may find it difficult to find datasets , But printed data sets can be used opencv Generated . In this part, I refer to the open source project digitx, It gives a very detailed scheme for generating data sets and data enhancement , And trained a simple CNN It realizes efficient digital recognition .

web Deploy

I use flask Realization , But after using the local computer as the server, it can only be accessed by the LAN , If you need to allow friends in China to visit , You can apply for a public server , For example, Alibaba cloud , Then put the code on it to run , Others can visit .

appendix

For the full code, see ：github

If there is a problem , You can add the following QR code for consultation