Robomaster visual tutorial (10) target prediction

RoboMaster Visual tutorial （10） Target forecast

The target prediction should be RM The most difficult and important part of vision , The prediction directly determines the hit rate after the bullet is fired , To be honest, my prediction is not good . There are many reasons why forecasting is difficult , I think the most difficult part is that prediction requires the cooperation of upper and lower computers , That is, wonderful calculation / Microcomputer and chariot STM32 We need to cooperate to achieve better results .

Although I can also write some embedded programs STM32 Also know better , But the level is too poor .17 I was responsible for the self-aiming and electronic control of the base during the competition in , At that time, I couldn't even control the rotation of the base. I always hit the wall during the game , Self aiming is done in vain ...

Get down to business , Although I do not predict how well , But I still have some experience to share with you .

Why should we make target prediction

First of all, let's talk about the necessity of prediction , Why should we make target prediction . Because of the limitations of camera and microcomputer performance , The frame rate will not be very high .

Every time the camera captures the armor of the combat vehicle 8ms Left right delay （ Press 120fps count ） Then add the processing time and serial port transmission time , In this way, it has been more than ten milliseconds from the appearance of the armor plate to the recognition and issuance of control commands .

After the lower computer receives the data, it also takes some time for the pan tilt to rotate to the specified position , When the pan tilt rotates in place, the armor plate is no longer in the position it appeared before .

If enemy vehicles keep moving , The self-aiming will always lag behind the actual position of the armor plate , If the enemy's vehicle is a small gyro spinning constantly, it will be more difficult to hit .

Several situations and analysis of prediction problems

• The camera is on the car chassis , The car doesn't move
• The camera is on the gun barrel of the car , The car doesn't move
• The camera is on the car chassis , Trolley movement
• The camera is on the gun barrel of the car , Trolley movement

The first case is the simplest . In the first case, the camera is the third angle , It is equivalent to that the trolley coordinate system is equivalent to the geodetic coordinate system . The position obtained by identifying the armor plate is the coordinate of the enemy combat vehicle armor plate in the geodetic coordinate system , In this way, the moving speed of the armor plate can be obtained through the time difference and position difference between the two adjacent frames , Then the prediction quantity can be obtained by multiplying the speed by the time for prediction compensation .

In the second case, the camera is the first view , The camera will move with the movement of the pan tilt , At this time, the measured position of the armor plate is relative to the camera, that is, relative to the barrel , If you want to predict, you need to know the current deflection angle of the barrel and then carry out coordinate conversion , The actual rotation angle of the object can be obtained by adding the current rotation angle of the barrel to the rotation angle converted from the position of the armor plate , Then the angular velocity can be calculated and the predicted quantity can be compensated .

In the third and fourth cases, the car is moving , This means that the trolley coordinate system and the earth coordinate system no longer coincide , If you want to get accurate prediction results, you need to specify an origin , On this basis, measure the displacement of the trolley , Then it is superimposed and converted with the position of the armor plate and the angle of the barrel to obtain the position in the geodetic coordinate system, and then compensated .

How to make predictions

From the above analysis we can see that , The most important thing to predict is to determine the enemy armor plate in the geodetic coordinate system （ You can use rectangular coordinate system or polar coordinate system ） The next position .

In reality, small and medium-sized cars are often in motion , Inertial navigation is hard to do , Therefore, we can use the trolley coordinate system to approximate the geodetic coordinate system . Because in actual combat, most of the time, people stop and fight , And when the car is not moving fast, the error will not be too large .

Generally speaking, everyone will install the camera on the barrel , The position of the armor plate obtained in this way is the position in the camera coordinate system with the location of the camera as the origin .

First of all, you need to set the position coordinate z Axis and y The axis is offset （ Suppose the camera is placed on the barrel and in the middle , be x The axis is just in the middle of the barrel and does not need to be offset ）.

After the offset, we get the coordinates with the PTZ as the origin , Then the coordinates of the armor plate at this time are converted into angles through the inverse trigonometric function （ It is easier to convert the position of the armor plate into an angle than to convert the barrel angle into three-dimensional coordinates ）, Then add the angle of the armor plate and the angle of the barrel to get the angle in the car coordinate system .

With this angle, the angular velocity can be obtained by Kalman filtering or simply subtracting two frames , Then multiply it by the prediction time to get the prediction quantity for compensation .

If you think it's difficult to make predictions, it's OK not to make predictions, but the effect is worse , The deflection angle of the armor plate relative to the camera is directly transmitted as an error to pid Control （pid This itself has the function of prediction ）, It can realize the tracking and aiming effect that there is no static error in speed but static error in position when the target moves, and the static error in position will be eliminated when the target stops , And the static error during motion will decrease with the increase of camera frame rate , In the use of 330fps If the camera recognizes the armor plate, it will not be lost and the error is within one armor plate .

If forecasting , Then it is predicted that STM32 It's better to do it on . After all, the sending frequency of the command of the upper computer is lower than the control frequency of the lower computer , You can send the corner converted after identifying the armor plate to the lower computer through the serial port , Then the lower computer predicts the position of the armor plate every control cycle, and then controls the rotation of the pan tilt , You can get better results .

About forecasting , The official of Xinjiang has an article you can read https://zhuanlan.zhihu.com/p/38745950

There is no code in this article, only the idea , Because I didn't do well myself , No prediction was made before this year's division competition , Because the relative position obtained by the camera is used as the speed to predict （ The pan tilt and the angle obtained at any time can be regarded as the relative speed , However, if the PTZ turns around, it will immediately become negative, which will offset the previous prediction effect ）, It never worked （ Be stupid to cry ）, In the end, because the electric control is not awesome, the barrel cannot be obtained pitch and yaw corner （ There are still several car programs to be written near the exam week , So electric control has no time to do this ） There is no prediction .

I applied for my own official account Notes on Jiangda , It is convenient for me to help others by summarizing my own learning and research experience . Interested friends can pay attention to .