Learning note 4 -- Key Technologies of high-precision map (Part 2)

This blog series includes 6 A column , Respectively ：《 Overview of autopilot Technology 》、《 Technical foundation of autopilot vehicle platform 》、《 Autopilot positioning technology 》、《 Self driving vehicle environment perception 》、《 Decision and control of autonomous driving vehicle 》、《 Design and application of automatic driving system 》.
This column is about 《 Autopilot positioning technology 》 Book notes .

---

---

2. Key technology of high-precision map

2.3 Synchronous positioning and map building

• Synchronous positioning and map building (Simultaneous Localization And Mapping,SLAM)： It refers to the robot starting from an unknown place in an unknown environment , In the process of movement, we can locate our own position and posture through the observed environmental features , Then build a map of the surrounding environment according to its own location , So as to achieve the goal of simultaneous positioning and map building ;
• SLAM There are two main problems to solve ：“ Where am I? " and " What's around me ”, The former corresponds to the problem of self positioning , The latter corresponds to the problem of map construction ;
• Under the current development ,SLAM There are two forms of implementation ： One is laser based on laser radar SLAM; Second, camera based vision SLAM;
  • laser SLAM： It mainly obtains the laser point cloud data of the surrounding environment of autonomous vehicle through laser radar , Lidar can measure the angle and distance of objects around the car with high accuracy , So it is very convenient to realize SLAM And obstacle avoidance ;
  • Vision SLAM： Mainly through the data collected by the camera for synchronous positioning and map construction ; The image information collected by visual sensor is richer than that obtained by laser radar , More conducive to post-processing ; Vision SLAM There are two main implementation paths ： One is based on the depth camera , Such as Kinect; The second is based on monocular 、 Binocular or fisheye Cameras ;
• laser SLAM And depth camera based VSLAM Through the collected laser point cloud data , Can directly calculate the distance of obstacles ; Based on monocular 、 Binocular 、 Fisheye camera VSLAM, Using multiple frames of images to estimate their own pose changes , Then calculate the distance of the object by accumulating the changes of position and posture , And carry out positioning and map construction ;

2.3.1 SLAM The classic framework of

Whole SLAM Problems that the system should actually solve ： How to estimate its own state through sensor data ( State is posture ); State estimation can be regarded as a mathematical modeling process , That is, how to pass the measurement data with noise , Estimate your state . namely ：
$$x_k=f(x_{k-1})+u_{k-1}\text{\hspace{1em}(1)}$$

$$y_k=h(x_k)+v_k\text{\hspace{1em}(2)}$$

among ：

• type (1) Is the equation of motion , It means that $k$ moment , The pose of the robot $x_k$ from $k-1$ The posture of the moment $f(x_{k-1})$ decision ; The actual environment will always cause errors , So add a noise amount $u_{k-1}$ Form certain constraints on state changes ;
• type (2) Is the observation equation , Express $k$ Sensor observation value at time $y$, From the robot's position and posture at the current moment $h(x_k)$ decision ; The impact of the actual environment , Bring in a certain observation error , namely $v_k$;
• The equation of motion describes the state $x_{k-1}$ How to change to $x_k$ Of , The observation equation describes how to start from $x_k$ Get the observation data $y_k$;

• Sensor data

  Sensor data in vision SLAM It is mainly about the reading and preprocessing of camera image information , In the laser SLAM It is mainly about the reading and preprocessing of laser point cloud data ;

• front end

  Vision SLAM in , The front end is equivalent to the visual odometer (Visual Odometer,VO), It mainly studies the transformation relationship between frames ; First extract the feature points of each frame , Rough registration of feature points using adjacent frame images , Then use the algorithm to remove the wrong matching , Further precise registration is carried out to obtain a pose information , At the same time, it can be combined with IMU The provided pose information is fused ; Each time the visual odometer estimates the motion between two images, there will be errors , This error will continue to accumulate, leading to drift ; The drift phenomenon is solved by loop detection and back-end optimization ;

  laser SLAM in , Front end scanning matching is the core step , The work is to know the position and pose of the previous frame and estimate the position and pose of the current frame by using the relationship between adjacent frames ; laser SLAM The mainstream front-end scanning matching algorithms mainly include iterative nearest neighbor algorithm and variants 、 Correlation scan matching (Correlative Scan Matching,CSM) Algorithm 、 Normal distribution transformation algorithm 、 Feature based matching (Feature-Based Matching,FBM) Algorithm etc. ;

• Back end

  The back end is mainly to optimize the output results of the front end , Use filtering theory or optimization theory to optimize trees or graphs , Finally, the optimal pose estimation is obtained ; The main methods are ： Kalman filter based on Bayesian filter 、 Extended Kalman filter, particle filter and other filtering theory methods , Optical flow method based on graph optimization 、 Intensive tracking and mapping (Dense Tracking And Mapping,DTAM)、 Wide range monocular (Large-Scale Direct Monocular SLAM,LSD-SLAM) Law and semi direct (Semi-Direct Monocular Odometry,SVO) Method and other optimization theory methods ;

• Loop back detection

  Loop detection mainly solves the problem of position estimation drifting with time , Require robots or any use SLAM Technical equipment has the ability to recognize the scenes that have been visited , Loop detection is mainly completed by judging the similarity between images ; If the test is successful , Then send the test results to the backend , The back end adjusts the travel path and the built map according to this information ;

• Map building

  Map output form ： Measurement map 、 Topological map ; Metric map emphasizes accurately representing the position relationship of objects in the map , According to the amount of point cloud data, it is divided into sparse map and dense map ; Sparse maps selectively ignore some information , Abstract all objects to a certain extent , So as to retain only some meaningful things ; Dense maps tend to retain all information , Model everything you see ; Use sparse map to locate autonomous vehicle , Dense map for navigation ;2D The metric map of consists of a small grid (Grid) form ,3D The metric map of consists of small squares (Voxel) form ; Each grid is occupied by 、 Idle and unknown 3 These states indicate whether there are objects in the grid ; Topological map emphasizes the relative relationship between objects in the map , Remove the details ;

2.3.2 SLAM Scheme comparison

• cost

  Lidar costs a lot , Tens of thousands to hundreds of thousands of Yuan ; Vision SLAM The camera cost is low ;

• Application scenarios

  laser SLAM It is mainly used in indoor and small environment ; Vision SLAM More extensive application scenarios , It can better adapt to the outdoor environment , But it is highly dependent on light , It doesn't work in dark or non textured areas ;

• Map accuracy

  laser SLAM The accuracy of the constructed map is high , Reach centimeter level , It can be directly used for positioning and navigation ; Vision SLAM The accuracy of the constructed map is low ;

• Ease of use

  laser SLAM And depth camera based vision SLAM It is through direct access to point cloud data in the environment , Estimate obstacles, distance and other information according to the generated point cloud data ; Based on monocular 、 Vision of binocular camera SLAM Do not directly obtain point cloud data , Instead, get gray or color images through the camera , Complete the extraction and matching of feature points , Then use various ranging methods to get the position information of obstacles ;

2.4 High precision map cloud service system

2.4.1 High precision map OTA service

• Over the air technology (Over The Air Technology,OTA) It's through mobile communication (GSM、CDMA etc. ) Network air interface technology for remote management of data and Applications ;
• OTA The most commonly used function is to get push information from mobile phones and upgrade the system , In this way, the mobile phone system can be updated through wireless download , All upgrades can be completed in a short time , Without backing up data, all data can be preserved intact , The basic flow ： Generate update package 、 Transfer update package 、 Install the update package ;
• In the automobile industry ,OTA It is divided into ： Firmware online upgrade (Fireware Over The Air,FOTA), That is, to the electronic control unit through the air interface (Electronic Control Unit,ECU) Wait for the on-board device to download the complete firmware image , Or patch existing firmware 、 Update memory ; The other is software online upgrade (Software Over The Air,SOTA), That is, the update of applications and map data ;

2.4.2 Sensor data return and update technology

• be based on OTA The derived sensor data return and update technology can transfer the data collected by the terminal ( Including vehicle end status 、 road 、 Road signs, etc ) Back to the cloud , Data preprocessing in the cloud 、 Data normalization 、 Data production and data release ;
• The contents of sensor data return include ：APP The run data 、 Vehicle status data 、 Body sensor data and map acquisition data ;
  • APP The run data ： finger APP Operation log of 、 Crash log 、API Call and other information ;
  • Vehicle status data ： It refers to the positioning data of the car 、 Speed 、 Information such as heading and data confidence ;
  • Body sensor data ： Refers to the wiper speed obtained by the sensor 、 Electric quantity 、 Tire pressure, interior temperature and other data ;
  • Map collection data ： It refers to the image collected by the sensor combined with high-precision map 、 Video and identified structured data ;
• The strategies of sensor data return include ： Active upload and task upload ;
  • Active upload ： Finger Dang APP The state of 、 Vehicle running state 、 Actively upload data when the road state changes ;
  • Task upload ： It refers to meeting the task requirements , For example, when the car arrives at the designated area 、 Upload data when the current state is the same as the defined state ;
• The sensor data return protocol includes ： Transfer protocol 、 Data protocol 、 Cache policy Protocol ;
  • Transfer protocol ： use HTTP and MQTT agreement ;
  • Data protocol ： use SensorIS and SFCD agreement ;
  • Cache policy Protocol ： Refers to the upload traffic restriction mechanism 、 Network anomaly detection mechanism 、 Detection mechanism for too small amount of data ;