Progressive multi grasp detection using grasp path for rgbd images

The current plane grab detection is mainly based on Jiang stay 2014 Rectangular grab representation proposed in , Its biggest disadvantage is , It is impossible to mark the grab truth value of the object in detail （ Or the existing data sets are not fully annotated ）, Lead to two problems ：

（1） Cannot provide perfect truth value for neural network ;

（2） Unable to accurately verify , Because a practical predictive crawl may not meet the verification criteria , namely IOU>0.25 & angle<30, vice versa .

An easy solution is , Annotate the data set at pixel level , Make the grab of the annotation as close to the true value as possible , But this plan is difficult to achieve , Alternative ,Chen A simplified method is proposed , That is, the paper introduced in this paper ：

@article{***,
  author    = {Lu Chen and Panfeng Huang and Zhongjie Meng},
  title     = {Convolutional multi-grasp detection using grasp path for {RGBD} images},
  journal   = {Robotics Auton. Syst.},
  volume    = {113},
  pages     = {94--103},
  year      = {2019}
}

The paper is divided into two parts ： Introduce the crawl path , Crawl detection network . Here we mainly introduce the concept of crawl path .

One 、 Grab path

Crawl path is introduced as the truth value of network learning , It's some curves on the image , Every point on the curve can be used as a grab point , The normal direction at the grab point is taken as the grab angle , The width and height of the grab frame are given by the predicted grab （ Let's introduce ）.

The crawl path is determined by Cornell The data set is generated by the original annotation rectangle , The specific way is ： Connect the midpoint of any two overlapping rectangular boxes . This operation is applicable to most situations , Here's the picture （ Left ）, But when all three rectangles have overlapping areas , Redundant crawl paths will be generated , When the shape of an object is complex , The generated crawl path may violate common sense , Here's the picture （ in ） and （ Right ）：

In order to cope with the above two situations , The author puts forward solutions respectively , among , In response to the second situation , Even using neural networks .

An example of the finally generated crawl path is shown in the following figure ：

Two 、 How to use crawl path

The author draws lessons from a few years ago ICRA A paper on the network structure for crawl detection , It's not detailed here , Here's the picture ：

In order to calculate the loss of the network , You need to know the truth value capture corresponding to the predicted capture , This truth grab is obtained by predicting the grab and the grab path . First, make a vertical line from the midpoint of the predicted grasp to the nearest grasp path , The intersection is the grab point of truth value grab , The angle of the vertical line is the grasping angle of the truth value grasp , The height and width of the truth grab are equal to the width and height of the predicted grab , Then calculate the loss and back-propagation . Again , The truth fetching used for validation in the validation phase is the same as above . The generation process of truth grab is shown in the following figure （ The solid line box is predictive crawl , Dashed boxes are truth grabs ）：

give the result as follows ：

The author is in another article （Detecting Graspable Rectangles of Objects in Robotic Grasping） In the said , The low accuracy is due to the simple network structure , After changing to a better network structure , Accuracy of 94.6%.

3、 ... and 、 summary

I think the starting point of this article is good , Optimize the training process by generating a more detailed grab truth , But there are still the following problems ：

（1） The width and height of the grab truth box should not be given by the predicted grab box , Because the predicted value may not meet the capture requirements , Such as too long or too short .

（2） The automatically generated crawl path is more detailed than the crawl box , But there are intermittent situations , It is even quite different from the edge of the object .

（3） Can another crawl path parallel to the crawl path and only one pixel away be taken as the true value , That is, capturing the truth value is not just a line , It should be an area .

 

Finally, shamelessly recommend your own B Station No , Occasionally upload robot Related videos