Abstract
Grasping a specified object from multi-object scenes is an essential ability for intelligent robots. This ability depends on the affiliation between the grasp position and the object category. Most existing multi-object grasp detection methods considering the affiliation rely on object detection results, thus limiting the improvement of robotic grasp detection accuracy. This paper proposes a decoupled single-stage multi-task robotic grasp detection method based on the Faster R-CNN framework for multi-object scenes. The designed network independently detects the category of an object and its possible grasp positions by using one loss function. A new grasp matching strategy is designed to determine the relationship between object categories and predicted grasp positions. The VMRD grasp dataset is used to test the performance of the proposed method. Compared with other grasp detection methods, the proposed method achieves higher object detection accuracy and grasp detection accuracy.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Hou X Y, Ao W, Song Q, et al. FUSAR-Ship: building a high-resolution SAR-AIS matchup dataset of Gaofen-3 for ship detection and recognition. Sci China Inf Sci, 2020, 63: 140303
Zhang W T, Jiang J W, Shao Y X, et al. Snapshot boosting: a fast ensemble framework for deep neural networks. Sci China Inf Sci, 2020, 63: 112102
Xie G, Shangguan A Q, Fei R, et al. Motion trajectory prediction based on a CNN-LSTM sequential model. Sci China Inf Sci, 2020, 63: 212207
Xie J, Pang Y W, Cholakkal H, et al. PSC-Net: learning part spatial co-occurrence for occluded pedestrian detection. Sci China Inf Sci, 2021, 64: 120103
Lenz I, Lee H, Saxena A. Deep learning for detecting robotic grasps. Int J Robot Res, 2015, 34: 705–724
Redmon J, Angelova A. Real-time grasp detection using convolutional neural networks. In: Proceedings of IEEE International Conference on Robotics and Automation, 2015. 1316–1322
Zhou X W, Lan X G, Zhang H B, et al. Fully convolutional grasp detection network with oriented anchor box. In: Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems, 2018. 7223–7230
Song Y, Gao L, Li X, et al. A novel robotic grasp detection method based on region proposal networks. Robot Comput-Integrated Manuf, 2020, 65: 101963
Mahler J, Goldberg K. Learning deep policies for robot bin picking by simulating robust grasping sequences. In: Proceedings of Conference on Robot Learning, 2017. 515–524
Zeng A, Song S, Yu K-T, et al. Robotic pick-and-place of novel objects in clutter with multi-affordance grasping and cross-domain image matching. In: Proceedings of IEEE International Conference on Robotics and Automation, 2018. 1–8
Zhang H, Lan X, Bai S, et al. ROI-based robotic grasp detection for object overlapping scenes. In: Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems, 2019. 4768–4775
Park D, Seo Y, Shin D, et al. A single multi-task deep neural network with post-processing for object detection with reasoning and robotic grasp detection. In: Proceedings of IEEE International Conference on Robotics and Automation, 2020. 7300–7306
Ren S Q, He K M, Girshick R, et al. Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Trans Pattern Anal Mach Intell, 2017, 39: 1137–1149
Asif U, Bennamoun M, Sohel F A. RGB-D object recognition and grasp detection using hierarchical cascaded forests. IEEE Trans Robot, 2017, 33: 547–564
Wang Z C, Li Z Q, Wang B, et al. Robot grasp detection using multimodal deep convolutional neural networks. Adv Mech Eng, 2016, 8: 1–12
Kumra S, Kanan C. Robotic grasp detection using deep convolutional neural networks. In: Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems, 2017. 769–776
Guo D, Sun F C, Kong T, et al. Deep vision networks for real-time robotic grasp detection. Int J Adv Robotic Syst, 2017, 14: 1–8
Guo D, Sun F, Liu H, et al. A hybrid deep architecture for robotic grasp detection. In: Proceedings of IEEE International Conference on Robotics and Automation, 2017. 1609–1614
Gualtieri M, Pas A T, Saenko K, et al. High precision grasp pose detection in dense clutter. In: Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems, 2016. 598–605
Guo D, Kong T, Sun F, et al. Object discovery and grasp detection with a shared convolutional neural network. In: Proceedings of IEEE International Conference on Robotics and Automation, 2016. 2038–2043
Jiang Y, Moseson S, Saxena A. Efficient grasping from RGBD images: learning using a new rectangle representation. In: Proceedings of IEEE International Conference on Robotics and Automation, 2011. 3304–3311
He K M, Zhang X Y, Ren S Q, et al. Deep residual learning for image recognition. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2016. 770–778
Lin T Y, Dollar P, Girshick R, et al. Feature pyramid networks for object detection. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2017. 936–944
Yang X, Sun H, Fu K, et al. Automatic ship detection in remote sensing images from Google Earth of complex scenes based on multiscale rotation dense feature pyramid networks. Remote Sens, 2018, 10: 132
Song Y, Pan Q K, Gao L, et al. Improved non-maximum suppression for object detection using harmony search algorithm. Appl Soft Computing, 2019, 81: 105478
Zhang H, Lan X, Wan L, et al. RPRG: toward real-time robotic perception, reasoning and grasping with one multi-task convolutional neural network. 2018. ArXiv:1809.07081
Acknowledgements
This work was supported by China Postdoctoral Science Foundation (Grant No. 2021M692778), National Key Research and Development Project of China (Grant No. 2018AAA0101704), Natural Science Foundation of Hubei Province (Grant No. 2021CFB368), and Research Project of Hubei Provincial Department of Education (Grant No. Q20201105).
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Song, Y., Gao, L., Li, X. et al. A novel vision-based multi-task robotic grasp detection method for multi-object scenes. Sci. China Inf. Sci. 65, 222104 (2022). https://doi.org/10.1007/s11432-021-3558-y
Received:
Revised:
Accepted:
Published:
DOI: https://doi.org/10.1007/s11432-021-3558-y