Abstract
To improve the detection accuracy and speed of palletizing robot positioning bolts in complex scenes, we proposed a positioning bolt (PB) detection method based on improved YOLO-V3. First, due to the actual detection requirement, we constructed the PB data set by using a series of data enhancement operations such as horizontal flip, ± 30degree rotation, and random luminance enhancement or decrease. Then, an improved anchor box mechanism based on the k-means++ algorithm was designed to obtain a more accurate anchor box for the PB data. According to the feature of the PB data in the palletizing robot, such as the existence of dust and dirt on the surface, the feature extraction network was further enhanced by adding a Densenet-4 module. In this way, the low-level semantics and high-level abstract features can be extracted effectively to improve detection performance. Finally, a new bounding box regression loss function was elaborated to accelerate the neural network training. The experimental results demonstrated the effectiveness of the proposed improvement mechanisms. The comparable results also show that our method is superior to the original YOLO-V3, SSD, and Faster R-CNN for PB data, and has a detection AP of 86.7%, a recall rate of 97%, and a detection speed of 25.47 FPS, which can achieve high-efficiency and high-precision detection in complex industrial scenarios.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Code Availability
Code generated or used during the study is available from the corresponding author by request.
References
Moura, F.M., Silva, M.F.: Application for automatic programming of palletizing robots. In: 2018 IEEE International Conference on Autonomous Robot Systems and Competitions (ICARSC), pp 48–53. IEEE (2018)
de Souza, J.P.C., Castro, A.L., Rocha, L.F., et al.: Adaptpack studio translator: translating offline programming to real palletizing robots Industrial Robot: The International Journal of Robotics Research and Application (2020)
Li, C., Ma, Y., Wang, S., et al.: Novel industrial robot sorting technology based on machine vision. In: 2017 9th International Conference on Modelling, Identification and Control (ICMIC), pp 902–907. IEEE (2017)
Wang, J., Zhang, X., Dou, H., et al.: Study on the target recognition and location technology of industrial sorting robot based on machine vision. J. Robot. Netw. Artif. Life 1(2), 108–110 (2014)
Chen, Z.N., Zhang, X., Peng, Z.R., et al.: Workpiece location and recognition based on machine vision. Electron. Sci. Technol. 29(4), 99–103 (2016)
Huang, C., Chen, D., Tang, X.: Implementation of workpiece recognition and location based on opencv. In: 2015 8th International Symposium on Computational Intelligence and Design (ISCID), vol. 2, pp 228–232. IEEE (2015)
Jinqiu, M., Tongshuai, Z., Zhiyu, Z.: An Approach for Picking T-shape workpiece based on monocular vision. In: 2018 3rd International Conference on Information Systems Engineering (ICISE), pp 1–5. IEEE (2018)
Choi, C., Taguchi, Y., Tuzel, O., et al.: Voting-based pose estimation for robotic assembly using a 3D sensor. In: 2012 IEEE International Conference on Robotics and Automation, pp 1724–1731. IEEE (2012)
Yang, L, Chong, M, Bai, C, et al.: A multi-workpieces recognition algorithm based on shape-SVM learning model. J. Phys. Conf. Series. IOP Publishing 1087(2), 022025 (2018)
Fu, T., Li, F., Zheng, Y., et al.: Dynamically grasping with incomplete information workpiece based on machine vision. In: 2019 IEEE International Conference on Unmanned Systems (ICUS), pp 502–507. IEEE (2019)
Zhao, Z.Q., Zheng, P., Xu, S., et al.: Object detection with deep learning: a review. IEEE Trans. Neural Netw. Learn. Syst. 30(11), 3212–3232 (2019)
Wang, X., Liu, M., Raychaudhuri, D.S., et al.: Learning person Re-Identification models from videos with weak supervision. IEEE Trans. Image Process. 30, 3017–3028 (2021)
Hu, H., Zhang, Z., Xie, Z., et al.: Local relation networks for image recognition. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp 3464–3473 (2019)
Jiang, W., Liu, M., Peng, Y., et al.: HDCB-Net: A neural network with the hybrid dilated convolution for pixel-level crack detection on concrete bridges. IEEE Trans. Indust. Inform. 17(8), 5485–5494 (2020)
Li, C.H.G., Chang, Y.M.: Automated visual positioning and precision placement of a workpiece using deep learning. Int. J. Adv. Manufact. Technol. 104(9), 4527–4538 (2019)
Lin, X., Wang, X., Li, L.: Intelligent detection of edge inconsistency for mechanical workpiece by machine vision with deep learning and variable geometry model. Appl. Intell. 50(7), 2105–2119 (2020)
Redmon, J., Farhadi A.: Yolov3: An incremental improvement. arXiv:1804.02767 (2018)
Kapoor, A., Singhal, A.: A comparative study of K-Means, K-Means++ and Fuzzy C-Means clustering algorithms. In: 2017 3rd International Conference on Computational Intelligence & Communication Technology (CICT), pp 1–6. IEEE (2017)
Huang, G., Liu, Z., Van Der Maaten, L., et al.: Densely connected convolutional networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 4700–4708 (2017)
Zhou, P., Ni, B., Geng, C., et al.: Scale-transferrable object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 528–537 (2018)
He, K., Zhang, X., Ren, S., et al.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 770–778 (2016)
Agarap, A.F.: Deep learning using rectified linear units (relu). arXiv:1803.08375 (2018)
Rezatofighi, H., Tsoi, N., Gwak, J.Y., et al.: Generalized intersection over union: a metric and a loss for bounding box regression. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 658–666 (2019)
Ren, S., He, K., Girshick, R., et al.: Faster r-CNN: towards real-time object detection with region proposal networks. IEEE Trans. Pattern Anal. Machine Intell. 39(6), 1137–1149 (2016)
Liu, W., Anguelov, D., Erhan, D., et al.: Ssd: Single shot multibox detector. European conference on computer vision, pp 21–37. Springer, Cham (2016)
Tang, Y., Li, B., Liu, M., et al.: Autopedestrian: an automatic data augmentation and loss function search scheme for pedestrian detection. IEEE Transactions on Image Processing (2021)
Girshick, R., Donahue, J., Darrell, T., et al.: Rich feature hierarchies for accurate object detection and semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 580–587 (2014)
Girshick, R.: Fast r-cnn. In: Proceedings of the IEEE International Conference on Computer Vision, pp 1440–1448 (2015)
Redmon, J., Divvala, S., Girshick, R., et al.: You only look once: unified, real-time object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 779–788 (2016)
Redmon, J., Farhadi, A.: YOLO9000: Better, faster, stronger. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 7263–7271 (2017)
Redmon, J., Farhadi A.: Yolov3: An incremental improvement. arXiv:1804.02767 (2018)
Lin, T.Y., Goyal, P., Girshick, R., et al.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp 2980–2988 (2017)
Wang, K, Ma, S, Chen, J, et al.: Approaches challenges and applications for deep visual odometry toward to complicated and emerging areas. IEEE Transactions on Cognitive and Developmental Systems (2020)
Wang, K, Ma, S, Ren, F, et al.: SBAS: Salient bundle adjustment for visual SLAM. IEEE Trans. Instrum. Meas. 70, 1–9 (2021)
Acknowledgements
The authors would like to acknowledge the support of the National Natural Science Foundation of China - Key Project 61733004, 62027810, 62076091 and 62133005.
Author information
Authors and Affiliations
Contributions
The overall study supervised by Yaonan Wang; Methodology, hardware, software, and preparing the original draft by Ke Zhao; Review and editing by Qing Zhu and Yi Zuo; The results were analyzed and validated by Chujin Zhang. All authors have read and agreed to the published version of the manuscript.
Corresponding author
Ethics declarations
Conflict of Interests
All the authors of this paper have no conflicts of interest, financial or otherwise.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Zhao, K., Wang, Y., Zuo, Y. et al. Palletizing Robot Positioning Bolt Detection Based on Improved YOLO-V3. J Intell Robot Syst 104, 41 (2022). https://doi.org/10.1007/s10846-022-01580-w
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s10846-022-01580-w